A couple weeks ago, a commissioned report on the performance of Intel's i9-9900K was put out more than a week before independent reviews, like our own, and this led to some interesting analysis of the numbers and practices used for the testing. According to the original version of the report, the 9900K could best the AMD Ryzen 7 2700X by 30% to 50% in some games, but some reviewers noticed the numbers shared for the 2700X were inconsistent with what they observed in their own tests. Going through the report, that included information on how the test systems were prepared, a number of issues or potential issues were identified, including the use of Game Mode on the 2700X, cutting its core and thread count in half, and using a different cooler on the 2700X than every other CPU tested. While a second version of the report was prepared, significant damage was done to Intel's reputation and that of Principled Technologies, which did the tests, and some would argue there were still issues with this second report.
On Friday, along with the embargo lifting on independent 9900K reviews, AMD decided to release a couple slides covering not only some of the identified issues but also its benchmarking best practices. The list is actually a pretty good one, going over ways to sanitize (in this case, make reproducible and reliable) the configuration of the operating system, the platform for both stock and overclocked situations, and sanitizing the data. The last two points made are for the reviewer to "remember the user!" It reminds us again that the testing was done with a 64 GB (4x 16 GB) memory kit that is far more expensive than the Ryzen 7 2700X CPU tested, and urges reviewers to always consider if the configuration, tests, and results make sense for a consumer.
Source: Legit Reviews
Back to original news post