Intel Caught Commissioning Apparently Bias Benchmark Data for Core i9 9900K [Updated]

[Guest_Jim_*]

Yesterday Intel officially announced many new CPUs, including the Core i9 9900K, an 8-core/16-thread CPU with a base clock of 3.6 GHz and a single-core boost of 5.0 GHz. The company calls it the world's best gaming processor, something that will likely be confirmed when the review embargo lifts on October 19. However, some early performance numbers are drawing more than just skepticism from some.

PCGamesN published performance data Intel had commissioned Principled Technologies (report PDF) to collect for it, comparing the 9900K against a number of CPUs, including the i7 8700K and Ryzen 7 2700X, across several games. According to this data, the 9900K possesses a significant lead in some games against the competing AMD CPU, in some cases reaching 30% to 50% better performance. The thing is, the numbers shared did not seem appropriate for some, including Steve Walton of TechSpot and Hardware Unboxed, who has previously and extensively benchmarked the already-released CPUs in at least some of these games, so he did some research. This research included not only looking through the report from Principled Technologies that is linked above from the PCGamesN article, but also trying to recreate the test systems.

Just going through the report, one issue identified was that the RAM was configured differently between the Intel and AMD systems. This is not in reference to the maximum supported speeds being used (2666 MHz for Intel and 2933 MHz for AMD) but to the Z390 system using an XMP profile with tightened timings, while the AMD system was left with the loose defaults, which will bias performance. Additionally, all of the DIMM slots were populated, which is apparently not optimal for Ryzen CPUs. Something else noted, though not by Steve, is that Game Mode was selected in the AMD Ryzen Master Utility, which on a Threadripper CPU, like the 2950X that was also part of the testing, will disable one of the two active dies, cutting the core count in half. Apparently this option is also present for the single-die 2700X, potentially cutting its core count in half as well, to just 4 cores.

After attempting to recreate the results shown between the 2700X and 8700K, the conclusion from Steve is "the Principled Technologies results are a load of rubbish." None of his tests showed the same gap indicated by the report, though the 8700K did consistently beat the 2700X. His testing included using identical memory configurations between the systems too, which improved performance for both Intel and AMD, but still not to what the report claims. As a result of this, PCGamesN has edited its original article, noting Steve's findings and has requested comment from Intel.

 

 

Update:

Steve has done some more testing and shared the results on the Hardware Unboxed Patreon page and while these results do actually come within margin of error of the Principled Technologies results, they do not ease the situation. These new results are from tests when Ryzen Master's Game Mode is enabled on the Ryzen 7 2700X, which Steve originally thought was only being used for the Threadripper 2950X. It turns out Game Mode is an option for the 2700X and does disable one of two CCX modules in on the die, making it a 4-core/8-thread CPU. Doing this with the loose timings gave him performance results in line with the Principled Technologies report, but it also means the report is pitting the 8-core/16-thread i9 9900K against the AMD CPU configured for 4-core/8-thread.

Source: TechSpot/Hardware Unboxed and PCGamesN

Back to original news post

[Guest_Jim_*]

Just updated the post because after the Game Mode setting was pointed out to Steve he did additional testing using it on the 2700X, and shared the results on Patreon. Apparently it does in fact disable one of the two CCX modules in the 2700X, making it a 4-core/8-thread CPU, which got him within margin of error of the Principled Technologies results.

So while we now have to fail to reject the commissioned results, being able to confirm them, I think we can reject the testing scenario, because who would actually run the 2700X like that? Game Mode exists for something like my 1950X and I would rather not play games that need it than disable half the CPU.

I'm wondering how the 1 CCX Ryzen 5 2400G, if overclocked to match the 2700X speed, would fair in these tests. It's one thing to compare the nearly $530 9900K against the $300 2700X, but if the results are actually similar to the $160 2400G, then that +50% claim seems very unimpressive to me, and that was in a single game (Ashes of the Singularity). The other results had smaller gaps.

 

Also noticed that some have spotted the Noctua NH-U14S was used for cooling the CPUs, except for the 2700X which used the stock Wraith Prism cooler it comes with. I managed to find HardOCP tested the Wraith Prism against other coolers, including the NH-U12S, so it is just 120 mm compared to the 140 mm U14S, on a Ryzen 7 1700. The U12S won in the tests by about 4 degrees when only the CPU was loaded and more when the GPU was loaded too, with the CPU crashing in those tests. The U14S, being larger, will be comparable or better than the U12S.

So we have inconsistent RAM timings, disabled cores, and lesser CPU cooling all as variables all present here and working against the Ryzen 7 2700X, and setting the test system apart from real-world computers.