Gadgetory

today we're unveiling the other half of our new CPU testing methodology for this year we already shared some of the workstation benchmarks which are actually getting further updating after that and now we're looking at the gaming benchmarks for what we're working with into 2019 and the upcoming rise in 3,000 CPU launch so this is our first round of tests for this testing method and it includes a good range an eclectic mix of CPUs if you will before that this video is brought to you by thermal grizzly and their high end thermal compounds thermal grizzly makes cryo not paste for high thermal performance and conductivity without being electrically conductive so you don't have to worry about shorting components cryo knot is particularly good for replacing stock GPU pastes as cryo knot is a non curing compound learn more at the link in the description below so a couple things here first of all the article below will have some additional testing methodology information if you are curious about the specific game settings used versions of various software solutions things like that and the workstation benchmark was already published if you're curious about Adobe Premiere Photoshop WinZip things like or 7-zip rather software solutions like that we're tested previously we are as a note adding an additional compile test and we're working with some of our community members on figuring out the best type of tests to do for this so we did GCC with cygwin and that showed AMD doing really well but there are other types of compiling where Intel does really well and they're both valid they're just looking at different aspects of the processor so our next step is to iterate on that and add another program compile test and then we'll have one showing the use case where AMD excels in one show in the use case we're Intel excels and then the programmers a money you can pick which one is most applicable to your use case so that should be pretty cool but that's getting worked on separately today we're looking at games and we've added several games we aside from increasing the count of games we also are now doing across all of them except for mostly one to resolution so 1080 and 1440 across the whole stack which obviously doubles the testing load but allows us to see at what point the GP bottleneck becomes a limiter and how relevant the different gaps between CPUs are at ten P if you're planning to do a higher-resolution that might become GPU about so that's part of we're looking at and then we have the test bench in the description below and the methodology in the description below as well in that article finally as for the CPUs chosen we just we kind of picked a bunch of ones that would allow us to validate the methods so the goal was to get a wide range on the stack where we have some common ones twenty-seven hundred ninety nine hundred K things like that that a couple that were a bit more sprawled out in terms of the performance differences so that we could determine if the testing was a good valid and reliable and repeatable method going forward so we will be adding more CPUs of course but this is just the first preview of what we're doing for testing with games for 2019 so let's get straight into the data little talk conclusions at the end the total war Warhammer to battle benchmark is basically the same thing as the total war Warhammer one test we ran in our previous methodology just using a different scene and with different armies the same problems with inconsistent 1% and 0.1% low is it from the first total Warhammer game effects this one as well but the overall FPS averages are still reliable the high preset is also still used as with the other games coming up the overclocked in 9900 K is the stuff the top scorer with slightly lower clocked at 9,700 K benchmarks trailing by an insignificant margin average FPS of the 5.1 gigahertz overclocked 9700 K is only 3 FPS beyond that only 5 gigahertz overclocked 6 core 6 thread 8600 k implying that here again as a game that doesn't much benefit from more than 6 threads and instead becomes more focused on higher frequencies this is a game that strongly benefits from the 2990 W X's game mode which brings the FPS average up from nineteen point five to 121 that's still worse than the other CPUs we've tested but at least it's not broken Corp Ryo appears to slightly raise performance as well but nowhere near as much as simply turning off cores with game mode here's where it gets interesting if you're looking at the stock r5 2600 9x results of 133 FPS average versus the stock r7 2700 results of 123 FPS average you might wonder about why that happened we'd already know that scam likes frequency more than threads so we can take a look at the frequency over time when this benchmark is running in this chart we ran the test on the 2600 versus the 2700 both stock and logged the frequency on each core during the test as illustrated by the chart the r5 2600 maintains a much higher frequency of 38 16 megahertz to 38 50 megahertz all core something that we can see plotted against all six core clocks in this chart the r7 2700 runs it closer to the 3467 megahertz all core range although we like recommending the r7 2700 9x it's normally under the stipulation of a user applied overclock and this is part of why there's still a lot of all core Headroom to gain here but this illustrates exactly why the our 727-100 gets outmatched by the cheaper r5 2600 in frequency bound titles fps it lowered for every CPU tested when the resolution was raised to 1440p so we're starting to encounter GP bottlenecking at this new resolution this is sort of the point with the 1440p test though as it's a degree of realism that can sometimes be lost at 1080p if you were to run these graphics settings on a 20 atti with this game the choice between an 8700 k 9700 k 9900 k or really anything above the 2600 at 4.2 gigahertz would be largely irrelevant they're all about the same the top half of the results that be the I 580 600 K through the i7 8700 K are all within margin of error if you're confused why the 8700 K is quote-unquote better than the 9900 K don't be it's not it's just that the results are within error margins and test variance and so we can either distinguish nor realize the actual differences between these processors there are some differences in point one percent one percent low values Illustrated primarily in the overclocked stat 9900 K 9700 8600 K but that's about the only difference these are all beyond the cutoff for GPU bottlenecking but below that level there's still some differentiation between the CPUs the 4.2 gigahertz 2700 and stock 2700 X are neck and neck again proving the value of AMD's non X CPU and overclocked the stock twenty six hundred and twenty six hundred but the 2600 X does have slightly better one percent lows the 1440 test doesn't provide much value beyond reminding us that GPA limitations can limit cpu differentiation when playing at higher settings the baked in total war Warhammer to campaign benchmark is much more CPU bound than the battle benchmark which is a good thing given the obvious GPU limitations of the 1440p battle benchmark the overclocked to 9900 K and 97 hard K lead in the 1080p test as well which makes sense the results should and do line up nearly the same as they do in the battle benchmark but with more differentiation at the high end overclocked in the 9900 K to 5 point 2 gigahertz gives a boost of about 6% over the Sox 9900 K with the 9700 K overclocked receiving a boost of 7% frame times are reasonable across most of the chart although there's still some natural run to run variance that builds a bigger error bar in the 0.1% lo category as for other rankings the 8600 KF 5 gigahertz surpasses the stock 97 RK and stock 87 RK which is a clear illustration that total war Warhammer is still heavily dependent upon frequency of each part this is further illustrated with the differentiation between the r7 2,700 x stock CPU plotting 135 FPS average and the r5 2600 xcp you plot a note 135 FPS average as well when stock the extra cores don't matter much here but the frequency ties things up the 2990 WX does terribly here predictably as the game doesn't quite understand what to do with the threads and scheduling get screwy and memory latency causes delays Corp Ryo doesn't fix anything either but restarting it's a game mode to disable half the cores does boost the 2990 W XOR seven 2700 levels of performance unlike the battle benchmarks the campaign benchmark at 1440p shows some scaling right up to the overclocked 9900 K and 9700 K where we do start to see some clipping off of the high end the higher scoring CPUs do have lower averages than they did at 1080p though so they are approaching a GPU limit a lot of rise and results are within margin of error like B 2600 at 4.2 gigahertz the 2700 X stock and the 2600 X stock f1 2018 is another returning title with the methodology unchanged for the last round except for newer drivers and a newer version of Windows both of which impacts the results the frame rates at 1080p and 1440p are high they're far beyond 144 Hertz monitors if you're trying to achieve that but there still see be limited so it's a good test of relative performance the overclocked to 9900 k97 Hut ceilidh at over 300 FPS average with their stock scores following it right behind the 9900 case stock in 9700 KOC results are roughly within margin of error of each other and our outside of our test resolution as it is now the chart has a clean intel AMD split with the typically higher frequency intel parts at the top and the typically higher thread count and d parts at the bottom there are 720 700 X runs at about 236 FPS average ranking it as functionally equals the overclocked 2700 at 4.2 gigahertz a surprise as they're about the same frequency here the r5 2600 X oxy vo ends up at 224 FPS average allowing the overclocks 2700 elite of about 5% it's clear that Intel is enabling higher overall framerate here with its high in two parts I'll be it more expensive on average but we are entering into territory where the question of practical versus theoretical differences emerges the relatively low frequency 2990 WX performed worst of all behind the stock 2,600 2,700 but this is expected for threader for parts in gaming scenarios the 2600 and 2700 end up in the same order as total war Warhammer which is the result of higher frequency on the stock 2600 when under all core workloads let's look at a frame time plot to better illustrate the differences as a reminder frame time plots given up close look to frame to frame creation time showing how many milliseconds it takes to draw each frames of the screen this is an important metric that can get obfuscated with average frame rates even 1% lows and helps illustrate consistency of frame delivery so 9700 K and 2,700 X both encounter occasional hiccups beyond 12 milliseconds but these are rare and unnoticeable in single frame doses the user would not notice these because they're not a big enough excursion to really be seen the 23 millisecond spike is getting more noticeable but the excursions to 9 to 12 milliseconds don't become particularly visible to the user unless several spikes happen in sequence which they don't hear overall both of these CPUs handle the game with fluidity going to 1440p the increased resolution knocks the Peaks off of the frame rates with some GPU limitation but only the 9700 K 9900 K and overclocked 8600 K have reached the level where noticeable CPU scaling stops so r7 2700 at 4.2 gigahertz and 2,700 X are within margin of error of each other and are also not too distant from the stock 8700 K which otherwise leading the pack of CPUs that are only occasionally hitting GPU limitations the r7 2700 stock CPU doesn't look great here comparatively but keep in mind that we're still at about 200 FPS in this title the point though is that it is limiting a 20 atti at 1440p although this bottleneck should further diminish as resolution increases the 2700 deserves an overclock as we've always said as its performance climbs notably to 220 FPS average when pushed to 4.2 gigahertz all core Assassin's Creed origins is next this is our most troublesome benchmark because of occasional stuttering that we have to correct for with extra bench passes but it's a good example of a game that respond strongly to overclocking similar in behavior to hitman 2 settings remain at the medium preset unchanged from previous testing the top 4 CPUs are nearly tied at 1080p approaching a definite GPU bottleneck and making this one of the games that will need to replace a revamp for our next round of testing still the overclocked to 9900 K and 9700 K hold a slight performance lead over stock the 8700 K stock is tied with 8600 K when clock to 5 gigahertz so we can say that origins that seems to benefit from higher thread counts interestingly the 2990 WX results confirm this even though the game suffered from hitching to the point of being unplayable average frame rate was slightly higher at stock than it was in game mode even though game mode didn't have stuttering and pulling the average down this is also a study in why we test with 1% lowest part one protect lures and frame time plots because these deficits in thread refers creator mode would not be shown if only looking at average FPS and last place was the stock R 5 2600 which performed up much better with a 4.2 gigahertz overclock and again further confirms that threads are helping here otherwise it'd be faster than the stock r7 2700 raising the resolution makes the GPU bottleneck more prevalent but there's enough fluctuation in framerate during the test at 1440p that it's still really only the top 4 CPUs that have hit the wall the 9700 K and 9900 K have a 1 FPS range between them which is well within test variants for this title the 87 arcade overclocked 280 600 K you have swap the leads but remain nearly tied as I have the 2700 stock and overclocked 2600 and the rest of the stack is ordered the same as it was at 1080 the same itching issues plagued the 2990 WX unless game mode is enabled which again slightly lowers the average FPS GTA 5 returns with almost identical settings to our previous tests but with high detail streaming while flying is enabled an extended distance scaling maxed out the segments of the built in benchmark that we logged for testing was too often hitting the 180 7.5 FPS engine limit on modern CPUs and these tweaks are intended to make it more stressful by increasing draw calls that the CPU has to juggle for increased geometry drawing GTA 5 is the oldest game we use receive you testing by a longshot and at some point it will need to be phased out at 1080p that overclocked 5.2 gigahertz 9900 Kay predictably leads the way trailed just slightly by the 5.1 gigahertz 9700 K again there's no tangible benefit from having more than eight threads in this title in fact given how close the stock 8600 K and 8700 K scores are it's unlikely that there's much benefit from having more than six as we fill out the chart with older four and six thread CPUs the answer will become more clear although that said we've seen in the past that having four threads or six threads can be detrimental if you're trying to hit that maximum frame rate limit because once you start hitting 180 7.5 and GTA 5 something like an i5 CPU without hyper threading will begin to hitch in a very noticeable fashion something we've published in the past the 2700 trails at roughly the same level as the 2990 WX which doesn't significantly benefit from game mode or Corp ryo outside of potentially more stable as your one-percent lows other than that i'm dolls law applies and GTA v favors frequency with threads preferred only to a point there's a clear division in this chart between Intel and AMD with and the falling universally below 100 FPS average which we saw in our last round of GTA 5 tests as well despite light lighter settings and so we can illustrate gaming advantages and Intel CPUs for this benchmark sometimes to the tune of 20% uplift as for the 9700 stock in 9,900 case stock will again remind you that these results are within margin of error at 1440p GPU limitations keep results at the high end of the chart squished closer together note that for the lower performance CPUs the scores are practically unchanged at either resolution while the 9900 KOC drops fully ten fps the partial GP bottleneck the stock 1900 km 97 or k have technically swapped places on the charge but the two stock CPUs perform essentially identically to each other at both 1080p and 1440p and remain within error margins the 2600 K and 2600 are also close to each other and the 0.1% lows of the 2990 WX are again slightly worse at stock than they were are with Corp Rio shadow of the Tomb Raider gives us a DirectX 12 representation the medium preset is used for this one the 1080p result shows a decent balance between thread count and frequency being the deciding factor of performance the overclocked to 9900 K still tops the chart but the stock and Corp Ryo enabled 2990 WX results are significantly higher than the game mode results and for once the 2990 WX exceeds both e stock 2700 and stock 2600 in performance the six core six thread 8600 k place is fairly high but the 4.2 gigahertz 2,700 plays better than the 4.2 gigahertz 2600 again confirming that Shadow the Tomb Raider is actually benefiting from thread count at least as high as 16 the 97 hardcase stock CPU ends up about 24% hired frame rate than the r7 2700 X 4 point of reference as 1440p we cut that test from this batch we discovered that it was GPU bound almost all the way down the stack and so there's no value in illustrating the same number of 15 different times civilization 6 updated and replaced its AI benchmark in the middle of redoing our tests again so we have fewer results for this title as of right now the new test is still 4 passes with 5 turns per pass but each turn is significantly longer and should magnify the differences between CPUs as CPU intensive as it is the AI benchmark doesn't make great use of higher thread counts this game like many others likes frequency the stock 2990 WX resolves turns faster than it did in game mode but not by much the only overclocked CPU we've run in civilization so far is the 80 600 K since we had to throw out our original set of data and the increase in frequency allowed it to surpass the other 12 16 and 64 thread CPUs at 30 seconds to complete a turn a full pass with 5 players would take 2.5 minutes that starts getting noticeable the slowdown from the 8600 k2 the 2700 X 36 second time when you're comparing to risky overclocked 8600 k is is pretty big that's another 30 seconds you'd be waiting for your next turn we'll revisit this test and add more CPUs as we go as six updates enough that it wipes all of our data out at least once a year we use the Miami bench scene for hitman to our next game the important settings for CPU testing our max LOD and Max simulation quality even with relatively high GP related settings CPUs have remained the bottleneck in our testing so far the overclocks 5.1 gigahertz 9700 k leads the pack although we need to run the 9900 KOC through here it's clear from the other results like the 2600 OC versus the 2700 OC that this title doesn't benefit much from thread counts greater than 8 the lowest scoring game CPU or sort of mainstream CPU was the r7 2700 thanks to its lower all core frequency than the r5 2600 wily 2990 WX scored lowest of all with no significant difference from either switching to game mode or running Corp Ryo for the services the results stack is exactly the same at 1080p or 1440p testing the 8700 K overclocked to a comparable frequency and seeing if it scores lower than the 9700 K as well as next on our to-do list the 2990 WX chugs Alon in last place again at 1440p with no uplift from game mode this would indicate again that it's because of just too many cores and threads and not enough frequency so that's it for this one as you can see we've added a few games and we're pretty happy with the games we have so far sip 6 frustratingly changed in a very substantial way in the middle of our testing so we had a full suite of test run with sub 6 and then it updated and the turn time changed again so the good news is that the update extended the turn duration for the benchmark so we end up with something that can better show the differences between CPUs where previously you might see a 1 second difference and that's kind of like ok whatever but what do you see a six-second difference or 10 second difference multiplied across however many AI players you have that starts to become significant and as many of you turn based strategy game players know the more AI you add the more that wait time really starts to pile on as you uncover more of the map and it get further in the game so that's changed and we've adopted the newest version of Civ 6-4 for the benchmark we think it's more accurate we've added more to total war so anyway there's a lot more we can do here and we're working on it so keep keep your eyes peeled for future content including especially the upcoming modified version of our compiler benchmark where we'll have the one that we already ran plus a new one showing when Intel pulls ahead in theory anyway so that's all coming up stay subscribed for that or get subscribed if you're not you can go to store like here in Texas net to pick up a mod mat they are now back in stock finally so we have the medium mod mats as on the table here and the large mod mat as under it both on store documents Nexus dotnet if you order them they'll go out right away so thank you for watching I'll see you all next time