Gadgetory


All Cool Mind-blowing Gadgets You Love in One Place

Deceptive Benchmarks Corrected, Intel Still Doesn't Get It

2018-10-12
so we weren't planning on doing this video but principal technologies came back with new test results with the rise in 2700 X tested mostly well better this time they used creator mode instead of game mode and it game mode despite its naming really meant more for thread Ripper and that's the disabled a bunch of cores that games don't know what the heck to do it but in the 2700 acts you don't have that problem so it should be left stock unmodified by the user which would be called creator mode they posted new results readin go through those today it looks a lot different than it did originally the he 60% differences between the 2700 X on the 900 K suddenly disappear so that's important to go through and then Intel also gave us a statement which just came in like minutes ago that we'll be reading and it is well it's it's something else before that this video is brought to you by the Thermaltake level 20 VT micro ATX case the level 20 VT takes the high quality at level 20 design and makes it more affordable and shrinks it down to a micro ATX form factor at that with fully modular paneling it's possible to rearrange this case into whatever configuration you prefer for a micro ATX case that can be a discussion piece in a home theater system click the link in the description below it's a quick recap very quick one we went over to the principal technologies offices we had an interview with them we went through their testing methods and why we thought there were problems in those methods they recorded the conversation they took the recording and they theoretically looked at some of it when redoing their tests and they did redo a bunch of tests if you don't know what came before that the answer is that principal technologies and Intel jointly released a third-party test document conducted by principal technologies consisting of 19 games for the 900k the 99 80 X E and some other Intel processors this is functionally a review the difference is that principal technologies is not under review embargo like we are so we can't come out and validate them or invalidate the results that's a problem the next part principal technologies is a third party paid by Intel so this is actually okay we're okay with paying a third-party test house to do validation the problem that we actually have is that the third-party test house is the only one permitted testing information and so consumers are left with a document which is full of what we very well has toned it down a bit thought thought the testing was flawed we disagreed with it vehemently and that's why we went to their offices and talks them about it so despite the despite being greeted at the door by one of the the co-founders mark at principal technologies who said we disagree with you the reality is that there new results showed that we were actually right as was Steve from Hardware and box and others who were involved in this so let me read it Intel's response first I want to first say this principal technologies has really done a lot to earn a little bit of trust towards them trying to do something a bit better its Intel that I now have a problem with more so than PT although PT of course we do ultimately think that their initial testing was very misleading whether they intended it to be or not so Intel released this extremely conceded statement in Tulsa given the feedback from the tech community we are pleased that principal technologies around additional tests they've now published these results along with even more detailed and the configurations used and the rationale the results continue to show that the 9th gen core I nine ninety nine hundred K is the world's best gaming processor we are thankful for principal technologies time and transparency throughout this process we always appreciate feedback from the tech community and are looking forward to comprehensive third-party reviews coming out on October 19th and this is following a previous statement where Intel said they were excited to see third-party reviewers show that the 99 heard K is the best processor in the world ever made ever so here's the thing no one I don't think anyone said that the 99 Harry K wouldn't be a chart topper in fact in our interview with PT I was establishing with them look I don't disagree gamers Naxos we don't disagree that the 9900 K is probably gonna be the best on the chart what we disagree with is the degree to which it is the best because here's the thing if you're already in the lead why do you have to cheat why do you have to throw caltrops at the guy behind you and trip them up there's no point you're already winning so and this isn't necessarily using the word cheat here if principal technologies really thought that game mode was the right thing to do and I'm okay with tanking them at their word for that bill didn't necessarily seem like he knew any better and they don't seem like they test a lot of gaming parts and I didn't talk to their technicians so if we all take them at their word for that we have to them look at Intel who came out and said we validated Petey's results and they were accurate Intel knows better Intel knows exactly what game mode does so Intel why why do you need this level of deception and then the bullshit coming after it where you say oh well we're still the best so we we have a big problem with the way Intel handle all this because info comes out acting smug about the results saying that they're still the best and again no one said Intel wasn't the best the word best however in this context can be an awful awful word because best technically extremely objectively and looking only at the FPS numbers yeah Intel chart tops here and therefore you could say they are the best in so far as their results for these games however Intel isn't necessarily the best in so far as its value proposition or in perhaps production performance or maybe in the other specific application performance maybe in power maybe in throws we don't know we haven't tested it yet so to say just outright best isn't really fair we will give Intel that they're at the top of the charts so for these gaming results as we said in our previous video as we said in the interview with PC world with Gordon yeah Intel technically is the best and tactical victory isn't always the best victory because there are a lot of other factors to consider that's where third-party reviews will come in but the statement by Intel was extremely conceited they didn't miss the point either in town that was damn well what the problem was but Intel is speaking to its shareholders and they're putting out a statement that will please its shareholders so anyway let's go through Petey's results we're just gonna read through them through the new tasks we're not going to cross validate it's not really the point here the point is how did the results change after that discussion and this is primarily with the change of game mode versus creator mode we're not really talking about any of the rest of the issues that we brought up during that interview so let's jump around and the results a bit to show what the differences are between game mode and creator mode we already pretty much knew all these differences from our own testing ages ago but it's good to revisit them with principal technologies testing total war Warhammer - really doesn't show that much of a difference which isn't a surprise we see a difference of actually a bit of a deficit in creator mode they're more or less equal within reasonable margins depending on how tight this testing methodology is so not a huge change here for the 2700 X of course the 2990 W X has a much better result in game mode but that is in fact why game mode was created it wasn't made for the 2700 X it was made for the 2990 W X and T V is similar to that like giveth Red River family so this shows why that exists but if we move on to something else we can see there are a couple games global Offensive another example these of course are listed at the very top of the document where you see the least change and these this one also shows functionally zero difference for the 2700 X however as we start to move through the document you'll see in Gears of War for the 2700 X in game mode is and this is the original result so despite and these sort of misnomer game mode is actually worse for the risin 7 family ZP use you see 129 fps and by the metric that principle technologies uses you'll see a 46% performance advantage they call it for the 9900 K and we have another issue with this but that's neither here nor there actually it's exactly here the issue is that fps scales not linearly and so if the gap will look bigger as you increase in in framerate distance but in reality as the frame rate is higher the frame time this difference is less significant and you're talking a couple milliseconds max maybe even one millisecond once you start getting up to the really high frame rate so it's not really the best metric but anyway 2,700 X is 129 FPS average in game mode creator mode the new test the retest is 151 point 5 FPS average and that is a it changes using their metric only from 46% advantage 900k to 25% pretty damn big difference and the next one Gears of War 4 there's some more information here for that but war thunder is another game where we see actually a bit of a deficit in creator mode this comes down to things like latency stuff like that but we also don't know if the testing is good here we haven't tested war thunder can't validate it and we're just looking at their data today ashes of the singularity this was the gigantic red flag for everyone 2700 X originally posted a 36 FPS 35.5 FPS average for a performance advantage in the night anarchy of 57.2% almost 60% better allegedly for the 9900 k that's a pretty damn big difference but if we look at creator mode it's 47 FPS average dropping the gap from the performance advantage metric that they use from 57.2% to 17.7% pretty damning and the original results and looking at much more respectable in the follow-up especially considering again the price difference and well we'll talk about this more later but Intel doesn't need to fudge the numbers here they're already the best in so far as just pure FPS numbers so why cheat and then Forza Motorsport is next this one 2700 X game-mode 151 FPS average creator mode 178 FPS average drop in the distance from a 35% quote performance advantage unquote to a 14.2% piron's advantage in quotes assassin's creed origins we see a score of 84 FPS average for the 2,700 x in game mode and 106 FPS average in creator mode dropping us from 42% quote performance advantage to 12% for Cry 5 up next this one doesn't show a massive difference but it does show a difference here from 103 FPS average to 113 by their testing methodology we'll skip through some of the other games here because a lot of these just to quickly go through my guess World of Warcraft's mostly the same results civilization 6 and not particularly exciting results although still improved in creator mode fortnight moves from 140 to 148 FPS average so that changes the performance advantage from 23% to 16% World of Tanks goes from it's actually not really my different in their testing methodology Rainbow six siege a bit of a difference it's 263 FPS average in game mode versus to 80 in creator mode moving from 17s percent to 11 ish percent in the performance advantage metric that they use pop G we go from 191 to 204 Tomb Raider 170 to 172 not really a change their shadow of war fish from 142 143 not a big change there either and some of these games show bigger changes than others some really not that exciting so that's it that's it for the quick recap again this video is completely unanticipated we had other video scheduled to go up at the time this one will be going up so well throw it together for you it's a really important news announcement we wanted to make it stand alone even though we could have shoved it into a news video in a week but we wanted to go up now because it's important and relevant now so significant difference between the 2700 X results the 9900 K results as originally conducted as everyone in our audience expected and again huge props to principal technologies for accepting an interview with us for being very transparent in this whole process and presumably trying to do the right thing here but Intel really has lost some favor in the tech community with this one and it's unfortunate because the 900k in the 998 exe and the 3175 that whole launch event was well received by the press and that's rare it's rare for a launch event to go over that well most launch events and the Nvidia and Intel results in in a lot of bickering and complaints about how how disorganized the launch event was so Intel had this momentum and they did things people wanted they added solder and then they just just obliterated all of that goodwill just lit a match and watched it burn it's a post results from another company PT whom they paid with incorrect testing that created differences that looked as big as 60 cent I'll remind you and then still come out after it's been changed from like 60% improvement to 17 and say well we're still the best so there's nothing more we can say here that we haven't already said that's it for this one this video again wasn't expected by important information to have sound off in the comments we're still still pretty pretty unhappy with way intel handled this to put it lightly the processor itself may be completely fine but Intel is souring the launch with with just gimmickry and stupid childish games that are completely based in total bullshit so anyway that's that's that's what we're dealing with for this one so anyways as always stored on Cameron's exit sign at table directly patreon.com slash gamers and access if you'd like to pick up our patreon tears that give you access to discord and things like that thank you for watching subscribe for more I'll see you all next time
We are a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for us to earn fees by linking to Amazon.com and affiliated sites.