Deceptive Benchmarks Corrected, Intel Still Doesn't Get It
Deceptive Benchmarks Corrected, Intel Still Doesn't Get It
2018-10-12
so we weren't planning on doing this
video but principal technologies came
back with new test results with the rise
in 2700 X tested mostly well better this
time they used creator mode instead of
game mode and it game mode despite its
naming really meant more for thread
Ripper and that's the disabled a bunch
of cores that games don't know what the
heck to do it but in the 2700 acts you
don't have that problem so it should be
left stock unmodified by the user which
would be called creator mode they posted
new results readin go through those
today it looks a lot different than it
did originally the he 60% differences
between the 2700 X on the 900 K suddenly
disappear so that's important to go
through and then Intel also gave us a
statement which just came in like
minutes ago that we'll be reading and it
is well it's it's something else before
that this video is brought to you by the
Thermaltake level 20 VT micro ATX case
the level 20 VT takes the high quality
at level 20 design and makes it more
affordable and shrinks it down to a
micro ATX form factor at that with fully
modular paneling it's possible to
rearrange this case into whatever
configuration you prefer for a micro ATX
case that can be a discussion piece in a
home theater system click the link in
the description below it's a quick recap
very quick one we went over to the
principal technologies offices we had an
interview with them we went through
their testing methods and why we thought
there were problems in those methods
they recorded the conversation they took
the recording and they theoretically
looked at some of it when redoing their
tests and they did redo a bunch of tests
if you don't know what came before that
the answer is that principal
technologies and Intel jointly released
a third-party test document conducted by
principal technologies consisting of 19
games for the 900k the 99 80 X E and
some other Intel processors this is
functionally a review the difference is
that principal technologies is not under
review embargo like we are so we can't
come out and validate them or invalidate
the results that's a problem the next
part principal technologies is a third
party paid by Intel so this is actually
okay we're okay with paying a
third-party test house to do validation
the problem that we actually have is
that the third-party test house is the
only one permitted
testing information and so consumers are
left with a document which is full of
what we very well has toned it down a
bit thought thought the testing was
flawed we disagreed with it
vehemently and that's why we went to
their offices and talks them about it so
despite the despite being greeted at the
door by one of the the co-founders mark
at principal technologies who said we
disagree with you the reality is that
there new results showed that we were
actually right as was Steve from
Hardware and box and others who were
involved in this so let me read it
Intel's response first I want to first
say this principal technologies has
really done a lot to earn a little bit
of trust towards them trying to do
something a bit better its Intel that I
now have a problem with more so than PT
although PT of course we do ultimately
think that their initial testing was
very misleading whether they intended it
to be or not so Intel released this
extremely conceded statement in Tulsa
given the feedback from the tech
community we are pleased that principal
technologies around additional tests
they've now published these results
along with even more detailed and the
configurations used and the rationale
the results continue to show that the
9th gen core I nine ninety nine hundred
K is the world's best gaming processor
we are thankful for principal
technologies time and transparency
throughout this process we always
appreciate feedback from the tech
community and are looking forward to
comprehensive third-party reviews coming
out on October 19th and this is
following a previous statement where
Intel said they were excited to see
third-party reviewers show that the 99
heard K is the best processor in the
world ever made ever so here's the thing
no one I don't think anyone said that
the 99 Harry K wouldn't be a chart
topper in fact in our interview with PT
I was establishing with them look I
don't disagree
gamers Naxos we don't disagree that the
9900 K is probably gonna be the best on
the chart what we disagree with is the
degree to which it is the best because
here's the thing if you're already in
the lead why do you have to cheat why do
you have to throw caltrops at the guy
behind you and trip them up
there's no point you're already winning
so
and this isn't necessarily using the
word cheat here if principal
technologies really thought that game
mode was the right thing to do and I'm
okay with tanking them at their word for
that bill didn't necessarily seem like
he knew any better and they don't seem
like they test a lot of gaming parts and
I didn't talk to their technicians so if
we all take them at their word for that
we have to them look at Intel who came
out and said we validated Petey's
results and they were accurate Intel
knows better Intel knows exactly what
game mode does so Intel why why do you
need this level of deception and then
the bullshit coming after it where you
say oh well we're still the best so we
we have a big problem with the way Intel
handle all this because info comes out
acting smug about the results saying
that they're still the best and again no
one said Intel wasn't the best the word
best however in this context can be an
awful awful word because best
technically extremely objectively and
looking only at the FPS numbers yeah
Intel chart tops here and therefore you
could say they are the best in so far as
their results for these games
however Intel isn't necessarily the best
in so far as its value proposition or in
perhaps production performance or maybe
in the other specific application
performance maybe in power maybe in
throws we don't know we haven't tested
it yet so to say just outright best
isn't really fair we will give Intel
that they're at the top of the charts so
for these gaming results as we said in
our previous video as we said in the
interview with PC world with Gordon yeah
Intel technically is the best and
tactical victory isn't always the best
victory because there are a lot of other
factors to consider
that's where third-party reviews will
come in but the statement by Intel was
extremely conceited they didn't miss the
point either in town that was damn well
what the problem was but Intel is
speaking to its shareholders and they're
putting out a statement that will please
its shareholders so anyway let's go
through Petey's results we're just gonna
read through them
through the new tasks we're not going to
cross validate it's not really the point
here the point is how did the results
change after that discussion and this is
primarily with the change of game mode
versus creator mode we're not really
talking about any of the rest of the
issues that we brought up during that
interview so let's jump around and the
results a bit to show what the
differences are between game mode and
creator mode we already pretty much knew
all these differences from our own
testing ages ago but it's good to
revisit them with principal technologies
testing total war Warhammer - really
doesn't show that much of a difference
which isn't a surprise we see a
difference of actually a bit of a
deficit in creator mode they're more or
less equal within reasonable margins
depending on how tight this testing
methodology is so not a huge change here
for the 2700 X of course the 2990 W X
has a much better result in game mode
but that is in fact why game mode was
created it wasn't made for the 2700 X it
was made for the 2990 W X and T V is
similar to that like giveth Red River
family so this shows why that exists but
if we move on to something else we can
see there are a couple games global
Offensive another example these of
course are listed at the very top of the
document where you see the least change
and these this one also shows
functionally zero difference for the
2700 X however as we start to move
through the document you'll see in Gears
of War for the 2700 X in game mode is
and this is the original result so
despite and these sort of misnomer game
mode is actually worse for the risin 7
family ZP use you see 129 fps and by the
metric that principle technologies uses
you'll see a 46% performance advantage
they call it for the 9900 K and we have
another issue with this but that's
neither here nor there actually it's
exactly here the issue is that fps
scales not linearly and so if the gap
will look bigger as you increase in in
framerate distance but in reality as the
frame rate is higher the frame time this
difference is less significant and
you're talking a couple milliseconds max
maybe even one millisecond once you
start getting up to the really high
frame rate so it's not really the best
metric but anyway 2,700 X is 129 FPS
average in game mode creator mode the
new test the retest is 151 point 5 FPS
average and that is a it changes using
their metric
only from 46% advantage 900k to 25%
pretty damn big difference and the next
one Gears of War 4 there's some more
information here for that but war
thunder is another game where we see
actually a bit of a deficit in creator
mode this comes down to things like
latency stuff like that but we also
don't know if the testing is good here
we haven't tested war thunder can't
validate it and we're just looking at
their data today ashes of the
singularity this was the gigantic red
flag for everyone
2700 X originally posted a 36 FPS 35.5
FPS average for a performance advantage
in the night anarchy of 57.2% almost 60%
better allegedly for the 9900 k that's a
pretty damn big difference but if we
look at creator mode it's 47 FPS average
dropping the gap from the performance
advantage metric that they use from
57.2% to 17.7% pretty damning and the
original results and looking at much
more respectable in the follow-up
especially considering again the price
difference and well we'll talk about
this more later but Intel doesn't need
to fudge the numbers here they're
already the best in so far as just pure
FPS numbers so why cheat and then Forza
Motorsport is next this one 2700 X
game-mode 151 FPS average creator mode
178 FPS average drop in the distance
from a 35% quote performance advantage
unquote to a 14.2% piron's advantage in
quotes assassin's creed origins we see a
score of 84 FPS average for the 2,700 x
in game mode and 106 FPS average in
creator mode dropping us from 42% quote
performance advantage to 12% for Cry 5
up next this one doesn't show a massive
difference but it does show a difference
here from 103 FPS average to 113 by
their testing methodology we'll skip
through some of the other games here
because a lot of these just to quickly
go through my guess World of Warcraft's
mostly the same results civilization 6
and not particularly exciting results
although still improved in creator mode
fortnight moves from 140 to 148 FPS
average so that changes the performance
advantage from 23% to 16%
World of Tanks goes from it's actually
not really my
different in their testing methodology
Rainbow six siege a bit of a difference
it's 263 FPS average in game mode versus
to 80 in creator mode moving from 17s
percent to 11 ish percent in the
performance advantage metric that they
use pop G we go from 191 to 204 Tomb
Raider 170 to 172 not really a change
their shadow of war fish from 142 143
not a big change there either and some
of these games show bigger changes than
others some really not that exciting so
that's it that's it for the quick recap
again this video is completely
unanticipated we had other video
scheduled to go up at the time this one
will be going up so well throw it
together for you it's a really important
news announcement we wanted to make it
stand alone even though we could have
shoved it into a news video in a week
but we wanted to go up now because it's
important and relevant now so
significant difference between the 2700
X results the 9900 K results as
originally conducted as everyone in our
audience expected and again huge props
to principal technologies for accepting
an interview with us for being very
transparent in this whole process and
presumably trying to do the right thing
here but Intel really has lost some
favor in the tech community with this
one and it's unfortunate because the
900k in the 998 exe and the 3175 that
whole launch event was well received by
the press and that's rare it's rare for
a launch event to go over that well most
launch events and the Nvidia and Intel
results in in a lot of bickering and
complaints about how how disorganized
the launch event was so Intel had this
momentum and they did things people
wanted they added solder and then they
just just obliterated all of that
goodwill just lit a match and watched it
burn it's a post results from another
company PT whom they paid with incorrect
testing that created differences that
looked as big as 60
cent I'll remind you and then still come
out after it's been changed from like
60% improvement to 17 and say well we're
still the best so there's nothing more
we can say here that we haven't already
said that's it for this one this video
again wasn't expected by important
information to have sound off in the
comments we're still still pretty pretty
unhappy with way intel handled this to
put it lightly the processor itself may
be completely fine but Intel is souring
the launch with with just gimmickry and
stupid childish games that are
completely based in total bullshit so
anyway that's that's that's what we're
dealing with for this one so anyways as
always stored on Cameron's exit sign at
table directly patreon.com slash gamers
and access if you'd like to pick up our
patreon tears that give you access to
discord and things like that thank you
for watching subscribe for more I'll see
you all next time
We are a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for us to earn fees by linking to Amazon.com and affiliated sites.