Gadgetory

at AMD's III event the company made great strides to showcase products that overall look good on paper especially on the CPU side even still as with anything promoted on a stage at the world's most bombastic gaming event there is a lot of flashy marketing that deserves scrutiny to keep perspective of reality will be looking both qualitatively and quantitatively at the scenario AMD used for its stream testing showing the differences between x264 fast medium slow and more while also showing tuning with process priority in FPS caps this video will discuss why we think AMD streaming presentation was needlessly misleading especially since we agree that the CPU should handle streaming workloads more capably than the 9900 k competitor if you're already better at the game there's no reason to stack the deck before that this video is brought to you by us and the GN store the best way to support our independent reporting is through store gamers and access net viewers like you allowed us to recently switch to paying for all of our own flights to product launch events rather than accepting flights from companies this is made possible with your purchases of merch like our GN medium mod matte in stock and shipping now and designed with GV teardown diagrams and grids our 100% custom-made two-tone shirt is also a great way to help and it's currently on sale the shirt uses 95% cotton and 5% to last in for a sporty fit with vibrant colors and was designed entirely by the GN team learn more at the link of the description below or go to store documents access net let's be very clear here we're not saying that Rison 3000 won't be good this is a matter of scrutinizing a specific part of a marketing presentation that's what it is and we've done this for just about everybody so it's important to do this because our job in the industry is to be a reality check and that's for the companies and for viewers from people who are buying the product and this did deserve a bit of a reality check so what we are saying one not that Rison 3000 is bad or going to be bad it actually looks promising on paper although wait for our benchmarks and then to we're saying that some of a.m. these presentations were misleading and do deserve some additional information shed on them so today we're focusing on the AMD streaming demonstration we already talked about Andes that measuring thermals with the risin cpu's in our navy and risin announcement video or the specs disclosure video and that was a whole separate thing but the streaming demonstration mostly amounted to laughing at intel's 9900 k sayin haha it does one FPS and that is probably for the most part of a reproducible number it's it's not we didn't a hundred percent replicate it but it's close enough that it doesn't matter because in a stream if you're not playing at a reasonable FPS and you're below ten it's irrelevant it's all bad so it is for a number that can be replicated the benchmark in that regard is sort of accurate it's just that it's misleading and that's the the hard line that we just redraw and we need to explore today in this video so one of the things i md said here is that with the twelve core cpu quote you can do things that you couldn't do before the other guy with eight cores sorry which is sort of throwing the and the 2700 acts under the bus as well which is a very capable streaming performer by the way although a.m. these sort of disclosed some of its testing procedure it was still misleading for being an inaccurate representation of the need to use the settings that were shown in the demonstration in a real environment so before anyone starts slamming the keyboard typing in the comments section below or remind you of this many enthusiasts felt that intel's principal technologies benchmarks were misleading so if those same people are arguing that AMD wasn't misleading strictly on the basis that it disclosed its testing approach that's a double standard we saw a lot of that on twitter and just wanted to address it now principal technology is disclosed in far greater detail how it tested to the extent that we were able to really burn them with it we literally showed up at pcs doorstep to ask about the benchmarks that I ran for Intel versus am the at the 9000 series launch so their results like a and these here and the stream and demonstration are replicable they're repeatable you can reproduce it in your own lap but that doesn't mean that they are realistic or representation representative of any real use case in the real world it's more similar to hiding tessellation under the world and then demonstrating that one product runs this workload better than the other while neither receives a visual benefit our quality comparisons today will help illustrate why slow settings don't only add zero value for gaming content but actually have no perceptible impact on quality when tested and compared properly this is true for locally recorded content and streamed content but especially streams and further even the core deficient ninety-nine hundred K relative to the twelve core CPU can be made the stream perfectly acceptable yet still high quality settings the Andy presentation may have made it seem to some like the 99 hundred K can't stream in-game at the same time but that's far from the truth as a quick aside our thanks to e post Vox on YouTube for helping converse with us about stream quality concepts and for talking through some of our ideas he posed box runs a channel based on improving stream quality and game capture visuals we'll link him below he provided some of the x264 flags that we've played around with for stream tuning today let's start with an on-screen comparison showing the different encoding modes helps to visualize things so we'll start with this and we'll explain more of what's going on later well change the different encoding modes on the screen occasionally is that you can see the differences between them as you're watching try to discern the differences if there are any between the different encoding modes as we change them try to guess which mode is fast which is slow and which is medium and keep in mind that the slower in name the in theory higher quality at least for the things they were designed for initially so for example slower or placebo or slow would be better quality in theory than medium or fast or faster very fast but the reason in theories in there is because it's not always the case as you'll see today so this was all captured on a 31 75 X there are never any dropped frames and we can strictly look at the image quality without the variable of performance if you've been waiting for the encoding mode to change while I've been talking we actually have been changing it well label each of them now with slow fast and medium watch these sequences playback for a moment and try to keep tabs on if you see any differences in the image quality a huge note here some benchmark scenes changed slightly like the clothes that characters wear or the vehicles on the road so it's important to pay attention only to static objects in some games like GTA 5 we're able to get much more consistency and avoid those issues together a lot of games like the Division two and Far Cry 5 which we'll show briefly as an example have dynamic lighting and even the benchmark scene this means that these particular games are useless for comparison because a lot of the arguments online revolves around people thinking that they see a difference in the shadows well if the clouds move and the lighting is affected it'll look different because of the game not the encoding choice so we've eliminated these types of changes as best we can and just gone with games that stay more static aside from elements like character clothing now that you've had some time to pick between the three settings for the best quality that you think you've seen it will reveal what each is actually we bait and switch to you all three of these were not only the same encoding option but were the same exact video nothing changed we did this because we're trying to encourage people to be open and self-aware about having a bias we all will so it's best to just acknowledge it early and figure out what it is there's no shame and a preference just make sure it's over something that actually has differences all three of these were exactly the same video overlaid and sort of fake switching between them well drop the tricks now and actually show you the difference is we're going to use one two and three labeling for a moment so that we don't influence your opinions with the actual titles of which setting is used these scenes are actually switching this time between three different captures with three different encoding presets and we want you to try and perceive a difference again the one two three labeling is a check against bias and we'll reveal them in a moment all three are used with a 10 megabit per second encoding option just like AMD did we've actually used this in the past too and they're locally captured but we also did separate streaming tests local capture was necessary for this comparison for one funny reason with the streaming tests although it worked for comparing them internally it's a terrible idea for this video because then we would have to download and reload the stream and then you just compound the compression so it's a terrible idea for this video because you'd be looking at results has happened compressed and rendered several times instead of just the ones that it needs to go up the first time aside from OBS obviously in the initial capture so by taking this route we're actually giving the AMD the benefit of the doubt much like they claimed to do for Intel although that's different story and we're doing that by uploading the highest quality possible-- within the settings used remember once uploaded you hit upload our compression by the website that further mitigates the differences between those quality choices that's a big part of the story here let's now review the basics while you watch these comparisons a little bit more keep in mind that smoke or fog effect changes our in-game dynamism and that they may have some procedural elements in the games as well that are not affected by encoding quality x264 is an encoding option available in OBS another capture software and in rendering software and is used in conjunction with the CPU to live encode video capture for stream viewers or for upload later in OBS you have the option of x264 CPU encoding or GPU encoding whether that's on Nvidia or AMD like NV encoder the x264 library has dozens of options that can be tweaked but most users will choose between ultra fast super fast very fast faster fastest low and slower that's all the presets really but you typically do that instead of hand-tuned your own options which is possible you could actually do that the slower the encoding option selected the more resources it requires in theory image quality increases but there is also a placebo level of quality at some point and there's also a right place to use each of the settings like slow and slower and it's not typically game streaming or capture if ever really but the bit rate at which games are streamed and captured is also lower than high quality media like you might get from a movie or something most streamers use very fast or faster if streaming on the host system might be the system playing the game and streaming at the same time with a bit rate likely of 6 megabits per second max twitch does not permit a stream greater than 6 megabits per second but YouTube does so we actually don't have too much of an issue with Andy's use of 10 megabits per second bitrate we actually use that bitrate for our own testing because we use that bitrate for our own streams to YouTube and so we aren't too concerned with that aspect but if you care only about twitch streaming just keep in mind that you're more interested in 6 not in 10 and for our benchmarks and you'll see them when the CPUs come out we do actually benchmark both 6 and 10 so you have a representation of each now determine if you think there was a difference between one two and three and make a note of which you thought looks best when I reveal them if you thought fast or medium looked best that means you're in potentially placebo territory and there's probably no actual meaningful difference between the captures you were just trying to find one which is okay we did that too when we first went through these tests and tried to discern the differences so let's reveal what those clips were now we'll explain some more while they play back with the proper name revealed you'll see these frames repeated from earlier and we might add a few additional quality settings and here for you very fast we'll start showing up and showing some compression differences where the others don't necessarily but we maintain that slow medium and fast all look functionally equivalent to locally and that should be doubly true once uploaded there may be some small pixel pixel differences but there won't be quality differences like better or worse it's just how YouTube at this point is processing it and if it is a problem we'll upload it somewhere else uncompressed although that is part of the the point here too is that YouTube does compress but anyway as for what each mode does while that runs through some more here's a breakdown slow versus fast which is usable on the 9900 KB by the way I will prove that later keeps adaptive quantization - mode 1 for both settings enabling adaptive quantization the ability to redistribute bits within a frame going up to mode 2 would redistribute bits across the entire video which isn't useful here and go into a cue mode 3 will net more distribution to dark parts of the scenes but that option is not enabled in any of the presets and we're on one for both of these to being compared slow changes be adapt to mode 2 from mode 1 with fast and very fast which changes to a slower and newer algorithm that suffers a speed reduction with a higher B frame setting or bi-directional frame but how our chart shows that B frames in both fast and slow are the same and B frames will provide a pixel Delta that contains only the information from changed pixel values frame the frame but not the absolute value for every pixel in the frame these act as an intermediary step in the temporal process for example the change from frame 0 to frame two the newer algorithm will slower but does not improve the quality output in a gaming stream it takes more resources to run without any added value remember a lot of these settings are for video transcodes and encode and as Ipoh Spock says again linked below more for pixel junkies and enthusiasts than they are for game streaming where the value is anywhere from low in the value add to non-existent here's a chart on the screen again to help illustrate these sourced originally from archived site bean dog d-block remains at the same between slow and fast this setting impacts encoding quality and time more heavily than some of the others but only changes with the ultra fast preset slow changes direct from spatial with fast to auto which means that slow will choose between spatial and temporal particularly in a multi pass and code where each option can be attempted and then the better choice can be kept it's sort of averages along the way in that second pass you get some additional data much of the usefulness is lost in a single pass game capture environment particularly one where we're working with relatively low quality to begin with more notably slow changes Emme from hex to umh from the ME GUI wikibooks entry this change is quote considerably slower than hex what searches a complex multi hexagon pattern in order to avoid harder to find motion factors unlike hex and dia the ME range parameter directly controls at um HS search radius allowing one to increase or decrease the size of the wide search this should theoretically help with motion ending the quotes there but as you'll see in our demos there really isn't any tangible difference with a game stream slow also changes look ahead to 50 measured in units of frames up from 30 with fast this means that 50 frames will be used for rate control and with regard to mbtree should theoretically increase video quality at the cost of speed but again a lot of this will depend on what kind of video you're encoding and for what Pervis h.264 after all wasn't invented for this purpose solely it wasn't made for game streaming it does a lot of other stuff and we use it to encode all of our videos too there are a few other changes and you'll see them in the chart but what else matters is the quality comparison in this use case it comes down to a qualitative look performance is the other angle of this getting into performance we'll be looking at stream benchmarks using the division - just like AMD but we're going to run at 1080p locally in streams this is because the 1440p local setting that AMD used created a GPU constraint on the host system they did stream at 1080 though and this artificially limited the player side performance to bring the Intel CPU even with AMD CPU it also creates enough GPU load with Andy running at nearly 100% GP load during its stream evidenced by the in-game UI on the right side of the screen it's enough GPU load that OBS will miss some of the frame renders we believe this removes Headroom for OBS is a scene compositing and when checking with a post box it sounds like he believes the same this introduces variables that AMD should have accounted for in fair testing but instead create it we'd be curious to see what the render lag metric was for Andy's test runs along with the percent delivered within 2% of 16.6 67 milliseconds as these were some of the only metrics not published starting with viewer performance this is the output that the viewer will see when watching the stream except represented as percentages this testing was done with real live streaming over a fiber network we tried a lot of settings especially now that we know how similar fast medium and slow and often very slow will look to each other and we found that the 9,900 k stock cpu without any overclocking at all with turbo duration limits enabled and running at in tiles at 95 watt TDP it's capable of game streaming with 10 megabits per second fast settings with the division - at Ultra settings again that's with turbo boost duration limits enabled so we're running this thing properly stocked more so than almost any motherboard will do for you out of the box by disabling the non stock mother would power override options with fast settings and no tuning the 99 hundred K delivered 100 percent of its frames to stream meaning that the viewer did receive a 60 FPS stream while the player maintained a 119 FPS average gaming framerate with a low as bottoming out just at 50 fps this is reasonable when considering the local streaming going on in the background the host system always hits a 0.1% low as hard even with the high-end HED t18 core CPUs and can be improved it primarily if not only by using a secondary system this is what seriously competitive streamers should be doing to avoid frame time drops if they are actually competitive in the games they're playing and they need that frame time consistency the only downside of the fast stream is that it's frame delivery pacing sits at 51% within plus or minus 2% of sixteen point six sixty seven millisecond delivery we're still within a few percent of sixteen point six six seven milliseconds overall enough that we hit the 60fps deliverable target but the frame pacing is imperfect and could be improved this is actually more results of the frame lag caused by the higher GPU load than by the CPU load created but we can make some changes on both sides of things to try and address this this is also something that AMD likely would have counted at its e3 presentation but did not talk about or discuss on stage we demonstrated this by carefully reducing the GPU load with in-game options but by keeping the CPU load exactly what was previously plus or minus about 1% error with these changes frame pacing improves to 93.3% within plus or minus 2% of sixteen point six sixty seven millisecond delivery and further demonstrates that AMD's test was flawed there was too much GPU load in the scenario creating external variables in frame lag because OBS is also running behind on the GPU resources it needs this is why we spend so much time explaining our streaming tasks when we do them and often dedicate in 10 to 15 minutes just doing stream tests there are a lot of variables and it requires a lot of care to do properly when I say dedicating 10 to 15 minutes I mean in one video where the whole video length might be 25 minutes anyway going back to the ultra settings by using real time process priority for OBS we can see that we improved the frame pacing to the viewer from the original 51 percent to 71 percent but the reduction in our other chart was to 0.1 percent lows for the player this too can be tuned out by keeping process priority targeted at OBS and setting an 80 FPS in-game cap we're now freeing up CPU resources from the game which is spewing potentially unnecessary frames for the player and locating them instead to the much more important OBS we now have a 97% delivery within plus or minus 2% of 16 milliseconds this results in a high quality stream for both the player and the viewer and can be tuned further with other settings of desired medium encoding delivers 59% of the frames to the stream so is where the ninety-nine hundred K starts to struggle with this title ultimately we found that medium encoding does work just fine for the ninety-nine hundred K in some games but it depends on how intensive the game is on the CPU in this instance the ninety-nine hundred K has trouble out of the box without any changes it can do fast and with some small changes it can start to do fast with a near-perfect delivery in terms of frame pacing but it's already pretty good it's good enough that most viewers won't really complain and if you're a professional streamer you'd either want some heavy tuning on there like some of the stuff we've done in here plus additional x264 tuning or you just get a separate system and really do it the proper way slow is too abusive on the 1900 K for it to handle on its own so I'm tuning with priority and FPS caffeine boosts it's a 30% frames delivered but that's obviously an entirely pointless stream with very low viewer quality and you will almost certainly win in this comparison but it's a pointless and misleading victory ultimately just like running a game with tessellation under the map Crysis 2 will make AMD look worse than Nvidia the actual visual quality is not improved even though the FPS looks different whenever we've used slow in demonstrations in the past we've been extremely careful to outline why it's not a useful comparison beyond a purely synthetic workload our issue with AMD's presentation is that it wasn't presented this way but instead mislead viewers to believe the quality would be meaningfully superior at the end of the day the comparison should have been something else because realistically the 900k can stream every bit as well as any of you will ever need for a single stream slower isn't actually better here anyone getting more serious should be using the secondary system anyway because the frame times will always be more consistent without background processes something we've extensively shown in the past a better demonstration would have been Andy demoing it's a likely capability of managing two simultaneous live streams both twitch and YouTube for me will have multiple audiences something that the 99 would struggle with without using any rebroadcast or service another great example that we would have loved to see AMD use would be doing a local recording with x264 and also doing a live stream this is useful if you're a streamer who wants to upload to twitch live and then maybe upload bits and pieces of your performance to YouTube later without having to read download the vaad through twitch so you retain a lot of that quality this is a valid comparison and one where the 12 Corizon cpu would probably win we don't know for sure because we don't have it but almost certainly be better than nine 900k added well certainly it would be actually and it probably would be good enough to do both of these things with very fast or maybe fast or faster settings not slow but something that's actually useful a dual stream test has some real use cases and would still have had AMD in the lead but in a more useful realistic metric we're not trying to say and II won't be better at this little comparison it's just that the comparison is useless and misleading because it provides no value and a massive performance hit if anything we're trying to help and the marketing better represent its own product finally looking at the chart for player side FPS we're hitting 147 FPS average with baseline performance this metric highlights the CPU without any streaming constraints functionally an unhindered gaming the frame rate and is useful for illustrating the impact to frame rate and time that streaming has going down to a 10 megabit per second and fast from baseline we maintain 80% of the original speed while also maintaining a 60 FPS livestream Albi it with mixed frame pace and results instituting a frame cap obviously caps the frame rate 280 FPS but does allow 0.1% lows to climb back to 50 FPS while improving frame pacing to near perfection on the stream side you can also see that these slow settings aside from the frame capped one maintain a high frame rate this begins to show a resource allocation issue compounded with an overall lack of sufficient resources on the Intel CPU for these unmodified presets and for the stock clock hardware that frame time hit by adding a stream always brings us down to 50 FPS for 0.1% lows in the best case in this game and we're looking at the better performing options you'll see that's true across both of them in previous testing we have Illustrated that this happens on both Intel and AMD sometimes with much worse percent lows and it's a nature of live encoding from the system that's also hosting the game so that's it that's really all this was it was okay they're testing this with slow settings let's explore this and a couple of things here we are happy to take credit for being the ones to popularize this type of test so we started doing this around the 7900 acts launch and our first set of testing or one of them was to do dual streams where you broadcast the two services at the same time without a rebroadcast service in between to eat quality or introduce latency and that's a valid use case sort of but it's rare one another use case we've tested in the past was local recording and streaming and that's where we think Andy should have focused its attention because it's a very valid use case for streamers who are serious about pulling some of their content uploading it to YouTube and bits and pieces before someone else captures it off of twitch and reuploads it under a different name so another valid use case AMD former one of the former marketing team members told us in person that he had borrowed some of this testing methodology for their own marketing presentations which is fine but it being that they did borrow this in part from us originally whether or not the onstage presentation did this at e3 we obviously caught we knew the deal like we've done these tests we know what we were looking for and knew it was reasonable and what wasn't so anyway the biggest problem we have with the test is that it has already propagated a mentality that this quality preset in OBS is somehow advantageous in any meaningful way that's the key word meaningful way we can't see it if there is one so there are places where these other quality settings with x264 or h.264 if you prefer are useful and as he posed Fox said pixel junkies maybe if you're converting files or transcoding videos that are higher quality dealing with with greater level of detail than something like a game would especially stream that 6 or 10 megabits per second that's where you might start seeing some advantages but for these purposes it is not a tangible advantage and that's the problem is a lot of the discussion around this from people who've never done any streaming before or don't really know anything about it I haven't done any testing of streaming before have no idea how its tested those are the people who are out in forums on reddit wherever and defending a presentation with really there's no reason to like Andy and their CPU they'll do just fine in game streaming they'll almost certainly be ahead in a reasonable use case for this application as well it's just the matter of how threads work with streaming content so this is something that Andy would have won regardless it just we would have liked to have seen it with a real use case not something that is synthetic to the point that again it's putting tessellation under the map in Crysis 2 and then showing that Nvidia tested better that's the same kind of thing that's happening here so anyway a lot of people thought that SLO would actually improve the stream quality and started parroting the presentation so we wanted to put this video together and beyond that there's this this sort of fascinating double standard with AMD where when intolerant Vidya get criticized by the media and we've done a lot of that for misleading marketing practices there's a sort of peanut gallery that cheers it on and then when AMD gets criticized there's a big jump to the defense of AMD but Andy's not a person being picked on they're not a kid in the playground and he's a big company too and Andy needs to needs to be challenged for its marketing as well especially as it gets bigger it gets for a confident that's going to become increasingly important so you don't need to find any reason to defend AMD is what we're saying and at the end of the day we do this to everybody just as a reminder because some people didn't do the usual Joel comments about this as a reminder we published a video at Computex last year titled Intel's 28 cores of bullshit and then we also went to PT principal technologies visited them we had a scene were kind of counted on counted from three down and flipped off the camera because of the testing methodology with Nvidia we put them in our disappointment bill and on a shirt that we sold a lot of for the disappointment build so if this happens to everybody from us it's not meant to be personal for those of you who are big game the fans please take a step back look at the quality comparisons we did and try to remove that bias and just think about well okay so this wasn't really a valid comparison but there are a lot of valid comparisons and let's just focus on those because again if you're already good at the game there's no reason to stack the deck and this is a matter of AMD has a lot of things that its processors are genuinely really good at so we'd like to see those focused on not creating these synthetic scenarios that will actually benefit nobody in the game streaming space which is where it was targeted so that's it for this one thank you for watching subscribe for more if you want to support this type of content you can go to store documents nexus net to pick up one of our mod maps larger on backorder you'll be guaranteed one if you backorder it and medium are in stock or you go to patreon.com/scishow and axis thanks for watching we'll see you all in that