Gadgetory


All Cool Mind-blowing Gadgets You Love in One Place

Why and how do OEMs cheat on benchmarking? - Gary explains

2017-02-09
how they mind ghosts in from and royalty now based marked eating has come up again in the news this time it was one plus has been accused of cheating the benchmark scores on the one plus three T they're not the first company to ever be acute of benchmark cheating so the question is why do OEM try to cheat on the benchmark and how do they do it well let me explain so first a bit about the types of people that buy smart phones I suppose broadly speaking there are three groups of people that buy smart phones the first group of person doesn't really care much about the specification or the speeds they'll go into the shop to buy the phone that are for sale the system they'll say yet the good phone they like the price they'll buy it at the end of a transaction done deal the second category of person are those that have a bit of technology understanding they might understand what the specs mean they'll understand the difference between four gigabytes of RAM and two gigabytes of RAM 16 gigabytes internal storage 32 gigabytes internal storage they might have screen resolution that kind of thing and they'll make an educated decision based on what they understand about each of the phones they may also have read some stuff all from the internet but it isn't their area of expertise but they know enough about technology to make an educated purchase and I suppose the third category of person is the one that is basically the geeks or nerds that knows everything about smartphones read all the reviews which is all the videos knows what's coming down the pipeline knows what was the best phone and you know they're the kind of person that yes yes hi oh hi yeah I'm making it making a video okay it'd be quick okay yep yeah it's a good phone it's good phone it what's the price Oh excellent price yes great off-contract brilliant okay but I've got to go okay so thank you bye bye fight fight fight fight sorry about that so they're the kind of person that you ring when you want to know what is a good phone to buy so the question is how do we M actually cheat well what actually happens is when a device is made the balance has to be struck between performance and power usage now really these two our enemies are at loggerheads with each other maybe give an example on your desktop you've probably got an Intel or AMD CPU it may be giving out between 50 a hundred maybe more watts of thermal heat it is got big heatsink on it it's got cooling fans leakages running from the mains electricity now for smart phones at the other end of that scale they are run by batteries they mustn't get too hot they've got no fans on them and basically the more performance that you want to get from a mobile device the more heat it produces and the more battery life it uses so there's a balance between getting good battery life and getting a good performance now when the device makers make a device they make various decisions in the hard way like the processor and the clocks means the processor and the clock speed of the GPU and other things like that and they also make some decisions in the software now the software can of course be tweet dynamically while you're using the phone the phone is actually always changing the speed of the CPU the speed of the GPU it's rotting is increasing there's a whole technology which I've covered in some of my other videos now every app that runs has a unique ID and those unique IDs are part of the information that's put on the Google Play Store and so the UID the unique ID of every app remains the same so antutu has the same UID today and tomorrow and it had it last year and basically it's not very hard for an OEM to put into its firmware something that detects when a particular app is running and if it is running it can tweak the software to say hey let's boost the CPU artificially during these few next few minutes boost the GPU now it's not the normal circumstances the phone would use because if it ran at that speed the whole time the batteries go flat very quickly the phone would heat up very quickly but for a 90 second test for a 2 minute test it's not going to make any long-term difference and so the way they're cheating is basically when they see antutu running or deep veins running or some of the the popular one basically the firm waited hey let's push up the cpos push up the GPU and then it gives out different results than it would be if it was using its normal settings its normal performance settings for when it's running a normal program now of course there is this question I'll throw it out there I'll say something I think it is cheating but let me judge this is it really cheating because actually it's not like the firmware has somehow managed to send messages or inject data into the benchmark and actually get it to display a full score that was the score that that device produced that is what that device is capable of doing but it's not what the device does day to day because to reach that capability it actually has to run the CPU much harder than it would under normal circumstances but let me look at it this way what about the speed things the car subject it will do nought to 60 in you know 4 seconds or something like that well how many people actually pull away from traffic lights to go north to 60 in four seconds well you don't just too dangerous it's not what you do but actually people will say wow look that's the speed of that car so you know they're artificial statistics that exist in all industries and maybe the benchmark are an artificial statistic in this way in fact they are artificial in the sense they don't really reflect the workload that a normal app that does I mean even pain quite complicated games doesn't actually reflect what you get in the benchmarks I've done some other articles about this that shows you how the benchmarks use the CPU and GPU the graph sort of produce and they're very different to what you get under normal games so Bayes marks are artificial to start with so if the smartphone is using the mark officially well is that really being issue I get it it I say it is cheating because people expect the phone to run that fast in normal usage and it doesn't but it's an interesting question I'd like to hear your thoughts in the comments below and so there you have it the OEMs are cheating on the benchmarks now this is not the only industry that is happening it's happening in the car engine we've seen that with emissions within that with fuel consumption figures this is going to happen in all industries because it's so cutthroat that people the OEMs are trying so hard to get head so their product sells the most doesn't means right but I'm just saying it's a reality my name's Gary Sims from Android source and I hope you enjoyed this video if you did please do give it a thumbs up don't forget to subscribe to Android authorities YouTube channel P tell me in the comments below what you think about Oh am cheating on benchmarks download the end or throws the app and don't forget to go over to Angela source calm because we are your source for all things Android
We are a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for us to earn fees by linking to Amazon.com and affiliated sites.