Samsung Isn’t the Only Android Maker Cheating in Benchmarks

A few months after the Samsung Galaxy S4 was released, it was revealed that Samsung had installed a special BenchmarkBooster application that overclocked the smartphone when benchmark tests were running. Thus, the Samsung GS4 experienced a 10% to 14% in benchmark results - enough to put it past competitors like the HTC One. The GS4 detected popular benchmark programs like 3DM, AnTuTu, AndEBench, Basemark, and Vellamo and increased its clock speed when these applications were running. Samsung denied that it was cheating - even though BenchmarkBooster was a real application found on the GS4. And now, it looks like Samsung is trying to pull the same tricks with the Galaxy Note 3. Reports have come out over the past week that reveal a special “high speed mode” on the recently-released Note 3. That high-speed mode causes the Note 3 to blast past the benchmark of other smartphones with similar specifications - like the LG G2. While the LG G2’s speed may look awful compared to the speed of the Note 3, the reality is that Samsung is artificially boosting its performance. Despite the fact that both the LG G2 and the GS4 feature the same 2.3GHz Snapdragon 800 processor, the two smartphones get totally different benchmark results.

Here’s how the Note 3’s cheating system works: -Upon detecting benchmarking apps, the Note 3 kicks its CPU to the highest possible speed: 2.3GHz -This boost lets it achieve abnormally high scores -After the benchmark test is complete, the CPU returns to a normal speed and loses any perceived advantage it had over its rivals

It’s not just Samsung

Samsung once again vehemently denied claims of benchmark cheating on the Note 3. But according to an excellent investigative report by Anand Lal Shimpi and Brian Klug of AnandTech, Samsung isn’t the only Android maker that tries to fudge its benchmark results. In fact, just about every Android maker uses benchmark boosting software. With the exception of Apple and Motorola, every major smartphone manufacturer has at least one device that performs CPU optimization when running benchmark software.   Here are the results of the AnandTech study:

As you can see, devices like the Samsung Galaxy Note 3 feature CPU optimization for nearly every major benchmark test. Notable non-cheaters include the Nexus 4, Nexus 7, Moto X, and Moto RAZRi. So what’s the point of running benchmark software anymore? Can we ever really trust the tech industry to play fair? Probably not. If you want to learn more about the methodology behind the Android benchmark boosting study and testing, then you really should check out the AnandTech article here. By the way, the most surprising part of this study for me is that Apple doesn’t fudge its benchmark results. I guess it’s because Apple restricts the number of iOS benchmarking tools allowed on its app store.