After benchmark results for the recently launched HTC One (M8) showed that the handset can offer great performance when compared to rivals – especially the Asian version which packs a slightly more powerful processor than the models that will be sold in other markets – new reports have suggested that HTC may be cheating significantly in benchmarks, at least when it comes to AnTuTu.
Taiwanese site ePrice has compared benchmarks for a variety of devices in both the old AnTuTu app – which Android OEMs managed to fool into showing higher scores in the past – and the AnTuTu X app and found different results for the Asian HTC One (M8) model. The model scored 38,815 in AnTuTu, but just 27,171 in AnTuTu X. The site also tested the LG G Pro 2, the Galaxy S5 and the Xperia T2 finding similar scores for each device in the two benchmarks.
While benchmark results may not be important for all mobile device makers, they’re clearly important for some customers, but also for the companies whose business is to offer benchmarks platforms. AnTuTu X was launched last year in response to reports that proved Samsung, HTC and other Android OEMs can optimize their devices for certain benchmark apps, in order to fight the OEM’s tricks and show actual scores.
BGR has found similar results in AnTuTu and AnTuTu X benchmarks when testing the UK version of the handset (image above). The handset scored 33,524 and 33,632, respectively, while running in default mode.
AnandTech, who called out Android OEMs for their optimizing practices related to benchmarks last year, found a “subtle form of benchmark optimization that we hadn’t seen in previous devices” when reviewing the new HTC One. At first, there’s no CPU optimization, but in time CPU frequency goes up when running a known benchmark application. According to the publication, the optimization targets many benchmark tests including GFXBench, BaseMark X and BaseMark OS II. AnTuTu tests were not mentioned.
Interestingly though, HTC has included a high-performance mode under Developer tools, allowing users to run the device in high-performance mode at all times, which is the mode apparently employed when known benchmarks are detected.
“I do appreciate that HTC is exposing the optimization control, the only thing missing is the ability to toggle the benchmark optimization off (not to mention that I’d prefer if it was disabled to begin with),” AnandTech wrote. “I fear that HTC’s justification in all of this is that everyone else is doing it so why opt out. The reality seems to be trending the other direction however. We’ll have to see what Samsung does with the Galaxy S 5, but I have a feeling that HTC is going to end up on the wrong side of history with this move. All of our benchmarks are already immune to the optimization, so it’s really a matter of sacrificing integrity for no real gain. There’s nothing more to say other than I’m disappointed.”
Screenshots showing ePrice‘s AnTuTu and AnTuTu X tests for the HTC One (M8) and competing flagship handsets follow below.
