Comparing the performance of two leading flagship phones is a good way for the tech media to help potential smartphone shoppers make an educated decision. For hardcore fans of a particular brand, however, these comparisons offer little use beyond bragging rights — which is obviously quite bizarre when you stop to think about it. Specs and benchmark test results would never convince a fanboy who stays loyal to one brand to switch to another. Instead, benchmarks are really only useful to smartphone shoppers who care more about performance and overall experience than they do about brand allegiance.
Regardless of how useful benchmarks really are to smartphone fans, they are and will continue to be a big part of the gadget news cycle. And with Samsung’s new Galaxy S9 and Galaxy S9+ becoming available for preorder this week, benchmark test results for Samsung’s new flagship phones are a hot topic. In particular, one set of benchmarks from a blog that is known for its thorough technical analyses is causing a bit of a stir.
Headlines like “Last year’s iPhones are destroying Samsung’s new Galaxy flagship smartphone in benchmark tests” have popped up on several different gadget blogs and Apple fan blogs over the past few days, following the publication of Anandtech’s Galaxy S9 benchmark test results. What’s odd about this is the fact that Anandtech, the source itself, stated repeated that something is very wrong with its benchmark test results.
Without diving too deep into the technical details, the short version is this: Anandtech probably tested a bad or improperly throttled device. The author on Anandtech made this fairly clear in its coverage, so we can only assume that some of the bloggers covering the story didn’t read far enough to get to that part.
While the results of some of the site’s tests seemed reasonable, most of the results for the Exynos 9810-powered Galaxy S9+ Anandtech was testing were awful. Some test results were even worse than phone models from last year, including Samsung’s own Galaxy S8. And while Samsung’s Snapdragon and Exynos variants are typically closely matched in performance, these tests found that the two chipsets are nothing alike.
Here are some examples:
If anything, Samsung’s Exynos-powered phones typically perform a bit better than their Snapdragon counterparts. And yet in these tests, the opposite is true by a wide margin. In fact, these tests show Apple’s iPhone 7 from 2016 beating the new Galaxy S9+ in several tests. Seems like something is fishy, no?
“What seems clear is that there is something is very very wrong with the Exynos 9810 S9+ that I tested,” Anandtech wrote in its post, clear as day. “It was barely able to distinguish itself from last year’s Exynos 8895, let alone the Snapdragon 845 in the Qualcomm Reference Device which we previewed earlier this month. I looked through the system and monitored frequencies and indeed the big cores were reaching the maximum 2.7GHz core frequency. The only explanation I have right now is that it’s possible that the DVFS configuration, as well as the scheduler, are currently so conservatively tuned that there is barely any activity on the big cores.”
Apparently Apple blogs missed this part.
The post continues, “One of the Samsung spokesmen confirmed to me that the demo unit were running special firmware for MWC and that they might not be optimized. I’m having a bit of a hard time believing they would so drastically limit the performance of the device for the show demo units and less so that they would mess around with the scheduler settings. I did get confirmation that Samsung is planning to ‘tune down’ the Exynos variant to match the Snapdragon performance – however the current scores which I got on these devices make absolutely no sense so I do hope this is just a mistake that will be resolved in shipping firmwares and we see the full potential of the SoC.”
It seems like Samsung may have broken something when tuning this pre-release device, but everything should be cleared up once reviews start hitting the web.
