We’ve seen a barrage of DxOMark camera reviews in the past few weeks, as several high-end smartphones came out, all of them sporting top-of-the-line camera experience. First, the camera pros declared the iPhone 8 Plus to be the best smartphone camera they ever tested. Its score was tied by the Galaxy Note 8 quick after that. Then came the Pixel 2 which took the top spot, according to DxOMark’s ranking system.
The problem with assigning scores to camera performance is that it’s not an entirely objective endeavor. While you can measure some camera characteristics with precision, there are various aspects concerning photo quality that are subjective. And a new video perfectly illustrates why DxOMark’s scoring system is significantly flawed.
Marques Brownlee explains that DxOMark scores aren’t entirely arbitrary, but they’re not adequately explained in these reviews. Moreover, the final grade a phone gets is not an average of all the photo and video scores. And that’s something we pointed out when looking at the Pixel 2 review, a phone that has a higher rating than the iPhone 8 Plus and Galaxy Note 8 but offers worse portrait and zoom features.
Brownlee also tackles one other issue we’ve identified with these scores. The Pixel 2’s record 99 score isn’t out of a maximum of 100. No, phones will surpass that mark, and some cameras are already ranked over 100 on DxOMark. But that’s not immediately apparent to readers, and you have to go through the entire review to find the fine print that explains it.
Finally, Brownlee also explains one issue we have not addressed in our coverage of these reviews, the fact that DxOMark works with certain companies on improving camera performance — like it did with OnePlus earlier this year. However, the DxOMark fails to identify in these reviews any potential conflict of interest when reviewing smartphones from companies that have a business relationship with DxOMark. Or I totally missed it.
For example, given the fact that Google bragged with the DxOMark Pixel 2 score on stage during its recent media event, I’d be inclined to think Google is a customer. And I may be wrong.
Check out Brownlee’s full video on DxOMark, which I personally score 99 out of to-be-decided, below.
