We’ve written often here about the different tests that attempt to measure your home broadband speed, which includes crowd-sourced apps from companies like Ookla. And how as soon as you point to results from a source like Ookla, you’ve got to couch them behind caveats like the fact that crowdsourced reports aren’t controlled, and differences in methodology between the networks can introduce bias into the data.

It’s a hard thing, measuring broadband speed, and to make matters more complicated is the fact none of the methods are “perfect.” That’s because everything from the equipment you use to the time of day you’re accessing the internet can affect network performance. Measuring everyone’s speed is not as simple or objectively straightforward as using a ruler to, say, measure the length of an object on your desk (in which case there’s one right answer, and it will be the same every time). This brings us to AT&T, and specifically to a step it’s taken in response to the FCC’s SamKnows speed test awarding the carrier a less-than-flattering score on its DSL internet service. What do you do if you’re a broadband provider and don’t like the results from a third-party’s test of your offering? Why, just pull out of the test and start your own, of course. 

That’s what AT&T did after it dropped out of the FCC speed test program, telling Ars Technica that the company “developed a best-in-class tool to measure its consumer broadband services. This tool measures performance on all AT&T IP broadband technologies and is more accurate, versatile, and transparent. For these and other reasons, our tool provides better and more useful information to our customers.” More useful information, that statement implies, than the FCC’s speed test — in which AT&T only wanted its IP broadband to be included, while its not-so-hot DSL broadband would be ignored.

Per Ars, the FCC in December of last year released both its 2017 and 2018 “Measuring Broadband America” reports, with at least one key difference between them. The 2017 report includes AT&T’s older DSL technology which provided 82% of advertised download speeds. In the same report for 2018, however, only AT&T’s newer IP broadband is included, leaving out the worse-performing DSL service.

Why does all this matter? For one thing, if AT&T can persuade unimpressive results not to be reported, imagine how much of an incentive it will have to keep bad results from being disclosed when it’s the one running the test. This also extends beyond speed testing, as AT&T, among others, has argued that it shouldn’t be required to submit in-depth 5G coverage maps to the FCC. And in a report the FCC released earlier this month, the commission noted its concern that carriers like Verizon and T-Mobile had “overstated their coverage” in their 4G LTE mobile broadband coverage maps.

Andy is a reporter in Memphis who also contributes to outlets like Fast Company and The Guardian. When he’s not writing about technology, he can be found hunched protectively over his burgeoning collection of vinyl, as well as nursing his Whovianism and bingeing on a variety of TV shows you probably don’t like.