- A research paper about the Sturgis bike rally in South Dakota that was held back in August attracted heavy media attention when it was published, because of its claim that more than a quarter of a million coronavirus cases in the US were attributable to attendees of the rally.
- However, new research suggests that paper may have gone too far in trying to make its case, with even researchers at Johns Hopkins arguing in a new paper that the model behind that original paper was “relatively weak.”
- The roughly 266,000 coronavirus cases the original paper blamed on the Sturgis rally represented 19% of all new coronavirus cases in the US for August.
Remember the Sturgis bike rally that was held in South Dakota back in August, the gathering that brought a few hundred thousand people together who seemed (at least from news reports) to have a devil-may-care attitude toward social distancing and wearing face masks? A study published by researchers from the Center for Health Economics and Policy Studies at San Diego State University, rocketed around the Internet earlier this month that, among other things, attributed more than a quarter of a million coronavirus cases to this one rally. Making the rally one of those things we call a “super spreader” event — one whose consequences are brought into even starker relief when you consider that for the month of August, the US recorded 1.4 million COVID-19 cases. The study blames some 266,000 of those on the Sturgis bike rally, which would mean this one event was responsible for a whopping 19% of the COVID-19 cases reported for the entire US last month.
Spoiler alert: Let’s just say, South Dakota Gov. Kristi Noem blasted this study as “fiction,” and without dismissing its findings entirely, she does kind of have a point.
It should probably wake up most people who were initially freaked out by the study’s findings to learn that while, yes, the right-wing editorial opinion section of The Wall Street Journal has poked holes in the study in recent days, so has everything and everyone from Slate (which has a decidedly leftist bent) to Johns Hopkins University researchers as well as Ashish K. Jha. He’s an adjunct professor of global health at Harvard T.H. Chan School of Public Health, as well as a professor of medicine at Harvard Medical School.
Before we dive into the re-evaluation of this study, though, let’s rewind the tape a few weeks and replay some of the breathless media coverage that parroted the study’s findings to label the Sturgis rally a COVID-19 super spreader event — coverage that came from sources that many news consumers would consider authoritative.
From The Washington Post: Worst case scenarios at Sturgis rally could link event to 266,000 coronavirus cases, study says.
From NBC News: Sturgis rally may have caused more than 250,000 new coronavirus cases, study finds.
From The Hill: Sturgis Motorcycle Rally was ‘superspreading event’ that cost public health $12.2 billion: analysis.
In brief, what the study claimed to find was the following. Based on cell phone data that pointed to where the rally-goers came from and where they went back home to, the researchers plotted coronavirus case increases in those areas both pre- and post- the rally. That helped them deduce that some 266,000 new COVID-19 cases seemed to be attributable in some way to attendance at the Sturgis gathering.
This led to lots of a-ha, see, we told you so media coverage. Flout the basic protective measures that health officials have been repeating for months, gather in large crowds and refuse to wear a face mask — and, well, you might just end up being responsible for 19% of all coronavirus cases in a given month. Or even more, who knows!
These findings, needless to say, have begun to be picked apart in recent days. A recent piece from the decidedly left-leaning Mother Jones, for example, gave credence to the basis of the WSJ editorial (though without agreeing with it outright): “I was open to (the WSJ piece) for a couple of reasons,” the Mother Jones writer explained. “First, like a lot of COVID-19 research, this study was released quickly and without peer review. Second, the figure of 266,000 really did seem awfully high given the timeframe and the number of infections attributed to actual attendees. So even though I normally don’t waste my time with Journal editorials, this time I did.”
And then there’s this, from Jha via Twitter:
I don't buy Sturgis study below.
Why? Really big effect. Makes me skeptical
One rule of thumb I teach is:
if effect size of a social phenomenon using noisy data is very large — be wary
So was Sturgis harmless?
No.
But I doubt it caused 250,000 cases.
Thread https://t.co/bCIlL5trWq
— Ashish K. Jha, MD, MPH (@ashishkjha) September 9, 2020
To Jha’s point about the fact that while he disputes the finding of 266,000 cases he still stresses the rally wasn’t harmless, there’s little reason to dispute that. A paper released by a team from Johns Hopkins University on Friday, in fact, says that same thing — that the rally no doubt contributed to at the very least a localized increases of cases in South Dakota (and probably beyond). While the new paper also argued that, at the same time, it’s awfully hard to follow rally attendees back to their home in, say, Los Angeles County (where, mind you, coronavirus was already spreading like wildfire) and attribute any post-rally increase there to the Sturgis attendees.
“The case data show relatively stable trends prior to the event and clear changes around the event, with little reason to believe that the changes in cases could have been caused by anything but the event,” the Johns Hopkins researchers note in their paper. “The overall conclusions that the Sturgis event caused a large increase in COVID-19 cases and infections are likely to be relatively robust to the specific statistical methodologies used.”
However, they go on to note that the study cited above which attracted so much media coverage has a “relatively weak” model that was used to arrive at its findings, which should be “interpreted cautiously.” For one thing, this new paper notes, it’s probable that the Sturgis rally caused many people linked to it to get a coronavirus test. And that testing increase could explain an increase in cases — with those tests identifying asymptomatic people who might not have realized they already had COVID-19, for example.
“There is evidence in that paper that Sturgis increased infections across the United States,” Elizabeth Stuart, a statistician and associate dean at the Johns Hopkins Bloomberg School of Public Health who reviewed the San Diego center’s study, told CBS MoneyWatch. “But do I believe that number is 200,000 or more? I am not sure.”
