MY GOODNESS, is it that time of year already? It’s been several months since the last round of articles warning that running too much will kill you–must be time another another one. What’s that? No new data to publish? That’s no problem, we’ll just republish the same data. The media never bothers to check these things, and always reports it as if it were brand new.
The new article is published in the Journal of the American College of Cardiology, analyzing data from the Copenhagen City Heart Study. (And here, on cue, is one of the requisite newspaper articles: “Fast running is as deadly as sitting on the couch, scientists find.”) The exact same data was published back in 2012 in the American Journal of Epidemiology. This time the authors are the same, but with the addition of James O’Keefe, who has been an author on pretty much every single one of the “running will kill you” studies.
As far as I can tell, the only new thing in the study (aside from the fact that a few more people have died) is that in addition to looking at hours of jogging per week, number of jogging days per week, and self-reported pace, they added a fourth category that combines the other three for an overall rating of “light,” “moderate,” or “vigorous” jogger.
So what do the numbers tell us?
First of all, I want to reiterate that the study is worth paying attention to, and we should take the results seriously if and when the data reaches any reasonable threshold of statistical significance. At this point, though, they’re nowhere near that threshold. The main problem is that sample sizes are large in the “less exercise” groups, which means they have a statistically significant reduction in mortality, but they are tiny in the “more exercise” groups, which means they don’t have a statistically significant reduction in mortality. This allows the authors to make the shamefully disingenuous argument that “strenuous joggers have a mortality rate not statistically different from that of the sedentary group”–which is almost a foregone conclusion, given that the sample size is less than a tenth as large.
The simplest and most objective way to make this point is simply to show the raw data. Here are the number of participants in each group, along with the number who died (of any cause) during the follow-up period:
QUANTITY OF JOGGING
- Sedentary: 413 / 128
- < 1 hour/week: 640 / 20
- 1-2.4 hours: 286 / 4
- 2.5-4 hours: 122 / 3
- > 4 hours: 50 / 1
FREQUENCY OF JOGGING
- Sedentary: 413 / 128
- < 1 time/week: 323 / 5
- 2-3 times: 474 / 7
- >3 times: 84 / 5
JOGGING PACE
- Sedentary: 413 / 128
- Slow: 178 / 7
- Average: 704 / 15
- Fast: 201 / 6
Now, you can search through those numbers looking for patterns. Does your risk really go up if you run more than 2.5 hours, but then go down again if you run more than four hours? Of course not. These are not real patterns, because we’re talking about one, two, three, or at most five or six deaths. No matter how interesting or important the question is, you can’t torture these numbers enough to force them to reveal the answers. They’re simply not there.
What about the combined metric of “light, moderate, or strenuous” jogger? Here’s a look at those numbers:
- Sedentary: 413 / 128
- Light: 576 / 7
- Moderate: 262 / 8
- Strenuous: 40 / 2
Yes, the conclusion of the study (that “strenuous” jogging is as bad as being sedentary) is based on two deaths over more than a decade of follow-up. (Thank goodness a third person didn’t die, or public health authorities would be banning jogging.)
In reality, of course, the statistical challenges are even more complex than what’s shown here. For example, the sedentary control group had an average age of 61.3, whereas the various running groups had an average age in their late 30s and 40s. So the comparison of death rates has to rely on imperfect statistical adjustment. You’ll notice that 31 percent of the sedentary subjects died during the decade or so of follow-up, compared to 5 percent of the strenuous joggers, who had an average age of just 37.0 at the start of the study. The researchers argue that this means the “hazard ratio” is about the same, but that requires an awful lot of assumptions about why people die in their 30s or 40s versus why they die in their 60s and 70s. Of course, with only two deaths in the strenuous group, it’s impossible to perform any sub-analysis on different causes of death. Did the joggers die of heart disease, as the paper suggests they should, or were they hit by a car or struck down by cancer? We have no idea.
The same issue arises with gender: 43.1 percent of the sedentary group was male, and 49.1 percent of the light joggers, but 80.0 percent of the strenuous joggers. Again, the researchers “adjust” for gender, but it’s not an apples-to-apples comparison, especially since the groups are so dramatically different in several traits.
Seriously, to publish this data once was legitimate. (In the original paper, researchers didn’t make all sorts of claims based on the sub-analysis of jogging dose.) To publish the same data a second time, this time making stronger claims about a “U-shaped” curve based on two (TWO!!!) deaths, is… well, you can make up your own mind. The data is right there.
