Do French Fries Kill You? A Lesson in Correlation and Causation
A study published last week in the <em>American Journal of Clinical Nutrition</em> suggested that those who eat fried potatoes two to three times a week or more are doubling their mortality risk.
It seems that every other week another food is out to get you. Everything from bacon to burnt toast has been found to contain carcinogens, substances that are capable of causing cancer. The French fry, tragically, is no exception.
In a study published last week in the American Journal of Clinical Nutrition, researchers tracked a group of 4,440 adults ranging in ages from 45 to 79 over the course of eight years, during which 236 of the subjects died. The study analyzed potato consumption among the participants and found that those who ate fried potatoes — e.g. French fries, hash browns, potato chips — two to three times a week or more were twice as likely to die earlier than those who did not. It concluded that the eating of fried potatoes is associated with an increase in mortality risk.
Key word being: associated.
If you’ve ever taken an introductory statistics course, there is one essential principle that the professors are sure to drill into your head: correlation (also known as association) does not equal causation.
Causation is when one variable directly contributes to another. For instance, it has been shown that smoking tobacco directly contributes to lung cancer.
Correlation, on the other hand, only means that there is some sort of relationship between two variables. In the case of potatoes, there is an association because the study authors found that a slightly higher number of people that consumed fried potatoes were more likely to die than their non-fried-potato eating counterparts. But that doesn’t necessarily mean that fried potatoes caused the deaths — or even contributed to them.
Now, no one will argue that French fries are good for you. But there are a number of problems with this recent study. First and foremost is the fact that the authors were actually conducting research on a form of arthritis until they decided to take a look at potatoes instead.
This fact stuck out to Philip Stark, associate dean of the division of mathematical and physical sciences and a professor of statistics at the University of California, Berkeley.
“Once you start switching the clinical endpoints of your study,” Stark told Seeker, “you can always pretty much find something that looks interesting.”
Secondly, the study was an observational study, meaning that the participants were observed in their daily life with no influence from the researchers. They were also in charge of self-reporting their potato consumption on questionnaires. It almost impossible to pinpoint causation in observational trials. Unless the researchers separated their participants into groups and force-fed half of them fries, there is no way to say definitively that potatoes were actually the reason people died.
This is because of the existence of confounders, hidden third variables that might be contributing to outcomes behind the scenes. In the case of potatoes, the possible confounders are almost endless.
“I can imagine a zillion different things that cause people who eat French fries to have worse health outcomes,” said Stark.
One obvious one is overall health. It makes sense that the people eating large amounts of French fries are less healthy overall than the people who were skipping the spuds. Obesity is another likely confounder.
The research exhibits the same pitfalls of so many other scientific studies, often revolving around the dangers of certain foods. Stark said that there has been a growing number of these pseudoscientific studies, likely due to the fact that it is now easier than ever to aggregate and comb through large sets of data.
This phenomenon has several names, including the term “p-hacking” named for the variable p that denotes whether a number is statistically significant. Basically, if you look at enough variables enough different ways, said Stark, you can find something significant.
It’s also known by another term: Cargo Cult Science. The term was coined by Richard Feynman, the renowned theoretical physicist, in a speech that he delivered at a 1974 Caltech commencement. He named it after a group of South Sea Islanders who built runways for planes to land in the hope of attracting cargo, not understanding that the simple construction of a runway wouldn’t guarantee the arrival of a plane.
“I call these things Cargo Cult Science,” Feynman said, “because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential.”
That essential missing ingredient is integrity, said Feynman, and a willingness to be your own harshest critic.
“If you’re doing an experiment,” he declared, “you should report everything that you think might make it invalid — not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked — to make sure the other fellow can tell they have been eliminated.”
The scientific method, as Stark put it, “is not looking for association, it’s trying to rule out explanations.”
WATCH: The Healthiest Diet in the World Isn't Just About Food