Our Community Platform providing Essential Information to Know what is Real
The Credibility Issue in Nutrition Science Is a Sign for All of Higher Ed
Publisher's note: The James G. Martin Center for Academic Renewal is a nonprofit institute dedicated to improving higher education in North Carolina and the nation. Located in Raleigh, North Carolina, it has been an independent 501(c)(3) organization since 2003. It was known as the John W. Pope Center for Higher Education Policy until early January 2017.
The author of this post is Anthony Hennen.
In recent years, psychology has dealt with a legitimacy crisis. Many influential psychological studies could not be reproduced by other psychologists, discrediting some key insights and weakening academic faith in the entire field.
Nutrition science has a similar problem.
The loudest critics argue that the methodologies relied on by researchers give bad data that are meaningless at best. Others worry that funding gives undue influence to the federal government, big business, or influential nonprofit associations. And some critics think nutrition science focuses on the wrong questions entirely about nutrition.
Defenders of the field say that the research methods used are reliable and improving with trial and error. They also argue that funding sources don't have as much influence as critics fear.
Critics and defenders, however, both agree that nutrition science is an extremely challenging field for getting good data. It's more like a social science than a hard science in that respect. The challenge for its defenders is building credibility. Whether they can also has implications for higher education research. Billions of dollars in taxpayer money go to colleges for research purposes, but the information that's produced might not be socially useful.
To collect data, nutrition researchers often use observational studies. Those studies rely on participants self-reporting their eating habits, usually by completing a survey or keeping a food journal. The problem is that people can misremember what they ate. Even tracking what someone ate for a week is hard because people might forget a snack or change their eating habits because they're being monitored.
Thus, with inaccurate survey data, what researchers tell the public to eat or to avoid can go back and forth. Eggs, for instance, have been healthy and unhealthy and healthy again. Fat has gone through something similar.
"The reason we know so little about what to eat despite decades of research is that our tools are woefully inadequate. Lately, as scientists try, and fail, to reproduce results, all of science is taking a hard look at funding biases, statistical shenanigans and groupthink. All that criticism, and then some, applies to nutrition," Tamar Haspel, a food columnist for The Washington Post, wrote.
Other critics are more blunt.
"This data is meaningless," Edward Archer, the chief science officer at EvolvingFX and former NIH research fellow and formerly at the Nutrition and Obesity Research Center at the University of Alabama at Birmingham, said of observational studies. Archer doesn't think those "memory-based methods" of research are helpful and prefers a different scientific approach in nutrition. "Replication...doesn't help us," he said. Rather than asking whether scientists can do the same work and get the same results, Archer wants researchers to focus on falsification instead. A scientific approach based on falsification would require trying to prove previous results wrong.
Changing the scientific paradigm, though, is a tall order. The U.S. Department of Agriculture. Department of Health and Human Services, and the National Institutes of Health spend about $5 billion annually on nutrition science research, Archer noted. If a researcher would publicly criticize the status quo, they could lose funding.
What happens if survey-based data suddenly become unacceptable? A lot of nutrition researchers will quickly discover it's a lot harder to find funding, conduct studies, and publish the kinds of articles that provide tenure, job security, and prestige. And these endangered scholars are the peers who pass judgment on the articles that appear in peer-reviewed journals. They have a powerful incentive not to rock the boat.
But defenders of the status quo aren't so radical about the need for change.
"There have been success stories, that seem to sometimes be overlooked with the 'drama' in the nutrition field," Brenda Davy, a nutrition scientist at Virginia Tech, said in an email to the Martin Center. Memory can be fickle, but tracking behavior can be reliable, Davy wrote in a 2015 paper. The point isn't to over-rely on one research approach, but to improve them.
Howard Bauchner, the editor-in-chief of the Journal of the American Medical Association (JAMA), argued that nutrition science academics are aware of the field's problems and actively work to improve it. "Research methods in general have improved tremendously over the last 20 years," he said. The quality of the data researchers use has improved, along with their research methods, such as controlling for confounding variables and bias.
Davy argued that good nutrition science can be extremely challenging, but wrote: "Evidence-based nutrition is best served by considering the totality of evidence across multiple study types including nutritional epidemiological studies, randomized controlled trials of behavioral interventions, and controlled feeding studies."
High-quality research methods aren't the only areas of concern. The funding sources for research worries nutrition scientists and others. Marion Nestle, a nutrition scientist at New York University, is wary of any research funded by food companies, calling this research "marketing, not science" in an email to the Martin Center. Bauchner is less suspicious so long as academic independence is protected. "I genuinely believe there's a way in which industry can fund nutrition research and that one can believe the science. I do think that there needs to be checks and balances," he said.
Archer is more concerned about government funding, given the amount of money and influence the USDA and NIH have. "Being in that network means you get more money," he said. The "academic industry-public policy network" can silence researchers who want to prove previous scientific findings wrong.
Not every problem in nutrition science is unique to it. Research methods across the social sciences suffer from the same credibility problems. Publishing pressures affect almost all researchers. And peer review is a weak point as well.
"I'm not sure this is a problem in my field that is any greater than other science fields," Davy said. "Researchers are expected to publish in order to advance their careers (promotion, tenure), and to show that their research has impacts on the field. Publishing research, and obtaining grant funding, can be very competitive processes. It is also true that it can be more difficult to publish studies with negative findings, and that can lead to publication bias."
Specialization also makes peer review difficult. Davy mentioned how hard it can be to find a few professors to review an article; they may not have the time or expertise to provide an academic journal with the skeptic eye it needs.
"I think that academia in general is in trouble," Edward Archer said.
Paying academics for peer review could make them prioritize the work. Currently, peer review is done for free as part of a professor's "service" commitment. Raising the standards for published research could also cause a change. Patrick Clinton has written about making p-values, a shorthand for how likely it is the result is a chance occurrence, more rigorous. John Ioannidis, a nutrition scientist at Stanford University, has argued for stronger standards along similar lines. JAMA has been willing to publish some strong critiques of nutrition science as a field (though not to the extent that Archer would like). Building public confidence could require more influential journals to embrace the critics and calls for radical change.
Nutrition science highlights a larger problem in higher education. If billions of dollars in funding haven't promoted high-quality research, then the modern research university could have quality-control issues. Research methods can improve, but the pressure to publish and biases within academic publishing hurts all academics. Nutrition science might be a field with more difficulties than most, but university departments of all stripes could have a (taxpayer-funded) replication crisis on their hands.
Anthony Hennen is managing editor of the James G. Martin Center for Academic Renewal.
Go Back
The author of this post is Anthony Hennen.
In recent years, psychology has dealt with a legitimacy crisis. Many influential psychological studies could not be reproduced by other psychologists, discrediting some key insights and weakening academic faith in the entire field.
Nutrition science has a similar problem.
The loudest critics argue that the methodologies relied on by researchers give bad data that are meaningless at best. Others worry that funding gives undue influence to the federal government, big business, or influential nonprofit associations. And some critics think nutrition science focuses on the wrong questions entirely about nutrition.
Defenders of the field say that the research methods used are reliable and improving with trial and error. They also argue that funding sources don't have as much influence as critics fear.
Critics and defenders, however, both agree that nutrition science is an extremely challenging field for getting good data. It's more like a social science than a hard science in that respect. The challenge for its defenders is building credibility. Whether they can also has implications for higher education research. Billions of dollars in taxpayer money go to colleges for research purposes, but the information that's produced might not be socially useful.
To collect data, nutrition researchers often use observational studies. Those studies rely on participants self-reporting their eating habits, usually by completing a survey or keeping a food journal. The problem is that people can misremember what they ate. Even tracking what someone ate for a week is hard because people might forget a snack or change their eating habits because they're being monitored.
Thus, with inaccurate survey data, what researchers tell the public to eat or to avoid can go back and forth. Eggs, for instance, have been healthy and unhealthy and healthy again. Fat has gone through something similar.
"The reason we know so little about what to eat despite decades of research is that our tools are woefully inadequate. Lately, as scientists try, and fail, to reproduce results, all of science is taking a hard look at funding biases, statistical shenanigans and groupthink. All that criticism, and then some, applies to nutrition," Tamar Haspel, a food columnist for The Washington Post, wrote.
Other critics are more blunt.
"This data is meaningless," Edward Archer, the chief science officer at EvolvingFX and former NIH research fellow and formerly at the Nutrition and Obesity Research Center at the University of Alabama at Birmingham, said of observational studies. Archer doesn't think those "memory-based methods" of research are helpful and prefers a different scientific approach in nutrition. "Replication...doesn't help us," he said. Rather than asking whether scientists can do the same work and get the same results, Archer wants researchers to focus on falsification instead. A scientific approach based on falsification would require trying to prove previous results wrong.
Changing the scientific paradigm, though, is a tall order. The U.S. Department of Agriculture. Department of Health and Human Services, and the National Institutes of Health spend about $5 billion annually on nutrition science research, Archer noted. If a researcher would publicly criticize the status quo, they could lose funding.
What happens if survey-based data suddenly become unacceptable? A lot of nutrition researchers will quickly discover it's a lot harder to find funding, conduct studies, and publish the kinds of articles that provide tenure, job security, and prestige. And these endangered scholars are the peers who pass judgment on the articles that appear in peer-reviewed journals. They have a powerful incentive not to rock the boat.
But defenders of the status quo aren't so radical about the need for change.
"There have been success stories, that seem to sometimes be overlooked with the 'drama' in the nutrition field," Brenda Davy, a nutrition scientist at Virginia Tech, said in an email to the Martin Center. Memory can be fickle, but tracking behavior can be reliable, Davy wrote in a 2015 paper. The point isn't to over-rely on one research approach, but to improve them.
Howard Bauchner, the editor-in-chief of the Journal of the American Medical Association (JAMA), argued that nutrition science academics are aware of the field's problems and actively work to improve it. "Research methods in general have improved tremendously over the last 20 years," he said. The quality of the data researchers use has improved, along with their research methods, such as controlling for confounding variables and bias.
Davy argued that good nutrition science can be extremely challenging, but wrote: "Evidence-based nutrition is best served by considering the totality of evidence across multiple study types including nutritional epidemiological studies, randomized controlled trials of behavioral interventions, and controlled feeding studies."
High-quality research methods aren't the only areas of concern. The funding sources for research worries nutrition scientists and others. Marion Nestle, a nutrition scientist at New York University, is wary of any research funded by food companies, calling this research "marketing, not science" in an email to the Martin Center. Bauchner is less suspicious so long as academic independence is protected. "I genuinely believe there's a way in which industry can fund nutrition research and that one can believe the science. I do think that there needs to be checks and balances," he said.
Archer is more concerned about government funding, given the amount of money and influence the USDA and NIH have. "Being in that network means you get more money," he said. The "academic industry-public policy network" can silence researchers who want to prove previous scientific findings wrong.
Not every problem in nutrition science is unique to it. Research methods across the social sciences suffer from the same credibility problems. Publishing pressures affect almost all researchers. And peer review is a weak point as well.
"I'm not sure this is a problem in my field that is any greater than other science fields," Davy said. "Researchers are expected to publish in order to advance their careers (promotion, tenure), and to show that their research has impacts on the field. Publishing research, and obtaining grant funding, can be very competitive processes. It is also true that it can be more difficult to publish studies with negative findings, and that can lead to publication bias."
Specialization also makes peer review difficult. Davy mentioned how hard it can be to find a few professors to review an article; they may not have the time or expertise to provide an academic journal with the skeptic eye it needs.
"I think that academia in general is in trouble," Edward Archer said.
Paying academics for peer review could make them prioritize the work. Currently, peer review is done for free as part of a professor's "service" commitment. Raising the standards for published research could also cause a change. Patrick Clinton has written about making p-values, a shorthand for how likely it is the result is a chance occurrence, more rigorous. John Ioannidis, a nutrition scientist at Stanford University, has argued for stronger standards along similar lines. JAMA has been willing to publish some strong critiques of nutrition science as a field (though not to the extent that Archer would like). Building public confidence could require more influential journals to embrace the critics and calls for radical change.
Nutrition science highlights a larger problem in higher education. If billions of dollars in funding haven't promoted high-quality research, then the modern research university could have quality-control issues. Research methods can improve, but the pressure to publish and biases within academic publishing hurts all academics. Nutrition science might be a field with more difficulties than most, but university departments of all stripes could have a (taxpayer-funded) replication crisis on their hands.
Anthony Hennen is managing editor of the James G. Martin Center for Academic Renewal.
Home
Contact
About
Service Rates
Privacy Policy
Follow on Facebook
Subscribe to Newsletters
Rss Feed
Keywords
Eastern NC Now & ENC NOW © Copyright 1998-2024 All Rights Reserved.
