Editor's note: Part 3 of a six-part USA TODAY series examining America's obesity epidemic.
Campaigns to cut fat and then sugar from America's diet didn't make a dent in the obesity epidemic. Nor did keto or paleo, Atkins, exercise plans or a TV reality show.
Kevin Hall knows, because he has studied all of them at the National Institutes of Health, where he has worked for 19 years. Hall considers it his mission to help people live healthier lives through food.
He has seen study participants lose weight on virtually every type of diet, regardless of the mix of fats, carbs, sugars or protein.
So a few years ago, when he decided to test the latest Diet Enemy No. 1, ultra-processed food, he expected to see much of the same.
In his study, two groups of 10 people each were fed either mainly chef-prepared whole foods for two weeks or mainly ultra-processed foods - the snack items, cereals and pre-made meals that make up about half of the typical American diet. Then they switched.
Both were offered the same number of calories and balance of fats, sugars, carbs, protein and salt. Both reported feeling equally full and equally satisfied with their food options.
Despite more than four decades fighting America's obesity epidemic, it's only gotten worse. To try to understand why, USA TODAY spoke with more than 50 experts for this six-part series, which explores emerging science and evolving attitudes toward excess weight.
But when they ate the mostly whole-food diet for two weeks, the volunteers lost about 2 pounds each. On the highly processed meal plan, the same people unintentionally consumed about 500 calories more a day on average, gaining both fat and 2 pounds each in just two weeks.
It was the first time he'd seen weight gain in one of his studies and Hall, whose research takes place at the National Institutes of Health Clinical Center in Bethesda, Maryland, couldn't help but conclude that ultra-processed food plays a significant role in the obesity epidemic.
"By changing people's food environment, you can change how much they're eating by a huge amount without them choosing," Hall said.
Highly processed foods are fast and filling. They are designed to taste good and be hard to stop eating. They can sit for months on store shelves. And they are cheap.
"At the end of a long day, with a couple of kids at home and working parents," Hall said, "frozen pizza looks pretty good." Ultra-processed foods "have a lot of positive attributes that are underappreciated."
"What's not to like?" he quipped.
It's not the tastiness or lack of fiber that makes people eat more ultra-processed food than they need, according to Dr. David Ludwig, an endocrinologist and researcher at Boston Children's Hospital.
Instead, he thinks the body responds to highly-processed carbohydrates by depositing excessive calories into fat tissue - leaving fewer calories available to meet the body's energy needs.
People overeat, according to this theory, not because they lack self-control, but to compensate for the calories being siphoned off into fat tissue.
From 1999 through 2018 the rate of Americans considered to have obesity jumped from just over 30% to more than 42%, according to the Centers for Disease Control and Prevention.
Over that time, ultra-processed food came to dominate the American pantry.
The distinguishing factor of a lot of ultra-processed food is that its water and fiber have been removed. Fiber and water lead to fullness.
Without the fiber, food gets digested faster and higher up in the digestive system, depriving the body of feeling full, Ludwig said.
He'd like to do a definitive study to determine whether a low-carb, high-healthy fats ketogenic diet could break the cycle of weight gain and metabolic problems and allow people to lose and keep off weight. But a long-term study like that could cost up to $100 million - much more than the typical government grant.
"To me, it's a public health failure that we've invested so much money into drugs and surgery," Ludwig said, while "government has never invested into a large-scale, multi-center low-carbohydrate-diet study."
Healthy weight tips
• Know that it's not your fault if you're carrying extra weight. The way the human body evolved, combined with easily available and low-cost processed food make weight gain likely and weight loss challenging.
• Fitness matters more for health than a number on a scale. Find physical activities you enjoy and commit to move a total of least 30 minutes a day, five days a week.
• Make sure to average at least 7 hours of sleep a night. Disconnecting from electronics for 90 minutes before bedtime can help.
• If you're concerned about weight, prioritize not gaining more and the quality of foods you eat.
• Chose whole foods, like fruits, vegetables, nuts, healthy oils, fermented dairy products and fatty fish, which have a healthier mix of nutrients and make you feel fuller.
At NIH, Hall is designing a short-term study with another set of volunteers to test whether eating ultra-processed foods made with extra fiber and water - and therefore fewer calories per bite - will lead to less overeating than off-the-shelf products. If so, he expects it will pressure the food industry to find a way to process food without taking out so much water and fiber.
Often portrayed as the "bad guy," the food industry has generally been responsive to nutrition research, Hall said.
When studies suggested eating a lot of fat led to obesity, the industry rapidly produced low-fat offerings. It substituted sugar and salt. When those were blamed for America's ever-widening waistline, shelves were suddenly full of low-salt, low-carb and sugar-free options.
Mike Gibney, a professor emeritus of food and health at University College Dublin in Ireland, said the problem is that people overeat energy-dense foods - ultra-processed or not.
"There are a lot of routes to getting fat," said Gibney, who consults for the food industry. People who eat in restaurants all the time gain weight even though they aren't eating much processed food, he said.
Counseling, lifestyle change, medications and gastric surgery are all needed to curb obesity. Solving the problem will require concerted effort and massive government spending, he said, not tinkering with the processing of food.
Feeding 9 billion people on the planet will require optimal food processing, said Dr. Dariush Mozaffarian, dean for policy at the Tufts Friedman School of Nutrition Science and Policy in Boston.
But changes to the food supply and food processing over the past half century must be contributing to the global epidemic of weight-related health problems, Mozaffarian said. Maybe plant and animal breeding has boosted production but left fewer nutrients on our plates, so we eat more to get the same nutritional value, he said.
Or perhaps, he said, we're altering the pace, location and effects of our digestion by removing the natural food structure and recombining highly processed components of food. "If we digest food without any intact natural structure, it's digested very, very quickly," he said.
By processing food differently, we're now consuming more refined sugar and starch, he said, which clearly affect metabolism, including the calories our bodies burn.
Additives in ultra-processed food, such as emulsifiers, stabilizers, guar gums, lecithin and artificial sweeteners and colors might also play a role in obesity, Mozaffarian said, though the evidence is still early and uncertain. Pesticides and chemicals such as endocrine disrupters, found in everything from plastic to lipstick, also could influence human health even at low levels, he said, but more research is needed.
"While excess starch, sugar and salt are clearly harmful, I used to discount and pooh-pooh the potential health harms of other, more trace additives," Mozaffarian said. He doesn't anymore. "We may have added things that we think are natural and fine, but they really may not be."
Even some newborns now are born with extra fat caused by these "obesogens," said Dr. Robert Lustig, a pediatric endocrinologist and emeritus professor at the University of California, San Francisco. Birth weight has risen by about a half pound in the past 25 years, Lustig said. "How could that be calories?"
Lab rats are fatter than they used to be, too, he said, suggesting there's something about our shared environment that's adding weight. He estimates about 20% of obesity is a result of these environmental factors.
"This is biochemical," he said. "This is not behavior."
What and how to eat instead?
At Tufts University Medical School, nutritionist Jose Ordovas has spent years examining the genetics of obese and thin people looking for ways to guide personalized eating plans. He's now convinced that genetics "represents only part of the equation or part of the puzzle. We have so much else going on."
Focused on health rather than just weight, Ordovas is now involved in a $170 million "precision nutrition" program funded by the NIH and designed to predict individual responses to food and dietary routines.
In one study, Ordovas and his team looked at older people at high risk for cardiovascular disease, following either a very low-fat diet, which is supposed to have heart benefits, or a Mediterranean diet with lots of healthy fats.
"For some people, depending on their genetic makeup, low-fat didn't do any good," he said. Instead, consuming more olive oil and nuts helped them avoid strokes. "The right diet for the right team was able to have a tangible effect."
Most nutrition researchers agree the best foods are digested slowly and add to feelings of fullness, with their natural high-fiber content intact. Those foods include low-starch vegetables, fruits, nuts, fish and healthy fats like olive oil.
There's some disagreement about how much grain should be part of a healthy diet. Some researchers support avoiding most high-carbohydrate foods, which include practically everything that has added sugar, as well as bread, rice and beans.
Several experts in metabolism support the idea of consuming very few carbohydrates.
To lose weight, people have to reduce their levels of insulin, the hormone that regulates sugar in blood, said Lustig, who wrote the 2021 book "Metabolical: The Lure and the Lies of Processed Food, Nutrition, and Modern Medicine."
The best approach is to eat foods that digest slowly, rather than spiking insulin levels with low fiber, sugar and simple carbohydrates, he said.
But others say eating beans and whole grains like brown rice and minimally processed wheat bread poses no problem.
A recent study from Stanford University found both an ultra-low carb ketogenic diet and a somewhat higher-carb Mediterranean diet boosted health for people with diabetes or pre-diabetes. The Mediterranean diet, which emphasizes vegetables, legumes, fruits, whole grains, olive oil and fish, was easier for people to stick to. The extra effort of the keto diet didn't seem worthwhile, they concluded.
Ludwig at Boston Children's said the added effort isn't worth it for people without serious metabolic or weight issues. They can get healthier just by cutting back on ultra-processed carbs, he said.
But he thinks those with a lot of excess weight or who are metabolically unhealthy - in other words, most Americans - would get additional benefit by going on a ketogenic diet with lots of healthy fats and few carbs.
Although the research remains thin, some people also swear by the idea of "mindful eating" - that the thoughtless consumption of food out of boredom or stress contributes to obesity.
Dr. Judson Brewer, a psychiatrist and neuroscientist at Brown University, is convinced that "people can make huge strides with mindful-eating approaches."
Savoring food and paying close attention while eating, rather than watching TV or reading, can add to the enjoyment of a meal and help people avoid overeating.
Mindful eating can encourage people to favor healthier foods, Brewer said. He used to consume a whole bag of gummy worms in one sitting, but now, "I've lost my taste for it," he said. "You could bury me in gummy worms and I just won't eat them."
SUBSCRIBE: Help support quality journalism like this.
There's also an aspect of self-acceptance to mindful eating, "stepping out of self-judgmental loops," he said. "I'd love to see people being kind to themselves."
Rob Knight, a professor at the University of California, San Diego, hopes to play a role in reversing obesity by learning how to manipulate the microbes that live in people's guts.
Gut bugs, he said, are intimately involved in weight gain, weight loss and the pounds that often creep back.
Studies have shown microbes inhabiting the intestines of mice that have lost weight are different from those in mice that have always been slim. Transplanting these bugs from the formerly obese mice into normal-weight ones led them to gain weight.
Learning to manipulate these bugs might help people sustain a lower weight once they've lost excess pounds, Knight said.
Even the timing of eating might make a difference in health, Ordovas said. People who generally eat earlier in the day tend to be healthier than those who eat later. "Being one 'chronotype' or another has even more effect than consuming one diet or another," he said.
"In terms of obesity, it's not only what we eat, it's not only how much we eat," Ordovas said. It may be "when you eat" that matters.
Finding a way
Scheduling his eating made a big difference for Lou Kim.
In adulthood, Kim, 43, of Arlington, Virginia, has weighed from 175 to close to 450 pounds.
Like many people who lose weight and keep it off, Kim, who works in private equity, set up a system of "guardrails" for himself to take away some of the hard work of decision-making.
Relying on an approach called intermittent fasting, Kim eats only during eight hours of the day, sometimes less. He doesn't have to track calories, just stop eating entirely for the rest of the day.
Valter Longo, a biologist at the Leonard Davis School of Gerontology at the University of Southern California, has spent his career studying intermittent fasting.
He's convinced that because the human body evolved to survive without food for extended periods, everyone would be better off not eating for 12 or more hours each day or fasting entirely on occasional days.
But like others in the obesity field, Longo can't afford the type of definitive research that might prove his theory. While drug companies can earn hundreds of millions if not billions of dollars from their products, there's not much to be made by encouraging people not to eat.
Kim said giving up sugar for a while also helped break the cravings. He has gone as long as a year without eating even fruit, but he loves citrus too much to stick with that regimen forever, he said with a chuckle.
Kim began his weight loss process by working on his personal and spiritual side, shifting his mental attitude, prioritizing self-care and taking ownership over his health. "When you start dealing with that stuff, everything else becomes easier," he said.
Like many people, Kim used food to dull the emotional pain of life - in his case, including the challenge of being the child of Korean immigrants. He was never Korean enough for his parents and never American enough for his peers.
Part of his motivation to lose weight was getting told he'd have to slim down before qualifying for weight loss surgery. If he was going to have to change his life anyway, he thought, why not just lose all the weight by himself? So, he found a way to do it.
"There's a path to healing ourselves," he said.
Contact Karen Weintraub at firstname.lastname@example.org.
Health and patient safety coverage at USA TODAY is made possible in part by a grant from the Masimo Foundation for Ethics, Innovation and Competition in Healthcare. The Masimo Foundation does not provide editorial input.
This article originally appeared on USA TODAY: Best diet for weight loss? Why health researchers don't have an answer