How to determine who is a legitimate science-based health expert and who is not. A simple method, a single criterion.

Reading Time: 5 minutes

Most popular health experts are unscientific

Many popular health experts claim to be science-based. They will often provide hundreds of references to support their claims. But citing many references does not mean one is scientific. It only means that one has collected many references.

Most self-proclaimed health experts are in fact very unscientific. They over-interpret the scientific literature. They create controversies where there are none. They form strong conclusions from weak data. They cite papers that support their views and ignore ones that don’t. They dedicate their entire body of popular work to buttressing an ideology or set of ideologies.

Their goal is not science per se but rather the promotion of a certain point of view. And, when credible people or organizations express disagreement with their conclusions, like clockwork they deploy conspiracy theories to dismiss that person or organization. They behave like cult leaders more than they behave like scientists.

What is sensational and sciency succeeds in the current media environment, not what is scientific

We want answers regarding how to be healthy. So when someone who is passionate provides us with an extensive, plausible-sounding series of arguments, we want to follow. How else, after all, are we going to become healthy, than by trying SOMETHING? And that something, provided by the extensive scientific-sounding claims from popular health experts, is likely to be better than nothing. So we do it.

But why THIS something? Why this advice?

Because it is sensational. Whatever is sensational generates greater buzz in the popular media. Thus, whatever we are exposed to is likely the result of a selection process, where the sensational survived and the drab perished. Sensational plus sciency equals pop health. (Sciency: appearing to be scientific but not actually being so.)

Whoever can be sensational and sciency at the same time, wins. The work of Gary Taubes and his countless low-carb followers is a perfect example of this. It is very sciency and very sensational. The work of Michael Greger and his followers (as well as his predecessor John McDougall) functions similarly. Sciency and sensational. Overturning the received wisdom. With “science.”

But what we really want are claims that are true. The current media environment that has gotten the claims to our doorstep is not giving us what we really want. We want the best way of achieving health according to a reasonable, scientific understanding of everything published to date. And the media is not giving us this because it rewards sciency and sensational, but it has no mechanism for ensuring that what is sciency is actually scientific.

Popular experts must have a scientific mentality if we want for their advice to be at the cutting edge of science

The true scientific mentality consists of a constant effort to understand the world and then to test–and if necessary–disprove that understanding. This is because the person with the scientific mentality really wants to know what is actually true. The person with the scientific mentality is happy if they discover that their current beliefs are not true, because that brings them one step closer to beliefs that actually ARE true.

For this person, false beliefs are a barrier to truth and must be rooted out. This is what people mean when they talk about falsification. Falsification is central to science because it is central to progress toward a final theory of reality, which is the scientist’s ultimate goal. This sounds very lofty, but it also has important practical applications.

A final theory of reality is extremely serviceable, because if we understand exactly how the world works, we can more readily create useful technologies. Thus, in the field of health sciences, science and benefit to patients go hand in hand.

A scientific mindset is the best way to serve patients: false and harmful/unhelpful theories are discarded while true and beneficial theories remain. It follows that an expert that is truly scientific should be trying to constantly overturn their own beliefs. And that promoting just one set of beliefs without criticizing them is a failsafe sign that one has identified someone who is not scientific.

We therefore reach our simple, singular criterion for identifying which popular health experts are science-based and which are not:

A science-based health expert will consistently and competently criticize the design, methods, and findings of studies that agree with the positions that they have taken previously.

They must demonstrate consistent evidence of a scientific mentality. Such an expert will consistently seek out the limitations to their positions. This is actually easy. Because almost all health science has limitations or flaws, we can constantly be gaining a deeper and deeper understanding of how the new and old science is or might be flawed.

Practical recommendations for nutrition and fitness are almost never 100% certain

Health scientists do not come to conclusions or recommendations based on certainty. In almost no area of health science does certainty exist. This is especially true for nutrition and fitness science. Rather, health scientists come to conclusions and make recommendations DESPITE limitations and flaws. Health scientists must weigh the different bits of evidence, with all of the flaws and limitations that are a part of each, against each other. The scientist will always look at evidence in health science and frown.

The recommendations that the health scientist will make are those that make him frown the least. Recommendations in health science are almost always a matter of nodding solemnly, wishing we had better evidence, and then making the recommendation. It is literally impossible to do that in an unbiased and informed way if one does not systematically understand the flaws and limitations of the different positions. Furthermore, since practical recommendations are always hedges, they should change over time as new evidence comes to light which then tips the scale in one direction or the other.

Therefore, a person who is not actively engaged in trying to understand the problems with the science underlying their own recommendations cannot possibly be coming to conclusions based on a scientific understanding of the evidence, and they cannot be updating their point of view in light of new evidence.

A person not constantly questioning their own beliefs in light of the evidence and growing cannot possibly be at the cutting edge of the science.

What most popular health experts do, versus what they should do

Therefore, ask yourself:

Does your preferred health expert seem to be concerned with findings the flaws and limitations of the evidence supporting his or her recommendations?
Do they go deep in looking at the scientific papers and evaluating the design, methods, or findings?

Because if not, then ask yourself:

What is that popular expert actually doing?

We take it for granted that a popular health expert does not behave in the above way, i.e. in a way that is scientific. We take it for granted that popular health experts constantly hype their beliefs, that they should represent a consistent certain point of view, year after year. And certainly this is a more successful strategy for getting a message out.

But shouldn’t we demand that this be balanced by at least some indication that the health expert is concerned with the objectivity of their position? And shouldn’t we be concerned when we see no indication that they are?

Our current system of health science communication has failed, but the way to change is clear

How has our current system of popular health experts helped us? How has it generated anything but confusion? I suggest that we hold popular health experts up to higher standards, to the standard of actual scientists:

What demarcates whether a person is science-based or not is whether they frequently criticize the evidence supporting their own positions. By this criterion, almost no current public health experts are science-based. And I think almost none in fact actually are, so I think this criterion gets things right. The problem is that we have as consumers incentivized hype.

If we want good health information, we need to start incentivizing earnestness. I believe that evaluating health experts on the basis of the above criterion will help to accomplish that.

In the meantime, we also need to explain to children starting at a young age just what science is. It is NOT the next, best cool thing. It is a laborious and constantly unfolding process, especially in the area of health. And the defining feature of this process is thoughtfulness and self-doubt, and the people who balance thoughtfulness and self-doubt with popular communication are the people we should trust. Not hypesters.

I wrote almost a year ago that I wanted to release a quack list. Well, a quack list could easily be generated based on the above criterion. It would include almost all popular health experts. Almost all claim to be science-based, but almost none actually are science-based.

And I really do think that this criterion is a fair one. It’s just radical. But maybe what we need in this age of insane health claims and gurus is exactly that: a radical point of view.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Producing high-quality scientific work like this takes many hours of work. This is made possible through the support of readers and viewers like you. Please consider donating below.

In just a few clicks, sign up to donate $5 monthly here:
Or $20 monthly here:

For one-time donations, or to become a patron on Patreon, click here.

Advertisements

Posted In:

3 Comments

  1. Nice post Kevin, I think that over-hyping is now common across many areas of science, but there is obviously a lot of public attention paid to the claims of popular health experts so their hype may be quite likely to be widely adopted.

    But I have heard the research hyping phenomena also referred as a form of the red-queen effect – if the majority of experts hype their results then a discerning reader will know to discount the value of a study by the average hype they expect to be added to it. This makes it very hard to present honest claims without hype as a reader will still the discount how much they expect experts to hype their results from your study. So then all experts have to hype their studies by at least he average in order to have them taken at their fair value (after discounting) which creates a nash-equilibria that maintains hyped research…

    It does seem very reasonable to expect public health experts to act scientifically, but the radical part would be finding a way for them to get attention and credibility on-par with others who have hyped their work.

  2. This is one of the best thing I’ve read lately.

    I wouldn’t have gone 5 years ago if I read this at the times! Luckily I stepped out of this nonsense early enough and embraced the plants; long live Kim Williams Jr, a true scientific spirit like you describe!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.