Reflections on the Role of “Consensus” in Scientific Research

Alan Kafka
Weston Observatory
Department of Earth and Environmental Sciences
Boston College

In a review of a paper in which we are exploring the extent to which some (or most, or “all”) Xs are Y, a reviewer wrote:

“The consensus in the community [studying this topic] is that all [Xs] are [Y].”

It wasn’t a “nasty” comment, but it did seem to be saying that we already know the answer (with some implication of so what’s the point of our study), and I don’t think we do know the answer (which is why we did the research and submitted the paper…).

So, I’ve been thinking: What is the role of “consensus” in a scientific research paper? I came across this, which gives me a starting point:

“Many people think that a scientific consensus refers to a large group of scientists who all agree that something is true. In reality, a scientific consensus is a large body of scientific studies that all agree with and support each other. The agreement among the scientists themselves is simply a by-product of the consistent evidence.” (www.facebook.com/thelogicofscience)

Is that it? Does science really work like that? But, doesn’t that leave a lot of “wiggle room”? For example, previous to the plate tectonics revolution, the scientific consensus was that continents don’t drift.

I’m not sure the scientific method is capable of “proving” that “all” of anything is ________. So it seems to me that the most we could accept would be that the consensus in the community [studying this topic] is that most [Xs] are [Y]. And, even if that is the case, then wouldn’t it be a very compelling question to explore what might be going on with the other Xs?

I think this overlaps with (but is not identical with) the question of how the public should relate to, and act upon, reports of scientific consensus in making public policy decisions. Scientific research can rarely (if ever?) “prove” something to be true. It can, however, in many cases provide a lot of guidance regarding how likely it is that something is true.

I think it should be left to each individual to decide the extent to which a reported scientific consensus should (or should not) be a compelling reason to believe it and/or act on it. But, making those kinds of informed decisions needs to be based on a firm understanding of how it all works within the world of scientific research.