Things are really grim for those of us who believe in science and who have spent a lifetime laboring in its vineyards. The Trumpian assault on science is simply beneath contempt. So great is the lure of financial and political power that some claim science is merely another form of domination. Ayurdhi Dhar (personal communication) recently noted that, “BMJ’s ex-editor wrote an opinion piece this month admitting that he thinks 20% of all published medical research is simply fraudulent—not a spin, not problematic, but downright fraudulent.” I have no doubt there is a good deal of truth in that shocking claim. But I do not address frauds and cheats (whether motivated by money, political power, unbridled narcissism, etc.) for they are beyond hope.
Today I address a vastly different (perhaps also 20% of us) audience—those of us who sought Truth via scientific investigations, played faithfully by the rules, and still published (what later turned out to be) false findings. I have published about 200 articles and chapters; it is quite possible that in roughly 40 of them, I did everything correctly and simply obtained the wrong answer. And because all answers look the same (and therefore all look true) initially, I hastened to publish them. How many readers can make a similar confession? (Is it true that the first step toward recovery involves admitting one is an addict?) Being an epistemic fallibilist implies that my scientific efforts are as likely as the next scientist’s efforts to fall victim to real problems that plague all of our efforts. But don’t take that admission too far. Are my beliefs more likely to be true than Donald Trump’s—I’ll place a lot of money on that wager. Anyone out there wants to take the Trump side of that bet? So, why do I prefer science to other sources of knowledge? Because science is an improvable body of practices and procedures. Let me start simply.
Donald T. Campbell, in an unpublished (I believe) paper entitled “Evolutionary epistemology and the fish scale model of omniscience,” spoke of an imaginary scientist who wished to study the lengths of fish in the ocean (the research question). Being committed empiricists, scientists immediately search for operational definitions whereby we might answer our research questions. One scientist dams a small inlet of the sea, waits for the water to evaporate, carefully catalogues the lengths of dead fish bodies, and concludes, “There are no fish in the ocean longer than three inches.” Another scientist drags the seas using a net of three-inch mesh. That scientist concludes, “There are no fish in the sea smaller than three inches in length.” Four years of graduate study immediately went up in smoke in my mind. Among other things, criterion validation was a fraud. Unreplicated, single studies were dangerous products in our efforts to construct valid research literatures (because they might represent our expected Type 1 errors which would lead us astray from our path toward Truth). We needed a science that properly understood, respected, and leveraged uncertainty in the creation of knowledge. Positivism taught us to make wild, untenable assumptions (e.g., Intelligence is whatever an intelligence test measures). So, if one created a new intelligence test, it was a good measure if it correlated close to +1.0 with the original intelligence test, and a bad measure if the criterion validation correlation was close to zero. No correlation coefficient was mathematically possible that would suggest that the second test was superior to the original test.
Have you noticed there is absolutely no room for tentativeness, uncertainty and the like for authoritarian belief systems (like Trumpism, Naziism, religious fanaticisms, etc.)? Concepts like uncertainty, method variance, failures to replicate, Type 1 errors, and the like demonstrated why it was problematic (in the extreme) to attempt to construct any web of belief—whether scientific, religious, political, or whatever. Fortunately, thinkers like Campbell, Fiske, Meehl, Kenny, Joreskog and Maxwell have enlightened our path toward construct (rather than criterion) validity. Similarly, our 70-year flirtation with Fisherian, inferential statistics (rather than different approaches such as Bayesian statistics) is largely responsible for our current replication crisis, our science of Type 1 errors, and the like. Over a 40-year span, my colleagues and I created a program of investigations entitled “Improving methodology via research on research methods.” If we believe that science is the best way to know Truth, why wouldn’t we use science to systematically study the components of our research practices? The yield from this program has been quite satisfying.
The path back to a better science of humans has been shown by construct validity efforts that utilize multi-trait / multimethod matrices analyzed via structural equation modeling. Similarly, our science of Type 1 errors can be repaired by new methodologies such as ORMA (copies available via email upon request). While our scientific web of belief is better than any other belief system (in my opinion), it is still fallible and imperfect. However, our scientific web of belief about humans is improving, and that has made all the difference for me. Returning to our quick discussion of criterion validation efforts of intelligence tests above, our more sophisticated efforts at construct validation demonstrated repeatedly that instances where new measures looked worse than the original tests were because the new tests were actually superior measures of intelligence. Our flawed, positivist assumption led us to draw the wrong conclusion time after time. Fortunately, we’ve now moved beyond that methodological pothole. We have analyzed uncertainty so thoroughly that we now have multiple methods for quantifying it. These concepts (e.g., error in measurement, correlated disturbances, shared method variance, etc.) better enable us to recognize Truth from Error, and to separate the wheat from the chaff. Once cleansed of the uncertainties that plague our efforts to tease out Truth from other misleading complexities of human behavior, our vision of the wellsprings of human action becomes more knowable.
However, I will end with an apology. If anyone has been offended by my thoughts on the distorting effects of politics, religion, and money on our beliefs, then I am truly sorry. While every retired academician would welcome a few extra shekels, I have never been tempted to fabricate scientific data for financial gain. That is part of why I prefer science (overall). Similarly, while I have some political beliefs, I’ve never been tempted to attack our Capitol to hang on to political power. Thus, it is not clear to me that our hot pursuit of filthy lucre, our extremist politics, and our religious fanaticisms are improving our beliefs in any way. But our science is definitely getting closer to the Truth. That’s why I’m still proud to call myself a scientist.
George spent fifty years in higher education, the last forty of which were at the University of Notre Dame. His last decade has involved founding, owning and serving on the Board of Directors of