Cautionary Tales from the Front Lines of Science

A review of On Fact and Fraud: Cautionary Tales from the Front Lines of Science by David Goodstein. This review appeared in the American Journal of Physics on July 14, 2010.

In 1609 Galileo turned toward the heavens a modified version of the telescope first invented by the Dutch spectaclemaker Hans Lippershey, and there he observed that satellites were orbiting Jupiter, that Venus had phases, and that there were mountains on the moon and spots on the sun. His claims challenged Aristotelian cosmology, which held that all objects in space must be perfectly round and perfectly smooth. After observing Saturn, however, Galileo wrote to Johannes Kepler, “Altissimum planetam tergeminum observavi” (“I have observed that the farthest planet is threefold”). He continued: “This is to say that to my very great amazement Saturn was seen to me to be not a single star, but three together, which almost touch each other.” Galileo saw Saturn not as a planet with rings as we see it today in even the tiniest of home telescopes, but as one large sphere surrounded by two smaller spheres.

Why did Galileo make this mistake? Two reasons: (1) Data—Saturn is twice as far away as Jupiter; thus what few photons of light there were streaming through the cloudy glass in his little tube made resolution of the rings problematic at best; and (2) theory—there was no theory of planetary rings. It is at this intersection of nonexistent theory and nebulous data that the power of belief is at its zenith and the mind fills in the blanks.

This problem of belief fulfillment is not fraud or deception. It is more a problem in self-deception, and there is no reason to believe that scientists are any less subject to the problem than anyone else. In fact, the scientific method is designed to deal with such belief expectations through colleague collaboration, peer review, experimental replication, and the like. If you don’t catch your errors and biases, someone else will, usually in a public forum, so there is a certain ethic among scientists so well described by the physicist Richard Feynman in a commencement address at Caltech: “It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty—a kind of leaning over backwards. For example, if you are doing an experiment, you should report everything that you think might make it invalid—not only what you think is right about it; other causes that could possibly explain your results; and things you’ve thought of that you’ve eliminated by some other experiment, and how they worked—to make sure the other fellow can tell they’ve been eliminated.”

The problem is that almost no one can live up to this ethic, and herein lies the foundation for scientific fraud. In his poignant and powerful new book, On Fact and Fraud, Feynman’s Caltech physics colleague David Goodstein has penned a primer on how science works (and doesn’t work) that should be read by anyone interested in the long term future of the most powerful knowledge generating machine in the history of humanity. “Fraud in science is, in essence, a violation of the scientific method,” Goodstein begins. “It is feared and denigrated by all scientists.” Examples: the Piltdown Man hoax was out and out fraud—a completely faked ancient hominid jaw and teeth. The British psychologist Cyril Burt’s fake twins and fake twin researchers is another clear-cut case of fraud, as was William Summerlin’s experiments on inducing healthy black skin grafts on white mice, which he enhanced with a black felt-tipped pen.

Goodstein begins by explaining how science works and does not work. For example, there is a certain type of what Goodstein calls “hypocrisy” in scientific papers, in which the narrative arc from theory to hypothesis to methods to results to further study sections is not at all what happens in a laboratory, where “every scientific experiment is chaotic—like war. You never know what’s going on; you cannot usually understand what the data mean. But in the end you figure out what it was all about, and then, with hindsight, you write it up as one clear and certain step after another.” Here is where potential trouble begins. “For example, it may be marginally acceptable, in writing up your experiment, to present your best data and casually refer to them as typical (because you mean typical of the phenomenon, not typical of your data), but is it not acceptable to move one data point just a little bit to make the data look better. All scientists would agree that to do so is fraud.”

The line of moral demarcation is fairly clear, so why do some (very few as it turns out) cross it while others do not? The devil is in the details, and Goodstein’s book primarily focuses on these cases of alleged fraud: That of Robert A. Millikan, whose historic measurement of the electron’s charge was maligned by accusations of fraud but whom Goodstein exonerates; Martin Fleischmann and Stanley Pons and their “discovery” of cold fusion, which was more likely a case of scientists who “convince themselves that they are in the possession of knowledge that does not in fact exist;” Victor Ninov and the supposed discovery of element 118 (fraud); Jan Hendrik Schön from Bell Laboratories and his work in semiconductors (fraud); and J. Georg Bednorz and Karl Müller’s discovery of high-temperature superconductivity, a seemingly impossible accomplishment that turned out to be real.

Creationists are fond of pointing out examples of scientific fraud, as if to say, “see, you can’t trust scientists so accept the text of Holy Scripture.” But as Goodstein notes, fraud in science is almost always exposed by scientists eventually. The problem with detecting intentional fraud is that “injecting falsehoods into the body of science is rarely, if ever, the purpose of those who perpetrate fraud. They almost always believe that they are injecting a truth into the scientific record.” Why would they do this? Here Goodstein’s book really shines as an insider’s perspective of how science works in the nitty-gritty, hardscrabble, competitive world of professional research. Goodstein, whose job it was as the vice provost at Caltech to investigate all allegations of scientific misconduct, outlines three motives, or what he calls “risk factors,” present in nearly all cases of scientific fraud, in which the perpetrators “(1) were under career pressure; (2) knew, or thought they knew, what the answer to the problem they were considering would turn out to be if they went to all the trouble of doing the work properly; and (3) were working in a field where individual experiments are not expected to be precisely reproducible.”

Of course, Goodstein adds, “it is by no means true that fraud always arises when these three factors are present;” however, “they do seem to be present whenever fraud occurs.” It’s an important distinction, and Goodstein ends this important book with Caltech’s policy statement on scientific misconduct that includes specific definitions of the different types of possible misconduct, proper steps in investigating potential misconduct to ensure a fair hearing for everyone involved, and what should be done with the findings.