Thursday, October 23, 2008

Scientific Misconduct (Part 1)

This is something I've previously published elsewhere. But I was thinking about it the other day, and I think it fits in with the overall content here. Oh, and last week didn't happen. This series will be continued, although I don't know when. :)

---

The use of the scientific method has, to put it succinctly, completely revolutionized the modern human experience. However, the underdiscussed backbone of this method is its dependence upon two critical components: A larger scientific community and a generally honest set of scientists. The larger community is necessary to provide critical review and to duplicate results. The honest individuals are necessary to prevent tremendous amounts of time and effort from being wasted running down dead ends.

These two critical components have, on occasion, broken down. The primary question is: What can we learn from these cases of scientific misconduct?

Scientific Misconduct by Individuals

In 2001, Jan Hendrik Schön (along with Hong Meng and Zhenan Bao) published a paper demonstrating that a thin layer of organic dye could be used as a transistor at the molecular scale. In this paper Schön claims “The fabrication of monolayer transistors and inverters might represent an important step towards molecular-scale electronics.” The paper is compelling, offering demonstrations of molecular structure and transistor characteristics of the materials he used.

Unfortunately, the results were a complete fabrication. In an editorial published along with the retraction of this paper, Nature questions what was going through Schön’s mind when he decided to fabricate his results.

“Was he trapped in a spiral of expectations that could only be fulfilled by deception?” (“Reflections on scientific fraud”, Nature, 2002, Vol. 319, p 417)

In the case of Schön’s work, the first hint that something was suspect came when someone noticed that the figures from several of his papers were identical. Following a lengthy investigation by Bell Labs (Schön’s employer), Schön’s was fired, his co-authors were exonerated, and journals were informed of the results. At the end of the 127 page report detailing these findings (which is no longer available through Bell Labs website), Schön responds to these findings:

“Although I have made mistakes, I never wanted to mislead anybody or misuse anybody’s trust. I realize that there is a lack of credibility in light of these mistakes, nevertheless, I truly believe that the reported scientific effects are real, exciting, and worth working for.”

The case of Jan Hendrik Schön is probably the most extreme form of scientific misconduct available to an individual, but it is certainly not the only form. Scientific misconduct generally falls into one of three larger categories: Fabrication, plagiarism, or ethical misconduct.

Fabrication such as that practiced by Schön has been rarely reported. Data on misconduct is scarce, but of 30 cases of scientific dishonesty reported by Denmark, just 2 were labeled ‘fabrication’ (Riis, 2001, J. of Clin. Pathology). A survey of just over 3,200 scientists being funded by the National Institutes of Health found that 0.3% had falsified data and 6.0% had failed to present data that contradicted one’s previous research. Even within the category of fabrication, often the kind of fabrication taking place is not as severe as that seen in the Schön case. Take, for example, the case of Dr. Robert Millikan.

Dr. Millikan was a physicist born in 1868 and prominent throughout the turn of the century. His greatest claim to fame is the oil drop experiment he did that established, once and for all, the charge on an electron. The experiment consists of balancing a small droplet of oil between a manually adjusted electrical field and the gravity of the earth. Since the gravitational field is known, and the electrical is known, the charge of the oil droplet can be calculated. What Millikan found was that the charge of the droplets was always a multiple of a single value: The charge of an electron.

When Dr. Millikan originally published his results, another scientist, Felix Ehrenhaft, published contradictory results suggesting a charge smaller than the electron charge existed. Millikan returned to the lab to obtain more precise results, and that is exactly what he got.

As it turns out, those precise results were partially the result of subtle data manipulation. Millikan measured close to 175 drops over the course of several months, but he reported the data of only 58 and claimed they were measured over 60 consecutive days. Some of the 175 could be legitimately removed due to obvious malfunctions in the machine used to conduct the experiments, others seemed to be removed solely to reduce the error in his final measurement. That these drops had gone unreported was not discovered until years after his death and upon closer examination of his lab notebooks. Millikan’s motivations were clear: He wanted to establish his reputation as a premier scientist. By selectively choosing the data he presented, he reduced his error by an order of magnitude and dismissed the objections of other scientists. In this case, his theories and results have proven accurate.

Another example of scientific misconduct demonstrates just how difficult it is to ascertain whether or not anything inappropriate has occurred. Gregor Mendel is commonly called the father of modern genetics. He lived in the latter half of the 19th century and raised pea pods in a garden in his monastery. Mendel’s primary contribution to science is the description of how traits are inherited. In 1865 Mendel published “Experiments on Plant Hybridization” in the Proceedings of the Natural History Society of Brunn, a minor journal in Moravia. For the next 35 years it was almost completely ignored, then in 1900 Carl Correns rediscovered the work and forced other scientists to acknowledge its significance. Gregor Mendel’s place in history was firmly establish over a decade after his death.

Almost immediately the scientific community noticed that the results Mendel got were too good to be true. W.R.F. Weldon, in a 1902 paper, noted using the newly developed χ2 test revealed Mendel’s results to be astonishingly close to the predicted values. In 1936 R.A. Fisher (in Annals of Science), showed rather convincingly that Mendel’s data must have been altered in some way. For the past 70+ years the debate has continued. Various explanations both plausible and apologetic have been put forth, from an eager-to-please assistant to selective acquisition of data to random chance. A review by Fairbanks and Rytting in the American Journal of Botany (2001) noted:

“There is substantial disagreement about his objectives, the accuracy of his presentation, the statistical validity of his data, and the relationship of his work to evolutionary theories of the day.” (p. 737)

Fairbanks and Rytting conclude that no real evidence that Mendel conducted misconduct exists, but this is far from a universally accepted interpretation. Other recent cases of fabrication can be similarly difficult to uncover. As authors Montgomerie and Birkhead lament in their 2005 article entitled “A Beginner’s Guide to Scientific Misconduct”:

“Indeed it should be possible for a clever scientist in our field [behavioral ecology] to fabricate all of their data without raising even the slightest suspicion...” (p. 21, Bulletin of the International Society for Behavioral Ecology)

Clearly, data fabrication is a problem of unknown extent upon which the scientific community can find little agreement. However, fabrication is hardly the only form of scientific misconduct that has been uncovered.

(Stay tuned for part II “Plagiarism” in the near future)

No comments: