Ed Yong has a piece in Nature on the problems of confirmation bias and replication in psychology. Yong notes that “It has become common practice, for example, to tweak experimental designs in ways that practically guarantee positive results.” The way this has been explained to me is that you perform an experiment, get a p-value of > 0.05 (significance). You know that your hunch is warranted, so just modulate the experiment, and hope that the p-value comes in at < 0.05, and you have publishable results! Obviously this is not just a problem in psychology; John Ioannidis has famously focused on medicine. But here’s a chart which shows that positive results are particular prevalent in psychology:
There are many angles to this story, but one which Ed did not touch upon is the political homogeneity of of psychology as a discipline. The vast majority of psychologists are political liberals. This issue of false positive results being ubiquitous is pretty well known within psychology, so I’m sure that that’s one reason Jonathan Haidt has emphasized the ideological blinkers of scholars so much. Let’s assume that ...
And in the end it is only what you feel that counts.
