Saturday, 15 June 2013

Why did you teach me to corrupt the scientific method? An open letter to my professors of 1993 and beyond.

Nijmegen,  15-06-2013

"There is no such thing as philosophy-free science; there is only science whose philosophical baggage is taken on board without examination."
- Daniell Dennett 

Esteemed prof. / dr. / PhD. / drs. / (emeritus),

almost exactly 20 years ago, in the summer of 1993, I was preparing for an exciting new chapter in my life as I was about to start my education as a student of Psychological Science at the same University at which I am currently employed as a docent and researcher. You probably do not remember me, I now know from personal experience what a lecture hall filled with 300+ students looks like from the professor's perspective. I followed your lectures, with great interest. 

During this first year you may have heard me ask some questions that were the topic of the books I was reading at the time: Gödel, Escher, Bach by Douglas Hofstadter and Consciousness Explained, by Daniel Dennett. Of course you were right to point out that, although interesting, such questions of philosophy were not the things empirical scientists like yourself cared much about. I know that now. Also, it was pointed out to me there was a philosophy course in the second year that pretty much covered all those things. 

(That course was indeed about philosophy, but not so much about the philosophy of psychological science. So I decided in my second year to write an essay about a story that appeared in a book edited by Hofstadter and Dennett. This you may remember, I won me the first prize in the 1995 university essay contest. Included was a short meet and greet with Daniel Dennett who was visiting in 1996 to promote his new book: Darwin's Dangerous Idea, from which I took the quote above this letter. By the way, do we still have an essay contest at our university?)

My question is about the following: 

In that same summer of 1993 an article appeared in the Journal of Experimental Education by Ronald Carver entitled: The Case Against Statistical Significance Testing, Revisited

Revisited, refers to the fact that Carver has to conclude that all the issues with statistical inference and the lack of replication of phenomena he identified in an article published in 1978 in the the Harvard Educational Review by the same title (save Revisited of course), were still around in 1993.

In fact, if you did not know the date of the publication, the excerpt from the article I copied below could very well have been published in the summer of 2013. The information in the abstract alone, contains all the answers one would need in order to change the corruption of the scientific method.

My question is: Why didn't you change? Your scientific work, the content of your lectures?

Why didn't we set up a study to quantify how bad this corruption really was? 
That would have been an excellent topic for a Master's thesis.

Why didn't we at least discuss these issues during my formative years as a scientist?


Your former student,

Fred Hasselman

ps. Carver is one of many scholars who have been warning us about these problems. If you missed his work, here's a selection of just a few of the publications that appeared just before and during the time I was a student that could, maybe should have caught your attention:

Cohen, J. (1990). Things I Have Learned (So Far). American Psychologist, 45(12), 1304–1312.
Cohen, J. (1994). The earth is round (p<. 05). American Psychologist, 49(12), 997–1003.
Freeman, W. J. (1997). Three centuries of category errors in studies of the neural basis of consciousness in intentionality. Neural Networks, 10(7), 1175–1183.
Kugler, P. N., Shaw, R. E., Vincente, K. J., & Kinsella-Shaw, J. (1990). Inquiry into intentional systems I: Issues in ecological physics. Psychological Research, 52(2), 98–121.
Meehl, P. E. (1990). Why Summaries of Research on Psychological Theories Are Often Uninterpretable. Psychological Reports, 66(1), 195.
Michell, J. (1997). Bertrand Russell’s 1897 Critique of the Traditional Theory of Measurement. Synthese, 110(2), 257–276.
Van Orden, G. C., & Paap, K. R. (1997). Functional neuroimages fail to discover pieces of mind in the parts of the brain. Philosophy of Science, 64(S1), 85–94.

pps. I have to exclude the advisors on my Master's Thesis who taught me the important lesson to publish data only when you are more than 100% sure they are reliable. Some of the pronunciation errors I had categorised could not be labelled unambiguously, or in other words, another rater would arrive at a different conclusion. So we did not publish.