Why did you teach me to corrupt the scientific method? An open letter to my professors of 1993 and beyond.

Nijmegen,  15-06-2013

"There is no such thing as philosophy-free science; there is only science whose philosophical baggage is taken on board without examination."

- Daniell Dennett 

Esteemed prof. / dr. / PhD. / drs. / (emeritus),

almost exactly 20 years ago, in the summer of 1993, I was preparing for an exciting new chapter in my life as I was about to start my education as a student of Psychological Science at the same University at which I am currently employed as a docent and researcher. You probably do not remember me, I now know from personal experience what a lecture hall filled with 300+ students looks like from the professor's perspective. I followed your lectures, with great interest. 

During this first year you may have heard me ask some questions that were the topic of the books I was reading at the time: Gödel, Escher, Bach by Douglas Hofstadter and Consciousness Explained, by Daniel Dennett. Of course you were right to point out that, although interesting, such questions of philosophy were not the things empirical scientists like yourself cared much about. I know that now. Also, it was pointed out to me there was a philosophy course in the second year that pretty much covered all those things. 

(That course was indeed about philosophy, but not so much about the philosophy of psychological science. So I decided in my second year to write an essay about a story that appeared in a book edited by Hofstadter and Dennett. This you may remember, I won me the first prize in the 1995 university essay contest. Included was a short meet and greet with Daniel Dennett who was visiting in 1996 to promote his new book: Darwin's Dangerous Idea, from which I took the quote above this letter. By the way, do we still have an essay contest at our university?)

My question is about the following: 

In that same summer of 1993 an article appeared in the Journal of Experimental Education by Ronald Carver entitled: The Case Against Statistical Significance Testing, Revisited

Revisited, refers to the fact that Carver has to conclude that all the issues with statistical inference and the lack of replication of phenomena he identified in an article published in 1978 in the the Harvard Educational Review by the same title (save Revisited of course), were still around in 1993.

In fact, if you did not know the date of the publication, the excerpt from the article I copied below could very well have been published in the summer of 2013. The information in the abstract alone, contains all the answers one would need in order to change the corruption of the scientific method.

My question is: Why didn't you change? Your scientific work, the content of your lectures?

Why didn't we set up a study to quantify how bad this corruption really was? 

That would have been an excellent topic for a Master's thesis.

Why didn't we at least discuss these issues during my formative years as a scientist?

Why?

Sincerely, 

Your former student,

Fred Hasselman

Carver, R. (1993). The Case Against Statistical Significance Testing, Revisited. The Journal of Experimental Educationl, 61(4), 287–292. Retrieved from http://www.jstor.org/stable/10.2307/20152382 

ps. Carver is one of many scholars who have been warning us about these problems. If you missed his work, here's a selection of just a few of the publications that appeared just before and during the time I was a student that could, maybe should have caught your attention:

Cohen, J. (1990). Things I Have Learned (So Far). American Psychologist, 45(12), 1304–1312.
Cohen, J. (1994). The earth is round (p<. 05). American Psychologist, 49(12), 997–1003.
Freeman, W. J. (1997). Three centuries of category errors in studies of the neural basis of consciousness in intentionality. Neural Networks, 10(7), 1175–1183.
Kugler, P. N., Shaw, R. E., Vincente, K. J., & Kinsella-Shaw, J. (1990). Inquiry into intentional systems I: Issues in ecological physics. Psychological Research, 52(2), 98–121.
Meehl, P. E. (1990). Why Summaries of Research on Psychological Theories Are Often Uninterpretable. Psychological Reports, 66(1), 195.
Michell, J. (1997). Bertrand Russell’s 1897 Critique of the Traditional Theory of Measurement. Synthese, 110(2), 257–276.
Van Orden, G. C., & Paap, K. R. (1997). Functional neuroimages fail to discover pieces of mind in the parts of the brain. Philosophy of Science, 64(S1), 85–94.

pps. I have to exclude the advisors on my Master's Thesis who taught me the important lesson to publish data only when you are more than 100% sure they are reliable. Some of the pronunciation errors I had categorised could not be labelled unambiguously, or in other words, another rater would arrive at a different conclusion. So we did not publish.

天

The thin line between science fact and science friction: Antisocial meateaters, Psi and other forms of empiarrhea

There is a beautiful paragraph in the history of science Michael Seevinck introduced me to. I immediately incorporated it into talks and papers. It is a letter by Albert Einstein he wrote to his friend Maurice Solovine on the 7th of April 1954 in which he explains his philosophy of science. Einstein distinguishes between two types of theories: Theories of construction and theories of principles. Theories of construction are found in a science that is immature. Those theories basically just describe a phenomenon in the empirical record (the data collected in experiments) using fancy language. My favorite example is the scientific notion about reading aloud written text. This is involves converting graphemes into phonemes according to scientific theory. What are graphemes? Well, they represent letters. What are phonemes? They represent sounds. So reading aloud is actually converting letters into sounds... Duh! Of course it is! Why do you need graphemes and phonemes to explain it?

Figure adapted from van Dongen (2010)

A theory of principles is completely different. It embarks from a set of postulates and axioms (A). It derives laws from those claims (S) and seeks to test how accurate those laws are. Einstein understood that a theory of principles “pays for its higher logical unity by having elementary concepts […], which are no longer directly connected with complexes of sense experiences.”. To create a theory of principles means sacrificing explanatory power of a theory. We cannot explain the way the universe works in terms of sensory experience. This was not just a coincidental oddity for Einstein, he firmly believed the fundamental notions of physics cannot be induced from experience and they cannot be justified a priori on the basis of our faculty of knowledge. (This strongly opposes Kant’s notion of incomensurability between Biology and Physics ).

How do scientist get to those systems of axioms and postulates then? Einstein is very clear about this and claims there is no logical route to arrive at A directly from E (direct experience). A formalism is created due to an intuitive leap, a creative act in which a conjecture is raised to a postulate. The arrow that doesn’t quite touch E and jumps upwards to A represents this creative act. The only route that is deducible by logic and reason flows from the formalism in A to the derived laws S and from those laws back to E. Theories of construction on the other hand infer “laws” from complexes of immediate sensations (sensory phenomena) in E. These laws are crudely glued together to construct ad-hoc theories that, in a way, save those phenomena for the constructed theory so far.

Well, what can I say... Sometimes it just doesn't feel entirely right to say "I told you so".

I am not saying that I expected anything like the fraud committed by social psychologist prof. Diederik Stapel, but the entourage, the context, the circumstances that allowed such fraud to go unnoticed and the responses of the universities and his colleagues... well let's just say that I am not entirely surprised it happened.

The reason for me not being surprised is due to the fact that I have been trying to figure out what kind of philosophy of science social psychologists actually adhere to, by studying the argumentation in the papers they write and the way theories are evaluated by the social psychology community. I have for instance been analyzing the paper by Darryl Bem who claims to have found evidence for the existence of Psi. More importantly perhaps, with the help of some bright students I have been trying to replicate some of the famous social psychology experiments. We failed to do so, but we also think we know why they did find something and we didn't, you know, what science is all about. No reason to be suspicious. I also didn't expect fraud because I know social psychologists are masters at inventing ingenious experiments. Really, they generate beautiful datasets by tweaking and fine tuning their stimuli and experimental procedures and they will continue to do so until they find the right conditions to support their hypotheses.

:)

Yes, indeed, that's not exactly the way science should work. I thought that was the problem. They have no real theories, but just produce an ever increasing empirical record of interesting and sometimes very cute phenomena that have no say whatsoever on the level of the major psychological theories about cognition, behavior or development. I call this enormous structureless record of so called scientific facts: empiarrhea (see Caught in the undertow).

Now when I say "them" or "they", to refer to social psychologists, of course I am generalizing: No I do not believe all social psychologists are pretesting designs until they work. What I do believe however, is that psychologists in general, but social psychologists in particular, should change the way they theorize about reality. "It is the theory that decides what we may observe" said Albert Einstein to Werner Heisenberg and he was right.

day 5: f.i.t Disk

[caption id="attachment_293" align="alignnone" width="526" caption="F.I.T"][/caption]

When I saw the F.I.T Mobile Mixer link pop up in google, I immediately thought someone had made a cool iPhone app to mix some tunes, an Ableton-like app or something.

Well.

It wasn’t

Apparently there is a great need among bodybuilders to create “smoothly mixed protein shakes” when there is no socket in the vicinity or while moving around a lot.

It is “A Portable And Lightweight Mixer For Protein Powders And More!” (Nudge nudge, wink wink, know what I mean? )

Here’s the most hilarious part:

“F.I.T. specializes in making mobile drink mixers. All of our products are durable, portable, and lightweight.”

You can specialize in anything of course and I commend the company for finding this niche in the market, but isn’t it kind of weird to promote your product to bodybuilders as being lightweight?

Nice.