In the past decades, brain imaging techniques have become all the rage in neuroscience research. Instead of bland studies that describe psychological processes in 8,000 word articles, brain imaging allows for pretty, compelling pictures of the brain (as we noted in a blog entry over a year ago).
But the pictures may not be telling us what we think.
A new study by Edward Vul in press in Perspectives on Psychological Science suggests that the validity of many studies that use brain imaging techniques — such as functional magnetic resonance imaging (fMRI) — may be in question:
In those studies, researchers used fMRI to measure blood oxygenation — a marker of neuronal activity — in specific brain regions during behavioral tasks. As is typical in fMRI studies, researchers divided up the brain into tiny cube-shaped regions called voxels and looked for activation within regions they believed were key to the behavior.
The problem, Vul says, is that there are fundamental flaws in the way most researchers determine which voxels to include in their analyses. Many only include voxels that reach a certain threshold of activation; if they hit that threshold, it's a correlation. Since they average these data across many individuals, even random "noise" in the data gets amplified into a false correlation — something Vul refers to as "voodoo correlation."
The problem is that if you have a study with sloppy design, and it gets published in a peer-reviewed journal, it becomes of the accepted literature for that topic. Researchers rarely go back and re-analyze all of those studies on a study-by-study basis and determine which are the “good” studies and which ones should be done away with because of these sloppy designs.
The news media, too, regularly will publish any fMRI study’s finding, regardless of whether it’s good research or not. Why? Because it’s compelling to hear that scientists are narrowing down specific areas in the brain to specific behaviors or emotions. It makes us feel like we’re beginning to understand the brain (when really all we’re doing is scratching the surface of our understanding).
All of which clouds our knowledge in this area, and makes us believe we have a far clearer understanding of the brain than we truly do. Vul’s research is valuable for pointing out how much of the fMRI research is fundamentally flawed and should be withdrawn from the journals they were published in.
And serves as another reminder to be careful of believing those pretty, compelling pictures of the brain.
