Nature reports:

In 2019, neuroscientist Scott Marek was asked to contribute a paper to a journal that focuses on child development. Previous studies had shown that differences in brain function between children were linked with performance in intelligence tests. So Marek decided to examine this trend in 2,000 kids.

Brain-imaging data sets had been swelling in size. To show that this growth was making studies more reliable, Marek, based at Washington University in St. Louis, Missouri (WashU), and his colleagues split the data in two and ran the same analysis on each subset, expecting the results to match. Instead, they found the opposite. “I was shocked. I thought it was going to look exactly the same in both sets,” says Marek. “I stared out of my apartment window in depression, taking in what it meant for the field.”

Now, in a bombshell 16 March Nature study, Marek and his colleagues show that even large brain-imaging studies, such as his, are still too small to reliably detect most links between brain function and behaviour.

As a result, the conclusions of most published ‘brain-wide association studies’ — typically involving dozens to hundreds of participants — might be wrong. Such studies link variations in brain structure and activity to differences in cognitive ability, mental health and other behavioural traits. For instance, numerous studies have identified brain anatomy or activity patterns that, the studies say, can distinguish people who have been diagnosed with depression from those who have not. Studies also often seek biomarkers for behavioural traits.

“There’s a lot of investigators who have committed their careers to doing the kind of science that this paper says is basically junk,” says Russell Poldrack, a cognitive neuroscientist at Stanford University in California, who was one of the paper’s peer reviewers. “It really forces a rethink.” [Continue reading…]