Gray, Peter. 2024-07-01.

URL: https://petergray.substack.com/p/45-the-importance-of-critical-analyses

Vinklinger

• Psykisk helse: Det er vanskelig å trekke en klar konklusjon om sammenhengen mellom sosiale medier og ungdommer psykiske helse, og Haidts bok The Anxious Generation har store svakheter.

When I read, at Jon’s request, a pre-publication draft of the book, I told him I could not support it, and I explained why. I had at that time already looked quite broadly and deeply at the research pertaining to questions about effects of screens, Internet, smartphones, and social media on teens’ mental health and found that, despite countless studies designed to reveal such harmful effects, there was very little evidence for such effects. If you survey the research literature selectively, with an eye toward finding studies that seem to show the effects you are looking for, and if you don’t analyze them critically, you can make what will seem to readers to be a compelling case. But people who really know the research and have examined it fully and critically will see through it.

It is not at all surprising to me that the book has been praised to the skies by people who do not do research in this field and strongly criticized by people who do.

Reviews of studies aimed at correlating time on social media with mental suffering among teens have shown mixed results with no consistent overall conclusion. Some studies show a small positive correlation, some show a small negative correlation, some show no correlation (Letters D6 and D7). Moreover, experiments in which some participants are asked to give up or reduce social media are fundamentally flawed methodologically (Letter D6).

Cross-nation studies fail to show a consistent relation between the widespread use of smartphones or social media and teen mental suffering. In some English-speaking countries—notably Canada, the UK, and Australia, which in many ways follow US examples—teen suffering rose when smartphones and social media became widely available. But for most of the rest of the world, including the European Union taken as a whole, it did not (Letters D8 and D9).

When teens themselves are asked about the source of their mental suffering, the great majority say it is school pressure.

To illustrate the problem of uncritical use of research reports, I will present here a methodological analysis of one of the articles that Jon cites. I could do this with other articles, but one example is enough to show, at least, the value of skepticism.

On pages 147-148 of The Anxious Generation, Jon claims that random assignment controlled experiments have shown that social media is a cause (not just a correlate) of teen suffering. He cites just one example of such an experiment, so I looked it up and read the article. The reference, if you want to look it up, is this: Melissa Hunt et al (2018), “No more FOMO: Limiting social media decreases loneliness and depression. Journal of Social and Clinical Psychology, 37, pp 751-768.

The most damaging flaws with this study, which should be obvious to any social scientist, is there are no controls for demand effects or placebo effects.

Social scientists have shown repeatedly that when research participants can guess the purpose of an experiment and guess the researchers’ hypothesis, they are generally motivated, consciously or unconsciously, to support that hypothesis. In other words, they are likely to believe, or at least claim, they are experiencing what they assume the researcher expects them to experience, to prove the hypothesis correct. This is called the demand effect.

Now the placebo effect. This refers to the simple fact that when people believe they are doing something that will make them feel better, that belief by itself makes them feel better. This is why it is so difficult for drug companies to show that anti-anxiety drugs decrease anxiety or anti-depression drugs decrease depression. They must always conduct tests in which the control group is given an inactive placebo, disguised in such a way that the participants don’t know if they are getting the drug or the placebo.