About Those Baby Brainwaves: Why “Policy Relevant” Social Science is Mostly a Fraud

But the difference in the effects may not be significant; the study may simply be underpowered for the effect to be significant in one group while the other may have had a more extreme result than they should have by chance alone.37 The proper way to test this difference is to test for an interaction in a singular model or to directly compute the significance of the difference in effects. [...] The first and third explanations had to do with the difference between the EEG and total samples, the second, with the existence of the effect. [...] The 2010 Head Start Impact Study conducted by the Office of Planning, Research and Evaluation looked at effects elicited by the nationally-implemented, well-funded, and widely celebrated Head Start program for three- and four-year-old children.57 Using a p-value cutoff of 0.10, they found significant positive effects on eight of fourteen cognitive tests given in the year of the program for the thr. [...] There was a significant effect of treatment relative to the cash group for the Bayley-III cognitive test and receptive language at the first follow-up in the comparison of the first treatment and the control group. [...] Does the blame lie with the journalists, who had all the opportunity in the world to look at the research, read the citations, and realize it did not hold up? Or is it the fault of the authors, who pushed their paper through PNAS’ easy track for submission and chose to promote an untenable interpretation of their findings to the wider world? It might be the fault of readers, who believed any of th.