A landmark study that asserted OpenAI’s ChatGPT enhances student learning outcomes has been retracted by Springer Nature nearly one year after its initial publication. The retraction cites "discrepancies" in the analysis and a lack of confidence in the conclusions, despite the paper accumulating hundreds of citations and gaining significant traction on social media.
"The paper's authors made some very attention-grabbing claims about the benefits of ChatGPT on learning outcomes," said Ben Williamson, a senior lecturer at the Centre for Research in Digital Education and the Edinburgh Futures Institute at the University of Edinburgh in Scotland, in an email to Ars. "It was treated by many on social media as one of the first pieces of hard, gold standard evidence that ChatGPT, and generative AI more broadly, benefits learners."
The retracted study aimed to measure "the effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking" by synthesizing results from 51 prior research studies. Its meta-analysis compared experimental groups that used ChatGPT in educational settings against control groups that did not utilize the AI chatbot to calculate an effect size across the studies.