Generative AI tools, like OpenAI's GPT-4, are increasingly integrated into educational settings. However, their long-term impact on skill acquisition remains under scrutiny. This report from the Wharton School of the University of Pennsylvania assesses how generative AI affects student learning.
The study involved nearly 1,000 students across three grades, evaluating two GPT-4-based tutors: GPT Base and GPT Tutor. The GPT Base provided a standard ChatGPT interface, while GPT Tutor incorporated safeguards to support learning without providing direct answers.
The results revealed that while GPT-4 significantly improved immediate performance on practice problems—by 48% for GPT Base and 127% for GPT Tutor—these gains did not translate into long-term learning. When access to GPT-4 was removed during exams, students who had used GPT Base performed 17% worse than those who never had access, indicating a detrimental effect on learning. However, GPT Tutor mitigated this negative impact, with performance differences becoming statistically insignificant.
The study underscores the potential of generative AI to enhance short-term performance but also highlights the risk of overreliance on these tools, which can inhibit the development of essential problem-solving skills.
As educational institutions increasingly turn to AI-driven tools, the findings stress the need for carefully designed safeguards to ensure that students continue to learn and retain critical skills over time.