Tool or Shortcut? What AI Is Really Doing to Our Learning
New findings from PISA 2025 and a long-term study from China point to the same conclusion: it is not whether students use AI that matters most. It is how they use it.
New findings from PISA 2025 and a long-term study from China point to the same conclusion: it is not whether students use AI that matters most. It is how they use it.
A tricky text can be summarized in seconds. An essay outline can appear after a single prompt. A math problem can be explained step by step or simply solved outright. Generative AI can remove an enormous amount of work from students’ shoulders. That is precisely what makes it such a powerful tool. What does that mean for learning? A good result is not the same as learning. Learning happens through thinking, trial and error, remembering, discarding, and correcting, often in the moments that feel most effortful.
This is the real fork in the road with generative AI: Are we using it as a tool that supports learning, or as a shortcut that skips the process altogether?
Two recent studies do not settle the question. They do, however, make a strong case for paying far more attention to how students use AI than to whether they use it.
Generative AI is already a fixture of everyday school life. According to the latest PISA results, 46 percent of students across OECD countries use AI chatbots at least weekly to support their learning.
The relationship between AI use and performance is not as simple as “more is better.” Students who use AI moderately as a learning aid score somewhat better than students who do not use AI at all. However, when AI is used regularly to generate draft texts for homework, the correlation turns negative.
It is important to stress that PISA shows correlation, not causation. We cannot conclude that frequent AI use causes weaker performance. The pattern does raise an important question, though: When does AI support learning, and when does it begin to work against it?
This is where a recent long-term study from China becomes especially interesting. In The Generative AI Learning Penalty: Evidence from Chinese Secondary Education, researchers tracked more than 26,000 students in grades 7 to 12 over 30 months. They compared homework scores and completion times with results from monthly exams completed without any aids.
At first, the data looks like a success story. Students become more productive. After roughly six months of AI use, homework scores rise by 18 percent, while average completion time falls from 64 to 45 minutes. The results are better and the work takes less time, which is exactly what makes AI so appealing.
The picture changes at exam time. Scores on the monthly unaided exams fall by around 20 percent. Students become faster and more successful at homework, but that improvement does not carry over when AI is unavailable.
This reveals the gap between outcome and process. AI can get a student to a good result quickly. If that result comes from skipping the cognitive effort that learning depends on, however, the student may save time while losing the learning opportunity that came with it.
We do not learn only when the right answer appears on the page. We learn while searching for it: remembering, connecting ideas, trying approaches, getting things wrong, and correcting them.
Generative AI can support that process, or it can complete the work on a student’s behalf. The China study offers one especially telling detail: learning losses are concentrated among the roughly 80 percent of AI-using students who sharply reduced their homework time after AI entered the picture. The researchers describe this pattern as “homework outsourcing.”
There is also a second group: students who use AI without cutting back on the time they invest. On average, they achieve exam scores comparable to those of peers who do not use AI, while also submitting stronger homework.
The researchers are careful not to draw a direct causal line between time spent and exam performance. Even so, the finding is striking: not all AI use is created equal.
A few simple examples make the difference clear.
A student can ask a chatbot to solve a problem outright, or ask for a hint on the next step while keeping the solution hidden.
A student can have a text summarized automatically, or write a summary first and then ask what important points may have been missed.
AI can produce a complete argument from scratch, or it can offer counterarguments, ask focused follow-up questions, and stress-test an argument the student developed independently.
The tool may be the same, but the learning process is entirely different. That is why AI literacy is about more than writing effective prompts. Students also need to learn when it is appropriate to let AI take the wheel and when the thinking itself is the point of the task.
This has real implications for how schools design assignments.
If AI can summarize a text in seconds, the key question is not whether summaries are still worth assigning. The key question is what the assignment was meant to teach in the first place.
If the goal is simply to produce a good summary quickly, AI can be a genuinely useful tool. If the goal is to help students identify what matters, weigh information, and express ideas in their own words, then a ready-made AI summary bypasses the learning the task was designed to encourage.
The same logic applies to presentations, arguments, research tasks, coding assignments, and more. Whether AI use makes sense is not a property of the task itself. It depends on the learning goal behind it.
That points to a question teachers will increasingly need to ask: What kind of thinking do I want this task to build, and how much of that thinking am I willing to hand over to AI?
Designing tasks around that question takes practice. Our course, Designing AI Fluency Missions by Phil Alcock, offers hands-on guidance for building assignments that use AI deliberately and with learning in mind.
None of this is an argument for keeping generative AI out of the classroom. AI is already deeply woven into how students live and learn, so a complete ban is unlikely to be realistic.
There will still be times when working without AI is the right choice. Foundations need to be built, skills need practice, and knowledge needs to be accessible without a digital tool.
Students also need meaningful opportunities to learn with AI. They need to see how it can explain concepts, provide feedback, ask useful questions, and support practice. They also need to scrutinize its answers and understand its limitations. Access to a chatbot is not the same as AI literacy.
Building that judgment, knowing when AI is helping students think and when it is beginning to think for them, is the focus of our course, AI Efficiency vs. AI Dependency: Protecting the Thinking in Student Work by Rachelle.
Research on the long-term effects of generative AI on learning is still developing. These studies do not answer every question. They do bring the most important questions into focus: Which skills do students still need to master without AI? When does AI support the learning process, and when does it replace it? How should homework and exams change when AI is always available? How can we design tasks so that AI does more than produce stronger outputs and genuinely contributes to learning?
The question is no longer simply, “Should students use AI?” It is this: When is AI a tool that helps students learn better, and when does it become a shortcut that skips the part they were supposed to learn?
No AI can make that judgment for us. That may be the AI skill most worth teaching students today.

AI in Primary School: Does It Really Need to Start This Early?