The AI Conundrum
What My KS3 Assessment Revealed About Learning
This week, I set one of the most revealing assessments I have ever used in KS3 Computer Science or indeed any other subject. Let me explain that I am not a Computer Science teacher; I’m an English teacher. However, in the way of being Leadership and in an International School, when we can’t recruit a position, we as Leadership step in. We also decided to use this as an opportunity to spend a year teaching our students how to use AI ethically and with more intentionality. For the past year, I have been teaching students about AI: not simply how to use AI tools, but how to think with them, question them, critique them, and use them deliberately.
When I was crafting my end of year assessment, I started from the viewpoint that when students can generate polished work in seconds, the real challenge is no longer the final product. It is whether they can think independently, reflect critically, and manage their own learning process. I also wanted to test their skills and ethical thinking, so I designed something very intentionally different.
Three quarters of the assessment was process-focused.
Students were given:
an open topic,
a checklist of required elements,
access to the rubric throughout,
optional graphic organisers,
and permission to use AI for every part of the task.
The assessment was not about whether they could produce a polished final outcome.
It was about whether they could manage their learning independently.
Could they:
plan?
iterate?
reflect?
follow instructions?
evaluate outputs?
explain their thinking?
demonstrate metacognition?
I even sent a message beforehand reminding students to:
charge their computers,
upload in the correct file formats,
and carefully follow submission instructions.
Because independence was part of the assessment.
As I stated at the start of this article, the results were revealing. Not because of the technology, but because of what it exposed about learning.
Here was the shock.
Around 15% of students, including some very academically strong students, could not complete basic process expectations such as ensuring that their laptops were charged, uploading the correct file formats and following basic instructions.
In a school where students use an LMS daily, I found that genuinely startling.
Some students rushed through the task, focusing almost entirely on producing a polished final product as quickly as possible. Others used the full 90 minutes, carefully refining prompts, revisiting criteria, and evaluating their own decisions throughout the process.
Some of the brightest students produced incredibly creative final products. Because the product format itself was open for the students to choose, many flourished creatively within that freedom.
However, some of those same students ignored major process requirements entirely.
They skipped reflection.
They ignored the rubric.
They failed to evidence iteration.
They treated metacognition as secondary because they viewed the “real work” as the final product.
That, to me, says everything about the current state of education in the age of AI.
For years, many students have been conditioned to believe that achievement is the product.
The grade.
The presentation.
The polished output.
However, AI destabilises that model completely.
When a machine can help generate polished outputs in seconds, process suddenly matters far more than product.
The students who succeeded most in this assessment were not always the students who produced the flashiest outcomes.
They were the students who could explain:
why they made decisions,
how they refined prompts,
where AI helped,
where it failed,
and how their thinking changed during the process.
One of my favourite submissions came from a student with SEND who refused to use AI at all. She used the reflection space to explain exactly why. No, she did not receive marks for AI implementation criteria she chose not to meet, but she earned marks for something arguably more important:
critical thinking,
self-awareness,
and metacognitive reasoning.
That mattered to me.
Because real learning is not blind compliance with technology.
It is conscious engagement with it.
Students were also required to copy and paste their prompts into the submission so I could see evidence of iteration. Prompt refinement itself was part of the assessment criteria.
Again, this revealed enormous differences between students.
Some treated prompting as a one-off command.
Others engaged in genuine iterative dialogue:
testing,
adjusting,
evaluating,
refining.
That iterative cycle matters neurologically as well as educationally.
Research into metacognition consistently shows that learning deepens when students actively monitor, evaluate, and regulate their thinking (Flavell, 1979). Reflection strengthens neural pathways associated with executive function, self-regulation, and long-term retention (EEF, 2021). Cognitive science also suggests that deeper learning occurs when students engage in retrieval, elaboration, and self-explanation rather than passive consumption. When students are forced to justify decisions, evaluate outputs, and revise their thinking, they strengthen schema formation and cognitive flexibility, both of which are essential for transferring knowledge into new contexts. Thinking routines help externalise cognition, making invisible processes visible both to students and teachers. Research from the Education Endowment Foundation also suggests that metacognitive strategies can add the equivalent of seven months of additional progress when explicitly taught and embedded into classroom practice (EEF, 2021).
This is precisely why the assessment worked.
Students could not simply rely on AI to generate an answer and disengage mentally. The structure required them to evaluate, interpret, refine, and explain their use of AI. Those actions increased cognitive engagement because students had to actively process information rather than passively receive it. Cognitive load theory also helps explain the differences I observed. Students who used scaffolds, reflection prompts, and iterative drafting reduced unnecessary cognitive overload and were able to focus more effectively on higher-order thinking. Students who rushed often focused only on surface-level completion.
Some parents later asked why there was not a traditional written assessment on paper.
My response was simple.
To assess a skill, students need to apply it. They also need to evaluate their own usage so they become conscious users of AI rather than passive consumers of it.
A rote memorisation test would tell me very little beyond who had a strong short-term memory. It would not show me who could actually use AI effectively, nor the level at which they could use it. It would not reveal who was simply copying and pasting outputs and who was critically reading, interpreting, adapting, and translating AI-generated information into their own understanding.
Most importantly, this assessment also reminded me that scaffolds only work when students choose to engage with them. Some students used graphic organisers and reflection routines to structure complex thinking beautifully. Others ignored every available support despite having full access to it.
AI does not remove the importance of metacognition. If anything, it magnifies it, because in an AI-supported classroom, the defining skill is no longer simply producing information. It is directing thinking.
This assessment left me convinced of something important:
The future of assessment cannot simply measure whether students can produce impressive outcomes.
We must assess:
independence,
decision-making,
reflection,
adaptability,
and the ability to think critically about both human and AI-generated ideas.
Otherwise we risk rewarding performance while missing learning entirely.
And honestly?
I think many schools are far less prepared for that shift than they realise.
Bibliography
Education Endowment Foundation (EEF) (2021) Metacognition and Self-Regulated Learning Guidance Report. London: Education Endowment Foundation. Available at:
https://educationendowmentfoundation.org.uk
(Accessed: 8 July 2025).
Flavell, J.H. (1979) ‘Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry’, American Psychologist, 34(10), pp. 906–911. Available at: https://doi.org/10.1037/0003-066X.34.10.906 (Accessed: 8 July 2025).