An AI Score Is Not Yet Evidence of Competence

A learner submits an essay. A system applies a rubric, produces a score and generates fluent feedback in seconds. The number looks precise, the comments look personalised and the institution appears to have solved a problem of scale.

But what has actually been established? Did the score represent the intended knowledge or merely the surface features of the response? Was the work produced by the learner, by an AI system or through a mixture the assessment was never designed to interpret? Does the model treat language, disability and developmental variation fairly? Can the learner explain and transfer the competence beyond the platform?

Artificial intelligence can assist assessment. An AI score is not yet evidence of competence.

The governing question is not whether a model agrees with a marker. It is whether the complete assessment process supports a defensible inference about what a particular learner knows, can do and can be trusted to do in the setting for which the award or decision matters.

The forty-second annual conference of the Association for Educational Assessment in Africa concluded in Arusha on 21 August. Hosted by Tanzania's National Examinations Council, the programme was organised around rethinking educational assessment in the age of artificial intelligence.

Its announced themes included personalised and inclusive assessment, e-assessment security, AI in technical and vocational assessment, data-informed policy, scenario-based simulation, continuous assessment, competency-based learning…