A Caring Voice Is Not a Caring Relationship

A chatbot can answer at midnight, remember a preference, adopt a warm voice and produce language that sounds patient, compassionate or prayerful. It does not become tired, embarrassed or visibly distracted. For a lonely learner, an overwhelmed minister or a person afraid to disclose distress, this responsiveness may feel safer than approaching another human being.

The experience can be useful. A person may organise thoughts, rehearse a difficult conversation, clarify a concept or locate an appropriate service. The ethical mistake begins when conversational fluency is treated as relationship. A system can imitate attention without knowing a person, mirror emotion without feeling it and recommend action without accepting responsibility for the result.

Education, ministry and public service therefore need a relational boundary as clear as their academic and data boundaries. AI may assist a relationship. It must not silently acquire the authority of teacher, mentor, pastor, counsellor, companion or accountable decision-maker.

In an interview published by Reuters on 15 September, Nobel Peace Prize laureate Maria Ressa argued that digital technology has moved through three dangerous waves: the capture of attention through social media, the manipulation of human needs through chatbot intimacy and the transfer of agency to systems that act on people's behalf. Reuters

Ressa is co-chair of the United Nations Independent International Scientific Panel on AI. The Panel was established to create a shared evidence…