AI-generated school essays are not better than the ones written by real students, according to a study published in the journal Applied Linguistics.
For the study, a team from the University of East Anglia, UK, compared the work of 145 real students and 145 essays generated by ChatGPT. The results showed that AI essays had correct answers, were written coherently, and were grammatically sound, but they lacked a personal human touch.
“We found that the essays written by real students consistently featured a rich array of engagement strategies, making them more interactive and persuasive. They were full of rhetorical questions, personal asides, and direct appeals to the reader – all techniques that enhance clarity, connection, and produce a strong argument,” said Prof Ken Hyland from UEA’s School of Education and Lifelong Learning.
“The ChatGPT essays, on the other hand, while linguistically fluent, were more impersonal. The AI essays mimicked academic writing conventions, but they were unable to inject text with a personal touch or demonstrate a clear stance. They tended to avoid questions and limited personal commentary. Overall, they were less engaging and less persuasive, and there was no strong perspective on a topic. This reflects the nature of its training data and statistical learning methods, which prioritise coherence over conversational nuance,” he added.
The authors suggest that this underlines the importance of fostering critical literacy and ethical awareness in this digital age. It is hoped that these findings could also help teachers and educators detect cheating in schools, colleges, and universities worldwide by teaching them to recognise machine-generated essays.
“Since its public release, ChatGPT has created considerable anxiety among teachers worried that students will use it to write their assignments. The fear is that ChatGPT and other AI writing tools potentially facilitate cheating and may weaken core literacy and critical thinking skills. This is especially the case as we don’t yet have tools to reliably detect AI-created texts,” said Prof Hyland. “In response to these concerns, we wanted to see how closely AI can mimic human essay writing, particularly focusing on how writers engage with readers.”
Despite its shortcomings, the authors do not completely dismiss the use of AI in the classroom. Instead, the researchers suggest that tools like ChatGPT should be used as teaching aids rather than shortcuts.
“When students come to school, college or university, we’re not just teaching them how to write, we’re teaching them how to think – and that’s something no algorithm can replicate,” concluded Prof Hyland.
Feng (Kevin) Jiang, Ken Hyland, Does ChatGPT Argue Like Students? Bundles in Argumentative Essays, Applied Linguistics, 2024; amae052, https://doi.org/10.1093/applin/amae052