ChatGPT vs human: How a Singapore professor could tell which essays were written by a reporter or the chatbot

• Published date: 02 February 2023
• Publication title: Malay Mail Online

In December, one of my more tech-savvy friends asked if I wanted him to generate a story about me using artificial intelligence (AI). As a writer, I was naturally intrigued, so I agreed.

To my surprise, his AI tool produced a half-decent, though bizarre story that ended with me serving a lifelong prison sentence after he'd put in the prompt 'female named Si Yuan lost her iPhone and committed a war crime'.

Although the story lacked depth, it was coherent and understandable while also fulfilling the weird parameters set by my friend; oddly impressive for an AI tool.

This led me down the rabbit hole of Chat Generative Pre-Trained Transformer, or ChatGPT for short, the hottest new AI breakthrough.

It is, in its own words, 'a language model developed by OpenAI, designed to generate human-like text based on input prompts'.

As a creative writing student, I was curious if it could produce my schoolwork for me since most of my projects involve writing creative works that can't just be plucked off the internet.

My editors, too, were curious about the possibilities of this new technology - could it eventually replace our newsroom of humans and make journalists obsolete?

So we put ChatGPT to the test and lined up my old schoolwork against material produced by the AI, to be judged by a professor.

I polished up an old essay, an opinion column and a piece of fiction, and fed ChatGPT prompts to write up three works with similar content to mine.

The six pieces were sent to Associate Professor Barrie Sherwood of Nanyang Technological University's English programme. He is an expert on creative writing and a published novelist.

Assoc Prof Sherwood was told that three were written by a human and the other three by ChatGPT, and he was asked to determine which was which.

How would I fare against a robot?

Read on to find out what happened.

The opinion pieces

Of the three types of writing, Assoc Prof Sherwood said that it was hardest to make the call on the opinion pieces.

He'd initially assumed that the chatbot's writing was the one with 'lapses in sense and grammar' and misused idioms and metaphors.

However, the lack of errors in the other piece of writing made him suspicious.

He noted that the first opinion piece swung wildly between diction levels, that is, the type of words chosen.

The writer uses the relatively highbrow 'boycotted' in one paragraph, then descends to finding herself 'getting slammed'. The piece also lacked the structure and clarity of the second piece.

Yet, the second piece...