English | EN 
We’re in the middle of the widespread introduction of Generative AI into many aspects of our lives.
Non-technical individuals may not directly see much of this transition. To the average consumer, the use of Generative AI as a customer support agent,where most consumers are likely to encounter it,is mostly invisible.Tt still looks like a Chatbot, though it seems to be better at responding meaningfully to natural language input.
As such, this may appear to be an incremental but meaningful, improvement to the customer support experience.
The more interesting potential of Generative AI seems to be in the area of writing, whether it be the prosaic work of creating PowerPoint presentations and technical documentation or the more interesting,and difficult process of creating narrative works of art.
The latter is certainly a more exciting possibility. We have hints of what Generative AI could do for writing in what we already see today in the creation of visual works of art.
However, there are two current problems with using Generative AI for writing narrative works:
- The quality of narrative writing work done by Generative AI seems to be poor at the moment.
- We, as technologists, are poor arbiters of what is good narrative writing.
Recently, a group of professional editors undertook to test ChatGPT’s ability to edit a work of fiction. The story chosen had already gone through several rounds of human editing, ultimately resulting in the story being published. This made the story a good test of how ChatGPT would do at editing a story compared to human editors.
“We chose it because we believe the literary genre, with its play and experimentation, poetry, and lyricism, offers rich pickings for complex editorial conversations. (And because we knew we could get permission from all participants in the process to share their feedback.)”
“Can ChatGPT edit fiction? 4 professional editors asked AI to do their job – and it ruined their short story“, The Conversation, Feb 12, 2024
The summary?
The editing recommendations made by ChatGPT were typically good, if somewhat stock and anodyne.
“ChatGPT picked up on the major themes and main characters. And the advice for more foreshadowing, dialogue and description, along with shorter paragraphs and an alternative ending, was generally sound.”
ibid
This set of first-pass suggestions is what might be found in a workshop on writing fiction,unsurprising, as this is likely the kind of content ChatGPT had available to it.
In a second, more hands-on, request to ChatGPT for editing advice, the program did very poorly.
“ChatGPT offered a radically shorter, changed story. The atmospheric descriptions, evocative imagery and nods towards (unspoken) mystery were replaced with unsubtle phrases…”
ibid
Clearly, ChatGPT did not have the ability to “understand” the point of the story, nor the importance of evocative imagery and subtle phrasing that are the hallmarks of good story writing.
In a third request where the story, already modified somewhat by human editors, was submitted to ChatGPT for further editing advice, the program offered the same suggestions as the first round.
However, asking ChatGPT for advice with tense, spelling, and punctuation, ChatGPT did a creditable job.
“But ChatGPT’s changes revealed its own writing preferences, which didn’t allow for artistic play and experimentation. For example, it reinstated prepositions like “in”, “at”, “of” and “to”, which slowed down the reading and reduced the creativity of the piece – and altered the writing style.”
ibid
ChatGPT did not ask for the authorship of the story, nor did it notice that the story was already published.
This is somewhat surprising, but may not represent a fatal problem with the use of ChatGPT as an editor. Human editors at publications will generally ask questions to confirm authorship. But it means that human editors must remain in the loop.
All is not lost, however. The article authors see a role for Generative AI, though carefully limited for the moment.
“This experiment shows how AI and human editors could work together. AI suggestions can be scrutinised – and integrated or dismissed – by authors or editors during the creative process.”
ibid
A hopeful story for the use of Generative AI in the human-dominated area of narrative writing? Sure, but I think we have a more fundamental problem with pushing Generative AI to do more.
We, as technologists, are generally terrible writers and editors.
We may be good at business writing, PowerPoints, technical documentation, and the like. But we are not trained as good narrative writers, and so are not good judges of what constitutes interesting, meaningful stories as edited by Generative AI.
We need to include non-technical, subject-matter experts in the training and review of Generative AI use in areas such as this, if we expect it to be more widely accepted and used by the general public. Which will be interesting, as many of those same experts fear being replaced by that very Generative AI.
Making those people, writers, editors, typesetters, part of a Generative AI/human team will certainly be the best approach.
