I have been thinking about AI slop for a while, partly because it seems increasingly difficult to avoid.
My dad watches quite a lot of videos online, and recently I have noticed him moving from one obviously AI-generated video to another. I find some of them quite jarring. Around the same time, I had a conversation with a colleague about AI-generated social content. We talked about how easy it is to produce something that looks perfectly acceptable but still feels generic, and how that kind of content doesn't necessarily lead to much engagement.
That got me thinking about something slightly closer to home: the articles and blog posts we had been creating for Flow Strategic.
There was nothing necessarily wrong with them. They could be well structured, informative and grammatically fine. But sometimes I felt strangely disconnected from the finished article. I could read something published under my name and think: I don't really feel like I wrote this.
That worries me more than whether somebody can spot the familiar signs of AI writing.
If producing content becomes increasingly easy, we could end up with businesses publishing more and more material that neither the writer nor the reader feels particularly connected to. Making AI generate another thousand words isn't difficult. Making those thousand words contain something worth saying is a different problem.
So we decided to play around with it.
Two people, two AI editorial assistants
A colleague, Karina, and I had a small hackathon around a fairly loose challenge: could we build an AI editorial assistant that helped reduce some of this generic AI content?
We worked independently. I built mine using ChatGPT and Karina built hers using Claude. Importantly, we didn't sit down beforehand and agree what the assistants should do or what "more human" content meant. We developed our approaches separately and only compared notes the following week.
My initial thinking was around voice. A business has a brand and certain principles it wants its content to follow, but I didn't think that was enough. Individual writers within that business should still sound different from one another.
That led me towards creating a writer profile as well as a business profile, looking at previous writing samples and trying to understand how someone naturally communicates. As I worked on it, though, I started to realise that voice was only part of the problem.
An AI can know how I tend to write without knowing what I think about the subject in front of me.
So the assistant gradually became less of a writing tool and more of an editorial process. Before drafting, it interviews me. It asks what I have observed, what I think people misunderstand, whether I have a relevant example, what I disagree with and where I'm uncertain. It plays the position back as a brief for approval before it writes the article.
Some of those stages weren't in my original plan. We figured them out as we went along.
Karina took a different route with Claude. When we eventually compared the assistants, one of the things that struck me was how quickly hers could produce a substantial article with a good amount of supporting research.
Neither of us had been trying to beat the other. What interested us was the logic the two assistants had independently developed around the same broad problem.
Then, somewhat accidentally, we turned it into an experiment.
What happens when you compare them?
We chose two topics relevant to Flow Strategic and started scoring the resulting articles against ten attributes: voice match, originality, specificity, credible authority, audience relevance, insight density, natural language, factual support, conversation potential and commercial restraint.
We also created direct-prompt versions as a comparison. This wasn't a one-line "write me a blog about AI" prompt deliberately designed to produce rubbish. The intention was to see what happened when we asked AI to write the content more directly rather than taking it through the editorial process.
On our first topic, about not allowing FOMO to dictate your AI strategy, the direct-prompt article scored 3.8 out of 5 using our framework. The article created through my ChatGPT editorial assistant scored 4.25. Karina's Claude editorial-assistant version scored 3.8.
The second topic was about businesses buying AI licences before properly understanding the work they want AI to improve. Here the differences were larger. The direct-prompt article scored 3.3, my editorial-assistant version 4.5 and Karina's version 3.7.
Those numbers are interesting, but I wouldn't read too much into the ranking.
We hadn't set out to run a controlled experiment. If we had, we would have designed it differently. The assistants were independently created, they used different models and different processes, and we hadn't accurately timed every stage. The assessment wasn't blinded either.
What we had really created was an exploratory test that raised some interesting questions.
A good prompt isn't necessarily the problem
One thing I don't want to conclude from this is that prompting doesn't work.
Our FOMO direct-prompt article scored 3.8. That's not terrible. Depending on what you're writing, a sufficiently detailed prompt may be all you need.
If I were looking at somebody else's AI content process now, I'd want to see both their prompt and the output before telling them they need anything more complicated.
The distinction becomes more interesting when the article is supposed to demonstrate subject-matter expertise.
You can put a lot into a prompt: the audience, business context, desired tone, length, structure, SEO requirements, things to avoid and perhaps even a description of the writer's voice. What you cannot put into it is an experience or opinion that you haven't yet thought to mention.
That was where I found the interview process useful.
For the AI-licence article, for example, the assistant asked about my own experiences of paying for software I stopped using. That reminded me of subscriptions I had bought for creating music and video. I had produced around 30 tracks towards a personal target of 50, stopped working on them, and continued paying for some of the tools afterwards.
It wasn't an extraordinary story. That was partly why I hadn't thought to put it in a prompt.
But it gave us something real to explore: buying access to a capability isn't the same as embedding it into your behaviour. From there, we could connect my own experience with things I had seen during digital-transformation projects.
The AI didn't invent my position. The conversation helped me find material I already had.
Research created a different trade-off
Karina's Claude assistant surprised me in a different way. It produced detailed, research-heavy articles quickly.
There was a lot I liked about that. If AI can reduce the time somebody spends searching for relevant evidence and give them useful sources to investigate, that's valuable.
Our review also highlighted another problem to solve, though. Having a source next to a statement isn't necessarily the same as the source supporting the precise statement being made.
In some places we found good research followed by a reasonable interpretation, but the interpretation was then written as though the research had established it directly. That's something I think an editorial assistant should challenge rather than merely checking whether a citation exists.
So the ideal process may need both forms of authority: the experience and judgement of the human, and reliable external evidence that supports, challenges or qualifies what that human thinks.
There is a cost to getting more from the human
There was another fairly obvious difference between the assistants: mine took longer.
The interviews I liked so much required me to answer questions. The assistant sometimes followed up. I then reviewed the proposed position before it drafted anything.
Karina's assistant could get to a researched article much faster.
We didn't accurately measure the time involved, so I'm not going to invent numbers after the event. But the trade-off was obvious enough for us to notice.
That raises what I think is one of the more useful questions to come out of the exercise. If additional human input improves the content, how much human time is it worth?
The goal probably isn't to make the human spend an hour being interviewed before every LinkedIn post. Nor do I think it should be to remove the human entirely so we can generate fifty articles while they get on with something else.
Perhaps the better design problem is: how do we extract the maximum amount of useful human insight with the minimum amount of the expert's time?
That is something I'd like to measure properly next.
We still didn't solve voice
There was also an uncomfortable finding in my own assistant.
Despite everything I had put into writer profiles, writing samples and voice instructions, I still didn't think it completely sounded like me.
One of the habits I noticed was its tendency towards short, sharp paragraphs and emphatic one-line statements. They weren't necessarily bad sentences, but my natural writing tends to develop thoughts through longer paragraphs.
That made me reconsider what we mean by "voice".
It isn't enough to tell an AI that someone is conversational, reflective or direct. Those descriptions could apply to thousands of people. Voice also lives in sentence length, paragraph structure, rhythm, how someone introduces examples, how certain they sound, the kinds of questions they ask and the way one thought leads into another.
So even the process that scored highest in our little test still had something fairly fundamental to improve.
What did we actually learn?
I don't think our mini experiment demonstrates that ChatGPT is better than Claude, or that an editorial assistant will always beat a good prompt. There were too many differences between the processes to make either claim.
What it has done is change the question I'm interested in.
I started by thinking about how we could get AI to write in a more human way. I'm now wondering whether that's slightly backwards.
Perhaps the useful role for AI isn't simply to imitate the human more convincingly. Perhaps it can help the human think about what they actually want to say, challenge the gaps, find evidence and then help turn that into something other people will find useful.
For some subjects, a detailed prompt may still be perfectly adequate. But when the reason for reading something is the knowledge and judgement of the person supposedly writing it, I'd like the AI to talk to that person before it starts pretending to be them.
Our next experiment needs to be better controlled. I'd like to compare the same models using both a structured direct prompt and an editorial workflow, measure the actual human time involved, blind the evaluation where possible and look specifically at what people perceive as generic or interchangeable writing.
Because reducing AI slop probably isn't going to be solved by asking AI to sound less like AI.
We may have to give it something genuinely human to work with first.