A post on Instagram caught my eye yesterday from the ever-entertaining Sophia Smith Galer (author of How To Kill A Language, available from all good booksellers). She had a rant about the introduction of an AI detection tool on Substack, arguing that the company is blaming AI writers for a flood of low-quality content created by its own demand for relentless, unpaid publishing. In her view, turning users into AI police avoids confronting the platform incentives that made automation attractive in the first place. I concur!
I’d previously tested third party slop detectors on some of my old blog posts from 10+ years ago that were written before I knew what an LLM was, and crazily some of them flag up 85-90% AI written. I don’t know, maybe I’ve got a bean-counter’s approach to rhetoric. Sadly Substack’s new AI dipstick only allows you to check Substack posts, and only those written after its introduction on the 21st July 2026. This severely limits one’s ability to test the thing’s efficacy. I did have a quick nose round with it though. I tested a post by my friend and writing mentor Gwyneth Box, who, though new to Substack, is a seasoned and accomplished writer and poet who I know wouldn’t touch AI with a barge pole, and the app said 100% human. I tried various other Substack posts in my feed and after a short while a disturbing pattern emerged. Every assessment was reading 100%, all the posts I checked were either 100% human or 100% AI generated. I will be very surprised if that is truly the case!
When I tried it on a note I’d just penned, Substack’s tool identified it as 100% AI written (it was 85% at most guv, honest!), then it invited me to write a message to any readers who tested my work and found it to be written by machine. I came up with “I use AI to accelerate the writing process. Ideation is my core business. Everything else can be leveraged with tools. A detector may score that as machine-made. I score it as how good writing gets done now.”
So basically if AI ‘offends’ you, my posts are not for you. Sorry about that, but we’re in a transitional period. In five years’ time I’d be really surprised if you can tell AI prose from human, as the improvement over each new model is staggering. If you don’t want your free ChatGPT essay to sound like AI slop, just run it through a better model like Claude’s Fable, it’ll detect and fix all the usual ‘tells’ such as triadic parallel binaries, cascaded negation-elevation antithesis, and my particular favourite, aphorism-stacking (I’m pointing at you “House of El” YouTube channel).
After a bit more digging about this I came across Substack’s original blog post announcing this measure and found that they were using a technology called Pangram, which has its own website, so I ventured across and tried one of my own blog posts in its free online AI checker. I chose “The Night the Internet Gods Turned On Me” which, like all my recent posts, starts off handwritten then gets the AI treatment. I’d say about 30-40% of it was ‘improved’ by AI. Not so says the detector, your post is 100% human (snigger). Engadget said of Substack: “The blog post announcing the feature is surprisingly spicy. There’s a dig at LinkedIn about the presence of AI-generated content on that service and it dubs attempting to create feigned human connection with AI slop ‘Claudefishing.’” Read more.
My view is Substack have screwed up majorly here. My guess is their PR advice has been to counter the growing anti-AI sentiment, which has as much to do with the environmental disasters of humongous data centres and the role AI plays in the ever growing onslaught of mass-state surveillance, but instead of looking to the future they’ve planted a foot in the past. Tech moves on and has a habit of leaving conscientious objectors behind. Remember I said this in ten years’ time when you’re talking to a local AI model so small it’s embedded in your phone and is ambivalent about having an Internet connection. Beam me up Scotty!