In a Veteran Speechwriter’s Hard-Wrought Work, Claude Finds Dozens of AI ‘Tells’

From a veteran speechwriter correspondent, who has been part of an ongoing discussion around the Professional Speechwriters Association, about AI.

The PSA’s AI guru Brent Kerrigan had sent around an elaborate prompt to help speechwriters detect AI writing in materials they get for inclusion in speeches and in other communications. 

“For some reason, I thought it would be fun to run this prompt against two recent speeches that I wrote,” the speechwriter wrote. She used the version of Claude that her organization allows.

“I want to say at the outset: I do not use AI to help write my speeches. I am not allowed to use AI to write my speeches. And I do not want AI to write my speeches.”

The results? “One of my speeches had 24 alleged tells, the other 28″—Including her use of the rule of three, which her speaker loves to employ. “So many writers I admire—and so many great editors whom I have had the privilege of working with—use the rule of three. As far as I’m concerned, Claude can stuff it.”)

Other results, with Claude in ital and the speechwriter’s comments in parens:

From the structural diagnosis in the speech with 24 alleged tells: The draft reads as human initiated but with significant AI assistance in expanding sections …  Almost every sentence is grammatically complete; there are no sentence fragments, no mid-thought pivots, and no places where the speaker seems to catch himself (Comment from me: Seriously????) The subheadings are genuinely creative and speaker-specific … but the body text under those headings resolves every thought neatly … The [framework] is doing real work and likely reflects the speaker’s authentic voice, but the connective tissue between those moments has the rhythmic smoothness and risk-averse hedging characteristic of model-generated … commentary. 

From the structural analysis of the speech with 28 alleged tells: The draft exhibits moderate AI patterning concentrated in transition language and structural signposting rather than in core content or forbidden vocabulary. … The primary tell is the abundance of transition scaffolding … These are the seams showing. The extended metaphor … reads as algorithmically constructed with each element mapped too neatly onto the target domain. (From me: Too neatly? I worked extremely hard on that metaphor. And I have no idea what Claude means by the “target domain.”) Em-dashes (from me: I didn’t use many em-dashes. But why can’t we use em-dashes anymore?) appear several times to insert explanatory phrases that feel like automated elaboration rather than natural parenthetical thought. Overall, this reads as a draft where a human built the argument and selected the data, but AI assisted with transitions, structure, and metaphor extension, leaving behind connective tissue that needs surgical removal.

The speechwriter concludes, “I can’t tell you how anxiety producing this is … It’s also hugely infuriating! I guess my question is, is AI good at detecting AI? Or are we all just screwed?”

To which Kerrigan replies, “One thing to remember—how did AI become AI? By literally scraping whatever existed publicly. We wrote the speeches it scraped. We and many, many others who had no idea what goes into a good speech. Mix ‘er all together and what we get are the current LLMs.”

To sum up: Borrowing rhetorical methods from the ancients, speechwriters write speeches. LLMs inhale all the speeches, then accuse speechwriters of stealing these methods from AI. Maybe we should stop raging and worrying about this, and start laughing instead.