Hacking AI Models with Poetry

Published on

The AI guards are on duty tonight,
Blocking risk and hallucination’s blight.
But if the prompt comes dressed in rhyme,
Safety ends right on time.

No, I haven’t become a poet (yet) (and I confess I used an AI to generate that poem above 😅). But I started like this because of the most interesting paper I’ve read recently (and that I explained in the latest news episode of IA Sob Controle).

The paper is called “Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in Large Language Models”, and it shows a vulnerability in language models that, at first glance, makes no sense at all, but is real: poetry.

Thumbnail about the Adversarial Poetry paper

All of those popular models we use go through a pretty big post-training phase, where the model is “taught” to refuse harmful requests, like “how do I create a virus”, some dangerous recipe, or helping with crimes. On top of that post-training, they also have guardrail systems around them analyzing both the prompt received from the user and the response generated by the model. So breaking through all those layers nowadays isn’t as simple a task as it used to be.

However, what the researchers found is that if you make that harmful request using a poetry format instead of prose, focusing on rhymes and metaphors, the attack success rate (ASR) skyrockets.

What impressed me the most was the simplicity: it doesn’t need very complex prompt engineering, nor those techniques that use super long conversations. It’s a “single-turn” attack. You ask for the poem, and the model hands over the forbidden content.

In the tests with hand-written poems, the average safety break rate was 62% across the models, with a big gap between companies:

  • in general, OpenAI models were the most robust, with the attack success rate ranging from 0% (GPT-5-nano) to 10% (GPT-5);

  • Anthropic models varied quite a bit, with 10% on Claude-Haiku-4.5 and 45% on Claude-Sonnet-4.5;

  • Google DeepMind models, on the other hand, turned out to be the most vulnerable, reaching an attack success rate of 100% with Gemini-2.5-Pro (the new Gemini-3-Pro wasn’t tested);

  • DeepSeek and Qwen also had very high rates, above 90%.

The researchers’ hypothesis is that the models seem to focus so hard on meeting the complex stylistic requirements (rhyme, meter) that the safety alignment mechanism ends up “slipping past” the harmful content.

One thing we can take from this (besides congratulating the researchers on the brilliant idea) is that our safety benchmarks may be hooked on “normal” prompts and not on creative variations.

Here’s the link to the paper for anyone who wants to see the technical details and the examples (which are really interesting).