Fast and Furious: Diffusion Models Generating Text
Published on
Remember the post I wrote a while back about diffusion models being used to generate text (and not just images, as was the standard)? This week, that got a really interesting new chapter.
Inception launched its first general chat model, Mercury. Before that, they had already released the code-specialized versions (Mercury Coder), but this new version works like a regular LLM, just MUCH, but MUCH faster.
To explain better, the LLMs we’re used to (ChatGPT, Gemini, Claude, etc) are autoregressive models. That means they generate text sequentially, word by word (token by token, actually). The model generates the token “The”, then “cat”, then “sat”, then “on”, then “the bed”, and so on, always based on the previous ones. It’s a queue process.
Diffusion models like Mercury, on the other hand, work differently. Thinking about image generation, they start with noise (like TV static) and iterate on that static until it turns into the image of the “cyberpunk winged unicorn” you asked Midjourney to generate. Since they’re so well established for image generation, using them to generate text was/is pretty surprising, but it works in a similar way: Mercury starts with a “draft” of the whole answer, full of noise, wrong data or blank spaces, and refines everything at once (or in very few steps). It’s this ability to generate the answer in parallel, and not word by word, that guarantees the absurd speed.
And “fast” here is no small thing. According to Artificial Analysis, this new Mercury can deliver performance at the level of a Claude 3.5 Haiku or GPT-4.1 Nano (smaller, really fast models), only with 7 TIMES the speed, at 708 tokens/second. It’s truly absurd to see this in practice (I’ll leave the link for you).
That super low latency is going to be extremely useful in tasks that need instant answers, like voice agents (imagine an AI call center attendant having to think and respond fast), real-time voice translation, among others.
It’s really interesting to see a strong bet on an architecture different from the one that dominates the market. While most companies iterate on autoregressive models, Inception chose to follow a parallel path, with diffusion, and they may reap great rewards from it. The great Yann LeCun has been saying for a while that we won’t get to the supposed AGI with autoregressive LLMs. Could diffusion be the next step on that path? We’ll see.
For those who got curious and want to try it, the model is already available via OpenRouter, Poe, Inception’s API, and their Playground. Check out how ABSURD the generation speed is!
Link to Inception’s Playground
Link to the post about Mercury’s launch
