Deep Research with Diffusion?

Published on

Haven’t I been saying that AI models using diffusion are the new trend?

After the launch of Google DeepMind’s “Gemini Diffusion” (we even interviewed on IA Sob Controle a Brazilian who worked on creating this model:

and Inception Labs’ Mercury, the folks at Google dropped a new paper called “Deep Researcher with Test-Time Diffusion”.

Many of today’s research agents (the famous and super useful Deep Research from ChatGPT, Gemini, Grok, Perplexity, etc.) work in a practically linear way: they plan, run a series of searches and, in the end, put everything together in a report. The problem is that in this process, a lot of context can get lost and the final result can, in some cases, end up losing a bit of cohesion.

In this paper, they tried to create this model inspired by how we write. Think about it: we don’t write a perfect text from start to finish. First, we create an initial, imperfect draft. Then, we review that draft, look for more information to fill the gaps, edit, and repeat that cycle several times.

Their technique does exactly that. It treats report generation as a diffusion process, a technique used A LOT in image generation models, and that has recently been gaining strength with text generation models too. Thinking about image generation, diffusion starts with noise (like TV static), and iterates on that static until it turns into the image of the “cyberpunk winged unicorn” you asked for.

The process they proposed in the paper has two really cool central ideas:

  • “Denoising with Retrieval”: the agent creates an initial, noisy draft of the deep research and uses it as a guide for the next searches. With each new piece of information found on the web, it “cleans the noise” (denoises) from the draft, making it better, more complete and more coherent. It’s a continuous feedback cycle between writing and research.

  • “Self-Evolution”: besides the main process, each component of the agent (the planner, the query generator, etc.) also goes through an optimization process to improve the quality of its outputs.

And of course, being a diffusion model, it does all of this at inference time.

In the paper’s benchmarks, the model beat the main Deep Research products we know (OpenAI, Perplexity, Grok) by A LOT, both for generating long reports and for answering complex questions that require multiple reasoning steps.

I found the approach super interesting, mainly because of the analogy with the human writing process, which has a linear side, but also this naturally iterative side (in fact, this post went through that iterative process of writing, reviewing, going back, etc. etc.). It’s a step beyond traditional RAG.

Here’s the link to the paper.