RecursiveMAS: AI agents communicating by telepathy

Published on

Nowadays, when we build a multi-agent AI system, one of the main bottlenecks is the communication between the agents. And researchers from Stanford, MIT and NVIDIA want to change that.

Since our multi-agent systems are based on LLMs (which are the brain of each agent), what happens in practice is that the first agent generates a wall of text and hands it to the next one, which has to read everything, interpret it, and generate another wall of text as a response. It’s a slow process, hard to optimize, and it burns an absurd amount of tokens.

That’s where a sensational paper from these folks that just came out comes in, RecursiveMAS (Recursive Multi-Agent Systems), and their insight proposes changing the whole paradigm. Instead of communicating through text, the idea is that the agents connect and talk entirely in the latent space where they generate the probability logits to predict the next tokens.

RecursiveMAS: AI agents communicating by telepathy

Making an analogy (don’t kill me), it’s almost as if the AIs communicated by “telepathy”. They created a very lightweight module called RecursiveLink, which takes the continuous “thoughts” of one model (its hidden states) and passes them straight to the next one, in the form of embeddings.

That way, all the reasoning, collaboration and correction among all the AI agents involved in the system happen in a recursive loop right in there. Only in the last round does the final agent translate all of that and spit out the answer in text format for us humans.

In the paper, they show the results this technique had on math, code generation and medical question benchmarks, and they were VERY impressive:

  • Average increase of 8.3% in answer accuracy.

  • Faster inference, with the speedup ranging from 1.2x to 2.4x.

  • Huge reduction in token usage, saving between 34.6% and 75.6% of the cost.

And best of all: the approach is agnostic. They tested it on sequential flows, Mixture-of-Experts (MoE), with external tools (tool-calling) and even in model distillation, and the architecture worked super well in all of them.

It’s an idea I found pretty elegant, and one that makes total sense for the orchestration of the multi-agent systems we’re going to be building in the near future.

Here’s the link to the paper and to the project’s GitHub repository.