Anthropic Teaches How to Build AI Agents
Published on
Anthropic opened the hood and showed how they built the multi-agent system behind Claude’s “Research” feature, which lets Claude search the web, Google Workspace and other integrations.
They published an article (their articles are wonderful, as I always say on the IA Sob Controle podcast) detailing the architecture and the lessons learned, and it’s pure gold for anyone in the field.
The main conclusion was that, for open-ended and more complex searches, where the path isn’t linear, multiple agents working together are MUCH more efficient. A multi-agent system with a lead orchestrator agent (using Claude Opus 4) and several subagents (using Claude Sonnet 4) outperformed a single Claude Opus 4 by 90.2% on their internal evals.
The lead agent breaks the task down into subtasks, and the subagents (each with its own context and tools) carry them out in parallel, exploring different aspects of the search. They basically “distill” the key insights from a sea of information before sending the “prime cut” of what they found back to the lead agent.
They shared SEVERAL lessons learned, and I picked out some that I found the most interesting:
- Surprisingly (or not), 80% of the performance variation can be explained by token usage. More agents = more tokens = more parallel reasoning capacity. But… that costs A LOT (like, 15x more tokens than a normal chat). So the task has to have a value that justifies the cost
- To coordinate those agents, you need good Prompt Engineering, to teach the orchestrator agent to delegate subtasks well to the subagents (with clear goals, output format, tools), to scale the effort according to the task’s complexity, and even to let the agents improve their own prompts (they even created a “tool-testing agent” that, upon finding an MCP tool with a bad description, tried using it and then rewrote the description to avoid failures in the future. Pretty meta! Hahah).
- To evaluate the systems, Anthropic used a mix of evaluating from the start with small test samples, LLM-as-a-judge (using another model to evaluate factual accuracy, citations, source quality, etc), and of course, human evaluation (we always talk about the “human-in-the-loop”) to catch the “edge cases”.
- For deployment, you shouldn’t update all the agents to a new version at the same time. Instead, they used rainbow deploys to avoid interruptions in the agents’ execution, gradually shifting traffic from the old versions to the new ones, keeping both running simultaneously.
I thought the level of transparency from Anthropic was FANTASTIC, sharing not only the successes, but also the struggles and learnings. It’s WELL worth the read for anyone who’s building or thinking about building more complex AI systems with agents.
Here’s the link to the article.
