SEAL: AIs That Learn to Learn

Published on

What if LLMs could “learn to learn”? That’s the idea behind a new MIT paper called “Self-Adapting Language Models (SEAL)”, which we covered on the IA Sob Controle podcast a few weeks ago.

Today, LLMs are powerful, but static. They don’t permanently update their “knowledge” (the neural network’s weights) when they receive new information (you’ve probably seen a message like “My knowledge cutoff is June 2024”).

For those who aren’t in the field, a neural network’s weights are basically the LLM’s “brain”. They’re billions of numerical parameters that, together, make up all of the model’s knowledge: its grammar, the facts it “knows”, its writing style. Changing the network’s weights is, literally, changing what the model knows.

The analogy the paper’s own authors use is pretty cool: a student usually doesn’t learn new concepts just by reading a book. They take notes, make summaries, create diagrams, do exercises… in other words, they use the book’s information as a base, but RESTRUCTURE it to make learning easier.

This proposed SEAL framework tries to do exactly that with an LLM. Given a new piece of data (a text, for example), the model first generates a “self-edit” (a new version of that data, rewritten in a way that’s more optimized for itself to learn). It can be a list of implications, a summary, a question-and-answer format, etc.

And how does it learn to make good “self-edits”? With our beloved Reinforcement Learning (I explain this topic in my book).

Then, it takes those self-edits and fine-tunes (kind of a retraining) itself. The model is rewarded whenever a “self-edit” it generated results in an improved version of itself after fine-tuning on that new data. Basically, it learns to be its own teacher.

The most impressive part is the results. In the test of incorporating new knowledge, a Qwen2.5-7B model (that is, just 7 billion parameters, REALLY small) using SEAL reached 47.0% accuracy. The interesting part is that the same model, when trained with synthetic data generated by GPT-4.1 for the same task, got 46.3%.

Pause to think about that: a model learned to create training data FOR ITSELF that is BETTER than data generated by a MUCH bigger and more powerful model (GPT-4.1). It became its own teacher, and a better teacher than GPT-4.1!

Of course, it’s not a magic solution. The paper shows that the model still suffers from “catastrophic forgetting” (when learning something new makes it forget what it already knew), but it’s a very promising first step.

This idea may point to a future with models that can continuously refine themselves, maybe one of the paths to real “continual learning”. Imagine an AI that, after each interaction, doesn’t just respond, but learns and permanently improves itself?

Here’s the link to the paper.