RLMs: A Glimpse into the Future of Software

A new pattern in programming is emerging.

The discourse right now is a scramble. In software, things are moving at light speed. Engineers are writing anxious LinkedIn posts about whether they'll have jobs in two years. Non-coders are delivering fully fledged applications in minutes. Framing this as "10x faster" or "10x fewer people" captures something, but it's like describing the invention of the printing press as "faster handwriting".

"Vibe coding" captured a moment, but it already feels stale. "Agents" is doing a lot of heavy lifting and means different things to everyone who uses it. And the perennial favorite "this will replace software engineers" mistakes a change in what programming is for a change in who does it.

Programming is moving to its next abstraction layer.

We are seeing a genuine shift in the nature of the work itself. The same kind of shift that happened when we moved from assembly to high-level languages, or from raw code to frameworks. The kind of shift where, once you see it, the old way of framing things stops making sense. During the phase change, completely new paradigms emerge.

This blog is about the transition.

In this first post I will be looking at a recent paper from MIT, Recursive Language Models, which formalizes one early instance of the shift. It's a good place to start because the results are awesome and the mechanism is clear. But the pattern it points toward is far bigger than long-context processing, and it's already showing up everywhere.

Let's start with the paper, then zoom out.

What are RLMs?

Recursive Language Models, by Alex Zhang and Omar Khattab at MIT, tackles a specific problem. Long-context processing.

Traditionally, when an LLM receives a prompt, the entire text gets flattened into a sequence of tokens and shoved through the neural network. The model has to hold everything in its "working memory" (the context window) simultaneously. This is like asking someone to solve a problem by reading an entire textbook at once, with no ability to flip back to a page, take notes, or use a calculator.

RLMs invert this completely. The prompt is not given to the neural network. Instead, it's stored as a variable in a Python REPL (similar to a Jupyter notebook) and the model is simply told: "there's a string called context in your environment. It's 10 million tokens long. Here's the query. Figure it out."

The model then does what any competent programmer would do. It peeks at the data. It greps for patterns. It chunks the data up. It writes loops. And critically, it calls itself or other LLMs as a function on subsets of the data.

The prompt goes from being input to the neural network to being data in a program that the neural network writes.

The Abstraction Layer RLMs Point Toward

Consider the history of abstraction in programming:

1. Machine code — manipulating registers and memory addresses directly
2. Assembly — symbolic names for operations, still hardware-bound
3. High-level languages — expressing intent ("sort this list"), the compiler handles the how
4. Frameworks and libraries — composing pre-built abstractions
5. LLM-assisted coding — describing what you want in natural language, the LLM writes code

RLMs add a sixth layer: the LLM writes code that calls LLMs that write code. This is the pattern, the general form. In the specific case of RLMs, it's applied to context decomposition. But the principle is universal. The "programmer" is no longer a human writing Python. It's an LLM deciding at runtime how to decompose a problem, writing the decomposition strategy as code, executing that code, observing the results, and iterating.

The LLM is simultaneously the programmer, the program, and the interpreter.

The boundary between "the model reasoning" and "the model programming" dissolves. A sub-LM call is just a function call. A regex filter is just a function call. The model decides which tool (itself, code, grep, web search etc.) is appropriate for each sub-problem, in code, at runtime.

Code and LLMs are falling back on each other. The LLM writes code to manipulate data. The code calls the LLM to understand data. The LLM reads the results and writes more code. This loop, i.e. this mutual recursion between symbolic computation and neural reasoning, is the actual core of what RLMs are.

Why This Works

When 200K tokens get dumped into a transformer, the attention mechanism has to spread across all of it. Information gets diluted. The model gets worse. This is because each unit of attention is spread thinner and thinner. It's like trying to stare at a 20 meter mural with your nose touching the wall.

RLMs solve this by ensuring the model never sees the full context at all. The root LM's context window stays clean and small. It only ever contains the query, its own code, and small snippets of output from code execution. When it needs semantic understanding of a chunk, it dispatches a sub-LM call on just that chunk. Each sub-LM also gets a clean, small context window. No single neural network call is ever overwhelmed.

Outside of LLMs this is called the out-of-core algorithm. A computer with 16GB of RAM can process a 1TB dataset by being clever about what it loads into memory and when. An LLM with a 128K context window can process 10M tokens by being clever about what it attends to and when.

Emergent Behavior

What elevates this from a clever engineering trick to something genuinely exciting is that nobody told the model how to do any of this. The system prompt just says: here's a REPL, here's your context as a variable, here are the functions you can call. The model then independently discovers strategies that any experienced programmer would recognize:

Peeking. At the start, the model knows nothing about the context except its size. So it looks at the first N characters to understand the data structure before committing to a strategy.

Grepping. Rather than reading everything, the model uses regex to narrow the search space before applying expensive semantic reasoning.

Map-reduce. The model chunks the data, dispatches sub-LM calls to each chunk (often in parallel), and aggregates the results. It independently reinvents distributed computing patterns.

Verification. The model uses sub-LM calls on small context windows to double-check answers it's already found — implicitly avoiding context rot by verifying in clean conditions.

Programmatic one-shotting. For tasks like applying git diffs, the model just writes the code to do it rather than trying to "reason" through it token by token. It recognizes when a problem is computational rather than semantic and acts accordingly.

These patterns emerge from first principles because they're the natural strategies for dealing with data that's too big to hold in your head at once. The model discovers them because they work. The examples above are from the paper. The more important framing is that these are the first things that happened, with models that were never trained to operate this way, at a recursion depth of just one.

It is entirely reasonable to expect that as models improve and recursion deepens, far more sophisticated strategies will emerge.

RLMs Are the Tip of the Iceberg

The general pattern is: LLMs become callable functions inside programs that LLMs write. RLMs apply this to context decomposition. But this pattern is going to drive a number of step changes across the field.

Entire complex codebases held in context and reasoned over as data. Models that write their own tools rather than choosing from a menu. Verification loops where the model tests its own answers in code before returning them. Self-optimising pipelines that rewrite themselves based on measured performance. And parallelism.

When models learn to fan out hundreds of sub-calls across a computation graph they generate at runtime, the inference compute demands will be enormous. But so will the value per query.

RLM's contribution demonstrates that when you formalise this pattern, and let the model control the decomposition, the results dramatically outperform human-engineered alternatives. I believe that finding will generalise.

Where This Is Going

Anyone who's spent time in machine learning will recognise the pattern in the RLM paper's closing line: "Agents are designed based on human intuition on how to break down a problem to be digestible for an LM. RLMs are designed based on the principle that fundamentally, LMs should decide how to break down a problem to be digestible for an LM."

This will ring true if you've ever over-engineered a feature set. One of the first things you learn in ML, usually the hard way, is that your human intuition about what matters is often wrong, and the model you were trying to help would have done better if you'd got out of the way. The same principle applies here, just one level up. Hand-crafted agent pipelines, human-designed retrieval chains, manually engineered decomposition strategies… they "feel" like good architecture. But the RLM results suggest that when you let the model control the decomposition, it massively outperforms the human-designed version.

This shouldn't be surprising. It's the same lesson the field keeps learning at every level of abstraction. Stop telling the model what to pay attention to. Give it the right environment and let it figure it out.

The real bet is larger than RLMs. The bet is that the code-LLM loop, models writing code that calls models, with each side compensating for the other's weaknesses, is not a trick or a scaffold. It's the natural next layer of computing abstraction.

People who build with LLMs are already feeling this. The patterns are emerging before the formalisms. RLMs are just the first case where someone ran controlled experiments and showed the results are dramatic. More formalisms will follow. More papers will name more specific instances of this loop. And each time, the finding will be the same: let the model control the decomposition, give it code as a medium, give it itself as a callable function, and the results will be better than anything a human engineers by hand.

This has profound implications for where AI is about to go.