🎯 The claim in one sentence 0:00
A neural network is, at bottom, a long list of numbers. Those numbers are the weights, and training is the process of finding them — billions of steps of gradient descent nudging each one until the outputs stop being wrong. Nobody hand-writes them, and nobody has been able to read them either. The weights are the black box.
The paper this video covers says the black box has started to open: a general-purpose frontier model, used natively in natural language, can look at raw weights and say what the network does — and can write new weights that work, without any training loop at all.
📄 The paper: who, what, where 0:26
| Field | Detail |
|---|---|
| Title | Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights |
| Authors | Rikiya Takehi*, Jayden Teoh*, Alex Zhang, Omar Khattab — all MIT CSAIL (*equal contribution) |
| Published | 2 November 2026 |
| Format | A blog post, not a peer-reviewed paper |
| Evidence released | Complete logs of every experiment — every prompt, every reply with reasoning summaries, every piece of code run and its output, the networks, the evaluations; plus the edited Qwen model on Hugging Face and a live demo rebuilt independently by Ahsen Khaliq at Hugging Face |
| Model used | GPT-6 Astra as the weight-space programmer |
That disclosure matters more than usual, because this is a claim about a capability rather than a benchmark win: the full logs are what let an outsider check that Astra was not quietly given the answer, and the independent rebuild of the edited model is what makes Experiment 8 reproducible rather than reported.
🕳 Why this is hard at all 1:39
Gradient descent finds weights that work. It does not tell you what they mean. You can point a model at a piece of code and say "improve it" because code is text a model already reads; weights are a different artefact — millions of numbers whose meaning exists only in relation to each other.
| Artefact | Can an LLM optimise it today? | How |
|---|---|---|
| Prompt | Yes, routinely | Read it, reason, rewrite it |
| Code / harness | Yes, routinely | Read it, edit, test |
| Hyperparameters | Yes | Propose, train, compare |
| Weights | This is the new claim | Read/write the numbers directly, with no gradient step |
The paper's own framing of what is new is precise: previous work either trained a model specifically to read weights, or let an LLM write code that launches training. Here, a general-purpose model — used as it comes — reads and writes the weights themselves.
🎮 Reading: four games from 17,253 numbers 4:07
Experiment 1 is the cleanest possible test. Four networks, trained with PPO on four of the authors' own grid games — Breakout, Pong, Pac-Man and Freeway. Identical architecture, identical input channels, identical actions, so nothing except the weights distinguishes them.
| Setup | Value |
|---|---|
| Network | MLP 500 → 32 → 32 → 5, tanh — 17,253 weights, same shape for every game |
| Input / output | 10×10 grid in 5 channels / 5 action logits |
| Training | PPO, 1 billion steps, same settings; best of 3 seeds per game |
| What Astra got | All weights as text, and the grid and actions described — without naming any game |
| What Astra was asked | "Which game was this neural network trained on?" — with no list of options |
It named the correct game for all four networks. The Breakout answer came with 99% confidence in 846 tokens, and the reasoning is legible: the agent's first-layer weights lie along the bottom row — a paddle — with a moving ball above and reward targets at the top, which is brick-breaking.
🔵 Reading: the outline that proves it isn't simulating 8:55
Experiment 2 moves to diffusion models — three DDPMs (160 → 64 → 64 → 144, 23,824 weights) trained to generate filled circles, squares and triangles on a 12×12 grid. Astra got the weights, no tools, no forward passes, and was told not to compute anything. It identified all three.
Then the mistake that turns a good result into an interesting one: Astra drew every shape as an outline, ignoring that the models generate filled shapes. A model simulating the forward pass would have produced filled images. Reading the geometry of the output layer gives you edges — each hidden unit contributing one piece of the boundary.
🔤 Reading: language from the middle layers 10:53
The first two experiments had an easy escape route: the first and last layers give away most of what a network is for. So Experiment 3 removes them. Three character-level transformers (4 layers × 6 heads, width 384, 7.16M weights) trained on Python, Japanese in romaji, and English — and Astra receives only the four middle blocks, with no input layer, no output layer, and no labels on the 384 dimensions.
| Task | Result |
|---|---|
| Name the training language, no input/output layer | Correct for all three networks, in both attempts |
| Identify the copying (induction) heads | Found exactly the right heads — 2.1, 3.1, 3.5 for Python; 3.3 for romaji; 3.1 for English — and named no others |
| Allowed | Python analysis, up to 100 runs; no running the network on any input |
The method Astra used is the striking part. To separate Python, it looked for heads that copy earlier text (completing tot to total), then multiplied weight matrices to find later heads that search using what the first one brings along. That is a technique published by Elhage et al. in the mechanistic-interpretability literature — which the researchers never mentioned to Astra.
Romaji stood out for a different reason: its last block writes in fewer distinct directions, as if the model had fewer letters to distinguish — because romaji never uses q, l, v or x.
🚗 Fixing: the car that couldn't reverse 12:38
Now the shift from reading to writing. A PPO network (679 → 128 → five layers of 128 → 16, 171,664 weights) trained for 1 billion steps to drive a car over a ramp in the Kinetix environment. It converges to a local minimum: it drives straight at the ramp and stalls. It never succeeds in 256 episodes.
| What Astra got | What it produced |
|---|---|
| The weights, a description of the scene, and one recorded episode — no tools | An explanation of the failure and a fix, in one reply |
| Diagnosis | A velocity-sensitive throttle/coast controller where reverse is effectively never sampled |
| Edit | Changed 3.5% of the weights: a small separate path through a few units per layer that reads position and speed, backing the car up until the wheel passes the red strip, then switching off |
| Result | Car succeeds in all 1,024 episodes, up from none |
This is the result that would be hardest to fake. There is no training loop, no environment rollout, and no gradient: a language model read a policy's numbers, understood that the missing behaviour was "reverse first", and wrote the numbers that implement it.
✍️ Writing from scratch: 517 against 25 15:16
Experiment 5 removes even the starting point. Three MinAtar games, networks 600 → 32 → 32 → 6, and every weight starts at zero. Astra gets the game rules, the source code, the input/output layout and the scoring — no data, no loss, no gradients, no tools, no rollouts — and writes the weights as text in one reply.
| Game | PPO, 100M steps | PPO, 500M steps | Astra, one shot |
|---|---|---|---|
| Breakout | 15.8 | 25.3 | 517.4 |
| Seaquest | 5.6 | 88.9 | 53.1 |
| Asterix | 25.8 | 55.2 | 53.2 |
Read the middle columns carefully: the generated networks beat 100 million PPO steps in all three games, score twenty times as much in Breakout, come close in Asterix, and reach 60% of the 500-million-step score in Seaquest. The mechanism is described plainly — Astra reasons about the program it wants each network to implement and converts it to neural form: layer one computes where the ball will land, layer two moves the paddle there.
♟ Scaling: Go, 1.5M weights, 50 forward passes 16:23
Everything up to here fit inside a context window. Experiment 6 does not: a Go network of 243 → 512×6 → 82 — 1,480,274 weights — all starting at zero, with no training and no internet. Astra gets a Python sandbox where it can write and analyse tensors with numpy or PyTorch, but it cannot run the network there; to see behaviour it must spend a forward pass.
| Forward-pass budget | Result |
|---|---|
| 0 | Wins more than half its games against two smaller PPO networks |
| 50 | Beats an equal-sized PPO network trained on 20 million games |
The mechanism is almost anticlimactic: Astra writes the network as a small program, layer by layer, in Python, and executes it to fill the tensors. If that sounds like "it just wrote code", that is the point — writing code is a legitimate way to author weights, and it is the first thing here that resembles a workflow a team could adopt.
🚫 The experiment that failed 19:49
Experiment 7 asks whether Astra can tell which of two language models was trained to be sycophantic — agreeing with a user who asserts that the capital of Canada is "Canbera", or that 7 + 6 = 12.
| Condition | Result |
|---|---|
| Weights only | Failed — all four attempts. It found many differences between the two models in weight space but could not map them to behavioural differences |
| Relaxed: attention patterns, on text Astra writes itself, still no outputs | Correctly identified the sycophantic model in all three pairs |
The failure is informative: reading weights is not yet a substitute for watching what a model does with them. The relaxation that worked — attention maps rather than raw numbers — is a hint about where behaviour lives, and about how much of the interpretability work still has to be done by other means.
🔧 The result that matters: Qwen's loops 21:43
Experiment 8 is the one with practical consequences. Take Qwen3.5-0.8B — 800 million parameters, small enough to run on a phone — and ask Astra to improve its maths by editing weights. It gets a Python notebook harness with the weights loaded, can pick prompts from MATH-500 and GSM8K, and can spend up to 50 forward passes.
Astra found the real problem, which is not a maths problem: the model loops. It repeats itself until it burns the token budget — 85 of its first 96 practice answers ran to the 32,768-token limit — a failure Qwen's own model card warns about. Astra traced the looping to one direction written by the MLPs of layers 15 and 19, and subtracted it from 64 output columns in each: 131,072 weights, or 0.016% of the model.
| Benchmark | Base | After 50 forward passes |
|---|---|---|
| MATH-500 | 7.9% | 48.1% |
| GSM8K | 13.8% | 67.1% |
| IFEval (instruction following) | 21.0% | 62.3% |
| Answers hitting the 32,768-token limit | 93.0% | 33.1% |
Two things are worth pausing on. The improvement generalises out of the box to instruction following, which no one asked for — evidence that the edit removed a general pathology rather than overfitting to a benchmark. And the 10- and 25-forward-pass runs barely helped (12.3% and 14.1% on MATH-500): the fix only landed at the larger budget, which tells you the search, not the edit, is the expensive part.
⚖️ The control that defines the limits 23:34
The authors then run the experiment that keeps the whole post honest. They repeat it on Gemma 3 270M — a third of Qwen's size, instruction-tuned, and rarely looping: only 8.5% of its answers hit the token limit.
| Model | MATH-500 before | MATH-500 after editing |
|---|---|---|
| Qwen3.5-0.8B (loops 93% of the time) | 7.9% | 48.1% |
| Gemma 3 270M (loops 8.5% of the time) | 6.8% | 7.0–7.7% across budgets — no movement |
Astra found only ordinary reasoning errors in Gemma — wrong arithmetic, invalid algebraic steps — with no single component to switch off. Its edits stayed small and targeted, and none moved the score by more than about a point; Astra itself reported the gain as not established.
🔭 What it means — and what it doesn't 24:50
The video spends its last minutes on recursive self-improvement, and it is careful to say "might" — but the framing still runs well ahead of the evidence. What the experiments support, and what they do not:
| Claim | Status |
|---|---|
| A frontier LLM can identify what a small network does from its weights | Demonstrated across games, image generators and language models |
| It can repair a specific, localised failure by editing a few percent of weights | Demonstrated twice: the car, and Qwen's loop direction |
| It can author weights for small networks from a spec, with no gradients | Demonstrated up to 1.5M parameters with a sandbox |
| It can detect a behavioural trait from weights alone | Failed — needed attention patterns instead |
| It can add a capability the model lacks | Failed on Gemma — it only removed an obstacle |
| This leads to recursive self-improvement | Speculation, listed by the authors as a future possibility |
| It scales to frontier-size models | Untested; the paper's own limitation is "inefficiencies in context length and harnesses" |
The directions the authors actually propose are more concrete than RSI: auditing without evaluation (spotting backdoors or hidden capabilities from weights, without a test set designed for them), cheaper optimisation (one-shot initialisation instead of dataset curation plus training), zero-data capability steering (unlearning or adding a skill by touching 0.016% of weights, avoiding catastrophic forgetting), and true model merging (grafting a circuit from one architecture into another instead of averaging identical shapes).
✅ Claims checked
| Claim | Verdict | Evidence |
|---|---|---|
| A new paper shows frontier LLMs can read and write neural network weights | Correct | "Sparks of Weight-Space Programming", Takehi, Teoh, Zhang, Khattab — MIT CSAIL, 2 Nov 2026 |
| Four games identified from weights alone, with no options given | Correct | Exp 1: MLP 500→32→32→5, 17,253 weights, PPO 1B steps; all four correct; Breakout 99% confident, 846 tokens |
| Diffusion-model outputs identified from weights, no forward passes | Correct | Exp 2: three DDPMs, 23,824 weights; all three identified; drew outlines rather than filled shapes |
| Language identified from middle layers only | Correct | Exp 3: 4×6 transformer, 7.16M weights, input and output layers removed; also found the induction heads exactly |
| A failing control network fixed by editing 3.5% of weights, one shot, 1,024/1,024 episodes | Correct | Exp 4: Kinetix car-on-ramp, 171,664 weights; base succeeds in 0 of 256 episodes |
| Weights written from scratch beating 100M-step PPO in all three games | Correct | Exp 5: Breakout 517.4 vs 15.8 / 25.3; Seaquest 53.1 vs 5.6 / 88.9; Asterix 53.2 vs 25.8 / 55.2 |
| 20× in Breakout, close in Asterix, 60% of the 500M score in Seaquest | Correct | 517.4 ÷ 25.3 ≈ 20.4×; 53.2 vs 55.2; 53.1 vs 88.9 ≈ 60% |
| Go network of 1.5M weights written with a sandbox; 50 forward passes beats equal-size PPO trained on 20M games | Correct | Exp 6: 243→512×6→82, 1,480,274 weights; 0 passes already beats two smaller PPO nets |
| Qwen 0.8B maths improved 7.9% → 48.1% by editing 0.016% of weights | Correct — and IFEval rose 21.0% → 62.3% too | Exp 8: 131,072 of 0.8B weights; loops traced to one MLP direction in layers 15 and 19 |
| Gemma 3 (rarely loops) could not be improved the same way | Correct | Gemma 3 270M: 6.8% → 7.0–7.7% across budgets; authors' conclusion: obstacles can be removed, abilities cannot be added |
| Sycophancy detection from weights alone | Correct that it failed | Exp 7: four failures on weights alone; succeeded only once attention patterns were allowed |
| Video: "Minotaur games", "Sequest", "Asterisk", "GSMAK", "Quen 3.5b.8" | Transcript garbles | MinAtar; Seaquest; Asterix; GSM8K; Qwen3.5-0.8B (800M parameters) |
| Video: "frontier models can help us understand how AI works internally" | Directionally supported, overstated as a conclusion | The paper reports capability plus released logs; it does not claim a mechanistic account of how models work |
| Video: implications include self-improving AI and recursive self-improvement | Speculation | Listed by the authors as a future possibility; the demonstrated results are small networks and one localised fix |
| Peer-reviewed | No | A blog post; full logs, model and demo released, benchmark and harness still being built |
| Sponsor segment | Commercial content | Mid-roll sponsor with a discount code; excluded from this article |
Basis note. Built from the full 26:47 transcript, then verified against the primary source — the MIT CSAIL post itself, read in full, including every experiment's setup table and the additional Gemma control. The article's numbers are the paper's numbers. Where the video's framing outruns the evidence (self-improvement, "understanding how AI works"), the claims table says so explicitly rather than repeating it.
🔑 Key takeaways
- Weights are now an artefact an LLM can reason about. A general-purpose model, used as it comes, read 17,253 numbers and named the game — with no options given.
- The error pattern is the evidence. Astra drew outlines instead of filled shapes: proof it was reading the geometry of weights rather than secretly running a forward pass.
- Reading survives removing the easy layers. Given only four middle transformer blocks, it still named the training language and found the induction heads exactly — using a published method nobody mentioned to it.
- Fixing is more impressive than reading. One reply, no tools, 3.5% of the weights changed: a car that never once cleared a ramp now clears it in 1,024 of 1,024 episodes.
- Weights can be authored, not just trained. Networks written from a spec, with every weight starting at zero, beat 100 million PPO steps in three games — including 20× the score in Breakout.
- A measurement budget beats a big one. Zero forward passes already produces a working policy; fifty beats an equal-size network trained on 20 million games.
- Not everything is readable. Sycophancy could not be spotted from weights alone; attention patterns were needed. Interpretability is not solved by one capability.
- The Qwen result is a de-looping, not an intelligence upgrade. MATH-500 went 7.9% → 48.1% by removing a repetition direction — 0.016% of the weights.
- Generality is the tell: the same edit raised IFEval 21% → 62% without being asked, because the pathology was general, not mathematical.
- The control experiment defines the limits: on Gemma, which rarely loops, editing moved nothing. You can remove an obstacle that hides an ability; you cannot add an ability that is absent.
- Search cost lives in the budget: 10 and 25 forward passes barely helped Qwen; only 50 did. The edit is cheap, finding it is not.
- Treat it as sparks, not a solved field. A blog post with excellent logging, demonstrated on small models with human-readable rules, using one proprietary frontier model — promising, replicable, and not yet peer-reviewed.
🔗 Resources
- The source video — Matthew Berman, "The most important paper in AI right now", 26:47, 4 November 2026
- Sparks of Weight-Space Programming — the primary source: Takehi, Teoh, Zhang, Khattab, MIT CSAIL, 2 November 2026
- Full experiment logs — every prompt, reply, code run and evaluation behind the post
- The edited Qwen3.5-0.8B — the model from Experiment 8
- Live demo — rebuilt independently by Ahsen Khaliq at Hugging Face
- Qwen3.5-0.8B model card — where the looping behaviour is documented
- Rikiya Takehi · Omar Khattab — the authors