🔍 Read the full analysis: Neural Network Weights As A Programming Interface For AI on ThorstenMeyerAI.com
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
TL;DR
Four MIT CSAIL researchers report that OpenAI’s GPT-6 Astra could identify, repair and create small neural networks by working directly with their weights. The experiments include a car controller fix, a language-model maths improvement and a Go network, but the findings come from a non-peer-reviewed blog post and have important limits.
Four researchers affiliated with MIT CSAIL report that OpenAI’s GPT-6 Astra could inspect and modify neural-network weights directly, identifying what small models did, fixing some failures and writing new networks. Their November 2 blog post describes the approach as weight-space programming; the results are logged publicly, but have not been peer reviewed and remain limited to a set of small experiments.
The researchers gave Astra the numerical weights of networks and asked it to infer their functions or produce changes. Unlike conventional training, this approach did not use an optimizer or gradient descent to adjust the weights. In some tests, Astra received weights in text form and answered in one reply. In others, code it wrote was run in a sandbox with a fixed test budget. The researchers say the model had no shell, web search or file-editing tools, and that they checked all 1,125 turns in the released logs for tool use.
Among the reported reading tests, Astra identified four small game-playing networks from their weights and named the underlying shapes for three small image generators. It also found the researchers’ designated copying, or induction, heads in three tiny language models. The source account notes a limitation in interpreting these demonstrations: software engineer Sean Goedecke said he could reproduce one of the visual patterns used to identify a game with a short Python script. The reported successes show that weight patterns can carry recognizable structure, but do not by themselves establish how much reasoning was needed.
The repair tests produced some of the strongest numerical results. A car-control network that recorded zero successes in 256 episodes before editing completed 1,024 of 1,024 episodes after Astra changed 3.5% of its weights. For Qwen3.5-0.8B, a change to 0.016% of weights accompanied a rise in the reported MATH-500 score from 7.9% to 48.1%. A Hugging Face engineer later rebuilt the edited Qwen model from the released logs and reported a 51.0% score on MATH-500. These are results from the described runs, not evidence that the same edits will work across models or evaluations.
Astra also generated small game networks from all-zero weights and produced a 1.5-million-weight 9×9 Go network. The source reports that, across 50 test runs, the Go network beat a same-size comparison model trained on about 20.5 million games. The post also records failures: Astra could not distinguish an honest model from a sycophantic one using weights alone, did not improve maths performance when editing Gemma 3 270M, and the authors say edits could not supply an ability a model did not already have.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
What Direct Weight Editing Could Change
If the approach holds up in independent tests, it could offer researchers and model developers another way to audit or repair small open models. Instead of retraining a network, a developer might ask a capable language model to inspect a failure and make a targeted change. The reported Qwen result, where a small fraction of weights was altered, is a concrete example of why that possibility is attracting attention.
The work also points to a potential change in how neural networks are treated: not only as systems trained by optimization, but as objects that another model can inspect and edit. That remains a research possibility, not a routine production method. The experiments are narrow, and a model’s weights do not necessarily reveal its behavior in a reliable or complete way. The inability to identify sycophancy from weights alone is a direct warning against treating weight inspection as a substitute for behavioral testing.
For organizations, the work may increase interest in the provenance and integrity of model weights, particularly if direct editing becomes more capable. For users, the report describes no immediate product change. It does not establish that deployed models can be safely or consistently repaired this way.
Popular right now
De'Longhi Rivelia Fully Automatic Espresso Machine with Milk Frother, White
TCL 65-Inch T7 Series 4K QLED Smart Google TV, 144Hz (65T7)
Shark AI Ultra Robot Vacuum, Black/Silver, AV2511AE
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, computer scientist Andrej Karpathy described neural networks as “Software 2.0”: programs whose behavior is encoded in weights produced by optimization rather than written line by line. He also observed that directly coding millions of weights is difficult. The MIT researchers’ work revisits that idea by asking whether a language model can reason about those numbers and edit them directly.
The researchers are Rikiya Takehi and Jayden Teoh, who made equal contributions, along with Alex Zhang and Omar Khattab. Their blog post, published November 2, 2026, presents the experiments and their logs. The source material says it is not a peer-reviewed paper and has not been posted on arXiv. It describes one run per setting, small networks and a single model family for each task, which limits how broadly the results can be generalized.
“Software 2.0”
— Andrej Karpathy, describing neural networks in 2017
How Far the Results Generalize
The evidence remains early and limited. The work is a blog-post report rather than a peer-reviewed paper, and the source describes one run per setting, small networks and only one model family per task. It is not yet clear whether the results will replicate across independent teams, larger models, different model architectures or repeated trials. The post’s logs allow inspection of the reported runs, but do not resolve those broader questions.
There are also open questions about what drives the results: whether Astra identified general, reusable structure in weights or relied on patterns specific to these tasks; how reliable its edits would be outside the reported benchmarks; and what safeguards would be needed before editing weights used in real systems. The available account does not establish that weight-only inspection can reliably identify deceptive or sycophantic behavior.
Independent Tests and Replication
The next step is independent replication across more tasks, model families and repeated trials, with clear reporting of test conditions and failures. Further evaluation would also need to check whether weight edits hold up on new data and whether improvements persist beyond the particular benchmarks described in the post.
The researchers have released logs that let others examine the experiments, and the reported Qwen reconstruction by a Hugging Face engineer offers one initial check on reproducibility. The source material does not specify a peer-reviewed publication or a timeline for a formal paper. Until more evidence is available, weight-space programming should be treated as an experimental research technique rather than a dependable model-maintenance tool.
Key Questions
What is weight-space programming?
It is an approach in which a language model reads or changes another neural network’s weights directly, rather than using a conventional optimizer such as gradient descent.
What did GPT-6 Astra do in the reported experiments?
The researchers report that Astra identified small networks, repaired some failures and wrote small networks from scratch, including a 1.5-million-weight Go model.
Was the Qwen maths improvement independently checked?
The source account says a Hugging Face engineer rebuilt the edited Qwen model from the released logs and reported a 51.0% MATH-500 score. That is an additional reported check, not a broad replication across models and settings.
Has the work been peer reviewed?
No. The source describes it as a blog post with public logs, not a peer-reviewed paper, and says it has not been posted on arXiv.
Can weight editing give a model abilities it does not have?
The researchers say their edits could not add an ability the model lacked. Their report does not show that direct editing can reliably create new capabilities or repair every kind of failure.
Source: ThorstenMeyerAI.com
