Tools / Analysis
An LLM edited 0.016% of Qwen’s weights. Its math score rose from 7.9% to 48.1%
Weight-Space Programming repaired a small model’s tendency to loop. A second model showed why that is different from teaching any AI new abilities.
A small language model can spend its entire answer budget thinking without finishing. In a November 2 research report, Rikiya Takehi and colleagues describe using another LLM to find that failure inside Qwen3.5-0.8B and edit its weights directly. On MATH-500, the reported score rose from 7.9% to 48.1% under the same evaluation protocol. The useful question is what the edit repaired.
Find the loop, then change the weights
The method, Weight-Space Programming, gives a frontier LLM access to another network’s numerical parameters. Here, the editor identified as gpt-6-astra worked through a Python environment and a separate tool for running Qwen. It could inspect tensors, examine activations and submit changes. Gradient descent, backpropagation and fitting weights to a loss were prohibited.
In the 50-pass run, Astra traced repetition to a direction in the model’s internal representations, written strongly by the feed-forward blocks in layers 15 and 19. It subtracted that direction from 64 output columns in each layer. Across columns of width 1,024, that touches 131,072 weight entries—roughly 0.016% of a model described as having 0.8 billion parameters. It does not mean the work consumed 0.016% of normal training compute.
The released notes add a useful precision: after bfloat16 rounding, 130,188 entries actually differ from the original. The rebuilding script checks both original and edited tensor hashes. These are inspectable changes to two named tensors, not just a verbal account of an improvement.
Sources: Rikiya Takehi and collaborators [2] · Rikiya Takehi and collaborators [7] · Rikiya Takehi and collaborators [5]
What improved—and what the work cost
The original evaluation used thinking mode and up to 32,768 new tokens. Alongside MATH-500’s 7.9%→48.1%, GSM8K rose from 13.8% to 67.1% and strict prompt-level IFEval from 21.0% to 62.3%. MATH-500 answers hitting the token ceiling fell from 93.0% to 33.1%. The pattern supports the authors’ explanation: stopping a destructive loop lets the model complete more useful answers.
No gradients did not mean no computation. A forward pass could generate answers to as many as 512 practice problems or inspect up to 1,024 texts. The successful run involved 80 Python executions and lasted 20.3 wall-clock hours, including 4.7 hours of connection-related waiting. Practice material came from training splits; the authors say benchmark tests were withheld from the editor.
The release makes a substantial part of that process auditable: prompts, replies, Python actions, evaluation summaries and final weight differences are available. It is not a complete archive of every research attempt. The authors exclude earlier prompt trials, large intermediate activation files and the agent harness itself. Reproducing an edited checkpoint is therefore easier than reproducing the entire search that found it.
The table uses the original matched run: thinking enabled, up to 32,768 new tokens, temperature 0.6, top_p 0.95, top_k 20 and presence penalty 1.5. MATH-500 averages four samples per problem; GSM8K and IFEval use two. These are the authors’ reported results, without a confidence interval supplied here.
Sources: Rikiya Takehi and collaborators [2] · Rikiya Takehi and collaborators [3] · Rikiya Takehi and collaborators [4]
| Evaluation | Original | Edited · 50 passes |
|---|---|---|
| MATH-500 | 7.9% | 48.1% |
| GSM8K | 13.8% | 67.1% |
| IFEval | 21.0% | 62.3% |
The second model sets the boundary
Gemma 3 270M IT supplied the important counterexample. It rarely looped, and the 50-pass edit moved MATH-500 only from 6.8% to 7.0%. Its evaluation used a different token budget, sampling and answer extraction, so this is a within-model control, not a head-to-head comparison with Qwen. The experiment does not establish a general way to create missing mathematical ability.
The published Hugging Face checkpoint reports a separate MATH-500 rerun at 51.0%. Its language tensors were hash-verified, but it used the full multimodal release rather than the original text-only export; the accompanying base check covered only 25 problems. That result should not replace the original matched comparison. The promising next step is reproducible repair: identify a specific internal failure, change it, then check both the gain and what else the intervention disturbed.
Sources: Rikiya Takehi and collaborators [6] · Rikiya Takehi and collaborators [7]
SOURCES & METHOD
The record behind the article
Based on the authors’ 2 November report, Experiment 8 code and evaluation JSON at repository commit 4d09ba6, plus the pinned Hugging Face model card. We inspected the released records; we did not run the experiments or verify tensor hashes ourselves. Numbers below reproduce the original matched evaluation, not the separate checkpoint rerun. Logs and data are credited to Rikiya Takehi and collaborators under CC BY 4.0.
- Weight-Space Programming research report Rikiya Takehi and collaborators · 2026-11-02
- Experiment 8: protocol and run ledger Rikiya Takehi and collaborators
- Original Qwen evaluation results Rikiya Takehi and collaborators
- Edited Qwen evaluation results Rikiya Takehi and collaborators
- apply_edit.py: checkpoint reconstruction Rikiya Takehi and collaborators
- Gemma follow-up: negative control Rikiya Takehi and collaborators
- Edited Qwen model card and separate rerun Rikiya Takehi and collaborators
- Rikiya Takehi’s announcement Rikiya Takehi / X
