🔍 Read the full analysis: GPT-6 Astra’s Single Matrix Edit Improved Qwen’s JSON Results on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
An internal experiment by OpenAI’s GPT-6 Astra, running as a Codex agent, used a rank-one edit to one matrix in Qwen3-4B-Instruct-2507. On a frozen test of synthetic text-to-JSON cases, exact answers rose from 49 of 120 to 70; the edit did not improve a separate table-extraction task, and independent review is still pending.
OpenAI’s GPT-6 Astra, acting as a Codex agent, reported that a rank-one edit to one matrix in Qwen3-4B-Instruct-2507 raised exact answers on a frozen text-to-JSON test from 49 of 120 to 70 of 120. The result is a measured gain on a narrow synthetic task, not evidence of a general improvement in the model: 50 final cases remained wrong, and a separate table-extraction task stayed at zero of 30.
The experiment tested whether Qwen could copy short texts containing tabs, line breaks, quotation marks or backslashes into JSON without changing the original text. A response counted as correct only if valid JSON could be parsed and the decoded field matched the source text exactly. For example, a tab must be represented in JSON as \t and recover as the original tab after decoding.
According to the experiment report, the edited model produced 70 exact answers out of 120, compared with 49 for the original. The report describes that as a gain of 17.5 percentage points, with 21 cases fixed and none of the previously correct cases broken. The test was frozen before the result was measured, according to the source account.
GPT-6 Astra selected and calculated a rank-one update to the output projection of the model’s final multilayer perceptron layer. The report says no gradient training was used and the other 397 stored tensors remained bit-identical. The model owner directed the agent to inspect Qwen, find a weakness and attempt an improvement; GPT-6 Astra then planned the experiment, created the test cases, applied the edit and wrote the report.
Infographic 01 · The measured result
Where the edit helped, and where it did not
Qwen3-4B-Instruct-2507, original versus one edited matrix, on synthetic texts that must survive the trip into JSON without losing a character.
Final test · 120 cases by group
- Texts with tabsthe target failureOriginal19/30Edit30/30
- Line breaksalready perfectOriginal30/30Edit30/30
- Mixed special characterstabs, quotes, backslashesOriginal0/30Edit10/30
- Second table rowstill unsolvedOriginal0/30Edit0/30
Development, validation and controls
- Developmentused to fit the edit · not independentOriginal24/40Edit40/40
- Validationnew cases · 9 wins, 0 lossesOriginal10/40Edit19/40
- Random directioncontrol on validation · same strengthOriginal10/40Edit10/40
- Plain-text tabscontrol · real tabs must surviveOriginal10/10Edit10/10
Exact answer = after normal JSON parsing, the text field matches the source text character for character. Same prompt, temperature 0, seed and output limit for both models; saved and reloaded edit, no hooks, no grammar constraints, no repair. Final test, validation and controls frozen before any independent output of the edited model. Fifty of 120 final answers are still wrong.
Sources: our experiment records, 5 Nov 2026 (Lexassay research, internal): experiment 3 result report and final result by GPT-6 Astra working as a Codex agent (OpenAI). Synthetic test cases, not published.
Powered by Thorsten Meyer AI
A Narrow Gain in Exact JSON Handling
The result suggests that a targeted change to model weights can improve a particular, measurable behavior without retraining the full model. That may interest researchers and developers studying whether model internals can be edited to address known failure modes. Here, the target was a concrete data-integrity problem: preserving special characters when text is carried through JSON.
The limits are as important as the score. The experiment tested one model, one specific task and synthetic cases; it does not establish that the edit improves everyday JSON generation or other capabilities. Fifty of 120 final answers were still incorrect, and the separate table-row extraction task remained at 0 of 30. The report also says the work has not yet received an independent second-AI review.
Popular right now
Samsung 65-Inch The Frame Pro Neo QLED 4K TV (2026) Bundle with Modern Brown Bezel, HDMI Cables, and Home Theater Guide
Bose QuietComfort Headphones (2nd Gen), Wireless Over-Ear Noise-Cancelling Headphones, Black
Samsung QN65LS03FAFXZA 65 The Frame QLED 4K Art Mode Vision AI Smart TV (2025)
As an affiliate, we earn on qualifying purchases.
How the Earlier Test Shaped This Edit
This was the third experiment in a series of weight-editing tests described by ThorstenMeyerAI.com. The series followed research on AI systems that inspect and modify neural-network weights directly. In an earlier experiment, Claude edited a local model called Ornith; a later review by GPT-6 Astra identified faults in that work, including a resume bug, a broken command-line interface and effects from weight conversion.
Before the Qwen JSON test, GPT-6 Astra had attempted a different edit on the same model, aimed at extracting fields from synthetic business messages. The original and edited models both scored 97 of 200 on that attempt’s frozen final test. The report says both versions also produced 47 invalid answers, many involving raw tab characters in JSON strings. That observation informed the next experiment: rather than search broadly for weaknesses, the agent focused on the known escaping failure.
Prompting alone had not improved the screening result for the new task: the original prompt and a more explicit escaping prompt each scored 16 of 32 screening cases, according to the source material. Those screening cases are separate from the 120-case final test.
“The starting hint for experiment 3 came from the previous attempt.”
— GPT-6 Astra’s experiment report
Independent Checks Still Pending
The reported gain comes from one internal experiment and has not yet been checked by a second AI or an independent research team, according to the source. The available account does not provide an external replication, a peer-reviewed paper or evidence that the change transfers to real-world text or other JSON tasks.
The test cases were synthetic, and the source does not describe their full construction or establish how representative they are of production data. It also remains unclear whether the improvement persists across different prompts, random seeds, model deployments or a broader range of inputs. The reported 21 fixes and zero regressions apply to this 120-case test, not to all possible uses of Qwen.
Replication and Broader Testing
The next useful step would be an independent review of the experiment records and a repeat of the frozen test, followed by tests on additional examples and settings. Such work could check whether the reported change is reproducible and whether it improves JSON escaping beyond the cases used here.
No date for a review, replication or follow-up experiment was given in the source material. Until those checks are available, the result should be read as a limited, reported benchmark gain rather than a broadly validated capability improvement.
Key Questions
What did GPT-6 Astra change?
It applied a calculated rank-one update to the output projection matrix in Qwen3-4B-Instruct-2507’s final MLP layer. The experiment report says the other 397 stored tensors were unchanged and no gradient training was used.
How much did the JSON score improve?
On the frozen synthetic test, exact answers increased from 49 of 120 to 70 of 120, a reported gain of 17.5 percentage points. The report says 21 cases were fixed and none of the previously correct cases became incorrect.
What task was the model tested on?
It had to place short text containing special characters, such as tabs and line breaks, inside JSON while preserving every character. A case counted as correct only when parsing the JSON returned text exactly matching the input.
Does the result show that Qwen is better at JSON generally?
No. The result covers one model and one narrow synthetic task. The source reports 50 incorrect answers out of 120 on the final test and no improvement on a separate table-row extraction task.
Has the experiment been independently verified?
Not according to the source account. It says an independent second-AI review has not yet taken place; no external replication or peer-reviewed validation is reported.
Source: ThorstenMeyerAI.com
