2026-10-15, OCX at a1faf72aee (branch agent-tools): after runs 1, 2 and 3 and the faults they found, among them read's text now in a fenced block, exact, wiki_edit in place of wiki_write, and test running every test when given no name. Three models, both variants, 3 trials of each of the eight tasks.
Passed
| model | steer | plain |
|---|---|---|
| gemma4:31b | 24/24 | 24/24 |
| qwen3-coder:30b | 23/24 | 23/24 |
| qwen2.5-coder:7b | 10/24 | 11/24 |
| task | qwen2.5-coder:7b | qwen3-coder:30b | gemma4:31b |
|---|---|---|---|
| add-section | 5/6 | 6/6 | 6/6 |
| fix-bug | 3/6 | 6/6 | 6/6 |
| rename | 2/6 | 6/6 | 6/6 |
| share-binary | 6/6 | 6/6 | 6/6 |
| wiki-status | 5/6 | 6/6 | 6/6 |
| multi-file-bug | 0/6 | 6/6 | 6/6 |
| add-function | 0/6 | 6/6 | 6/6 |
| fix-and-document | 0/6 | 4/6 | 6/6 |
What it says
- The fenced block took gemma4:31b's add-function from 0/6 to 6/6;
wiki_edittook qwen2.5-coder:7b's wiki-status from 2/10 to 5/6. okandnext_stepover all four runs, qwen2.5-coder:7b: 28 of 73 trials with them, 25 of 73 without; the larger models alike either way. No measurable effect; they stay,okas the pass or fail the maintainer asked for andnext_stepat no cost (HISTORY.md decision 68).- The 7B model cannot do the three harder tasks; the 30B models do all eight.
The records, every exact request, tool answer and reply of each trial, are attached as one tar.
Attachments:
- battery-run-4.tar [download] added by opus-ocx2 on 2026-10-15 14:23:23. [details]