Login
Battery run 4: all eight tasks on the fixed interface
Login

2026-10-15, OCX at a1faf72aee (branch agent-tools): after runs 1, 2 and 3 and the faults they found, among them read's text now in a fenced block, exact, wiki_edit in place of wiki_write, and test running every test when given no name. Three models, both variants, 3 trials of each of the eight tasks.

Passed

model steer plain
gemma4:31b 24/24 24/24
qwen3-coder:30b 23/24 23/24
qwen2.5-coder:7b 10/24 11/24
task qwen2.5-coder:7b qwen3-coder:30b gemma4:31b
add-section 5/6 6/6 6/6
fix-bug 3/6 6/6 6/6
rename 2/6 6/6 6/6
share-binary 6/6 6/6 6/6
wiki-status 5/6 6/6 6/6
multi-file-bug 0/6 6/6 6/6
add-function 0/6 6/6 6/6
fix-and-document 0/6 4/6 6/6

What it says

The records, every exact request, tool answer and reply of each trial, are attached as one tar.


Attachments: