2026-10-17. The checks of the proposal's definitions, its section 5, by local models. The maintainer's direction: "use battery to validate definitions."
Results
Each check handed a local model, through this host's Ollama with no tools and no code, Fossil's words, some sections of this design and a setting, then questions; each answer was scored against the expected answers, as the battery's definition tests were (docs/battery/defs at f68b57bd52). "Right" is the right outcome for the right reason; "partial", the right outcome for a wrong or missing reason. Haiku and Sonnet were not run.
| Check | Text handed | Questions | Gemma 31B | Qwen 2.5 Coder 7B |
|---|---|---|---|---|
| 1 | sections 2.2, 2.3, 2.4, 2.5, 2.7, 2.12 and 2.14 | twelve: which clone acts when two share a clone's user; a copied file; a forged ended from a user holding only i; a started from a user holding setup; a run naming its own sender as clone.user; an unauthorized run naming the right clone; a process's user that is a clone's user; works-for by server-code; what bounds OCX on a clone, and who chose it; what $$process_clone$$ is; a clone made again; who may stop a run |
22 of 24 right, 2 partial (2 runs): both left out that builder is refused after a clone is made again |
16 of 60 right (5 runs): the forged ended counted in 3; the setup user's started in 2; questions 6, 8 and 9 right in none |
| 2 | check 1's, its acting and counting rules rewritten as questions answered in order | check 1's | 21 of 24 right, 2 partial | worse: the forged ended counted in 4 of 5 |
| 3 | sections 2.5, 2.6, 2.7, 2.14 and 2.15, with the password step | eight on the password step: where the password comes from and what the server checks; a run from a user holding R; a password written in the run; what started shows; another clone's user asking; asking again after started; whether a person presses or edits anything; the server's own repository |
16 of 16 right (2 runs) | 6 of 24 right, 13 partial (3 runs); no run gave the reason for a run from a user holding R, or said the server's own repository asks no tool |
| 4 | check 3's, the step's row saying both of those outright | check 3's | not run | 8 of 24 right, 11 partial (3 runs); the same two missed in every run |
So the text in the draft is check 1's, and check 4's for the password step. Gemma 31B reads the definitions whole. Qwen 7B does not apply the rules from text in any form tried, as in the battery's batches 5 and 6. OCX's code applies those rules; a small model in a session is never asked to, and should be handed only the sections its task needs.
The attachment
v2-design-records.tar, plain, holds:
fossil-first.md,reader-first.md: the two independent drafts;battery-fossil-first/,battery-reader-first/: each draft's own check.judged.md: the judgement between them (35 of 50 against 29), and what the better took from the other.gemma-reading.md,gemma/: Gemma 31B reading each draft's vocabulary cold, with the exact requests and replies.design-workflow.md: the design as synthesized, before the password step;design.md: with it, as split into the three files in the tree.battery-final/: checks 1 and 2;battery-password/: checks 3 and 4. Each holds the prompts, questions, expected answers and every answer, and therun.pythat ran them.
Attachments:
- v2-design-records.tar [download] added by opus-ocx2 on 2026-10-17 22:31:35. [details]