The plan-of-record for semantic lookup over the repository — documentation, wiki, tickets, code, agent instructions, and remote media — served to both LLM runners and human search. Definitions here are the single source of truth — TODO.md, BACKLOG.md, and tickets link to these anchors instead of re-explaining. Renders via Fossil at /doc/trunk/docs/vector-search.md.

Why this exists

The architecture names RAG-style context awareness as the way the native runner compensates for a smaller context window — the local-first, consult-minimally goal of the whole project. This document makes that concrete: the Fossil repository is both the corpus and the index store. No second database product, no external vector service, no sidecar file — embeddings live inside the repository's own SQLite database and travel with it.

The axioms apply directly:

• Clarity — the search core is a brute-force cosine scan you can read in one sitting; nothing approximate, nothing probabilistic, until measurement forces it.
• Interchangeability — embeddings come through the existing Provider seam; swapping nomic-embed-text for another model is a config change plus a re-index, not a code change.
• Accountability — every chunk is keyed to a content-addressed artifact hash. A search hit is traceable to the exact blob, check-in, and file it came from.

Vocabulary

• Chunk — a contiguous slice of one artifact's text (a markdown section, a Crystal def, a wiki page), small enough to embed and cite whole. The unit of retrieval.
• Embedding — the Float32 vector an embedding Model returns for a chunk's text. Derived data: regenerable at any time from the artifact + the model id.
• Kind — the corpus a chunk belongs to: doc · wiki · technote · ticket · forum · code · instruction · media · api. Queries filter by kind so a runner priming itself pulls instruction, a coding task pulls code + doc + api.
• Project — the repository slug that sourced a chunk. Empty for chunks from the local repo; set to the peer's Fossil project-name for imported peers. Present on every search hit; optional filter argument on vec-search.
• Manifest — a small checked-in record standing in for a bulky remote asset (video, audio, imagery): URI, integrity hash, mime, size, and a pointer to its transcript/caption text. The manifest is versioned; the bytes are not local.

Design decisions

Vectors are stored in two Fossil layers {#derived}

Embeddings are derived data: regenerable from artifact + model id. Content addressing does the invalidation — a hash that hasn't changed still has a valid embedding across renames, branches, and rewrites.

The storage model uses two layers that Fossil already provides, with no sidecar and no file outside the repository:

Layer 1 — fx_vec_* tables (local operational index)

Tables prefixed fx_ survive fossil rebuild (Fossil reserves the prefix for user tables) and live in the repository database alongside the content. They are the fast-path for brute-force cosine scan. They are not synced by fossil clone or fossil sync — they are a local working cache, not the transfer store.

Phase-0 spike (2026-07-04, Fossil 2.27 99675884a9): vec_* tables (no prefix) were dropped by rebuild; fx_vec_* tables (all three, with seeded rows) survived intact. The fx_ prefix is confirmed correct on the deployed version.

Layer 2 — Fossil unversioned content (fossil uv) (the transfer store)

The embedding vectors and chunk metadata are stored as Fossil unversioned files — inside the repository's own infrastructure, transferred with fossil clone --unversioned and fossil sync --unversioned. No model invocation is required to clone or sync: the unversioned blobs are the vectors, already computed. A new server clones the repo and the vectors arrive as a simple copy.

Format: one unversioned file per artifact hash — vec/HASH.bin — starting with a 4-byte version header VE01 (bytes 0x56 0x45 0x30 0x31), followed by chunk records. Each chunk record: seq(u16) + proj_len(u16) + project(UTF-8, proj_len bytes) + text_len(u32) + text(UTF-8, text_len bytes) + vec(L2-normalized Float32[dim], dim×4 bytes little-endian). Blobs without the VE01 header are old format and must be regenerated via vec-index (or migrated with vec-normalize). Stale files (hash no longer reachable from tip) are pruned on vec-index. The fx_vec_* tables are populated from the unversioned blobs by vec-import; no model invocation is needed for that step.

Workflow summary:

No sidecar, ever. Both layers are inside the Fossil repository file or its unversioned content mechanism. There is no separate file on disk to lose, back up separately, or keep in sync manually.

Schema {#schema}

fx_vec_* tables live in the repo database (Layer 1). The unversioned blobs (vec/HASH.bin) are opaque binary files managed by fossil uv (Layer 2); their packed format is defined in vec_store.cr.

CREATE TABLE fx_vec_chunk(
  id       INTEGER PRIMARY KEY,
  hash     TEXT NOT NULL,   -- artifact hash the text came from
  kind     TEXT NOT NULL,   -- doc|wiki|technote|ticket|forum|code|instruction|media|api
  path     TEXT,            -- file path, wiki name, ticket id, manifest path
  seq      INTEGER NOT NULL,-- chunk ordinal within the artifact
  text     TEXT NOT NULL,
  project  TEXT DEFAULT '', -- repo slug; empty = this repo; set for multi-repo aggregation
  UNIQUE(hash, seq)
);
CREATE TABLE fx_vec_embedding(
  chunk_id INTEGER NOT NULL REFERENCES fx_vec_chunk(id),
  model    TEXT NOT NULL,   -- provider model id that produced it
  dim      INTEGER NOT NULL,
  v        BLOB NOT NULL,   -- little-endian Float32[dim]
  PRIMARY KEY(chunk_id, model)
);
CREATE TABLE fx_vec_state(  -- one row: index bookkeeping
  model      TEXT NOT NULL,
  dim        INTEGER NOT NULL,
  last_ckout TEXT           -- check-in hash the index is current to
);

fx_vec_embedding is keyed (chunk_id, model) — structurally multi-model, permitting transitional coexistence during a re-index — while fx_vec_state.model defines what VecSearch actually scans. Changing the embedding model changes fx_vec_state.model and invalidates everything — deliberate and honest; there is no cross-model vector algebra worth pretending otherwise.

Multi-repo dimension {#multi-repo}

The project column on fx_vec_chunk (and the proj_len + project fields in each vec/*.bin chunk record) carries the originating repository slug. For the single-repo case the value is empty or the local Fossil project-name, which costs nothing and preserves backward compatibility.

When the index is eventually aggregated across peers, two hard constraints apply and must be enforced by callers:

1. One embedding model for the whole aggregate. Cosine similarity is only meaningful between vectors produced by the same model at the same dimensionality. Mixing indexes built with different models produces wrong numbers, not degraded results. The model id is recorded in fx_vec_state; a caller aggregating across repos must verify equality before mixing.
2. Project on every hit. Results from an aggregate always identify their source repo. The vec-search CLI and AgentTool.semantic_search carry project on every result object; the optional --project= / project: filter restricts the scan to one repo's chunks.

Exact search first {#exact-search}

Embedding vectors are L2-normalized at index time — both stored chunks and query embeddings — so the similarity score reduces to a bare dot product (the sqrt denominators cancel to 1.0). The search core is a brute-force dot-product scan in plain Crystal over the Float32 blobs — roughly forty readable lines, exact by construction, zero dependencies. At 768 dimensions, tens of thousands of chunks scan in under ten milliseconds on the deployment host with a release build; this repo's corpus will not stress it for a long time.

Approximate-nearest-neighbour indexing is explicitly deferred. If the corpus ever measures past the point where exact scan hurts (~hundreds of thousands of chunks), the sanctioned complex dependency is sqlite-vec — single-file C, MIT, widely deployed — loaded behind the same VecSearch interface so nothing above the seam changes. Until a measurement says so, it does not exist here.

Embedding through the Provider seam {#embedding}

An embedding endpoint is just another Provider: base URL + model id. The default is local Ollama /api/embed with nomic-embed-text; Anthropic/OpenAI embedding endpoints are the same shape if ever wanted. Configuration reuses ollama-codex-providers — no new configuration concept. The embedder is one file with one job: text in, Float32 blob out, model id recorded.

Chunking per kind {#chunking}

• doc / wiki / technote — split on markdown headings; a heading section is the natural citable unit. Oversize sections split on paragraph boundaries with a hard token ceiling.
• ticket / forum — one chunk per comment/post; they arrive pre-sized.
• code — one chunk per top-level def / class / module, found by an honest, readable regex pass. The compiler's own parser is the upgrade path only if the regex measurably misses; do not start there.
• instruction — the runner priming texts. The architecture doc already flags LOCAL_CODEX_PREPROMPT / CLAUDE_PREPROMPT as project-specific content that belongs under agent-docs/; moving them there makes them checked-in artifacts this index covers. A runner can then retrieve the relevant slice of its doctrine instead of carrying all of it in-context — the same economy the consult budget serves.
• media — the manifest's transcript/caption text, below.

Remote bulky media: the manifest pattern {#manifests}

Video, audio, and large imagery stay remote; what the repository versions is a manifest — one small JSON file per asset under media/:

{
  "uri": "https://media.example.org/talks/2026-06-fossil.mp4",
  "sha3": "…",
  "mime": "video/mp4",
  "bytes": 812345678,
  "transcript": "media/talks/2026-06-fossil.txt"
}

The transcript/caption/description text is checked in beside it and embedded with kind=media. A search hit returns the manifest: the caller gets the remote URI plus the integrity hash to verify on fetch. Local cost is text only; the repository stays lean; the asset stays accountable. (Same shape git-annex proved; same shape as Chronicle's attachment thinking.)

External contract {#external-contract}

All four surfaces — the in-process tool (AgentTool.semantic_search), the MCP tool (mcp__ollama-codex__semantic_search), the CLI (agent-tool vec-search), and the future /ext/search HTTP endpoint — return results as the same frozen hit shape:

kind=technote rule: path carries the technote artifact id (the short uuid), not a file path. Consumers that link to Fossil's technote viewer should use /technote/<path>.

This shape is stable. Additive fields are permitted in a future minor version. Removing or renaming a field, changing the type of score, or changing the meaning of path for any kind is a breaking change and must propagate via a daemon-mediated cross-project ticket to all registered consumers before the change lands.

The seam

box "query" "(runner · MCP · CLI · /ext)" fit
arrow
box "VecSearch" "dot-product over fx_vec_embedding" fit
arrow
box "fx_vec_* tables" "repo database, derived" fit fill 0xeeeeee
arrow from 2nd box .s down 0.4 then right
box "VecIndexer" "chunk → embed → store" fit
arrow
box "Embedding Provider" "ollama /api/embed" fit

One binary, as ever: the native runners call AgentTool.semantic_search in-process; the claude runner reaches the same tool over MCP (mcp__ollama-codex__semantic_search); operators use agent-tool vec-search; humans use /ext/search. All four are interfaces onto the same VecSearch.

Files, one responsibility each:

• src/vec_store.cr — schema, open/attach, read/write of chunks + blobs
• src/vec_chunker.cr — kind-aware text → chunks
• src/vec_embedder.cr — Provider call, text → Float32 blob
• src/vec_indexer.cr — walk tip, diff against vec_state, embed only new hashes
• src/vec_search.cr — query → embed → cosine scan → ranked hits

Phases

Phase 0 — spikes {#phase-0}

All three spikes complete. Results recorded here; spec files are spec/vec_embed_roundtrip_spec.cr and spec/vec_scan_timing_spec.cr.

1. Rebuild safety ✓ — fx_ prefix survives fossil rebuild on Fossil 2.27 99675884a9. Two-layer model settled. See #derived.

2. Embed round-trip ✓ — Crystal → Ollama /api/embed (nomic-embed-text, 768-dim) → Float32 BLOB → SQLite in-memory → back. All 768 values recover within 1e-6. Cosine checks: same sentence = 1.0, unrelated sentences = 0.38, similar > unrelated confirmed (0.64 vs 0.33).

3. Scan timing ✓ — dot-product scan on the deployment host (all well within limits):

The debug timings used cosine similarity (two extra norm accumulators + sqrt); the release timings use the bare dot product enabled by L2 normalization at index time. Combined improvement from --release optimisations and the simpler inner loop: 41–52×. The SQLite BLOB path overhead (~2.5× vs in-memory at 10k in debug) is proportionally unchanged; not re-measured under release since the corpus will remain well under 10k chunks for the foreseeable future.

Phase 1 — indexer {#phase-1}

agent-tool vec-index walks the tip manifest plus wiki/technotes, chunks (doc/wiki/technote first), embeds hashes not yet in vec_chunk, records last_ckout. agent-tool vec-rebuild drops and re-indexes from scratch — the fossil rebuild analogue. Idempotent by construction: re-running against an unchanged tip does nothing.

Phase 2 — search core + tool exposure {#phase-2}

vec_search.cr; agent-tool vec-search "query" [--kind=KIND] [--project=PROJECT] [-k N]; the same method registered as AgentTool.semantic_search (in-process for native runners) and as mcp__ollama-codex__semantic_search via bin/mcp-server. Hits return {score, kind, project, path, hash, text} — enough to cite or to fetch the full artifact. The project field is present on every hit; the optional project filter restricts the scan to one repo's chunks. Vectors are L2-normalized at index time (VecEmbedder.l2_normalize); the score field is therefore a bare dot product in [0, 1].

agent-tool vec-normalize migrates existing stored vectors in-place (pure vector math, no re-embed) and rewrites all vec/HASH.bin blobs with the VE01 header.

Phase 3 — human surface {#phase-3}

/ext/search: query box, kind/project filters, ranked results linking to /doc/<hash>/, /wiki/<name>, /technote/<id>, /tktview/<hash>, /info/<hash>. Sits beside Fossil's keyword /search, not replacing it — keyword and semantic answer different questions. Served as a console route in src/chat_cgi_app.cr via src/ui.cr (render_search_page); the project filter is a <select> dropdown defaulting to the current repo.

Phase 4 — code + instruction corpora, RAG wiring {#phase-4}

Code corpus: src/vec_indexer.cr walks all .cr files (excluding *_spec.cr) at the tip manifest via CODE_LIST_SQL and chunks them as kind=code via VecChunker.split_code.

Instruction corpus: preprompts live under prompts/ (prompts/local-codex-preprompt.md, prompts/claude-preprompt.md) and are indexed as kind=instruction — index_docs detects the prompts/ prefix via instruction_path?.

RAG pre-turn: OllamaTaskRunner.rag_context discovers the repo via Config.repository_from_checkout, checks VecStore#empty? (silent no-op when no index exists), retrieves top-3 hits by embedding the task prompt, and prepends them as a "Related context" block. All errors are swallowed so a missing index or unreachable Ollama never breaks the task runner.

Crystal stdlib as kind=api is deferred to Phase 6 (gated on the import-project milestone).

Phase 5 — media manifests {#phase-5}

src/vec_manifest.cr — VecManifest module with Manifest record (uri, sha3, mime, bytes, transcript), load_json parser, and verify? integrity helper (SHA3-256 via OpenSSL).

src/vec_indexer.cr walks media/**.json files at tip via MEDIA_LIST_SQL; for each valid manifest it reads the transcript file via fossil cat and embeds the transcript text as kind=media, keyed to the manifest's blob UUID. Files under media/ are skipped by index_docs to avoid double-indexing transcript .txt files as kind=doc.

A search hit for kind=media returns the manifest path in path and the transcript chunk in text; the caller fetches the remote URI from the full manifest and calls VecManifest.verify? to validate the download against sha3 before consuming it.

Phase 6 — keep-current + scale (conditional) {#phase-6}

The persistent task-worker -d daemon re-runs the incremental indexer after commit/merge events, so the index tracks tip without operator action. sqlite-vec behind VecSearch only if Phase-0 scan timings, revisited against the real corpus, demand it.

Open items

• ~~Rebuild-safety / in-repo vs sidecar~~ Resolved 2026-07-04. Two-layer model: fx_vec_* tables (local operational index, survives rebuild) + Fossil unversioned content vec/HASH.bin (transfer store, cloned with --unversioned, no model invocation on clone). No sidecar. See #derived.
• Embedding-model choice: nomic-embed-text (768d) is the working default; benchmark one alternative before Phase 4 locks the corpus in.
• Whether forum posts index continuously or on thread close (churn vs freshness).
• Cross-checkout behaviour: fx_vec_* tables are shared across all checkouts of the same repo DB (tip-scoped, correct). Unversioned blobs are synced per-repo with --unversioned.