| Ticket Hash: | 8150605d31f9d19d68e239c2fc997677477d4868 | ||
| Title: | Use the kind=summary layer as a docs-staleness detector (flag docs whose summaries contradict current code) | ||
| Status: | Closed | Type: | feature |
| Severity: | Normal | Priority: | medium |
| Subsystem: | vector-search | Resolution: | Overcome_By_Events |
| Last Modified: |
2026-11-09 21:53:27 34.3 hours ago |
Created: |
2026-07-07 00:04:56 127.3 days ago |
| Version Found In: | |||
| Description: | ||||
|
Idea surfaced 2026-07-06: the kind=summary layer (per-doc + per-section LLM summaries) is a natural STALENESS DETECTOR. A summary that describes behavior the code no longer implements is a flag that the doc is out of date.
Build an automated (or semi-automated) staleness check that leverages the summary layer: - For each doc summary, cross-check its claims against current reality — e.g. embed the summary and find the nearest CODE chunks; if a summary asserts a mechanism/name/flag that no longer exists (fails a symbol/lookup or contradicts the code), flag the doc as possibly stale. - Surface flags in a report (agent-tool docs-staleness) and/or the web Search/Docs UI (a "possibly stale" badge on a doc). - Optionally re-generate a doc summary and diff against the stored one after a code change to detect drift. This makes the docs-freshness audit ONGOING instead of a one-time sweep. Acceptance: - A tool/report lists docs whose summary claims are contradicted by current code (with the specific contradicted claim). - Low false-positive rate on a spot-check; flags map to genuinely stale content. - Runs off the existing kind=summary index (no new corpus). Depends on: a complete summary index (kind=summary rebuild). Relates to: the docs-freshness audit ticket, and Vector Search milestone 27c526c72b. | ||||
| User Comments: | ||||
|
added on 2026-11-09 21:53:27:
Closed in OCX's ticket sort, batch 6 (ollama post 8f6c9afbb7): tracked under the semantic-search row of docs/parity-v1.md (0e2d18fbea), held: the maintainer chose Fossil's full-text search first. Reopen this when semantic search is taken up. | ||||