statecraft.
Catalog/aicortex

040-retrieval-evaluation

Evaluation: a golden corpus, recall and precision at k, and a gate that refuses a silent regression

Lifecycle approvedImplementation pendingVerification unknownQualification unknown

Retrieval quality is the product, and it degrades invisibly. This spec builds the instrument: a committed golden corpus of memories and queries with judged relevance, a runner that loads it into a temporary instance and executes every query, metrics that are the standard ones rather than invented, a baseline file per configuration, and a gate that fails a change which moves a metric down by more than its tolerance without an explicit, justified baseline update. Constitution XIV becomes a command with an exit code.

Read the full pinned specification ↗

Relationships

Claimed units

  • establishes
    {
  "kind": "file",
  "path": "crates/aicortex-eval/Cargo.toml"
}
  • establishes
    {
  "kind": "file",
  "path": "crates/aicortex-eval/src/lib.rs"
}
  • establishes
    {
  "kind": "file",
  "path": "crates/aicortex-eval/src/corpus.rs"
}
  • establishes
    {
  "kind": "file",
  "path": "crates/aicortex-eval/src/metrics.rs"
}
  • establishes
    {
  "kind": "file",
  "path": "crates/aicortex-eval/src/runner.rs"
}
  • establishes
    {
  "kind": "file",
  "path": "crates/aicortex-eval/src/report.rs"
}
  • establishes
    {
  "kind": "file",
  "path": "crates/aicortex-eval/src/main.rs"
}
  • establishes
    {
  "kind": "file",
  "path": "crates/aicortex-eval/tests/metrics.rs"
}
  • establishes
    {
  "kind": "file",
  "path": "eval/README.md"
}
  • establishes
    {
  "kind": "file",
  "path": "eval/golden/"
}
  • establishes
    {
  "kind": "file",
  "path": "eval/baselines/"
}
  • extends
    {
  "nature": "additive",
  "spec": "018-retrieval-and-recall-trace",
  "unit": {
    "kind": "file",
    "path": "crates/aicortex-recall/src/lib.rs"
  }
}

Source headings

  1. 040: Evaluation
  2. 1. Purpose
  3. 2. Territory
  4. 3. Behavior
  5. 4. Functional requirements
  6. 5. Acceptance criteria
  7. 6. Out of scope
  8. 7. Resolved decisions
  9. Verification

Frozen anchors

No frozen anchors declared.