023-real-data-quality-qualification

Semantic quality and calibration qualification on real labeled data

Lifecycle approvedImplementation completeVerification unknownQualification unknown

The governed path from "mechanics only" to a scoped empirical quality claim that backlog item RUSTEV-011 and owner decision R-04 require: how an appropriately licensed public dataset or an independently human-labeled dataset is admitted, with provenance, license, privacy basis, labeling method, annotator independence, splits and limitations recorded; how the evaluator configuration is frozen before the final-test split is read; how quality, calibration, abstention and regression are measured against exact and rules baselines under spec 004; and how every resulting claim is scoped to the evaluated population, task and binding identity. Approved (A-22). Admitted datasets and their per-case details stay outside this repository, held by the host (OD-7); approval authorizes the procedure and its refusal checks, not any particular dataset, live call or spend beyond R-29.

Read the full pinned specification ↗

Relationships

Claimed units

  • extends
    {
      "nature": "additive",
      "spec": "004-evaluation-and-replay",
      "unit": {
        "kind": "directory",
        "path": "crates/rustev-eval/"
      }
    }

Source headings

  1. 023: Semantic quality and calibration qualification on real labeled data
  2. 1. Purpose
  3. 2. Territory
  4. 3. Behavior
  5. 3.1 Dataset admission
  6. 3.2 Contamination and exposure
  7. 3.3 Freezing before the final test
  8. 3.4 Measurement
  9. 3.5 Claim scoping
  10. 4. Out of scope
  11. 5. Observable negative cases
  12. Acceptance
  13. Verification
  14. Implementation record
  15. 6. Questions settled at approval

Frozen anchors

No frozen anchors declared.