023-real-data-quality-qualification
Semantic quality and calibration qualification on real labeled data
The governed path from "mechanics only" to a scoped empirical quality claim that backlog item RUSTEV-011 and owner decision R-04 require: how an appropriately licensed public dataset or an independently human-labeled dataset is admitted, with provenance, license, privacy basis, labeling method, annotator independence, splits and limitations recorded; how the evaluator configuration is frozen before the final-test split is read; how quality, calibration, abstention and regression are measured against exact and rules baselines under spec 004; and how every resulting claim is scoped to the evaluated population, task and binding identity. Approved (A-22). Admitted datasets and their per-case details stay outside this repository, held by the host (OD-7); approval authorizes the procedure and its refusal checks, not any particular dataset, live call or spend beyond R-29.
Read the full pinned specification ↗Relationships
depends on
Claimed units
- extends
{ "nature": "additive", "spec": "004-evaluation-and-replay", "unit": { "kind": "directory", "path": "crates/rustev-eval/" } }
Source headings
- 023: Semantic quality and calibration qualification on real labeled data
- 1. Purpose
- 2. Territory
- 3. Behavior
- 3.1 Dataset admission
- 3.2 Contamination and exposure
- 3.3 Freezing before the final test
- 3.4 Measurement
- 3.5 Claim scoping
- 4. Out of scope
- 5. Observable negative cases
- Acceptance
- Verification
- Implementation record
- 6. Questions settled at approval
Frozen anchors
No frozen anchors declared.