Decide what correct means first
A model cannot tell you it is wrong, because detecting that needs ground truth it does not have. Somebody has to define the answer before anything is measured against it.
- Project
- AIKYA
- Ground truth
- A labelled benchmark
- Attribute states
- Agree, disagree, unknown
- Veto rules
- Checked against the schema at startup
Labelled materials
2,017
across 6 CPSEs, synthetic benchmark
Source
AIKYA / BENCHMARK

