AIKYA
A material-code harmonization engine that refuses to merge anything it cannot justify.
- Python
- FastAPI
- scikit-learn
- rapidfuzz
- SQLite
- Next.js
- Docker
Repository private while the problem statement is live. Opening after evaluation.
Measured
- Precision, unattended auto-merge
- 100%
- 5-fold cluster-grouped CV, engine.eval --veto-model Synthetic benchmark.
- Precision, naive fuzzy baseline
- 61.0%
- Same split, engine.eval --baseline Synthetic benchmark.
- Hard negatives merged in error
- 0 of 2,562
- Planted hard negatives; the baseline merged 1,854 of them Synthetic benchmark.
- Precision including the human review queue
- 98.2%
- Same split, 73 uncertain pairs routed to a steward Synthetic benchmark.
What it is
Every Central Public Sector Enterprise runs its own SAP material master, so the same bolt exists as CPCL-FAST-1001, IOCL-FAST-1002 and GAIL-FAST-1004, in three description styles and three units. Multiply that by two to five lakh line items per organisation. AIKYA reads those masters, works out which rows describe the same physical item, mints one Common National Material Code, and keeps a permanent mapping back to every original code.
The interesting engineering is in what it declines to do. SS304 and SS316 differ by one character and score about 97% similar on any string metric, but SS316 contains molybdenum and is specified for chloride service. Merging them puts the wrong bolt on a seawater line. So attribute comparison is three-state rather than boolean: AGREE is positive evidence, DISAGREE on a critical attribute vetoes the match whatever the score, and UNKNOWN routes the pair to a human instead of guessing. The veto configuration is validated at startup against the extractor's schema, so a safety rule the extractor cannot actually produce fails the build rather than silently never firing.
Accuracy is measured with 5-fold cluster-grouped cross-validation. Grouping by cluster matters more than it sounds: splitting pairs at random would put CPCL's and IOCL's spelling of the same bolt on opposite sides of the split and leak the answer into the training set. There is a test, test_evaluation_leak.py, whose entire job is to catch that class of mistake, and CI runs lint, veto-config validation, the test suite, a safety regression gate and the accuracy report on every push.
Decisions worth defending
- Three-state attribute comparison, where a conflict on a safety-bearing attribute vetoes the merge regardless of similarity score
- Veto configuration validated against the extractor schema at startup, so an unenforceable safety rule fails the build
- 5-fold cluster-grouped cross-validation, chosen specifically to prevent train/test leakage between spellings of the same item
- A dedicated regression test for evaluation leakage, plus a CI accuracy gate on every push
- Deterministic gradient-boosted classifier over structured features rather than an LLM, so decisions stay reproducible and auditable years later
- Benchmark generator that refuses to plant unlearnable traps, which would quietly cap the achievable score
Not built yet
Planned work, with the reason it is next. None of this exists today.
Hybrid retrieval with vector embeddings over the description corpus
Candidate generation is currently lexical, so an item described in genuinely different words never reaches the classifier at all. Embeddings widen recall without touching the veto, which stays the safety boundary.
Active learning on steward decisions
Every review is a labelled example being thrown away. Retraining on them should shrink the review queue over time rather than leaving it constant.
Calibration of the match probability
The auto-merge and review thresholds are currently tuned numbers. Calibrated probabilities would let the bands be set from a stated false-merge budget instead.