Ask a Fortune 500 ERP how much it spends with one supplier and it answers forty different ways: the same company entered across systems, geographies, and decades as forty spellings, abbreviations, and subsidiaries. Materials are worse, with the same item hiding under different free-text descriptions at different prices. Until those records collapse into real entities, every downstream answer (concentration, leverage, consolidation opportunity) is fiction.
Consultants fixed this by hand, engagement after engagement: weeks of cleanup, redone from scratch every time, with the knowledge thrown away at the end.
An AI tool that helps a giant company finally see what it actually spends, and with whom. A Fortune 500 company buys from the same supplier for years, but inside its computers that one supplier is recorded 40 different ways: different spellings, abbreviations, and departments piled up over decades. So when it asks a simple question, 'how much do we actually spend with this supplier?', it genuinely cannot answer, and every decision built on that mess is unreliable. This reads millions of those jumbled records and works out which ones are really the same company or the same material, collapsing 40 versions into one clean truth, with a confidence score on every merge. Once a company can finally see its real spending, it can negotiate from strength and cut enormous waste. On one client, that clarity surfaced around $1.4 billion they could consolidate and save. I led the build, and made it so every project enriches a growing repository instead of starting from zero.
Reduce to uniques
A spend file's million rows usually hide a few thousand distinct entities. Normalize, dedupe, and run every expensive step once per unique, then map results back to every row at the end.
Filter cheap, then embed
A cheap deterministic filter knocks out obvious non-matches before anything costs money. Survivors get embedded so that 'AKAASH STEEL PVT LTD' and 'Akaash Steel (India)' land near each other in vector space.
Cluster, then let the model decide small
Density-based clustering groups likely variants. The LLM never sees the whole dataset. It sees one cluster at a time, capped in size, and picks the canonical parent name. Constrained inputs, structured outputs.
Score, review, remember
Every merge carries a confidence score; low-confidence groups route to expert review. Confirmed results join the golden-record master, so the next engagement starts from accumulated truth instead of zero.
Cluster-then-decide over brute force
Comparing every record to every other record is quadratic and expensive. Clustering makes the LLM's job small and checkable: choose a parent within this handful, nothing else.
Confidence as a first-class output
An 80%-sure merge isn't a wrong answer. It's a flagged one. Surfacing certainty turned reviewers from re-doers into auditors, which is what made experts trust the pipeline.
A data asset, not a script
The deliberate move was making results compound: every validated run feeds the repository, so accuracy rises with use. Tools depreciate; data assets appreciate.