As LLMs have grown more powerful, hallucinations have proven stubbornly difficult to avoid. Errors pop up in even the smartest models, and while there are ways to catch those errors, the industry is still figuring out the best way to do it.
Probably, which just raised $9 million in seed funding from Andreessen Horowitz, is trying to build a more rigorous way to catch those errors.
As founder Peter Elias (pictured above) puts it, the company’s goal is to prevent hallucinations and simple factual errors from ever reaching the user, and achieve the kind of 99.99% accuracy that’s common in deterministic systems but much more difficult to reach with AI. As it turns out, bringing LLMs to that level of accuracy requires rethinking many of the basic assumptions of AI engineering.
Probably’s first product is a data science tool, built to produce quick answers from complex datasets. Each result comes with a citation and an audit trail for how it was developed, an increasingly common practice among AI tools.
But keeping errors from creeping into those summaries required an elaborate harness system that Elias describes as a “data science mech suit.” The LLM’s first-pass answers are checked against a deterministic validator system, which bounces back any results that don’t match the dataset. Crucially, the LLM has been trained against the validator, and the whole system is optimized for fast and accurate answers, the company said.
“What we learned building this was that the better your harness engineering is, the weaker the model can be,” Elias says. “If you can refine the context enough, the model does not have to work very hard to do the right thing. Basically, it’s an exercise in reducing ambiguity.”
That allows Probably’s data science tool to run on significantly smaller AI models. Elias …