In October 2025, I cofounded an AI governance startup called Lioren Technologies. Lioren was based around a governor that caught unsafe responses at inference. I believed it needed token-level data to detect hallucinations, so I set out to collect as much as I could across different kinds of adversarial prompts. I built a harness to capture that data and tune the governor against each model.
Collecting this data exposed a problem with the governor. Confidence and entropy flag a model that's unsure, but a confident hallucination isn't unsure, it's wrong with high confidence, and at the token level that looks identical to being right. The signals the governor was built on couldn't see the failure that mattered most, which is what led me to look for a new method of detecting confident hallucinations.