The Confident-Hallucination Detector
A hallucination detector built in a field I had no background in. I directed it, and an AI coding agent wrote and ran the code on a borrowed machine, built from the start to run as cheaply as possible.
I lost access to the remote machine and the project stopped there. The final state is archived. Nothing below is peer reviewed, and the last section says plainly what I cannot claim.
Where it started
This project came out of an earlier one. At my startup I built a governor, a product meant to catch bad model output before it reached the user, and a red team harness to test it. The governor looked for hallucinations by asking the model the same question several times and checking whether the answers agreed. The harness data showed the flaw: that approach cannot catch the wrong answer a model states calmly and identically every time, which was the one failure I cared about. It left a sour taste in my mouth.
The current work had moved inside the model, to its internal activations. I did not know that field, and I was not going to spend months becoming an interpretability researcher before I could start. So I started a new project to build a detector of my own. What follows is the order it happened in: each problem I hit, and what got built to get past it.
Learning as I Built
So every result had to pass a test whose outcome I could read without following the mechanism behind it, and the pass marks were fixed before the data existed.
Testing protocol
Two experiments had their pass and fail marks committed to version control before any data was collected. One passed. One missed, and the miss is still in the record.
The clearest check: refit the detector on deliberately scrambled labels. If the signal is real, scrambling should destroy it.
Across 30 shuffles the scrambled version scored 0.493, a coin flip. The real one held 0.896.
The first tool could not see inside the model
The collector was moved to Hugging Face transformers, which reads the model's internal state layer by layer. Every collector wrote the same file format, so swapping the backend never meant rewriting the analysis on top. One 208-line collector ended up handling all 25 models.
Adapting the harness
The first two collectors read only the output: the top ten token probabilities, then the model's full vocabulary. The second came back negative, which showed the output side had nothing more to give, and the tool running it could not expose the layers inside the model at all.
Every claim needed an answer key
Each sentence a model writes is a claim, and the detector learns from claims marked true or false. Those marks are the labels. A jury of three models from different vendors graded them against the actual Wikipedia article, never their own memory, after an early audit caught a judge inventing a fact to grade an answer. Grading was the slow, expensive stage, so each claim was graded once and the same claims were run through every model.
Why grading once is sound
Whether a sentence is true does not depend on which model reads it, so every model was scored against the identical answer key. That is what makes 25 models comparable. The real objection is that a model reading someone else's sentence might behave differently from one writing its own. That was tested directly (step 6): 2,068 claims the model generated itself, collected and graded fresh. No model graded itself or its own family.
The free tiers ran out every day
I wanted to see how cheaply this could be done, so the jury ran on free API tiers wherever it could. Free tiers have daily caps, not just per-minute ones, and the jury kept hitting them:
- A six-worker burst exhausted a whole day's quota mid-run. The graders were rebuilt to run throttled and checkpointed, so any of them could be killed at any point and resumed in the next quota window. About 56,000 verdicts were accumulated that way.
- One juror had its own per-model daily cap, which turned a re-grade into a weeks-long drip. The grading chain became a loop that retried every three hours, took whatever quota had freed up, and only exited when the job was fully done.
- A dead quota makes a juror return nothing but abstains, which would quietly poison the saved progress. A guard stopped the run cleanly after 15 empty answers in a row.
- One provider's rate-limit replies said to wait hours, which looked like the connection hanging. The client was capped to wait at most 60 seconds and retry.
- Quota pressure forced jurors to be swapped out twice. No new juror counted until it was checked against the old one's stored verdicts on an overlap slice. The two swaps agreed at 96.6% and 95.3%.
Running unattended on a borrowed, shared machine
The Mac Studio had 192 GB of RAM and still could not hold the range of models I wanted. So the system downloaded the next model, ran the tests, erased it, and started again, driven by 64 lines of shell that checked every five minutes whether the current run was still alive. About 65 runs went through that way with nobody watching.
The output was bigger than both machines, so a small daemon drained the box over a private mesh network: pull, verify, back up to Drive, and only then delete the remote copy. It only touched files that had gone five minutes without a write, so it never grabbed one mid-write. Every file existed in at least two places at every moment of its life. Over its lifetime it moved 177 files, 183.3 GB, and lost zero bytes.
The shared box was power-cycled for maintenance on an intermittent basis. Runs were chained behind completion markers so a restart skipped finished work, and every transfer retried until a window opened and resumed rather than restarting. One 5.5 GB pull crossed a dozen link drops without going back to zero. When the local disk filled to literal zero, the rule became: confirm a pack on Drive by name and size, then delete the local copy.
Claude Science integration
Claude Science runs in the Claude app. It has no shell and no access to the machine; the only thing it could reach was my Google Drive. I started using it two weeks after it came out. Its first job was a map of the field: 578 papers screened by abstract and narrowed to 252 on the topic, to place what I had against published work. Once it could read the data, it reproduced my results on its own, and its numbers matched mine. Its analysis was better than mine, so from there I let it take a bigger role in directing the project.
At first I pasted its output into Claude Code and back again. That did not hold up against the run queue and my limited access to the Mac. Both agents could read and write Google Drive, so I had Claude Code design the communication system I had in mind: one status file overwritten every loop and marked ready only after the last byte uploaded, and a request queue in the other direction that the executing side could decline.
Running it exposed two failures. The two agents cited 0.932, 0.943 and 0.934 for the same result, so every headline number was stored alongside its exact configuration, and the drift stopped. And one request sat unread because the watcher looked for a fixed file name; it was changed to pick up the newest file instead.
The first experiment over it was written by Claude Science, which had never seen the machine, then reviewed and run by Claude Code. It went from first test to launch in under 40 minutes and needed two messages relayed by hand.
Real conversations are not single-turn
Single prompts were the baseline, but real use is a conversation, and I wanted the detector attacked, not just sampled. A second model was put in a closed loop as a live attacker, reading the defender's answers and pushing harder each turn. It ran unattended across twelve defender architectures.
What it found
As recorded by the collection logs: 26 days · ~19,500 lines across two repos · ~200 GiB of activations across 25+ models · ~75,000 juror verdicts
On the answers I was targeting, the model's own confidence barely sorts true from false. A detector reading its internal state does substantially better.
On the calmest tenth of claims: internal signal 0.91 to 0.92, the model's own confidence 0.61 to 0.73.
The resampling method the governor was built on was measured directly. It landed at exact chance on the calm, confident answers, because 17.4% of those hallucinations came back identical across ten tries. There was nothing for it to measure.
What I cannot claim
I am not an interpretability researcher and none of this was peer reviewed. As I worked on it, I realized designing the engineering is where I learned the most.
The final results, which finished around July 14, were never audited while the project ran. In September I had Claude Science audit them before publishing this page. The protocol claims and the headline numbers it could recompute checked out. The collection figures (file counts, gigabytes, verdict totals) are consistent across every document. They come from the machine's own logs and from Google Drive: once the local Mac ran out of room, everything was written there, and that archive is what remains. The machine itself has since been wiped.
The four gaps
- The robustness testing used the weakest class of attacker. The stronger ones, the sort that defeat comparable systems in the published literature, were never run.
- Transfer between different model architectures was never demonstrated cold.
- Labels came from an automated jury graded against Wikipedia, not human adjudication. Two human audit sheets are built and still ungraded.
- Validation against a third external dataset never happened.
The documents below were produced during the project. The engineering and analysis write ups were written by the agent that did the work.