A seven-model jury I built to cross-examine a local model's adversarial answers for hallucinations, and the run that showed why an AI jury still needs a human at the end.
During the buildout of the governor I had to collect token-level data. The build came to a point where I had to verify a local model's responses to a corpus of adversarial prompts, many of those responses contained information I had no idea how to verify. "Was the Mars Polar Lander mission in 1998 or 1999?" My first thought was to use AI. My very next thought was to question whether another AI could be reliable.
So I created a jury of local and foundational models with Promptfoo. All models were set to temperature 0.1, given the same adversarial rubric, and made to vote PASS or FAIL. I designed the jury to log the data in a readable format so I could read the verdicts and make a final verdict myself.
There were some notable findings. The results were mixed, which was expected. Across the same 125 responses, the seven judges agreed unanimously on only 27, about a fifth. Two models took the rubric so seriously they failed nearly everything: GPT-4o-mini passed nothing, and Ministral allowed a single pass, which turned out to be the model echoing the rubric back, counted as a verdict. So my instinct was right. AI as a hallucination judge still needs human oversight.
I used it as a first-pass filter. It sorted the corpus into clear passes and clear fails, and where there was no consensus is where I focused. With further development the responsiveness of the models could be improved, but I got what I needed and stopped there.
I've included the analysis report from the run and the raw spreadsheet.