Detect Hallucinations for RAG-Based Systems

RAG reduces hallucinations, but it doesn't remove them. A system can retrieve exactly the right passage and still answer with something that passage never said, and from the outside that answer looks the same as a correct one. Most detection methods work from the prompt and the response alone, while RAG applications have something extra to check against: the context the system actually retrieved. Comparing an answer to the passage it came from is a narrower question than comparing it to the world, which is why detection gets easier here.

We implement four detectors and compare them on accuracy, precision, recall, latency and cost: an LLM prompt-based detector scoring each sentence from 0 to 1 against the context, semantic similarity between the answer and the passage, a BERT stochastic checker, and token similarity. None of them are state of the art, and that's deliberate. They are simple enough to add to an existing pipeline in an afternoon, which counts for more when the alternative is no detection at all.

Two implementation details matter more than the choice of method. First, give the judge its own model, memory and parameters, because a judge sharing chat history with the model that produced the answer ends up grading its own reasoning and tends to agree with itself. Second, tune the cut-off between grounded and hallucinated on your own data, since that boundary shifts by domain and a threshold borrowed from a benchmark will sit in the wrong place.