RAG Answer Unfaithfulness / Groundedness Failure
Model Accuracy & ReliabilityDescription
A RAG-based system retrieves the correct source documents but the generated response contradicts, overstates, or departs from the retrieved context. Distinct from hallucination (which fabricates facts absent from any source): here the source is correct and the grounding step fails. In BFSI this causes material harm — a model answers with a wrong interest rate, outdated regulatory deadline, or incorrect product term even though the correct value was in the retrieved chunk.
A mortgage bot retrieves the RBI MCLR circular containing the current repo rate but the LLM responds with a rate from its training-data memory, not the retrieved document. Customer is quoted a wrong EMI.
Multiple confirmed production incidents establish this risk: lawyers submitted AI-hallucinated case citations to a federal court (Mata v. Avianca, 2023), and Air Canada's chatbot provided factually incorrect refund policy guidance that a tribunal held the airline legally bound by (2024). Both cases involved authoritative-sounding incorrect outputs causing real-world downstream harm.
Primary mitigations
- Answer-faithfulness scoring (e.g. RAGAS faithfulness metric)
- source-citation enforcement
- retrieval context size limits to avoid truncation
- post-generation groundedness verification gate
- abstain-if-ungrounded policy.
Detection signals
Per-response faithfulness score; source-contradiction rate; sampled QA audit against retrieved chunks; NLI-based entailment checking.
Mitigating controls
0No directly mapped controls.