Lexical Similarity

Explanation

Factuality/Faithfulness checks whether an AI system's output is actually supported by its source material rather than fabricated, which is critical in RAG and summarisation where a confident but unfounded statement, such as a wrong interest rate or invented policy clause, can mislead customers and breach the expectation that the system is trustworthy. The Factuality/Faithfulness Score rates how well each claim in the generated output is grounded in the retrieved or reference context, typically by checking whether the supplied source entails the statements the model makes. To implement it, capture both the model output and the exact context passages used to produce it, then run a faithfulness evaluator (for example an entailment or claim-verification check) that flags statements not backed by the source, logging the score and the unsupported claims as evidence for each evaluated response across pre-release testing and sampled production monitoring. The thresholds are a target of at least 0.90, with investigation below 0.85 and remediation below 0.80 (proposed): scores in the investigate band prompt a reviewer to examine grounding quality, while scores in the remediate band require corrective action such as tightening retrieval, adjusting prompts, or constraining generation. Its Immediate priority reflects that ungrounded output is among the most damaging accuracy failures in BFSI.