Auditability

Explanation

This auditability control checks whether failures in a RAG system can be traced back to their true cause — whether a bad answer came from poor retrieval, a flawed source document, or the generation step itself. It matters because when a banking RAG assistant gives a wrong answer, the institution must be able to show exactly where the breakdown occurred, both to fix it and to satisfy accountability obligations; an unattributable failure is one nobody can remediate or defend. It is measured by the RAG Failure Attribution Rate: the share of failures that the system can correctly attribute to a specific pipeline stage. No explicit formula is provided, so report the attribution rate without inventing terms. Implement it by logging, for every RAG response, the query, the retrieved passages and their scores, the assembled prompt, and the final output, all linked by a trace id, so that when a failure is flagged it can be decomposed and assigned to retrieval, source, or generation as audit evidence. The target is an unattributed-failure rate of less-than-or-equal-to 1% (ideally 0); any breach raises an alert, indicating gaps in tracing that must be closed so failures stop being black boxes.