Retrieval Quality

Explanation

Retrieval quality checks whether a RAG system fetches the right source passages before the model answers — because in BFSI a chatbot or summarizer is only as trustworthy as the documents it pulls (a wrong clause retrieved from the wrong policy leads to a confidently wrong answer about, say, a customer's interest rate). It is measured by Retrieval Precision@K: of the top K passages the retriever returns, the proportion that are actually relevant to the query. No calculation method is specified beyond this metric, so do not invent a formula — report the standard precision-at-K against a labelled relevance set. Implement it by maintaining a gold set of queries with known-relevant passages, periodically running the live retriever over them, and logging the precision score, the value of K used, and the misretrieved passages as evidence; instrument this at the retrieval stage of the pipeline, before generation. Operate against the target of greater-than-or-equal-to 0.90; investigate when precision falls below 0.85 (tune chunking, embeddings, or reranking) and remediate below 0.80, where retrieval is degraded enough that generated answers cannot be trusted and the index or retriever needs correction.