Context Efficiency

Explanation

Context efficiency checks how much of the model's available context window a RAG system is actually consuming when it assembles retrieved passages plus the prompt — because over-stuffing the window wastes cost, slows responses, and can crowd out the most relevant evidence, while running near the limit risks truncation that silently drops information a banking answer depended on. It is measured by the Context Utilization Ratio: the share of the context capacity in use. No explicit formula is given, so report utilization as the proportion of the window consumed without inventing additional terms. Implement it by instrumenting the prompt-assembly step to record tokens used versus the model's window size on every request, logging the ratio alongside the request id and retrieved-chunk count as evidence, and tracking the distribution over time. Operate it against a capacity SLA: the system should stay within the agreed capacity, and when utilization reaches 80% the control should auto-scale (for example shrink or rerank context, raise capacity) or alert operators, so that requests do not hit hard truncation limits during peak load and answer completeness is preserved.