Inference with Human Oversight Rate

Explanation

Inference with Human Oversight Rate measures the share of in-scope model decisions that pass through a human checkpoint before taking effect, ensuring people remain accountable for consequential AI outputs — a core expectation of RBI's Safety, Resilience and Sustainability principles, NIST MANAGE 2.1/3.3 and ISO Clause 6.1.2. It matters because fully automated decisions in lending, fraud holds or account closures can cause real harm if the model errs; a human gate provides a chance to catch and correct mistakes and preserves a responsible owner. The metric is Human-in-loop, expressed as the proportion of in-scope decisions that are human-gated, with a proposed target of 100%. No calculation method is specified, so operationalise it as decisions reviewed by a human divided by total in-scope decisions. To implement, define which decision types are 'in-scope' (typically high-impact or agentic/RAG actions), route those to a reviewer queue where a person must approve, reject or modify before the action commits, and log the reviewer identity, timestamp and disposition as evidence. Operate dashboards tracking the oversight rate by decision type. Any in-scope decision that executes without a recorded human gate is a breach, triggering alert, review of the bypass and remediation of the routing control.