Every bank can point to a human in its AI-decision workflow. Far fewer can show that the human is actually forming an independent judgment rather than rubber-stamping a recommendation that arrived pre-packaged with a confidence score and a plausible explanation. That gap — between a human present in the process and a human exercising genuine judgment inside it — is where automation bias lives, and it's invisible in any workflow diagram because the diagram shows a review step either way.
The order information arrives in determines whether review is real
A reviewer who sees the AI's recommendation before independently forming a view of the case is anchored to that recommendation whether or not they realize it — this is a well-established finding in decision research, not a banking-specific quirk, and it shows up reliably in controlled studies on automation bias. The practical fix is sequencing: show the reviewer the underlying case facts first, let them form a provisional judgment, and only then reveal the AI's recommendation for comparison. Most banks' current workflows do the opposite — lead with the AI score because it's the fastest thing to render on screen — which is efficient and is also exactly the design choice that produces rubber-stamping.
If approving takes one click and overriding takes a form, you've designed the bias in
A workflow where accepting an AI recommendation is a single click and overriding it requires a multi-field written justification is not neutral — it's a friction asymmetry that pushes every marginal, uncertain case toward acceptance regardless of the reviewer's actual judgment, because disagreement costs more effort than agreement. Override documentation exists for a legitimate reason — accountability needs a record — but that record requirement should apply symmetrically, or at minimum shouldn't be so much heavier on the override side that it functions as a soft penalty for disagreeing with the model.
An override rate is a starting point, not an answer, until you check whether the overrides were right
A 5–20% override rate is roughly where a well-functioning human-AI review process tends to land for most consequential decision types — enough disagreement to show genuine independent judgment, not so much that the AI isn't adding value. A rate below 2% on a high-volume queue is the number that should trigger an automation-bias investigation, not a congratulatory note in a model performance report. But the rate alone doesn't tell you which direction the bias runs — the only way to know whether an override rate is healthy is retrospective outcome analysis: when humans did override, were the outcomes better, and when they agreed with the AI, would an override have produced a better outcome in hindsight. Skipping that outcome analysis and reporting the override rate on its own is reporting a number that looks like oversight and measures almost nothing about its quality.
Reviewers can't exercise judgment on an explanation they can't actually use
Meaningful review requires an explanation a human reviewer can genuinely evaluate against their own expertise — SHAP-based feature importance for a traditional model, highlighted supporting evidence, a calibrated confidence indicator — not a bare score. A reviewer handed a number with no basis for independent assessment isn't being given the tools to disagree even if they wanted to, which means the "human in the loop" in that workflow is structurally set up to agree regardless of the individual reviewer's competence or diligence.
Regulators across jurisdictions are converging on substance over form here
RBI's FREE-AI framework's accountability pillar, the EU AI Act's human-oversight provisions, and Bank of England expectations all require meaningful review of consequential AI decisions, and the examination trend across these frameworks is toward testing substance — can this reviewer actually explain why they agreed, not just that they clicked agree — rather than accepting a reviewer's signature as sufficient evidence on its own. . A bank designing its human-oversight control around "we have a sign-off step" rather than "we can demonstrate the sign-off reflects genuine judgment" is optimizing for the wrong audit question.
India layers statutory review rights and personal accountability on top of the general principle
The DPDP Act 2023 gives customers a statutory right to request human review of an adverse automated decision, which only means something if the bank has designated a specific responsible individual and a defined response timeline — a generic customer service queue is not a DPDP-compliant human review mechanism. RBI's Fair Practices Code requires that a credit rejection come with actual, individual-level decision factors, which forces exactly the per-application explainability that meaningful human review depends on. PMJDY and other financial-inclusion account segments need their own oversight monitoring specifically for rejection-rate disparities, because a model performing well in aggregate can still be systematically harsher on a segment the bank has a policy commitment to serve. And under PMLA, the named Principal Officer's personal accountability for SAR accuracy means human review of AI-assisted AML outputs is not an operational nicety — it is the mechanism by which legal liability actually attaches to a named individual.
What a bank should actually do
- Redesign review workflows to show case facts before the AI recommendation, so reviewers form an independent judgment before seeing what the model concluded.
- Equalize friction between acceptance and override — stop making disagreement structurally more expensive than agreement.
- Pair override-rate monitoring with retrospective outcome analysis; investigate both unusually low and unusually high rates, and don't trust the rate alone.
- Give reviewers an explanation they can actually evaluate (feature importance, highlighted evidence, calibrated confidence) — a bare score is not a basis for meaningful oversight.
- Stand up a named, timeline-bound human-review process for DPDP Act requests and PMJDY/priority-sector rejection-rate monitoring specifically — don't rely on a generic customer service channel to satisfy either obligation.
Related Articles
Override rate monitoring built into governance.
Zytra tracks override rates by queue, reviewer, and model — detecting automation bias before it becomes a regulatory finding. SHAP-based explanations per case give reviewers something to review, not just a recommendation to accept.
Talk to sales