Most AI governance programs produce metrics as a reporting exercise — a dashboard that stays green until the day it doesn't. The programs that actually manage AI risk use metrics as a decision-support system instead: something that tells the model risk committee where to look, tells the board when risk appetite is being approached, and produces the evidence trail an examiner will actually ask for. The right metric depends entirely on who's reading it. A board needs strategic signals because it decides on risk appetite and resourcing, not individual validation timelines; a committee needs operational detail because it decides which models get escalated attention; and examination evidence has to be evidentiary — it has to demonstrate compliance, not assert it.
A dashboard that's been green for six straight months is worth a second look before it's worth a compliment. In my experience, that's usually a sign the thresholds were set too loose to ever trip, not a sign the AI program is actually healthy.
Model risk and validation metrics
| Metric | Definition | Target | Audience |
|---|---|---|---|
| Validation currency rate | % of production AI models with current independent validation | 100% Critical, 100% High, ≥95% Medium | Board, Committee, Examination |
| Open high-severity findings | Count of open Critical/High findings, aged by overdue status | Zero unmitigated Critical findings; High findings have committee-approved timelines | Board, Committee |
| Finding remediation rate | % of findings remediated within committed timeline | ≥90% on-time | Committee, Examination |
| Inventory completeness rate | % of known AI deployments registered with all required fields | ≥95% with all 9 fields populated; annual shadow-AI discovery completed | Committee, Examination |
Fairness and bias metrics
| Metric | Definition | Target | Audience |
|---|---|---|---|
| Fairness testing coverage | % of customer-facing AI with current testing including India proxy variables | 100% of Critical/High customer-facing models | Board, Committee, Examination |
| Proxy group approval rate ratio | Approval rate ratio for each India proxy group vs. general population, within credit quality band | A documented internal threshold — many banks use the US four-fifths/80% convention as a working benchmark, since RBI has not codified a specific figure | Committee, Examination |
| PMJDY population performance gap | Approval rate for PMJDY-enrolled vs. general population, same credit quality band | ≤5 percentage point gap without a documented inclusion justification | Committee, Examination (India-specific) |
| Fairness monitoring coverage | % of production credit/fraud AI with real-time fairness monitoring, not manual periodic review | 100% of Critical customer-facing models | Committee, Examination |
AI security metrics
| Metric | Definition | Target | Audience |
|---|---|---|---|
| Guardrail false positive rate | % of legitimate interactions blocked by AI safety guardrails | Below the bank's own defined customer-impact ceiling | Committee (customer impact) |
| Prompt injection detection rate | % of confirmed injection attempts detected before reaching the model | As close to 100% as the bank's own red-teaming can validate; zero undetected injections confirmed in post-hoc review | Committee, Examination |
| AI incident mean time to detection | Average time from occurrence to detection | An internal target the committee sets and defends — faster for security incidents than for slower-moving performance/fairness drift | Committee, Examination |
Operational and regulatory metrics
| Metric | Definition | Target | Audience |
|---|---|---|---|
| Monitoring coverage rate | % of production models with automated monitoring vs. manual review | 100% Critical, ≥90% High | Committee, Examination |
| Regulatory notification compliance | % of Critical AI incidents notified to the relevant regulator within the bank's own defined internal timeline | 100% against whatever timeline the bank's own policy and counsel have set | Board, Examination |
| Vendor contract compliance rate | % of AI vendor contracts including all required governance clauses | 100% for Critical/High vendors; ≥80% overall with documented exceptions | Committee, Examination |
| Adverse action explanation quality rate | % of AI-generated notices passing quality review (decision-specific, plain language, India-specific factors named) | ≥99% pass rate; failures escalated as governance incidents | Committee, Examination |
Metrics by audience
Board, quarterly — 5 to 7 risk-posture indicators: % of Critical/High models with current validation; open Critical findings count; a residual-risk heat map by domain; Critical AI incident count and status; fairness compliance status; regulatory examination status.
MRM Committee, monthly — the full operational picture: all metrics above, plus inventory exceptions pending approval, models approaching validation due dates, fairness metric trends by proxy group, open vendor contract gaps, and pending decommissioning approvals.
Examination, on request — evidentiary documentation: 24 months of validation currency trend, finding remediation timeline evidence, proxy-variable fairness test results on file, internal notification records with timestamps, vendor contract compliance evidence, and adverse-action explanation samples.
India-specific metrics with no international equivalent
Proxy group approval rate ratio across all four India dimensions. Beyond standard gender/age disparate impact, track caste-proxy (pincode-based), religion-proxy (UPI-pattern-based), language-community, and disability-proxy approval rate ratios continuously, with alerts when any ratio drops below the bank's documented threshold within a credit quality stratum.
DPDP Act data subject rights fulfillment rate. The percentage of deletion, correction, and access requests tied to AI decision data fulfilled within the Act's timeline — a metric that doesn't exist in US or EU AI governance programs because neither regime forces this specific tension between an individual deletion right and a multi-year regulatory retention requirement.
Regional language explanation availability rate. For banks with significant non-English customer bases, the percentage of AI-generated adverse-action notices actually available in the customer's preferred language. A bank serving rural Maharashtra that only generates English notices has a financial inclusion gap and a regulatory exposure gap at the same time — track toward 100% availability across the top languages in your service area, not just English plus Hindi.
What to fix on the dashboard this quarter
- If any metric has stayed green for two consecutive quarters, check whether the threshold is actually tight enough to ever trip.
- Add the proxy group approval rate ratio for all four India dimensions if it isn't already a standing line item, not a one-off test result.
- Set an explicit internal notification-timeline target for AI incidents and get the board to approve it — don't rely on an assumed regulatory number.
- Build the DPDP data subject rights fulfillment rate as a tracked metric, not a manual log reviewed only when a request arrives.
- Confirm the regional-language explanation availability rate actually gets reported to the committee, not just mentioned in passing.
Zytra's governance dashboard tracks all of these in real time, with role-specific views for board, committee, and examination — and the examination evidence package is auto-generated from the same data driving the dashboard. Request a demo to see it.
Related: AI governance program and reporting
Track all 15 AI governance metrics with role-specific views for board, committee, and examination
Zytra's governance dashboard tracks validation currency, finding remediation, India proxy variable fairness ratios, security incidents, monitoring coverage, and regulatory notification compliance in real time. The examination evidence package is auto-generated from the same data — no manual assembly.