AI Explainability for Banking: SHAP, LIME, and Regulatory Evidence

Explainability is not a letter in an acronym — it is the practical substance behind RBI's "Understandable by Design" sutra, one of the seven principles in the FREE-AI committee's framework report . It is also a binding obligation under Articles 13 and 14 of the EU AI Act. For banking AI, explainability is the precondition for human oversight: Article 14 only works if the person reviewing a decision can actually see what drove it, not a confidence score and a shrug. This guide covers SHAP and LIME methodology for BFSI, the explainability depth required at each decision tier, and the evidence an RBI or OCC examiner will sit down and read line by line.

SHAP vs LIME for banking AI: choosing the right method

SHAP (SHapley Additive exPlanations) is the method regulators will accept as evidence. Attribution values sum to the prediction's deviation from the base rate, which is what makes comparison across customers and across time periods valid — an examiner can line up ten adverse decisions and check whether the same features drive them consistently. TreeSHAP gives exact attribution on tree-based models; on neural networks you're working with an approximation, and you should say so in your documentation rather than imply precision you don't have. SHAP's real cost shows up at inference scale: kernel SHAP on a large feature set with a complex model can add meaningful latency, which is why most production deployments store SHAP values at decision time rather than computing them on demand when a customer asks.

LIME (Local Interpretable Model-Agnostic Explanations) is useful for model debugging and dead on arrival as regulatory evidence. It builds a local linear approximation around one prediction, so explanations are instance-specific and can genuinely differ between two runs on the same input — that's not a bug, it's how the method works, and it's exactly why it fails as evidence. We've watched an examiner compare two LIME explanations for near-identical loan denials and ask why the top feature changed between them; there was no good answer, because LIME was never built to give one. Use LIME internally for feature-selection sanity checks. Do not put it in front of an examiner and call it your explainability program.

Explainability levels: calibrating depth to decision impact

Not every banking AI decision needs the same explainability depth, and RBI's FREE-AI report and the EU AI Act both imply the same proportionality logic: explanation depth scales with the decision's impact on the customer. Running full SHAP-per-decision storage on an internal fraud-review heuristic wastes engineering budget; running model-level feature importance on an automated credit denial gets you an adverse finding in an RBI inspection. The five tiers below are how we've seen banks actually operationalize this without either extreme.

• L1 — Internal operational AI. Back-office automation, internal analytics, fraud scoring for internal review only. You need model-level feature importance for the governance audit trail and periodic aggregate analysis — nothing customer-facing. Example: an internal risk-scoring model feeding portfolio analytics.
• L2 — Customer-facing AI with low material impact. Product recommendations, personalized content, communication targeting. Document model-level feature importance; make customer-specific explanation available on request, but you don't need to generate it proactively. Example: a cross-sell recommendation engine.
• L3 — Customer-facing AI with material impact. Credit approvals, loan pricing within normal parameters. Store SHAP attribution per decision, make it retrievable within the regulatory response timeline, and keep examination evidence on file. Example: automated loan approval at standard rates.
• L4 — Adverse decisions. Credit refusal, adverse terms, unfavorable insurance pricing, KYC rejection. Store and retrieve SHAP attribution, issue an adverse-action notice naming the top contributing factors, and document the human review pathway — EU AI Act Article 14 makes human oversight mandatory here. Example: an automated credit denial.
• L5 — High-stakes agentic decisions. Agent-executed transactions, autonomous loan modifications, collections actions. You need the full context chain — what instructions the agent received, what tools it invoked, what reasoning led to the action — and Forensic Replay capability to reconstruct it after the fact. Example: an agentic loan-restructuring decision.

The tier assignment belongs in your model inventory, signed off by risk, not left to the data science team's judgment call on launch day. That one governance step is where most gaps actually get caught.

What examination-ready explainability evidence looks like

An RBI or OCC examiner reviewing your explainability program checks five things, roughly in this order. First, technical documentation: model architecture, feature set, which explainability method you chose and why, and calibration evidence that SHAP values actually track observable changes in model behavior. Second, decision-level records: SHAP attribution stored per decision for customer-facing models, retrievable by customer ID and date, for as long as your board-approved retention schedule requires . Third, cohort-level analysis showing feature importance stability over time, so you catch drift before it quietly changes what your explanations mean. Fourth, proof you can actually respond to a customer explanation request within the required timeline — have sample adverse-action notices ready, not hypothetical ones. Fifth, the bias interaction check: does SHAP show the same high-attribution features correlating with protected characteristics? That's the seam between your explainability program and your fairness program, and examiners increasingly ask for both in the same sitting.

Regulatory mapping

Zytra: AI explainability for BFSI

Zytra's governance platform stores SHAP attribution per decision at inference time, indexed for retrieval by customer ID and date range. Examination evidence packs include cohort-level SHAP analysis, feature importance stability reports, and customer explanation response capability, formatted for RBI, EU AI Act, and SR 11-7 requirements. At L5 (agentic), Forensic Replay reconstructs the full context chain.

What to do before your next examination

1. Map every customer-facing AI use case to an explainability tier (L1–L5) in the model inventory, signed off by risk — not defaulted by whichever team built the model.
2. Store SHAP, not LIME, attribution at the decision level for every L3-and-above model before it reaches production; retrofitting this after an examiner asks is the expensive way to learn the lesson.
3. Build the adverse-action notice pipeline for L4 decisions now: top contributing factors, human review pathway, and a response SLA, with a sample ready to show on request.
4. Run quarterly cohort-level SHAP stability checks to catch feature drift before it silently changes what your stored explanations actually mean.
5. For any L5 agentic use case, stand up Forensic Replay before launch — not after an incident forces the conversation.