OCR Noise Tolerance

Explanation

OCR Noise Tolerance checks how well the model keeps extracting information correctly when the documents it reads are degraded or noisy rather than clean - for example faded ink, printing irregularities, or scans of historical records, which are common in banking Document Extraction over cheques, statements, and legacy paperwork. It is measured by the OCR Degradation Rate, calculated as F1_noisy - F1_clean, where F1_clean is the F1 score (the harmonic mean of precision and recall) for text extraction on ideal, high-quality documents and F1_noisy is the F1 score on documents that have been intentionally degraded or are naturally noisy. The closer this gap is to zero, the more robust the model is to noise. To implement it, assemble paired clean and degraded versions of representative documents (or synthetically apply realistic noise), run extraction on both, compute the two F1 scores and their difference at evaluation time, and log per-run results as evidence. The threshold is a degradation of 1% or less, with a target of 0, and an alert on any breach (proposed). Exceeding this gap should trigger investigation and remediation - such as improving preprocessing, denoising, or model fine-tuning on noisy samples - before relying on the extractor for noisy production documents. It maps to RBI Safety, Resilience and Sustainability, NIST MEASURE 2.7 robustness and security, and ISO Clause 6.1.2 with 9.1 monitoring (all proposed).