Document Accessibility Bias

Explanation

Document accessibility bias checks whether a document-extraction system performs equally on high-accessibility documents (clear, high-contrast, well-structured) versus low-accessibility ones (faint print, poor contrast, low resolution, cluttered layout), so customers submitting harder-to-read documents are not unfairly penalised with extraction errors. It matters because document quality often correlates with access to resources, and a performance gap quietly disadvantages vulnerable customers in onboarding and claims. It is measured by the Visual Fairness Score, computed as F1_high_access − F1_low_access: the difference in extraction F1 between high- and low-accessibility document groups. To implement it, curate labelled test sets stratified by accessibility characteristics, compute F1 for the high- and low-access groups, take the difference, and log per-group scores plus representative failing samples as evidence on a fairness dashboard. Run it pre-deployment and on a recurring cadence, re-testing after model or imaging-pipeline changes. The proposed thresholds target at least 0.90, with investigation below 0.85 and remediation below 0.80; dropping under 0.85 triggers investigation and under 0.80 triggers mandatory remediation — enhance image-handling, contrast normalisation, or model training for low-accessibility inputs, re-test, and record sign-off — in line with proposed RBI Fairness and Equity and NIST MEASURE 2.11 bias-evaluation expectations.