Structural Awareness

Explanation

Structural Awareness checks whether the model correctly understands and preserves the structure of a document when extracting from it - recognising tables, rows and columns, sections, key-value pairs, and nested layouts rather than flattening them into jumbled text. This matters in banking Document Extraction because a misread table or merged column in a statement, schedule, or financial summary can silently corrupt the extracted values. It is measured by Structure Parsing Accuracy, calculated as Structure accuracy = Correct_Structures / Total_Structures, the fraction of structural elements the model parses correctly out of all structures present. To implement it, build an evaluation set annotated with the true structural elements of each document, run extraction, compare the parsed structure against the ground truth, compute the accuracy, and log per-document results as evidence so problematic structures can be diagnosed. Running this at evaluation and on production samples helps catch regressions when new document formats appear. The thresholds are tiered: a structure accuracy of at least 0.90 is the target, below 0.85 should be investigated, and below 0.80 should be remediated (proposed). Crossing these levels should trigger the corresponding alert and corrective action - such as improving table or layout parsing - before relying on the extractor. It maps to RBI Safety, Resilience and Sustainability, NIST MEASURE 2.7 robustness and security, and ISO Clause 6.1.2 with 9.1 monitoring (all proposed).