Inside the EU’s Emergency AI Audit — and What It Found

The European AI Office, established under the EU AI Act, conducted its first emergency audit of a deployed high-risk AI system in February. The subject of the audit was a credit scoring system operated by a major pan-European bank. The findings, released last month after a 90-day confidentiality period, are instructive about both the regulatory machinery and the systems it’s designed to govern.

The audit found three significant issues. First, the system’s training data included historical loan approval decisions that encoded discriminatory lending patterns from the pre-GDPR era, and the system had replicated those patterns in its current decisions at a statistically measurable rate. Second, the bank’s documentation of the system — required under the AI Act’s high-risk system provisions — described the system’s decision logic at a level of abstraction that made the audit difficult to conduct and would make it difficult for affected individuals to exercise their right to explanation. Third, the system’s ongoing monitoring process had failed to detect the discriminatory pattern despite operating for 18 months, because the monitoring metrics were not designed to test for the specific demographic disparities that the audit identified.

What This Tells Us

The audit’s findings are consistent with what AI auditors and regulators have observed in other contexts: the problem is often not the AI system itself but the documentation and monitoring infrastructure around it. Systems that are making consequential decisions are frequently surrounded by governance frameworks that were designed to satisfy compliance requirements rather than to actually identify when the system is malfunctioning.

The bank has been given 60 days to remediate the identified issues. It has not been fined — the AI Act’s fine provisions apply to ongoing violations, and the bank moved promptly to suspend the system pending remediation. The audit represents the EU’s first use of its emergency audit powers, and both the process and the findings will shape how the AI Office conducts future audits.