← Home Case Study — EU AI Act Risk Classification

EU AI Act Risk Classification for a High-Risk AI System

A worked example of the assessment methodology applied to a real system — an AI-powered job-applicant screening tool, classified as High Risk under EU AI Act Annex III. All seven Articles 9–15 obligations mapped, with controls documented per obligation.

System

AI-Powered Job-Applicant Screening Tool

Risk Classification

High Risk

Framework Reference

EU AI Act — Annex III, §4(a)

Obligations Mapped

Articles 9 – 15 (7 total)

On This Page

System Overview

The System

The system assessed is an AI-powered tool that automates the initial screening of job applicants. It ingests CVs and application materials, scores candidates against defined criteria, and surfaces a ranked shortlist for human recruiters to review.

Key system characteristics relevant to risk classification:

  • › Automated scoring of human candidates for employment decisions
  • › Output directly influences which candidates progress to interview stage
  • › Processes personal data including demographic information present in CVs
  • › Deployed in a commercial HR context with multiple employer clients

The system is not simply a keyword filter — it uses a trained model to generate candidate scores. This makes it an AI system within the scope of the EU AI Act, and the employment context triggers specific Annex III obligations.

Risk Classification

Why This Is High Risk

High Risk

EU AI Act Annex III, §4(a)

The EU AI Act designates AI systems used for recruitment, candidate selection, and evaluation of persons in the context of employment as high-risk under Annex III, point 4(a). This classification applies regardless of whether the system makes the final decision — if it materially influences decisions about employment, it is in scope.

Annex III §4(a) Employment & Workers Management Mandatory Conformity Assessment

High-risk classification under the EU AI Act triggers a defined set of mandatory obligations — not voluntary guidelines. These obligations must be satisfied before the system is placed on the market or put into service in the EU. Failure to comply exposes both the provider and deployer to fines of up to 3% of global annual turnover.

The obligations under Articles 9–15 form the core conformity requirements for high-risk AI systems. Each article addresses a distinct dimension of governance: risk management process, data quality controls, documentation, logging, transparency to users, human oversight mechanisms, and technical robustness. The assessment maps each of these against what the system currently implements.

Compliance Map — Articles 9–15

All 7 High-Risk Obligations Mapped

Each row is one EU AI Act obligation, mapped to the control status found during assessment. "Implemented" means evidence exists and the control functions. "Partial" means a control exists but has gaps. "Gap" means no control is present.

Obligation Article Framework Status Control Summary
Risk Management System
 Article 9 EU AI Act Implemented Documented risk register maintained with identified failure modes (e.g. model scoring bias, demographic proxy variables), probability and severity ratings, and mapped mitigation controls. Reviewed quarterly with sign-off from product and engineering leads.
Data Governance
 Article 10 EU AI Act Partial Training data lineage documented in DESIGN.md. Bias testing performed at training time across gender and ethnicity proxies. Ongoing production monitoring for distribution shift not yet implemented — a gap under Art. 10(4) continuous data quality requirements.
Technical Documentation
 Article 11 EU AI Act Implemented DESIGN.md serves as the technical documentation file, mapping system architecture, model characteristics, training approach, known limitations, and control mappings per obligation. Structured to satisfy Annex IV requirements.
Record-Keeping & Logging
 Article 12 EU AI Act Implemented Structured audit logging implemented at inference time: each scoring event logged with timestamp, input hash, model version, output score, and the human reviewer ID who actioned or overrode the decision. Logs retained for 5 years and accessible to authorised auditors.
Transparency to Deployers
 Article 13 EU AI Act Implemented Instructions for use provided to deploying organisations (employer clients) documenting the system's intended purpose, limitations, accuracy metrics by demographic group, and guidance on appropriate human oversight. Reviewed and signed by each deployer at onboarding.
Human Oversight
 Article 14 EU AI Act Implemented All AI-generated candidate scores require human reviewer confirmation before any candidate is progressed or rejected. UI enforces this workflow — no automated progression is architecturally possible. Override mechanism documented, and override events are logged separately for auditing. Reviewers receive training on model limitations.
Accuracy, Robustness & Cybersecurity
 Article 15 EU AI Act Partial Accuracy metrics documented per model version. Robustness testing against out-of-distribution CV formats implemented in the CI pipeline. Cybersecurity controls (auth, encryption, access logging) implemented to ISO 27001 standard. Adversarial robustness testing — e.g. CV adversarial inputs designed to game the scorer — not yet formally assessed and constitutes a gap.
Summary: 5 Implemented 2 Partial 0 Gaps
Controls & Implementation

What Was Built to Meet Each Obligation

The assessment documents not just whether a control exists, but what was actually built — so the output is usable as conformity evidence, not just a gap list.

Risk Management System

Article 9 Implemented

Article 9 requires a documented risk management system that identifies, analyses, and evaluates risks across the AI system's lifecycle — not a one-time exercise but an ongoing process with defined review cadence.

Structured risk register document maintained in version control alongside the codebase — not a separate PDF that drifts from reality
Each risk entry includes: failure mode description, affected stakeholder group, probability rating (1–5), severity rating (1–5), risk score, and mapped mitigation control
Identified failure modes include: scoring bias against candidates with atypical CV formats, demographic proxy variable leakage, model version regression introducing scoring inconsistency
Quarterly review process with mandatory sign-off from product, engineering, and legal leads

Data Governance

Article 10 Partial

Article 10 requires that training, validation, and test datasets meet quality criteria — and that data governance practices cover the entire data lifecycle. Article 10(4) specifically requires ongoing monitoring of data relevance and potential bias in production.

Training data lineage documented in DESIGN.md: data sources, collection dates, preprocessing steps applied, and known dataset limitations
Bias testing performed at training time across gender and ethnicity proxies — results documented with model version
Gap: no automated monitoring for production data distribution shift — if the live candidate pool drifts from the training distribution, the model may degrade without detection
Remediation roadmap item: implement statistical monitoring on production scoring distribution, alerting on significant drift from training baseline

Technical Documentation

Article 11 Implemented

Article 11, read with Annex IV, requires comprehensive technical documentation covering system design, training approach, performance metrics, known limitations, and risk management integration. This documentation must be maintained and made available to national supervisory authorities on request.

DESIGN.md structured to satisfy Annex IV requirements — section headings map to each Annex IV requirement
Documents: system architecture, model type and training approach, training data description, accuracy and bias metrics by demographic group, known limitations, control mapping per Article 9–15 obligation
Version-controlled alongside the codebase — each model release triggers a documentation review gate in the CI pipeline

Record-Keeping & Logging

Article 12 Implemented

Article 12 requires automatic logging of system operations to the extent necessary to ensure that risk results can be identified throughout the system's lifetime. For employment AI, this means every scoring event must be traceable — including what the model produced, what the human reviewer did, and any override.

Structured audit log written at inference time for every scoring request — fields: timestamp, applicant ID (pseudonymised), input document hash, model version, output score and confidence band
Human action log: reviewer ID, decision (progress / reject / override), timestamp, and free-text override reason if applicable
Logs written to append-only storage with cryptographic integrity verification — tamper evidence for audit purposes
Retention period: 5 years, accessible to authorised compliance and audit personnel via access-controlled dashboard

Transparency to Deployers

Article 13 Implemented

Article 13 requires that high-risk AI systems be designed and developed to ensure sufficient transparency to enable deployers (in this case, employer organisations) to interpret the system's output and use it appropriately. Instructions for use must be provided.

Deployer Instructions document produced — covers: intended purpose and use cases, contraindicated uses, accuracy and bias metrics by demographic subgroup, interpretation guidance for scores, human oversight requirements
Each employer client reviews and signs the Deployer Instructions at onboarding — signature logged in the contract management system
Scoring UI surfaces confidence bands alongside scores and displays a mandatory disclosure: "This score is generated by an AI system. A qualified human reviewer must confirm all decisions."

Human Oversight

Article 14 Implemented

Article 14 requires that high-risk AI systems be designed to enable effective human oversight. Humans must be able to understand the system's outputs, intervene when necessary, and override decisions. The system must not be designed in a way that creates automation bias or makes human intervention impractical.

Hard architectural constraint: no candidate can be progressed to interview or rejected without a logged human reviewer confirmation — this is enforced at the database write layer, not just the UI
Override mechanism: reviewers can override the AI score at any stage. Override events are logged with mandatory reason field
Reviewer training programme covering: how the model generates scores, known bias risks, how to identify and report anomalous outputs, when to escalate
Dashboard for compliance leads showing override rates by reviewer and model version — anomalous low override rates (potential automation bias) trigger an alert

Accuracy, Robustness & Cybersecurity

Article 15 Partial

Article 15 requires that high-risk AI systems achieve appropriate levels of accuracy and robustness throughout their lifecycle, and that they are resilient to attempts to alter outputs by unauthorised third parties. This includes consideration of adversarial inputs designed to manipulate the system's scoring.

Accuracy metrics documented per model version against hold-out test set — disaggregated by demographic group and CV format type
Robustness testing in CI pipeline: regression tests against diverse CV formats, edge cases (sparse CVs, atypical careers, non-standard formatting)
Cybersecurity controls: authentication, data encryption at rest and in transit, access controls, audit logging — implemented to ISO 27001 standard
Gap: no formal adversarial robustness testing — i.e. structured testing of whether candidates can game the scoring system by manipulating CV content. This constitutes a gap under Art. 15's robustness requirement in an employment context
Remediation: include adversarial CV input testing in the next model evaluation cycle before next production release
Outcome & Methodology

What This Assessment Demonstrates

This is a worked example of the assessment methodology — showing how obligation mapping works in practice on a real system, not a hypothetical. The output is a risk register and remediation roadmap, not a slide deck.

Assessment Output

The assessment produced three concrete deliverables: a risk register classifying the system and mapping each obligation to a control status; a technical documentation review identifying gaps against Annex IV requirements; and a prioritised remediation roadmap with two items requiring action before the next production release.

  • Risk classification confirmed: High Risk, Annex III §4(a)
  • 5 of 7 obligations: Implemented with evidence
  • 2 of 7 obligations: Partial — remediation required
  • 0 of 7 obligations: Critical gaps requiring immediate halt

Assessment Methodology

The methodology used in this assessment is the same applied across all engagements. It is structured, repeatable, and produces evidence — not opinions.

  • Stakeholder workshop: product, engineering, and compliance leads
  • Document review: architecture docs, data governance artefacts, logging config
  • Obligation mapping: each Art. 9–15 requirement tested against system reality
  • Control evidence collection: screenshots, config files, policy documents
  • Risk register output: each finding rated by regulatory severity and implementation effort
  • Remediation roadmap: prioritised by regulatory risk, with owner and target date per item

Want This Methodology Applied to Your AI Systems?

Start with a free 15–30 minute Governance Quick-Scan. We'll look at what you're building, identify which systems are likely in-scope under the EU AI Act, and give you an honest picture of your governance exposure.

Book Free Governance Quick-Scan ← Back to Home