Composite scenario, not a real company. Assembled from common industry patterns; every number is illustrative and its assumptions are listed at the end.
This case belongs to Governance and responsible AI in practice.
Not legal advice. This page shows how engineers can read the text of Regulation (EU) 2024/1689 (Official Journal, 12 July 2024, ELI), checked on 2026-10-22. The Commission's AI Act page, shown as last updated 19 October 2026, describes later changes under an "AI Omnibus" and says: "Starting on 2 December 2027, high-risk AI systems will be subject to strict obligations before they can be put on the market" (the page then lists them). Quotes below are the text as first published. Check the current text and that page, and ask counsel, before relying on any scope or date.
Context
- A software company hires for about 12 roles a month, with about 400 applications each.
- Its in-house assistant scores each CV against the job advert and ranks applicants. Recruiters open the top 20.
- Role: Article 3(11) defines putting into service as "the supply of an AI system for first use directly to the deployer or for own use in the Union for its intended purpose". A company that builds a tool for its own recruiters may therefore be a provider and a deployer. Counsel should confirm.
| Question | What the text says | Reading |
|---|---|---|
| Is the use in an Annex III area? | Point 4(a) of Annex III, "Employment, workers’ management and access to self-employment": "AI systems intended to be used for the recruitment or selection of natural persons, in particular to place targeted job advertisements, to analyse and filter job applications, and to evaluate candidates" | Yes: ranking applicants is filtering and evaluating them |
| Can Article 6(3) lift it? | An Annex III system "shall not be considered to be high-risk where it does not pose a significant risk of harm", for example for "a preparatory task". It is "always" high-risk "where the AI system performs profiling of natural persons" | Open. A ranker that decides who a person reads is hard to call preparatory, and "profiling" is defined by reference to GDPR Article 4(4) (Article 3(52)). Default: treat it as high-risk |
Problem statement
| Measure | Value |
|---|---|
| Applications a month | 4,800 (12 x 400) |
| Read by a person | 240 (12 x 20), 5% |
| Declined with no human reading | 4,560, 95% |
| Target | A written classification, no decline without a logged human decision, and a measured miss rate per role |
Constraints
- Fixed: reading every CV costs 400 x 2 minutes = 800 minutes (13.3 hours) a role.
- Fixed: only criteria published in the job advert may be used.
- Limit: the form collects no group data, so group comparisons need separate, lawfully held data (a legal question).
Why the traditional approach fell short
Reading every CV does not scale. A keyword filter drops "K8s" when the advert says "Kubernetes". A model trained on past hires can copy the past: recital 57 warns that such systems "may perpetuate historical patterns of discrimination, for example against women, certain age groups, persons with disabilities, or persons of certain racial or ethnic origins or sexual orientation".
Options considered
| Option | Verdict | Reason |
|---|---|---|
| A. Application form with must-have questions and a recruiter skim (non-AI) | Kept for roles under 50 applicants | Visible rules, no model; whether a rules-only filter is an "AI system" (Article 3(1)) is for counsel |
| B. Keep the hidden auto-filter | Rejected | No classification, no human decision on 95% of declines |
| C. Treat as high-risk: ranking with evidence, recruiter decides, sample check, logs, monitoring | Accepted | Keeps the time saving and puts a person on every decline |
| D. No ranking, only per-criterion summaries | Rejected for volume | Costs the full 13.3 hours; does not settle Article 6(3) |
Solution architecture
Figure 1. Where the human decision sits
How to read this: the system declines no one; only the Approval service produces a decision.
Figure 2. One role, from posting to decision
How to read this: the alt block is the checkpoint that tests the ranking before anyone is declined on its word.
How AI solved it, step by step
Step through the design, then switch to "Without the fix".
Under the hood
| Checkpoint | Here | Cost per role |
|---|---|---|
| Before the action | A recruiter reads all 400 | 800 minutes |
| By sampling | Top 40 plus 36 random: 76 CVs | 152 minutes (2.5 hours) |
| By escalation | "Unclear" flags and applicant requests for a person | Only the flagged CVs |
Article 14(4) lists what oversight must enable. Two items matter here: "to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)", and "to decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output".
The sample is a detector, not a proof. If 5% of the 360 others were suitable (18), a random 36 shows at least one with probability 1 - C(342,36) / C(360,36) = 0.857. See Under the hood: Human oversight checkpoints.
What went wrong / failure modes
| Failure | Mitigation |
|---|---|
| The recruiter accepts the ranked list unread | Hidden-rank sample; track how often decisions differ from the ranking |
| The model leans on a proxy not in the advert | Evidence quotes per criterion; the Policy engine rejects criteria not in the posted version |
| One group is advanced less often | Monthly comparison where lawful data exists; Article 10(2)(f) names "examination in view of possible biases" for data sets, so ask counsel how it applies here |
| A model update changes the order | Pin versions, rerun a labelled set, log the version |
| Applicants are not told | Article 26(11): deployers of Annex III systems that make or assist decisions about people "shall inform the natural persons that they are subject to the use of the high-risk AI system"; say so in the advert |
Results
All numbers are illustrative targets, not measurements.
| Measure | Before | Illustrative target |
|---|---|---|
| CVs read by a person, per role | 20 of 400 (5%) | 76 of 400 (19%) |
| Recruiter time, per role | 40 minutes | 152 minutes |
| Declines with no logged human action | 4,560 a month | 0 |
| Classification record | None; filed as "productivity" | Written, reviewed when the use changes |
| Chance the sample sees at least one missed suitable applicant | Not measured | 0.857 if 5% were missed |
Tradeoffs and when NOT to use this approach
- Sampling is not reading everyone. Whether regulators accept a sample as sufficient oversight is not answered by the text; ask counsel.
- Few applicants. Under about 50, read them all.
Design rules extracted
- Classify before you build: write down the Annex III area, the Article 6(3) argument and who is provider and deployer.
- A model may order work; only a person declines, and the log says who.
- Test the ranking with a blind sample before acting on it.
- Pin criteria and model versions, and log the evidence per criterion.
- Tell the people affected, and give them a route to a human.
Assumptions
All numbers are illustrative.
- Volume: 12 roles x 400 applications = 4,800 a month. Before: top 20 read, 12 x 20 = 240 (5%); 4,800 - 240 = 4,560 (95%) declined unread.
- Reading time 2 minutes per CV: 400 x 2 = 800 minutes = 13.3 hours; 20 x 2 = 40 minutes; 76 x 2 = 152 minutes = 2.5 hours.
- Review set: 40 + 36 = 76 per role, 76 / 400 = 19%. The sample is drawn from the other 400 - 40 = 360.
- If 5% of the 360 are suitable, 18 are; P(at least one in a random 36) = 1 - C(342,36) / C(360,36) = 0.857 (hypergeometric, computed with Python). The widen threshold of 3 is a team choice, not a regulatory figure.
- Quotes are from the Official Journal text and the Commission page in the note. The company and counts are invented.
Last verified: 2026-10-22