Composite scenario, not a real company. Assembled from common industry patterns; every number is illustrative and its assumptions are listed at the end.
This case belongs to Governance and responsible AI in practice.
Not legal advice. This page shows how engineers can read the text of Regulation (EU) 2024/1689 (Official Journal, 12 July 2024, ELI), checked on 2026-10-22. The Commission's AI Act page, shown as last updated 19 October 2026, says: "The transparency rules of the AI Act will come into effect in August 2026." It also describes later changes under an "AI Omnibus". Quotes below are the text as first published. Check the current text and that page, and ask counsel, before relying on any scope or date.
Context
- An online retailer runs an order-help chatbot: parcel status, returns, delivery questions. About 2,000 chats a day.
- The bot has a human first name, a photo and short typing delays. Nothing says it is an AI.
- It reads orders through a read-only tool. Refunds above a limit, payment terms and anything about credit go to staff.
- An internal check of 30 sampled transcripts found 0 that told the customer they were talking to an AI.
| Question | What the text says | Reading |
|---|---|---|
| Is it in an Annex III area? | Annex III lists areas such as employment and "essential private services", including "the creditworthiness of natural persons" (point 5(b)) | Not on these facts: order help is not a listed use. It would change if the bot decided, for example, eligibility for a credit product |
| Which rule applies? | Article 50(1): "Providers shall ensure that AI systems intended to interact directly with natural persons are designed and developed in such a way that the natural persons concerned are informed that they are interacting with an AI system, unless this is obvious from the point of view of a natural person who is reasonably well-informed, observant and circumspect, taking into account the circumstances and the context of use" | A transparency duty. The Article's heading is "Transparency obligations for providers and deployers of certain AI systems"; the Commission's page calls this level "Transparency risk", and "limited risk" is this course's shorthand, not an official heading |
| Who is the provider? | Article 3(3): a provider "develops an AI system ... or that has an AI system ... developed and ... puts the AI system into service under its own name or trademark" | Open: the retailer or its chatbot vendor, depending on who built it and under whose name it runs. A contract question for counsel |
Problem statement
| Measure | Value |
|---|---|
| Chats a day | 2,000 |
| Sampled transcripts that disclose AI | 0 of 30 |
| Proof that any customer was told | None; no event is logged |
| Target | Disclosure visible at the first message in 100% of sessions, every showing logged with its version, an alert on any session without one |
Constraints
- Fixed: Article 50(5): the information must be given "in a clear and distinguishable manner at the latest at the time of the first interaction or exposure", and "shall conform to the applicable accessibility requirements".
- Fixed: the friendly persona stays; the retailer will not drop the name.
- Limit: no change of chat vendor this year.
- Limit: the model must answer truthfully if a customer asks whether it is a person.
Why the traditional approach fell short
A help-centre page is not a conversation, so no one has to be told anything, but it cannot answer "where is my parcel". A button-driven bot is plainly automated, yet it fails on free-text questions. A line in the terms of service sits where nobody is reading when the chat opens, which is hard to square with "at the latest at the time of the first interaction".
Options considered
| Option | Verdict | Reason |
|---|---|---|
| A. No chatbot: order-status page and a contact form (non-AI) | Kept as the "no AI" path | Always available beside the chat; slower for customers |
| B. Disclosure in the terms only | Rejected | Not clear or distinguishable at the first interaction |
| C. Rely on "obvious" | Rejected | A human name, photo and typing delay make it less obvious, and the test is a legal standard you cannot measure |
| D. Header label, first-message disclosure, logged event, build test, handoff to a person | Accepted | Simple, visible, provable |
Solution architecture
Figure 1. Disclosure before the first model reply
How to read this: nothing reaches the Model gateway until the disclosure event exists, and the Eval service can prove it afterwards.
Figure 2. One chat session
How to read this: the alt block is the second safety: anything outside order help, or any request for a person, leaves the bot.
How AI solved it, step by step
Step through one session, then switch to "Without the fix".
Under the hood
Article 50(1) leaves one door open: "unless this is obvious from the point of view of a natural person who is reasonably well-informed, observant and circumspect". An architect cannot measure that standard, so the design does not use it. It states the fact in words at the first message.
The event is small: session id, disclosure version, time, channel. The version points to a line in the system card, the short record of the bot's intended purpose, its scope fence (no credit or eligibility decisions) and the exact disclosure text. If the scope ever grows, the card is where the classification is revisited. See Under the hood: Model and system cards and Under the hood: Audit logging that survives an audit.
What went wrong / failure modes
| Failure | Mitigation |
|---|---|
| The label is small or scrolls away | Persistent header label, first message, contrast and screen-reader checks |
| A widget release drops the disclosure | Build test on the first message; daily sessions-versus-events check |
| The server logs "sent" but the client never rendered it | The client confirms the render, not the server |
| The persona says "yes, I am a person" when asked | Instruction forbids it; a set of 50 "are you a human?" prompts runs on every release |
| Scope creeps into credit or eligibility | Policy engine blocks it; reclassify before any such feature |
| A voice or email channel is added | Each channel gets its own disclosure and event |
| Article 50(2) (machine-readable marking of synthetic text) | Open: whether it reaches this retailer depends on its role. Ask counsel; this page designs no marking |
Results
All numbers are illustrative targets, not measurements.
| Measure | Before | Illustrative target |
|---|---|---|
| Sessions showing the disclosure at the first message | 0 of 30 sampled | 2,000 of 2,000 a day |
| Sessions with no logged event | Not measurable | 0, alert on any |
| Log volume | None | 146 MB a year |
| "Are you human?" prompts answered truthfully | Not tested | 50 of 50 per release |
Tradeoffs and when NOT to use this approach
- Customer reaction. A disclosure may change how customers behave; measure it, do not assume.
- Scope. If the bot starts deciding eligibility or credit, this page no longer applies: see the credit case.
- Other Article 50 duties. Emotion recognition, deep fakes and published text have their own paragraphs (50(3) and 50(4)).
Design rules extracted
- Say it in words at the first message; do not rely on "obvious".
- Make the disclosure a gate: no event, no model reply.
- Log the version shown, and reconcile sessions against events daily.
- Fence the scope in a policy layer and write it in the system card.
- Always leave a route to a person.
Assumptions
All numbers are illustrative.
- 2,000 sessions a day: 2,000 x 365 = 730,000 events a year; at 200 bytes each, 730,000 x 200 = 146,000,000 bytes = 146 MB.
- The internal check of 30 transcripts, the 50-prompt test set and the disclosure wording are invented.
- Quotes are from the Official Journal text and the Commission page in the note. The retailer and its vendor are invented.
Last verified: 2026-10-22