What you'll learn: how to answer an enterprise RAG design question in the six-step framework, where access control and conflict handling belong, and how to score the answer against a rubric.
At a glance
Figure 1. The worked answer: ingestion and query paths
How to read this: dashed arrows are asynchronous work (ingestion and tracing), solid arrows are calls the user waits for, and the access filter sits inside the Retriever, not after it.
- Access control is a filter inside the retrieval query, set by code from the user's identity.
- No evidence means no answer.
- Conflicting current documents are both named; superseded ones are filtered out.
- Quality is measured on a hand-labelled question set.
Scenario
Composite scenario, not a real company. Lumen Industrial Supply, a fictional distributor, has about 30 product documents (datasheets, FAQs, compatibility notes) and a catalog of 40 products. One document is partner-only, two documents disagree on one specification, and one datasheet has been superseded. Sales engineers, some employed by partners, ask product questions. The interviewer says: "Design the assistant, and tell me how you would know it works." All figures are illustrative.
What you will be able to do
- Answer an enterprise RAG prompt in six steps.
- Place access control inside the retrieval query and say why a post-filter fails.
- Name four failure modes and a check for each.
- Score the answer on five criteria.
Prerequisites
- The interview loop and an answer framework.
- Theory: Data and retrieval.
- Hands-on: Metadata filters, Context assembly and citations.
Core ideas
Step 1 and 2: clarify, then fix constraints
| Question | If the answer is... | Then the design... |
|---|---|---|
| Who may see which documents? | Public and partner users | Filter by role inside retrieval |
| What does a wrong answer cost? | A wrong spec reaches a quote | Refuse, and cite |
| Do documents change? | Yes, new revisions | Chunks carry a status |
| How many questions? | 1,200 a day | One synchronous path |
Step 3: the design
Ingestion splits documents at headings and tags each chunk with source, SKUs, status and visibility. The orchestrator decides before retrieval (Figure 2); the model gateway sees only retrieved text and the question.
Figure 2. What the orchestrator does with one question
How to read this: each diamond is a decision made by code before the model writes anything, and the first two exits stop before any answer is written.
Step 4: trade-offs
| Decision | Default | Switch when |
|---|---|---|
| Search | Keyword first: part numbers match literally | Wording differs: add vector search |
| Chunking | Split at headings | Answers missed in long sections: add overlap |
| Filter | Inside the query | Never after retrieval |
| Weak evidence | Refuse, using a coverage threshold | Users reject refusals: tune the threshold |
Step 5 and 6: failure modes and measurement
| Failure | What the user sees | Caught by |
|---|---|---|
| Partner-only text reaches a public user | A leaked answer | Access-control test cases, target zero leaks (capstone, 36 questions: 5 leaks without the filter, 0 with it) |
| Superseded datasheet cited | An outdated figure | A status tag excluded in retrieval, and a test on known revisions |
| Two documents disagree | Silent choice of one | A conflict case in the question set |
| No relevant document | A confident guess | Unanswerable questions, refusal rate |
The refusal check is a cheap heuristic: in the capstone it reaches 97.2% refusal accuracy on 36 questions, and one question that should be refused (q36) gets clarifying questions.
In the interview: say the failure table aloud and name the number you would watch first.