Skip to content
    All guides

    Forward Deployed Engineer

    Extended Interview Bank 3: Security, Reliability, Behavioral

    Six more interview questions: a leaked key, safe logging, a provider outage, test versus production quality, delivering bad news and deciding with thin data.

    By Published 14 min read

    Share

    This lesson is part of FDE Interview: Technical Question Banks. Enroll for free, then sign in, to read it in full.

    Enroll to unlockAlready enrolled?

    What you'll learn: how to answer six more interview questions (a committed API key, what to log, a model provider outage, a gap between test results and production, delivering bad news and deciding with incomplete information) with a first move, a number, a bound and a reflection.

    At a glance

    Figure 1. The first move for each of the six questions

    flowchart TD accTitle: First move for each question in this bank accDescr: Six questions in three groups. For security, revoke the key first and log metadata, not content. For reliability, fail fast and degrade, and treat the test set as unrepresentative until sampled. For behavioral, bring options with bad news and name what would show a decision was wrong. A["Six questions"] --> B["Security"] A --> C["Reliability"] A --> D["Behavioral"] B --> B1["Q1: revoke and rotate first"] B --> B2["Q2: log metadata, not content"] C --> C1["Q3: fail fast and degrade"] C --> C2["Q4: sample real traffic"] D --> D1["Q5: bad news with options"] D --> D2["Q6: name what would prove you wrong"]

    How to read this: every branch ends in a first action, because interviewers listen for what you do in the first minute.

    • Optional extension: six shorter questions in the same shape as the reference pages.
    • Security and reliability sketches cite primary sources and use illustrative numbers with the arithmetic shown.
    • The two behavioral sketches are labelled composites, not anyone's real experience. Bring your own stories.
    • The capstone has no retries, no fallback and an in-sample evaluation, and the sketches say so where it matters.
    • Nothing here describes how any company interviews.

    Scenario

    Composite scenario, not a real company. A candidate with a background in back-office integrations is preparing for the last round of a forward deployed role. The practice customer is Lumen Industrial Supply, the fictional distributor. A friend plays an engineering manager who will ask three technical questions and three behavioral ones, and the candidate wants one structure that works for both kinds. All numbers are illustrative.

    What you will be able to do

    • Handle a leaked credential in the right order and say why rewriting history is not the first step.
    • Decide what a service should log, what it should not, and how you would debug without content.
    • Use Little's law to show why fail-fast timeouts limit the damage of a provider outage.
    • Tell a behavioral story with a measure, a decision and a reflection, without inventing details.

    Prerequisites

    Core ideas

    In the interview: "What would you do first?" For security and reliability, the first action must stop the damage or limit it; analysis follows. For behavioral questions, the first thing you say is the situation's stakes, not the background.

    Q1. A teammate committed a model provider's API key to the shared repository. What do you do in the first hour?

    What it assesses. Whether you act in the right order: stop the key from working, then find the scope, then clean up. Rewriting history first is the usual wrong answer.

    Answer framework.

    1. Treat the key as exposed and revoke or rotate it first.
    2. Check the provider's usage since the commit for anything unexpected.
    3. Put the new key where the service reads it at runtime, not in the repository.
    4. Remove it from history if you need to, and tell the people who cloned the repository.
    5. Add a check that blocks the next one, and write a short note.

    Strong answer sketch.

    First I rotate the key, because once it is revoked it cannot be used, and that may be sufficient on its own. GitHub's guidance says to revoke or rotate a secret as the first step, and warns that the commit can still be reachable from clones, forks and pull requests, so cleaning history does not make the old key safe. I would then read the provider's usage for the time since the commit, and treat any unfamiliar usage as an incident with a cost. The new key goes in the deployment's secret store. Last I would add secret scanning to the pipeline so the same mistake fails a build.

    One common mistake. Rewriting the history, force-pushing and declaring it fixed while the original key still works.

    One follow-up. The key was shared by three services. How do you rotate it without an outage?

    Q2. An engineer wants to log every prompt and answer to debug a customer's bad answers. What do you say?

    What it assesses. Whether you balance the need to debug against the duty not to copy customer text into places with weaker access control.

    Answer framework.

    1. Agree the debugging need: what question do the logs have to answer?
    2. Separate metadata from content.
    3. Log metadata by default; capture content only on a narrow, time-limited path.
    4. Restrict who reads content logs and how long they live.
    5. Make sure error messages do not leak details to callers.

    Strong answer sketch.

    Log by default Capture only on a narrow path Do not log
    Trace id, caller role, counts, timings Request text and retrieved passages for flagged requests Tokens, passwords and keys
    Token counts, cost, outcome Model output for a reviewed sample Whole request headers
    Ids of the documents used Stage-by-stage timings for a flagged request Personal data that is not needed

    Most debugging questions are "which documents did it use and what did each stage do?" and ids and timings answer those. I would log metadata by default, add a trace id, and keep request text out. If we need content, I would capture it for a flagged sample with redaction, short retention and a smaller group of readers. The OWASP guidance on sensitive information disclosure (LLM02:2025, still LLM02 in the 2026 edition) asks for least privilege on sensitive data and no leaking through error messages. It does not address logging directly, so the log rule is a design choice built on those principles. The capstone's logs follow the metadata rule and drop fields named for the request text and the bearer token.

    One common mistake. Turning on full prompt logging "temporarily" in production, then leaving it on.

    One follow-up. A customer reports a wrong answer from last Tuesday and you only have metadata. What can you still do?

    Q3. The model provider is down for thirty minutes during business hours. What does the user see, and what do you do?

    What it assesses. Whether you design the failure path: fail fast, degrade usefully, tell the truth, and avoid making a struggling dependency worse.

    Answer framework.

    1. Detect: set a short timeout and watch the error rate.
    2. Fail fast so requests do not pile up.
    3. Degrade: show the part that does not need the model, such as search results.
    4. Say what is happening and signal it to callers.
    5. Recover gently: bounded retries, not a flood.

    Strong answer sketch.

    By Little's law, requests in flight equal arrival rate times time spent waiting. At 5 requests a second and a 30 second timeout, 150 requests sit open; with a 3 second timeout it is 15. So I would shorten the timeout and open a breaker after repeated failures. The user would see the search results with a note that generated answers are paused. For callers, HTTP defines 503 for a server temporarily unable to handle a request, with an optional Retry-After header. When the provider returns, I would retry with limits, not all at once. The capstone returns a generic 500 and has no fallback; it is a reference service, not a production design.

    One common mistake. Retrying every failure at once, which multiplies the load on a provider that is already struggling.

    One follow-up. The fallback search page must still apply access control. How do you make sure it does?

    Q4. Your evaluation scores are stable, but users say quality has dropped. What do you check?

    What it assesses. Whether you doubt your own test set and know how to compare it with real traffic without overreading small samples.

    Answer framework.

    1. Assume the test set is unrepresentative until sampled traffic says otherwise.
    2. Sample recent real requests within your privacy rules and label them by hand.
    3. Compare the failure rate with its uncertainty, not as a single number.
    4. Group the failures: new topics, new document types, changed data.
    5. Add the failures to the set, and keep part of it unseen when tuning.

    Strong answer sketch.

    The capstone's golden set is 36 questions written by the same author who tuned the pipeline, so even its stable score only says nothing regressed on those questions. Suppose 3 of the 36 fail: 8.3%. I sample 50 recent requests, label them, and 9 fail: 18%. With 50 samples the uncertainty is wide, roughly plus or minus 11 points at 95%, so I would take 100 more before concluding anything, and in the meantime read the nine failures, which usually show a pattern such as a document type the golden set never had.

    One common mistake. Tuning the pipeline on the new failures and then reporting the same set as the accuracy.

    One follow-up. You are not allowed to read production requests. What proxies do you have?

    Q5. Tell me about a time you had to give bad news to a sponsor.

    What it assesses. Whether you bring facts, options and a recommendation when the news is bad, and how early you speak.

    Answer framework.

    1. Situation: what was promised and what the stakes were.
    2. Task: your responsibility in the decision.
    3. Action: when you told them, what evidence you brought, what options you offered.
    4. Result: what was decided and how it turned out.
    5. Reflection: what you would say earlier next time.

    Strong answer sketch. This is a composite and not a real person's experience.

    A pilot's acceptance bar was 85% on 40 agreed questions, which is 34 passes. The latest run passed 28, or 70%. The engineer told the sponsor the same day, with the 12 failing questions grouped by cause, and offered three options: a two-week extension, cutting one document source from scope, or accepting 70% for a smaller audience. The sponsor chose the extension. Looking back, the engineer would have shared the failing run a week earlier and would add a mid-pilot checkpoint to every plan.

    Composite example, not a real person. Numbers are illustrative.

    One common mistake. Delivering the bad news with no options, so the sponsor must solve it alone.

    One follow-up. The sponsor says the bar was a guess and asks you to lower it. What do you do?

    Q6. Tell me about a decision you made with incomplete information.

    What it assesses. Whether you can decide without perfect data, limit the cost of being wrong and notice quickly if you were.

    Answer framework.

    1. Situation: what was unknown and why waiting had a cost.
    2. Task: the decision and its deadline.
    3. Action: how you made the choice reversible or small.
    4. Result: what you learned and when you learned it.
    5. Reflection: what signal you would add earlier.

    Strong answer sketch. This is a composite and not a real person's experience.

    A team had to choose between waiting three weeks for a supplier's API and building a nightly export in two days. The engineer chose the export, wrote the decision down with the signal that would reverse it ("if the data is more than a day stale in more than 1 answer in 20, switch"), and told the customer the data would be dated on screen. In week four the stale rate was 1 in 50, so the export stayed. The engineer would add the stale-rate measure on day one next time.

    Composite example, not a real person. Numbers are illustrative.

    One common mistake. Describing a decision as brilliant without saying what would have shown it was wrong.

    One follow-up. What if the signal had shown you were wrong in week two? What would have been lost?

    Worked example

    Assumptions: arrival rate, timeouts and sample counts are illustrative. Little's law: the average number of requests in the system equals the arrival rate times the average time each spends there.

    Timeout Arrival rate Requests open during the outage
    30 seconds 5 per second 5 x 30 = 150
    3 seconds 5 per second 5 x 3 = 15

    Fail-fast cuts the pile-up tenfold. In a thirty-minute outage (1,800 seconds) at 5 requests a second, 9,000 requests arrive. With a breaker open, most of them get the search fallback at once, instead of waiting out a timeout.

    For Q4, the sample uncertainty uses a normal approximation:

    • Standard error = sqrt(0.18 x 0.82 / 50) = 0.054, and 1.96 x 0.054 = 0.106, so 18% plus or minus about 11 points.
    • The range is 7.4% to 28.6%, and the golden-set rate (8.3%) sits just inside it, so the sample does not prove a drop.
    • It is a reason to sample more and read the failures.

    Predict first. A teammate suggests retrying each failed model call three times immediately. During the outage, how many calls does the provider receive for 9,000 requests, and is that a good idea?

    Answer

    Each request makes 1 call plus 3 retries, so 4 calls: 9,000 x 4 = 36,000 calls to a provider that is already failing. Immediate retries multiply load without adding successes during an outage. Bounded retries with waits, and a breaker that stops calling for a while, protect the provider and your own capacity.

    Practice task

    Spot the flaw. What is wrong with this log line from a service that handles customer requests?

    Pseudocode.

    log.info("discovery request=%s headers=%s", body, request.headers)
    Answer

    It writes the customer's full request text and every header, including the Authorization header with the bearer token, to the application log. Anyone who can read the log can read customer text and replay tokens. Log ids, counts and timings instead, and drop sensitive fields by name.

    Then write the replacement line using only: trace id, caller role, length of the request, outcome and elapsed time.

    Explain it back. In three sentences, explain to a customer's security lead why you log metadata by default. A good explanation names what the logs are for, what you leave out and why, and how content can still be captured when it is needed.

    Exercise

    Write one behavioral story of your own in five lines, one per step of the structure, using a real event. Add one number and one decision.

    Acceptance criteria: the situation names the stakes; the action says what you decided and why; the result has a measure; the reflection names something you would do differently. Done when a friend can ask "what was your part, exactly?" and you can answer from the page.

    Hint 1: choosing the story

    Pick an event where you made a call under time pressure and something was at stake. A story where nothing could have gone differently has no decision to describe.

    Hint 2: finding the number

    Ask what you could count before and after: hours, failures, days saved, tickets. If there is no number, say what you could not measure and why.

    Common mistakes

    • Rewriting history before rotating a leaked key.
    • Logging full requests and headers for convenience.
    • Retrying a failing dependency immediately and at full volume.
    • Reporting a small sample's percentage with no sense of its uncertainty.
    • Bringing bad news without options.
    • Telling a behavioral story with no measure, or borrowing a story that did not happen to you.

    Knowledge check

    1. Why rotate a leaked key before cleaning the repository history?

    A revoked key cannot be used, and that stops the risk immediately. History cleanup is slower, and the commit may still be reachable from clones, forks and pull requests, so the old key stays dangerous until it is revoked.

    2. What is the difference between metadata and content logging, and which is the default?

    Metadata is trace ids, counts, timings, roles and outcomes; content is the text of requests, retrieved passages and answers. Metadata is the default because it answers most debugging questions without copying customer text into a log that more people can read.

    3. Why does a shorter timeout reduce the damage of an outage?

    By Little's law, requests in flight equal the arrival rate times the time each waits. Cutting the timeout from 30 to 3 seconds at 5 requests a second drops 150 open requests to 15, so threads, connections and memory are not exhausted while the dependency is down.

    4. Why is 18% on 50 sampled requests not proof that quality fell from 8.3%?

    A sample of 50 gives roughly plus or minus 11 points at 95% confidence, so the real rate could be near 8%. It is a reason to sample more and read the failures, but the interval is too wide to call a finding by itself.

    5. What makes bad news easier for a sponsor to act on?

    Early timing, evidence and options. A sponsor who hears the number the same day, sees the failures grouped by cause and can choose between extending, cutting scope or lowering the audience can decide in one meeting, while a bare problem forces them to invent the solution.

    You can now

    • Order the first hour of a leaked-credential response.
    • Decide what to log by default and what to capture on a narrow path.
    • Use Little's law to justify a fail-fast timeout.
    • Read a small-sample failure rate with its uncertainty.
    • Tell a behavioral story from your own work with a measure and a reflection.

    Further reading

    Last verified: 2026-11-02

    Next

    FDE scenario interview questions: four customer cases

    Enroll to keep reading

    Lesson 5 of 5 · FDE Interview: Technical Question Banks

    Six more interview questions: a leaked key, safe logging, a provider outage, test versus production quality, delivering bad news and deciding with thin data.

    In the rest of this lesson

    • Worked example
    • Practice task
    • Exercise
    • Common mistakes
    • Knowledge check
    • You can now

    Enrolling is free. It unlocks every lesson in FDE Interview: Technical Question Banks, tracks your progress across devices and keeps your quiz scores. The newsletter is optional.

    EnrollSign in

    Ask about this article

    Get answers grounded in this post. AI-generated — based on this article, and may be imperfect.

    FDE Interview: Technical Question Banks· Page 5 of 5
    PreviousExtended Interview Bank 2: Python, RAG and Agents
    Scaled AI Weekly

    Enjoyed this? Get more like it every Monday.

    Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.

    Get new articles by email

    Free. We email a confirmation link first, and you can unsubscribe from any email.

    Join engineers building production AI systems

    Comments

    Back to all guides