Skip to content

    Forward Deployed Engineer

    Extended Interview Bank 2: Python, RAG and Agents

    Six more interview questions on Python details, latency, chunking, retrieval choice, runaway agent loops and agent evaluation, with sketches and mistakes.

    By Published 13 min read

    Share

    This lesson is part of FDE Interview: Technical Question Banks. Enroll for free, then sign in, to read it in full.

    What you'll learn: how to answer six more technical interview questions (a Python default-argument trap, finding where latency went, choosing a chunk size, choosing a retrieval method, stopping a runaway agent loop, and evaluating an agent, not just its answers) with a structure, a number and a test you can name.

    At a glance

    Figure 1. Where the six questions sit in one request

    flowchart LR accTitle: The six questions placed along a request path accDescr: A request enters a Python service. Q1 is about Python code inside the service. Q2 is about timing each stage. Q3 and Q4 are about retrieval, chunking and the choice of search method. Q5 is the bounded agent loop. Q6 evaluates the whole path. A["Request"] --> B["Python service: Q1 language details"] B --> C["Stage timing: Q2"] C --> D["Retrieval: Q3 chunks and Q4 method"] D --> E["Agent loop: Q5 limits"] E --> F["Answer"] F -.-> G["Q6: evaluate outcome and path"]

    How to read this: Q2 and Q6 look at the whole path, and the other four each zoom into one box.

    • Optional extension: six more questions in the same shape as the reference pages, each shorter.
    • Two sketches use tested code from the capstone lesson folder; the rest use illustrative numbers with the arithmetic shown.
    • Technical answers are scored on four things: a measurement, a number, a bound and a test.
    • The capstone's retrieval is BM25 keyword search and its default provider is a rule-based mock, so no answer here claims behaviour of a real model.
    • None of this describes how any company interviews.

    Scenario

    Composite scenario, not a real company. A Node.js engineer is preparing for a technical round for a forward deployed role. The practice customer is Lumen Industrial Supply, the fictional distributor whose product discovery copilot is the course capstone. The round is one hour: two coding questions and four design questions. The candidate wants answers that name a measurement and a test, because those are what an interviewer can check. All numbers are illustrative.

    What you will be able to do

    • Explain the mutable default argument trap and the standard fix, and show it with a test.
    • Locate a latency regression by timing stages, and explain why the added stage is not always the slow one.
    • Choose a chunk size by testing against a question set, using a token budget to bound the choice.
    • State the limits of an agent loop and how you would evaluate its path, not only its final answer.

    Prerequisites

    Core ideas

    In the interview: "Walk me through how you would find out." Technical answers earn marks for a first measurement, a number that would change your decision, a bound on cost or time, and a test you would add so it stays fixed.

    Q1. What does this Python function return on its second call, and why?

    What it assesses. Whether you know that a default value is created once, when the function is defined, and can fix and test it.

    Answer framework.

    1. Predict the output before you explain it.
    2. State the rule: the default is evaluated once, at definition time.
    3. Say which defaults are risky: lists, dictionaries and most objects.
    4. Show the fix: None as the default and create the object inside.
    5. Name the test that would have caught it.

    Strong answer sketch. The function and its fix are both in tested code.

    Verified: capstone/fde-product-discovery-copilot/lessons/M8-L7/m8_l7_gotchas.py

    def add_tag_buggy(tag: str, tags: list[str] = []) -> list[str]:  # noqa: B006 (the bug is the point)
        tags.append(tag)
        return tags
    
    
    def add_tag(tag: str, tags: list[str] | None = None) -> list[str]:
        if tags is None:
            tags = []
        tags.append(tag)
        return tags

    The first call returns ["a"] and the second returns ["a", "b"], because the empty list was created once, when def ran, and every call that omits tags shares it. The Python tutorial gives the same warning. The fix is a None default and a fresh list inside the function. I would add a test that calls the function twice and checks the second result is ["b"], so a refactor cannot bring the bug back.

    One common mistake. Saying "Python passes by reference, so the list is shared". The cause is when the default is evaluated, not how arguments are passed.

    One follow-up. A FastAPI endpoint has def handler(items: list[str] = []). Is it affected, and what would you use instead?

    Q2. After you added retrieval, the endpoint's response time doubled. How do you find out why?

    What it assesses. Whether you measure before you guess, and whether you know that the stage you added is not always the stage that got slow.

    Answer framework.

    1. Reproduce the slowness with a fixed request set.
    2. Time each stage of one request, with the same clock and the same names.
    3. Compare before and after, stage by stage.
    4. Fix the stage that grew, not the stage you suspect.
    5. Keep the timing in production, so the next regression is visible.

    Strong answer sketch. A small timer, built with a fake clock in its test, adds up seconds per named stage.

    Verified: capstone/fde-product-discovery-copilot/lessons/M8-L7/m8_l7_gotchas.py

    class StageTimer:
        """Adds up the seconds spent in each named stage of one request."""
    
        def __init__(self, clock: Callable[[], float] = time.perf_counter) -> None:
            self._clock = clock
            self.stages: dict[str, float] = {}
    
        @contextmanager
        def stage(self, name: str) -> Iterator[None]:
            start = self._clock()
            try:
                yield
            finally:
                self.stages[name] = self.stages.get(name, 0.0) + (self._clock() - start)

    I would wrap each stage in the timer and print the table for the same ten requests before and after. The finally records a stage even when it raises. Then I read the table, and I do not assume it is the retrieval code. The worked example shows why.

    One common mistake. Profiling the new code because it is new, when the time went somewhere else.

    One follow-up. Your timings come from your laptop. How do you trust them for production?

    Q3. How would you choose a chunk size for the product documents?

    What it assesses. Whether you treat chunk size as an experiment against a question set, bounded by the context budget, and not as a number from a blog post.

    Answer framework.

    1. Write twenty to forty real questions, each with the passage that answers it.
    2. Try three sizes and one overlap, holding everything else fixed.
    3. Measure whether the right passage is in the top results and in the final context.
    4. Read the failures: split answers, lost headings, noise.
    5. Choose the smallest size that passes, and recheck when documents change.

    Strong answer sketch. The context budget limits the answer: the capstone's default budget is 700 tokens and it estimates 1.3 tokens per word, so about 538 words fit.

    Chunk window (words) Full chunks in the budget Risk
    60 8 The answer is split across chunks
    120 4 A balanced starting point
    240 2 Two chunks can fill the context with noise

    Arithmetic: 700 / 1.3 = 538 words; 538 / 120 = 4.5, so 4 full chunks; 538 / 240 = 2.2, so 2; 538 / 60 = 8.97, so 8.

    I would not pick from the table; I would run the three sizes against the question set and look at recall in the final context and at the failures. The capstone's window of 120 words is a maximum, because it splits at headings first, so real chunks are often shorter. I would pick the smallest size where the failures are not split answers, and rerun after any big document change.

    One common mistake. Choosing the size by feel and never reading the failures.

    One follow-up. Half your documents are tables. What changes?

    Q4. Keyword search, embeddings or both? How do you decide?

    What it assesses. Whether you pick a retrieval method from the failures in your own data, not from fashion, and whether you know what each method misses.

    Answer framework.

    1. Look at the real queries: exact identifiers, or natural language?
    2. Run the simplest method first and list what it misses.
    3. Add the next method only for a miss class you can name.
    4. Combine only if the combination beats each alone on your set.
    5. Count the cost: an index to build and update, a model to host, a result to explain.

    Strong answer sketch.

    Part numbers and protocol names such as an SKU are exact strings, and keyword search is good at them. The capstone's own README says a paraphrase with no shared words is missed. So I would start with keyword search, run the question set, and sort the misses. If many are paraphrases, embeddings are worth the extra index and the extra thing to keep fresh. If the misses are mixed, a hybrid that merges both ranked lists may help, and I would prove it on the same questions. If a small set of misses is left, a rule or a synonym list can be cheaper than a new component.

    One common mistake. Adopting embeddings because "everyone does RAG that way", with no list of misses.

    One follow-up. The customer asks for the keyword results and the embedding results side by side. Is that a good idea?

    Q5. Your agent calls the same search tool again and again. What do you do?

    What it assesses. Whether you bound a loop with several independent limits and treat a loop as a design problem, not as bad luck.

    Answer framework.

    1. Cap the number of steps, and cap the elapsed time.
    2. Detect an identical repeated call and skip or stop it.
    3. Make tool errors visible to the plan, and keep them short and safe.
    4. Record every step so you can see the loop.
    5. Decide what the user sees when a limit trips: a partial answer and a note, not silence.

    Strong answer sketch. The capstone uses three bounds: a step cap, a time limit and duplicate-call detection.

    Bound Capstone default What it stops
    max_steps 14 A plan that never says "final"
    workflow_timeout_s 5.0 seconds A slow loop
    Duplicate-call check the same tool and arguments twice A model repeating itself

    I would not trust one limit. The step cap bounds cost, the time limit bounds waiting, and the duplicate check catches the loop early. The runner checks time between steps, so one slow call can overshoot it; that is why each tool also needs its own timeout. When a bound trips, the answer says it stopped early and its confidence is capped at medium.

    One common mistake. Adding "do not repeat yourself" to the prompt and calling the loop fixed.

    One follow-up. How would you test that the step cap holds when the model never says "final"?

    Q6. How do you evaluate an agent, as opposed to a single answer?

    What it assesses. Whether you know that a correct final answer can hide a bad path, and that the path has its own measures.

    Answer framework.

    1. Define the outcome: was the task done correctly?
    2. Define path measures: steps used, tools called, repeated calls, permission errors, cost.
    3. Build cases that target the path, such as a request that tempts a forbidden action.
    4. Score outcome and path separately.
    5. Review a sample by hand each release.

    Strong answer sketch.

    A final answer can be right while the agent took twelve steps and tried a tool it should not have. So I would score two things. Outcome: did the recommendation match the expected product and cite a real source? Path: how many steps, were any calls repeated, did any call fail on permission? I would add cases built to tempt the agent into a forbidden action, because the path is where those show up. I would also read ten traces by hand each release, since scores miss problems nobody thought to measure.

    One common mistake. Reporting a single success rate for an agent whose steps nobody looked at.

    One follow-up. The success rate is the same after a change, but the average steps went from 4 to 9. Do you ship it?

    Worked example

    Assumptions: the timings below are illustrative seconds for one request, as the stage timer from Q2 would print them. Before the change there is no retrieval, and the compose model call sees a short prompt. After it, retrieved text is added to the prompt.

    Predict first. Before you read the table below, which stage do you expect to change most when you add retrieved text to a prompt, and why? Say it, then check the answer.

    Answer

    The stage that sends the prompt to the model. Model time grows with the amount of input and output text, so adding retrieved passages makes that call slower even though the retrieval stage itself is fast. A stage table lets you see this in one look, while guessing from the diff points at the new code.

    Stage Before After Change
    Extract (model call) 0.90 0.90 0.00
    Retrieve 0.00 0.08 +0.08
    Compose (model call) 1.30 3.40 +2.10
    Validate 0.00 0.02 +0.02
    Other 0.05 0.05 0.00
    Total 2.25 4.45 +2.20

    The total went from 2.25 to 4.45 seconds, a factor of 1.98, so "doubled". The retrieval code added 0.08 seconds. The compose call added 2.10 of the 2.20 seconds, which is 95% of the increase. The model call got slower because it now reads more text. The fix is to shrink what goes into it: a smaller context budget or fewer chunks. Optimising the retrieval code would recover at most 0.08 seconds.

    Practice task

    Run the timer and the tests, then use the timer on a stage of your own.

    1. From capstone/fde-product-discovery-copilot, run ./.venv/Scripts/python -m pytest lessons/M8-L7 -q and confirm the tests pass.
    2. Wrap two functions of your own, a fast one and a slow one, in StageTimer.stage and print timer.stages.
    3. Write one sentence on which stage you would investigate and why.

    Decision exercise. A customer's questions are 70% exact part numbers and 30% descriptions in natural language. Options: (A) keyword search only, (B) embeddings only, (C) keyword first, then add embeddings for the descriptions if a test shows misses. Choose and justify.

    Model answer

    Option C. Exact identifiers are where keyword search is strongest, so most of the traffic is already served. The natural-language 30% is where misses are likely, but the size of the gap is unknown until you measure it. Option B risks making the 70% worse and adds a component before any evidence. Choose A only if the test shows no meaningful misses.

    Exercise

    Write a stage-timing test for a function of your own that calls a slow dependency, using a fake clock so the test is instant and exact.

    Acceptance criteria: the test uses an injected clock and no sleep; it checks two stages with different durations; it checks a stage that raises is still recorded. Done when the test passes in under a second and fails if you rename a stage in the code.

    Hint 1: the fake clock

    Use an iterator of numbers and return the next one on each call: a start value then an end value per stage. The durations are the differences you chose.

    Hint 2: the raising stage

    Use try and except around the with block in the test, and check the recorded duration afterwards. The timer's finally block is what makes the record exist.

    Common mistakes

    • Explaining the default-argument bug as pass-by-reference.
    • Optimising the stage you added, not the stage that grew.
    • Picking a chunk size without a question set or without reading the failures.
    • Adding embeddings with no list of what keyword search missed.
    • One limit on an agent loop, usually a prompt instruction.
    • Judging an agent by its final answer only.

    Knowledge check

    1. Why does a mutable default argument keep its contents between calls?

    The default expression runs once, when the function definition is executed, and the same object is reused whenever the argument is omitted. Appending to it changes that one object, so each call sees the earlier calls' changes, which is why the usual fix is a None default.

    2. In the worked example, why is optimising the retrieval code a poor use of time?

    Retrieval added 0.08 of the 2.20 seconds of growth, about 4%. The compose call grew by 2.10 seconds because its prompt got longer, so the saving is in sending less text to the model, not in faster retrieval.

    3. How does the context budget limit the choice of chunk size?

    A budget of 700 tokens at about 1.3 tokens per word holds roughly 538 words. With 240-word chunks only two fit, so a poor top result wastes half the context; with 60-word chunks eight fit but an answer may be split across them.

    4. When would you add embeddings to a keyword search system?

    When the miss list from your own questions shows paraphrases that share no words with the documents, and the gain on those justifies a new index to build and keep fresh. If the misses are few or exact-identifier problems, a rule or synonym list is cheaper.

    5. Why score an agent's path as well as its final answer?

    A correct answer can come from a wasteful or unsafe path, such as many repeated calls or an attempt to use a forbidden tool. Those problems cost money and create risk, and they show up in the step count and the permission errors before they show up in accuracy.

    You can now

    • Explain the mutable default trap and write the test that catches it.
    • Time stages of a request and read a before-and-after table.
    • Bound a chunk-size experiment with a token budget.
    • Choose a retrieval method from a list of misses.
    • Name three limits on an agent loop and score both outcome and path.

    Further reading

    Last verified: 2026-11-02

    Next

    Extended interview bank, part 3 (security, reliability, behavioral)

    Enroll to keep reading

    Lesson 4 of 5 · FDE Interview: Technical Question Banks

    Six more interview questions on Python details, latency, chunking, retrieval choice, runaway agent loops and agent evaluation, with sketches and mistakes.

    In the rest of this lesson

    • Worked example
    • Practice task
    • Exercise
    • Common mistakes
    • Knowledge check
    • You can now

    Enrolling is free. It unlocks every lesson in FDE Interview: Technical Question Banks, tracks your progress across devices and keeps your quiz scores. The newsletter is optional.

    Ask about this article

    Get answers grounded in this post. AI-generated — based on this article, and may be imperfect.

    Scaled AI Weekly

    Enjoyed this? Get more like it every Monday.

    Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.

    Get new articles by email

    Free. We email a confirmation link first, and you can unsubscribe from any email.

    Join engineers building production AI systems

    Comments