Skip to content

    Forward Deployed Engineer

    Workflow Limits, Step Log and Resumable State in Python

    Bound a tool-calling workflow with a step cap and a wall-clock limit, record each step in a log, and save the state as JSON so a stopped run can resume.

    By Published 8 min read

    Share

    This lesson is part of Forward Deployed Engineer: Agents, Tools and Guardrails. Enroll for free, then sign in, to read it in full.

    What you'll learn: how the capstone's WorkflowRunner stops a loop with a step cap and a time limit, what the step log holds and why it holds no private data, and how to save a stopped run as JSON and resume it without repeating work.

    At a glance

    Figure 1. The runner, its two budgets and what it records

    The workflow runner with its budgets and step logBudgets to Orchestrator (stops the loop); Orchestrator to Model gateway (plans); Orchestrator to Tool server (validated call); Orchestrator to Step log (summary only)Budgets: max steps, secondsBudgetsmax steps,secondsOrchestrator: WorkflowRunnerOrchestratorWorkflowRunnerModel gateway: plans next stepModel gatewayplans next stepTool server: read-only toolsTool serverread-only toolsStep log: one record per stepStep logone record perstepstops the loopplansvalidated callsummary only

    How to read this: the two budgets are checked before every step, and the step log keeps a short summary of each step.

    • A model loop needs two limits: a step cap and a time limit.
    • The step cap bounds steps, not model calls. A repaired reply makes one step cost two calls.
    • The time limit is checked between steps. It does not interrupt a running step.
    • The step log holds summaries, not arguments or results, so it is safe to keep.
    • Observations plus the log are the whole state, so a stopped run can be saved and resumed.

    Scenario

    Composite scenario, not a real company. At Lumen Industrial Supply, a fictional distributor, a sales engineer pastes in a customer request: vibration monitoring on 40 pumps, a humid plant, an existing OPC UA setup. The assistant searches the catalog, reads product records and checks protocols.

    In a pilot, one request makes the planner repeat searches with slightly different wording. Nobody notices until the model bill arrives. The team wants a hard stop, a record of each step, and a way to continue a run that stopped early. All numbers are illustrative.

    What you will be able to do

    • Explain what max_steps and timeout_s bound, and what they do not.
    • Read a step log and say which step failed and why, without any private data in it.
    • Predict the status and step count of a run from its limits and a fake clock.
    • Save a stopped run as JSON and resume it so finished calls are not repeated.

    Prerequisites

    Core ideas

    Two limits, checked before each step

    Limit Parameter and default Checked What it does not do
    Steps max_steps=14 By the for loop Does not count model calls
    Wall clock timeout_s=5.0 At the top of each step Does not cancel a running step

    Anthropic's Building effective agents says it is common to include stopping conditions, such as a maximum number of iterations, to keep control. The capstone adds a time limit too, because a step cap alone does not protect a user from a slow tool.

    Verified: capstone/fde-product-discovery-copilot/app/workflow/runner.py

        def run(self, requirements: Requirements) -> WorkflowResult:
            result = WorkflowResult("max_steps")
            started = self.clock()
            seen: set[str] = set()
            for step in range(1, self.max_steps + 1):
                if self.clock() - started > self.timeout_s:
                    result.status = "timeout"
                    return result
                tick = self.clock()
                try:
                    decision = complete_structured(
                        self.provider, plan_messages(requirements, result.observations, self.registry.describe()), PlanDecision
                    )
                except StructuredOutputError:
                    result.status = "planner_error"
                    return result
                if decision.action == "final":
                    result.status = "completed"
                    return result
                if decision.action == "request_approval":
                    self._request_approval(decision, step, tick, result)
                    return result
                self._run_tool(decision, step, tick, seen, result)
            return result  # status stays "max_steps"
    NOTE

    The status starts as max_steps. If the loop ends without a return, that is the answer. Every other exit sets its own status first.

    The clock is a parameter (clock=time.monotonic by default). Python's documentation says time.monotonic() cannot go backwards and is not affected by system clock updates, and that only the difference between two calls is meaningful. So the runner subtracts two readings, and a test can pass a fake clock.

    Predict first. A fake clock moves 2 seconds every time anyone reads it. The runner has timeout_s=3, max_steps=14, and a planner with ten different searches queued. How many steps run, and what is the status?

    Answer

    One step, status timeout. The clock reads 0 (start), 2 (the check, not above 3), 4 (step start) and 6 (step end). At the next check it reads 8, and 8 is above 3, so the loop stops. The limit is checked only between steps, so one step overshoots a small budget.

    Figure 2. States of one run and the exits that limits add

    stateDiagram-v2 accTitle: States of one run and the exits that limits add accDescr: Each step first checks the clock. If the time limit is exceeded the run ends as Timeout. Otherwise the planner is asked. Two unreadable replies end the run as PlannerError. A final decision ends it as Completed. An approval request that is accepted ends it as AwaitingApproval, and one that is refused or unavailable ends it as Completed. A tool decision runs the tool and returns to the clock check, unless the step cap is reached, which ends the run as MaxSteps. [*] --> CheckClock CheckClock --> Timeout: elapsed above limit CheckClock --> Planning: within limit Planning --> PlannerError: two unreadable replies Planning --> Completed: final, or approval refused Planning --> AwaitingApproval: approval request accepted Planning --> RunningTool: tool RunningTool --> CheckClock: steps left RunningTool --> MaxSteps: step cap reached Timeout --> [*] PlannerError --> [*] Completed --> [*] AwaitingApproval --> [*] MaxSteps --> [*]

    How to read this: only Completed and AwaitingApproval are planned endings; the other three are limits or failures, and the run keeps what it learned.

    Steps are not model calls

    A reply that is not valid JSON is repaired once, and the repair is a second model call inside the same step. The worst case is two calls per step.

    Verified: capstone/fde-product-discovery-copilot/lessons/M4-L4/test_m4_l4_resume.py

    def test_each_step_may_cost_two_model_calls(ctx: ToolContext) -> None:
        """max_steps bounds steps, not model calls: a malformed reply is repaired once, and the repair is a second call."""
        texts = [t for i in range(3) for t in ("not json", _search(i))]
        provider = ScriptedProvider(texts)
        result = WorkflowRunner(provider, default_registry(), ctx, max_steps=3).run(Requirements())
        assert result.status == "max_steps" and len(result.steps) == 3 and len(provider.calls) == 6

    With the default max_steps=14, budget for up to 28 model calls, not 14.

    The step log: summaries, never arguments or results

    Figure 3. What a run records

    erDiagram accTitle: What a workflow run records accDescr: A workflow result has one status, many step records and many observations. A step record holds the step number, action, tool name, success flag, a short summary and elapsed milliseconds. An observation holds the tool name, the arguments, the success flag and either the data or the error. Tool steps have both a step record and an observation, matched by position. WORKFLOW_RESULT ||--o{ STEP_RECORD : logs WORKFLOW_RESULT ||--o{ OBSERVATION : "feeds the next prompt" STEP_RECORD |o--o| OBSERVATION : "same step, matched by position" WORKFLOW_RESULT { string status } STEP_RECORD { int step string action string tool bool ok string summary float elapsed_ms } OBSERVATION { string tool json args bool ok json data_or_error }

    How to read this: the log (step records) is for people and metrics, the observations are for the model, and the two are separate on purpose.

    Verified: capstone/fde-product-discovery-copilot/app/workflow/runner.py

    @dataclass(frozen=True)
    class StepRecord:
        step: int
        action: str
        tool: str | None
        ok: bool
        summary: str
        elapsed_ms: float
    
    
    @dataclass
    class WorkflowResult:
        status: Status
        observations: list[dict[str, Any]] = field(default_factory=list)
        steps: list[StepRecord] = field(default_factory=list)
        approval: ApprovalRequest | None = None

    A step record is frozen, so nothing edits history. For a tool step the summary is one of "2 products", "ok" or "error: ...". Arguments and tool data live only in the observations, because they can hold what the customer typed. You can ship the log to a dashboard; observations need a retention policy.

    WARNING

    An error summary includes the error text, and a tool may echo an argument in it. The capstone's LookupError says No product LUM-XX-000 found., which repeats the SKU. Decide per tool whether that is acceptable.

    Three exits write no record of their own: a final decision (status completed), timeout and planner_error. That is why the worked example below asks the planner seven times and logs six steps.

    Resumable state

    The state of a run is its observations plus its step log. The planner prompt is built from the observations, so a restarted run plans the next step, not the first. The capstone runner does not offer this, so the lesson code adds it as a subclass.

    Verified: capstone/fde-product-discovery-copilot/lessons/M4-L4/m4_l4_resume.py

    def result_to_json(result: WorkflowResult) -> str:
        """Only the parts that are plain data. A pending approval is not saved here (see M4-L5)."""
        return json.dumps({"status": result.status, "observations": result.observations, "steps": [asdict(s) for s in result.steps]})
    
    
    def result_from_json(text: str) -> WorkflowResult:
        raw = json.loads(text)
        return WorkflowResult(raw["status"], raw["observations"], [StepRecord(**s) for s in raw["steps"]])
    
    
    def call_key(tool: str, args: dict[str, Any]) -> str:
        return f"{tool}:{json.dumps(args, sort_keys=True)}"  # the same key the runner uses for duplicates

    Two rules decide what resumes. Only max_steps, timeout and planner_error are resumable: a completed run has nothing left, and an approval is a separate flow. And max_steps is a total across runs, so resuming with the same cap does nothing.

    Worked example

    Lumen's request has a scripted plan of six tool steps (a search, two spec reads, two protocol checks and one gateway spec read) and then a final decision. The mock planner is a rule-based stand-in, so a real model's plan will vary.

    Assumptions: illustrative; max_steps=2 for the first run, the default for the second, and the saved JSON is stored between runs.

    Run Planner asked Tools run Step log
    Uninterrupted 7 times 6 steps 1 to 6, completed
    First run, max_steps=2 2 times 2 steps 1 and 2, max_steps
    Resume from the saved JSON 5 times 4 steps 3 to 6, completed

    The resumed run ends with the same six tools in the same order as the uninterrupted one. Restarting from scratch would have repeated two tool calls and two planner calls.

    At the customer: a buyer asks, "What if it gets stuck?" A good answer names the two limits and what they do not cover (a running step is not cancelled), says what the user sees (the partial result and the status), and says the log holds no customer text.

    Build it

    Run the lesson tests. They use a fake clock and scripted planners, so they need no network and no key.

    cd capstone/fde-product-discovery-copilot
    ./.venv/Scripts/python -m pytest lessons/M4-L4 -q

    Verified: capstone/fde-product-discovery-copilot/lessons/M4-L4/m4_l4_resume.py

    class ResumableRunner(WorkflowRunner):
        def resume(self, requirements: Requirements, saved: WorkflowResult) -> WorkflowResult:
            if saved.status not in RESUMABLE:
                raise ValueError(f"A run that ended as '{saved.status}' has nothing left to resume.")
            result = WorkflowResult("max_steps", list(saved.observations), list(saved.steps))
            seen = {call_key(o["tool"], o["args"]) for o in result.observations}
            started = self.clock()
            for step in range(len(result.steps) + 1, self.max_steps + 1):
                if self.clock() - started > self.timeout_s:
                    result.status = "timeout"
                    return result

    The rest of the loop is a copy of run. Two things are rebuilt from the saved state: the finished calls (seen) and the step number to continue from. The clock restarts, so the time limit applies to each run on its own.

    NOTE

    Copying the loop is a teaching shortcut. In your own service, change run to accept the saved result.

    In test_the_time_limit_is_checked_between_steps, change timeout_s to 9. Predict the step count first (it is 2: the clock reads 8 at the second check, under 9, and 14 at the third).

    Exercise

    Make the time limit cover the whole workflow, not each run. Change resume to subtract the seconds already spent in the saved steps (the sum of elapsed_ms, divided by 1000) from the budget.

    Acceptance criteria:

    • With timeout_s=5 and a saved run whose steps took 4 seconds in total, a resume allows one more second.
    • A fake clock drives the test; nothing sleeps.
    • The existing tests in the folder still pass.

    Done when a new test shows a resumed run ending as timeout after the saved steps used most of the budget.

    Hint 1: where the budget is used

    Leave the clock alone and change what you compare it with.

    Hint 2: what to subtract

    Work out the seconds already spent before the loop and compare the clock with what is left. Build the saved run from StepRecord values you write by hand.

    Common mistakes

    • Counting model calls as steps. One step can cost two calls, so estimates based on max_steps are too low.
    • Expecting the time limit to cancel a slow tool. Give tools their own timeouts.
    • Logging arguments and tool data in the step log. They can hold customer text. Log summaries only.
    • Resuming with the same cap. max_steps counts across runs, so nothing runs. Raise it deliberately.

    Knowledge check

    1. The default is `max_steps=14`. What is the largest number of model calls one run can make, and why?

    Twenty-eight. Each step asks the planner once, and asks again with a repair message if the reply is not valid JSON, so 2 calls times 14 steps is 28.

    2. A tool takes 40 seconds and `timeout_s` is 5. What does the user wait for, and what is the status?

    The user waits the full 40 seconds, then the status is timeout. The limit is only checked at the top of the next step, so a running step is never cancelled.

    3. Why does the step log keep `summary` and not the arguments or data?

    Arguments and results can contain what a customer typed or private product data. A short summary such as "2 products" shows where a run went wrong and is safe to send to a dashboard.

    4. Which statuses can be resumed, and why not `completed`?

    max_steps, timeout and planner_error can, because the run stopped before the planner said it was done. A completed run has nothing left to do, and awaiting_approval waits for a person, which is a different flow.

    5. After resuming, why does the duplicate-call check still work?

    The resume code rebuilds the set of finished calls from the saved observations, using the runner's key: the tool name plus the arguments as sorted JSON. Without that rebuild the set would be empty and an old call would run again.

    You can now

    • State what the step cap and the time limit bound, and what they leave open.
    • Predict a run's status and step count from its limits and a fake clock.
    • Explain why the step log holds summaries only.
    • Save a stopped run as JSON and resume it without repeating finished calls.

    Further reading

    Last verified: 2026-11-02

    Next

    Human approval for sensitive actions

    Enroll to keep reading

    Lesson 4 of 8 · Forward Deployed Engineer: Agents, Tools and Guardrails

    Bound a tool-calling workflow with a step cap and a wall-clock limit, record each step in a log, and save the state as JSON so a stopped run can resume.

    In the rest of this lesson

    • Worked example
    • Build it
    • Exercise
    • Common mistakes
    • Knowledge check
    • You can now

    Enrolling is free. It unlocks every lesson in Forward Deployed Engineer: Agents, Tools and Guardrails, tracks your progress across devices and keeps your quiz scores. The newsletter is optional.

    Ask about this article

    Get answers grounded in this post. AI-generated — based on this article, and may be imperfect.

    Scaled AI Weekly

    Enjoyed this? Get more like it every Monday.

    Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.

    Get new articles by email

    Free. We email a confirmation link first, and you can unsubscribe from any email.

    Join engineers building production AI systems

    Comments