Composite scenario, not a real company. Assembled from common industry patterns; every number is illustrative and its assumptions are listed at the end.
This case belongs to Agents, Tools and MCP.
Context
A software company with 300 engineers builds an internal assistant for questions such as "why did the checkout build fail after PR 4812, and is there a ticket for it?". The assistant reaches the company's systems through two MCP servers (Model Context Protocol, a standard for exposing tools to model applications): a repo server with search_code, read_file, get_diff and list_commits, and a ticket server with search_tickets and get_ticket. Each is owned by a different team. The redesign below replaces whole-object tools such as get_diff with scoped ones such as list_changed_files and get_file_diff.
Problem statement
Pilot traces showed that a typical question needed about 6 tool calls (7 model calls, the last one writing the answer), and the tools returned whole objects: a large diff, a ticket with its entire comment thread. The average result was 18,000 tokens. With a 6,000-token base (instructions plus tool definitions) and an assumed 128,000-token context window, the final call of a typical session carried 114,000 tokens. A question that needed a seventh tool call would carry 132,000 and fail, because that is over the window. Even below that point, answers began to miss details from early in the session, and a typical question processed 420,000 input tokens across its seven model calls.
Goals: median session under 75,000 input tokens, no context-length failures, and every answer citing its file or ticket.
Constraints
- Fixed: the assistant is read-only, and repo access must follow the asking engineer's own permissions.
- Fixed: tickets can contain customer data; only approved model providers may see them.
- Fixed: the MCP servers belong to other teams.
- Limit: a 30-second answer target for typical questions.
Why the traditional approach fell short
Code search finds strings but cannot connect a failing build, the diff that caused it and the ticket that describes it, so engineers hop between three tools. Wikis, runbooks and a scripted bot cover the questions someone expected. A model with tools can connect the evidence, but only if it fits in its context.
Options considered
| Option | Idea | Verdict |
|---|---|---|
| A. A dashboard linking build, PR and ticket (non-AI) | One page per failing build with the diff summary and the matching tickets | Kept as the baseline and as a link target in answers |
| B. Assistant with scoped, paginated, truncating tools and a session budget | Smaller tools, capped results, cursors, a token budget per session with a defined behaviour at the limit | Accepted |
| C. Keep the broad tools and buy a bigger context window | Move the failure point further out | Rejected: the same session still processes about 5.7 times the tokens (see Results) and the window only delays the failure |
Solution architecture
Figure 1. Assistant components with two MCP tool servers
How to read this: tool results pass through the Orchestrator before the model sees them, so that is where the size limit and the budget are enforced.
Figure 2. One question, with the budget-exhausted path
How to read this: steps 5 to 14 repeat for each tool call, and the alt block is the decision the Orchestrator makes after every result.
How AI solved it, step by step
- The engineer asks the question. The Orchestrator identifies the user, so the repo server applies that user's permissions.
- At session start the Orchestrator lists the tools and sends only the definitions it needs.
- The model calls
list_changed_filesfor the PR. The result is capped, shows 25 of 40 files and carries a cursor. - The model picks the files that touch checkout and calls
get_file_difffor one path. The server returns at most 2,000 tokens and says how much was cut and how to ask for the next page. - The model calls
search_ticketswith a limit of 5. Results use a concise format: title, id, status and one line, not the thread. - After each result the Orchestrator adds the result's tokens to the session total and compares it with the budget.
- The model answers citing the file and ticket it used. If the budget is spent first, the Orchestrator returns what was read and asks the engineer to narrow the question.
Under the hood
MCP messages are JSON-RPC 2.0. In the specification revision 2025-11-25, a session starts with an initialize request carrying the client's protocol version, capabilities and clientInfo; the server answers with its own capabilities (for example tools with listChanged), and the client sends notifications/initialized. After that, tools/list returns the tool definitions and tools/call runs one, over stdio or Streamable HTTP. The latest revision on the specification site as of 2026-10-21, 2026-07-28, drops the handshake: every request carries the protocol version, and its compatibility page says a modern client fails against a legacy server. Pin the revision.
What matters for context: the tool definitions go into every model request, and Anthropic's documentation counts the tools parameter, tool_use blocks and tool_result blocks as input tokens. The MCP tools page (revision 2026-07-28) defines no maximum result size, so limiting is your job. It also says servers SHOULD return tools in a deterministic order, which helps prompt-cache hit rates. Anthropic's "Writing effective tools for agents" says Claude Code restricts tool responses to 25,000 tokens by default, recommends pagination, filtering, range selection and truncation with sensible defaults, and shows a concise response of 72 tokens against 206 for the detailed one. Lifecycle details are in the lesson's MCP section.
What went wrong / failure modes
- A diff or thread floods the context. Mitigation: a default cap of 2,000 tokens per result, cursors, and a concise format by default.
- Truncation hides the key line. Mitigation: the result says what was cut and how to ask for it; Anthropic's post suggests steering agents to "many small and targeted searches".
- Redundant calls. Mitigation: the Orchestrator returns the earlier result and counts the call against a limit of 12.
- Tool list growth. Every added server adds definitions to every request. Mitigation: a cap on tool count.
- Permission leak. A shared service token exposes private repos. Mitigation: the server uses the asking user's identity and the Policy engine checks the repo argument.
- Protocol drift. A library upgrade changes the MCP revision. Mitigation: a contract test (
tools/listand onetools/call) in CI. - Customer data and injected text in tickets. Mitigation: read-only tools, approved providers only, ticket text treated as untrusted.
Results
These are illustrative targets, not measurements. For a typical session of seven model calls, the input tokens processed are the sum of 6,000 + (k - 1) x r over k = 1 to 7. With r = 18,000 that is 7 x 6,000 + 18,000 x 21 = 420,000; with r = 1,500 it is 42,000 + 31,500 = 73,500, a ratio of 420,000 / 73,500 = 5.7. A session that needs a seventh tool call would carry 6,000 + 7 x 18,000 = 132,000 tokens into the next call in the first design, over the 128,000 window, and 6,000 + 7 x 1,500 = 16,500 in the second. The second design meets the 75,000-token target.
Tradeoffs and when NOT to use this approach
- Smaller tools need more calls for broad questions, so latency rises while tokens fall.
- Truncation can drop the one detail that mattered; keep a cursor to the full text.
- Skip MCP for a single in-house tool; a direct function call is simpler (MCP vs function calling).
- Do not launch if repo permissions cannot be enforced per user.
Design rules extracted
- Size every tool result: a default cap, a cursor, a concise default format.
- Set a per-session token budget and decide in advance what the user sees at the limit.
- Say what was omitted inside the result so the model can narrow its next call.
- Run tools as the asking user, not as a shared service account.
- Pin the protocol revision and contract-test every server.
Assumptions
- All numbers are chosen for illustration: a 128,000-token window, a 6,000-token base, 7 model calls (6 tool results and the answer), an average of 18,000 tokens per result in the first design and 1,500 in the second.
- Input tokens for call k are 6,000 + (k - 1) x r; the model's own output tokens are ignored. Sums: 18,000 x (0 + 1 + ... + 6) = 18,000 x 21 = 378,000, plus 42,000, gives 420,000; 1,500 x 21 = 31,500, plus 42,000, gives 73,500.
- The 2,000-token cap, the 25-of-40 file page, the limit of 12 calls and the 75,000-token goal are illustrative choices.
- The company, its repo and ticket systems, and PR 4812 are invented.
Last verified: 2026-10-22