What you'll learn: the three common places a Python AI service can sit in a company that runs Spring Boot and Node.js services, how to choose between them, and what stays true whichever you pick: one contract, identity checked at the edge, a deadline on every call.
At a glance
Figure 1. The three placements of one Python AI service
How to read this: a request enters through the gateway and reaches the Python service by route A, B or C; only the Python service talks to the model provider.
- A Python AI service is a service like any other: private, one contract, one trace id.
- A routes to it directly, B puts a Spring Boot service in front, C a Node.js service.
- Pick by who owns the business rules and the user's session, not by language.
- A hop costs milliseconds; a model call costs seconds.
- Identity is checked at the edge and again in the service.
Scenario
Composite scenario, not a real company. Lumen Industrial Supply, the fictional distributor, runs a Spring Boot quote service and a Node.js portal backend behind one API gateway. The product discovery copilot is a new Python service. The portal team wants to call it from Node.js, the quote team wants it in the quote flow, and the platform team asks why it cannot have its own public route. You have a week to propose where it goes. Numbers are illustrative.
What you will be able to do
- Compare three placements by owner, identity, failure handling and change cost.
- Choose a placement for an estate and defend it in three sentences.
- List what every placement must provide: contract, edge identity check, deadline, trace id.
Prerequisites
FastAPI: an AI-ready service, Human approval for sensitive actions and Cloud platforms for AI.
Core ideas
Three placements
Predict first. The Python service must be called by both the portal (Node.js) and the quote flow (Spring Boot). Which placement needs the fewest changes to the existing services? Decide before you open the table.
Answer
Placement A changes neither service: both call the same gateway route. B and C each add a client to one codebase, and the second caller needs its own. A's price is an AI route open to everyone who reaches the gateway.
| A. Direct route behind the gateway | B. Called from Spring Boot | C. Called from Node.js | |
|---|---|---|---|
| Who calls Python | The gateway, for any client | One Spring Boot service | One Node.js service |
| Who owns the user session | Gateway and Python service | The Spring service | The Node service |
| Where business rules stay | In the callers, which treat Python as a lookup | In Spring, next to quotes and prices | In Node, next to the portal |
| Good when | Several clients need the same capability | The output feeds a transaction the Java team owns | The output is part of a portal screen |
| Watch for | An AI route open to every client | Each caller writing its own client | Hand-rolled retries, no typed contract |
| Change cost | Low for callers | A new client per caller | A new client per caller |
My default is A for the first release, with the gateway doing authentication and rate limits, and B or C only when a caller must combine the result with its own data. I would not choose A when the output is joined with records the gateway cannot see, or security forbids AI routes on the shared gateway.
What stays the same in every placement
Figure 2. Choosing a placement
How to read this: the questions run from how many callers, to who owns the joined data, to what the gateway may do.
Every placement needs the same four things:
- One contract. FastAPI generates an OpenAPI schema and serves it at
/openapi.json, with interactive documentation at/docs(FastAPI first-steps page). Hand it to the Spring and Node teams. - Identity checked at the edge, and again in the service. The next lesson covers how.
- A deadline on every call. The service must not outlast the caller's wait.
- A trace id the caller passes inward and finds in the service logs.
Calling it from Spring Boot and from Node.js
As of 2026-11-02 the Spring Framework reference lists RestClient (synchronous), WebClient (non-blocking, reactive) and RestTemplate, which it marks as deprecated in favour of RestClient from Framework 7.0. RestClient delegates to a request factory, so set the timeout explicitly.
Node.js has a global fetch, based on undici and no longer marked experimental since v21.0.0 (Node.js globals page). In Python, httpx times out after 5 seconds of network inactivity by default (httpx documentation), shorter than many model calls. State the deadline in every client.
FastAPI's CLI server ignores X-Forwarded-* headers unless you list trusted proxies with --forwarded-allow-ips. The documentation's "*" trusts all incoming addresses; do not copy it into a service others can reach.
Worked example
Assumptions: illustrative. The gateway adds 5 ms, a Spring Boot or Node.js service 8 ms, the Python pipeline 120 ms, and one model call takes 3000 ms.
- Placement A: 5 + 120 + 3000 = 3125 ms.
- Placement B or C: 5 + 8 + 120 + 3000 = 3133 ms.
- The extra hop adds 8 ms, which is 8 / 3125 = 0.26 percent of the total.
So the hop is not the reason to choose; ownership and failure are. In A the gateway can only return an error when the AI service is down.
Deadlines shrink going inward: the gateway waits 30 s, the calling service at most 25 s, and the Python service caps its model call at 10 s. Each layer can then answer before the one outside gives up.
At the customer: the platform lead asks, "why not expose it publicly?" A good answer: the service holds partner-only data behind a role check, so every request needs a verified caller; a public route adds a second identity path, and the gateway already gives rate limits and logging.
Practice task
Place the service in Lumen's estate, or one you know.
- Draw the gateway, your services and the Python service; mark A, B or C on each route.
- Fill the table's rows, including what the caller shows when the service is down.
- Write the four shared requirements as a pull-request checklist.
Decision exercise. The quote team wants the copilot in a quote screen, joined with the customer's price list. The portal team wants it on the search page, shown unchanged. Choose a placement for each and justify it.
Model answer
Quote screen: B, because the answer is joined with prices the Spring service owns, and the quote team decides what to show on failure. Search page: A, because the portal shows the answer as is. If the shared gateway may not carry AI routes, the search page falls back to C.
Exercise
Write a one-page placement note for Lumen: a choice for each caller, the contract file you hand over, and each layer's deadline.
Acceptance criteria:
- Each caller has a placement and one sentence of reasons.
- It names where identity is checked and three deadlines, each shorter than the one outside.
- It says what each caller shows when the Python service is down.
Done when a Spring Boot developer who has never seen the capstone could write the client from your note and the OpenAPI file.
Hint 1: start from the failure
Write what the user sees when the Python service is down. That shows who owns the call.
Hint 2: nest the deadlines
Start from the gateway's wait; subtract a margin at each layer inward.
Explain it back. Tell the platform lead in three sentences why the Python service stays on the private network. A good one says who can reach it, what identity check it still performs, and what the gateway adds.
Common mistakes
- Choosing by language comfort. "Our team writes Java" justifies B only if Java owns the joined data.
- A public route "for the pilot". Pilots get remembered. Keep it private.
- Trusting forwarded headers. A header a client set is a claim, not a fact.
- Deadlines that grow inward. The inner call keeps working after the user has gone.
Knowledge check
1. Why does placement A change the fewest services, and what does it cost?
Both callers use the same gateway route, so neither codebase gains an integration. The cost is an AI route open to every client that reaches the gateway, so authentication, rate limits and role checks must be right.
2. In the worked example, why does an extra hop not decide the choice?
The hop adds 8 ms to a 3125 ms request, 0.26 percent: the model call dominates.
3. When would you choose B over A?
When the result must be combined with data or a transaction a Spring Boot service owns, such as a price list in a quote. That service can decide what the user sees when the AI service fails; a gateway cannot.
4. Why should deadlines shrink going inward?
An inner call that outlasts the outer one keeps spending model tokens after the outer layer has given up. Shorter inner deadlines let each layer answer or fail cleanly first.
5. What does the OpenAPI file give you across Spring Boot and Node.js?
FastAPI produces it from the code, so it matches the service. Both teams can generate or check clients from it instead of guessing fields and status codes.
You can now
- Compare placements A, B and C by owner, identity, failure handling and change cost.
- Choose a placement for a caller and defend it in three sentences.
- List the four things every placement must provide.
- Nest deadlines from the gateway down to the model call.
Further reading
- FastAPI: First steps, the OpenAPI schema and docs.
- FastAPI: Behind a Proxy, forwarded headers and trusted proxies.
- Spring Framework: REST clients,
RestClient,WebClientand request factories.
Last verified: 2026-11-02
Next
Next lesson: Passing identity: authn, authz and tenant isolation