Spring AI Getting Started: Your First ChatClient

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;

@Service
public class SummaryService {

    private final ChatClient chatClient;

    public SummaryService(ChatClient.Builder builder) {
        this.chatClient = builder
            // The system text sets stable behaviour for every call made through this client.
            .defaultSystem("You summarise customer notes for a claims team. "
                + "Be factual, use plain English and never invent details.")
            .build();
    }

    public String summarise(String note) {
        return chatClient.prompt()
            // {note} is a template placeholder, filled from param() below.
            .user(u -> u.text("Summarise this note in three short bullet points:\n\n{note}")
                        .param("note", note))
            .call()      // synchronous mode
            .content();  // runs the request and returns the reply text
    }
}

Expose it from a controller:

import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
class SummaryController {

    private final SummaryService service;

    SummaryController(SummaryService service) {
        this.service = service;
    }

    @PostMapping("/summaries")
    String summarise(@RequestBody String note) {
        return service.summarise(note);
    }
}

Run the application with the key in the environment and post a note with curl. You will get a few bullet points back. That is the complete loop: a Spring bean, a client, a model.

Prompts and responses

The prompt you build has parts, and the reply has more than text.

A prompt is a list of messages, each with a role. The system message sets behaviour and rules, the user message carries the request, the assistant messages are earlier model replies (relevant once you keep conversation history), and tool messages carry results of tool calls (lesson 3). defaultSystem(...) and .system(...) set the first; .user(...) sets the second.

content() returns just the text. When you need more, ask for the ChatResponse:

import org.springframework.ai.chat.model.ChatResponse;

ChatResponse response = chatClient.prompt()
    .user("Explain optimistic locking in two sentences.")
    .call()
    .chatResponse();

String text = response.getResult().getOutput().getText(); // the reply
var usage = response.getMetadata().getUsage();             // token counts reported by the provider
Integer inTokens = usage.getPromptTokens();
Integer outTokens = usage.getCompletionTokens();

Tokens are the unit that providers bill and limit. The reference notes that a token is roughly three quarters of a word, which is a useful rule of thumb for budgeting. Log getPromptTokens() and getCompletionTokens() from the first prototype so you can see what a feature costs.

For typed replies, entity(MyRecord.class) converts the answer into a Java type. That is the subject of lesson 3, so keep this one line in mind: you do not have to parse free text yourself.

Per-request options

Defaults come from properties. You can override them for one call. In the 2.0 line, options(...) takes a builder (not a built object), and a portable ChatOptions.builder() works with any provider:

import org.springframework.ai.chat.prompt.ChatOptions;

String shortAnswer = chatClient.prompt()
    .user("Name three causes of a database deadlock.")
    .options(ChatOptions.builder()
        .temperature(0.0)   // as deterministic as the provider allows
        .maxTokens(200))    // cap the reply length and cost
    .call()
    .content();

Use provider-specific options classes (such as OpenAiChatOptions or AnthropicChatOptions) only when you need a setting that the portable builder does not have.

Streaming at a high level

A model produces a reply token by token. Without streaming, the user waits for the whole answer; with streaming you send chunks as they arrive, which makes a chat window feel immediate.

import reactor.core.publisher.Flux;

Flux<String> chunks = chatClient.prompt()
    .user("Explain eventual consistency to a new developer.")
    .stream()      // streaming mode instead of call()
    .content();    // Flux<String> of text pieces

Two facts from the reference documentation decide how you build this. First, stream() needs the reactive stack, for example spring-boot-starter-webflux, while call() needs the servlet stack such as spring-boot-starter-web. Second, there is no built-in reactive conversion from a stream to a Java type yet, so streaming and typed output do not combine as neatly as call().entity(...). A WebFlux controller can return the Flux<String> with produces = MediaType.TEXT_EVENT_STREAM_VALUE. The details (Server-Sent Events, Spring MVC with SseEmitter, front-end handling, error cases) are in Streaming LLM responses in Spring AI.

Conversation memory in one paragraph

A model keeps no state between calls. If a user asks a follow-up question, you must send the earlier messages again. Spring AI packages that as ChatMemory plus a memory advisor:

import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;
import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;

ChatMemory chatMemory = MessageWindowChatMemory.builder()
    .maxMessages(10)          // keep a sliding window of recent messages
    .build();

ChatClient chat = builder
    .defaultAdvisors(MessageChatMemoryAdvisor.builder(chatMemory).build())
    .build();

String reply = chat.prompt()
    .user("And what about renters insurance?")
    // The conversation id is required with memory advisors; there is no default.
    .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
    .call()
    .content();

The documentation states that the conversation id is mandatory (omitting it throws IllegalArgumentException) and recommends deriving it on the server, not trusting a value from the browser. Storage defaults to an in-memory repository; the reference lists JDBC, Cassandra, Neo4j, MongoDB and Redis repositories for durable storage. Note that the JDBC repository page says tool-call messages are not stored. Lesson 7 returns to memory design.

Testing without calling a paid model

Most of your service is ordinary Java and should have ordinary tests. The model call is the only nondeterministic, billed part, so isolate it.

Level 1: a stub ChatModel in a unit test. ChatModel has a single abstract method, call(Prompt); call(String), getOptions() and stream(Prompt) have default implementations (the default stream throws UnsupportedOperationException, so override it if you test streaming). That makes a stub easy and keeps the test free of mock libraries.

import java.util.List;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.model.Generation;
import org.springframework.ai.chat.prompt.Prompt;

/** A fake model: records the prompt it received and returns a canned answer. */
class StubChatModel implements ChatModel {

    Prompt lastPrompt;
    private final String answer;

    StubChatModel(String answer) {
        this.answer = answer;
    }

    @Override
    public ChatResponse call(Prompt prompt) {
        this.lastPrompt = prompt;
        return new ChatResponse(List.of(new Generation(new AssistantMessage(answer))));
    }
}

import static org.assertj.core.api.Assertions.assertThat;
import org.junit.jupiter.api.Test;

class SummaryServiceTest {

    @Test
    void sendsTheNoteAndReturnsTheModelText() {
        StubChatModel model = new StubChatModel("- Water damage\n- Claim opened\n- Follow up Monday");
        // ChatClient.builder(model) is fine in a unit test; it skips Spring Boot's auto-configuration.
        SummaryService service = new SummaryService(ChatClient.builder(model));

        String result = service.summarise("Kitchen flooded on 3 May, claim opened on 4 May.");

        assertThat(result).contains("Water damage");
        // Assert on what we sent, not on what a model would say.
        assertThat(model.lastPrompt.getContents()).contains("Kitchen flooded");
    }
}

This tests your prompt construction, parameter binding and defaults. It cannot tell you whether a real model gives good answers; that is an evaluation problem (lesson 4 covers retrieval evaluation, lesson 7 covers answer quality).

Level 2: a local model for integration tests. An Ollama instance gives realistic behaviour with no per-call bill. The reference lists Testcontainers support in the spring-ai-spring-boot-testcontainers module, with service-connection factories for an OllamaContainer and for several vector stores. Local models are slower and weaker than hosted ones, so use them to test plumbing, not quality.

Level 3: a small, separate suite against the real provider. Keep a handful of end-to-end checks that run on demand or nightly, not on every commit, and cap their token budget. The reference documentation describes RelevancyEvaluator and FactCheckingEvaluator, which use a model as a judge and therefore need a model to run.

The anatomy of a Spring AI application

Keep the AI code in a small number of places so that it is easy to replace, test and observe.

The adapter is a plain Java class (such as SummaryService above, or a more specific ClaimsAssistant) with a method per use case. The rest of your system calls summarise(note), not chatClient.prompt().... That gives you one place for prompts, defaults, timeouts, caching and metrics, and one seam for tests.

Three use cases to try

1. Note summariser. Input: a long free-text note. Output: three bullets. Building blocks: defaultSystem, a user template, content(). Risk: the model adds details that are not in the note, so keep the instruction "never invent details" and review samples.
2. Ticket triage. Input: a support email. Output: a category and priority. Building blocks: a short prompt with the allowed categories listed, low temperature, later entity(...) for a typed result (lesson 3). Risk: categories drift, so validate the result against an enum.
3. Internal explainer. Input: a developer's question about the company's conventions. Output: a short answer with a streamed reply. Building blocks: stream(), memory with a server-side conversation id. Risk: the model does not know your conventions, which is the problem retrieval solves (lesson 4).

Choosing how to call

Pick the call style from the need, not the other way round.

Trade-offs

In practice: a summary endpoint for an operations team

The situation. An operations team wants a button in their internal tool that summarises long case notes. Engineering has two days to deliver a prototype, and finance wants to know what the feature will cost before it goes further.

Starting architecture. A Spring Boot service exposes case data. The tool shows raw notes, which can run to several pages.

What changes.

1. Add spring-ai-starter-model-openai (or the starter of the approved provider) and the BOM. Put the key in the secret store and bind it with ${...}.
2. Create SummaryService as above, with an explicit model name and max-tokens or maxTokens capped on the call.
3. Add a StubChatModel test that asserts the note reaches the prompt and that the default system text is applied.
4. Log getPromptTokens() and getCompletionTokens() with the case id (not the note text) so the team can estimate cost per summary.
5. Add a feature flag so the button can be switched off without a deploy.

Result. The team should see summaries appear in a few seconds, a unit test that runs in milliseconds with no network, and a first token-usage figure per request. Multiplying the average tokens by the provider's published price and the expected daily volume (assumed to be a few hundred requests) gives finance a cost estimate to challenge. Nothing else in the architecture moves: the new dependency is one outbound HTTPS call.

Architect's notes

• Wrap the client. Give each use case a method on a class you own. When the model, prompt or provider changes, one class changes.
• Pin the model name. A provider's default is a decision someone else makes. An explicit name in configuration is reviewable and reversible.
• Treat the key as a production secret. Use environment variables or a secret manager, rotate it, and give non-production environments a separate key with a spending cap if the provider offers one.
• Mind what you send. Notes may contain personal data. Decide before the prototype what may leave the process, and log identifiers instead of content. Spring AI's prompt and completion logging is off by default; the observability page warns that switching it on can expose sensitive data.
• Plan for slow and down. Model calls are slower than database calls and fail differently. Configure timeouts and decide what the user sees when the call fails.
• Prefer the simplest call. A one-shot call with a good system message solves many problems. Memory, retrieval and tools each add moving parts; add them when a failing example requires it.

Common mistakes

• Using ChatClient.create(chatModel) inside Spring beans. The reference notes this bypasses the auto-configured builder, so observability and customisers are ignored. Fix: inject ChatClient.Builder.
• Putting a key in application.yml. It ends up in Git. Fix: use ${OPENAI_API_KEY} style placeholders backed by the environment.
• Copying property names from an older article. The .options segment changed in 2.0. Fix: check the provider page for your version; an unknown property is silently ignored by Spring Boot, so verify the effective model with a log line or an actuator view.
• Forgetting the conversation id. Memory advisors throw without it. Fix: pass ChatMemory.CONVERSATION_ID on every call and derive it server-side.
• Streaming on the servlet stack. stream() needs the reactive stack. Fix: add WebFlux, or use call() for non-interactive work.
• Testing by calling the real model in every build. It is slow, costs money and fails randomly. Fix: stub the ChatModel for unit tests and keep a small, capped live suite.
• Leaving output length uncapped. A long reply costs more and takes longer. Fix: set max tokens in properties or per call.

Key takeaways

• A model starter plus a few properties gives you a ChatModel bean and an auto-configured ChatClient.Builder.
• ChatClient is the fluent layer: prompt().system(...).user(...).call().content().
• call() selects synchronous mode; the model runs when you ask for content(), chatResponse() or entity().
• Set the model name, key placeholder, timeout and token limit explicitly, and keep secrets out of the repository.
• stream() needs a reactive stack; use it for interactive replies and call() elsewhere.
• Memory needs a conversation id that your server controls.
• Isolate the model behind your own class and stub ChatModel in unit tests; keep a small, separate suite for quality against a real model.
• Log token usage from day one.

Practice exercises

Starter (about 30 minutes). Create the project from this lesson with one provider starter. Read the API key from an environment variable, set an explicit model name, and implement SummaryService with a POST /summaries endpoint. Send three different notes with curl and compare the results. Then change temperature in the options and observe the difference over five repeated calls.

Intermediate (about 2 hours). Add the StubChatModel test from this lesson and extend it: assert that the system text is the first message in lastPrompt.getInstructions(), and add a second test where the stub throws a RuntimeException, then make SummaryService return a friendly fallback message in that case. Next, log prompt and completion token counts from chatResponse() with the note length, and plot or tabulate them for ten notes of different sizes.

Advanced (half a day). Build the streaming version: add WebFlux, expose GET /summaries/stream returning Flux<String> as Server-Sent Events, and write a small HTML page that renders the text as it arrives. Then add MessageChatMemoryAdvisor with a conversation id derived from a signed-in user, and demonstrate that a follow-up question ("shorten that") works. Write down two ways memory could leak data between users and the check that prevents each.

Related reading

• Spring AI: Prompting, structured output and tool calling — the next lesson, typed results and tools.
• Spring AI: RAG and vector stores — grounding answers in your own documents.
• Spring AI: Production architecture — timeouts, retries, metrics and security.
• Streaming LLM responses in Spring AI — a deep dive on SSE with WebFlux and MVC.
• Spring AI enterprise integration guide — the single-post version of this ground.
• LLM observability guide — what to measure once the feature is live.

Official documentation

• Getting started — BOM, repositories and Spring Initializr.
• ChatClient API — builder, defaults, response types, streaming, memory advisors.
• Chat model API — ChatModel, Prompt and ChatResponse.
• OpenAI chat, Anthropic chat and Ollama chat — provider properties.
• Chat memory — ChatMemory, repositories and the conversation id.
• Testcontainers support — service connections for local containers.
• Upgrade notes — property and API changes between lines.