CCA Developer M1: MSO Foundations

Temperature is NOT the only source of non-determinism. Even with temperature=0, outputs can vary because:

• Hardware floating-point differences across server instances
• Model weight updates (new model versions change behaviour)
• Token budget pressure (a model close to its max_tokens limit may abbreviate)
• Prompt caching introduces subtle timing differences

Designing for non-determinism. Don't fight it — design around it:

• Define expected output properties in your eval, not exact strings
• Use structured output (JSON schema or tool use) to constrain format
• Test across multiple samples to catch edge-case behaviours

1.4 The Claude Model Family

As of 2026, the Claude model family has four tiers:

Model selection rule. Start with Sonnet 5. Move to Opus only if Sonnet fails your eval benchmarks on the specific task. Move to Haiku only if Sonnet passes but your cost/latency budget is not met.

Accessing models via API. Models are identified by their ID string. There are two kinds of IDs:

• Alias: claude-sonnet-5 — always resolves to the latest stable Sonnet 5 release. Automatic updates; use for development.
• Pinned: claude-sonnet-5-20251101 — resolves to a specific release. No surprise changes; required when you have a validated eval suite. See M5 for detailed versioning strategy.

1.5 Reasoning Modes

Claude has two reasoning modes: basic (default) and extended thinking.

Basic mode. The model produces a direct response. Reasoning happens implicitly in the generation process. Fast, suitable for the vast majority of tasks.

Extended thinking. The model generates a scratchpad of reasoning tokens (visible to you as a thinking block in the response) before producing the final answer. This significantly improves accuracy on:

• Multi-step math and logic problems
• Complex code review that requires holding many constraints simultaneously
• Long-horizon planning tasks

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    thinking={
        "type": "enabled",
        "budget_tokens": 10000   # max tokens for the thinking block
    },
    messages=[{"role": "user", "content": "Prove that sqrt(2) is irrational."}]
)

for block in response.content:
    if block.type == "thinking":
        print("Thinking:", block.thinking)
    elif block.type == "text":
        print("Answer:", block.text)

When to use extended thinking.

1.6 Prompting Modes

Three prompting modes form a progression of specificity:

Zero-shot. Instructions only, no examples. Works well when the task is clearly specified and the model's training already covers the pattern.

{"role": "user", "content": "Classify this review as positive, neutral, or negative: 'The delivery was late but the product is excellent.'"}

Few-shot (one-shot / multi-shot). Instructions plus one or more demonstrations. Use when:

• Output format matters and words alone don't fully specify it
• The task requires a style or reasoning pattern the model needs to see

system = """Classify product reviews.

Example:
Review: "Broke after a week, terrible quality."
Classification: negative

Example:
Review: "Decent for the price, but nothing special."
Classification: neutral
"""

Guidelines for few-shot examples:

• Quality over quantity: three excellent, diverse examples beat ten mediocre ones
• Cover the hard cases: easy examples add no information; edge cases teach the boundary
• Match the format exactly: if production input has typos, include a typo in an example
• Keep examples adjacent to the instruction they illustrate

Multi-turn. Conversation history passed as the messages array. The model has access to the full prior exchange. Use for interactive assistants; be deliberate about how much history you retain (context management — see M2).

1.7 SDK vs REST vs Batch

When to use the official SDK.

The Python and TypeScript SDKs add:

• Automatic retry with exponential backoff (rate limit errors handled for you)
• Streaming response parsing (event-by-event without manual SSE parsing)
• Type-safe request and response objects
• Helper utilities (token counting, tool schema validation)

Use the SDK for all production integrations. Use raw REST only when the SDK is unavailable (unusual runtime, edge function with size limits).

# SDK — retries, types, streaming built in
client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

Synchronous vs streaming vs batch — choose based on the user experience.

Streaming example (Python SDK):

with client.messages.stream(
    model="claude-sonnet-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarise this report: ..."}]
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Batch example — submit 500 documents:

import anthropic

client = anthropic.Anthropic()

requests = [
    {
        "custom_id": f"doc-{i}",
        "params": {
            "model": "claude-haiku-4-5",
            "max_tokens": 256,
            "messages": [{"role": "user", "content": f"Summarise: {documents[i]}"}]
        }
    }
    for i in range(500)
]

batch = client.messages.batches.create(requests=requests)
print(batch.id)  # poll this ID to get results when ready

M1 Checkpoint

Key facts to remember from this module:

• 1 English word ≈ 1.3 tokens; non-English and code are more expensive
• Context = input + output combined — always budget both
• temperature=0 ≠ fully deterministic — hardware drift and model updates still cause variation
• Start with Sonnet 5 — move up/down based on eval results, not intuition
• Extended thinking costs output tokens — enable it only when basic mode fails your eval
• Streaming is default for user-facing output; Batches API for bulk non-urgent work