Back to Hub
Certification Study Notes

Claude Platform & API

Production engineering on the Claude Messages API: choosing a model and effort level, request shape and roles, explicit criteria and few-shot examples, the tool-use loop and its pairing rules, stop_reason handling, streaming events, structured outputs and schema design, validation and retry loops, adaptive thinking, context windows, compaction and prompt caching, preserving facts in long sessions, Message Batches, multi-pass review, retriable vs. terminal errors, evals and cost levers, human review and confidence calibration, deployment platforms and data residency, and security for apps that read untrusted content.

Author: Michaël Bettan
Track: Anthropic / Applied AI
Test Yourself: Claude API Practice Quiz →
00

Overview & Mental Model

Definition

The Claude API is a stateless HTTP interface. The core endpoint, POST /v1/messages, takes the whole conversation (system prompt, alternating user/assistant turns, tool definitions) and returns one assistant message made of typed content blocks plus a stop_reason and token usage. Official SDKs (Python, TypeScript, Go, Java and others) wrap the same API and add retries, streaming helpers and types.

What your code owns

The API remembers nothing between calls and never runs your tools. Your application owns history (what to resend, prune or compact), tool execution (and authorization), branching on stop_reason, retries, validation of outputs, and cost/latency instrumentation. Almost every exam-style defect is one of these responsibilities done wrong.

Mental model

Treat every response as a state transition: read stop_reason first, then decide whether to show text, run tools, continue, compact, fall back or stop. Code that reads content[0].text without checking stop_reason is the root of most production bugs.

01

Build the request

Model ID, max_tokens, system, messages, tools, optional thinking/effort and output format.

02

Call (sync, stream or batch)

Pick the transport by who is waiting and how much work there is.

03

Branch on stop_reason

end_turn, tool_use, max_tokens, refusal, and the rest.

04

Append & loop

Append the assistant turn unchanged, add tool results, repeat until done.

01

1. Model Family, Effort & Choosing a Model

TierProfileTypical fitWatch out
HaikuFastest, lowest cost per tokenHigh-volume classification, routing, extraction, real-time triage, sub-agent workersValidate on your eval set before trusting it on nuanced steps
SonnetBalanced capability, speed and costDefault starting point for most apps and agents, coding, tool useDon't upgrade the whole pipeline when only one step fails
OpusMost capable, slowest, highest costHardest reasoning, long multi-step agent work, planning, review of critical outputPaying top-tier prices on steps a smaller tier passes

Two independent levers

  • Model tier sets the capability ceiling and the price per token.
  • Reasoning depth is a per-request setting: adaptive thinking plus output_config.effort (low, medium, high, xhigh, max; the accepted set varies by model).
  • Tune them separately. A smaller tier at higher effort can beat a larger tier at low effort on some tasks, and vice versa; only an eval tells you which.

Selection rule & Planner → Executor routing

  • Start on the balanced tier with the least reasoning that passes.
  • Move individual steps up or down only on eval evidence; route easy vs. hard inputs to different tiers.
  • Planner → Executor tier routing (Spec-Driven pattern): Use Opus for the read-only architectural specification and ambiguity-resolution pass, then start a fresh context window on Sonnet with only the finalized spec for implementation and tool-heavy execution (and Haiku for bounded read-only search workers).
  • Treat every model or effort swap as a release: pinned ID, baseline comparison, rollback target.
  • Turn thinking off or effort down on high-volume classifiers: thinking tokens are billed as output.

Sampling parameters on current models

On Claude 4.7 and later, setting temperature, top_p or top_k to any non-default value returns HTTP 400. Steer behavior with prompting, examples, structured outputs and effort instead. Even on older models where temperature: 0 is accepted, output was never fully deterministic: test properties, not exact strings.

Common trap

"Use the best model everywhere" is still a tier decision, just the most expensive one. And "set temperature to 0 for consistency" fails twice: it 400s on the newest models and never guaranteed identical text anyway.

02

2. Messages API: Request Shape and Roles

FieldRequired?What it does
modelYesModel ID. Pin a fixed ID in production (see §17).
max_tokensYesCap on output tokens for this response. Does not create context room or fix validation errors.
messagesYesConversation as user / assistant turns; content is a string or an array of blocks.
systemNoTop-level system prompt (string or text blocks) for instructions that apply to the whole conversation. Newer models also accept mid-conversation role: "system" messages inside messages, placed after a user turn, to change instructions later without invalidating the cached prefix (check per-model support).
tools / tool_choiceNoTool definitions and how Claude may use them (§4).
thinking / output_configNoAdaptive thinking, effort, and format for structured outputs (§7, §9).
streamNoServer-sent events instead of one JSON body (§6).
stop_sequencesNoCustom strings that end generation (stop_reason: "stop_sequence").
inference_geoNoFirst-party data residency: "global" (default) or "us" (§17).
Input content blocks
text, image, document, tool_result (in user turns); prior text, tool_use, thinking, redacted_thinking (in assistant turns you echo back).
Response object
id, role: "assistant", content[], stop_reason, stop_sequence, usage (input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens).
Transport choice
User waiting + short answer → sync. User waiting + long answer → streaming. Many concurrent in-app calls → async client. Bulk offline work → Message Batches.
System Prompt AltitudeWhat it looks likeFailure mode / why
Too specific (brittle state machine)Hardcoded if/else trees, exact regexes, rigid step-by-step tool sequences in proseBreaks on the first edge case; move deterministic rules into code, hooks, or schema validators
Goldilocks zone (heuristics & 4-step framework)Clear role/goal, tool-selection heuristics, and a 4-step operating loop (Explore → Plan → Confirm on ambiguity → Act & Verify)Gives the model strong guardrails while preserving flexibility to recover from unexpected tool outputs
Too vague (assumed shared context)"Be a helpful assistant and follow best practices"Forces the model to guess domain rules, tool boundaries, and stopping criteria
import anthropic
client = anthropic.Anthropic()          # reads ANTHROPIC_API_KEY from the environment

resp = client.messages.create(
    model=PINNED_MODEL_ID,
    max_tokens=1024,
    system="You are a support assistant for Acme. Answer only from the provided policy.",
    messages=[
        {"role": "user", "content": "Can I return an opened item?"},
    ],
)
if resp.stop_reason == "end_turn":
    answer = next(b.text for b in resp.content if b.type == "text")

Common trap

Prefilling the final assistant turn to force a format is rejected with a 400 on Claude 4.6 and later. Use structured outputs (§7) instead. Also, content[0] is not always text: with thinking on, the first block is usually a thinking block. Select blocks by type.

03

3. Explicit Criteria and Few-Shot Examples

Why it matters

In review, triage and extraction pipelines, precision (how many reported items are real) decides whether people trust the output. Vague instructions produce noisy output; confidence wording does not fix it. Define what qualifies, then show it.

Vague instructionWhy it failsExplicit replacement
"Check that comments are accurate"Everything slightly imprecise gets flagged, so false positives pile up"Flag a comment only when the behavior it claims contradicts what the code does"
"Be conservative" / "Only report high-confidence findings"Doesn't define the boundary; the model's notion of "confident" is unchanged, so precision barely movesCategorical report / skip lists: report bugs, security flaws, data loss; skip style, naming, and patterns used consistently in the repo
"Rate the severity"Labels drift between runs and reviewersA severity rubric with one concrete code example per level (critical / major / minor)

Precision and trust

  • One noisy category makes users ignore the accurate categories too. Measure precision per category, not just overall.
  • If one category is mostly noise, temporarily disable it (stop reporting it) while you rewrite its criteria and re-measure on an eval set; re-enable it once precision recovers.

Few-shot (multishot) examples

  • The most reliable fix when detailed instructions still give inconsistent format or judgment.
  • Anthropic's prompting guidance: make examples relevant, diverse, and wrapped in <example> tags (inside <examples>); 3–5 examples usually work best.
  • Examples teach a judgment that generalizes to new cases; a rule list matches only what you enumerated.
  • Keep examples varied: near-identical examples get copied on surface features (length, wording, field order).
Use few-shot to…What the examples show
Handle ambiguous cases (which tool, escalate or not, is this a real gap)The chosen action and why it beats the plausible alternative
Fix output formatThe exact finding shape: location, issue, severity, suggested fix
Cut false positivesContrast pairs: an acceptable pattern (not reported) next to a genuine issue (reported)
Reduce extraction hallucinationCorrect extraction from varied layouts (inline citations vs. bibliography, informal amounts like "a couple of cups"), including an explicit null when a field is absent
<examples>
  <example>
    <code>if user.is_admin or True:</code>
    <finding>{"location": "auth.py:42", "issue": "condition is always true, so the admin check is bypassed",
              "severity": "critical", "suggested_fix": "remove 'or True'"}</finding>
  </example>
  <example>
    <code>for i in range(len(items)):   # idiom used across this repo</code>
    <finding>none: style preference consistent with the codebase, not reported</finding>
  </example>
</examples>

Common trap

Fixing a noisy reviewer by adding "only report findings you are very confident about". It doesn't define what counts. Write categorical criteria, add contrast examples, and measure precision per category.

04

4. The Tool-Use Loop

01

Define tools

name, description (what it does, when to use it and when not to), input_schema.

02

Claude requests

Response has tool_use blocks (each with an id) and stop_reason: "tool_use".

03

Your code executes

Authorize, validate inputs, run. Add a human gate before irreversible actions.

04

Return results

Append the assistant turn unchanged, then one user turn with a tool_result per call.

Pairing ruleWhat breaks if you ignore it
Every tool_use gets a tool_result whose tool_use_id equals the call's id400: result references an unknown ID, or a call is left unanswered
Results go in the very next user message; nothing in between400 on the next request
In that user message, tool_result blocks come first; any text goes after themValidation error, or the turn ends early
Parallel calls in one turn are all answered in one user messageOrphaned calls; splitting results across turns fails
Failures are returned as a tool_result with is_error: true and a short messageAn empty string reads as "no data", so Claude confidently reports a false result
messages.append({"role": "assistant", "content": resp.content})   # echo unchanged
results = []
for block in resp.content:
    if block.type == "tool_use":
        try:
            out = run_tool(block.name, block.input)                    # your authz + validation inside
            results.append({"type": "tool_result", "tool_use_id": block.id, "content": out})
        except ToolError as e:
            results.append({"type": "tool_result", "tool_use_id": block.id,
                            "content": f"lookup failed: {e.short}", "is_error": True})
messages.append({"role": "user", "content": results})                # all results, one turn, first
tool_choiceBehaviorNotes
{"type": "auto"} (default)Claude decides whether to call a toolAdd "disable_parallel_tool_use": true inside this object for at most one call per response
{"type": "any"}Must call some toolForced use: not allowed with manual extended thinking, and some of the newest models return 400
{"type": "tool", "name": "..."}Must call that toolSame restrictions as any; prefer auto + strict: true or structured outputs where forcing is rejected
{"type": "none"}No tool callsTools stay defined (keeps the cache prefix stable)

Tool design & multimodal tool_result

  • Description = purpose + when to use + an exclusion sentence ("not for refunds").
  • Small, well-named tool set; overlapping tools cause wrong picks.
  • Mark truly required fields in required; use enums for closed sets.
  • strict: true guarantees tool inputs match the schema.
  • Multimodal tool_result: content can be an array of text and image blocks (e.g. Playwright browser screenshots), enabling closed-loop visual UI verification.

Client vs. server tools

  • Client tools: you execute them and return tool_result.
  • Server tools (e.g. web search): Anthropic executes them inside the turn.
  • A long server-tool loop can stop with pause_turn: send the assistant content back as-is to continue.
ArchitectureHow tools & context are handledScaling limits & best fit
Workflow / Hybrid RAGFixed code path; static chunking + embeddings + reranking injected before calling ClaudePredictable latency; fails on multi-hop code navigation (under-retrieval, over-retrieval, and misaligned retrieval across imports)
Shallow ReAct Loop / Standard MCPSingle messages[] loop calling JSON tools one turn at a time4 scaling bottlenecks: (1) upfront tool-schema bloat, (2) K-turn sequential ping-pong latency & repeated prefix billing, (3) pretraining mismatch (LLMs are trained on millions of real TypeScript/Python repos vs. synthetic JSON tool_use traces), and (4) JSON schemas cannot express loops, conditionals, or error-handling chains
Deep Agent Harness (e.g., Claude Code)Explicit planning tool (TodoWrite) + isolated subagents + virtual/real filesystem (Glob/Grep/paginated Read)Solves context crowding and goal drift over 10–100+ step tasks
Code-orchestrated tool calling (Anthropic programmatic tool calling; Cloudflare Code Mode)Anthropic programmatic tool calling: Claude writes Python that runs in Anthropic's code execution container; tools opt in via allowed_callers, and intermediate results stay out of context. Cloudflare Code Mode (separate third-party variant): converts MCP tool schemas into a typed TypeScript API (declare namespace); LLM emits one TypeScript script run inside a network-isolated V8 isolate (~ms startup, no filesystem/env access) where tools are exposed as outbound RPC bindings (await env.ServerName.toolName(...))Collapses K sequential tool turns into 1 LLM inference pass and keeps intermediate tool payloads inside V8 memory (out of the LLM context window) until console.log outputs the final distilled result

Common trap

"Unknown tool_use_id" is a wiring bug, not a schema or description bug. And disable_parallel_tool_use is not a top-level request field: it lives inside tool_choice.

05

5. stop_reason Values and Handling

stop_reasonMeaningCorrect handling
end_turnNatural finishUse the response
tool_useClaude is calling one or more toolsRun them, return paired tool_results, call again
max_tokensHit your max_tokens cap (or the model maximum)Raise the cap or continue the response; never parse truncated JSON as complete
model_context_window_exceededGeneration filled the context windowTreat as truncated; prune/compact history before the next turn
stop_sequenceOne of your stop_sequences firedRead stop_sequence to see which
pause_turnA long server-tool loop was pausedSend the assistant content back unchanged to continue
refusalClaude declined (safety classifiers)Returned as HTTP 200. Read stop_details; don't blind-retry the same request; reset or rephrase the refused turn, or retry on a fallback model where offered

Refusals in practice

  • stop_details can carry a policy category (may be null) and an explanation that isn't guaranteed stable.
  • Leaving the refused turn in history leads to continued refusals: remove or rephrase it, or clear history.
  • When streaming, the refusal arrives on the message_delta event; discard the partial turn.
  • An HTTP-status-only error check misses refusals entirely.

max_tokens vs. context window

  • Input alone larger than the window → 400 "prompt is too long", before any generation, on every model.
  • Input fits, but input + max_tokens exceeds the window → newer models accept it and stop with model_context_window_exceeded if they run out.
  • The API never silently drops old turns: history management is your job.

Common trap

A refusal is a content decision, not a transient error: exponential backoff on the same request just repeats it. And max_tokens limits output length only; raising it does nothing for an oversized prompt or a crowded context.

06

6. Streaming Events

Event (in order)CarriesHandler action
message_startMessage shell, initial usageOpen a buffer for the turn
content_block_startBlock index and type (text, tool_use, thinking)Start a per-index accumulator
content_block_deltatext_delta, input_json_delta (partial tool JSON), thinking deltasRender text; append JSON fragments, don't parse yet
content_block_stopBlock completeNow parse tool input JSON and validate
message_deltaTop-level changes: stop_reason, cumulative output usageBranch on stop_reason (refusal, tool_use, max_tokens…)
message_stopMessage completeOnly now commit the assistant turn to history
pingKeep-aliveIgnore

Why stream

  • Improves perceived latency (time to first token) for a waiting user.
  • Recommended for long requests: very long non-streamed calls risk timeouts, and the Python SDK requires streaming above roughly 21k max_tokens.
  • Does not reduce total tokens, cost or request count.

Interrupted streams

  • "The read loop ended" ≠ "the message is complete". Only message_stop means complete.
  • On a drop, discard the partial turn and retry from the last complete turn.
  • SDK helpers (e.g. a final-message accumulator) do the bookkeeping for you.

Common trap

Saving a half-streamed assistant turn leaves a malformed tool_use in history, and every later request fails validation, often surfacing only on retries. Parse tool input at content_block_stop; commit the turn at message_stop.

07

7. Structured Outputs

ApproachGuaranteeUse when
System-prompt output styles (format/persona instructions in system, e.g. YAML/HTML/tutor style)None: best effort (steers tone, verbosity, or non-JSON formats like YAML/Markdown)Human-facing presentation or custom personas; never rely on prompt prose for downstream JSON parsers
JSON outputs: output_config.format = {"type": "json_schema", "schema": {...}}The response text parses and matches the schema (constrained decoding)The answer itself must be machine-readable (extraction, classification, reports)
Strict tool use: "strict": true on a toolTool name and inputs match input_schemaAgents whose tool calls feed real systems
resp = client.messages.create(
    model=PINNED_MODEL_ID, max_tokens=1024,
    messages=[{"role": "user", "content": email_text}],
    output_config={"format": {"type": "json_schema", "schema": {
        "type": "object",
        "properties": {"name": {"type": "string"}, "demo_requested": {"type": "boolean"}},
        "required": ["name", "demo_requested"],
        "additionalProperties": False}}},
)
if resp.stop_reason == "end_turn":                      # check before parsing
    data = json.loads(next(b.text for b in resp.content if b.type == "text"))

Production constraints

Common trap

When a parser breaks on "almost JSON", don't add a third paragraph of formatting instructions or a regex repair step. Move format enforcement into the API with output_config.format or strict: true.

Extraction via tool use

Define a tool whose input_schema is the record you want, then read the tool_use block's input (already a parsed object). Nothing has to be executed: the tool call is the output.

tool_choice for extractionGuaranteeUse it when
{"type": "auto"}Claude may answer in text instead of calling the toolYou can tolerate a text reply, or the prompt states clearly when the tool applies
{"type": "any"}Must call one of the tools; Claude picks whichSeveral extraction schemas (invoice, receipt, contract) and the document type is unknown
{"type": "tool", "name": "extract_metadata"}Must call that toolA specific extraction must run first (e.g. metadata before enrichment steps)
Schema design choiceWhyExample
Put in required only fields every source containsA required field the document lacks pushes the model to invent a valueinvoice_number required; po_number optional
Make may-be-missing fields nullable ("type": ["string", "null"])"Not stated" becomes a valid answer instead of a fabricationdue_date nullable + "use null if not stated". Required-but-nullable keeps the key always present
Enum + "other" + a detail stringExtensible categories without losing informationcategory: [..., "other"] + category_detail
Add an "unclear" enum valueThe model can flag ambiguity instead of guessingsentiment: [positive, negative, neutral, unclear]
Put format-normalization rules in the promptThe schema fixes types, not how messy source formats are rendered"Dates as YYYY-MM-DD; amounts as numbers; weights in grams"
record_invoice = {
    "name": "record_invoice", "strict": True,
    "description": "Record fields from ONE invoice. Use null when a field is not stated.",
    "input_schema": {"type": "object", "additionalProperties": False,
        "properties": {
            "invoice_number":  {"type": "string"},
            "due_date":        {"type": ["string", "null"], "description": "YYYY-MM-DD, or null if absent"},
            "category":        {"type": "string", "enum": ["hardware", "software", "services", "other"]},
            "category_detail": {"type": ["string", "null"]},
            "stated_total":    {"type": "number"}},
        "required": ["invoice_number", "due_date", "category", "category_detail", "stated_total"]}}

resp = client.messages.create(model=PINNED_MODEL_ID, max_tokens=1024, tools=[record_invoice],
                              tool_choice={"type": "tool", "name": "record_invoice"},
                              messages=[{"role": "user", "content": invoice_text}])
record = next(b for b in resp.content if b.type == "tool_use").input   # dict, schema-valid

Strict-schema limits (verify against current docs)

Objects need additionalProperties: false; numeric (minimum, maximum) and string-length constraints aren't supported (400), so enforce ranges in your validator; complexity caps include 20 strict tools per request, 24 optional parameters and 16 union-type (e.g. nullable) parameters across strict schemas. Enum capitalization isn't guaranteed: compare case-insensitively.

Common trap

Marking every field required "for completeness". When the source lacks the value, a schema-valid but invented value comes back. Make it nullable or optional and say "null if not stated".

08

8. Validation, Retry and Feedback Loops

Error classExampleRemoved by strict schema?Fix
Syntax / shapeBroken JSON, wrong type, extra keyYes (strict tools, JSON outputs)Nothing more to do
SemanticLine items don't sum to the total; value in the wrong field; impossible dateNoValidate in code → retry with error feedback → human review
Information absentThe total is only in an attachment you didn't sendNo, and retries can't helpReturn null / flag it, fetch the missing source, or route to a human

Retry with error feedback

  • The follow-up request carries the original document, the failed extraction and the specific validation errors ("line items sum to 1,140.00 but stated_total is 1,240.00; check for a missing line or misread digit").
  • Retries work for format mismatches, structural mistakes, misplaced fields and slips the source can resolve.
  • Retries fail when the information isn't in the input: pushing harder makes the model invent a value that passes your check. Cap attempts (1–2), then flag or route to review.

Fields that make errors visible

  • calculated_total vs. stated_total: the model sums the line items and copies the printed total; your code compares them and flags mismatches.
  • conflict_detected (boolean + note): the source contradicts itself (two different dates for one event). Report it; don't let the model pick one silently.
  • detected_pattern on review findings: name the construct that triggered the finding (e.g. string-built-sql, unchecked-return). When developers dismiss findings, group dismissals by pattern to see which criteria cause false positives, then rewrite those criteria (§3).
def extract_with_retry(doc, max_retries=2):
    messages = [{"role": "user", "content": doc}]
    for attempt in range(max_retries + 1):
        resp = client.messages.create(model=PINNED_MODEL_ID, max_tokens=2048, tools=[record_invoice],
                                      tool_choice={"type": "tool", "name": "record_invoice"}, messages=messages)
        call = next(b for b in resp.content if b.type == "tool_use")
        errors = validate(call.input)            # sums, ranges, cross-field business rules
        if not errors:
            return call.input
        messages += [{"role": "assistant", "content": resp.content},
                     {"role": "user", "content": [{"type": "tool_result", "tool_use_id": call.id, "is_error": True,
                         "content": "Validation failed:\n" + "\n".join(errors) +
                                    "\nRe-read the document and fix only these fields. Use null if the value is not in it."}]}]
    return route_to_review(doc, call.input, errors)  # don't loop forever on absent data

Common trap

"Retry up to five times until validation passes." If the value isn't in the source, the loop eventually produces a fabricated value that passes. Absent data is an input problem: flag it or fetch the source.

09

9. Adaptive Thinking

What changed

The older extended thinking mode made you allocate a fixed token budget. Adaptive thinking lets the model judge how much to reason per request; you steer depth with effort.

  • API shape: thinking={"type": "adaptive"} plus output_config={"effort": "high"}.
  • Manual {"type": "enabled", "budget_tokens": N} is deprecated on Claude 4.6 and returns 400 on 4.7 and later.
  • Migration: drop budget_tokens, switch the type to adaptive, control depth with effort.

Display and billing

  • thinking.display: "summarized" returns readable summaries; "omitted" returns an empty thinking field.
  • You are billed for the full internal thinking as output tokens either way; display changes only what you see.
  • Higher effort is not always better: it adds latency and cost. Match effort to task difficulty.
Rule in tool-use loopsWhy
Echo the assistant message exactly, including thinking and redacted_thinking blocks, when you return tool resultsThinking blocks carry a cryptographic signature; editing, summarizing or filtering them gives a 400
Keep one thinking configuration per assistant turnA tool-use loop is one assistant turn; change settings only between turns
Manual extended thinking allows only tool_choice auto or noneForced tool use is incompatible with manual mode
To save context, prune or compact older completed turnsNever "save tokens" by stripping blocks from the active turn

Common trap

Changing thinking or effort between requests invalidates the messages prompt cache (and on some models the system and tool caches too). Pick a setting per workload and keep it stable.

10

10. Context Window, Compaction and Prompt Caching

Context strategy (Write / Select / Compress / Isolate)HowUse when
Prune (Compress)Drop or truncate old tool results and stale turns in your codeTool outputs dominate the history
Context editing (Compress)Beta context_management edits that clear old tool uses (and thinking) server-side past a triggerLong agent loops with bulky, one-time tool output
Compaction (Compress)Beta context_management edit compact_20260112: older history is summarized into a compaction block (default trigger 150k input tokens; optional pause_after_compaction returns stop_reason: "compaction")Persistent agents that run for hours or days
Progressive disclosure & tool scoping (Select)Pass only relevant tools per turn or load skills/docs on demand via retrieval instead of dumping 50+ schemas up frontLarge tool catalogs where schema tokens bloat every turn and degrade tool selection
Virtual filesystem offloading (Write + Select)Persist plans, scratchpads, and bulky intermediate results to external state/files; read back targeted slicesMulti-step tasks that must survive compaction without losing exact specs or artifacts
Sub-agent handoff (Isolate)Give a bounded sub-task its own fresh messages[] array; pass an explicit context contract and return only the synthesized answer + required raw artifactsExploration, deep search, or independent verification that would flood the main context
MeasureToken-counting endpoint before sending; watch usageAlways; production payloads are larger than test fixtures

Diagnosis rule: Turn 1 vs. Turn 5+

An agent that picks the wrong tool on Turn 1 has a schema/description collision or tool bloat problem (too many overlapping tools; fix with Select / progressive disclosure or sharper descriptions). An agent that picks the right tool on turns 1–4 and the wrong one from Turn 5+ does not have a schema problem: accumulated tool_result output is crowding out instructions (fix with Compress / prune / edit / compact, not by rewriting descriptions or raising max_tokens).

Context Rot Taxonomy & Recovery

  • Context poisoning: A hallucination or faulty premise in an early turn (or tool output) is referenced repeatedly by later turns. Warning: Summarization/compaction preserves poisoned premises in condensed form—you must truncate/reset history prior to the bad turn.
  • Context confusion: Unrelated documents, logs, or 50+ unused tool definitions dilute attention and degrade accuracy. Fix via tool scoping and progressive disclosure.
  • Context clash: Contradictory instructions or superseded requirements accumulate across turns. Fix by consolidating state into a single external spec or resetting with a clean handoff.

Multi-Agent Context Contracts & Builder–Reviewer

  • Self-contained handoff: A subagent starts with a blank messages[] array—never assume it inherits parent conversation history.
  • Raw artifact return: If a downstream step needs exact code, SQL, or JSON produced by a subagent, require the subagent to write it to a shared store/file or return it verbatim rather than returning a prose summary.
  • Bounded waves & when NOT to parallelize: Dispatch parallel subagents in bounded waves (e.g., 3–5 concurrent calls) to avoid workspace 429s. Never parallelize sequential data dependencies or overlapping writes to the same state/record.
  • Builder–Reviewer separation: Run verification/code review in a separate messages.create() call with a fresh messages[] array containing only the spec and candidate artifact—never ask the same context that generated the artifact to grade its own work.

Prompt caching mechanics

  • Mark the end of a reusable prefix with cache_control: {"type": "ephemeral"}; up to 4 breakpoints per request.
  • Prefix order is tools → system → messages. A change at one level invalidates that level and everything after it.
  • Cache hits need an identical prefix: static content first, dynamic content after the last breakpoint.
  • Default TTL is 5 minutes (refreshed on use); "ttl": "1h" is optional.
  • Track cache_creation_input_tokens vs. cache_read_input_tokens in usage.

Caching economics

  • Cache write: 1.25× base input (5-min TTL) or 2× (1-hour TTL).
  • Cache read: about 0.1× base input (lower on some newer models; check pricing).
  • Break-even: a 5-min entry pays off after 1 read; a 1-hour entry after 2 reads.
  • Also cuts time to first token on long prefixes.
  • Silent invalidators: editing tool definitions, changing tool_choice, toggling images, changing thinking/effort, or swapping system output styles mid-session (invalidates both system and downstream messages caches).

Common trap

A 0% hit rate with a timestamp, user ID or ticket text placed before the breakpoint. Each request writes a new entry and never reads one. A longer TTL only makes those writes cost more; move the dynamic text after the breakpoint.

11

11. Preserving Critical Facts in Long Sessions

The problem

The API is stateless, so every request must carry the conversation. As sessions grow, the tempting fixes (summarize, compact, drop turns) quietly lose the exact facts the task depends on.

RiskSymptomFix
Progressive summarization"Refund of 1,240.18 for order A-7731, promised by 14 March" becomes "customer wants a refund"Keep a persistent case facts block outside the summarized history
Tool-output bloatAn order lookup returns 40+ fields when 5 matter; resent on every later turnTrim tool output to the relevant fields in your wrapper before appending the tool_result
Lost in the middleFindings buried in the middle of a long aggregated input get skippedPut a key-findings summary first; organize details under explicit section headers or XML tags
Partial historyOnly the latest user message is sent; the model loses commitments and contextSend the complete (or deliberately managed) history every request

Case facts block

  • Extract transactional facts (amounts, dates, order IDs, statuses, promised actions, the customer's stated expectation) into a structured block your code owns and updates when tools return new data.
  • Re-inject it on every request. For multi-issue sessions, keep one entry per issue keyed by order or ticket ID.
  • Place it after the last cache breakpoint so updates don't invalidate the cached static prefix (§10).

Long inputs (lost-in-the-middle mitigation)

  • Models handle the start and end of a long input more reliably than the middle.
  • Anthropic's long-context guidance: put long documents at the top and the query and instructions at the end (tests showed up to ~30% better answers on complex multi-document inputs); wrap each document in <document> tags with source metadata.
  • When aggregating subagent or tool results, lead with a short summary of key findings, then the detail under headers.
  • Ask upstream agents to return structured data (key facts, dates, source locations, citations, relevance scores) rather than verbose prose and reasoning when the downstream context budget is tight. Orchestration patterns live in the Claude Agent SDK guide.
case_facts = {"customer_id": "C-1029",
              "issues": {"A-7731": {"amount": 1240.18, "ask": "refund", "promised_by": "2026-03-14",
                                    "status": "return label sent"}}}
system = [{"type": "text", "text": STATIC_POLICY, "cache_control": {"type": "ephemeral"}},
          {"type": "text", "text": "<case_facts>\n" + json.dumps(case_facts) + "\n</case_facts>"}]

Common trap

Trusting compaction or a rolling summary to keep amounts, dates and IDs. Summaries paraphrase. Keep authoritative facts in a structured block your code maintains, and trim tool output at the source.

12

12. Message Batches

What it is
An asynchronous endpoint that accepts many Messages requests in one submission and processes them in the background at 50% of standard price (input and output).
Limits
Capped at 256 MB total payload or 100,000 requests per batch (whichever threshold is reached first). Most batches finish within an hour; anything not done in 24 hours expires.
custom_id
Required, unique per request (1–64 chars, letters, digits, _, -). Results can come back in any order, so match on custom_id, never on position.
Lifecycle
Create → poll until processing_status is ended → fetch results (each succeeded, errored, canceled or expired). Results stay downloadable for 29 days.
No latency SLA
"Most finish within an hour" is typical, not promised; under heavy demand more requests can expire at 24 hours.
One response per request
A batched request can include tools and prior turns, but your code can't run a tool mid-request and hand back the result. If a result stops on tool_use, the next step needs a new request. Batches suit single-shot work; forced extraction tools (§7) fit because the tool_use input is the output.
WorkloadBest transportWhy
Nightly classification of 40k tickets, read next morningMessage BatchesHalf price, no rate-limit fight, latency doesn't matter
Pre-merge PR check a developer is waiting onSynchronous APIBlocking workflow; batch has no latency guarantee
Agent that must query your database between stepsSynchronous tool loopBatch can't run your tools mid-request
Chat UI with long answersStreamingA person is waiting; perceived latency
Backend fan-out of a few hundred calls per user actionAsync client with bounded concurrencyNeeds results in seconds
Same 5k-token instructions on every batched itemBatches + prompt cachingThe discounts can be combined (batch cache hits are best effort)

Operating a batch pipeline

Common trap

Splitting a list into chunks and calling messages.create() per item is not batching: it is serial synchronous traffic that hits rate limits. Smaller chunks or more backoff don't change that; the Batches API does. And the reverse trap: moving a blocking check (pre-merge, user waiting) to batches for the 50% saving, which removes any latency guarantee.

13

13. Multi-Instance and Multi-Pass Review

Why self-review is weak

The session that generated an artifact keeps its own reasoning in context and tends to defend it. "Now review your work critically" in the same thread, or more thinking, doesn't remove that anchor.

PatternHowUse it for
Independent reviewer instanceA separate messages.create with a fresh messages list holding only the spec/rubric and the artifact, without the generator's reasoningCatching subtle bugs in generated code, contracts, extractions
Per-file local passOne focused request per file: local bugs, security, logicLarge PRs where a single prompt dilutes attention
Cross-file integration passA separate pass over the per-file structured findings plus interfaces: data flow, contracts, consistencyIssues no single file shows; resolving contradictory local findings
Confidence-annotated verificationEach finding carries a self-reported confidence; route by thresholdSending low-confidence findings to humans; thresholds must be calibrated (§16)
01

Generate

Builder session produces the artifact (code, extraction, report).

02

Local passes

One fresh request per file; structured findings with detected_pattern and confidence.

03

Integration pass

New request over the findings and interfaces: cross-file data flow and contradictions.

04

Route

High-confidence findings auto-comment; low-confidence ones go to a human.

Common trap

Reviewing a 30-file PR in one prompt, or asking the generating session to "double-check". The first dilutes attention and yields contradictory findings; the second inherits the generator's blind spots. Split into local + integration passes and use an independent instance.

14

14. Errors and Retries

Status / typeClassAction
400 invalid_request_error (bad shape, prompt too long, unsupported param, spend limit you set)TerminalFix the request; retrying the same body fails identically
401 authentication · 403 permission · 404 not foundTerminalFix key, permissions or model ID; fail fast
402 billing_error · 413 request_too_largeTerminalFix billing; shrink the payload below the endpoint's size limit
429 rate_limit_errorRetriableWait retry-after, then backoff + jitter. A spend-cap 429 has no retry-after and keeps failing
500 api_error · 504 timeout_errorRetriableExponential backoff; for 504 on long requests, stream instead
529 overloaded_errorRetriableBackoff with jitter; consider a fallback model in the orchestration layer
HTTP 200 + stop_reason: "refusal"Not an HTTP errorContent decision: handle as in §5, don't backoff-retry

SDK defaults

  • Official SDKs retry connection errors, 408, 409, 429 and ≥500 2 times by default with short exponential backoff, honoring retry-after.
  • Configure with max_retries (client-wide or per request); 0 disables.
  • Log the request-id response header for support.

Reliability placement

  • Retry: next to the call, owned by one layer.
  • Circuit breaker: at the service boundary, on aggregate error rate.
  • Fallback (other tier, cached answer, human queue): in orchestration.
  • Budgets: max turns / tool calls per agent run.

Common trap

Wrapping SDK calls in your own 5-attempt loop with default SDK settings gives up to 15 attempts per request against a rate limit. One layer owns retries. And a bare except Exception: pass retries 400s pointlessly while hiding the real error.

15

15. Evals and Cost Levers

Output shapeGraderCatches / misses
Label, number, enum, JSON fieldExact match / code assertionCheap, deterministic; can't judge free text
Structured object, tool trajectoryCode checks on properties (fields, ranges, expected/forbidden tools, call counts)Robust to wording; misses tone and nuance
Free-text answer, summary, toneLLM judge with a rubricScales; must be calibrated against human labels
High-stakes or novel behaviorHuman reviewGold standard; slow and costly

Eval discipline

Cost leverEffectNumber / rule
Model tier per stepLargest price driverStart balanced; move steps on eval evidence
Effort / thinkingThinking billed as outputLow or off for classifiers and routers
Prompt cachingCheaper repeated prefixes, faster first tokenWrite 1.25× / 2×, read ≈0.1×
Message BatchesHalf price for offline work50%, ≤24 h
Context sizeEvery resent token is billed againPrune, edit, compact; count tokens
Call count / fan-outMulti-agent runs multiply tokensFan out only for truly independent subtasks
StreamingPerceived latency onlyNo cost change
US-only residency (first party)Premium1.1× on Claude 4.6+ with inference_geo: "us"
Workspace spend caps & rate limitsHard blast-radius control per team/envSeparate workspaces + monthly spend limits in Console (settings/limits); spend-cap 429s have no retry-after

Common trap

Quoting the per-call cost of a pilot. Instrument tokens, latency and errors per call from day one, multiply by production volume, size for the heavy tail and p95 rather than the median, and set the reliability floor before tuning cost.

16

16. Human Review and Confidence Calibration

The core risk

An aggregate number hides segment failures. "97% field accuracy" can mean 99% on typed invoices and 70% on handwritten receipts, or one field (say tax_id) failing everywhere. Slice accuracy by document type × field before you reduce human review.

PracticeHowWhat it prevents
Segmented accuracyReport accuracy per document type, per field, per sourceAutomating a segment that quietly fails
Field-level confidenceThe model outputs a confidence per field (alongside "unclear" / conflict_detected flags)One document-level score hiding one bad field
Calibration on labeled dataOn a labeled validation set, measure real accuracy per confidence band; set thresholds per field / doc type where accuracy meets the targetTrusting raw self-reported confidence, which isn't calibrated until checked
Risk-based routingLow confidence, conflicts, ambiguous or contradictory sources → human queue, highest-risk firstSpending scarce reviewer time on easy items
Stratified random samplingKeep sampling auto-approved, high-confidence items within each stratum (doc type, field, source)Unmeasured error rates and missed new error patterns or drift; plain random sampling under-samples rare types

Feedback loop

Reviewer corrections become labeled data → recalibrate thresholds, add few-shot examples for the new error patterns (§3), extend the eval set (§15).

Common trap

"Overall accuracy is 97%, so remove human review", or "review only low-confidence items". The first ignores segments; the second never measures errors among auto-approved items. Validate per segment, calibrate thresholds on labeled data, and keep stratified sampling running.

17

17. Deploying: Platforms, Data Residency and Model-ID Pinning

PlatformOperated by / identityResidency controlPick it when
Claude API (first party)Anthropic; API keys and workspacesinference_geo: "global" (default) or "us" (1.1×), Claude 4.6+ only. No EU-only value; older models 400 on the paramNo binding cloud or EU constraint; newest features and betas first
Claude Platform on AWSAnthropic-operated, AWS billingSame global / us geographiesFirst-party features with AWS procurement
Amazon BedrockAWS-operated; IAM, AWS billGlobal endpoint, single-region endpoints, or geographic inference profiles (e.g. eu.-prefixed IDs); regional costs 10% moreWorkload and compliance posture already live in AWS
Google Cloud (Vertex AI)Google-operated; Cloud IAM and billingGlobal, multi-region (us / eu) or regional (some models); 10% premium off globalWorkload and compliance posture already in Google Cloud; EU residency
Microsoft FoundryAnthropic-operated, on AzureVaries by deployment typeCustomer standardised on Azure

Model-ID pinning

  • Claude 4.6 and later: the dateless ID (e.g. claude-sonnet-4-6) is a fixed snapshot; updates ship under new IDs.
  • Earlier models: dated IDs (claude-haiku-4-5-20251001); the short alias moves to the newest dated snapshot.
  • Platform formats differ: Bedrock anthropic.claude-… (dated models end in -v1:0; optionally global. / us. / eu. prefixed), Google Cloud …@YYYYMMDD for dated models and the API format from 4.6 on.
  • Every ID has its own deprecation and retirement date.

Release discipline

  • Pin the full model ID in config, not code.
  • Keep the previous pin as the rollback target.
  • Promote a new model only when evals meet or beat the pinned baseline.
  • Feature availability differs by platform: verify each beta or tool you depend on.
  • Measure latency from the customer's region with a real payload; compare total cost (fees, egress, integration), not just token price.

Common trap

"Inference must stay in the EU" cannot be met with first-party inference_geo (global or US only). Use Bedrock regional/EU routing or the Google Cloud eu multi-region, decided at scoping because residency is pass/fail. Encryption at rest does not change where inference runs.

18

18. Security for API Apps

RiskControl that actually bounds itNot enough on its own
Indirect prompt injection via fetched pages, emails, files or tool outputTreat tool/fetched content as data; delimit it; scan it before it enters context; isolate readers from action toolsFiltering only the user's prompt
Excessive agency (agent sends, deletes, pays)Least privilege at the tool registry: remove tools the task doesn't need; read-only credentials"Never send email" in the system prompt
Side-effecting tool misuseDeterministic authorization in your code before execution (allowlist, caller identity, amount scope), failing closed; human approval for irreversible actionsAn output filter that runs after the tool already executed
Forged identity ("I am an admin")Inject verified role and tenant server-side from your auth layerAsking the model to verify the claim
Malformed or hostile tool inputsstrict: true plus server-side validation; parameterised queries; fixed write pathsTrusting model-generated arguments
Leaked secrets and PIIKeys in env/secret manager, never in prompts; pass reference IDs, not raw PII; per-tenant keys or workspaces"Don't reveal secrets" instructions
Unsafe rendering of model outputEscape or sanitize before display; never execute model HTML/JSAssuming model output is trusted
Programmatic tool calling / LLM-generated codeRun generated code inside a capability-restricted sandbox (e.g., V8 isolate with zero ambient network, filesystem, or env-var access; only explicit outbound RPC bindings)Running LLM-generated Python/JS on the host or passing raw API credentials into the sandbox

Compliance scoping

Common trap

"All our users are trusted employees, so injection isn't a risk." The attacker is whoever wrote the web page, email or document the agent reads. Prompt rules and bigger models are probabilistic; capability removal and pre-action authorization are not.

19

19. Failure Modes and Find the Defect

Symptom / scenarioRoot cause → correct fixTempting wrong answer
"tool_result references unknown tool_use_id"Result ID ≠ call ID, or result not in the next user turn → echo block.id; results first in the next messageRewriting the tool description
400 after "trimming" history to save tokens with thinking onThinking block edited/dropped, signature mismatch → echo turns unchanged; compact older turnsRaising max_tokens
Validation error only on retries after a dropped streamHalf-streamed turn saved → commit at message_stop; discard partial turnsChanging the tool schema
Correct tool on turns 1–4, wrong tool from turn 5Context crowded by tool output → prune, edit or compact; measure tokensClarifying descriptions or raising max_tokens
Agent repeats a hallucinated API path even after context compactionContext poisoning survived summarization → truncate/reset history to before the poisoned turnRunning compaction again or lowering temperature
Orchestrator spawns a subagent to generate a config file, gets "Completed!" back, and lacks the actual configSubagent returned a summary instead of the raw artifact → require verbatim output or write to a shared store/fileIncreasing the subagent's max_tokens
Agent with 40 MCP tools suffers 12-turn sequential tool ping-pong and 80k schema tokens per turnStandard ReAct loop bottleneck → use progressive tool disclosure (Select) or code-orchestrated tool calling (Anthropic programmatic tool calling, or Code Mode in a V8 isolate)Switching to a larger model tier
Model says "no orders found" during a DB outageFailure returned as "" → tool_result with is_error: true"Never say no orders found" in the prompt
Answer silently missing; logs show HTTP 200stop_reason: "refusal" or max_tokens not checked → branch on stop_reasonRetrying on HTTP status only
Same refusal repeats every turnRefused turn kept in history → remove/rephrase it or resetExponential backoff
Nightly job hits 429 every nightSync loop over items → Message Batches + custom_idSmaller chunks
Batch results attached to the wrong ticketsMatched by position → match on custom_idSorting results by timestamp
429 storms despite "retries"Own loop stacked on SDK retries → one retry owner, honor retry-after, jitterMore attempts
400 after model upgrade on temperature / budget_tokensUnsupported on 4.7+ → omit sampling params; adaptive thinking + effortLowering the values
0% cache hit rateDynamic text before the breakpoint, or tool_choice/effort changing per call → static prefix first, stable settings1-hour TTL
Parser breaks after an upstream release; can't roll backMoving alias → pin the full ID, keep prior pin, eval-gate promotionAdding parser retries
EU-only residency requirement on first-party APIinference_geo has no EU value → Bedrock/Google Cloud EU routinginference_geo: "us" + encryption
Summarizer agent forwards a thread externallyInjection via fetched email + unneeded send tool → treat content as data; remove the toolStricter user-input regex
Code reviewer floods PRs with style nits; developers start ignoring real bugsVague criteria → categorical report/skip rules + contrast few-shot examples; disable the noisiest category while fixing it"Only report high-confidence findings"
Extractor returns plausible but invented po_number valuesField is required but often absent → nullable/optional + "null if not stated"More retries
Retry loop never fixes total; the value is only in an attachment that wasn't sentInformation absent → stop retrying; flag, fetch the source or route to reviewRaising the retry cap
Support agent forgets the refund amount and deadline after 30 turnsRolling summary dropped exact facts → persistent case-facts block; trim tool outputsSwitching to a bigger model
97% overall accuracy, but handwritten receipts are often wrongAggregate hides segments → accuracy per doc type × field; stratified sampling; calibrated thresholdsRemoving human review
Pre-merge check moved to Message Batches; developers wait hoursBatch has no latency SLA → sync API for blocking checks, batch for overnightPolling the batch more often

Find the defect: a migrated tool-use turn

This code ran on an older model. After moving to a Claude 4.7+ pinned ID it fails or misbehaves. Find at least three defects before revealing.

resp = client.messages.create(
    model=NEW_PINNED_ID, max_tokens=4096,
    temperature=0,                                            # (a)
    thinking={"type": "enabled", "budget_tokens": 8000},      # (b)
    tools=TOOLS, messages=messages,
)
messages.append({"role": "assistant",                         # (c)
                 "content": [b for b in resp.content if b.type != "thinking"]})
call = next(b for b in resp.content if b.type == "tool_use")
try:
    out = run_tool(call.name, call.input)
except Exception:
    out = ""                                                  # (d)
messages.append({"role": "user", "content": [
    {"type": "text", "text": "Here is the result:"},          # (e)
    {"type": "tool_result", "tool_use_id": call.id, "content": out},
]})

Reveal the Fixes (Click)

Fixes. What must change, and why?

(a) Remove temperature: non-default sampling values return 400 on Claude 4.7+. Steer with the prompt instead. (b) Replace the manual budget with thinking={"type": "adaptive"} and output_config={"effort": "high"}; budget_tokens returns 400 on 4.7+. (c) Append resp.content unchanged: filtering thinking blocks breaks the signature check on the next call. (d) Return failures as a tool_result with is_error: true and a short message, never "". (e) Put tool_result blocks first; text after. Bonus: check resp.stop_reason == "tool_use" before looking for a call, and loop over all tool_use blocks in case of parallel calls.
20

Glossary, Practice Quiz & Related Guides

Content block
Typed unit of a message: text, image, document, tool_use, tool_result, thinking, redacted_thinking.
stop_reason
Why generation stopped: end_turn, tool_use, max_tokens, stop_sequence, pause_turn, refusal, model_context_window_exceeded.
tool_use_id
Field on a tool_result that must equal the id of the tool_use block it answers.
is_error
Flag on a tool_result telling Claude the tool failed, so it can retry, ask, or stop instead of trusting empty data.
tool_choice
auto (default), any, tool, none; holds disable_parallel_tool_use.
Strict tool use
strict: true on a tool: inputs are guaranteed to match its input_schema.
output_config.format
JSON outputs: constrains the response text to a JSON schema.
Adaptive thinking
thinking: {type: "adaptive"}; the model sizes its own reasoning, steered by output_config.effort.
Effort
Per-request reasoning depth: low → max (set varies by model). Higher = slower and costlier.
Signature
Cryptographic field on thinking blocks; any edit makes the next request fail.
message_stop
Final streaming event; the only safe point to commit a streamed turn.
input_json_delta
Streaming fragment of tool input JSON; parse only at content_block_stop.
Cache breakpoint
cache_control marker ending a reusable prefix; up to 4 per request; TTL 5m (default) or 1h.
Compaction
Server-side summarization of older context into a compaction block so long-running agents keep going.
4 Context Strategies
Write (persist outside context), Select (progressive disclosure/scoped tools), Compress (prune/edit/compact), and Isolate (subagents with clean messages[]).
Context Rot Taxonomy
Context poisoning (hallucinated premise survives compaction; requires reset), Context confusion (unrelated docs/tools dilute attention), and Context clash (contradictory turns).
Programmatic tool calling
Anthropic feature where Claude writes Python that runs in the code execution container and calls your tools (opted in via allowed_callers), so chained calls and intermediate results stay out of the context window. Cloudflare Code Mode is a separate third-party variant (TypeScript in a V8 isolate with RPC bindings).
custom_id
Your unique key per batched request; the only reliable way to match out-of-order results.
inference_geo
First-party residency parameter: global or us (Claude 4.6+).
Pinned model ID
An ID whose underlying model never changes; the unit of release and rollback.
Indirect prompt injection
Instructions hidden in content the app reads (pages, emails, tool output) rather than typed by the user.
Few-shot (multishot) examples
A few varied, tagged input → output examples that teach format and judgment for ambiguous cases.
Explicit criteria
Categorical report/skip rules (with examples per severity) that replace vague "be careful / be confident" wording.
Nullable field
Schema type such as ["string", "null"] so "not stated" is a valid answer instead of a fabricated value.
Retry with error feedback
Re-asking with the document, the failed output and the specific validation errors; useless when the data is absent.
calculated_total / stated_total
Paired fields (computed vs. printed) whose mismatch your code flags as a semantic error.
detected_pattern
Field naming the construct behind a finding, so dismissals can be analyzed by pattern.
Case facts block
Structured, code-maintained facts (amounts, dates, IDs, statuses) re-injected every request outside summarized history.
Lost in the middle
Long-input effect where content in the middle is used less reliably than the start and end.
Independent reviewer
A separate API call with a fresh context (spec + artifact only) instead of self-review.
Multi-pass review
Per-file local passes followed by a cross-file integration pass.
Stratified sampling
Sampling auto-approved items within each segment (doc type, field) to keep measuring error rates.
Calibrated confidence
Confidence thresholds set from measured accuracy on a labeled validation set, per field or segment.

Architecting Claude Platform & API Video

A visual walkthrough of the Messages API request/response lifecycle, tool-use loops, adaptive thinking & effort, SSE streaming, prompt caching, server-side compaction, and multi-cloud deployment.

Audio Deep Dive & Study Downloads

Listen to the Claude Platform & API audio deep dive, or open the 2-column study guide PDF, the 23-slide Architecting Claude deck, and the AI API Architecture Overview infographic in the Hub Viewer.

Production Engineering with the Claude API (Podcast)
Open Study Notes, Slide Deck & Infographic in Hub Viewer Launch Interactive Claude API Practice Quiz →
  • Claude Agent SDK: agent loops, subagents and orchestration on top of the API.
  • Claude Code: configuration, hooks, CLAUDE.md, skills, plugins, Agent SDK, Managed Agents.
  • Anthropic: philosophy, Constitutional AI, interpretability and safety.
  • Prompt Engineering: prompt anatomy and techniques.
  • MCP and RAG: tool protocols and retrieval pipelines.

Self-Assessment Questions (Click to Reveal)

Q1. Claude returns two tool_use blocks in one response. How must the next request be structured?

Append the assistant message unchanged, then a single user message whose content starts with two tool_result blocks, each with the matching tool_use_id (any text goes after them). Failed calls use is_error: true.

Q2. How do max_tokens and model_context_window_exceeded differ, and what happens when the input alone is too large?

max_tokens means the output hit your cap; raise it or continue. model_context_window_exceeded means generation filled the context window; treat it as truncated and shrink history. An input that alone exceeds the window is rejected with a 400 before generation.

Q3. Why is a refusal not handled by your retry logic, and what should the app do?

It arrives as HTTP 200 with stop_reason: "refusal", so status-based retry never sees it, and resending the same request repeats it. Read stop_details, remove or rephrase the refused turn (or reset history), or retry on a fallback model where offered.

Q4. In a streaming handler, when is it safe to parse tool input and when is it safe to commit the turn?

Accumulate input_json_delta fragments and parse at that block's content_block_stop. Commit the assistant turn to history only after message_stop; on a dropped stream, discard the partial turn.

Q5. A team migrates to a Claude 4.7+ model and gets 400s. The request sets temperature: 0.2 and thinking: {type: "enabled", budget_tokens: 6000}. What changes?

Omit the sampling parameter and steer with prompting. Replace manual thinking with thinking: {type: "adaptive"} plus output_config: {effort: ...}.

Q6. A prompt with a 6k-token static system prompt shows 0% cache reads. The breakpoint is on a system block that begins with today's date. Fix?

Move the date (and any per-request text) after the last breakpoint so the cached prefix is byte-identical across requests; keep tool_choice, tools and effort stable too. A longer TTL would only raise write costs.

Q7. An EU insurer requires inference to stay in the EU. Which deployment choices satisfy it?

Not first-party inference_geo, which supports only global or us. Use Amazon Bedrock regional/EU routing or the Google Cloud eu multi-region (both carry a regional premium), pin the model ID, and verify feature availability on that platform.

Q8. Your code wraps each SDK call in its own five-attempt retry loop. What's wrong?

The SDK already retries retriable errors twice by default and honors retry-after, so attempts multiply against a rate limit. Let one layer own retries (or set max_retries), fail fast on 4xx terminal errors, and put fallbacks in orchestration.