Claude API Deep Dive

Your ticket classifier has run in production for a month: Claude Opus 4.8, a 6,000-token system prompt resent at full price on every call, one synchronous request per ticket, overnight, with nobody watching the output stream. The invoice arrives and it is roughly ten times what the job should cost. Nothing is broken. Every request simply paid retail, because the API's entire discount structure (caching, batching, routing) sat unused.

One endpoint, six levers

Lesson 2.11 drew the three tiers of getting model work done in a program and told you when the bottom one wins: a plain API call is right when the task is inference over input you already have. No loop, no tools you did not build, no repo access. This appendix is that bottom tier taken seriously, because at production volume the plain API call is where the money is, and the difference between a naive integration and a tuned one is routinely 10x on the same workload.

Everything here is one endpoint. POST /v1/messages carries streaming, tool use, caching, batching hooks, and structured outputs as parameters and siblings, not separate products. The minimum request is three fields:

curl -s https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-haiku-4-5",
    "max_tokens": 256,
    "messages": [
      {"role": "user", "content": "Classify this ticket: my invoice shows the wrong amount"}
    ]
  }'

Two response fields deserve operator-level attention, not just parsing. stop_reason tells you why generation stopped: end_turn (natural finish), max_tokens (you truncated it), stop_sequence, tool_use (the model wants a function run), pause_turn (a long turn paused; send the response back as-is to continue), or refusal. Code that only handles the happy path will eventually mistake a truncation for an answer. And usage is the meter: input_tokens, output_tokens, plus two cache fields that the caching section below turns into money. Every cost claim in this appendix is checkable in that object. Log it on every call.

Streaming: progress for humans, insurance for long outputs

Set "stream": true and the API returns server-sent events instead of one JSON body. The event sequence is fixed and worth knowing cold, because you will end up debugging against it: message_start carries an empty message shell, then for each content block a content_block_start, a series of content_block_delta events (for text, deltas of type text_delta), and a content_block_stop. A message_delta near the end carries the final stop_reason and usage, then message_stop closes the stream. ping events arrive interspersed as keepalives, and the docs are explicit that new event types may be added, so switch on known types and ignore the rest instead of throwing.

The part naive handlers miss: errors arrive inside the stream. Under load you can receive an error event carrying overloaded_error, the in-stream equivalent of an HTTP 529, after tokens have already arrived. A handler that treats an open stream as a committed success will hand partial output downstream as if it were complete.

The SDKs wrap all of this. In TypeScript:

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const stream = client.messages.stream({
  model: "claude-opus-4-8",
  max_tokens: 2048,
  messages: [{ role: "user", content: "Draft the migration plan." }],
});

stream.on("text", (delta) => process.stdout.write(delta));
const finalMessage = await stream.finalMessage();
console.error(finalMessage.usage);

Here is the position, since people stream reflexively: streaming is a latency-perception feature and a connection-safety feature, not a throughput feature. Stream when a human is watching tokens render, and stream long generations, where holding a silent HTTP connection open for minutes is a timeout waiting to happen. If the output lands in a database and nobody watches it generate, streaming buys you nothing, and the batches section below actively beats it on price.

Tool use from scratch: the loop under the harness

Lesson 2.11 warned that hand-building a full agent loop around plain API calls means you should move up a tier. That advice stands. But you should build the minimal loop once, by hand, because every harness you will ever operate (Claude Code included) is this loop with better tooling, and debugging harnesses goes faster when you have written the thing they wrap.

The mechanics: you pass a tools array where each tool has a name, a description, and an input_schema in JSON Schema. When the model decides to call one, the response comes back with stop_reason: "tool_use" and one or more tool_use content blocks, each carrying an id, the tool name, and parsed input. You execute the function yourself, then append two things to the conversation: the assistant message exactly as received, and a user message containing a tool_result block per call, matched by tool_use_id. Then you call the API again. That is the whole loop.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const tools: Anthropic.Tool[] = [
  {
    name: "get_order_status",
    description:
      "Look up the current status of an order. Call this whenever the user asks where an order is or mentions an order number.",
    input_schema: {
      type: "object",
      properties: {
        order_id: { type: "string", description: "Order ID, e.g. ORD-1042" },
      },
      required: ["order_id"],
    },
  },
];

// Stand-in for your real system. The loop does not care what it talks to.
async function getOrderStatus(input: unknown): Promise<string> {
  const { order_id } = input as { order_id: string };
  return JSON.stringify({ order_id, status: "shipped", eta: "2026-07-08" });
}

export async function answer(userInput: string): Promise<string> {
  const messages: Anthropic.MessageParam[] = [
    { role: "user", content: userInput },
  ];

  while (true) {
    const response = await client.messages.create({
      model: "claude-opus-4-8",
      max_tokens: 1024,
      tools,
      messages,
    });

    if (response.stop_reason !== "tool_use") {
      const text = response.content.find((b) => b.type === "text");
      return text?.text ?? "";
    }

    messages.push({ role: "assistant", content: response.content });

    const results: Anthropic.ToolResultBlockParam[] = [];
    for (const block of response.content) {
      if (block.type !== "tool_use") continue;
      try {
        results.push({
          type: "tool_result",
          tool_use_id: block.id,
          content: await getOrderStatus(block.input),
        });
      } catch (err) {
        results.push({
          type: "tool_result",
          tool_use_id: block.id,
          content: err instanceof Error ? err.message : "tool failed",
          is_error: true,
        });
      }
    }
    // Every tool_result from this turn goes back in ONE user message.
    messages.push({ role: "user", content: results });
  }
}

Three details in there carry most of the production pain. First, the model may emit several tool_use blocks in one turn (parallel calls are the default); all of their results must return in a single user message, not one message each. Second, a failed tool gets a tool_result with is_error: true, never a dropped result, so the model can recover instead of the conversation desynchronizing. Third, tool_choice is your override: auto is the default, any forces some tool call, a specific tool by name forces that one, none forbids tools, and disable_parallel_tool_use: true caps it at one call per turn. Forcing a specific tool used to be the standard trick for guaranteed JSON; structured outputs, below, replaced it.

One economics note before moving on: tool definitions and the tool-use system prompt are input tokens on every request. The pricing page publishes the overhead per model (a few hundred tokens depending on tool_choice), and your schemas ride on top of that. A fat tool list on a high-volume endpoint is a cache candidate, which is exactly where the next section comes in.

Prompt caching: the discount you have to earn

Prompt caching is the highest-leverage cost lever on the API and the easiest one to silently get wrong. The mechanics first, then the economics, then the trap.

Caching is a prefix match. The request renders in a fixed order (tools, then system, then messages), and a cache_control breakpoint says: everything up to here is reusable. A later request whose rendered prefix matches byte for byte reads the cached prefix instead of reprocessing it. Any change anywhere in that prefix, one byte, invalidates everything after it. There are two ways in: automatic caching, a single top-level cache_control field that places the breakpoint on the last cacheable block and moves it forward as a conversation grows, and explicit breakpoints on individual content blocks, up to four per request, for prompts where different sections change at different rates.

jq -n --rawfile sys system.txt '{
  model: "claude-haiku-4-5",
  max_tokens: 100,
  system: [
    {type: "text", text: $sys, cache_control: {type: "ephemeral"}}
  ],
  messages: [{role: "user", content: "Define term 12 in one line."}]
}' > req.json

curl -s https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d @req.json | jq .usage

Now the economics, as of July 2026, from the pricing page. A cache write with the default 5-minute TTL costs 1.25x base input price; a write with the 1-hour TTL ("ttl": "1h") costs 2x; a cache hit costs 0.1x. On Opus 4.8 that is $5.00 per million tokens base, $6.25 to write for five minutes, $10.00 to write for an hour, and $0.50 to read. So the break-even math is short: with the 5-minute TTL, the second request already wins (1.25x plus 0.1x is 1.35x, against 2x uncached). The 1-hour TTL needs a third request to pay for its doubled write cost. That gives you the doctrine: cache any prefix that repeats at least once within the TTL, which in practice means every system prompt, tool list, and document context on any endpoint with real traffic. Choose 1h only when your traffic has gaps longer than five minutes; steady traffic keeps the 5m cache warm by itself, because hits refresh it. And when first-request latency matters, note that max_tokens: 0 is a legal request that populates the cache without generating anything: a pre-warm.

There is a floor. The minimum cacheable prompt length, as of July 2026, is 1,024 tokens on Opus 4.8 and Sonnet 5, 4,096 on Haiku 4.5, and 512 on Fable 5. Shorter prompts are processed without caching and without any error, which is the first of two silent failure modes.

The second is worse because it costs you the write premium forever. A breakpoint on content that changes per request writes a fresh cache entry every time and reads none. The usage object is your instrument: cache_creation_input_tokens counts what you wrote (and paid the premium on), cache_read_input_tokens counts what you read at 0.1x. If reads sit at zero across identical-looking requests, some byte in your prefix is not identical.

Batches: half price for patience

The Message Batches API takes the same request bodies you have been sending and processes them asynchronously at 50 percent of standard prices. All usage: input, output, cache tokens, everything. As of July 2026 the shape is: up to 100,000 requests or 256 MB per batch, most batches finish in less than an hour, processing is capped at 24 hours (requests still unfinished expire), and results stay downloadable for 29 days.

You create a batch by wrapping ordinary Messages requests, each with a custom_id:

curl -s https://api.anthropic.com/v1/messages/batches \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "requests": [
      {
        "custom_id": "ticket-001",
        "params": {
          "model": "claude-haiku-4-5",
          "max_tokens": 64,
          "messages": [{"role": "user", "content": "Classify as billing, bug, or feature: invoice shows wrong amount"}]
        }
      },
      {
        "custom_id": "ticket-002",
        "params": {
          "model": "claude-haiku-4-5",
          "max_tokens": 64,
          "messages": [{"role": "user", "content": "Classify as billing, bug, or feature: app crashes on login"}]
        }
      }
    ]
  }' | jq -r .id

Then you poll the batch by ID until processing_status moves from in_progress to ended, fetch the results_url from the batch object, and stream back one JSON line per request. Each line carries the custom_id and a result.type of succeeded, errored, canceled, or expired. Handle all four: an errored result with an invalid-request error needs a fixed body, a server error can be retried as-is, and expired requests get resubmitted. Results arrive in whatever order processing finished, so key everything by custom_id and never by position. Code that zips results against the input array by index works in the demo and corrupts data in production.

The routing doctrine between the last two sections is a single test: who consumes the output, and when. If a human is waiting, stream. If a machine consumes it and tonight is soon enough, batch, and take the 50 percent. The cold-open classifier fails this test twice, streaming synchronously to an audience of nobody at full price. Most pipeline workloads (evaluations, enrichment, moderation, bulk generation) are batch workloads wearing a synchronous costume because that is how the first prototype was written.

Structured outputs: the contract moves into the decoder

Lesson 2.11 called a prompted JSON contract "text that is usually JSON" and showed the SDK's outputFormat fixing it at the agent tier. The raw API primitive underneath is structured outputs, and as of July 2026 it is generally available on the Claude API (Fable 5, Opus 4.8, Sonnet 5, Haiku 4.5, and the recent 4.x models) with no beta header. The older top-level output_format parameter still works for a transition period; the current shape is output_config.format.

curl -s https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-haiku-4-5",
    "max_tokens": 200,
    "output_config": {
      "format": {
        "type": "json_schema",
        "schema": {
          "type": "object",
          "properties": {
            "category": {"type": "string", "enum": ["billing", "bug", "feature", "other"]},
            "urgent": {"type": "boolean"}
          },
          "required": ["category", "urgent"],
          "additionalProperties": false
        }
      }
    },
    "messages": [{"role": "user", "content": "Classify: my card was charged twice and I need it fixed today"}]
  }' | jq -r '.content[0].text' | jq .

This is not a stronger prompt. It is constrained decoding: the schema compiles to a grammar and the sampler cannot emit tokens that violate it, so the text in response.content[0].text parses, has the required fields, and has the declared types, every time. Its sibling is strict tool use: put strict: true on a tool definition (with additionalProperties: false and a required list in the schema) and tool inputs get the same guarantee, which kills the classic agent failure of a tool call missing a required argument.

The guarantees have edges you should design around rather than discover. The schema dialect is restricted: no recursive schemas, no numerical constraints like minimum or multipleOf, no minLength or maxLength, and additionalProperties must be false on every object; an unsupported keyword is a 400, not a silent downgrade. The first request with a new schema pays grammar-compilation latency, after which the compiled grammar is cached for 24 hours from last use. And changing output_config.format invalidates the prompt cache for that conversation thread, so a schema you tweak per request quietly forfeits the caching section's discount. Schema validity also says nothing about truth: the JSON will parse, and the classification can still be wrong. Verification of content stays your job; structured outputs only retires the parsing failure class.

Routing: pick the model like you pick an instance type

The lineup as of July 2026, from the models and pricing pages (prices per million tokens, input/output):

Prices move and models ship; treat this table as a snapshot and the pricing page as the source. The GET /v1/models endpoint lists what your key can reach, so route against live data rather than hardcoded assumptions.

The routing doctrine this course takes: route by failure cost and by who catches failures, not by benchmark vibes. High-volume, narrow, machine-verified tasks go to Haiku 4.5, because your verification harness, not the model's ceiling, is what guarantees quality there, and a 5x price gap compounds brutally at volume. Interactive product work and general pipeline reasoning default to Sonnet 5, the deliberate speed-and-intelligence middle. Long-horizon agentic work, hard synthesis, and anything where a wrong answer is expensive to detect goes to Opus 4.8, which is also the documentation's own default recommendation for complex agentic coding. Fable 5 is the escalation tier: reach for it when a task measurably defeats Opus 4.8 and the outcome justifies double the token price, not as a default.

Two mechanical constraints shape routing more than people expect. Caches are per-model, so bouncing a conversation between models rewrites the cache each hop: route per task, not per turn. And a cheap model plus the Batches API stacks: Haiku at batch rates is $0.50 per million input tokens, one tenth of synchronous Opus 4.8, which is the arithmetic behind the cold open's 10x.

Where it breaks

The cache that never hits. The costliest failure in this appendix because it is invisible without instrumentation: a breakpoint plus a volatile prefix means you pay the 1.25x write premium on every single request while reads stay at zero. Alert on the ratio of cache_read_input_tokens to input_tokens for any endpoint that claims to cache. Below the model's minimum prefix length the request silently processes uncached too, and Haiku's 4,096-token floor surprises teams who tuned their prompt on Opus's 1,024.

Batch expiry and ordering assumptions. A batch is not a guarantee, it is a 24-hour window. Requests that do not finish return expired and need resubmission, so a pipeline without an expired-handling branch loses data on its worst day. And results are unordered by design; matching by array index instead of custom_id is the classic first-week bug.

Schema rejections and schema rigidity. Structured outputs 400 on unsupported JSON Schema features, which bites hardest when you generate schemas from a typed model whose defaults emit minLength or open objects. The subtler failure is 2.11's lesson replayed at the API tier: an overconstrained enum meets an input that fits nothing, and the output is forced into the least-wrong bucket. Escape hatches (an other category, optional-by-design fields) are schema engineering, not sloppiness.

Streams that end in errors. An error event after 500 delivered tokens is still a failed request. Buffer, check for a clean message_stop, then commit.

python3 -c "print('Glossary. ' + ' '.join(
    f'Term {i}: a padded glossary entry used to exceed the minimum cacheable prefix.'
    for i in range(700)))" > system.txt

BATCH_ID=msgbatch_...   # from step 6's create call

curl -s https://api.anthropic.com/v1/messages/batches/$BATCH_ID \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" | jq '{processing_status, request_counts}'

Knowledge check

Sources

• Messages API reference: https://platform.claude.com/docs/en/api/messages.md (fetched July 2026)
• Streaming messages: https://platform.claude.com/docs/en/build-with-claude/streaming.md (fetched July 2026)
• Tool use with Claude: https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview.md (fetched July 2026)
• Prompt caching: https://platform.claude.com/docs/en/build-with-claude/prompt-caching.md (fetched July 2026)
• Batch processing: https://platform.claude.com/docs/en/build-with-claude/batch-processing.md (fetched July 2026)
• Structured outputs: https://platform.claude.com/docs/en/build-with-claude/structured-outputs.md (fetched July 2026)
• Models overview: https://platform.claude.com/docs/en/about-claude/models/overview.md (fetched July 2026)
• Pricing: https://platform.claude.com/docs/en/about-claude/pricing.md (fetched July 2026)