unit U2 — 2 of 4

Token economics

budgets, truncation, caching

Every call to an LLM bills you twice — once for the tokens you send, once for the tokens it generates — and both count against a fixed context window. On a flow that fires per record, a few thousand wasted input tokens per call becomes a line item finance will ask about. Token economics is the discipline of deciding, before the request leaves the HTTP Request Piece, exactly how much context is worth sending and how much output you are willing to pay for.

Three levers live in the flow. First, cap the response with max_tokens so a chatty model can’t run up a bill or blow your latency SLA. Second, trim the input in an Edit Fields step or Code Piece upstream — strip boilerplate, drop old turns, summarise long documents before they reach the prompt. Third, exploit prompt caching: mark a stable system prefix as cacheable so repeated calls reuse it at a fraction of the cost. Measure real usage from the response’s token counts in the run history, not a guess.

Where it breaks: the unbounded loop. Wiring a full conversation history or an entire uploaded PDF straight into every iteration means input grows without limit — until you either hit the context ceiling and get a hard error mid-run, or quietly pay ten times what the task actually needed. Truncate deliberately, summarise the long tail, and cache the constant parts so only the fresh delta costs full price.

worked example

A digest automation summarising a client’s daily Slack export, where the system prompt is identical on every run.

{
  "model": "{{ connections.llm.model }}",
  "max_tokens": 400,
  "system": [
    {
      "type": "text",
      "text": "You summarise team chatter into five bullet points. Keep to the house style.",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    { "role": "user", "content": "{{ step_2.dailyExportTrimmed }}" }
  ]
}

field checklist

Cap every call with max_tokens to bound cost and latency.
Trim boilerplate and stale turns before the prompt leaves the flow.
Mark the stable system prefix cacheable to reuse it cheaply.
Truncate or summarise long documents instead of sending them whole.
Read actual token counts from the run history, not estimates.

common failure — Runaway bill from unbounded input

A nightly enrichment ran over ten thousand records, and each call forwarded the entire prior thread plus a full product catalogue. Input tokens climbed every iteration, the monthly invoice tripled, and one run finally errored on the context limit. The fix stacks the three levers: cap max_tokens, project only the fields the task needs in an Edit Fields step, and cache the unchanging catalogue prefix so it stops being re-billed.

check your understanding

A nightly enrichment over ten thousand records forwards the entire prior thread plus a full product catalogue on every call. The invoice tripled and one run finally errored on the context limit. Which lever addresses the catalogue specifically?

next unit opens once this is passed

sandbox validation

The check above confirms you followed the unit. Marking the module COMPLETED takes more: build the automation in your own engine and submit the exported flow and its run evidence, signed, to your unique validation URL. See the module page for that spec.