unit U3 — 3 of 4

Failure modes

refusals, timeouts, schema drift

An LLM call fails in ways an ordinary HTTP request never does. The transport can return a clean 200 while the body is a polite refusal, a truncated half-object because you hit max_tokens, or valid JSON whose priority field arrived as a number instead of your expected string. None of these throw at the network layer, so a naive flow treats them as success and writes nonsense downstream. In production you must inspect the semantic result, not just the status code.

The engine gives you the branches to handle this. On the HTTP Request Piece, set a request timeout and enable retry-on-failure with backoff so a slow provider under load doesn’t wedge the run. Read the response’s finish or stop reason: a value of length means the output was cut off and your JSON is incomplete; a refusal or safety stop means there is no data to parse at all. Feed the parsed object through a Code Piece that checks each field’s type against the contract, then use a Router step to send clean results, refusals, and drift down separate paths.

Where it breaks: the single happy path. A flow that pipes the model output straight into a database write, with no branch for the abnormal cases, corrupts a row the first time the model refuses or drifts — and because nothing threw, no alert ever fires. Build the failure branches before you go live, not after the client finds bad data sitting in their CRM.

worked example

A guard in a Code Piece rejecting refusals, truncation, and schema drift before an invoice-extraction result is stored.

export const code = async (inputs) => {
  const res = inputs.modelResponse;              // mapped from the HTTP step

  if (res.stop_reason === 'refusal') {
    throw new Error('model refused — route to needs_human');
  }
  if (res.stop_reason === 'max_tokens') {
    throw new Error('output truncated — raise max_tokens or shrink input');
  }

  const out = JSON.parse(res.content);
  if (typeof out.total !== 'number' || typeof out.currency !== 'string') {
    throw new Error('schema drift — total/currency wrong type');
  }

  return out;
};

field checklist

Set a request timeout and retry-with-backoff on the HTTP step.
Check the stop reason before you trust any content.
Reject truncated output caused by hitting max_tokens.
Type-check every field against the contract in a Code Piece.
Route refusals and drift down a separate Router branch.

common failure — A 200 that was actually a refusal

An invoice extractor treated every 200 as success and fed the body straight into an accounting sync. One vendor PDF tripped a safety refusal, so the model returned an apology sentence rather than JSON; a lenient parser defaulted the missing total to zero, and a bogus invoice entered the ledger. Nothing errored, so it surfaced only at month-end reconciliation. Read stop_reason first, validate field types, and branch refusals to a human queue.

check your understanding

An HTTP 200 came back from the model. Select everything that must still be verified before the parsed result is written downstream.

select every one that applies — partial answers are marked wrong

next unit opens once this is passed

sandbox validation

The check above confirms you followed the unit. Marking the module COMPLETED takes more: build the automation in your own engine and submit the exported flow and its run evidence, signed, to your unique validation URL. See the module page for that spec.