Error Handling

When a raw API error or unknown thrown error is normalized, the original error is preserved on err.cause, in case the normalized type/status/issues aren't enough to diagnose the failure. Existing LLMError instances are returned unchanged, so internally-created errors such as timeout, parse, and validation errors may not have a cause.

import { isLLMError } from 'vern-llm';

try {
  const result = await llm.call({ systemPrompt, userContent, schema });
} catch (err) {
  if (isLLMError(err)) {
    switch (err.type) {
      case 'validation':
        console.log(err.issues); // the schema's error object, or the tool contract issues
        break;
      case 'invalid_params':
        console.log(err.message); // the call site's own input was malformed
        break;
      case 'api':
        console.log(err.status); // HTTP status, if the provider returned one
        console.log(err.cause); // the original error thrown by the provider client
        console.log(err.retryAfterMs); // Retry-After from the last attempt, if the provider sent one
        console.log(err.code); // more specific machine readable error code, when available
        break;
      case 'circuit_open':
        // provider is down, back off and do not retry immediately
        break;
    }
  }
  throw err;
}

Error types

Error codes

LLMError.code provides a more specific machine readable error code when LLMError.type alone does not describe the failure precisely. type and code are independent: code is not owned by a single type, so the same code can in principle appear under a different type as VernLLM learns to detect more.

The currently defined tool contract codes are:

The currently defined caller input codes are:

Everything else under invalid_params stays uncoded: the thrown message already names the exact problem, and a code only earns its place when a caller would realistically branch on it. These six are exactly the ones whose message already embeds structured data a caller might want directly instead of re-parsing it out of prose: a joined list of names/ids for the five history- and tools-shape checks, or the specific missing capability for unsupported_capability; see issues below for the typed payload each one carries.

The currently defined rate-limiting codes are:

See Rate Limiting for the full rateLimit option and what triggers each of these. See Retry Budget for retry_budget_exhausted specifically.

The currently defined HTTP status codes are:

These are derived purely from the HTTP status code, so they apply the same way regardless of which adapter or client raised the error.

The currently defined connectivity code is:

The currently defined fallback code is:

See Provider Fallback for the full fallback option and what makes fallbackOn move to the next target versus stop.

Soft failure detection

A response can parse cleanly and pass schema validation without actually being a good answer. A model that returns a placeholder, an empty-but-technically-present string, or a low-confidence refusal looks like a success to retries and the circuit breaker, since nothing threw. The detectSoftFailure option runs once per attempt, right after a response is shaped, and can turn that result into a real failure before it reaches the caller:

const llm = new VernLLM({
  client: fromOpenAI(openai),
  model: 'gpt-4o',
  detectSoftFailure: (result, meta) => {
    // meta: { requestId, model, providerName, isFallback, attempt }
    if (typeof result === 'string' && result.trim() === 'N/A') {
      return 'soft_failure_detected';
    }
    return undefined;
  },
});

Returning undefined leaves the result as a success. Returning an LLMErrorCode throws LLMError('api', code), using soft_failure_detected as a reasonable default when no more specific code applies.

A soft failure flows through the exact same retry and circuit breaker paths a real failure would: with maxRetries configured it can still succeed on a later attempt, and with circuitBreaker configured, repeated soft failures count toward the same threshold repeated real failures would. See Retries and Circuit Breaker.

A throwing hook is caught, logged, and treated the same as returning undefined, so a broken hook doesn't fail every call it's attached to.

For stream: true calls, detectSoftFailure runs at the same point, once the stream has fully accumulated, and a soft failure rejects finalResult the same as any other post-stream failure.

fallback targets each accept their own detectSoftFailure. A target that leaves it unset inherits the parent instance's hook, unlike circuitBreaker and rateLimit, which are always independent per target and never inherited.

VernLLM already fails a call outright when a response arrives with no content and no tool calls at all, regardless of whether detectSoftFailure is configured. detectSoftFailure is for judgment calls beyond that baseline: a specific placeholder string, a truncated JSON shape, or anything else that's a failure for your application but not for VernLLM in general.

The currently defined soft failure code is:

try {
  await llm.call({
    userContent: 'Use the available tools.',
    tools: [weatherTool],
  });
} catch (error) {
  if (isLLMError(error) && error.code === 'unknown_tool') {
    console.log('The model requested a tool that was not offered.');
  }
}

code is optional. Errors without a more specific machine readable code leave code undefined.

issues

Three fields split one failure three ways: message is prose for a human, code is a stable string a program can === against, and issues is the structured data behind that specific failure, for a program to act on without re-parsing it out of message. Two calls can throw the same code with completely different specifics (which tool names collided, which history index was bad); issues is where those per-instance specifics live.

LLMErrorIssuesByCode maps each code that carries structured issues to that payload's exact shape:

interface LLMErrorIssuesByCode {
  unknown_tool: ToolIssue[];
  duplicate_tool_call_id: ToolIssue[];
  duplicate_tool_names: { names: string[] };
  unknown_tool_choice: { requested: string; available: string[] };
  duplicate_tool_result_ids: { historyIndex: number; ids: string[] };
  unknown_tool_result_ids: { historyIndex: number; ids: string[] };
  missing_tool_results: { historyIndex: number; ids: string[] };
  unsupported_capability: { capability: string };
}

Not every code appears there. Most invalid_params failures are a single deterministic fact the message already states in full (an empty tools array, toolChoice set without tools, a schema combined with jsonMode: false), so a typed issues entry for them would only duplicate the message into a field, the same near-duplicate-code problem code itself avoids by staying uncoded there. The codes that do appear are exactly the ones whose message already string-joins a list.

code stays the only discriminator; hasIssues just gives that existing check a typed return instead of a manual cast:

import { isLLMError, hasIssues } from 'vern-llm';

try {
  await llm.call({ userContent: 'hi', tools: [weatherTool, weatherTool] });
} catch (err) {
  if (isLLMError(err) && hasIssues(err, 'duplicate_tool_names')) {
    console.log(err.issues.names); // string[], fully typed, no cast
  }
}

Schema-validation failures (type: 'validation', no code) are the one deliberate exception: issues there is the caller's own Zod-compatible validator's error object, a shape VernLLM can't know in advance since it accepts any compatible validator. That case stays unknown on LLMError.issues rather than being forced into LLMErrorIssuesByCode.

Tool contract issues

A single model response can contain multiple tool contract problems. When this happens, LLMError.issues contains the complete set of detected issues, while the top level code identifies the primary contract error.

Each issue identifies the affected tool, tool call ID, and machine readable error code:

interface ToolIssue {
  name: string;
  toolCallId: string;
  code: 'unknown_tool' | 'duplicate_tool_call_id';
  detail?: unknown;
}

FallbackExhaustedError

When fallback is configured and every target fails, call() throws FallbackExhaustedError instead of the last target's raw error. It extends LLMError, so isLLMError and any instanceof LLMError check still passes, and it carries attempts, a snapshot of every target's own error, in the order they were tried:

class FallbackExhaustedError extends LLMError {
  readonly attempts: FallbackAttempt[];
}

interface FallbackAttempt {
  index: number; // -1 for the primary
  provider: string;
  model: string;
  error: LLMErrorSnapshot;
  request?: LLMRequestSnapshot;
}

FallbackExhaustedError.type is always 'fallback_exhausted', its own identity, rather than inheriting the last attempted target's own type: every target failing is a meaningfully different event from any single target's own failure. status and retryAfterMs still inherit from the last attempt, and retryable on this class defers to the last attempt's own retryable rather than anything about the fallback_exhausted type itself, since the type alone carries no retry signal.

A lone target with no fallback configured, or a chain where fallbackOn returns 'stop' on the very first failure, throws the plain, unwrapped LLMError instead, exactly as it did before fallback existed. FallbackExhaustedError only appears once more than one target was actually tried. See When every target fails.

For stream: true, this applies to opening the stream, not to what happens after. A failure before the first chunk arrives goes through the same fallback chain as a non-streaming call, so call() itself can reject with FallbackExhaustedError once every target's stream fails to open. Once a target's stream has opened and call() has returned { chunks, finalResult }, a failure partway through never falls over: finalResult rejects with the normalized LLMError from whichever target was streaming, and fallback is not attempted. See Streaming: open failures only for why.

retryable

LLMError.retryable is computed purely from type/code, independent of any specific call's nonRetryableStatus list or whether its signal has since aborted:

if (isLLMError(err) && !err.retryable) {
  // safe to give up immediately, without inspecting type/code yourself
}

It is false when:

• type is 'parse', 'validation', 'invalid_params', or 'aborted'
• code is one of the tool contract codes (unknown_tool, duplicate_tool_call_id, tool_choice_none_violated, unexpected_tool_calls)
• code is one of the local rate limit codes (rate_limit_queue_full, rate_limit_queue_timeout, rate_limit_capacity_exceeded, retry_budget_exhausted)

FallbackExhaustedError overrides this to defer to the last attempted target's own retryable, since type: 'fallback_exhausted' by itself says nothing about whether retrying could help.

What gets retried

shouldRetry skips a retry when any of these are true:

• The signal has already aborted, or error.type is 'aborted'
• error.type is 'parse', 'validation', or 'invalid_params' because these are deterministic input or response processing failures rather than transient provider faults
• error.code is 'unknown_tool', 'duplicate_tool_call_id', 'tool_choice_none_violated', or 'unexpected_tool_calls' because these are deterministic tool contract failures
• error.code is 'rate_limit_queue_full' or 'rate_limit_queue_timeout' because the wait already happened and retrying would only requeue behind the same limit with nothing changed, or 'rate_limit_capacity_exceeded' because the call could never fit the configured tokensPerMinute capacity in the first place, so no amount of retrying changes that. See Interaction with retries.
• error.code is 'retry_budget_exhausted' because a configured retryBudget decided this target has already had too many retries recently; further retrying against it is refused until the trailing window's ratio drops. See Retry Budget.
• error.status is in nonRetryableStatus (default [400, 401, 403, 404, 422])

Everything else, including timeouts, 5xx errors, network failures, and unknown errors, is retried up to maxRetries. If the failed attempt's error carries a Retry-After header (delta seconds or HTTP date form), that value is honored for the wait, capped at the same maximum delay as backoff. Otherwise the wait falls back to exponential backoff with jitter.

Tool contract errors are not retried

unknown_tool, duplicate_tool_call_id, tool_choice_none_violated, and unexpected_tool_calls are deliberately non retryable. See Tool Calling for the canonical contract workflow and handling guidance. These errors therefore fail immediately instead of consuming the remaining maxRetries.

For raw API or unknown thrown errors, the underlying error is preserved on err.cause, so it is still available even though the top level message is a generic 'LLM request failed'. Existing LLMError instances are not wrapped again, so some error types such as timeout, parse, and validation may not have a cause.

kind on events versus type on errors

VernLLM's event stream uses a kind discriminant, not type, and kind: 'rate_limited' shares a string with LLMErrorType: 'rate_limited' without meaning the same thing. A rate_limited event fires whenever a call sat in the local rate limiter's queue before clearing, success included: that's the entire point of the event, reporting that rate limiting affected timing even when nothing failed. A rate_limited error only exists when a call did not clear: the queue timed out, the queue was full, or the request could never fit at all. Reading kind: 'rate_limited' as a failure count would be a real, plausible misreading, since most events with that kind correspond to calls that went on to succeed.

Debugging a failed call

When debug: true is set, a failure is also logged through the injected logger's debug method, independent of catching the thrown LLMError yourself:

const llm = new VernLLM({ client, model, debug: true });

[vern:<requestId>] error:
<the provider's error message or response body>

This is useful when a failure happens somewhere you aren't directly awaiting the call() (for example, inside cachedCall's coalesced callers), since the log fires regardless of which caller ends up seeing the thrown error.

err.cause itself isn't guaranteed to be safe to JSON.stringify on its own, since some SDKs throw errors with circular references. JSON.stringify(err) leaves cause out of its own serialization for that reason, along with the structured type/code/status/issues fields it does include. issues gets its own check: for a schema validation failure it is the caller's own validator's error object, a shape VernLLM cannot know in advance. A circular issues value (or one nested inside attempts, which carries its own issues per recorded failure) is replaced with a short marker string in the serialized output rather than dropped silently or left to throw. It also adds message and retryable, since a plain Error subclass without this wouldn't show either: message is non-enumerable on Error, and retryable is a getter, not an own property. err.attempts works the same way and never carries cause.

Request snapshots

Each recorded attempt carries both sides of the story: attempt.error is what came back, and attempt.request is what was sent. attempt.request is an LLMRequestSnapshot, holding the provider, the model, the request body as actually sent for that attempt, and the wall clock time the attempt started:

interface LLMRequestSnapshot {
  provider: string;
  model: string;
  body: unknown;
  headers?: Record<string, string>;
  startedAt: number;
}

Auth headers are always removed before a snapshot is built, so headers never includes an API key or bearer token. Use err.attempts[i].request alongside err.attempts[i].error to see the exact payload that produced a given failure:

if (isLLMError(err)) {
  for (const attempt of err.attempts ?? []) {
    console.log(attempt.error.message, attempt.request?.body);
  }
}

request is optional on RetryAttempt, so it is absent on attempts recorded before this field existed.