Retries

const llm = new VernLLM({
  client: fromOpenAI(openai),
  model: 'gpt-4o',
  maxRetries: 3, // retries after the first attempt, so 4 attempts total
  baseDelayMs: 500, // base for exponential backoff between retries
});

Every call() retries transient failures automatically. maxRetries defaults to 1 (2 attempts total), so this is active even if you never touch the option.

Backoff and jitter

Each retry waits longer than the last, and the exact wait is randomized within a range rather than fixed:

exp = min(baseDelayMs * 2 ** attempt, maxDelayMs);
delay = random() * exp;

This is full jitter: the delay lands anywhere between zero and the computed exponential value, instead of every caller waiting the identical amount. When many callers hit a failure at the same time, full jitter spreads their retries out instead of having them all retry in lockstep and hit the provider again at the same instant. See AWS's Exponential Backoff and Jitter for why full jitter is preferred over equal jitter.

The delay is capped at DEFAULT_MAX_DELAY_MS (10 seconds) regardless of attempt, so a high maxRetries never produces an unbounded wait.

Honoring Retry-After

If the failed attempt's error carries a Retry-After header, that value is used for the wait instead of the computed backoff:

recoverDelay = retryAfterMs ?? getBackoffDelay(baseDelayMs, attempt);

Millisecond headers and malformed values

A provider's own timing guidance is honored ahead of the default backoff curve. Some providers surface a Retry-After-Ms or X-Retry-After-Ms header for finer granularity than a whole second allows, and that value is checked before falling back to the standard Retry-After header.

A value that gives no usable timing, a negative delta or a date already in the past, is treated the same as a missing header rather than as an immediate retry. It falls through to the computed backoff instead.

Backing off harder for rate limits and server errors

When no explicit timing is present, the computed backoff itself is not one single curve. A rate-limited response with no explicit timing is still a clear signal to slow down, so it waits longest. A server fault waits somewhat longer than the default curve, since it is a transient failure the provider did not choose to send. Every other retryable failure keeps the default curve described above.

What gets retried

shouldRetry skips a retry when any of these are true:

• The signal has already aborted
• error.type is parse or validation, since these are deterministic response processing failures that a retry will not fix
• error.status is in nonRetryableStatus (default 400, 401, 403, 404, 422)

Everything else, including timeouts, 5xx errors, network failures, and unknown errors, is retried up to maxRetries.

SDK-internal retries

Every option on this page governs VernLLM's own retry loop, the one wrapping client.chat.completions.create() (or the equivalent per adapter). It has no visibility into what happens inside that call. fromOpenAI, fromAnthropic, fromGemini, and fromBedrock each wrap an official provider SDK, and most of those SDKs retry transient failures, including 429s, on their own, by default, before VernLLM's create() call ever returns. @google/genai is the exception: it only retries when the client is explicitly configured to.

fromFetch is the one exception when left at its default: it calls native fetch directly, so VernLLM's own retry loop is the only one in play there. If you swap in a custom request or requestStream transport via FetchAdapterConfig, that transport (e.g. axios) may add its own retry layer, the same as the official SDK adapters above.

This is invisible, not incorrect, but it has real consequences worth knowing about:

• Amplified retries. A failure can be retried by the SDK, then, if it still fails, retried again by VernLLM, each with its own separate backoff policy, uncoordinated with the other.
• A blind circuit breaker. A burst of 429s the SDK quietly retries through to eventual success never reaches VernLLM as a failure, so the breaker never sees it, even though the provider was genuinely under strain.
• A blind rate limiter. AIMD's reactive shrink only fires once a 429 actually reaches VernLLM; an SDK that retries the 429 away internally gives AIMD nothing to react to until the SDK's own retries are exhausted too.
• Blind telemetry. onEvent's retry events, the logger, and usage metering all see nothing while a retry happens at the SDK layer.
• A shared timeout budget. timeoutMs wraps the whole create() call, SDK-internal retries included, so they can eat into the budget before VernLLM's own retry loop gets a turn.

If you want VernLLM to be the sole retry authority, disable the SDK's own retries when constructing the client:

const openai = new OpenAI({ apiKey, maxRetries: 0 });
const anthropic = new Anthropic({ apiKey, maxRetries: 0 });
// AWS SDK v3: new BedrockRuntimeClient({ maxAttempts: 1, ... })
// @google/genai: check the SDK's own retry-options field for the installed version

Cancelling mid retry

If a signal fires while a retry is waiting out its backoff delay, the pending wait is cancelled immediately rather than sitting idle until the delay finishes. An abort during backoff is never retried, it is treated as a deliberate cancellation. See Cancellation & Timeouts for the full abort lifecycle.

Reading attempt history

Every normalized LLMError carries an optional attempts array. Each entry has the attempt's index and a snapshot of that attempt's error: an LLMErrorSnapshot with message, type, code, status, issues, retryAfterMs, retryable, and its own nested attempts if that attempt was itself the terminal failure of a retry loop. It's a snapshot, not a live LLMError, since a past attempt is a record, not something you'd catch or rethrow. cause isn't part of it. cause is meant to be read on the live error you actually caught (err.cause), not carried inside recorded history. Each entry can also carry request, an LLMRequestSnapshot of what was actually sent for that attempt, with auth headers always removed. See Error Handling for cause and for the full LLMRequestSnapshot shape.

try {
  await llm.call({ userContent: 'hello' });
} catch (err) {
  if (isLLMError(err)) {
    for (const attempt of err.attempts ?? []) {
      console.log(attempt.index, attempt.error.type);
    }
  }
}

attempts is absent when nothing was retried, for example a call that failed on its first and only try. With fallback configured, each FallbackAttempt on a FallbackExhaustedError extends this same shape, adding provider and model, and that attempt's own error.attempts still holds the retries made against that one target. See Provider Fallback for FallbackExhaustedError itself.

Options reference

See Configuration for every option alongside timeout and circuit breaker settings.