Cancellation & Timeouts

call() and cachedCall() accept a standard AbortSignal, which composes with VernLLM's own per-attempt timeout rather than replacing it.

const controller = new AbortController();

const promise = llm.call({
  systemPrompt,
  userContent,
  signal: controller.signal,
});

// cancel from anywhere
controller.abort();

How it composes with timeouts

Each attempt gets its own internal timeout controller. If you also pass a signal, VernLLM combines both into a single signal via AbortSignal.any([yourSignal, internalTimeoutSignal]), either one firing cancels the in-flight request.

If the internal timeout fires, VernLLM throws LLMError('timeout'). If the signal you provide fires, VernLLM throws LLMError('aborted'). This lets callers distinguish between requests that exceeded their time limit and requests that were intentionally cancelled.

The internal timer is always cleared afterward, whether the call succeeds, fails, or is aborted, so nothing is left running in the background.

Fail-fast on an already-aborted signal

If signal.aborted is already true when you call call() or cachedCall(), it rejects immediately with LLMError('aborted') before any request is dispatched, no network call, no wasted attempt.

Aborting during a retry wait

If the signal fires while waiting out a backoff delay between retries, the pending timer is cancelled immediately and the wait rejects right away, it won't sit idle until the delay would have finished on its own.

Cancelling a streaming call

signal works the same way for a stream: true call. If it fires before the stream opens, call() throws LLMError('aborted') before any request is dispatched, same as a non-streaming call. If it fires after the stream has opened, the in-flight request is cancelled and finalResult rejects with LLMError('aborted'); any chunks already delivered through chunks are not retracted. See Streaming for the full contract.

Setting a total time budget

timeoutMs bounds a single attempt and resets on every retry. deadlineMs is different: it is one clock for the whole call, spanning every retry and every fallback target, starting the moment call() is invoked.

const result = await llm.call({
  systemPrompt,
  userContent,
  deadlineMs: 30_000,
});

Once deadlineMs elapses, VernLLM throws LLMError('aborted', { code: 'deadline_exceeded' }), even mid retry or mid fallback. If your own signal fires first instead, the error still has type aborted but no deadline_exceeded code, so the two causes stay distinguishable.

A deadline only bounds getting to a final result: choosing a target, retrying, and opening a stream. Once a stream has actually opened, deadlineMs no longer applies to it: timeoutMs only bounds opening the stream and receiving its first chunk, and every gap after that is bounded separately by chunkIdleTimeoutMs. Neither option, nor deadlineMs, imposes a total time limit on the rest of an already-opened stream. See Per-chunk idle timeout for the full contract.

Cached calls and cancellation

cachedCall() also accepts a caller-provided signal.

Each coalesced caller has its own signal and cancellation lifecycle. Aborting one coalesced caller does not cancel the shared in-flight operation for other callers.

const controller = new AbortController();

const result = await llm.cachedCall({
  cacheKey: `cv:${cvId}`,
  ttl: 3600,
  signal: controller.signal,
  call: {
    systemPrompt,
    userContent,
  },
});