Reliable LLM calls,
by default.
The LLM call framework. Resilience, observability, and control for every call. Retries, timeouts, provider fallback, rate limiting, circuit breaking and more, dependency-light and typed from the start.
$npm install vern-llm$pnpm add vern-llm$yarn add vern-llm$bun add vern-llm$npm install vern-llm$pnpm add vern-llm$yarn add vern-llm$bun add vern-llm$npm install vern-llm$pnpm add vern-llm$yarn add vern-llm$bun add vern-llm$npm install vern-llm$pnpm add vern-llm$yarn add vern-llm$bun add vern-llmimport Anthropic from '@anthropic-ai/sdk';
import OpenAI from 'openai';
import { fromAnthropic, fromOpenAI, VernLLM } from 'vern-llm';
const openai = fromOpenAI(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }));
const anthropic = fromAnthropic(new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }));
export const llm = new VernLLM({
client: openai,
model: 'gpt-4o',
fallback: { client: anthropic, model: 'claude-sonnet-5', circuitBreaker: true },
rateLimit: { requestsPerMinute: 500, tokensPerMinute: 100_000, maxConcurrent: 20 },
retryBudget: { windowMs: 60_000, minCalls: 20, retryRatio: 0.2 },
maxRetries: 3,
timeoutMs: 10_000,
defaultMaxTokens: 1000,
defaultReasoningEffort: 'medium'
});
const result = await llm.cachedCall({
cacheKey: 'weather:new-york',
ttl: 3600,
call: { userContent: "What's the weather in New York?" }
});What this call actually does.
fallback:→ Falls over to a backup target on failure, in processcircuitBreaker: true→ Stops repeated failures from cascadingrateLimit:→ Queues locally under a per-minute ceilingretryBudget:→ Caps how much recent traffic can be retriesmaxRetries: 3→ Retries transient failures with backoff and jittertimeoutMs: 10_000→ Prevents attempts from hanging indefinitelydefaultMaxTokens: 1000→ Applied to any call that omits its owndefaultReasoningEffort: 'medium'→ Sets reasoning depth across providerscachedCall→ Returns cached results without another API call
LLM calls fail in ways plain SDK calls do not handle: timeouts, rate limit errors, a provider having an outage, or a request that just hangs. VernLLM adds retries with backoff, a circuit breaker, provider fallback, rate limiting, and caching around your existing client, so a single bad call does not take down your app.
VernLLM runs in your own process, so there is no extra network hop or proxy to maintain. It is built around small interfaces rather than one config object, so caching, rate limiting, and the circuit breaker can each be swapped for your own implementation. Running in-process also means it can react to your own application logic, not just the request and response, catching failures a gateway watching traffic from outside your app would miss. A gateway is still the better choice for one shared setup across many services or languages.
Calling the client directly means you own retries, timeouts, circuit breaking, and caching yourself, code most teams end up rewriting per project. VernLLM ships those as configurable options on one class, so you keep your existing provider client and wrap it instead of reimplementing the resilience layer.
Yes. cachedCall accepts any adapter implementing get/set (delete is optional), so Redis, a database, or a custom store can replace the built-in in-memory cache without changing how you call it.
Yes, through middleware. transform edits or redacts an outgoing request before it is sent, and wrap runs around a whole logical call, retries and fallback attempts included, for logging, tracing, or cost tracking.
Yes, written in TypeScript from the ground up. Structured output schemas, call params, and errors are all typed, so mistakes surface at compile time instead of at runtime.
Zero runtime dependencies. VernLLM does not bundle Zod or provider SDKs, it relies on compatible interfaces instead, so you bring your own provider clients and schema validators while keeping your dependency tree minimal.