What is VernLLM

Get production-grade LLM calls, without any bloat

VernLLM is the LLM call framework. Resilience, observability, and control for every call. A lightweight layer that adds retries, timeouts, circuit breakers, provider fallback, rate limiting, caching, streaming, structured outputs, schema validation, usage tracking, and observability events with sensible defaults and minimal dependencies.

import Anthropic from '@anthropic-ai/sdk';
import OpenAI from 'openai';
import { fromAnthropic, fromOpenAI, VernLLM } from 'vern-llm';

const llm = new VernLLM({
  client: fromOpenAI(new OpenAI({ apiKey: process.env.OPENAI_API_KEY })),
  model: 'gpt-4o',

  // Reliability defaults
  maxRetries: 3,
  timeoutMs: 10_000,
  circuitBreaker: true,
  rateLimit: { requestsPerMinute: 500, maxConcurrent: 20 },

  // Falls over to a declared backup target when the primary fails
  fallback: {
    client: fromAnthropic(new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY })),
    model: 'claude-sonnet-5',
    circuitBreaker: true,
  },

  // Observability
  onUsage: ({ totalTokens }) => {
    console.log(`Used ${totalTokens} tokens`);
  },
  onEvent: (event) => {
    if (event.kind === 'fallback') console.warn(`falling over ${event.from} -> ${event.to}`);
  },
});

const getWeather = {
  name: 'get_weather',
  description: 'Gets the current weather for a city',
  parameters: { type: 'object', properties: { city: { type: 'string' } }, required: ['city'] },
};

const city = 'Denver';

const { chunks, finalResult } = await llm.cachedCall({
  cacheKey: `weather:${city}`,
  ttl: 60,
  call: {
    userContent: `What's the weather in ${city}?`,
    tools: [getWeather],
    stream: true,
  },
});

for await (const chunk of chunks) {
  if (chunk.type === 'text-delta') process.stdout.write(chunk.delta);
}

const result = await finalResult; // cached, retried, and streamed, tool calls included

Why VernLLM?

• Retries with backoff: transient failures get retried automatically. Parse errors, validation errors, and non retryable status codes fail fast instead
• Provider fallback: declare an ordered list of backup targets, tried in order after the primary, with no scoring or health-checking involved
• Client-side rate limiting: queue locally against requests-per-minute, tokens-per-minute, and concurrency ceilings instead of letting the provider reject the call
• Structured output: pass a Zod schema and get a typed, validated result back
• Provider native JSON Schema mode: constrain generation itself, not just validate after the fact
• Streaming: set stream: true on any call and get live chunks alongside the same validated result call() would have returned
• Caching: wrap any LLM call with cachedCall, and bring your own adapter such as Redis or Upstash
• Middleware: transform outgoing requests, and observe or wrap the outcome of a call, for logging, tracing, redaction, or cost tracking
• Circuit breaker: trips after repeated failures and recovers automatically once the provider is healthy again
• Observability: one onEvent stream reports retries, fallovers, circuit transitions, and rate-limit waits, so nothing about a call's reliability path is a black box
• One interface, every provider: OpenAI, Groq, Mistral, DeepSeek, Cerebras, Together, Fireworks, Ollama, Anthropic, Gemini, Bedrock, or raw HTTP through fromFetch
• Zero runtime dependencies: VernLLM is provider- and validator-agnostic. It works with compatible LLM clients and any schema validator exposing a safeParse interface, such as Zod.

Who's using VernLLM?

• Applera uses VernLLM as its primary LLM call layer, backed by the Groq adapter and a custom Upstash cache adapter, for structured resume/job matching and application generation.