Getting Started

Install

pnpm add vern-llm

Dont forget to install your AI provider!

In this example we will be using OpenAI. See the Adapters documentation for supported AI providers.

pnpm add openai

Basic usage

import OpenAI from 'openai';
import { VernLLM, fromOpenAI } from 'vern-llm';

const openai = fromOpenAI(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }));

const llm = new VernLLM({
  client: openai,
  model: 'gpt-4o',
});

const parsed = await llm.call({
  systemPrompt: 'Return JSON: { "skills": string[] }', // JSON mode enabled by default
  userContent: 'Extract skills from: ...',
});

systemPrompt is optional, omit it and no system message is sent at all:

const answer = await llm.call({
  userContent: 'What is the capital of France?',
  jsonMode: false,
});

Per-call overrides

const llm = new VernLLM({ client: openai, model: 'gpt-4o-mini' });

await llm.call({
  systemPrompt: '...',
  userContent: '...',
  model: 'o3',
  reasoningEffort: 'high', // passed through as `reasoning_effort` for supported models
});

See Per-call Overrides for the full guide, including which fields fall back to instance defaults, cheap-default/expensive-escalation patterns, and how overrides interact with usage tracking and caching.

Streaming

Pass stream: true to get incremental chunks back alongside the same validated result:

const { chunks, finalResult } = await llm.call({
  userContent: 'Write a short poem about the ocean.',
  jsonMode: false,
  stream: true,
});

for await (const chunk of chunks) {
  if (chunk.type === 'text-delta') process.stdout.write(chunk.delta);
}

const result = await finalResult;

See Streaming for the full contract, including tool calling, caching, and retry behavior while streaming.

Next steps

A single call above already gets retries, timeouts, and JSON parsing for free. Beyond that, VernLLM ships a few more resilience primitives worth knowing about early:

• Provider Fallback: declare backup targets tried in order when the primary fails
• Rate Limiting: queue locally to stay under a provider's requests/tokens-per-minute ceiling
• Circuit Breaker: stop hammering a provider that's already down
• Observability: one onEvent stream for retries, fallovers, circuit transitions, and rate-limit waits

See Features Overview for everything opt-in beyond the defaults above.