OpenRouter API for AI Agents

• Making chat completions via OpenRouter API
• Selecting appropriate models and variants
• Implementing streaming responses
• Using tool/function calling
• Enforcing structured outputs
• Integrating web search
• Handling multimodal inputs (images, audio, video, PDFs)
• Managing model routing and fallbacks
• Handling errors and retries
• Optimizing cost and performance

API Basics

Making a Request

POST https://openrouter.ai/api/v1/chat/completions

{
  'Authorization': `Bearer ${apiKey}`,
  'Content-Type': 'application/json',
  // Optional: for app attribution
  'HTTP-Referer': 'https://your-app.com',
  'X-Title': 'Your App Name'
}

const response = await fetch('https://openrouter.ai/api/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${apiKey}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'anthropic/claude-3.5-sonnet',
    messages: [
      { role: 'user', content: 'Your prompt here' }
    ]
  })
});

Response Structure

{
  "id": "gen-abc123",
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "Response text here"
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 20,
    "total_tokens": 30
  },
  "model": "anthropic/claude-3.5-sonnet"
}

• choices[0].message.content

  - The assistant's response

• choices[0].finish_reason

  - Why generation stopped (stop, length, tool_calls, etc.)

• usage

  - Token counts and cost information

• model

  - Actual model used (may differ from requested)

When to Use Streaming vs Non-Streaming

stream: true

• Real-time responses needed (chat interfaces, interactive tools)
• Latency matters (user-facing applications)
• Large responses expected (long-form content)
• Want to show progressive output

• Processing in background (batch jobs, async tasks)
• Need complete response before processing
• Building to an API/endpoint
• Response is short (few tokens)

const response = await fetch('https://openrouter.ai/api/v1/chat/completions', {
  method: 'POST',
  headers: { /* ... */ },
  body: JSON.stringify({
    model: 'anthropic/claude-3.5-sonnet',
    messages: [{ role: 'user', content: '...' }],
    stream: true
  })
});

for await (const chunk of response.body) {
  const text = new TextDecoder().decode(chunk);
  const lines = text.split('\n').filter(line => line.startsWith('data: '));

  for (const line of lines) {
    const data = line.slice(6); // Remove 'data: '
    if (data === '[DONE]') break;

    const parsed = JSON.parse(data);
    const content = parsed.choices?.[0]?.delta?.content;
    if (content) {
      // Accumulate or display content
    }
  }
}

Model Selection

Model Identifier Format

provider/model-name[:variant]

• anthropic/claude-3.5-sonnet

  - Specific model

• openai/gpt-4o:online

  - With web search enabled

• google/gemini-2.0-flash:free

  - Free tier variant

Model Variants and When to Use Them

:free

:online

:extended

:thinking

:nitro

:exacto

Default Model Choices by Task

anthropic/claude-3.5-sonnet

openai/gpt-4o

• Balanced quality, speed, cost
• Good for most tasks

anthropic/claude-3.5-sonnet

openai/gpt-4o

• Strong code generation and understanding
• Good reasoning

anthropic/claude-opus-4:thinking

openai/o3

• Deep reasoning capabilities
• Higher cost, slower

openai/gpt-4o-mini:nitro

google/gemini-2.0-flash

• Minimal latency
• Good for real-time applications

google/gemini-2.0-flash:free

meta-llama/llama-3.1-70b:free

• No cost with limits
• Good for high-volume, lower-complexity tasks

anthropic/claude-3.5-sonnet:online

google/gemini-2.5-pro:online

• Web search built-in
• Real-time data

anthropic/claude-3.5-sonnet:extended

google/gemini-2.5-pro:extended

• 200K+ context windows
• Document analysis, codebase understanding

Provider Routing Preferences

{
  provider: {
    order: ['anthropic', 'openai', 'google'],
    allow_fallbacks: true,
    sort: 'price' // 'price', 'latency', or 'throughput'
  }
}

• Have preferred provider arrangements
• Need to optimize for specific metric (cost, speed)
• Want to exclude certain providers
• Have BYOK (Bring Your Own Key) for specific providers

Model Fallbacks

{
  models: [
    'anthropic/claude-3.5-sonnet',
    'openai/gpt-4o',
    'google/gemini-2.0-flash'
  ]
}

• High reliability required
• Multiple providers acceptable
• Want graceful degradation
• Avoid single point of failure

• Tries first model
• Falls to next on error (5xx, 429, timeout)
• Uses whichever succeeds
• Returns which model was used in

  model

  field

Parameters You Need

Core Parameters

• Which model to use
• Default: user's default model
• Always specify for consistency

• Conversation history
• Structure:

  { role: 'user'|'assistant'|'system', content: string | ContentPart[] }

• For multimodal: content can be array of text and image_url parts

• Enable Server-Sent Events streaming
• Use for real-time responses

• Controls randomness
• 0.0-0.3: Deterministic, factual responses (code, precise answers)
• 0.4-0.7: Balanced (general use)
• 0.8-1.2: Creative (brainstorming, creative writing)
• 1.3-2.0: Highly creative, unpredictable (experimental)

• Maximum tokens to generate
• Always set to control cost and prevent runaway responses
• Typical: 100-500 for short, 1000-2000 for long responses
• Model limit: context_length - prompt_length

• Nucleus sampling - limits to top probability mass
• Use instead of temperature when you want predictable diversity
• 0.9-0.95: Common settings for quality

• Limit to K most likely tokens
• 1: Always most likely (deterministic)
• 40-50: Balanced
• Not available for OpenAI models

Sampling Strategy Guidelines

temperature: 0.1-0.3, top_p: 0.95

temperature: 0.0-0.2

temperature: 0.8-1.2

temperature: 1.0-1.5

temperature: 0.6-0.8

Tool Calling Parameters

• Available functions for model to call
• Structure:

{
  type: 'function',
  function: {
    name: 'function_name',
    description: 'What it does',
    parameters: { /* JSON Schema */ }
  }
}

• Control when tools are called

• 'auto'

  : Model decides (default)

• 'none'

  : Never call tools

• 'required'

  : Must call a tool

• { type: 'function', function: { name: 'specific_tool' } }

  : Force specific tool

• Allow multiple tools simultaneously
• Set

  false

  for sequential execution

• Need to query external APIs (weather, search, database)
• Need to perform calculations or data processing
• Building agentic systems
• Need structured data extraction

Structured Output Parameters

• Enforce specific output format

{ type: 'json_object' }

• Model returns valid JSON
• Must also instruct model in system message

{
  type: 'json_schema',
  json_schema: {
    name: 'schema_name',
    strict: true,
    schema: { /* JSON Schema */ }
  }
}

• Model returns JSON matching exact schema
• Use when structure is critical (APIs, data processing)

• Need predictable response format
• Integrating with systems (APIs, databases)
• Data extraction
• Form filling

Web Search Parameters

{ model: 'anthropic/claude-3.5-sonnet:online' }

{
  plugins: [{
    id: 'web',
    enabled: true,
    max_results: 5
  }]
}

• Need current information (news, prices, events)
• User asks about recent developments
• Need factual verification
• Topic requires real-time data

Other Important Parameters

• Stable identifier for end-user
• Set when you have user IDs
• Helps with abuse detection and caching

• Group related requests
• Set for conversation tracking
• Improves caching and observability

• Custom metadata (max 16 key-value pairs)
• Use for analytics and tracking
• Keys: max 64 chars, Values: max 512 chars

• Stop sequences to halt generation
• Common:

  ['\n\n', '###', 'END']

Handling Responses

Non-Streaming Responses

const response = await fetch(/* ... */);
const data = await response.json();

const content = data.choices[0].message.content;
const finishReason = data.choices[0].finish_reason;
const usage = data.usage;

const toolCalls = data.choices[0].message.tool_calls;
if (toolCalls) {
  // Model wants to call tools
  for (const toolCall of toolCalls) {
    const { name, arguments: args } = toolCall.function;
    const parsedArgs = JSON.parse(args);
    // Execute tool...
  }
}

Streaming Responses

let fullContent = '';
const response = await fetch(/* ... */);

const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  const chunk = decoder.decode(value);
  const lines = chunk.split('\n').filter(line => line.startsWith('data: '));

  for (const line of lines) {
    const data = line.slice(6);
    if (data === '[DONE]') break;

    const parsed = JSON.parse(data);
    const content = parsed.choices?.[0]?.delta?.content;
    if (content) {
      fullContent += content;
      // Process incrementally...
    }

    // Handle usage in final chunk
    if (parsed.usage) {
      console.log('Usage:', parsed.usage);
    }
  }
}

// Tool calls stream across multiple chunks
let currentToolCall = null;
let toolArgs = '';

for (const parsed of chunks) {
  const toolCallChunk = parsed.choices?.[0]?.delta?.tool_calls?.[0];

  if (toolCallChunk?.function?.name) {
    currentToolCall = { id: toolCallChunk.id, ...toolCallChunk.function };
  }

  if (toolCallChunk?.function?.arguments) {
    toolArgs += toolCallChunk.function.arguments;
  }

  if (parsed.choices?.[0]?.finish_reason === 'tool_calls' && currentToolCall) {
    // Complete tool call
    currentToolCall.arguments = toolArgs;
    // Execute tool...
  }
}

Usage and Cost Tracking

const { usage } = data;
console.log(`Prompt: ${usage.prompt_tokens}`);
console.log(`Completion: ${usage.completion_tokens}`);
console.log(`Total: ${usage.total_tokens}`);

// Cost (if available)
if (usage.cost) {
  console.log(`Cost: $${usage.cost.toFixed(6)}`);
}

// Detailed breakdown
console.log(usage.prompt_tokens_details);
console.log(usage.completion_tokens_details);

Error Handling

Common HTTP Status Codes

• Invalid request format
• Missing required fields
• Parameter out of range
• Fix: Validate request structure and parameters

• Missing or invalid API key
• Fix: Check API key format and permissions

• Insufficient permissions
• Model not allowed
• Fix: Check guardrails, model access, API key permissions

• Insufficient credits
• Fix: Add credits to account

• Request took too long
• Fix: Reduce prompt length, use streaming, try simpler model

• Too many requests
• Fix: Implement exponential backoff, reduce request rate

• Provider error
• Fix: Use model fallbacks, retry with different model

• Service overloaded
• Fix: Retry with backoff, use fallbacks

Retry Strategy

async function requestWithRetry(url, body, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      const response = await fetch(url, body);

      if (response.ok) {
        return await response.json();
      }

      // Retry on rate limit or server errors
      if (response.status === 429 || response.status >= 500) {
        const delay = Math.min(1000 * Math.pow(2, attempt), 10000);
        await new Promise(resolve => setTimeout(resolve, delay));
        continue;
      }

      // Don't retry other errors
      return response;
    } catch (error) {
      if (attempt === maxRetries - 1) throw error;
      const delay = Math.min(1000 * Math.pow(2, attempt), 10000);
      await new Promise(resolve => setTimeout(resolve, delay));
    }
  }
}

Graceful Degradation

{
  models: [
    'anthropic/claude-3.5-sonnet',  // Primary
    'openai/gpt-4o',                // Fallback 1
    'google/gemini-2.0-flash'        // Fallback 2
  ]
}

• Log errors but continue
• Fall back to simpler features
• Use cached responses when available
• Provide degraded experience rather than failing completely

Advanced Features

When to Use Tool Calling

• Querying external APIs (weather, stock prices, databases)
• Performing calculations or data processing
• Extracting structured data from unstructured text
• Building agentic systems with multiple steps
• When decisions require external information

1. Define tools with clear descriptions and parameters
2. Send request with

   tools

   array
3. Check if

   tool_calls

   present in response
4. Execute tools with parsed arguments
5. Send tool results back in a new request
6. Repeat until model provides final answer

references/ADVANCED_PATTERNS.md

When to Use Structured Outputs

• API responses (need specific schema)
• Data extraction (forms, documents)
• Configuration files (JSON, YAML)
• Database operations (structured queries)
• When downstream processing requires specific format

1. Define JSON Schema for desired output
2. Set

   response_format: { type: 'json_schema', json_schema: { ... } }

3. Instruct model to produce JSON (system or user message)
4. Validate response against schema
5. Handle parsing errors gracefully

{
  response_format: { /* ... */ },
  plugins: [{ id: 'response-healing' }]
}

When to Use Web Search

• User asks about recent events, news, or current data
• Need verification of facts
• Questions with time-sensitive information
• Topic requires up-to-date information
• User explicitly requests current information

{
  model: 'anthropic/claude-3.5-sonnet:online'
}

{
  model: 'openrouter.ai/auto',
  plugins: [{
    id: 'web',
    enabled: true,
    max_results: 5,
    engine: 'exa' // or 'native'
  }]
}

When to Use Multimodal Inputs

• OCR, image understanding, visual analysis
• Models:

  openai/gpt-4o

  ,

  anthropic/claude-3.5-sonnet

  ,

  google/gemini-2.5-pro

• Speech-to-text, audio analysis
• Models with audio support

• Video understanding, frame analysis
• Models with video support

• Document parsing, content extraction
• Requires

  file-parser

  plugin

references/ADVANCED_PATTERNS.md

Best Practices for AI

Default Model Selection

anthropic/claude-3.5-sonnet

openai/gpt-4o

• Good balance of quality, speed, cost
• Strong at most tasks
• Wide compatibility

• Need speed →

  openai/gpt-4o-mini:nitro

  or

  google/gemini-2.0-flash

• Complex reasoning →

  anthropic/claude-opus-4:thinking

• Need web search →

  :online

  variant
• Large context →

  :extended

  variant
• Cost-sensitive →

  :free

  variant

Default Parameters

{
  model: 'anthropic/claude-3.5-sonnet',
  messages: [...],
  temperature: 0.6,  // Balanced creativity
  max_tokens: 1000,   // Reasonable length
  top_p: 0.95        // Common for quality
}

• Code:

  temperature: 0.2

• Creative:

  temperature: 1.0

• Factual:

  temperature: 0.0-0.3

When to Prefer Streaming

• User-facing (chat, interactive tools)
• Response length unknown
• Want progressive feedback
• Latency matters

• Batch processing
• Need complete response before acting
• Building API endpoints
• Very short responses (< 50 tokens)

When to Enable Specific Features

Cost Optimization Patterns

• Testing and prototyping
• Low-complexity tasks
• High-volume, low-value operations

{
  provider: {
    order: ['openai', 'anthropic'],
    sort: 'price',  // Optimize for cost
    allow_fallbacks: true
  }
}

user

session_id

Performance Optimization

• Use

  :nitro

  variants for speed
• Use streaming for perceived speed
• Set

  user

  ID for caching benefits
• Choose faster models (mini, flash) when quality allows

• Use provider routing with

  sort: 'throughput'

• Parallelize independent requests
• Use streaming to reduce wait time

{
  provider: {
    sort: 'latency'  // or 'price' or 'throughput'
  }
}

Progressive Disclosure

Parameters Reference

references/PARAMETERS.md

• Complete parameter reference (50+ parameters)
• Types, ranges, defaults
• Parameter support by model
• Usage examples

Error Codes Reference

references/ERROR_CODES.md

• All HTTP status codes
• Error response structure
• Error metadata types
• Native finish reasons
• Retry strategies

Model Selection Guide

references/MODEL_SELECTION.md

• Model families and capabilities
• Model variants explained
• Selection criteria by use case
• Model capability matrix
• Provider routing preferences

Routing Strategies

references/ROUTING_STRATEGIES.md

• Model fallbacks configuration
• Provider selection patterns
• Auto router setup
• Routing by use case (cost, latency, quality)

Advanced Patterns

references/ADVANCED_PATTERNS.md

• Tool calling with agentic loops
• Structured outputs implementation
• Web search integration
• Multimodal handling
• Streaming patterns
• Framework integrations

Working Examples

references/EXAMPLES.md

• TypeScript patterns for common tasks
• Python examples
• cURL examples
• Advanced patterns
• Framework integration examples

Ready-to-Use Templates

templates/

• basic-request.ts

  - Minimal working request

• streaming-request.ts

  - SSE streaming with cancellation

• tool-calling.ts

  - Complete agentic loop with tools

• structured-output.ts

  - JSON Schema enforcement

• error-handling.ts

  - Robust retry logic

Quick Reference

Minimal Request

{
  model: 'anthropic/claude-3.5-sonnet',
  messages: [{ role: 'user', content: 'Your prompt' }]
}

With Streaming

{
  model: 'anthropic/claude-3.5-sonnet',
  messages: [{ role: 'user', content: '...' }],
  stream: true
}

With Tools

{
  model: 'anthropic/claude-3.5-sonnet',
  messages: [{ role: 'user', content: '...' }],
  tools: [{ type: 'function', function: { name, description, parameters } }],
  tool_choice: 'auto'
}

With Structured Output

{
  model: 'anthropic/claude-3.5-sonnet',
  messages: [{ role: 'system', content: 'Output JSON only...' }],
  response_format: { type: 'json_object' }
}

With Web Search

{
  model: 'anthropic/claude-3.5-sonnet:online',
  messages: [{ role: 'user', content: '...' }]
}

With Model Fallbacks

{
  models: ['anthropic/claude-3.5-sonnet', 'openai/gpt-4o'],
  messages: [{ role: 'user', content: '...' }]
}

baseURL: 'https://openrouter.ai/api/v1'