LLM

Unified interface for multiple LLM providers with automatic routing

Overview

The LLM abstraction layer provides seamless integration with multiple AI providers, allowing you to switch between OpenAI, Claude, Gemini, and Ollama without changing your code. It handles provider-specific implementations, message formatting, and streaming while providing a consistent API across all providers.

Supported Providers

Astreus supports four major LLM providers with automatic model routing:

Examples select models explicitly; existing supported defaults are unchanged. Retired native models need an explicit current replacement.

OpenAI

Current direct API model IDs:

• gpt-6-astra
• gpt-5.6-sol
• gpt-5.6-terra
• gpt-5.6-luna
• API Key: Set OPENAI_API_KEY environment variable

For direct OpenAI requests, GPT-6 Astra tool calling requires the Responses API. The adapter handles that protocol; configured OpenAI-compatible gateways use Chat Completions. Do not set temperature or top_p for GPT-6 Astra. See the OpenAI model guide.

Anthropic Claude

Current direct API model IDs:

• claude-fable-5-1
• claude-opus-5
• claude-sonnet-5
• claude-haiku-4-5-20251001
• API Key: Set ANTHROPIC_API_KEY environment variable

Google Gemini

All 12 supported models:

• Latest: gemini-2.5-pro, gemini-2.5-pro-deep-think, gemini-2.5-flash, gemini-2.5-flash-lite
• Stable: gemini-2.0-flash, gemini-2.0-flash-thinking, gemini-2.0-flash-lite, gemini-2.0-pro-experimental, gemini-1.5-pro, gemini-1.5-flash, gemini-1.5-flash-8b, gemini-pro
• API Key: Set GEMINI_API_KEY environment variable

Ollama (Local)

All 31 supported models:

• Latest: deepseek-r1, deepseek-v3, deepseek-v2.5, deepseek-coder, deepseek-coder-v2, qwen3, qwen2.5-coder, llama3.3, gemma3, phi4
• Popular: mistral-small, codellama, llama3.2, llama3.1, qwen2.5, gemma2, phi3, mistral, codegemma, wizardlm2
• Additional: dolphin-mistral, openhermes, deepcoder, stable-code, wizardcoder, magicoder, solar, yi, zephyr, orca-mini, vicuna
• Configuration: Set OLLAMA_BASE_URL (default: http://localhost:11434)

Configuration

# OpenAI
export OPENAI_API_KEY="${OPENAI_API_KEY:?Set OPENAI_API_KEY in your environment}"
export OPENAI_BASE_URL="https://api.openai.com/v1"  # Optional

# Anthropic Claude
export ANTHROPIC_API_KEY="${ANTHROPIC_API_KEY:?Set ANTHROPIC_API_KEY in your environment}"
export ANTHROPIC_BASE_URL="https://api.anthropic.com"  # Optional

# Google Gemini
export GEMINI_API_KEY="${GEMINI_API_KEY:?Set GEMINI_API_KEY in your environment}"

# Ollama (Local)
export OLLAMA_BASE_URL="http://localhost:11434"  # Optional

import { Agent } from '@astreus-ai/astreus';

const agent = await Agent.create({
  name: 'MyAgent',
  model: 'gpt-6-astra',  // Explicit selection, not a default change
  maxTokens: 2000
});

Hosted free models through OpenRouter

OpenRouter uses the existing openai adapter, not a fifth provider. Set both the gateway URL and its matching key before creating an agent or LLM instance. OPENROUTER_API_KEY below is your environment variable, mapped to the adapter’s OPENAI_API_KEY.

export OPENAI_BASE_URL="https://openrouter.ai/api/v1"
export OPENAI_API_KEY="${OPENROUTER_API_KEY:?Set OPENROUTER_API_KEY in your environment}"

On 2026-09-13, the OpenRouter model catalog listed zero input and output token prices and tool support for these IDs:

• openrouter/free
• nvidia/nemotron-3.5-lightning:free
• google/gemma-4-31b-it:free
• google/gemma-4-26b-a4b-it:free

import { Agent } from '@astreus-ai/astreus';

const agent = await Agent.create({
  name: 'HostedFreeAgent',
  model: 'openrouter/free',
  maxTokens: 1000
});

const response = await agent.ask('Explain how a hash table works.');
console.log(response);

Hosted free access has rate and availability limits, not unlimited service. If a free model is unavailable, handle the error or choose another free model explicitly; Astreus does not automatically fall back to paid models. A free framework license does not make hosted APIs free, and local Ollama still needs hardware and electricity.

These namespaced IDs require the OpenRouter endpoint. To return to direct OpenAI, restore OPENAI_BASE_URL=https://api.openai.com/v1 and an OpenAI-issued key together. Changing the base URL affects every request using that adapter instance.

Discover models without credentials

getModelsByProvider reads the framework’s local catalog without API credentials or authenticated clients. It does not prove that your account has access or that a hosted model is currently available.

import { getModelsByProvider } from '@astreus-ai/astreus';

const modelIds = getModelsByProvider('openai');
console.log(modelIds);

Usage Examples

Basic LLM Usage

import { getLLM } from '@astreus-ai/astreus';

const llm = getLLM();

// Generate response
const response = await llm.generateResponse({
  model: 'claude-sonnet-5',
  messages: [{ role: 'user', content: 'Explain quantum computing' }],
  maxTokens: 1000
});

console.log(response.content);

Streaming Responses

// Stream response in real-time
for await (const chunk of llm.generateStreamResponse({
  model: 'gpt-6-astra',
  messages: [{ role: 'user', content: 'Write a story about AI' }],
  stream: true
})) {
  if (!chunk.done) {
    process.stdout.write(chunk.content);
  }
}

Function Calling

const response = await llm.generateResponse({
  model: 'gpt-6-astra',
  messages: [{ role: 'user', content: 'What\'s the weather in Tokyo?' }],
  tools: [{
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get current weather information',
      parameters: {
        type: 'object',
        properties: {
          location: { 
            type: 'string',
            description: 'City name'
          }
        },
        required: ['location']
      }
    }
  }]
});

// Handle tool calls
if (response.toolCalls) {
  response.toolCalls.forEach(call => {
    console.log(`Tool: ${call.function.name}`);
    console.log(`Args: ${call.function.arguments}`);
  });
}

LLM Options

Configure LLM behavior with these options:

interface LLMRequestOptions {
  model: string;              // Required: Model identifier
  messages: LLMMessage[];     // Required: Conversation history
  temperature?: number;       // Model-dependent; omit for GPT-6 Astra
  maxTokens?: number;         // Max output tokens (default: 4096)
  stream?: boolean;           // Enable streaming responses
  systemPrompt?: string;      // System instructions
  tools?: Tool[];             // Function calling tools
}

Parameter Details

• temperature: Optional and model-dependent. Omit it for GPT-6 Astra; not every model accepts sampling controls.
• maxTokens: Maximum tokens in the response (varies by model)
• stream: Enable real-time streaming for long responses
• systemPrompt: Sets behavior and context for the model
• tools: Enable function calling capabilities

Provider Features

Vision Models

Each provider supports specific models for image analysis:

OpenAI Vision

• gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-4-vision-preview
• gpt-4o-2024-08-06, gpt-4o-2024-05-13

Claude Vision

• claude-3-5-sonnet-20241022, claude-3-5-sonnet-20240620
• claude-3-opus-20240229, claude-3-sonnet-20240229, claude-3-haiku-20240307

Gemini Vision

• gemini-1.5-pro, gemini-1.5-flash
• gemini-1.0-pro-vision-latest, gemini-pro-vision

Ollama Vision

• llava, llava:7b, llava:13b, llava:34b
• llava-llama3, llava-phi3, moondream

Embedding Models

For knowledge base and semantic search:

OpenAI Embeddings

• text-embedding-3-large (3072 dimensions)
• text-embedding-3-small (1536 dimensions)
• text-embedding-ada-002 (1536 dimensions)

Gemini Embeddings

• text-embedding-004
• embedding-001

Ollama Embeddings

• nomic-embed-text, mxbai-embed-large
• all-minilm, snowflake-arctic-embed

Note: Claude/Anthropic does not support embeddings. Use OpenAI or Gemini for embedding generation.

Additional Environment Variables

# Dedicated embedding API keys (optional)
OPENAI_EMBEDDING_API_KEY="your-embedding-key"
OPENAI_EMBEDDING_BASE_URL="https://api.openai.com/v1"
GEMINI_EMBEDDING_API_KEY="your-gemini-embedding-key"

# Dedicated vision API keys (optional)
OPENAI_VISION_API_KEY="your-vision-key"
OPENAI_VISION_BASE_URL="https://api.openai.com/v1"
ANTHROPIC_VISION_API_KEY="your-anthropic-vision-key"
ANTHROPIC_VISION_BASE_URL="https://api.anthropic.com"
GEMINI_VISION_API_KEY="your-gemini-vision-key"

Model Selection Guide

For Code Generation

• Best: gpt-4o, claude-3-5-sonnet-20241022, deepseek-coder
• Fast: gpt-4o-mini, claude-3-5-haiku-20241022

For Reasoning Tasks

• Best: claude-opus-4-20250514, gpt-6-astra, o3
• Balanced: claude-sonnet-4-20250514, gpt-4o

For Creative Writing

• Best: gpt-6-astra, claude-3-opus-20240229
• Fast: gemini-2.5-pro, gpt-4o-mini

For Privacy/Local Use

• Best: deepseek-r1, llama3.3, qwen3
• Code: deepseek-coder, codellama