LLM

Unified interface for multiple LLM providers with automatic routing

Overview

The LLM abstraction layer provides seamless integration with multiple AI providers, allowing you to switch between OpenAI, Claude, Gemini, and Ollama without changing your code. It handles provider-specific implementations, message formatting, and streaming while providing a consistent API across all providers.

Supported Providers

Astreus supports four major LLM providers with automatic model routing:

OpenAI

All 14 supported models:

• Latest: gpt-4.5, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, o4-mini, o4-mini-high, o3
• Stable: gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-4, gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-3.5-turbo-instruct
• API Key: Set OPENAI_API_KEY environment variable

Anthropic Claude

All 9 supported models:

• Latest: claude-sonnet-4-20250514, claude-opus-4-20250514, claude-3.7-sonnet-20250224
• Stable: claude-3-5-sonnet-20241022, claude-3-5-sonnet-20240620, claude-3-5-haiku-20241022, claude-3-opus-20240229, claude-3-sonnet-20240229, claude-3-haiku-20240307
• API Key: Set ANTHROPIC_API_KEY environment variable

Google Gemini

All 12 supported models:

• Latest: gemini-2.5-pro, gemini-2.5-pro-deep-think, gemini-2.5-flash, gemini-2.5-flash-lite
• Stable: gemini-2.0-flash, gemini-2.0-flash-thinking, gemini-2.0-flash-lite, gemini-2.0-pro-experimental, gemini-1.5-pro, gemini-1.5-flash, gemini-1.5-flash-8b, gemini-pro
• API Key: Set GEMINI_API_KEY environment variable

Ollama (Local)

All 31 supported models:

• Latest: deepseek-r1, deepseek-v3, deepseek-v2.5, deepseek-coder, deepseek-coder-v2, qwen3, qwen2.5-coder, llama3.3, gemma3, phi4
• Popular: mistral-small, codellama, llama3.2, llama3.1, qwen2.5, gemma2, phi3, mistral, codegemma, wizardlm2
• Additional: dolphin-mistral, openhermes, deepcoder, stable-code, wizardcoder, magicoder, solar, yi, zephyr, orca-mini, vicuna
• Configuration: Set OLLAMA_BASE_URL (default: http://localhost:11434)

Configuration

# OpenAI
export OPENAI_API_KEY="your-openai-key"
export OPENAI_BASE_URL="https://api.openai.com/v1"  # Optional

# Anthropic Claude
export ANTHROPIC_API_KEY="your-anthropic-key"
export ANTHROPIC_BASE_URL="https://api.anthropic.com"  # Optional

# Google Gemini
export GEMINI_API_KEY="your-gemini-key"

# Ollama (Local)
export OLLAMA_BASE_URL="http://localhost:11434"  # Optional

import { Agent } from '@astreus-ai/astreus';

const agent = await Agent.create({
  name: 'MyAgent',
  model: 'gpt-4.5',  // Model automatically routes to correct provider
  temperature: 0.7,
  maxTokens: 2000
});

Usage Examples

Basic LLM Usage

import { getLLM } from '@astreus-ai/astreus';

const llm = getLLM();

// Generate response
const response = await llm.generateResponse({
  model: 'claude-sonnet-4-20250514',
  messages: [{ role: 'user', content: 'Explain quantum computing' }],
  temperature: 0.7,
  maxTokens: 1000
});

console.log(response.content);

Streaming Responses

// Stream response in real-time
for await (const chunk of llm.generateStreamResponse({
  model: 'gpt-4.5',
  messages: [{ role: 'user', content: 'Write a story about AI' }],
  stream: true
})) {
  if (!chunk.done) {
    process.stdout.write(chunk.content);
  }
}

Function Calling

const response = await llm.generateResponse({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'What\'s the weather in Tokyo?' }],
  tools: [{
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get current weather information',
      parameters: {
        type: 'object',
        properties: {
          location: { 
            type: 'string',
            description: 'City name'
          }
        },
        required: ['location']
      }
    }
  }]
});

// Handle tool calls
if (response.toolCalls) {
  response.toolCalls.forEach(call => {
    console.log(`Tool: ${call.function.name}`);
    console.log(`Args: ${call.function.arguments}`);
  });
}

LLM Options

Configure LLM behavior with these options:

interface LLMRequestOptions {
  model: string;              // Required: Model identifier
  messages: LLMMessage[];     // Required: Conversation history
  temperature?: number;       // Creativity level (0.0-1.0, default: 0.7)
  maxTokens?: number;         // Max output tokens (default: 4096)
  stream?: boolean;           // Enable streaming responses
  systemPrompt?: string;      // System instructions
  tools?: Tool[];             // Function calling tools
}

Parameter Details

• temperature: Controls randomness (0.0 = deterministic, 1.0 = very creative)
• maxTokens: Maximum tokens in the response (varies by model)
• stream: Enable real-time streaming for long responses
• systemPrompt: Sets behavior and context for the model
• tools: Enable function calling capabilities

Provider Features

Vision Models

Each provider supports specific models for image analysis:

OpenAI Vision

• gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-4-vision-preview
• gpt-4o-2024-08-06, gpt-4o-2024-05-13

Claude Vision

• claude-3-5-sonnet-20241022, claude-3-5-sonnet-20240620
• claude-3-opus-20240229, claude-3-sonnet-20240229, claude-3-haiku-20240307

Gemini Vision

• gemini-1.5-pro, gemini-1.5-flash
• gemini-1.0-pro-vision-latest, gemini-pro-vision

Ollama Vision

• llava, llava:7b, llava:13b, llava:34b
• llava-llama3, llava-phi3, moondream

Embedding Models

For knowledge base and semantic search:

OpenAI Embeddings

• text-embedding-3-large (3072 dimensions)
• text-embedding-3-small (1536 dimensions)
• text-embedding-ada-002 (1536 dimensions)

Gemini Embeddings

• text-embedding-004
• embedding-001

Ollama Embeddings

• nomic-embed-text, mxbai-embed-large
• all-minilm, snowflake-arctic-embed

Note: Claude/Anthropic does not support embeddings. Use OpenAI or Gemini for embedding generation.

Additional Environment Variables

# Dedicated embedding API keys (optional)
OPENAI_EMBEDDING_API_KEY="your-embedding-key"
OPENAI_EMBEDDING_BASE_URL="https://api.openai.com/v1"
GEMINI_EMBEDDING_API_KEY="your-gemini-embedding-key"

# Dedicated vision API keys (optional)
OPENAI_VISION_API_KEY="your-vision-key"
OPENAI_VISION_BASE_URL="https://api.openai.com/v1"
ANTHROPIC_VISION_API_KEY="your-anthropic-vision-key"
ANTHROPIC_VISION_BASE_URL="https://api.anthropic.com"
GEMINI_VISION_API_KEY="your-gemini-vision-key"

Model Selection Guide

For Code Generation

• Best: gpt-4o, claude-3-5-sonnet-20241022, deepseek-coder
• Fast: gpt-4o-mini, claude-3-5-haiku-20241022

For Reasoning Tasks

• Best: claude-opus-4-20250514, gpt-4.5, o3
• Balanced: claude-sonnet-4-20250514, gpt-4o

For Creative Writing

• Best: gpt-4.5, claude-3-opus-20240229
• Fast: gemini-2.5-pro, gpt-4o-mini

For Privacy/Local Use

• Best: deepseek-r1, llama3.3, qwen3
• Code: deepseek-coder, codellama