Adding a New LLM Provider

This guide walks through adding a new LLM provider to CortexPrism.

Steps

1. Create the Provider File

Create packages/ai/src/llm/your-provider.ts. Most providers extend OpenAICompatibleProvider:

import { OpenAICompatibleProvider } from "./openai-compatible.ts";

export class YourProvider extends OpenAICompatibleProvider {
  readonly name = "your-provider";
  readonly defaultModel = "your-default-model";

  constructor(apiKey?: string, baseUrl?: string) {
    super(
      apiKey ?? "",
      baseUrl ?? "https://api.your-provider.com/v1",
      "your-provider",
      "your-default-model",
    );
  }
}

For non-OpenAI-compatible providers, implement the LLMProvider interface directly:

import type { LLMProvider, CompletionOptions, CompletionResult, CompletionChunk } from "./types.ts";

export class YourProvider implements LLMProvider {
  readonly name = "your-provider";
  readonly defaultModel = "your-default-model";

  constructor(private apiKey: string) {}

  async complete(options: CompletionOptions): Promise<CompletionResult> {
    // Implement complete API call
  }

  async *stream(options: CompletionOptions): AsyncIterable<CompletionChunk> {
    // Implement streaming API call
  }
}

2. Register in Router

Add to packages/ai/src/llm/router.ts in the createProvider() function.

3. Add Config Schema

Add the provider to the ProviderConfig type union in packages/core/contracts/config.ts.

4. Add to Setup Wizard

Add the provider option to the interactive setup in packages/cli/src/cli/setup.ts.

5. Add Model Listing (if applicable)

Create a yourProviderModels() function and register it in packages/server/src/server/models.ts.

6. Add Provider-Specific Settings (optional)

If the provider has unique parameters, add them to CompletionOptions in packages/ai/src/llm/types.ts and wire them through in the provider adapter.

7. Document

Add to the README provider list
Update the Provider Guide wiki page
Update Configuration — add provider-specific config example
Add a CHANGELOG entry

LLMProvider Interface

interface LLMProvider {
  readonly name: string;
  readonly defaultModel: string;
  complete(options: CompletionOptions): Promise<CompletionResult>;
  stream(options: CompletionOptions): AsyncIterable<CompletionChunk>;
}

CompletionOptions

interface CompletionOptions {
  messages: Message[];
  systemPrompt?: string;
  temperature?: number;
  maxTokens?: number;
  topP?: number;
  model?: string;
  stream?: boolean;
  signal?: AbortSignal;
  // Provider-specific:
  reasoningEffort?: "low" | "medium" | "high";
  repetitionPenalty?: number;
  numCtx?: number;
  keepAlive?: string;
  // ... and more
}

Currently Supported Providers

CortexPrism ships with 30 providers: Anthropic, OpenAI, Google Gemini, Groq, Ollama, DeepSeek, Cerebras, Perplexity, DeepInfra, Hyperbolic, MiniMax, Zhipu/GLM, Replicate, Cloudflare Workers AI, xAI/Grok, Mistral, Together AI, Fireworks AI, Novita, OpenRouter, Bedrock, LM Studio, LiteLLM, Alibaba/Qwen, Pinecone (embeddings), Voyage AI (embeddings), Cohere (embeddings), Nomic (embeddings), Jina AI (embeddings), and OpenAI (embeddings).