Custom model gateways

Custom model gateways allow you to implement private or specialized LLM provider integrations by extending the MastraModelGateway base class.

Overview

Gateways handle provider-specific logic for accessing language models:

Provider configuration and model discovery
Authentication and API key management
URL construction for API endpoints
Language model instance creation

Create custom gateways to support:

Private or enterprise LLM deployments
Custom authentication schemes
Specialized routing logic
Gateway versioning with unique IDs

Creating a Custom Gateway

Extend the MastraModelGateway class and implement the required methods:

typescript

import { MastraModelGateway, type ProviderConfig } from '@mastra/core/llm';
import { createOpenAICompatible } from '@ai-sdk/openai-compatible-v5';
import type { LanguageModelV2 } from '@ai-sdk/provider-v5';

class MyPrivateGateway extends MastraModelGateway {
  // Required: Unique identifier for the gateway
  // This ID is used as the prefix for all providers from this gateway
  readonly id = 'private';

  // Required: Human-readable name
  readonly name = 'My Private Gateway';

  /**
   * Fetch provider configurations from your gateway
   * Returns a record of provider configurations
   */
  async fetchProviders(): Promise<Record<string, ProviderConfig>> {
    return {
      'my-provider': {
        name: 'My Provider',
        models: ['model-1', 'model-2', 'model-3'],
        apiKeyEnvVar: 'MY_API_KEY',
        gateway: this.id,
        url: 'https://api.myprovider.com/v1',
      },
    };
  }

  /**
   * Build the API URL for a model
   * @param modelId - Full model ID (e.g., "private/my-provider/model-1")
   * @param envVars - Environment variables (optional)
   */
  buildUrl(modelId: string, envVars?: Record<string, string>): string {
    return 'https://api.myprovider.com/v1';
  }

  /**
   * Get the API key for authentication
   * @param modelId - Full model ID
   */
  async getApiKey(modelId: string): Promise<string> {
    const apiKey = process.env.MY_API_KEY;
    if (!apiKey) {
      throw new Error(`Missing MY_API_KEY environment variable`);
    }
    return apiKey;
  }

  /**
   * Create a language model instance
   * @param args - Model ID, provider ID, and API key
   */
  async resolveLanguageModel({
    modelId,
    providerId,
    apiKey,
  }: {
    modelId: string;
    providerId: string;
    apiKey: string;
  }): Promise<LanguageModelV2> {
    const baseURL = this.buildUrl(`${providerId}/${modelId}`);

    return createOpenAICompatible({
      name: providerId,
      apiKey,
      baseURL,
      supportsStructuredOutputs: true,
    }).chatModel(modelId);
  }
}

Registering Custom Gateways

During Initialization

Pass gateways as a record when creating your Mastra instance:

typescript

import { Mastra } from '@mastra/core';

const mastra = new Mastra({
  gateways: {
    myGateway: new MyPrivateGateway(),
    anotherGateway: new AnotherGateway(),
  },
});

After Initialization

Add gateways dynamically using addGateway:

typescript

const mastra = new Mastra();

// Add with explicit key
mastra.addGateway(new MyPrivateGateway(), 'myGateway');

// Add using gateway's ID
mastra.addGateway(new MyPrivateGateway());
// Stored with key 'my-private-gateway' (the gateway's id)

Using Custom Gateways

Reference models from your custom gateway using the gateway ID as prefix:

typescript

import { Agent } from '@mastra/core/agent';

const agent = new Agent({
  id: 'my-agent',
  name: 'My Agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/model-1', // Uses MyPrivateGateway
});

mastra.addAgent(agent, 'myAgent');

When you create an agent or use a model, Mastra's model router automatically selects the appropriate gateway based on the model ID. The gateway ID serves as the prefix. If no custom gateways match, it falls back to the built-in gateways.

TypeScript Autocomplete

Automatic Type Generation in Development

When running in development mode (MASTRA_DEV=true), Mastra automatically generates TypeScript types for your custom gateways!

Set the environment variable:
bash
```
export MASTRA_DEV=true
```

Register your gateways:

typescript

const mastra = new Mastra({
  gateways: {
    myGateway: new MyPrivateGateway(),
  },
});

Types are generated automatically:
- When you add a gateway, Mastra syncs with the GatewayRegistry
- The registry fetches providers from your custom gateway
- TypeScript types are regenerated in ~/.cache/mastra/
- Your IDE picks up the new types within seconds

Autocomplete now works:

typescript

const agent = new Agent({
  model: 'my-gateway-id/my-provider/model-1', // Full autocomplete!
});

How It Works

The GatewayRegistry runs an hourly sync that:

Calls fetchProviders() on all registered gateways
Generates TypeScript type definitions
Writes them to both global cache and your project's dist/ directory
Your TypeScript server automatically picks up the changes

:::tip The first time you add a gateway, it may take a few seconds for types to generate. Subsequent updates happen in the background every hour. :::

Manual Type Generation Alternatives

If you're not running in development mode or need immediate type updates:

Option 1: Use type assertion (simplest)

typescript

const agent = new Agent({
  id: 'my-agent',
  name: 'my-agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/model-1' as any, // Bypass type checking
});

Option 2: Create a custom type union (type-safe)

typescript

import type { ModelRouterModelId } from '@mastra/core/llm';

// Define your custom model IDs
type CustomModelId =
  | 'private/my-provider/model-1'
  | 'private/my-provider/model-2'
  | 'private/my-provider/model-3';

// Combine with built-in models
type AllModelIds = ModelRouterModelId | CustomModelId;

const agent = new Agent({
  id: 'my-agent',
  name: 'my-agent',
  instructions: 'You are a helpful assistant',
  model: 'private/my-provider/model-1' satisfies AllModelIds,
});

Option 3: Extend ModelRouterModelId globally (advanced)

typescript

// In a types.d.ts file in your project
declare module '@mastra/core/llm' {
  interface ProviderModelsMap {
    'my-provider': readonly ['model-1', 'model-2', 'model-3'];
  }
}

This extends the built-in type to include your custom models, giving you full autocomplete support.

Gateway Management

getGateway(key)

Retrieve a gateway by its registration key:

typescript

const gateway = mastra.getGateway('myGateway');
console.log(gateway.name); // 'My Private Gateway'

getGatewayById(id)

Retrieve a gateway by its unique ID:

typescript

const gateway = mastra.getGatewayById('my-private-gateway');
console.log(gateway.name); // 'My Private Gateway'

This is useful when:

Gateways have explicit IDs different from their registration keys
You need to find a gateway by its ID across different instances
Supporting gateway versioning (e.g., 'gateway-v1', 'gateway-v2')

listGateways()

Get all registered gateways:

typescript

const gateways = mastra.listGateways();
console.log(Object.keys(gateways)); // ['myGateway', 'anotherGateway']

Gateway Properties

Required

Property	Type	Description
`id`	`string`	Unique identifier for the gateway, used as the gateway prefix for the model string
`name`	`string`	Human-readable gateway name

Methods

Method	Description
`fetchProviders()`	Fetch provider configurations
`buildUrl(modelId, envVars?)`	Build API URL for a model
`getApiKey(modelId)`	Get API key for authentication
`resolveLanguageModel(args)`	Create language model instance
`getId()`	Get gateway ID (returns `id` or `name`)

Provider Configuration

The fetchProviders() method returns a record of ProviderConfig objects:

typescript

interface ProviderConfig {
  name: string;                    // Display name
  models: string[];                // Available model IDs
  apiKeyEnvVar: string | string[]; // Environment variable(s) for API key
  gateway: string;                 // Gateway identifier
  url?: string;                    // Optional API base URL
  apiKeyHeader?: string;           // Optional custom auth header
  docUrl?: string;                 // Optional documentation URL
}

Gateway IDs vs Keys

Understanding the distinction:

Key: The registration key used when adding the gateway to Mastra (record key)
ID: The gateway's unique identifier (via id property or name if not set)

typescript

class VersionedGateway extends MastraModelGateway {
  readonly id = 'my-gateway-v2';     // Unique ID for versioning and prefixing
  readonly name = 'My Gateway';       // Display name
}

const mastra = new Mastra({
  gateways: {
    currentGateway: new VersionedGateway(), // Key: 'currentGateway'
  },
});

// Retrieve by key
const byKey = mastra.getGateway('currentGateway');

// Retrieve by ID
const byId = mastra.getGatewayById('my-gateway-v2');

// Both return the same gateway
console.log(byKey === byId); // true

Model ID Format

Models accessed through custom gateways follow this format:

[gatewayId]/[provider]/[model]

Examples:

private/my-provider/model-1

Advanced example

Token-based gateway with caching:

typescript

class TokenGateway extends MastraModelGateway {
  readonly id = 'token-gateway-v1';
  readonly name = 'Token Gateway';

  private tokenCache: Map<string, { token: string; expiresAt: number }> = new Map();

  async fetchProviders(): Promise<Record<string, ProviderConfig>> {
    const response = await fetch('https://api.gateway.com/providers');
    const data = await response.json();

    return {
      provider: {
        name: data.name,
        models: data.models,
        apiKeyEnvVar: 'GATEWAY_TOKEN',
        gateway: this.id,
      },
    };
  }

  async buildUrl(modelId: string, envVars?: Record<string, string>): Promise<string> {
    const token = await this.getApiKey(modelId);
    const siteId = envVars?.SITE_ID || process.env.SITE_ID;

    const response = await fetch(`https://api.gateway.com/sites/${siteId}/token`, {
      headers: { Authorization: `Bearer ${token}` },
    });

    const { url } = await response.json();
    return url;
  }

  async getApiKey(modelId: string): Promise<string> {
    const cached = this.tokenCache.get(modelId);

    if (cached && cached.expiresAt > Date.now()) {
      return cached.token;
    }

    const token = process.env.GATEWAY_TOKEN;
    if (!token) {
      throw new Error('Missing GATEWAY_TOKEN');
    }

    // Cache token for 1 hour
    this.tokenCache.set(modelId, {
      token,
      expiresAt: Date.now() + 3600000,
    });

    return token;
  }

  async resolveLanguageModel({ modelId, providerId, apiKey }: {
    modelId: string;
    providerId: string;
    apiKey: string;
  }): Promise<LanguageModelV2> {
    const baseURL = await this.buildUrl(`${providerId}/${modelId}`);

    return createOpenAICompatible({
      name: providerId,
      apiKey,
      baseURL,
      supportsStructuredOutputs: true,
    }).chatModel(modelId);
  }
}

Error handling

Provide descriptive errors for common failure scenarios:

typescript

class RobustGateway extends MastraModelGateway {
  // ... properties

  async getApiKey(modelId: string): Promise<string> {
    const apiKey = process.env.MY_API_KEY;

    if (!apiKey) {
      throw new Error(
        `Missing MY_API_KEY environment variable for model: ${modelId}. ` +
        `Please set MY_API_KEY in your environment.`
      );
    }

    return apiKey;
  }

  async buildUrl(modelId: string, envVars?: Record<string, string>): Promise<string> {
    const baseUrl = envVars?.BASE_URL || process.env.BASE_URL;

    if (!baseUrl) {
      throw new Error(
        `No base URL configured for model: ${modelId}. ` +
        `Set BASE_URL environment variable or pass it in envVars.`
      );
    }

    return baseUrl;
  }
}

Testing Custom Gateways

Example test structure:

typescript

import { describe, it, expect, beforeEach } from 'vitest';
import { Mastra } from '@mastra/core';

describe('MyPrivateGateway', () => {
  beforeEach(() => {
    process.env.MY_API_KEY = 'test-key';
  });

  it('should fetch providers', async () => {
    const gateway = new MyPrivateGateway();
    const providers = await gateway.fetchProviders();

    expect(providers['my-provider']).toBeDefined();
    expect(providers['my-provider'].models).toContain('model-1');
  });

  it('should integrate with Mastra', () => {
    const mastra = new Mastra({
      gateways: {
        private: new MyPrivateGateway(),
      },
    });

    const gateway = mastra.getGateway('private');
    expect(gateway.name).toBe('My Private Gateway');
  });

  it('should resolve models by ID', () => {
    const mastra = new Mastra({
      gateways: {
        key: new MyPrivateGateway(),
      },
    });

    const gateway = mastra.getGatewayById('my-private-gateway');
    expect(gateway).toBeDefined();
  });
});

Best practices

Use descriptive IDs for versioning: Set explicit id values when you need to version your gateways
typescript
```
readonly id = 'my-gateway-v1';
```
Implement proper error handling: Throw descriptive errors with actionable messages
Cache expensive operations: Cache tokens, URLs, or provider configurations when appropriate
Validate environment variables: Check for required environment variables in getApiKey and buildUrl
Document your gateway: Add JSDoc comments explaining the gateway's purpose and configuration
Follow naming conventions: Use clear, consistent naming for providers and models
Handle async operations: Use async/await for network requests and I/O operations
Test thoroughly: Write unit tests for all gateway methods

Built-in Gateways

Mastra includes built-in gateways as reference implementations:

NetlifyGateway: Netlify AI Gateway integration with token exchange
ModelsDevGateway: Registry of OpenAI-compatible providers from models.dev

See Netlify, OpenRouter, and Vercel for examples of gateway usage.

Custom gateways | Models | Mastra

Custom model gateways

Overview

Creating a Custom Gateway

Registering Custom Gateways

During Initialization

After Initialization

Using Custom Gateways

TypeScript Autocomplete

Manual Type Generation Alternatives

Gateway Management

getGateway(key)

getGatewayById(id)

listGateways()

Gateway Properties

Required

Methods

Provider Configuration

Gateway IDs vs Keys

Model ID Format

Advanced example

Error handling

Testing Custom Gateways

Best practices

Built-in Gateways