# Register an LLM Service Provider

LLM Service Providers are organization-level resources that represent connections to upstream LLM APIs (e.g., OpenAI, Anthropic, AWS Bedrock). Once registered, they are exposed through an AI Gateway and can be attached to agents across any project in the organization.

## Prerequisites[​](#prerequisites "Direct link to Prerequisites")

* Admin access to the WSO2 Agent Manager Console
* At least one AI Gateway registered and active (see [Register an AI Gateway](/agent-manager/docs/next/tutorials/register-ai-gateway/.md))
* API credentials for the target LLM provider (e.g., an OpenAI API key)

***

## Step 1: Navigate to LLM Service Providers[​](#step-1-navigate-to-llm-service-providers "Direct link to Step 1: Navigate to LLM Service Providers")

1. Log in to the WSO2 Agent Manager Console (`http://localhost:3000`).

2. Go to the Organization level by closing the projects section from the top navigation.

3. In the left sidebar, click **LLM Service Providers** under the **RESOURCES** section.

   > The LLM Service Providers page lists all registered providers with their Name, Template, and Last Updated time.

***

## Step 2: Add a New Provider[​](#step-2-add-a-new-provider "Direct link to Step 2: Add a New Provider")

1. Click the **+ Add Service Provider** button.

2. Fill in the **Basic Details**:

   | Field                    | Description                                                                    | Example                               |
   | ------------------------ | ------------------------------------------------------------------------------ | ------------------------------------- |
   | **Name** *(required)*    | A descriptive name for this provider configuration                             | `Production OpenAI Provider`          |
   | **Version** *(required)* | Version identifier for this provider configuration                             | `v1.0`                                |
   | **Short description**    | Optional description of the provider's purpose                                 | `Primary LLM provider for production` |
   | **Context path**         | The API path prefix for this provider (must start with `/`, no trailing slash) | `/my-provider`                        |

3. Under **Provider Template**, select one of the pre-built provider templates:

   | Template             | Description                     |
   | -------------------- | ------------------------------- |
   | **Anthropic**        | Claude models via Anthropic API |
   | **AWS Bedrock**      | AWS-hosted foundation models    |
   | **Azure AI Foundry** | Azure AI model deployments      |
   | **Azure OpenAI**     | OpenAI models hosted on Azure   |
   | **Gemini**           | Google Gemini models            |
   | **Mistral**          | Mistral AI models               |
   | **OpenAI**           | OpenAI models (GPT-4, etc.)     |

   Selecting a template auto-populates the upstream URL, authentication type, and API specification.

4. Provide the Credentials for the selected template. (Follow the official documentation of the respective providers for getting an api key/ credential)

5. Click **Add provider**.

***

## Step 3: Configure Provider Settings[​](#step-3-configure-provider-settings "Direct link to Step 3: Configure Provider Settings")

After creation, the provider detail page appears with six configuration tabs.

### Overview Tab[​](#overview-tab "Direct link to Overview Tab")

Displays a summary of the provider:

| Field              | Description                                                      |
| ------------------ | ---------------------------------------------------------------- |
| **Context**        | The context path (e.g., `/test`)                                 |
| **Upstream URL**   | The backend LLM API endpoint (e.g., `https://api.openai.com/v1`) |
| **Auth Type**      | Authentication method (e.g., `api-key`)                          |
| **Access Control** | Current access policy (e.g., `allow_all`)                        |

The **Invoke URL & API Key** section shows:

* **Gateway**: Select which AI Gateway exposes this provider.
* **Invoke URL**: The full URL agents use to call this provider through the gateway (auto-generated).
* **Generate API Key**: Generate a client API key for agents to authenticate against this provider.

***

### Connection Tab[​](#connection-tab "Direct link to Connection Tab")

Configure the upstream connection to the LLM Provider API:

| Field                     | Description                                         | Example                     |
| ------------------------- | --------------------------------------------------- | --------------------------- |
| **Provider Endpoint**     | The base URL of the upstream LLM API                | `https://api.openai.com/v1` |
| **Authentication**        | Auth method for the upstream call                   | `API Key`                   |
| **Authentication Header** | HTTP header used to pass the credential             | `Authorization`             |
| **Credentials**           | The API key or secret for the upstream LLM provider | `sk-...`                    |

Click **Save** to persist changes.

***

### Access Control Tab[​](#access-control-tab "Direct link to Access Control Tab")

Control which API resources are accessible through this provider:

* **Mode**: Choose `Allow all` (default – all resources permitted) or `Deny all` (whitelist only).
* **Allowed Resources**: List of API operations permitted (e.g., `GET /assistants`, `POST /chat/completions`).
* **Denied Resources**: List of API operations explicitly blocked.

Use the arrow buttons to move resources between the Allowed and Denied lists. You can also **Import from specification** to populate the resource list from an OpenAPI spec.

***

### Security Tab[​](#security-tab "Direct link to Security Tab")

Configure how to authenticate to this provider via the gateway:

| Field              | Description                           | Example     |
| ------------------ | ------------------------------------- | ----------- |
| **Authentication** | Auth scheme for inbound calls         | `apiKey`    |
| **Header Key**     | HTTP header name carrying the API key | `X-API-Key` |
| **Key Location**   | Where the key is passed               | `header`    |

***

### Rate Limiting Tab[​](#rate-limiting-tab "Direct link to Rate Limiting Tab")

Set backend rate limits to protect the upstream LLM API:

* **Mode**: `Provider-wide` (single limit for all resources) or `Per Resource` (limits per endpoint).
* **Request Counts**: Configure request-per-window thresholds.
* **Token Count**: Configure token-per-window thresholds.
* **Cost**: *(Coming soon)* Cost-based limits.

***

### Guardrails Tab[​](#guardrails-tab "Direct link to Guardrails Tab")

Attach content safety policies to this provider:

* **Global Guardrails**: Apply to all API resources under this provider. Click **+ Add Guardrail** to attach one.
* **Resource-wise Guardrails**: Per-operation guardrails for individual API endpoints (e.g., `POST /chat/completions`).

***

## Verifying the Provider[​](#verifying-the-provider "Direct link to Verifying the Provider")

The registered provider appears in the **LLM Service Providers** list showing its name and the template used (e.g., `OpenAI`). From the Overview tab, select your active AI Gateway to see the **Invoke URL** — this is the endpoint agents use to call the LLM through the gateway.

***

## Notes[​](#notes "Direct link to Notes")

* The **context path** must be unique per organization. It forms part of the invoke URL: `<gateway-host><context-path>`.
* Credentials entered in the Connection tab are stored securely and never exposed in the UI.
* A provider must be associated with at least one AI Gateway to be callable by agents.
* Multiple providers can share the same gateway but must have distinct context paths.