> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gp.scale.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Inference Overview

> An overview of Scale Generative Platform's Inference Capabilities

<Info>
  This uses SGP's V5 apis which are still in development
</Info>

SGP supports an OpenAI-compatible interface that allows you to use the same API across model providers and self-hosted models.

For instance, we can use OpenAI's client to do inference with Anthropic's `claude-3.5-sonnet`

```python theme={null}
import openai
from constants import SGP_API_KEY, SGP_TEST_ACCOUNT_ID

API_KEY = ...
ACCOUNT_ID = ...

client = openai.Client(
    api_key=API_KEY,
    base_url="https://api.sgp.scale.com/v5/",
    default_headers={
        "x-api-key": API_KEY,
        "x-selected-account-id": ACCOUNT_ID,
    },
)
response = client.chat.completions.create(
    model="anthropic/claude-3-5-sonnet-20240620",
    messages=[
        {"role": "user", "content": "the most interesting thing you ever learned"}
    ],
)
print(response.choices[0].message.content)
```

You can also use our SDK, [`scale-gp`](https://pypi.org/project/scale-gp/)

```python theme={null}
from scale_gp import SGPClient

sgp_client = SGPClient(
    api_key=...,
    account_id=...,
)
response = sgp_client.beta.chat.completions.create(
    model="anthropic/claude-3-5-sonnet-20240620",
    messages=[
        {"role": "user", "content": "the most interesting thing you ever learned"}
    ],
)
print(response.choices[0].message.content)
```

## Specifying which Model to do Inference On

As you might have noticed in the example, you need to specify the model names in a special way. Instead of just `claude-3.5-sonnet` it's `anthropic/claude-3-5-sonnet-20240620`

We need to do this because we have many providers, some of which provide the same model.

1. To distinguish them, the first part is the **provider**: `anthropic`, `openai`, `llmengine` etc.

2. The second part is the **model name**. You'll sometimes see a different model name than you would if you used the provider directly. This is because the versions we use are pinned to a specific point in time (meaning the model stays the same over time).

The two parts are joined together by a slash to get the overall name `openai/gpt-4o` or `anthropic/claude-3-5-sonnet-20240620`

For a complete list of model names for models not hosted on SGP (e.g., OpenAI and Anthropic models), you can view the documentation for the open source project we use to route to the correct provider here [https://docs.litellm.ai/docs/providers](https://docs.litellm.ai/docs/providers)&#x20;

For models hosted on SGP, you can visit [https://llm-engine.scale.com/model\_zoo/](https://llm-engine.scale.com/model_zoo/) to get the model names. Note that for these models, the provider is `llmengine`.