List All Model Deployments

JavaScript

import SGPClient from 'sgp';

const client = new SGPClient({
  apiKey: 'My API Key',
});

// Automatically fetches more pages as needed.
for await (const modelDeployment of client.models.deployments.listAll()) {
  console.log(modelDeployment.id);
}

{
  "items": [
    {
      "name": "<string>",
      "id": "<string>",
      "created_at": "2023-11-07T05:31:56Z",
      "account_id": "<string>",
      "created_by_user_id": "<string>",
      "created_by_identity_type": "user",
      "status": "<string>",
      "model_creation_parameters": {},
      "model_endpoint_id": "<string>",
      "model_instance_id": "<string>",
      "vendor_configuration": {
        "min_workers": 0,
        "max_workers": 1,
        "per_worker": 10,
        "vendor": "LAUNCH"
      },
      "deployment_metadata": {}
    }
  ],
  "total_item_count": 123,
  "current_page": 123,
  "items_per_page": 123
}

GET

model-deployments

JavaScript

import SGPClient from 'sgp';

const client = new SGPClient({
  apiKey: 'My API Key',
});

// Automatically fetches more pages as needed.
for await (const modelDeployment of client.models.deployments.listAll()) {
  console.log(modelDeployment.id);
}

{
  "items": [
    {
      "name": "<string>",
      "id": "<string>",
      "created_at": "2023-11-07T05:31:56Z",
      "account_id": "<string>",
      "created_by_user_id": "<string>",
      "created_by_identity_type": "user",
      "status": "<string>",
      "model_creation_parameters": {},
      "model_endpoint_id": "<string>",
      "model_instance_id": "<string>",
      "vendor_configuration": {
        "min_workers": 0,
        "max_workers": 1,
        "per_worker": 10,
        "vendor": "LAUNCH"
      },
      "deployment_metadata": {}
    }
  ],
  "total_item_count": 123,
  "current_page": 123,
  "items_per_page": 123
}

Authorizations

x-api-key

string

header

required

Query Parameters

sort_by

enum<string>[]

Available options:

model_creation_parameters:asc,

model_creation_parameters:desc,

model_endpoint_id:asc,

model_endpoint_id:desc,

model_instance_id:asc,

model_instance_id:desc,

vendor_configuration:asc,

vendor_configuration:desc,

deployment_metadata:asc,

deployment_metadata:desc,

status:asc,

status:desc,

id:asc,

id:desc,

created_at:asc,

created_at:desc,

account_id:asc,

account_id:desc,

created_by_user_id:asc,

created_by_user_id:desc,

created_by_identity_type:asc,

created_by_identity_type:desc,

name:asc,

name:desc

account_id

string

Optional filter by account id

page

integer

default:1

Page number for pagination to be returned by the given endpoint. Starts at page 1

Required range: x >= 1

limit

integer

default:100

Maximum number of artifacts to be returned by the given endpoint. Defaults to 100 and cannot be greater than 10k.

Required range: 1 <= x <= 10000

Response

Successful Response

items

ModelDeploymentResponse · object[]

required

The data returned for the current page.

Show child attributes

items.name

string

required

items.id

string

required

The unique identifier of the entity.

items.created_at

string<date-time>

required

The date and time when the entity was created in ISO format.

items.account_id

string

required

The ID of the account that owns the given entity.

items.created_by_user_id

string

required

The user who originally created the entity.

items.created_by_identity_type

enum<string>

required

The type of identity that created the entity.

Available options:

user,

service_account

items.status

string

required

Status of the model's deployment.

items.model_creation_parameters

Model Creation Parameters · object

items.model_endpoint_id

string

items.model_instance_id

string

items.vendor_configuration

LaunchDeploymentVendorConfiguration · object

LaunchDeploymentVendorConfiguration
LLMEngineDeploymentVendorConfiguration

Show child attributes

items.vendor_configuration.min_workers

integer

default:0

items.vendor_configuration.max_workers

integer

default:1

items.vendor_configuration.per_worker

integer

default:10

The maximum number of concurrent requests that an individual worker can service. Launch automatically scales the number of workers for the endpoint so that each worker is processing per_worker requests, subject to the limits defined by min_workers and max_workers.

If the average number of concurrent requests per worker is lower than per_worker, then the number of workers will be reduced. - Otherwise, if the average number of concurrent requests per worker is higher than per_worker, then the number of workers will be increased to meet the elevated traffic.

Here is our recommendation for computing per_worker:

Compute min_workers and max_workers per your minimum and maximum throughput requirements. 2. Determine a value for the maximum number of concurrent requests in the workload. Divide this number by max_workers. Doing this ensures that the number of workers will "climb" to max_workers.

items.vendor_configuration.vendor

string

default:LAUNCH

Allowed value: "LAUNCH"

items.deployment_metadata

Deployment Metadata · object

total_item_count

integer

required

The total number of items of the query

current_page

integer

required

The current page number.

items_per_page

integer

required

The number of items per page.

Get Model usage by model name Get Model usage for one deployment

⌘I

Knowledge Bases

Chunks

Agents

Completions

Chat Completions

Models

Users

Accounts

Organizations

Question Sets

Evaluations

Evaluation Configs

Evaluation Datasets

Studio Projects

Application Specs

Questions

Knowledge Base Data Sources

Model Templates V3 (Beta)

Model server

API Reference

Fine Tuning Jobs V3 (Beta)

Training Datasets V3 (Beta)

package deployments

Beta

Applications

ChatThreads

Interactions

MonitoringDashboard

Chat Themes

account groups

List All Model Deployments

Authorizations

Query Parameters

Response