Models

Create Model

Description

Creates and hosts a model based on a model template.

Base embedding models, chunk ranking functions, and LLMs are often not sufficient for customer use cases. We have shown in various blogs that fine-tuning these models on customer data can lead to significant improvements in performance.

Details

Before creating a model, you must first create a model template. A model template serves 2 purposes. First, it provides common scaffolding that is static across multiple models. Second, it exposes several variables that can be injected at model creation time to customize the model.

For example, a model template can define a docker image that contains code to run a HuggingFace or SentenceTransformers model. This docker image code also accepts environment variables that can be set to swap out the model weights or model name. Refer to the Create Model Template API for more details.

To create a new model, users must refer to an existing model template and provide the necessary parameters the the model template requires in its model_creation_parameters_schema field. The combination of the model template and the model creation parameters will be used to create and deploy a new model.

Once a model has been created, it can be executed by calling the Execute Model API.

Coming Soon

Some of our EGP APIs depend on models, for example Knowledge Base APIs depend on embedding models, Chunk Ranking APIs depend on ranking models, and Completion APIs depend on LLMs.

In the near future, if a model is created from a model template that is compatible with one of these APIs (based on the model template’s model_type field), the model will automatically be registered with the API. This will allow users to immediately start using the model with those API without any additional setup.

POST

models

import SGPClient from 'sgp';

const client = new SGPClient({
  apiKey: process.env['SGP_API_KEY'], // This is the default and can be omitted
});

async function main() {
  const modelInstance = await client.models.create({
    account_id: 'account_id',
    model_type: 'COMPLETION',
    name: 'name',
  });

  console.log(modelInstance.id);
}

main();

{
  "name": "<string>",
  "display_name": "<string>",
  "model_vendor": "OPENAI",
  "model_type": "COMPLETION",
  "base_model_id": "<string>",
  "base_model_metadata": {
    "ui_model_section_type": "PARTNER",
    "model_developer": "<string>",
    "model_license_url": "<string>",
    "delivery_date": "<string>",
    "modelDetails": {
      "number_of_parameters": 123,
      "token_context_window": 123,
      "languages": 123,
      "alignments": 123
    }
  },
  "model_creation_parameters": {},
  "model_card": "<string>",
  "training_data_card": "<string>",
  "description": "<string>",
  "model_template_id": "<string>",
  "model_group_id": "<string>",
  "id": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "account_id": "<string>",
  "created_by_user_id": "<string>",
  "request_schema": {},
  "response_schema": {},
  "deployment_count": 123,
  "supports_multi_turn": true
}

Authorizations

x-api-key

string

header

required

Headers

x-selected-account-id

string

Body

application/json

name

string

required

model_type

enum<string>

required

An enum representing the different types of models supported.

Attributes: COMPLETION: Denotes that the model type is completion. CHAT_COMPLETION: Denotes that the model type is chat completion. AGENT: Denotes that the model type is agent. EMBEDDING: Denotes that the model type is embedding. RERANKING: Denotes that the model type is reranking. GENERIC: Denotes that the model type is generic. BUNDLE: "Not to be used directly - type to surface a model bundle in the UI - TODO: Explicitly type a bundle

Available options:

COMPLETION,

CHAT_COMPLETION,

AGENT,

EMBEDDING,

RERANKING,

GENERIC,

BUNDLE

account_id

string

required

The ID of the account that owns the given entity.

display_name

string

model_vendor

enum<string>

An enum representing the different types of model vendors supported.

Attributes: OPENAI: Denotes that the model vendor is OpenAI. COHERE: Denotes that the model vendor is Cohere. GOOGLE: Denotes that the model vendor is Google. ANTHROPIC: Denotes that the model vendor is Anthropic. LLMENGINE: Denotes that the model vendor is LLM Engine. OTHER: Denotes that the model vendor is Other.

Available options:

OPENAI,

COHERE,

GOOGLE,

VERTEX_AI,

ANTHROPIC,

LAUNCH,

LLMENGINE,

BEDROCK,

OTHER

base_model_id

string

base_model_metadata

object

model_creation_parameters

object

model_card

string

training_data_card

string

description

string

model_template_id

string

model_group_id

string

Response

200

application/json

Successful Response

name

string

required

model_type

enum<string>

required

An enum representing the different types of models supported.

Available options:

COMPLETION,

CHAT_COMPLETION,

AGENT,

EMBEDDING,

RERANKING,

GENERIC,

BUNDLE

string

required

The unique identifier of the entity.

created_at

string

required

The date and time when the entity was created in ISO format.

account_id

string

required

The ID of the account that owns the given entity.

created_by_user_id

string

required

The user who originally created the entity.

request_schema

object

required

JSON schema for the requests to the model instance

response_schema

object

required

JSON schema for the response to the model

deployment_count

integer

required

Number of deployments of this model instance

supports_multi_turn

boolean

required

True if a model supports multi-turn conversations natively

display_name

string

model_vendor

enum<string>

An enum representing the different types of model vendors supported.

Available options:

OPENAI,

COHERE,

GOOGLE,

VERTEX_AI,

ANTHROPIC,

LAUNCH,

LLMENGINE,

BEDROCK,

OTHER

base_model_id

string

base_model_metadata

object

model_creation_parameters

object

model_card

string

training_data_card

string

description

string

model_template_id

string

model_group_id

string

List Models Get Model

import SGPClient from 'sgp';

const client = new SGPClient({
  apiKey: process.env['SGP_API_KEY'], // This is the default and can be omitted
});

async function main() {
  const modelInstance = await client.models.create({
    account_id: 'account_id',
    model_type: 'COMPLETION',
    name: 'name',
  });

  console.log(modelInstance.id);
}

main();

{
  "name": "<string>",
  "display_name": "<string>",
  "model_vendor": "OPENAI",
  "model_type": "COMPLETION",
  "base_model_id": "<string>",
  "base_model_metadata": {
    "ui_model_section_type": "PARTNER",
    "model_developer": "<string>",
    "model_license_url": "<string>",
    "delivery_date": "<string>",
    "modelDetails": {
      "number_of_parameters": 123,
      "token_context_window": 123,
      "languages": 123,
      "alignments": 123
    }
  },
  "model_creation_parameters": {},
  "model_card": "<string>",
  "training_data_card": "<string>",
  "description": "<string>",
  "model_template_id": "<string>",
  "model_group_id": "<string>",
  "id": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "account_id": "<string>",
  "created_by_user_id": "<string>",
  "request_schema": {},
  "response_schema": {},
  "deployment_count": 123,
  "supports_multi_turn": true
}

Knowledge Bases

Chunks

Agents

Completions

Chat Completions

Models

Users

Accounts

Organizations

Question Sets

Evaluations

Evaluation Configs

Evaluation Datasets

Studio Projects

Application Specs

Questions

Knowledge Base Data Sources

Model Templates V3 (Beta)

Model server

API Reference

Fine Tuning Jobs V3 (Beta)

Training Datasets V3 (Beta)

package deployments

Beta

Applications

ChatThreads

Interactions

MonitoringDashboard

Chat Themes

account groups

account

Guardrails

Create Model

Description

Details

Coming Soon

Authorizations

Headers

Body

Response