POST
/
v4
/
evaluation-configs
import SGPClient from 'sgp';

const client = new SGPClient({
  apiKey: process.env['SGP_API_KEY'], // This is the default and can be omitted
});

async function main() {
  const evaluationConfig = await client.evaluationConfigs.create({
    account_id: 'account_id',
    question_set_id: 'question_set_id',
  });

  console.log(evaluationConfig.id);
}

main();
{
  "evaluation_type": "studio",
  "question_set_id": "<string>",
  "studio_project_id": "<string>",
  "auto_evaluation_model": "gpt-4-32k-0613",
  "auto_evaluation_parameters": {
    "temperature": 1,
    "batch_size": 13
  },
  "id": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "account_id": "<string>",
  "created_by_user_id": "<string>"
}

Authorizations

x-api-key
string
header
required

Headers

x-selected-account-id
string | null

Body

application/json
question_set_id
string
required
account_id
string
required

The ID of the account that owns the given entity.

evaluation_type
default:
llm_auto

Evaluation type

Available options:
llm_auto
studio_project_id
string
auto_evaluation_model
enum<string>
default:
gpt-4-turbo-2024-04-09

The name of the model to be used for auto-evaluation

Available options:
llama-3-1-70b-instruct,
gpt-4-turbo-2024-04-09,
llama-3-70b-instruct-bedrock
auto_evaluation_parameters
object

Execution parameters for auto-evaluation

Response

200
application/json
Successful Response
evaluation_type
enum<string>
required

Evaluation type

Available options:
studio,
llm_auto,
human,
llm_benchmark
question_set_id
string
required
id
string
required

The unique identifier of the entity.

created_at
string
required

The date and time when the entity was created in ISO format.

account_id
string
required

The ID of the account that owns the given entity.

created_by_user_id
string
required

The user who originally created the entity.

studio_project_id
string
auto_evaluation_model
enum<string>

The name of the model to be used for auto-evaluation

Available options:
gpt-4-32k-0613,
gpt-4-turbo-preview,
gpt-4-turbo-2024-04-09,
llama-3-70b-instruct,
llama-3-1-70b-instruct,
llama-3-70b-instruct-bedrock
auto_evaluation_parameters
object

Execution parameters for auto-evaluation