POST
/
v4
/
evaluation-datasets
/
{evaluation_dataset_id}
/
generation-jobs

Authorizations

x-api-key
string
headerrequired

Headers

x-selected-account-id
string

Path Parameters

evaluation_dataset_id
string
required

Body

application/json
num_test_cases
integer

Number of test cases to generate for the evaluation dataset

group_by_artifact_id
boolean
default: false

If this flag is true, for every generated test case, the chunks used to generate it will be guaranteed to be from the same document (artifact).

harms_list
string[]

List of harms to be used for the evaluation dataset generation. If not provided, generation will use the knowledge base id.

advanced_config
object

Advanced configuration for the evaluation dataset generation job.

custom_instructions
string

Custom instructions for test case generation

Response

200 - application/json
generation_job_id
string
required

ID of the async job associated with this evaluation dataset generation

status
enum<string>
required

Status of the async job

Available options:
Pending,
Running,
Completed,
Failed,
Canceled
created_at
string
required

The timestamp at which the upload job started.

updated_at
string
required

The timestamp at which the upload job was last updated.

num_test_cases
integer

Optional number of test cases input to the job

num_completed_test_cases
integer

Number of test cases that have been generated

total_chunk_count
integer

Number of chunks in the knowledge base

failure_reason
string

Reason for the job's failure, if applicable

Get Evaluation Dataset Generation JobsGet Evaluation Dataset Generation Job