Skip to main content
POST
Python

Path Parameters

evaluation_dataset_id
string
required

Body

application/json
num_test_cases
integer

Number of test cases to generate for the evaluation dataset

group_by_artifact_id
boolean
default:false

If this flag is true, for every generated test case, the chunks used to generate it will be guaranteed to be from the same document (artifact).

harms_list
string[]

List of harms to be used for the evaluation dataset generation. If not provided, generation will use the knowledge base id.

advanced_config
Advanced Config · object

Advanced configuration for the evaluation dataset generation job.

custom_instructions
string

Custom instructions for test case generation

Response

Successful Response

generation_job_id
string
required

ID of the async job associated with this evaluation dataset generation

status
enum<string>
required

Status of the async job

Available options:
Pending,
Running,
Completed,
Failed,
Canceled
created_at
string<date-time>
required

The timestamp at which the upload job started.

updated_at
string<date-time>
required

The timestamp at which the upload job was last updated.

num_test_cases
integer

Optional number of test cases input to the job

num_completed_test_cases
integer

Number of test cases that have been generated

total_chunk_count
integer

Number of chunks in the knowledge base

failure_reason
string

Reason for the job's failure, if applicable