Get Test Case Result

account_id

string

required

The ID of the account that owns the given entity.

application_spec_id

string

required

created_at

string

required

The date and time when the entity was created in ISO format.

created_by_user_id

string

required

The user who originally created the entity.

evaluation_dataset_id

string

required

evaluation_dataset_version_num

string

required

evaluation_id

string

required

id

string

required

The unique identifier of the entity.

label_status

enum<string>

required

The status of the test case result. This should not be explictly set when creating a test case result. When patching a test case result, this field can be set to 'PENDING' to prevent the test case result from being marked 'COMPLETED'.

Available options:

PENDING,

COMPLETED,

FAILED

test_case_evaluation_data

object

required

test_case_id

string

required

annotated_by_user_id

string

The user who annotated the task.

annotation_results

object[]

application_test_case_output

object

application_test_case_output.account_id

string

required

The ID of the account that owns the given entity.

application_test_case_output.application_variant_id

string

required

application_test_case_output.created_at

string

required

The date and time when the entity was created in ISO format.

application_test_case_output.evaluation_dataset_id

string

required

application_test_case_output.id

string

required

The unique identifier of the entity.

application_test_case_output.output

object

required

application_test_case_output.schema_type

enum<string>

required

Available options:

GENERATION,

FLEXIBLE

application_test_case_output.test_case_id

string

required

application_test_case_output.test_case_version_id

string

required

application_test_case_output.application_interaction_id

string

application_test_case_output.application_test_case_output_group_id

string

application_test_case_output.interaction

object

application_test_case_output.interaction.aggregated

boolean

required

Boolean of whether this interaction has been uploaded to s3 bucket yet, default is false

application_test_case_output.interaction.application_spec_id

string

required

application_test_case_output.interaction.application_variant_id

string

required

application_test_case_output.interaction.created_at

string

required

The date and time when the entity was created in ISO format.

application_test_case_output.interaction.duration_ms

integer

required

How much time the step took in milliseconds(ms)

application_test_case_output.interaction.id

string

required

application_test_case_output.interaction.input

object

required

application_test_case_output.interaction.operation_status

enum<string>

required

The outcome of the operation

Available options:

SUCCESS,

ERROR

application_test_case_output.interaction.output

object

required

application_test_case_output.interaction.start_timestamp

string

required

application_test_case_output.interaction.chat_thread_id

string

application_test_case_output.interaction.interaction_source

enum<string>

Available options:

EXTERNAL_AI,

EVALUATION,

SGP_CHAT,

AGENTS_SERVICE

application_test_case_output.interaction.operation_metadata

object

The JSON representation of the metadata insights emitted through the execution. This can differ based on different types of operations

application_test_case_output.interaction.trace_spans

object[]

application_test_case_output.interaction.trace_spans.application_interaction_id

string

required

The id of the application insight this step belongs to

application_test_case_output.interaction.trace_spans.duration_ms

integer

required

How much time the step took in milliseconds(ms)

application_test_case_output.interaction.trace_spans.id

string

required

Identifies the application step

application_test_case_output.interaction.trace_spans.node_id

string

required

The id of the node in the application_variant config that emitted this insight

application_test_case_output.interaction.trace_spans.operation_input

object

required

The JSON representation of the input that this step received.

application_test_case_output.interaction.trace_spans.operation_output

object

required

The JSON representation of the output that this step emitted.

application_test_case_output.interaction.trace_spans.operation_status

enum<string>

required

The outcome of the operation

Available options:

SUCCESS,

ERROR

application_test_case_output.interaction.trace_spans.operation_type

enum<string>

required

Type of the operation, e.g. RERANKING

Available options:

TEXT_INPUT,

TEXT_OUTPUT,

COMPLETION_INPUT,

COMPLETION,

KB_RETRIEVAL,

KB_INPUT,

RERANKING,

EXTERNAL_ENDPOINT,

PROMPT_ENGINEERING,

DOCUMENT_INPUT,

MAP_REDUCE,

DOCUMENT_SEARCH,

DOCUMENT_PROMPT,

CUSTOM,

INPUT_GUARDRAIL,

OUTPUT_GUARDRAIL,

CODE_EXECUTION,

DATA_MANIPULATION,

EVALUATION,

FILE_RETRIEVAL

application_test_case_output.interaction.trace_spans.start_timestamp

string

required

The start time of the step

application_test_case_output.interaction.trace_spans.operation_expected

object

The JSON representation of the expected output for this step

application_test_case_output.interaction.trace_spans.operation_metadata

object

The JSON representation of the metadata insights emitted through the execution. This can differ based on different types of operations

application_test_case_output_id

string

archived_at

string

The date and time when the entity was archived in ISO format.

audit_comment

string

audit_required

boolean

audit_status

enum<string>

Available options:

UNAUDITED,

FIXED,

APPROVED

completed_at

string

metrics

object

result

object

The result of the test case evaluation, in JSON form where the key is the question ID and the value is the result.

task

object

task.account_id

string

required

The ID of the account that owns the given entity.

task.id

string

required

task.priority

integer

required

task.status

enum<string>

required

Available options:

PENDING,

COMPLETED

task.task_entity_id

string

required

task.task_entity_parent_id

string

required

task.task_type

enum<string>

required

Available options:

EVALUATION_ANNOTATION

task.assigned_to

object

task.assignment_expires_at

string

The date and time when the task assignment expires in ISO format.

task.task_entity

object

The entity that the task is associated with.

task.task_entity.account_id

string

required

The ID of the account that owns the given entity.

task.task_entity.application_spec_id

string

required

task.task_entity.created_at

string

required

The date and time when the entity was created in ISO format.

task.task_entity.created_by_user_id

string

required

The user who originally created the entity.

task.task_entity.evaluation_dataset_id

string

required

task.task_entity.evaluation_dataset_version_num

string

required

task.task_entity.evaluation_id

string

required

task.task_entity.id

string

required

The unique identifier of the entity.

task.task_entity.label_status

enum<string>

required

The status of the test case result. This should not be explictly set when creating a test case result. When patching a test case result, this field can be set to 'PENDING' to prevent the test case result from being marked 'COMPLETED'.

Available options:

PENDING,

COMPLETED,

FAILED

task.task_entity.test_case_evaluation_data

object

required

task.task_entity.test_case_id

string

required

task.task_entity.annotated_by_user_id

string

The user who annotated the task.

task.task_entity.application_test_case_output_id

string

task.task_entity.archived_at

string

The date and time when the entity was archived in ISO format.

task.task_entity.audit_comment

string

task.task_entity.audit_required

boolean

task.task_entity.audit_status

enum<string>

Available options:

UNAUDITED,

FIXED,

APPROVED

task.task_entity.completed_at

string

task.task_entity.result

object

The result of the test case evaluation, in JSON form where the key is the question ID and the value is the result.

task.task_entity.test_case_evaluation_data_schema

enum<string>

default:

GENERATION

Available options:

GENERATION

task.task_entity.time_spent_labeling_s

integer

The time spent labeling in seconds.

test_case_evaluation_data_schema

enum<string>

default:

GENERATION

Available options:

GENERATION

test_case_version

object

time_spent_labeling_s

integer

The time spent labeling in seconds.

Knowledge Bases

Chunks

Agents

Completions

Chat Completions

Models

Users

Accounts

Organizations

Question Sets

Evaluations

Evaluation Configs

Evaluation Datasets

Studio Projects

Application Specs

Questions

Knowledge Base Data Sources

Model Templates V3 (Beta)

Model server

API Reference

Fine Tuning Jobs V3 (Beta)

Training Datasets V3 (Beta)

package deployments

Beta

Applications

ChatThreads

Interactions

MonitoringDashboard

Chat Themes

account groups

account

Guardrails

Get Test Case Result

Description

Details

Authorizations

Path Parameters

Query Parameters

Response