1. Instantiate Client
1. Instantiate Client
Follow the instructions in the Quickstart Guide to setup the SGP ClientFetch a Knowledge Base ID from:
https://egp.dashboard.scale.com/knowledge-bases
2. Create safety dataset
2. Create safety dataset
For safety evaluation datasets, a generation job workflow is created to generate test cases.
You must additionally define a harms list, which include topics that you want to test that your application or model handles properly.
Advanced configs for emotions, moods, methods, tones can also be provided
3. Start generation job
3. Start generation job
Start the generation job. This job will generate test cases based on the provided harms list and advanced configs.
4. Approve auto-generated test cases
4. Approve auto-generated test cases
Before publishing the dataset, review the auto-generated test cases and approve/decline each test case. Publishing is blocked until
all test cases are reviewed.
5. Publish the dataset
5. Publish the dataset
Publishing the dataset allows it to be available for use in evaluations