Overview
This guide is designed for human annotators to understand how to label evaluations created through the human evaluations creation flow. For information on creating contributor evaluations, see Contributor Evaluation.Flag for Review and auditing are off by default. Ask your Scale team to enable them for your account.
Start Labeling

Labeling Tasks

- Save: There is no save button, and answers are not saved until you submit. If you close a task with unsubmitted answers, the UI asks Discard changes?, and discarding loses them.
- Skip: Skip. If you are unsure about how to answer this task, use the Skip button. Skipping unassigns the task from you, moves it to the end of the queue, and opens the next task.
- Submit: Submit. When you are done with a task, use the Submit button to submit it. Submitting a task allows it to be audited by an auditor. You are allowed to edit a task after you submit if it has not yet been audited.
- Flag for Review: If you want to submit a task but have concerns about the task you want to flag to an auditor, toggle the flag for review button and leave a comment. The auditor will see your comment and see that you flagged this for review.