> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gp.scale.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent Evaluation

> Leverage specialized agents to produce evaluation results.

## Agent-as-a-Judge

Use the `auto_evaluation.agent` task type for configuring auto evaluation tasks with specialized agents.

<Expandable title="configuration properties" defaultOpen={true}>
  <ResponseField name="model" type="string" required>
    The identifier of the model (in `<model_provider>/<model_name>` format) that the Agent will use to execute the task.
  </ResponseField>

  <ResponseField name="output_rules" type="string[]" required>
    A space to provide additional instructions to the agent.
  </ResponseField>

  <ResponseField name="output_type" type="string">
    If not provided, `output_type` is inferred from output rules.
  </ResponseField>

  <ResponseField name="temperature" type="float">
    For the `APEAgent` (Default Agent) you can configure a temperature
  </ResponseField>
</Expandable>

### Example Usage

The following illustrates a basic example in which an agent task is defined with an Instruction Following Agent to evaluate the adherence of a given response to a prompt.

```python theme={null}
client.evaluations.create(
  name="Example Instruction Following Evaluation,
  data=[
    {
      "prompt": "You must either describe the process of photosynthesis in exactly 20 words or explain why it’s important in a single sentence under 10 words — choose whichever option is more precise.",
      "response": "Photosynthesis converts sunlight, water, and carbon dioxide into glucose and oxygen, sustaining plant life and fueling ecosystems.",
    },
    ...
  ],
  tasks=[
    {
      "task_type": "auto_evaluation.agent",
      "alias": "instruction-following-agent",
      "configuration": {
        "name": "instruction-following-agent",
        "definition": "Perform comprehensive analysis of instruction following",
        "output_rules": [
            "Score adherence on a 1-5 scale", 
            "List specific violations and well-followed instructions"
        ],
        "output_type": "integer",
        "designated_to": {
                "agent_name": "IFAgent",
                "config": {
                    "model": "openai/o3-mini"
                }
            }
      }
    }
  ]
)
```