# AWS Architecture Reference
Source: https://docs.gp.scale.com/docs/infrastructure/aws/aws-architecture-reference

Overview of the SGP AWS architecture

## Architecture Reference

<img />

## Cloud Services

| Service                              | Usage                                                                    |
| ------------------------------------ | ------------------------------------------------------------------------ |
| Elastic Kubernetes Service (EKS)     | SGP applications are deployed to this cluster                            |
| Identity and Access Management (IAM) | Role and policy management for SGP applications access to cloud services |
| Secrets Manager                      | Secret and configuration management for SGP applications                 |
| Key Management Service (KMS)         | Encryption of sensitive data at rest                                     |
| Simple Storage Service (S3)          | Object storage for SGP applications (knowledge bases, models, etc.)      |
| Simple Queue Service (SQS)           | Karpenter spot instance interruption queue                               |
| OpenSearch                           | Search provider for SGP applications                                     |
| Elastic Container Registry (ECR)     | Container registry for custom agent images                               |
| Aurora PostgreSQL                    | Relational databases for SGP applications                                |
| DocumentDB                           | Document-oriented database for SGP applications                          |
| ElastiCache for Redis                | In-memory caching for SGP applications                                   |
| CloudWatch                           | Logging for cloud services                                               |

## Networking

SGP is deployed into a VPC spanning three Availability Zones with separate public and private subnets. All application workloads run in private subnets. Outbound internet access is provided by a NAT Gateway.

### Ingress

| Property             | Default                                                     |
| -------------------- | ----------------------------------------------------------- |
| Load balancer type   | Application Load Balancer (ALB)                             |
| Listener port        | 443 (TLS)                                                   |
| Placement            | External (internet-facing)                                  |
| WAFv2                | Optional — associate an existing Web ACL by name            |
| VPC Endpoint Service | Optional — enables AWS PrivateLink for private connectivity |

<Note>
  The load balancer can be configured as internal (private subnet only) for deployments that require no direct internet exposure.
</Note>

### Service Mesh

Istio is deployed as a cluster add-on and manages all east-west traffic between SGP services. An internal Istio ingress gateway handles traffic routing within the cluster.

## EKS Cluster Default Configuration

### Node Groups

| Node Group   | Size                         | Limits   | Machine Type  | Machine Image                     | Taints                             |
| ------------ | ---------------------------- | -------- | ------------- | --------------------------------- | ---------------------------------- |
| System       | 1                            | —        | m6a.2xlarge   | EKS Default AMI                   | CriticalAddonsOnly=true:NoSchedule |
| Default      | Karpenter Managed Node Group | 1000 CPU | c\*, m\*      | Karpenter Bottlerocket AMI Family |                                    |
| High Storage | Karpenter Managed Node Group | 100 CPU  | c\*, m\*, r\* | Karpenter Bottlerocket AMI Family |                                    |
| GPU          | Karpenter Managed Node Group | 385 CPU  | g5.\*         | Karpenter Bottlerocket AMI Family | nvidia.com/gpu:NoSchedule          |
| GPU (Large)  | Karpenter Managed Node Group | 100 CPU  | p4d.\*        | Karpenter Bottlerocket AMI Family | nvidia.com/gpu:NoSchedule          |

### Cluster Add-ons

| Add-on                       | Purpose                                                            |
| ---------------------------- | ------------------------------------------------------------------ |
| Karpenter                    | Workload-driven node autoscaling for Karpenter-managed node groups |
| Cluster Autoscaler           | Scaling for the system node group                                  |
| Istio                        | Service mesh for east-west traffic management                      |
| cert-manager                 | TLS certificate provisioning and renewal                           |
| AWS Load Balancer Controller | Kubernetes-native ALB and NLB provisioning                         |
| external-dns                 | Automatic Route 53 DNS record management                           |
| AWS Node Termination Handler | Graceful pod drain on spot interruption or node termination        |
| NVIDIA GPU Operator          | GPU driver and device plugin lifecycle management                  |
| AWS EBS CSI Driver           | Persistent volume support for EKS workloads                        |

## Aurora PostgreSQL Default Configuration

| Instance                                              | Service                           | Instance Count | Engine       | Instance Class |
| ----------------------------------------------------- | --------------------------------- | -------------- | ------------ | -------------- |
| agents-service-\<workspace\_id>                       | Agents                            | 2              | PostgreSQL16 | db.r6g.xlarge  |
| duc-\<workspace\_id>-duc-api-aurora-db                | Document Understanding Capability | 3              | PostgreSQL15 | db.r6g.large   |
| reducto-\<workspace\_id>-aurora-db                    | Reducto                           | 3              | PostgreSQL15 | db.r6g.large   |
| sae-\<workspace\_id>-sae-postgres                     | Semantic Analysis Engine          | 1              | PostgreSQL17 | db.r6g.large   |
| scale-egp-\<workspace\_id>-agentex-aurora-db          | Agentex                           | 3              | PostgreSQL15 | db.r6g.large   |
| scale-egp-\<workspace\_id>-egp-api-aurora-db          | EGP API                           | 3              | PostgreSQL15 | db.r6g.large   |
| scale-egp-\<workspace\_id>-identity-service-aurora-db | Identity Service                  | 3              | PostgreSQL15 | db.r6g.large   |
| scale-egp-\<workspace\_id>-llm-engine-aurora-db       | LLM Engine                        | 2              | PostgreSQL14 | db.r6g.large   |
| scale-egp-\<workspace\_id>-spicedb-postgres           | SpiceDB                           | 3              | PostgreSQL16 | db.r6g.large   |
| scale-egp-\<workspace\_id>-temporal-aurora-db         | Temporal                          | 3              | PostgreSQL15 | db.r6g.large   |
| scale-egp-\<workspace\_id>-train-aurora-db            | SGP Train                         | 3              | PostgreSQL15 | db.t3.medium   |

## DocumentDB Default Configuration

| Instance                                 | Service | Instance Count | Instance Class |
| ---------------------------------------- | ------- | -------------- | -------------- |
| scale-egp-\<workspace\_id>-compass-docdb | Compass | 1              | db.t3.medium   |
| \<workspace\_id>-agentex-mongo           | Agentex | 1              | db.r5.large    |

## Customer Sensitive Data

Sensitive data may be stored in the following services, all data is encrypted at rest and in transit:

* Aurora PostgreSQL: For storing metadata, persistent data, and service-specific databases (SGP API Backend, Identity Service, LLM Engine, SpiceDB, Agents Service, SGP Train, and Temporal)
* DocumentDB: For storing Agentex agent data/workflows and Workflows document processing state
* S3 Buckets: For storing large data objects and datasets
* OpenSearch: For managing text embeddings and vector search capabilities


# AWS SGP Deployment
Source: https://docs.gp.scale.com/docs/infrastructure/aws/aws-sgp-deployment

End-to-end guide to deploying SGP in an AWS account.

## Overview

This guide will walk you through the process of deploying SGP in an AWS account. SGP AWS infrastructure is defined by terraform modules managed by Scale.

## Prerequisites

* Access to an AWS account with sufficient permissions to create resources
* The following tools installed:
  * [AWS CLI](https://aws.amazon.com/cli/)
  * [Terraform](https://www.terraform.io/)
  * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
* The following from Scale:
  * The SGP AWS Infrastructure Terraform modules
  * A `workspace_id` and `registration_secret` unique to your deployment
  * `install.sh` script to bootstrap the SGP cluster
* A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional)
* A custom domain for your deployment (optional)

## Installation

### Step 1: Prepare Terraform Root Module

In a new directory, create a `.tf` file to invoke the SGP AWS infrastructure Terraform modules. Reference the `egp-hybrid-cloud-customer-deployment` module as the entrypoint for the SGP infrastructure.

```terraform theme={null}
module "egp_hybrid_deployment" {
  source                = "<path to egp-hybrid-cloud-customer-deployment module>"
  scale_account_id      = "<workspace_id>"
  registration_secret   = "<registration_secret>"  # sensitive — use TF_VAR_registration_secret env var

  domain = "<workspace_id>.workspace.egp.scale.com"  # or your custom domain

  # Core services — enable the components you need
  deploy_egp_api    = true   # EGP API backend — required for the SGP platform
  deploy_spellbook  = false  # Spellbook prompt management service (optional)
  deploy_sgp_models = false  # SGP model serving infrastructure (optional)
  deploy_llm_engine = false  # Model Engine service (optional)
  deploy_agentex    = false  # Agentex agentic workflow service (optional)
  create_agentex_tool_cache = false  # Persistent tool cache for Agentex (optional)
  enable_cloud_build = false # Cloud-based container build service for Agentex (optional)
  add_agents_service = false # Agent Service for building and running agents (optional)
  add_document_understanding_capability = false  # Document Understanding (Dex) service (optional)
  enable_reducto    = false  # Reducto OCR and document parsing service (optional)
  enable_train      = false  # SGP model training infrastructure (optional)
  enable_compass    = false  # Workflows workflow orchestration service (optional)
  enable_ukbp       = false  # Universal Knowledge Base Pipeline (optional)
  enable_system_manager = true  # System Manager deployment orchestrator — required
  enable_monitoring = false  # Observability stack (optional)

  create_temporal_postgresql = true  # Recommended; set false to use an in-cluster Cassandra instead

  omit_egp_control_plane = false  # Set true to disable communication with Scale's hosted control plane

  base_repository = "<base container registry URL>"  # Only needed if not pulling from Scale's registry

  openai_api_key    = "<openai_api_key>"     # sensitive — use TF_VAR_openai_api_key env var
  openai_org_id     = "<openai_org_id>"
  anthropic_api_key = "<anthropic_api_key>"  # sensitive — use TF_VAR_anthropic_api_key env var
  launch_api_key    = "<launch_api_key>"     # sensitive — use TF_VAR_launch_api_key env var

  authentication_type = "SAML"  # "SAML" or "OIDC"

  use_cmk = true  # Recommended for production: encrypts data at rest using AWS KMS

  # If deploying into an existing VPC rather than creating a new one, provide the IDs here
  # vpc_id             = "<existing VPC ID>"
  # private_subnet_ids = ["<subnet-id-1>", "<subnet-id-2>"]
  # public_subnet_ids  = ["<subnet-id-1>", "<subnet-id-2>"]

  access_cidrs = []  # List of CIDR blocks to allow access; defaults to open to internet

  # Load balancer configuration
  load_balancer_type = "application"  # "application" or "network"
  # waf_name          = "<WAFv2 Web ACL name>"  # Optional; only applies to application load balancers
  internal_lb        = false  # Set true to prevent internet access to the load balancer
  # additional_certs  = ["<certificate ARN>"]  # Additional TLS certificates for the load balancer

  # permissions_boundary_arn = "<ARN>"  # Optional: IAM permissions boundary for provisioned roles

  # See the module documentation for the full list of available variables
}

# Example Terraform provider and backend configuration
provider "aws" {
  region = "<your-aws-region>"
}

terraform {
  required_version = ">= 1.1.7"

  backend "s3" {
    bucket       = "sgp-<workspace_id>-terraform-state"
    key          = "sgp-<workspace_id>-sgp.tfstate"
    region       = "<your-aws-region>"
  }
}
```

### Step 2: Provision Infrastructure via Terraform

Run the following commands in the same directory as the previous step's .tf file.

```bash theme={null}
terraform init

terraform plan -out=tfplan

# Inspect this plan to review the planned resources to be provisioned before proceeding
terraform show tfplan 

terraform apply tfplan
```

*This step may take significant time due to resource creation dependencies*

<Note>
  If a pre-created certificate and domain entry are not provided, you will likely encounter the following error:

  ```bash theme={null}
  Error: creating ELBv2 Listener (arn:aws:elasticloadbalancing:us-east-1:12345678910:loadbalancer/net/scale-egp-<workspace_id>/12345678910): operation error Elastic Load Balancing v2: CreateListener, https response error StatusCode: 400, RequestID: 12345678910, api error UnsupportedCertificate: The certificate 'arn:aws:acm:us-east-1:12345678910:certificate/12345678910' must have a fully-qualified domain name, a supported signature, and a supported key size.
  ```

  Resolve this by properly configuring DNS and TLS for your domain before re-running `terraform apply`.
</Note>

### Step 3: Bootstrap the Cluster

Note the following values from the previous step's apply output:

```bash theme={null}
export SYSTEM_MANAGER_IRSA_ROLE_ARN=$(terraform output -raw system_manager_irsa_role_arn)
export SPICEDB_DATASTORE_URI=$(terraform output -raw spicedb_connection_string)
export AWS_REGION="<your-aws-region>"
```

#### Configure EKS access

Configure access for yourself to the provisioned EKS cluster:

1. Navigate to "Elastic Kubernetes Service" on the AWS console
2. Go to the required cluster
3. Go to *Access*
4. Under *IAM access entries* click *Create Access Entry*
5. Select your current AWS IAM role from the dropdown, leave the *Type* as *Standard*
6. Add the `AmazonEKSAdminPolicy` and `AmazonEKSClusterAdminPolicy` to the *Cluster* scope
7. Click create

#### Allowlist your IP (if needed)

1. Navigate to "Elastic Kubernetes Service" on the AWS console
2. Go to the required cluster
3. Go to *Networking*
4. On the right, click *Manage* and click *Endpoint Access*
5. Under *Advanced settings* add your IP address

#### Execute install.sh

Run the `install.sh` script to create the following Kubernetes resources:

* AWS Secret Manager CSI Driver
* Spicedb
* [SGP System Manager](/docs/infrastructure/system-manager)

```bash theme={null}
cd <path to install.sh>

# Set the correct kubectl context
aws eks update-kubeconfig --name scale-egp-<workspace_id> --region <aws_region>

# Execute the install.sh script
./install.sh
```

System Manager is the deployment orchestrator for the SGP platform. It will automatically begin deploying the other services required for the SGP platform within the cluster.

### Step 4: Verify the Deployment

Wait for all services to be ready:

```bash theme={null}
kubectl get helmreleases -A  # All should be Ready=True
kubectl get pods -A          # All should be Running
```

### Step 5: Configure Identity Provider

#### SAML Configuration

In your Identity Provider, configure a new SAML application with the following settings:

* Service Entity ID: `https://auth.<domain>`
* Redirect URI: `https://auth.<domain>/dashboard/org/saml/callback`

Modify the configuration `scale-egp-<workspace_id>/identity-service-saml-secrets` (either via System Manager or directly in AWS Secrets Manager):

```json theme={null}
{
  "id": "<workspace_id>",
  "samlConfiguration": {
        "entityId": "<entity id from your identity provider>",
        "x509Cert": "<x509 certificate from your identity provider>",
        "ssoUrl": "<sso URL from your identity provider>",
        "attributeMappings": {
            "email": "<email attribute name from your identity provider>",
            "firstName": "<first name attribute name from your identity provider>",
            "lastName": "<last name attribute name from your identity provider>"
        }
    }
}   
```

#### OIDC Configuration

In your Identity Provider, configure a new OIDC application with the following settings:

* Redirect URI: `https://auth.<domain>/dashboard/org/oidc/callback`

Modify the configuration `scale-egp-<workspace_id>/identity-service-oidc-secrets` (either via System Manager or directly in AWS Secrets Manager) with the following JSON:

```json theme={null}
{
  "id": "<workspace_id>",
  "oidcConfiguration": {
    "clientId": "<client ID from your identity provider>",
    "clientSecret": "<client secret from your identity provider>",
    "issuer": "<issuer from your identity provider>",
    "authorizationUrl": "<authorization URL from your identity provider>",
    "tokenUrl": "<token URL from your identity provider>",
    "userInfoUrl": "<user info URL from your identity provider>"
  }
}
```

After modifying the secret, restart the `sgp-system-manager` deployment to apply the changes.

```bash theme={null}
kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager
```

## Accessing the Platform

If all goes smoothly, you should be able to navigate to the SGP platform at `https://<workspace_id>.workspace.egp.scale.com` (or your custom domain) and authenticate via the configured identity provider.


# Azure Architecture Reference
Source: https://docs.gp.scale.com/docs/infrastructure/azure/azure-architecture-reference

Overview of the SGP Azure architecture

## Architecture Diagram

```mermaid theme={null}
graph TD
    USER(["Client"])
    INTERNET(["Internet"])

    subgraph AZ["Azure Subscription"]
        FD["Azure Front Door<br/>WAF · CDN · TLS termination"]

        subgraph VNET["Virtual Network — 10.0.0.0/16"]
            INGRESS["Istio Ingress<br/>Internal Load Balancer"]
            NAT["NAT Gateway<br/>Outbound egress"]

            subgraph AKS["AKS Cluster — private API server"]
                SYS["System Node Pool<br/>3× Standard_D4s_v3"]
                APP["User Node Pool<br/>3–10× Standard_D16s_v3"]
            end

            BASTION["Azure Bastion"]
            JUMP["Jump Host VM"]
        end

        KV["Key Vault<br/>Secrets + CMK keys"]
        LAW["Log Analytics Workspace"]

        PG[("PostgreSQL<br/>Flexible Server")]
        REDIS[("Redis Cache<br/>Premium")]
        STOR[("Storage Account")]
        SB[("Service Bus<br/>Premium")]
    end

    USER --> FD
    FD -->|"Private Link"| INGRESS
    INGRESS --> APP
    APP -->|"Private Endpoint"| PG
    APP -->|"Private Endpoint"| REDIS
    APP -->|"Private Endpoint"| STOR
    APP -->|"Private Endpoint"| SB
    APP --> KV
    SYS --> KV
    APP --> LAW
    SYS --> LAW
    APP --> NAT --> INTERNET
    BASTION --> JUMP -->|"kubectl"| AKS
```

***

## Resources by Type

### Compute Resources

| Resource         | Count | Purpose                      |
| ---------------- | ----- | ---------------------------- |
| AKS Cluster      | 1     | Kubernetes orchestration     |
| System Node Pool | 1     | System pods (fixed 3 nodes)  |
| User Node Pool   | 1     | Application workloads        |
| GPU Node Pool    | 0-1   | AI/ML workloads (optional)   |
| Cassandra Pool   | 0-1   | Temporal database (optional) |

**Total VMs:** 6-24 (3 system + 3-10 user + 0-5 GPU + 0-6 Cassandra)

***

### Network Resources

| Resource          | Count | Purpose                  |
| ----------------- | ----- | ------------------------ |
| Virtual Network   | 1     | Network boundary         |
| Subnets           | 4-5   | Network segmentation     |
| NSGs              | 4-5   | Traffic control          |
| Route Tables      | 3-4   | Traffic routing          |
| Private DNS Zones | 7     | Internal name resolution |
| Private Endpoints | 7     | Secure PaaS access       |
| Bastion Host      | 0-1   | Secure VM access         |
| Public IPs        | 0-1   | Bastion endpoint         |

***

### Data & Storage Resources

| Resource          | Count | Purpose                     |
| ----------------- | ----- | --------------------------- |
| PostgreSQL Server | 1     | Relational database         |
| Redis Cache       | 1     | Distributed cache           |
| Storage Account   | 1     | Blob/File storage           |
| AI Search Service | 0–1   | Full-text search (optional) |
| OpenAI Service    | 0–1   | LLM models (optional)       |

***

### Security Resources

| Resource              | Count | Purpose                |
| --------------------- | ----- | ---------------------- |
| Key Vault             | 1     | Secrets management     |
| Managed Identity      | 1-2   | Service authentication |
| RBAC Role Assignments | 10+   | Access control         |

***

### Monitoring Resources

| Resource                | Count | Purpose             |
| ----------------------- | ----- | ------------------- |
| Log Analytics Workspace | 1     | Centralized logging |
| Data Collection Rule    | 1     | AKS metrics         |
| Diagnostic Settings     | 7+    | Resource logging    |
| Datadog Connection      | 0-1   | External monitoring |

***

## Network Architecture

### Address Space Planning

```
VNet: 10.0.0.0/16 (65,536 IPs)
├── AKS Subnet: 10.0.1.0/24 (256 IPs)
├── Bastion Subnet: 10.0.2.0/26 (64 IPs)
├── Database Subnet: 10.0.3.0/24 (256 IPs)
└── Private Endpoints: 10.0.4.0/25 (128 IPs)

Pod CIDR: 10.244.0.0/16 (65,536 IPs)
Service CIDR: 10.243.0.0/16 (65,536 IPs)
```

### Traffic Flow

**Egress (Internet):**

```
Pods/VMs → NAT Gateway → Public IP → Internet
(Stateful, return traffic allowed)
```

**Ingress (Internal):**

```
Service IP → Load Balancer → Pod IP (via CNI)
```

**Database Access:**

```
AKS Pods → Private Endpoint → Private Link → PostgreSQL
(DNS: server.postgres.database.azure.com)
```

**External Service Access:**

```
AKS Pods → API Gateway / Load Balancer → OpenAI / AI Search
(Via Private Endpoints)
```

***

## AKS Configuration Deep Dive

### API Server Access

**Type:** Private cluster (recommended)
**Endpoint:** Internal only
**Access Method:** Bastion host or VPN
**DNS:** k8s..azmk8s.io (private)

### Network Policies

**Engine:** Azure Network Policy
**Scope:** Pod-to-pod communication
**Default:** Allow all (unrestricted)
**Configuration:** Define in Kubernetes manifests

### Container Registry

**Integration:** Azure Container Registry (optional)
**Authentication:** Managed identity or pull secrets
**Pulling:** Private endpoint (optional)

### Monitoring & Observability

**Azure Monitor Agent:** Deployed in kube-system
**Metrics:** CPU, Memory, Disk, Network
**Logs:** Container stdout/stderr, Kubernetes events
**Dashboards:** Pre-built in Log Analytics


# Azure SGP Deployment
Source: https://docs.gp.scale.com/docs/infrastructure/azure/azure-sgp-deployment

End-to-end guide to deploying SGP in an Azure subscription.

## Overview

This guide walks you through deploying SGP in an Azure cloud subscription using the SGP Azure Terraform modules. SGP Azure infrastructure is defined by terraform modules managed by Scale.

## Prerequisites

* Access to an Azure subscription with sufficient permissions to create resources (Contributor + User Access Administrator roles, or equivalent)
* The following tools installed:
  * [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) (`az login` completed)
  * [Terraform](https://www.terraform.io/) (`>= 1.1.7, < 2.0.0`)
  * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
* The following from Scale:
  * The SGP Azure Infrastructure Terraform modules (`azure-terraform/infra`)
  * A `workspace_id` and `registration_secret` unique to your deployment
* A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional)
* A custom domain for your deployment (optional)

## Installation

### Step 1: Build Configuration

Configuration is split across three files. This allows you to reuse the same configuration for multiple environments.

```
infra/
├── main.tfvars.json                         # Selects which domain + environment to deploy
└── config/
    └── <domain_code>/
        ├── default.yaml                     # Domain-wide defaults (all environments)
        └── <location>/
            └── <environment_type>.yaml      # Environment-specific overrides
```

#### File 1: `main.tfvars.json`

This file selects which configuration to load. Edit it to point at your customer and environment before running Terraform.

```json theme={null}
{
  "environment_type": "<dev|qa|staging|uat|prod>",
  "domain_code": "<A unique 3-9 alphanumeric character string to identify yourself>",
  "location": "<Azure region>"
}
```

| Field              | Description                                          | Allowed values                                 |
| ------------------ | ---------------------------------------------------- | ---------------------------------------------- |
| `environment_type` | Environment tier                                     | `dev`, `qa`, `staging`, `uat`, `prod`          |
| `domain_code`      | Customer identifier (3–9 alphanumeric chars/hyphens) | e.g. `acme`, `contoso`                         |
| `location`         | Azure region                                         | e.g. `eastus2`, `northeurope`, `southeastasia` |

#### File 2: `config/<domain_code>/default.yaml`

Customer-wide baseline settings that apply to **all** environments unless overridden. This file is the right place for stable policy and posture decisions: tenant identity, tagging standards, and security baselines for Key Vault, PostgreSQL, Redis, Storage, and other services.

```yaml theme={null}
# Identity
business_unit: "<your_business_unit>"
tenant_id: "<your_azure_tenant_id>"

default_tags:
  costcenter: "<cost_center>"
  product: "<domain_code>"
  deploymenttype: "new"
  safe-to-delete: "no"

# CMK key names (Terraform creates these keys when cmk.create=true in the env YAML)
cmk:
  create: true
  encryption_key_name: "encryptionCMK"
  k8s_key_name: "k8sEncryptionCMK"
  service_bus_key_name: "serviceBusEncryptionCMK"

# Key Vault network posture — secure-by-default baseline.
# Override public_network_access_enabled and network_acls_ip_rules in your env YAML
# if you need temporary public access during bootstrap from a laptop or CI runner.
keyvault:
  public_network_access_enabled: false
  enable_rbac_authorization: true
  sku_name: "standard"
  soft_delete_retention_days: 90    # Cannot be changed after vault creation
  purge_protection_enabled: true
  network_acls_default_action: "Deny"
  network_acls_bypass: "AzureServices"
  network_acls_ip_rules: []

# PostgreSQL baseline
psql:
  public_network_access_enabled: false
  sku_name: "GP_Standard_D2s_v3"
  psql_version: "15"
  backup_retention_days: 7
  bootstrap_aad_principals: false
  auth:
    active_directory_auth_enabled: true
    password_auth_enabled: true
  storage_mb: 32768
  storage_tier: "P10"
  auto_grow_enabled: true
  azure_extensions: "uuid-ossp,ltree"

# Redis baseline
redis:
  public_network_access_enabled: false
  sku_name: "Premium"
  family: "P"
  capacity: 1
  active_directory_authentication_enabled: true
  access_policy_name: "Data Owner"
  non_ssl_port_enabled: false
  tls_version: 1.2
  redis_version: 6

# Storage account baseline
storage_account:
  public_network_access_enabled: false
  min_tls_version: "TLS1_2"
  account_tier: "Standard"
  account_replication_type: "LRS"
  allow_nested_items_to_be_public: false

# OpenAI — leave client.mode empty to let Terraform infer from open_ai.create
open_ai:
  client:
    mode: ""
    key: ""
    org_id: ""
    custom_url: ""

# NAT Gateway baseline
nat_gateway:
  sku_name: "Standard"
  public_ip_sku: "Standard"
  public_ip_allocation_method: "Static"
  zones: ["1", "2", "3"]
  idle_timeout_in_minutes: 10

# AKS baseline
aks:
  load_balancer_sku: "standard"
  only_critical_addons_enabled: true
  key_vault_secret_rotation_enabled: true

# Front Door baseline
frontdoor:
  create: true
  ssl_mode: managed
  minimum_tls_version: "TLS12"
  forwarding_protocol: HttpOnly
  private_link_service_name: "sgp-ingress-lb"
  subdomains: ["auth", "api", "@", "admin"]
  extra_subdomains: []
  dns_ttl_seconds: 60
  ip_filtering:
    enabled: false
    allowed_ip_ranges: []

# System Manager bootstrapping baseline
bootstrapping:
  az_cli_version: "2.59.0"
  helm_chart_version: "2.1.0"
  base_repository: "<your_container_registry_url>"
```

#### File 3: `config/<domain_code>/<location>/<environment_type>.yaml`

Environment-specific configuration. This is where you set everything that differs per environment: resource names, subscription, CIDRs, feature flags, node pool sizing, and bootstrapping state.

```yaml theme={null}
# ── Environment identity ───────────────────────────────────────────────────────
location: eastus2
subscription_id: "<your_azure_subscription_id>"
name_suffix: "<workspace_id>"        # e.g. "acme01"
deployment_id: "<workspace_id>"
deployment_url: "<workspace_id>.workspace.egp.scale.com"
temporal_db_mode: postgresCommonHosted  # Recommended; or "cassandraK8s" (not recommended)

# ── Auth / secrets ────────────────────────────────────────────────────────────
authType: default    # Options: default, SAML, OIDC
RBAC: false
registration_secret: ""     # Auto-generated if empty
identity_service_jwt: ""    # Auto-generated if empty

# ── OIDC (required only when authType = OIDC) ─────────────────────────────────
oidc:
  clientId: ""
  clientSecret: ""
  issuer: ""
  authorizationUrl: ""
  tokenUrl: ""
  userInfoUrl: ""

# ── SAML (required only when authType = SAML) ─────────────────────────────────
saml:
  X509Cert: ""
  SSOUrl: ""
  emailAttrName: ""
  firstNameAttrName: ""
  lastNameAttrName: ""

# ── SSL ───────────────────────────────────────────────────────────────────────
sslMode: e2e    # Options: e2e, managed

# SSL cert/key for Istio ingress (PEM file paths or base64). Required when sslMode=e2e.
istio:
  sslCert: "<path_to_pem_certificate>"
  sslKey: "<path_to_pem_private_key>"

# ── Resource group + networking ───────────────────────────────────────────────
resource_group: "rg-eus2-<domain_code>-<workspace_id>-dev-01"

network:
  vnet: "sgpaz<workspace_id>-vnet"
  int_rt_name: "sgpaz<workspace_id>-rt-int"
  int_nsg_name: "sgpaz<workspace_id>-nsg-int"
  private_cluster: true
  nsg_rules:
    postgresql: true
    redis: true

private_dns:
  create: true
  private_cluster: true

subnets:
  psql:
    name: "sgpaz<workspace_id>-snet-psql"
    type: int
    address_prefixes:
      - "<cidr_block>"    # e.g. "10.95.23.160/27"
    service_endpoints:
      - Microsoft.Storage
    service_delegation:
      name: Microsoft.DBforPostgreSQL/flexibleServers
      actions:
        - Microsoft.Network/virtualNetworks/subnets/join/action

  deployment_scripts:
    name: "sgpaz<workspace_id>-snet-deployscripts"
    type: int
    address_prefixes:
      - "<cidr_block>"
    service_delegation:
      name: Microsoft.ContainerInstance/containerGroups
      actions:
        - Microsoft.Network/virtualNetworks/subnets/action

  aks:
    name: "sgpaz<workspace_id>-snet-aks"
    type: int
    address_prefixes:
      - "<cidr_block>"
    private_endpoint_network_policies: Disabled
    service_endpoints:
      - Microsoft.CognitiveServices

  bastion:
    name: AzureBastionSubnet
    type: int
    address_prefixes:
      - "<cidr_block>"

  jump_host:
    name: "sgpaz<workspace_id>-snet-jumphost"
    type: int
    address_prefixes:
      - "<cidr_block>"

# ── Workload Identity + NAT Gateway ───────────────────────────────────────────
workload_identities:
  enabled: true

nat_gateway:
  enabled: true
  name: "sgpaz<workspace_id>nat"
  zones: []    # [] = regional (no zone pinning)

# ── CMK ───────────────────────────────────────────────────────────────────────
# Set create=true for new deployments. If CMK keys already exist, set create=false
# and provide the key URIs to avoid "already exists (import required)" errors.
cmk:
  enabled: true
  create: true
  # encryption_key_uri: "https://<keyvault>.vault.azure.net/keys/encryptionCMK/<version>"
  # k8s_key_uri: "https://<keyvault>.vault.azure.net/keys/k8sEncryptionCMK/<version>"
  # service_bus_key_uri: "https://<keyvault>.vault.azure.net/keys/serviceBusEncryptionCMK/<version>"

# ── Key Vault ─────────────────────────────────────────────────────────────────
# Override public access settings here if bootstrapping from a laptop or CI runner.
keyvault:
  name: "sgpaz<workspace_id>keyvault"    # Must be globally unique; 3-24 characters
  soft_delete_retention_days: 90          # Must match existing vault if already created
  public_network_access_enabled: true     # Set false once stable; keep true during bootstrap
  network_acls_ip_rules:
    - "<your_public_ip>/32"              # Your laptop or CI runner IP

# ── Core services ─────────────────────────────────────────────────────────────
law:
  create: true

psql:
  create: true
  name: "sgpaz<workspace_id>postgres"    # Must be globally unique
  admin_user: postgres
  bootstrap_aad_principals: true         # Runs once after cluster is up; requires kubectl access

redis:
  create: true
  name: "sgpaz<workspace_id>redis"       # Must be globally unique

storage_account:
  create: true
  name: "sgpaz<workspace_id>storage"    # Must be globally unique; no hyphens
  enable_file_private_endpoint: true

service_bus:
  create: true
  name: "sgpaz<workspace_id>servicebus"
  sku: Premium                            # Premium required for private endpoints
  capacity: 1
  premium_messaging_partitions: 1
  private_endpoint_enabled: true
  public_network_access: false

# ── AI services ───────────────────────────────────────────────────────────────
# Modes: use_openai_via_azure | use_openai_via_custom_endpoint | no_openai
# NOTE: Do not commit real API keys. Keep `key` as a local uncommitted change.
open_ai:
  create: false
  name: "sgpaz<workspace_id>openai"
  client:
    mode: "use_openai_via_custom_endpoint"
    key: "<openai_api_key>"
    custom_url: "<your_azure_openai_endpoint>"

ai_search:
  create: false
  name: "sgpaz<workspace_id>aisearch"

# ── AKS ───────────────────────────────────────────────────────────────────────
aks:
  create: true
  name: "sgpaz<workspace_id>aks"
  node_resource_group: "rg-aks-eus2-<domain_code>-<workspace_id>-dev-01"
  dns_prefix: "egp-k8scluster"
  sku: Free                               # Free | Standard (use Standard for production)
  kubernetes_version: "1.30"
  run_command_enabled: true
  enable_encryption_at_host: true
  use_azure_managed_flux: true
  zones: ["1", "2", "3"]
  istio:
    enabled: true
    ingress_mode: External
    revisions:
      - asm-1-20
  node_pools:
    system:
      name: default
      vm_size: Standard_D4s_v3
      count: 3
      min_count: 3
      max_count: 6
    user:
      enabled: true
      name: user
      vm_size: Standard_D16s_v3
      count: 3
      min_count: 3
      max_count: 10
    gpu:
      enabled: false                      # Set true to enable GPU workloads
      name: gpu
      vm_size: Standard_NV72ads_A10_v5
      min_count: 0
      max_count: 5
    cassandra:
      enabled: false                      # Required only if temporal_db_mode is cassandraK8s
  network:
    pod_cidr: "10.244.0.0/16"
    service_cidr: "10.243.0.0/16"
    dns_service_ip: "10.243.0.10"
    outbound_type: userDefinedRouting

# ── Bastion / jump host ───────────────────────────────────────────────────────
bastion:
  create: true
  admin_username: azureuser
  ssh_public_key: "<your_ssh_public_key>"

# ── Front Door overrides ──────────────────────────────────────────────────────
# Most Front Door settings inherit from default.yaml. Override only what differs.
frontdoor:
  extra_subdomains: []    # e.g. ["chat"] if Agentex UI is enabled

# ── Feature flags ─────────────────────────────────────────────────────────────
feature_flags:
  models: false
  agentex:
    create: false    # Set true to provision Agentex service
  compass:
    create: false    # Set true to provision Workflows service
  dex:
    create: false    # Set true to provision Dex (Document Understanding) service
  reducto:
    create: false    # Set true to provision Reducto service

# ── Bootstrapping (System Manager) ────────────────────────────────────────────
bootstrapping:
  enabled: true
  use_managed_flux: true
  system_manager_version: "<system_manager_image_tag>"
  desiredState: |
    {
      "version": "0.1",
      "packs": [
        { "name": "cert-manager" },
        { "name": "egp" },
        { "name": "identity-service" },
        { "name": "spicedb" },
        { "name": "sgp-apps" },
        { "name": "sgp-models" }
      ]
    }

# ── Policy assignments ────────────────────────────────────────────────────────
policy_assignments:
  enabled: false    # Set true to enforce Azure Policy tag rules

# ── Observability ─────────────────────────────────────────────────────────────
observiqidp:
  monitoring:
    enabled: false
  insights:
    enabled: false
```

<Note>
  **Naming constraints for Azure resources:**

  * Key Vault names: 3–24 alphanumeric characters and hyphens, globally unique
  * Storage Account names: 3–24 lowercase letters and numbers only (no hyphens), globally unique
  * PostgreSQL and Redis names: globally unique within Azure
  * All names must remain stable after first apply — many Azure resources cannot be renamed
</Note>

### Step 2: Provision Infrastructure via Terraform

Navigate to the `infra` directory and initialize:

```bash theme={null}
cd infra

terraform init
```

Review and apply the plan using the `main.tfvars.json` file you configured in the previous step:

```bash theme={null}
terraform plan -var-file=main.tfvars.json -out=tfplan

# Inspect the plan to review resources before proceeding
terraform show tfplan

terraform apply tfplan
```

If the Datadog monitoring integration is enabled in your YAML, pass the API key via environment variable rather than committing it to file:

```bash theme={null}
TF_VAR_datadog_api_key="<your_key>" terraform plan -var-file=main.tfvars.json -out=tfplan
```

*This step may take significant time (30–60 minutes) due to resource creation dependencies, particularly the AKS cluster and PostgreSQL Flexible Server.*

### Step 3: Bootstrap the Cluster

The Azure infrastructure **automatically bootstraps** [SGP System Manager](/docs/infrastructure/system-manager) via an Azure Deployment Script. When `bootstrapping.enabled: true` is set in your configuration, Terraform provisions an Azure Container Instance that:

1. Installs Flux CD on the AKS cluster (using Azure Managed Flux if `use_managed_flux: true`)
2. Applies the System Manager `HelmRepository` and `HelmRelease` Flux CRDs
3. Waits for System Manager to reconcile

Monitor bootstrap progress in the Azure Portal under **Deployment Scripts** in your resource group, or check System Manager logs after bootstrap:

```bash theme={null}
# Get AKS credentials (from inside the VNet or via Bastion — cluster is private by default)
az aks get-credentials \
  --resource-group <resource_group> \
  --name sgpaz<workspace_id>aks \
  --overwrite-existing

# Verify System Manager is running
kubectl get pods -n sgp-system-manager

# Watch System Manager deploy remaining services via Flux
kubectl get helmreleases -A
```

#### Accessing the Private AKS Cluster

Because the AKS cluster is private by default, you must access it from within the provisioned VNet. Two options are provided:

**Option A: From the jump host VM (via Azure Bastion)**

The Bastion host and jump host VM are provisioned when `bastion.create: true`. Connect via the Azure Portal (Bastion blade) or using the helper script:

```bash theme={null}
python3 scripts/connect_private_aks.py --run-mode=local --bootstrap-jump-host
```

**Option B: Using `az aks command invoke`**

When `aks.run_command_enabled: true`, you can run kubectl commands without VPN access:

```bash theme={null}
az aks command invoke \
  --resource-group <resource_group> \
  --name sgpaz<workspace_id>aks \
  --command "kubectl get helmreleases -A"
```

### Step 4: Configure DNS

After `terraform apply` completes, retrieve the Front Door endpoint hostname:

```bash theme={null}
terraform output frontdoor_endpoint_host_name
terraform output frontdoor_dns_zone_id
```

Configure a CNAME record in your DNS provider pointing your `deployment_url` to the Front Door endpoint (the `azurefd.net` hostname from the output above).

<Note>
  If using Azure DNS (the DNS zone is managed by Terraform when `frontdoor.create: true`), CNAME records are created automatically. Verify with:

  ```bash theme={null}
  az network dns record-set list \
    --resource-group <resource_group> \
    --zone-name <deployment_url>
  ```
</Note>

### Step 5: Verify the Deployment

Wait for all services to be ready:

```bash theme={null}
kubectl get helmreleases -A  # All should show Ready=True
kubectl get pods -A          # All should be Running or Completed
```

System Manager continuously reconciles the desired state. The `bootstrapping.desiredState` value in your environment YAML is written to a secret in Azure Key Vault during `terraform apply`, and System Manager reads from that secret at runtime. If a HelmRelease shows `Ready=False`, check its events:

```bash theme={null}
kubectl describe helmrelease <name> -n <namespace>
```

### Step 6: Configure Identity Provider

#### SAML Configuration

Set `authType: "SAML"` in your environment YAML, then configure your Identity Provider with:

* **Service Entity ID**: `https://auth.<deployment_url>`
* **Redirect URI**: `https://auth.<deployment_url>/dashboard/org/saml/callback`

Update the `is-saml-secret` secret in Key Vault (or via System Manager GUI):

```json theme={null}
{
  "id": "<workspace_id>",
  "samlConfiguration": {
    "entityId": "<entity_id_from_your_identity_provider>",
    "x509Cert": "<x509_certificate_from_your_identity_provider>",
    "ssoUrl": "<sso_url_from_your_identity_provider>",
    "attributeMappings": {
      "email": "<email_attribute_name>",
      "firstName": "<first_name_attribute_name>",
      "lastName": "<last_name_attribute_name>"
    }
  }
}
```

#### OIDC Configuration

Set `authType: "OIDC"` in your environment YAML, then configure your Identity Provider with:

* **Redirect URI**: `https://auth.<deployment_url>/dashboard/org/oidc/callback`

Update the `is-oidc-secret` secret in Key Vault (or via System Manager GUI):

```json theme={null}
{
  "id": "<workspace_id>",
  "oidcConfiguration": {
    "clientId": "<client_id_from_your_identity_provider>",
    "clientSecret": "<client_secret_from_your_identity_provider>",
    "issuer": "<issuer_from_your_identity_provider>",
    "authorizationUrl": "<authorization_url>",
    "tokenUrl": "<token_url>",
    "userInfoUrl": "<user_info_url>"
  }
}
```

After modifying an identity secret, restart System Manager to apply the changes:

```bash theme={null}
kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager
```

## Accessing the Platform

If all goes smoothly, you should be able to navigate to the SGP platform at `https://<workspace_id>.workspace.egp.scale.com` (or your custom domain) and authenticate via the configured identity provider.


# GCP Architecture Reference
Source: https://docs.gp.scale.com/docs/infrastructure/gcp/gcp-architecture-reference

Overview of the SGP GCP architecture

## Architecture Diagram

```mermaid theme={null}
graph TD
    USER(["Client"])
    ENGINEER(["Engineer"])

    subgraph GCP["GCP Project"]
        GLB["Global Load Balancer<br/>Static IP · TLS 1.2+"]
        IAP["Cloud IAP"]

        subgraph VPC["VPC — 10.0.0.0/16"]
            INGRESS["Istio Ingress"]

            subgraph GKE["GKE Cluster"]
                SYS["System Pool<br/>3× n2-standard-4"]
                CPU["CPU Pool<br/>4–30× n2-standard-16"]
            end

            BASTION["Bastion VM<br/>e2-micro"]
        end

        subgraph PEERED["VPC-Peered — Cloud SQL"]
            PG_MAIN[("Cloud SQL<br/>PostgreSQL 17 (main)")]
            PG_TEMP[("Cloud SQL<br/>PostgreSQL 17 (temporal)")]
        end

        SM["Secret Manager"]

        subgraph GCS_GROUP["Cloud Storage"]
            BUCKET[("GCS — platform")]
            KB[("GCS — knowledge base")]
        end

        AR["Artifact Registry<br/>Docker + Helm"]
        KMS["Cloud KMS<br/>CMEK (optional)"]
    end

    USER --> GLB
    GLB --> INGRESS
    INGRESS --> CPU
    CPU -->|"VPC Peering · SSL enforced"| PG_MAIN
    CPU -->|"VPC Peering · SSL enforced"| PG_TEMP
    CPU -->|"Workload Identity"| SM
    CPU -->|"Workload Identity"| BUCKET
    CPU -->|"Workload Identity"| KB
    AR -->|"Workload Identity"| CPU
    KMS -.->|"encrypts"| PG_MAIN
    KMS -.->|"encrypts"| PG_TEMP
    KMS -.->|"encrypts"| BUCKET
    ENGINEER --> IAP --> BASTION -->|"private API"| GKE
```

***

## Terraform Structure

SGP's GCP infrastructure is provisioned in two separate Terraform phases with different privilege levels:

```mermaid theme={null}
graph LR
    CREDS(["Personal GCP<br/>Credentials"])

    subgraph PS["projectsetup/ — run once with personal credentials"]
        SA["Terraform<br/>Service Account"]
        AR_PS["Artifact Registries<br/>Docker + Helm"]
        SMK["SA Key stored in<br/>Secret Manager"]
    end

    subgraph DEP["deployments/name/ — SA credentials auto-loaded from Secret Manager"]
        GKE_D["GKE Cluster"]
        SQL_D["Cloud SQL × 2"]
        VPC_D["VPC + Subnets"]
        SEC_D["Secrets + IAM<br/>+ Workload Identity"]
    end

    CREDS -->|"terraform apply"| PS
    SMK -->|"credentials<br/>auto-read at plan time"| DEP
```

| Phase                   | Directory             | Credentials                                          | Scope                                                |
| ----------------------- | --------------------- | ---------------------------------------------------- | ---------------------------------------------------- |
| Privileged bootstrap    | `projectsetup/`       | Your personal GCP identity                           | Service account, Artifact Registries, API enablement |
| Deprivileged main infra | `deployments/<name>/` | Terraform service account (read from Secret Manager) | GKE, Cloud SQL, networking, IAM, secrets             |

The main infrastructure Terraform reads the service account key directly from Secret Manager — no key file needs to exist on disk during the infrastructure run.

***

## Resources by Type

### Compute Resources

| Resource                     | Count | Purpose                                    |
| ---------------------------- | ----- | ------------------------------------------ |
| GKE Cluster                  | 1     | Kubernetes orchestration                   |
| System Node Pool (`default`) | 1     | System pods — tainted `CriticalAddonsOnly` |
| CPU Node Pool (`cpu`)        | 1     | Application workloads                      |
| GPU Node Pool (`gpu`)        | 0–1   | AI/ML workloads (optional)                 |
| Cassandra Node Pool          | 0–1   | Temporal database (optional)               |
| Bastion Host                 | 0–1   | Private cluster access via IAP (optional)  |

**Default node pool sizing:**

| Pool               | Machine Type     | Min Nodes | Max Nodes               |
| ------------------ | ---------------- | --------- | ----------------------- |
| System (`default`) | `n2-standard-4`  | 3         | 10                      |
| CPU (`cpu`)        | `n2-standard-16` | 4         | 30                      |
| GPU (`gpu`)        | `a2-highgpu-1g`  | 0         | 0 (disabled by default) |
| Cassandra          | `n2-standard-4`  | 3         | 6                       |

All pools use `COS_CONTAINERD` image type and are preemptible by default (configurable via `node_pool_config.preemptible`).

***

### Network Resources

| Resource            | Count | Purpose                             |
| ------------------- | ----- | ----------------------------------- |
| VPC Network         | 1     | Network boundary                    |
| Subnetwork          | 1     | Kubernetes nodes                    |
| Secondary IP Ranges | 2     | GKE pod and service CIDRs           |
| Cloud DNS Zone      | 1     | Internal + external name resolution |
| Global Static IP    | 1     | Load balancer ingress endpoint      |
| SSL Policy          | 1     | Minimum TLS 1.2 enforcement         |
| Firewall Rules      | 4–5   | Traffic control (offline mode)      |
| VPC Peering         | 1     | Private connectivity to Cloud SQL   |
| Private Route       | 0–1   | Google APIs access in offline mode  |

***

### Data & Storage Resources

| Resource                      | Count | Purpose                       |
| ----------------------------- | ----- | ----------------------------- |
| Cloud SQL (PostgreSQL 17)     | 1     | Main platform database        |
| Cloud SQL (PostgreSQL 17)     | 1     | Temporal workflow database    |
| GCS Bucket (main)             | 1     | Platform object storage       |
| GCS Bucket (knowledge base)   | 1     | KB document storage           |
| GCS Bucket (monitoring)       | 0–1   | Observability data (optional) |
| Vertex AI Vector Search Index | 0–N   | Vector embeddings (optional)  |
| Cloud Firestore               | 0–1   | Agentex state (optional)      |

Both Cloud SQL instances are private-only (no public IP) and connected to the VPC via VPC peering. SSL is enforced for all database connections (`ENCRYPTED_ONLY`).

***

### Security Resources

| Resource                                                 | Count | Purpose                                     |
| -------------------------------------------------------- | ----- | ------------------------------------------- |
| Secret Manager Secrets                                   | 10+   | Platform configuration and credentials      |
| Cloud KMS Key Ring + Key                                 | 0–1   | Customer Managed Encryption Keys (optional) |
| Service Account (`sgp-<workspace_id>-sa`)                | 1     | Main SGP workload identity SA               |
| Service Account (`sgp-<workspace_id>-node-pool-creator`) | 1     | Node pool creation SA                       |
| Service Account (`sgp-tf-lp-<name>`)                     | 1     | Terraform execution SA (projectsetup)       |
| Workload Identity Pool                                   | 1     | GKE pod → GCP SA binding                    |
| IAP Tunnel (Bastion)                                     | 0–1   | Private cluster access                      |

***

### Artifact Resources (projectsetup phase)

| Resource                   | Count | Purpose              |
| -------------------------- | ----- | -------------------- |
| Artifact Registry (Docker) | 1     | SGP container images |
| Artifact Registry (Helm)   | 1     | SGP Helm charts      |

Both registries are named `sgp-<workspace_id>-docker-repository` and `sgp-<workspace_id>-helm-repository`.

***

### Monitoring Resources

| Resource                   | Count | Purpose                                      |
| -------------------------- | ----- | -------------------------------------------- |
| Cloud Logging              | 1     | GKE system components, API server, workloads |
| VPC Flow Logs              | 0–1   | Network traffic sampling (optional)          |
| GKE Vulnerability Scanning | 1     | Basic vulnerability mode enabled by default  |

***

## Network Architecture

### Address Space

```
VPC: sgp-<workspace_id>-network
└── Subnet: sgp-<workspace_id>-network-kubernetes-subnet
    ├── Primary range:   10.0.0.0/16   (65,536 IPs — GKE nodes)
    ├── Secondary range: 10.2.0.0/16   (65,536 IPs — Kubernetes services)
    └── Secondary range: 10.4.0.0/16   (65,536 IPs — Kubernetes pods)

Cloud SQL VPC Peering:
    └── Reserved range:  /16 block      (managed by service networking)

GKE Control Plane (private):
    └── Master CIDR:     10.5.0.0/28   (configurable via private_gke_master_ipv4_cidr_block)
```

### Traffic Flow

**Ingress (External):**

```
Internet → Global Load Balancer (static IP) → Istio Ingress → Pod
(TLS terminated at load balancer; SSL policy enforces TLS 1.2+)
```

**Database Access:**

```
GKE Pod → VPC Peering → Cloud SQL (private IP)
(SSL enforced; no public IP on Cloud SQL instances)
```

**Google APIs (offline mode):**

```
GKE Pod → Private Google Access → Google APIs (199.36.153.4/30)
(Dedicated route; no default internet gateway route created)
```

**Bastion Access:**

```
Engineer → IAP Tunnel → Bastion VM (e2-micro) → GKE API (private endpoint)
```

***

## GKE Configuration

### Cluster Features

| Feature                    | Value                                        |
| -------------------------- | -------------------------------------------- |
| Datapath provider          | `ADVANCED_DATAPATH` (eBPF-based)             |
| IP stack                   | Dual-stack IPv4/IPv6                         |
| Workload Identity          | Enabled (`<project>.svc.id.goog`)            |
| Secret Manager integration | Enabled                                      |
| Vulnerability scanning     | Basic mode                                   |
| Private nodes              | Enabled (when `offline_mode = true`)         |
| Private endpoint           | Configurable (`enable_gke_private_endpoint`) |
| Master authorized networks | Configurable per deployment                  |
| DNS endpoint               | Enabled (allows external cluster DNS access) |

### Workload Identity

GKE pods authenticate to GCP services using Workload Identity rather than node-level service account keys. Kubernetes service accounts are bound to GCP service accounts via the workload identity pool:

```
<project>.svc.id.goog[<namespace>/<k8s-service-account>]
  → GCP Service Account
    → Secret Manager, GCS, Artifact Registry, etc.
```

Key bindings provisioned by Terraform:

| Kubernetes Identity                             | GCP Role                                                        |
| ----------------------------------------------- | --------------------------------------------------------------- |
| `sgp-system-manager` (system-manager namespace) | `secretmanager.viewer`, `secretmanager.secretAccessor`          |
| `sgp-system-manager-pre-install`                | `secretmanager.viewer`, `secretmanager.secretAccessor`          |
| `egp-api-backend` (egp namespace)               | `secretmanager.secretAccessor`, GCS access via SA impersonation |
| `egp-api-backend-db-setup`                      | `secretmanager.secretAccessor`                                  |
| `agents` (agents namespace)                     | `secretmanager.secretAccessor`                                  |

***

## Secret Manager Secrets

Key secrets provisioned by Terraform:

| Secret Name                             | Contents                                                                       |
| --------------------------------------- | ------------------------------------------------------------------------------ |
| `<prefix>-system-manager-config`        | System Manager runtime configuration (URLs, workspace ID, cloud provider info) |
| `<prefix>-system-manager-desired-state` | Initial desired state (pack list) — managed externally after first apply       |
| `<prefix>-saml-config-secret`           | SAML IdP configuration                                                         |
| `<prefix>-oidc-config-secret`           | OIDC IdP configuration                                                         |
| `terraform-service-account-key-secret`  | Terraform SA private key (used by `deployments/` as provider credentials)      |

The `secret_name_prefix` variable (typically `sgp-<workspace_id>`) scopes secrets for deployments in shared GCP projects.

***

## Customer Managed Encryption Keys (CMEK)

When `useCustomerManagedEncryptionKey = true`, Terraform provisions a Cloud KMS key ring and symmetric encryption key:

| Resource   | Name                            |
| ---------- | ------------------------------- |
| Key Ring   | `sgp-<workspace_id>-key-ring`   |
| Crypto Key | `sgp-<workspace_id>-crypto-key` |

The key is applied to:

* Cloud SQL instances (both main and Temporal)
* GCS buckets

CMEK is recommended for production deployments to maintain cryptographic control over data at rest.

***

## Optional Capabilities

Each optional capability provisions dedicated infrastructure:

| Capability                   | Variable                           | Additional Resources                                                        |
| ---------------------------- | ---------------------------------- | --------------------------------------------------------------------------- |
| Agentex                      | `deployAgentex`                    | Firestore, dedicated service account, GCS bucket                            |
| Workflows                    | `deployCompass`                    | Cloud SQL database, GCS bucket, service account                             |
| Dex (Document Understanding) | `enable_dex`                       | Cloud SQL (`db-custom-4-15360`), GCS bucket, service account                |
| Reducto                      | `enable_reducto`                   | Cloud SQL (`db-custom-2-7680`), GCS bucket, Vision API key, service account |
| Model Engine                 | `deployModelEngine`                | Vertex AI, additional node pools                                            |
| Monitoring                   | `enable_monitoring`                | GCS bucket, IAM bindings                                                    |
| Cloud Build                  | `enable_cloud_build`               | GCS bucket, Artifact Registry, service accounts                             |
| Vertex AI Search             | `vertex_ai_indices`                | Vector Search indices and endpoints                                         |
| LiveKit                      | `provision_livekit_infrastructure` | GCS bucket (audio), IAM bindings                                            |


# GCP SGP Deployment
Source: https://docs.gp.scale.com/docs/infrastructure/gcp/gcp-sgp-deployment

End-to-end guide to deploying SGP in a GCP project.

## Overview

This guide walks you through deploying SGP in a GCP project using the SGP GCP Terraform modules. SGP GCP infrastructure is defined by terraform modules managed by Scale.

GCP deployment is split into two Terraform phases with different privilege levels:

* **`projectsetup/`** — Run once with privileged GCP credentials. Creates a dedicated Terraform service account (with least privileges), enables required GCP APIs, and provisions Artifact Registries for SGP images and Helm charts.
* **`deployments/<your-deployment>/`** — Run with the service account created above. Provisions the GKE cluster, Cloud SQL, networking, and all SGP infrastructure.

This split ensures the main infrastructure Terraform never requires broad privileged credentials.

## Prerequisites

* Access to a GCP project with permissions to manage IAM, APIs, and compute resources
* The following tools installed:
  * [gcloud CLI](https://cloud.google.com/sdk/docs/install) (`gcloud auth login` completed)
  * [Terraform](https://www.terraform.io/) (`>= 1.1.7`)
  * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
  * [helm](https://helm.sh/docs/intro/install/) (for cluster bootstrap)
  * [crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane) or [Docker](https://docs.docker.com/engine/install/) (for copying images)
  * \[sgpctl] ([https://github.com/scaleapi/sgpctl](https://github.com/scaleapi/sgpctl)) (cli tool to execute system manager related commands)
* The following from Scale:
  * The SGP GCP Infrastructure modules (`gcp/`)
  * A `workspace_id` (8-digit number) and `registration_secret` unique to your deployment
* A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional)
* A custom domain for your deployment (optional)

## Installation

### Step 1: Bootstrap the Project

Navigate to the `projectsetup/` directory. This phase creates the Terraform service account and Artifact Registries.

Edit the `locals` block at the top of `projectsetup/main.tf`:

```hcl theme={null}
locals {
  project              = "<your-gcp-project-id>"
  region               = "<gcp-region>"         # e.g. "us-east1"
  zone                 = "<gcp-zone>"            # e.g. "us-east1-a"
  service_account_name = "<workspace_id>-sa"    # Suffix for the service account name
  workspaceID          = "<workspace_id>"        # 8-digit number from Scale, e.g. "90000001"
}
```

Then initialize and apply:

```bash theme={null}
cd projectsetup

terraform init
terraform apply
```

This creates:

* A Terraform service account (`sgp-tf-lp-<service_account_name>`) with the IAM roles needed to provision SGP infrastructure
* A service account key stored as `terraform-service-account-key-secret` in Secret Manager
* Docker and Helm Artifact Registries (`sgp-<workspace_id>-docker-repository`, `sgp-<workspace_id>-helm-repository`)
* All required GCP APIs enabled on the project

The Terraform outputs include the `service_account_email` of the newly created service account, which is useful for auditing. The service account key is automatically stored in Secret Manager and read by the main infrastructure Terraform — no manual key management is required.

### Step 2: Copy SGP Images and Helm Charts

*This step is only required if your deployment is configured to block internet access from the cluster (`offline_mode` in step 3).*

Before provisioning the main infrastructure, copy SGP's Docker images and Helm charts from Scale's registry into your GCP Artifact Registry.

**Download the manifest from Scale's control plane**:

```bash theme={null}
sgpctl manifest download \
  --control-plane-id <workspace_id> \
  --registration-token <token-from-scale-engineer> \
  --version <consolidated-version> \
  --channel <release|dev> \
  --output <version>.manifest.yaml
```

Note: sgpctl manifest download requires internet access to [https://control.egp.scale.com](https://control.egp.scale.com), so it must be run from a connected machine.
Users with air-gapped operator workstations should run it from their laptop and SCP the output to wherever they'll run the mirror commands.

**Authenticate to source + destination registries**:

```bash theme={null}
# Source: Scale's ECR (credentials from Scale engineer)
crane auth login 022465994601.dkr.ecr.us-west-2.amazonaws.com \
  --username AWS \
  --password-stdin

# Destination: customer's GCP Artifact Registry (uses gcloud creds)
gcloud auth configure-docker <region>-docker.pkg.dev
```

**Mirror images and charts**:

```bash theme={null}
# Container images
sgpctl mirror-manifest \
  --manifest-file <version>.manifest.yaml \
  --dest-docker-registry <region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-docker-repository

# Helm charts
sgpctl mirror-helm-charts \
  --manifest-file <version>.manifest.yaml \
  --dest-helm-repo oci://<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-helm-repository
```

**Verify the mirror**:

```bash theme={null}
gcloud artifacts docker images list \
  <region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-docker-repository \
  --include-tags --limit=20

gcloud artifacts docker images list \
  <region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-helm-repository \
  --include-tags --limit=20
```

### Step 3: Configure the Deployment

Copy the `deployments/prototype/` directory and rename it for your deployment:

```bash theme={null}
cp -r deployments/prototype deployments/<your-deployment-name>
cd deployments/<your-deployment-name>
```

Edit the `locals` block at the top of `main.tf`. The deployment reads its Terraform service account credentials automatically from Secret Manager — no key file is required on disk.

```hcl theme={null}
locals {
  project  = "<your-gcp-project-id>"
  region   = "<gcp-region>"                                               # e.g. "us-east1"
  zone     = "<gcp-zone>"                                                 # e.g. "us-east1-b"

  # ── Deployment identity ────────────────────────────────────────────────
  workspaceID        = "<workspace_id>"                                   # 8-digit number from Scale
  registrationSecret = "<registration_secret>"                            # From Scale
  deploymentURL      = "<workspace_id>.workspace.egp.scale.com"          # Or your custom domain

  # ── Repositories ──────────────────────────────────────────────────────
  # baseRepository: used by workloads running inside the cluster to pull images
  # publicBaseRepository: used for initial image pulls during bootstrap (before private DNS resolves)
  # Both point to the same registry when using a private Artifact Registry.
  baseRepository       = "<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-docker-repository"
  publicBaseRepository = "<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-docker-repository"

  # ── System Manager ────────────────────────────────────────────────────
  systemManagerImageTag = "<system_manager_image_tag>"  # From Scale; must match manifest.yaml
  offline_mode          = true                          # true when using your own Artifact Registry

  # ── Auth ──────────────────────────────────────────────────────────────
  authType = "SAML"   # "SAML" or "OIDC"

  # ── Bootstrap ─────────────────────────────────────────────────────────
  # deploy_system_manager=false: Terraform creates infra only; run manual-helm-install.sh to bootstrap
  # deploy_system_manager=true:  Terraform also installs System Manager into the cluster
  deploy_system_manager = false

  # ── DNS ───────────────────────────────────────────────────────────────
  createDNSRecords = false   # Set true if Terraform should manage Cloud DNS records

  # ── Encryption ────────────────────────────────────────────────────────
  useCustomerManagedEncryptionKey = false   # Set true for CMEK (recommended for production)

  # ── Networking ────────────────────────────────────────────────────────
  gke_config = {
    private_endpoint = true
    master_authorized_networks = [
      { cidr_block = "10.0.0.0/16", display_name = "cluster vpc primary range" },
      { cidr_block = "10.2.0.0/16", display_name = "cluster vpc services range" },
      { cidr_block = "10.4.0.0/16", display_name = "cluster vpc pod range" },
    ]
  }
}
```

**If using SAML,** place your IdP's x509 certificate (without BEGIN/END lines) as `x509.cer` in the deployment directory. Update the `samlConfigSecret` in the `module "sgp"` block:

```hcl theme={null}
module "sgp" {
  source = "../../modules/sgp/"
  samlConfigSecret = jsonencode({
    "id" = local.workspaceID
    "samlConfiguration" = {
      "entityId" = "https://auth.${local.deploymentURL}"
      "x509Cert" = file("${path.module}/x509.cer")
      "ssoUrl"   = "<your-idp-sso-url>"
      "attributeMappings" = {
        "email"     = "<email-attribute-from-idp>"
        "firstName" = "<first-name-attribute-from-idp>"
        "lastName"  = "<last-name-attribute-from-idp>"
      }
    }
  })
  # ... other variables
}
```

**If using OIDC,** update `oidcConfigSecret` instead:

```hcl theme={null}
oidcConfigSecret = jsonencode({
  "id"               = local.workspaceID
  "clientId"         = "<client-id-from-idp>"
  "clientSecret"     = "<client-secret-from-idp>"
  "issuer"           = "<issuer-url>"
  "authorizationUrl" = "<authorization-url>"
  "tokenUrl"         = "<token-url>"
  "userInfoUrl"      = "<user-info-url>"
})
```

### Step 4: Provision Infrastructure via Terraform

From the deployment directory, initialize and apply:

```bash theme={null}
terraform init -upgrade

terraform plan -out=tfplan

# Inspect the plan before proceeding
terraform show tfplan

terraform apply tfplan
```

*This step can take 30–60 minutes due to GKE cluster provisioning and Cloud SQL setup.*

After apply completes, connect to the cluster:

```bash theme={null}
gcloud container clusters get-credentials sgp-<workspace_id>-kubernetes-cluster \
  --region <gcp-region> \
  --project <your-gcp-project-id>
```

<Note>
  If `gke_config.private_endpoint = true`, the cluster API server is only accessible from within the VPC. Use the provisioned bastion host (via IAP) or a network-connected runner to access it. Set `bastion_enabled = true` and add your user email to `bastion_iap_members` in the `security_compliance` block to enable bastion access.
</Note>

### Step 5: Bootstrap the Cluster

[SGP System Manager](/docs/infrastructure/system-manager) orchestrates SGP service deployment. If you set `deploy_system_manager = false` in the previous step, bootstrap it manually using the provided script.

Edit `manual-helm-install.sh` in your deployment directory with the correct values:

```bash theme={null}
PROJECT_ID="<your-gcp-project-id>"
PROJECT_NUMBER="<your-gcp-project-number>"    # From: gcloud projects describe <project-id> --format='value(projectNumber)'
BASE_REPOSITORY="<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-docker-repository"
SYSTEM_MANAGER_IMAGE_TAG="<system_manager_image_tag>"
OFFLINE_MODE=true

# Obtain from Cloud SQL instance → "Connect to this instance" in the console, or from Terraform state
SPICEDB_IP_ADDRESS="<cloud-sql-private-ip>"
SPICEDB_PASSWORD="<spicedb-db-password>"
TEMPORAL_DB_PASSWORD="<temporal-db-password>"
```

Then run the bootstrap script:

```bash theme={null}
chmod +x manual-helm-install.sh
./manual-helm-install.sh
```

This installs System Manager into the cluster. System Manager will then begin reconciling the `desired_state.json` file and deploying SGP services automatically.

#### Desired State

The `desired_state.json` file in your deployment directory defines which SGP packs System Manager installs. Update it to reference your GCP Artifact Registry:

```json theme={null}
{
  "version": "0.1",
  "packs": [
    { "name": "flux" },
    {
      "name": "sgp-helm-repository",
      "properties": {
        "helm-repo": {
          "url": "oci://<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-helm-repository"
        }
      }
    },
    { "name": "istio" },
    {
      "name": "sgp-base",
      "properties": {
        "helm-base": {
          "value_overrides": {
            "refreshRegcred": { "enabled": false },
            "gcp": {
              "presharedCertificates": ["sgp-<workspace_id>-ssl-certificate"]
            }
          }
        }
      }
    },
    { "name": "spicedb" },
    { "name": "identity-service" },
    { "name": "temporalf" },
    { "name": "sgp-apps" }
  ]
}
```

See [Step 6](#step-6-configure-ssl-certificates) for the SSL certificate options and how to set the correct value in the `gcp` block above.

### Step 6: Configure SSL Certificates

Three options are available, in order of preference:

#### Option 1: Google Certificate Manager (Recommended)

Provisions a wildcard certificate (`*.your-domain.com`) via DNS authorization. Covers all subdomains automatically.

Enable it in `main.tf`:

```hcl theme={null}
module "sgp" {
  source = "../../modules/sgp/"
  enable_certificate_manager = true
  # ...
}
```

After `terraform apply`, note the two outputs:

* `certificate_manager_dns_auth_record` — a CNAME you must add to your DNS zone for domain validation
* `certificate_map_name` — typically `sgp-<workspace_id>-cert-map`

Add the CNAME to your DNS provider, then update `desired_state.json`:

```json theme={null}
{
  "name": "sgp-base",
  "properties": {
    "helm-base": {
      "value_overrides": {
        "gcp": {
          "certificateMapName": "sgp-<workspace_id>-cert-map"
        }
      }
    }
  }
}
```

The certificate typically provisions within 10–60 minutes of the DNS record being in place.

#### Option 2: Google Managed Certificate

Google provisions and auto-renews per-subdomain certificates. Does not require uploading a certificate, but requires DNS to resolve to the load balancer IP before provisioning, and does not support wildcards.

This is the default when neither `certificateMapName` nor `presharedCertificates` is set in the `sgp-base` pack's `gcp` block. Simply omit those keys from `desired_state.json`.

#### Option 3: Preshared Certificate

Use a certificate you manage and upload to Google Cloud. Required when DNS is not publicly resolvable or Certificate Manager is not available.

Upload the certificate:

```bash theme={null}
gcloud compute ssl-certificates create sgp-<workspace_id>-ssl-certificate \
  --certificate=fullchain1.pem \
  --private-key=privkey1.pem \
  --project <your-gcp-project-id>
```

Reference it in `desired_state.json`:

```json theme={null}
{
  "name": "sgp-base",
  "properties": {
    "helm-base": {
      "value_overrides": {
        "gcp": {
          "presharedCertificates": ["sgp-<workspace_id>-ssl-certificate"]
        }
      }
    }
  }
}
```

<Warning>
  Preshared certificates must be renewed manually. Let's Encrypt certificates expire after 90 days.
</Warning>

After changing the SSL configuration in `desired_state.json`, restart System Manager to apply it:

```bash theme={null}
kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager
```

### Step 7: Configure DNS

Get the external IP address of the load balancer:

```bash theme={null}
# From inside the cluster:
kubectl get svc istio-ingress --namespace istio-ingress -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

# Or from gcloud (the global address is created by Terraform):
gcloud compute addresses describe sgp-<workspace_id>-external-ip \
  --global \
  --project <your-gcp-project-id> \
  --format="value(address)"
```

Create DNS A records in your DNS provider pointing to this IP for:

* `<deployment_url>` (apex)
* `api.<deployment_url>`
* `auth.<deployment_url>`
* `app.<deployment_url>`

If `createDNSRecords = true` in your `main.tf` locals, Terraform manages a Cloud DNS zone and creates these records automatically. Retrieve the name servers from the Terraform output:

```bash theme={null}
terraform output name_servers
```

Then delegate the zone by configuring these as NS records at your domain registrar.

### Step 8: Verify the Deployment

Wait for all services to be ready:

```bash theme={null}
kubectl get helmreleases -A  # All should show Ready=True
kubectl get pods -A          # All should be Running or Completed
```

System Manager continuously reconciles the desired state. If a HelmRelease shows `Ready=False`, check its events:

```bash theme={null}
kubectl describe helmrelease <name> -n <namespace>
```

## Accessing the Platform

Once all HelmReleases are ready and DNS resolves correctly, navigate to `https://<workspace_id>.workspace.egp.scale.com` (or your custom domain) and authenticate via your configured identity provider.


# System Manager
Source: https://docs.gp.scale.com/docs/infrastructure/system-manager

Overview of the SGP System Manager deployment orchestrator

## Overview

System Manager is a Kubernetes operator used to deploy and manage the SGP platform. It is responsible for deploying the SGP platform services and agents to an existing Kubernetes cluster.

## Installation

System Manager is installed as a Helm chart into the Kubernetes cluster during the deployment of the SGP platform. See your cloud provider's corresponding deployment guide for more information.

## Configuration

System Manager is configured via `system-manager-config.json`. This file is stored in the cloud provider's secret manager. To modify the configuration, you can either update the secret directly or use the System Manager GUI, then restart the System Manager deployment to apply the changes.

```bash theme={null}
kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager
```

An example configuration file is shown below. The `aws` block is only present for AWS deployments; GCP and Azure deployments use equivalent `gcp` and `azure` blocks instead. The `baseRepository` format also varies by cloud provider (e.g., `oci://<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-helm-repository` for GCP, `oci://<account_id>.dkr.ecr.<region>.amazonaws.com/sgp-<workspace_id>-helm-repository` for AWS).

```json theme={null}
{
  "cloudProvider": "<aws|azure|gcp>",
  "baseRepository": "<helm-repository-oci-url>",
  "samlSetupEnabled": true,
  "oidcSetupEnabled": true,
  "deploymentURL": "https://<workspace_id>.workspace.egp.scale.com",
  "workspaceId": "<workspace_id>",
  "authType": "<saml|oidc>",
  "baseDomain": "<base_domain>",
  "scaletrain_tenant_prefix": "<scaletrain_tenant_prefix>",
  "train_tenant_prefix": "<train_tenant_prefix>",
  "deployAgentex": true,
  "deploySae": true,
  "aws": {
    "accountId": "<aws_account_id>",
    "region": "<aws_region>",
    "prefix": "<prefix>",
    "modelEngineS3Bucket": "scale-egp-<workspace_id>-ml",
    "sqsQueuePolicyTemplate": "",
    "sqsQueueTagTemplate": "",
    "clusterName": "<cluster_name>",
    "karpenterIrsaArn": "<karpenter_irsa_arn>",
    "targetGroupArn": "<target_group_arn>",
    "nodeSubnets": "<node_subnets>",
    "nodeSecurityGroup": "<node_security_group>",
    "postgresHostTemporal": "<postgres_host_temporal>",
    "compassBucketName": "<compass_bucket_name>",
    "compassMongoHost": "<compass_mongo_host>",
    "compassRedisHost": "<compass_redis_host>",
    "reductoBucketName": "<reducto_bucket_name>",
    "reductoDatabaseUrl": "<reducto_database_url>",
    "reductoIrsaRoleArn": "<reducto_irsa_role_arn>",
    "reductoAzureVisionEndpoint": "<reducto_azure_vision_endpoint>",
    "reductoAzureVisionKey": "<reducto_azure_vision_key>",
    "codeBuildProjectName": "<codebuild_project_name>",
    "codeBuildS3Bucket": "<codebuild_s3_bucket>",
    "codeBuildRegistryUrl": "<codebuild_ecr_registry_url>",
    "codeBuildServiceRoleArn": "<codebuild_service_role_arn>",
    "cloudDeployEnabled": true,
    "dex": {
      "irsaRoleArn": "<duc_api_backend_irsa_role_arn>",
      "prefix": "duc-<workspace_id>"
    },
    "identities": {
      "sgpModels": {
        "irsaArn": "<sgp_models_irsa_arn>",
        "secretArns": {
          "backend": "<sgp_models_backend_secret_arn>",
          "model-providers": "<sgp_models_provider_secret_arn>"
        }
      }
    },
    "train": {
      "irsaRoleArn": "<train_irsa_role_arn>",
      "sagemakerExecutionRoleArn": "<train_sagemaker_execution_role_arn>",
      "sagemakerSecurityGroupId": "<train_sagemaker_security_group_id>",
      "databaseHost": "<train_database_host>",
      "dataBucket": "<train_s3_data_bucket_name>",
      "checkpointsBucket": "<train_s3_checkpoints_bucket_name>",
      "outputBucket": "<train_s3_output_bucket_name>",
      "stagingBucket": "<train_s3_staging_bucket_name>"
    },
    "registryProxy": {
      "irsaRoleArn": "<registry_proxy_irsa_role_arn>",
      "ecrRepositoryUrl": "<registry_proxy_ecr_repository_url>",
      "jwtSecret": "<registry_proxy_jwt_secret>",
      "workspaceConfig": "<workspace_config_json>"
    }
  },
  "frontDoorSSLCertB64": "<istio_certificate_b64_resolved>",
  "frontDoorSSLKeyB64": "<istio_key_b64_resolved>",
  "initialDesiredState": "<desired_state_json>",
  "datadog": {
    "enabled": true,
    "env": "<datadog_context>",
    "clusterName": "<cluster_name>",
    "secretName": "<datadog_secret_name>",
    "irsaRoleArn": "<datadog_irsa_role_arn>"
  }
}
```

## Architecture

System Manager runs as a deployment in the `sgp-system-manager` namespace. Its GUI is accessible on port 8000.

```bash theme={null}
kubectl port-forward deployment/sgp-system-manager -n sgp-system-manager 8000:8000
```

Then navigate to `http://localhost:8000` in your browser to access the GUI.

```mermaid theme={null}
graph TD
    ADMIN(["Platform Admin"])

    subgraph CLUSTER["Kubernetes Cluster"]
        subgraph SM_NS["sgp-system-manager namespace"]
            GUI["Web GUI<br/>:8000"]
            API["REST API<br/>/api/v3"]
            WATCHER["Secret Watcher<br/>polls every 3s"]
            INSTALLER["Pack Installer"]
        end

        subgraph RESOURCES["Cluster Resources"]
            NS["Namespaces"]
            SEC["Secrets + Pull Secrets"]
            HR["HelmRelease CRDs"]
        end

        subgraph FLUX_NS["flux-system namespace"]
            FLUX["Flux Helm Controller"]
        end

        PODS["SGP Services<br/>(Helm-managed pods)"]
    end

    subgraph CLOUD["Cloud Provider"]
        SM_SECRET["Secret Manager<br/>desired-state · config"]
        HELM_REPO["Helm Repository<br/>OCI Artifact Registry"]
    end

    ADMIN -->|"browser"| GUI
    GUI --> API
    API -->|"read / write"| SM_SECRET
    WATCHER -->|"detect external changes"| SM_SECRET
    API --> INSTALLER
    INSTALLER --> NS
    INSTALLER --> SEC
    INSTALLER --> HR
    FLUX -->|"watches"| HR
    FLUX -->|"pull chart"| HELM_REPO
    FLUX -->|"deploy"| PODS
```

### Packs

System Manager organizes services into "packs". Each pack is a collection of resources that are deployed together. Generally packs are composed of [FluxCD HelmRelease](https://fluxcd.io/flux/components/helm/helmreleases/) as well as other resources necessary to support a particular service.

When a pack is installed, System Manager renders its resource templates and writes the resulting Kubernetes resources — namespaces, secrets, and FluxCD HelmRelease CRDs — to the cluster. FluxCD then picks up the HelmRelease CRDs and handles pulling and deploying the Helm charts.

### FluxCD

System Manager offloads resource reconciliation to FluxCD. FluxCD is a tool that allows you to manage the lifecycle of your Kubernetes resources. It is responsible for ensuring that the desired state of the resources is maintained. For more information on FluxCD, see the [FluxCD documentation](https://fluxcd.io/flux/components/helm/helmreleases/).

### desired-state.json

The collection of packs that System Manager will deploy is defined in the `desired-state.json` file. This file is stored in the cloud provider's secret manager. To modify the desired state, you can either update the secret directly or use the System Manager GUI, then trigger reconciliation via the System Manager GUI. A sample desired state file is shown below:

```json theme={null}
{
  "version": "0.1",
  "packs": [
    { "name": "flux" },
    { "name": "sgp-helm-repository" },
    { "name": "istio" },
    { "name": "spicedb" },
    { "name": "identity-service" },
    { "name": "temporalf" },
    { "name": "sgp-apps" }
  ]
}
```


# Creating an Agent
Source: https://docs.gp.scale.com/docs/v5/agents/agent-service/lifecycle/creating-agent-service-agents

How to create and publish an agent in the SGP UI.

<Warning>
  #### Recommendation & Future Development

  While Agent Service remains supported, **Agentex is the recommended path** for all new projects. Future development, new features, and enhancements will be focused primarily on Agentex.
</Warning>

## 1. Give your agent a name

The first step is to give your agent a name. This will be used to identify the agent in the SGP UI.

1. Navigate to the [agents page](https://egp.dashboard.scale.com/beta/build).

2. Create a new Agent Service Agent using the top right **Create Agent** button.

3. Fill in the name of the agent and an optional description and click **Create Agent**.

<Frame>
  <img alt="Create Agent Service Agent" />
</Frame>

A draft version of the agent will be created and you will be redirected to the agent's CI/CD page.

## 2. Configure your agent

Now you should see your agent's CI/CD page with a YAML editor featured prominently. Any updates you make to the YAML will be automatically saved.

<Frame>
  <img alt="Blank YAML editor" />
</Frame>

This YAML configuration is what is used to build the agent on the fly using Agent Service. You can copy and paste or drag and drop an existing YAML file from your local device.

Alternatively, you can get started by selecting a template from the **Use a template** menu in the bottom. This is a library of preconfigured YAML files or snippets that we are constantly expanding. Selecting the template will immediately update the YAML in the editor, which you can modify.

For example, you could swap the `llm_model` from `gpt-4o-mini` to another one like `llama-3-1-70b-instruct` for an open source alternative. Note that these models are directly interfacing with SGPs model API and can be called by name or by using the `model_deployment_id`.

<Frame>
  <img alt="Template Agent" />
</Frame>

Often, it is useful to visualize this workflow. To do that, you can swap to the **Graph** view on the top left, which will show a node visualization of your workflow or state machine application. Clicking on any of the nodes shows its configuration.

<Frame>
  <img alt="Graph View" />
</Frame>

## 3. Test your agent

You can easily test the current configuration (as long as it is a valid YAML). To do that click **Test** in the bottom menu.

<Frame>
  <img alt="Test Agent" />
</Frame>

By changing your YAML configuration and continuously testing its output, you can iteratively adjust your agent until it has reached the desired state. The **Threads** panel on the right supports multi-turn by default. If you want to reset context and start a new thread, just click **New Thread**.

The YAML configuration is auto-saved continuously to ensure progress is lost.

## 4. Publish your agent

Once you want to go ahead and publish the configuration for deployment, you can hit **Publish Agent** in the top right. After an agent is published its configuration is immutable and cannot be changed. Only its name and description can be updated.

<Frame>
  <img alt="Published Agent" />
</Frame>

## What's next?

Using the templates captures only a tiny fraction of what is possible with Agent Service. To write you own custom configurations you need to learn the set of available nodes, connections and techniques.&#x20;

Learn more in our v4 documentation on [Building Agents](/docs/agents/hello-world).


# Deploying an Agent
Source: https://docs.gp.scale.com/docs/v5/agents/agent-service/lifecycle/deploying-agent-service-agents

How to deploy and interact with agents in the SGP UI.

<Warning>
  #### Recommendation & Future Development

  While Agent Service remains supported, **Agentex is the recommended path** for all new projects. Future development, new features, and enhancements will be focused primarily on Agentex.
</Warning>

## Introduction

Agents that are configured with YAML in SGP and are fully executed on the platform's backend. The application logic defined in your YAML configuration determines how the agent processes inputs and produces outputs.

## Live Execution from YAML Editor

As you configure your agent in YAML, you can execute it within SGP's interface to see how it behaves.
The execution output is shown in the right-hand panel, on a per-node basis. This allows you to debug and validate your agent's flow in real time:

<Frame>
  <img alt="Test Agent" />
</Frame>

Outputs can be viewed either as raw JSON or in a more readable, formatted display. Toggle between these modes to best understand or debug your agent's responses.

## Providing User Inputs

In the Agent Service CI/CD page, you will be prompted to provide inputs that are designated as `user_question` in the YAML config.

```yaml theme={null}
workflow:
  - name: query_llm
    type: generation
    config:
      llm_model: gpt-4o-mini
      temperature: 0.2
    inputs:
      input_prompt: user_question
```

These inputs will be rendered in the inputs panel. You can provide them as either raw JSON or fill them in via form fields. If you do not specify user inputs or type them, you will have to use JSON inputs. Hitting submit starts an execution of the agent as a run.

<Frame>
  <img alt="Submit Input" />
</Frame>

## Deploying an Agent

From the Agent Service CI/CD page, click **Deploy Agent**. Then enter a custom slug (a unique identifier) for your url. By default this is set to the agent's id.
Click **Deploy**, which creates a simple web-based chat interface you can access.

<Frame>
  <img alt="Deploy Agent" />
</Frame>

When an agent is deployed, you will see a success message and a link to the deployed agent in the header.

<Frame>
  <img alt="Deployed Agent" />
</Frame>

## Interacting with a Deployed Agent

Once deployed, you can visit the web app at the slug you specified. You can chat with the agent in a more intuitive, conversational format. The chat interface supports threads, user feedback, and dynamic inputs (if the agent requires more than one input per turn).

<Frame>
  <img alt="Chat Interface" />
</Frame>

## Customizing a Deployed Agent

Once an agent is deployed, you can customize it by clicking the **Manage Deployment** tab. From there you can:

* Change the slug (URL) of the deployed agent
* Change the theme of the deployed agent
* Change the evaluation settings of the deployed agent

### Changing the slug

To change the slug of the deployed agent, simply enter a new slug in the input field and click **Redeploy**.

<Frame>
  <img alt="Redeploy Agent" />
</Frame>

### Changing the theme

To change the theme of the deployed agent, simply select a new theme from the dropdown. This will apply the theme to the deployed agent without having to redeploy.

You can also create a new theme by clicking **Create New Theme** in the dropdown. This will open the theme editor, where you can customize the theme to your liking.

<Frame>
  <img alt="Create Theme" />
</Frame>

### Changing the evaluation settings

You can enable or disable continuous evaluation for your agent by toggling the **Online Evaluation** switch.

<Frame>
  <img alt="Online Evaluation" />
</Frame>


# Monitoring an Agent
Source: https://docs.gp.scale.com/docs/v5/agents/agent-service/lifecycle/monitoring-agent-service-agents

How to monitor and evaluate agents in the SGP UI.

<Warning>
  #### Recommendation & Future Development

  While Agent Service remains supported, **Agentex is the recommended path** for all new projects. Future development, new features, and enhancements will be focused primarily on Agentex.
</Warning>

## Monitoring Executions

Every time an agent is run (during testing or otherwise), it is recorded and can be viewed on the traces page. This provides a comprehensive view of all threads and their inputs/outputs, helping you track performance and diagnose issues.

You can navigate to the traces page by clicking the **View Traces** button in the agent header.

<Frame>
  <img alt="View Traces" />
</Frame>

From the traces page you can:

* View the inputs and outputs of each thread
* View the performance metrics of each thread
* View the metadata of each thread
* Filter and search for specific threads

<Frame>
  <img alt="Traces Page" />
</Frame>

Clicking on a trace will open the trace details in a modal.

<Frame>
  <img alt="Individual Trace" />
</Frame>


# Introduction to Agent Service
Source: https://docs.gp.scale.com/docs/v5/agents/agent-service/overview

An overview of SGP's Agent Building Framework

<Warning>
  #### Recommendation & Future Development

  While Agent Service remains supported, **Agentex is the recommended path** for all new projects. Future development, new features, and enhancements will be focused primarily on Agentex.
</Warning>

Welcome to the introductory guide for **Agent Service**. This document is designed for engineers who are new to Agent Service and want to learn how to build GenAI agents using our framework. Agent Service provides the backend for SGP applications, enabling the creation, deployment, and execution of powerful AI solutions based on Large Language Models (LLMs) and other AI models.

To get a more in depth overview of Agent Service, check out the [v4 docs](/docs/agents/agent-service) page.

## What is Agent Service?

Agent Service is a software framework that provides a set of modular building blocks that make it easy to create, deploy, and run AI applications. These applications can perform a variety of tasks, including:

* **Document Analysis**
* **Chatbot Responses**
* **Custom AI Solutions**

While Agent Service is primarily designed to support LLM-based applications, it can also accommodate other types of models (for example, multi-modal LLMs) on an ad-hoc basis.

## Key Features

* **Modular Design:**
  Build applications using pre-configured components that can be combined like Lego blocks.
* **Flexible Architecture:**
  Support both simple, linear workflows and complex, interactive applications with ease.
* **Seamless Integration with SGP:**
  Easily integrate with existing SGP tools and APIs for functionalities such as information retrieval and text generation.

## Core Components

Agent Service applications are constructed using several key components:

### Node

* **Definition:**
  A node represents a single operation such as generation, retrieval, or data transformation.
* **Examples:**
  * **Generation Nodes:** Handle LLM-based text generation and chat responses.
  * **Reranker & Retriever Nodes:** Enhance search capabilities and ensure relevance when working with large datasets.
  * **Citation & Code Execution Nodes:** Provide operations like reference generation or executing code for data transformation.

### Workflow

* **Definition:**
  A workflow is a sequence of nodes arranged in a specific order. It aggregates data from user inputs or previous outputs.
* **Features:**
  * Supports parallel processing for maximum execution speed.
  * Offers flexibility in how nodes are combined and executed.

### State Machine

* **Definition:**
  A state machine is a structured flow that transitions between different states (or workflows) based on outcomes.
* **Capabilities:**
  * Supports branching and conditional paths.
  * Incorporates additional user inputs for highly interactive applications.

<Frame>
  <img alt="schematic overview of agent service" />
</Frame>


# Agentex CI/CD
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/agentex-monitoring

Understand how to set up CI/CD for Agentex Agents

## Navigating to the Agentex CI/CD page

1. To view all Agentex agents in SGP navigate to the [agents page](https://egp.dashboard.scale.com/beta/build).

2. Click on an Agentex agent to navigate to the Agentex CI/CD page.

<Frame>
  <img alt="Agentex CI/CD page" />
</Frame>

## Agentex CI/CD functionality

From the Agentex CI/CD page you can:

* View the active version of the Agentex agent
* Open the agent chat interface
* Navigate to the agent's traces
* See the git commits corresponding to each agent deployment

## Agentex versioning

The active version of the Agentex agent is displayed at the top of the page. Agentex agents are automatically deployed when a change is merged into Github.

To create a new version of the Agentex agent, just push a new commit to Github and merge it into the main branch. After it's merged
you'll be able to see the new version deployed in SGP, as well as the commit hashes and the timestamp of the deployment.


# API and SDK
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/api

Programmatic access to Agentex cloud builds and deployments via the REST API and Python/TypeScript SDKs.

## Overview

The Build and Deploy API and SDKs give you programmatic control over Agentex cloud builds and deployments. You can create builds, poll for status, stream logs, create deployments, and tear them down through REST endpoints under `/v5/builds` and `/v5/agentex/deployments`, or through the equivalent Python and TypeScript SDK methods.

The API and SDKs expose the same underlying resources as the UI, CLI, and CI/CD interfaces, so anything you create or modify programmatically is visible everywhere else. Use them when you need to integrate builds and deployments into custom automation or tooling.

For full SDK documentation, see the [Python SDK reference](https://sgp-dev.stldocs.app/api/python) and [TypeScript SDK reference](https://sgp-dev.stldocs.app/api/typescript).

## Authentication

Every request to the Build and Deploy API requires two headers:

| Header                  | Description                                                               |
| ----------------------- | ------------------------------------------------------------------------- |
| `x-api-key`             | Your SGP API key. Found in the IAM or admin section of the SGP dashboard. |
| `x-selected-account-id` | Your SGP account ID. Found in the admin section of the SGP dashboard.     |

<CodeGroup>
  ```bash curl theme={null}
  curl -s $SGP_CLIENT_BASE_URL/v5/builds \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID"
  ```

  ```python Python theme={null}
  from scale_gp_beta import SGPClient

  client = SGPClient(
      api_key="your-api-key",
      account_id="your-account-id",
  )
  ```

  ```typescript TypeScript theme={null}
  import SGPClient from 'scale-gp';

  const client = new SGPClient({
    apiKey: process.env.SGP_API_KEY,
    accountId: process.env.SGP_ACCOUNT_ID,
  });
  ```
</CodeGroup>

<Note>
  The Python and TypeScript SDKs handle authentication headers automatically once you initialize the client. The examples in the rest of this page assume you have already created a `client` instance as shown above.
</Note>

## Build lifecycle

A build packages your agent source code into a container image. The lifecycle is: **create** the build, **poll** its status until it reaches a terminal state, and optionally **stream logs** or **cancel** it. See the [build status lifecycle](/docs/v5/agents/agentex/cloud-build-and-deploy/overview#build) for the full state diagram.

### Create a build

To create a build, send a multipart form request with your build context archive (a `.tar.gz` of your agent source code), the target image name and tag, and the agent name.

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST $SGP_CLIENT_BASE_URL/v5/builds \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
    -F "context_archive=@build-context.tar.gz" \
    -F "image_name=my-agent" \
    -F "image_tag=v1.2.0" \
    -F "agent_name=my-agent"
  ```

  ```python Python theme={null}
  build = client.build.create(
      context_archive="./build-context.tar.gz",
      image_name="my-agent",
      image_tag="v1.2.0",
      agent_name="my-agent",
  )
  print(f"Build created: {build.id} (status: {build.status})")
  ```

  ```typescript TypeScript theme={null}
  const build = await client.build.create({
    context_archive: "./build-context.tar.gz",
    image_name: "my-agent",
    image_tag: "v1.2.0",
    agent_name: "my-agent",
  });
  console.log(`Build created: ${build.id} (status: ${build.status})`);
  ```
</CodeGroup>

The response includes the build ID and an initial status of `queued`. Use the build ID to poll for status updates and retrieve logs.

Optional parameters:

| Parameter    | Description                                               |
| ------------ | --------------------------------------------------------- |
| `build_args` | Key-value pairs passed as Docker build arguments          |
| `platform`   | Target platform for the image build (e.g., `linux/amd64`) |

### Get build status

Poll `GET /v5/builds/{build_id}` to check the current status of a build.

<CodeGroup>
  ```bash curl theme={null}
  curl -s $SGP_CLIENT_BASE_URL/v5/builds/$BUILD_ID \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID"
  ```

  ```python Python theme={null}
  build = client.build.retrieve("build_abc123")
  print(f"Status: {build.status}")
  ```

  ```typescript TypeScript theme={null}
  const build = await client.build.retrieve("build_abc123");
  console.log(`Status: ${build.status}`);
  ```
</CodeGroup>

### Cancel a build

You can cancel a build that is in `queued` or `running` state.

```bash theme={null}
curl -X POST $SGP_CLIENT_BASE_URL/v5/builds/$BUILD_ID/cancel \
  -H "x-api-key: $SGP_API_KEY" \
  -H "x-selected-account-id: $SGP_ACCOUNT_ID"
```

## Deploy lifecycle

A deployment takes a successfully built container image and rolls it out to a Kubernetes cluster. The lifecycle is: **create** the deployment, **poll** its status until it becomes `healthy` or `failed`, and inspect **logs** for debugging. See the [deployment status lifecycle](/docs/v5/agents/agentex/cloud-build-and-deploy/overview#deployment) for the full state diagram.

### Create a deployment

To create a deployment, provide the `manifest_file` (your `manifest.yaml` content as a string), the `environment_config` (your `environments.yaml` content as a string), and a `build_id` referencing a successful build.

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST $SGP_CLIENT_BASE_URL/v5/agentex/deployments \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
    -H "Content-Type: application/json" \
    -d '{
      "manifest_file": "name: my-agent\nversion: 1.0\n...",
      "environment_config": "environment: production\nnamespace: agents\n...",
      "build_id": "build_abc123"
    }'
  ```

  ```python Python theme={null}
  deployment = client.deploy.create(
      manifest_file="name: my-agent\nversion: 1.0\n...",
      environment_config="environment: production\nnamespace: agents\n...",
      build_id="build_abc123",
  )
  print(f"Deployment created: {deployment.id} (status: {deployment.status})")
  ```

  ```typescript TypeScript theme={null}
  const deployment = await client.deploy.create({
    manifest_file: "name: my-agent\nversion: 1.0\n...",
    environment_config: "environment: production\nnamespace: agents\n...",
    build_id: "build_abc123",
  });
  console.log(`Deployment created: ${deployment.id} (status: ${deployment.status})`);
  ```
</CodeGroup>

Instead of a `build_id`, you can supply `image_name` and `image_tag` directly to deploy a pre-existing image.

To create a **preview deployment** (ephemeral, with automatic teardown), add the following optional parameters:

| Parameter       | Description                                                      |
| --------------- | ---------------------------------------------------------------- |
| `preview`       | Set to `true` to create a preview deployment                     |
| `preview_label` | A label for the preview (e.g., `pr-482`). Maximum 30 characters. |
| `expires_at`    | ISO 8601 timestamp for when the preview should be torn down      |

### Get deployment status

Poll `GET /v5/agentex/deployments/{deployment_id}` to check the current status and view deployment events.

<CodeGroup>
  ```bash curl theme={null}
  curl -s $SGP_CLIENT_BASE_URL/v5/agentex/deployments/$DEPLOYMENT_ID \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID"
  ```

  ```python Python theme={null}
  deployment = client.deploy.retrieve("deploy_xyz789")
  print(f"Status: {deployment.status}")
  ```

  ```typescript TypeScript theme={null}
  const deployment = await client.deploy.retrieve("deploy_xyz789");
  console.log(`Status: ${deployment.status}`);
  ```
</CodeGroup>

### Delete a deployment

To tear down a deployment and release its cluster resources:

```bash theme={null}
curl -X DELETE $SGP_CLIENT_BASE_URL/v5/agentex/deployments/$DEPLOYMENT_ID \
  -H "x-api-key: $SGP_API_KEY" \
  -H "x-selected-account-id: $SGP_ACCOUNT_ID"
```

## Polling pattern

Builds and deployments are asynchronous operations. After creating either resource, poll the status endpoint until the resource reaches a terminal state.

**Build statuses:** `queued` → `running` → `success` | `failed` | `cancelled` | `timed_out` | `error`

**Deploy statuses:** `pending` → `in_progress` → `healthy` | `failed` | `cancelled`

<CodeGroup>
  ```bash curl theme={null}
  # Poll a build until it reaches a terminal state
  while true; do
    STATUS=$(curl -s $SGP_CLIENT_BASE_URL/v5/builds/$BUILD_ID \
      -H "x-api-key: $SGP_API_KEY" \
      -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
      | jq -r '.status')
    echo "Build status: $STATUS"
    case $STATUS in
      success|failed|cancelled|timed_out|error) break ;;
    esac
    sleep 5
  done
  ```

  ```python Python theme={null}
  import time

  build = client.build.retrieve("build_abc123")
  terminal_statuses = {"success", "failed", "cancelled", "timed_out", "error"}

  while build.status not in terminal_statuses:
      time.sleep(5)
      build = client.build.retrieve("build_abc123")
      print(f"Build status: {build.status}")

  if build.status != "success":
      raise RuntimeError(f"Build ended with status: {build.status}")
  ```

  ```typescript TypeScript theme={null}
  const terminalStatuses = new Set(["success", "failed", "cancelled", "timed_out", "error"]);

  let build = await client.build.retrieve("build_abc123");

  while (!terminalStatuses.has(build.status)) {
    await new Promise((r) => setTimeout(r, 5000));
    build = await client.build.retrieve("build_abc123");
    console.log(`Build status: ${build.status}`);
  }

  if (build.status !== "success") {
    throw new Error(`Build ended with status: ${build.status}`);
  }
  ```
</CodeGroup>

<Note>
  A polling interval of 5 seconds is a reasonable default. Builds typically take 2-10 minutes depending on image complexity. Deployments typically take 1-5 minutes depending on image size and cluster capacity.
</Note>

## Log streaming

Build logs and deployment logs use different retrieval mechanisms.

### Build logs (SSE)

Build logs are streamed via Server-Sent Events (SSE) from `GET /v5/builds/{build_id}/logs`. Each event contains a log line from the image build process.

```bash theme={null}
curl -N $SGP_CLIENT_BASE_URL/v5/builds/$BUILD_ID/logs \
  -H "x-api-key: $SGP_API_KEY" \
  -H "x-selected-account-id: $SGP_ACCOUNT_ID"
```

The response is a stream of `text/event-stream` data. Each line is a log entry from the build. The stream closes when the build completes.

### Deployment logs (cursor-based pagination)

Deployment logs are retrieved with cursor-based pagination from `GET /v5/agentex/deployments/{deployment_id}/logs`. Each response includes a page of log lines, a `next_cursor` for fetching the next page, and a `has_more` flag.

<CodeGroup>
  ```bash curl theme={null}
  # Fetch the first page of deployment logs
  CURSOR=""
  while true; do
    RESPONSE=$(curl -s "$SGP_CLIENT_BASE_URL/v5/agentex/deployments/$DEPLOYMENT_ID/logs?limit=100&cursor=$CURSOR" \
      -H "x-api-key: $SGP_API_KEY" \
      -H "x-selected-account-id: $SGP_ACCOUNT_ID")
    echo "$RESPONSE" | jq -r '.lines[]'
    HAS_MORE=$(echo "$RESPONSE" | jq -r '.has_more')
    if [ "$HAS_MORE" != "true" ]; then
      break
    fi
    CURSOR=$(echo "$RESPONSE" | jq -r '.next_cursor')
  done
  ```

  ```python Python theme={null}
  cursor = None

  while True:
      params = {"limit": 100}
      if cursor:
          params["cursor"] = cursor

      response = client.deploy.logs(
          "deploy_xyz789",
          **params,
      )

      for line in response.lines:
          print(line)

      if not response.has_more:
          break
      cursor = response.next_cursor
  ```

  ```typescript TypeScript theme={null}
  let cursor: string | undefined;

  while (true) {
    const response = await client.deploy.logs("deploy_xyz789", {
      limit: 100,
      cursor,
    });

    for (const line of response.lines) {
      console.log(line);
    }

    if (!response.has_more) break;
    cursor = response.next_cursor;
  }
  ```
</CodeGroup>

## Listing and filtering

Use the list endpoints to retrieve builds and deployments for an agent, filtered by common criteria.

### List builds

Retrieve builds filtered by agent name, with control over result count and sort order.

<CodeGroup>
  ```bash curl theme={null}
  curl -s "$SGP_CLIENT_BASE_URL/v5/builds?agent_name=my-agent&limit=10&sort_order=desc" \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID"
  ```

  ```python Python theme={null}
  builds = client.build.list(
      agent_name="my-agent",
      limit=10,
      sort_order="desc",
  )
  for build in builds:
      print(f"{build.id}: {build.status} ({build.image_tag})")
  ```

  ```typescript TypeScript theme={null}
  const builds = await client.build.list({
    agent_name: "my-agent",
    limit: 10,
    sort_order: "desc",
  });
  for (const build of builds) {
    console.log(`${build.id}: ${build.status} (${build.imageTag})`);
  }
  ```
</CodeGroup>

### List deployments

Retrieve deployments, optionally filtered by preview label to find the latest deployment for a specific branch or PR.

<CodeGroup>
  ```bash curl theme={null}
  # Get the latest deployment for a specific PR
  curl -s "$SGP_CLIENT_BASE_URL/v5/agentex/deployments?preview_label=pr-482&limit=1" \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID"
  ```

  ```python Python theme={null}
  deployments = client.deploy.list(
      preview_label="pr-482",
      limit=1,
  )
  if deployments:
      latest = deployments[0]
      print(f"Latest PR deploy: {latest.id} ({latest.status})")
  ```

  ```typescript TypeScript theme={null}
  const deployments = await client.deploy.list({
    preview_label: "pr-482",
    limit: 1,
  });
  if (deployments.length > 0) {
    const latest = deployments[0];
    console.log(`Latest PR deploy: ${latest.id} (${latest.status})`);
  }
  ```
</CodeGroup>

### Undeployed builds

To find agents that have completed builds but no healthy deployment, use the undeployed builds endpoint:

```bash theme={null}
curl -s $SGP_CLIENT_BASE_URL/v5/builds/undeployed \
  -H "x-api-key: $SGP_API_KEY" \
  -H "x-selected-account-id: $SGP_ACCOUNT_ID"
```

This is useful for identifying agents that may have been built but never deployed, or agents whose deployments have failed and need attention.

## API and SDK reference

For full request and response schemas, see the generated API reference and SDK documentation:

**API reference:**

* [Agentex Cloud Build](/reference/v5/agentex-cloud-build): endpoints for creating, listing, polling, and cancelling builds
* [Agentex Cloud Deploy](/reference/v5/agentex-cloud-deploy): endpoints for creating, listing, inspecting, and deleting deployments
* [SGP Cloud Secrets](/reference/v5/sgp-cloud-secrets): endpoints for managing account-level secrets

**SDK reference:**

* [Python SDK](https://sgp-dev.stldocs.app/api/python): `scale_gp_beta` package (`client.build`, `client.deploy`, `client.secrets`)
* [TypeScript SDK](https://sgp-dev.stldocs.app/api/typescript): `scale-gp` package (`client.build`, `client.deploy`, `client.secrets`)

<Note>
  The Build and Deploy API does not support webhooks or push notifications. You must poll the status endpoints to track build and deployment progress. There is no callback mechanism to notify your system when a build completes or a deployment becomes healthy.
</Note>

## Next steps

* [sgpctl CLI](/docs/v5/agents/agentex/cloud-build-and-deploy/sgpctl): the same operations from the terminal with a simpler interface
* [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): automate builds triggered by git push
* [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets): manage credentials via API/SDK or the dashboard
* [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments): ephemeral deployments for branch and PR testing


# CI/CD with Gitea Actions
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd

Automate agent builds and deployments with the provided Gitea Actions workflow templates.

## Overview

The agentex-agents repository template ships with three Gitea Actions workflows that automate the build and deploy lifecycle for your agents. These workflows use the SGP Build and Deploy API to package your agent source code into container images and deploy them to SGP-managed Kubernetes clusters.

Out of the box, merging to `main` triggers an automatic build and deploy. Pushes to feature branches trigger build-only validation. You can also trigger builds and deploys manually via workflow dispatch.

You do not need to write these workflows from scratch. The template repository includes them pre-configured. Your responsibility is to understand when each workflow runs, how to configure your agent directories, and how to set up the required secrets.

## How it works

When you push code that changes an agent, the CI/CD pipeline detects the change, builds a container image, and (on the main branch) deploys it. The following diagram shows the end-to-end flow for a merge to main:

```mermaid theme={null}
flowchart LR
    A[Push to main] --> B[Detect changed agents]
    B --> C[Package build context]
    C --> D[POST /v5/builds]
    D --> E[Poll build status]
    E --> F{Build succeeded?}
    F -- Yes --> G[POST /v5/agentex/deployments]
    F -- No --> H[Fail workflow]
    G --> I[Poll deploy status]
    I --> J{Deploy healthy?}
    J -- Yes --> K[Agent serving traffic]
    J -- No --> L[Fetch logs & fail]
```

The pipeline auto-detects which agents changed by running `git diff` against the previous commit. Only agents with modified files are built, so a single push that touches multiple agents triggers parallel builds for each one.

## The three workflows

The template repository includes three workflow files under `.gitea/workflows/`. Each serves a different purpose in the development lifecycle.

| Workflow                              | Trigger                                         | Builds? | Deploys? | Use case                                    |
| ------------------------------------- | ----------------------------------------------- | ------- | -------- | ------------------------------------------- |
| `cloud-build.yaml`                    | Push to non-main branches (agent paths)         | Yes     | No       | Validate builds on feature branches         |
| `cloud-build-and-deploy-on-push.yaml` | Push to `main` (agent paths) or manual dispatch | Yes     | Yes      | Build and deploy on merge to main           |
| `cloud-deploy.yaml`                   | Manual dispatch or workflow call                | No      | Yes      | Deploy a specific image version or rollback |

### cloud-build.yaml: build on feature branches

This workflow triggers on pushes to any branch **except main** when files under `teams/**/agents/**` change. It auto-detects which agents were modified, packages the build context for each one, and submits a build request to the SGP Build API via `POST /v5/builds`. The workflow polls the build status until it succeeds or fails, with a 600-second timeout.

This workflow does not deploy. Its purpose is to validate that your agent builds successfully before you merge.

### cloud-build-and-deploy-on-push.yaml: build and deploy on main

This workflow triggers when files under `teams/**/agents/**` change on the main branch, or via manual dispatch. It builds container images for all changed agents and then deploys them.

For each agent, the workflow:

1. Packages the build context based on the agent's `manifest.yaml` build configuration
2. Submits the build to `POST /v5/builds` and polls until complete
3. Reads the agent's `manifest.yaml` and `environments.yaml`
4. Submits a deployment to `POST /v5/agentex/deployments` with the resulting `build_id`
5. Polls the deployment status until it reaches `healthy` or fails

The workflow uses a `resolve_inputs.py` helper script to derive the image name from the `agent.name` field in the manifest and to auto-select the environment from `environments.yaml` (auto-selected when only one environment is defined, which is the typical case).

### cloud-deploy.yaml: standalone deploy

This workflow is manual-only. It deploys an existing image without rebuilding. You can trigger it via manual dispatch or call it from another workflow using `workflow_call`.

Inputs include:

* `agent-path` (required): path to the agent directory
* `agent-environment` (required): environment key from `environments.yaml`
* `build-id` OR `image-name` + `image-tag`: specify either a previous build ID or a direct image reference

The workflow submits `POST /v5/agentex/deployments`, polls until the deployment is healthy, and fetches deploy logs on failure.

<Tip>
  Use the standalone deploy workflow for rollbacks. If a deployment fails, you can redeploy the previous known-good build by providing its `build-id`.
</Tip>

## Workflow runs

After a workflow triggers, you can monitor its progress in the Gitea Actions UI. Each workflow run shows the individual steps, their status, and logs for each stage of the build and deploy process.

<Frame>
  <img alt="Gitea workflow run detail page showing build and deploy steps" />
</Frame>

The workflow run page displays:

* Each step in the pipeline with pass/fail indicators
* Timing information for build and deploy phases
* Console output for debugging failed steps

If a build or deployment fails, expand the failed step to see the error output. Common failures include Dockerfile syntax errors, missing dependencies, and secret reference mismatches.

## Manual dispatch

You can trigger a build and deploy for a specific agent without pushing code by using the manual dispatch form in Gitea Actions.

<Frame>
  <img alt="Gitea manual dispatch form with input fields for agent path and configuration" />
</Frame>

To trigger a manual dispatch:

1. Navigate to the **Actions** tab in your Gitea repository
2. Select the `cloud-build-and-deploy-on-push.yaml` workflow (or `cloud-deploy.yaml` for deploy-only)
3. Click **Run workflow**
4. Fill in the inputs:
   * **agent-path** (required): the path to your agent directory (for example, `teams/my-team/agents/my-agent`)
   * **image-name** (optional): overrides the image name derived from `manifest.yaml`. Should match the `agent.name` in your manifest.
   * **agent-name** (optional): overrides the agent name. Should match `image-name` (the `resolve_inputs.py` helper sets both to the same value by default).
   * **image-tag** (optional): overrides the auto-generated tag. Use a version number (e.g., `v1.2.0`) or a commit SHA. Defaults to the current commit SHA.
   * **agent-environment** (optional): must match an environment key in your `environments.yaml`. Auto-selected when only one environment is defined.
5. Click **Run workflow** to start the pipeline

<Note>
  Manual dispatch is useful during initial setup, testing, or when you need to deploy a specific image tag. For routine development, merging to main triggers the build and deploy automatically.
</Note>

## Repository structure

The CI/CD workflows expect a specific directory layout. Each agent lives in its own directory under `teams/<team>/agents/<agent-name>/` and contains the configuration files the workflows need.

```
teams/
  my-team/
    agents/
      my-agent/
        manifest.yaml          # Build and agent configuration
        environments.yaml      # Deployment settings (namespace, resources, etc.)
        Dockerfile             # Container build instructions
        .dockerignore          # Files to exclude from build context
        src/
          main.py              # Agent source code
          requirements.txt     # Python dependencies
          ...
      another-agent/
        manifest.yaml
        environments.yaml
        Dockerfile
        src/
          ...
```

The workflows detect changes at the `teams/**/agents/**` path level. When you push a commit that modifies any file inside an agent directory, the corresponding workflow picks up that agent for building.

## manifest.yaml

The `manifest.yaml` file defines your agent's identity, build context, credentials, and deployment defaults. It is the primary configuration file the CI/CD workflows read when building and deploying your agent.

```yaml theme={null}
kind: Agent

build:
  context:
    # Root directory for the build context, relative to the agent directory
    root: ../
    # Paths to include in the build context (relative to root)
    include_paths:
      - golden_agent
    # Path to the Dockerfile (relative to root)
    dockerfile: golden_agent/Dockerfile
    # Path to the .dockerignore file (relative to root)
    dockerignore: golden_agent/.dockerignore

agent:
  # Agent Communication Protocol type: "async" or "sync"
  acp_type: async
  # Unique agent name (used as the image name by default)
  name: golden-agent
  # Human-readable description shown in the SGP dashboard
  description: Generic Claude Agent SDK harness.
  # Temporal workflow configuration (for async agents)
  temporal:
    enabled: true
    workflows:
      - name: GoldenAgentWorkflow
        queue_name: golden_agent_queue
  # Secrets injected as environment variables at runtime
  credentials:
    - env_var_name: ANTHROPIC_API_KEY
      secret_name: anthropic-api-key
      secret_key: api-key

deployment:
  image:
    # Left empty; populated by the CI/CD pipeline
    repository: ""
    tag: "latest"
  imagePullSecrets:
    - name: agentex-acr-pull-secret
  global:
    replicaCount: 1
    resources:
      requests:
        cpu: "500m"
        memory: "1Gi"
      limits:
        cpu: "1000m"
        memory: "2Gi"
```

### Key fields

**`build.context`** controls what files are sent to the cloud build service. Set `root` to the common ancestor directory, then use `include_paths` to specify which subdirectories to include. The `dockerfile` and `dockerignore` paths are relative to `root`.

**`agent`** defines the agent's identity and runtime configuration:

* `acp_type` determines whether the agent uses synchronous request-response (`sync`) or asynchronous task-based (`async`) communication.
* `name` is used as the default image name and must be unique within your account.
* `temporal` configures Temporal workflow settings for async agents. Each workflow entry specifies the workflow class name and task queue.
* `credentials` maps secrets stored in SGP to environment variables in the running container. Each entry references a secret by name and key, and injects it as the specified environment variable. See [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets) for how to create and manage these secrets.

**`deployment`** sets default resource requests and replica counts. These values can be overridden in `environments.yaml`.

<Note>
  The `deployment.image.repository` field is intentionally left empty. The CI/CD pipeline populates it automatically with the managed registry path during the build phase.
</Note>

## environments.yaml

The `environments.yaml` file defines deployment settings for your agent's target environment: the Kubernetes namespace, authentication principal, resource limits, and Helm value overrides.

The file has a top-level `environments` map with a named key for each environment. In practice, you define one environment per agent. The key name (e.g., `dev.aws` below) is arbitrary; the CI/CD pipeline's `resolve_inputs.py` helper auto-selects it when only one environment is defined.

```yaml theme={null}
schema_version: "v1"

environments:
  dev.aws:
    environment: dev
    kubernetes:
      namespace: "sgp-golden-agent"
    auth:
      principal:
        user_id: ac48a4a5-ca7f-4fcd-9724-f08dbc6b081f
        account_id: 69c69407ee5d19e1dce57d57
    helm_overrides:
      command:
        - uvicorn
        - project.acp:acp
        - --host
        - 0.0.0.0
        - --port
        - "8000"
      replicas: 1
      resources:
        requests:
          cpu: 500m
          memory: 1Gi
        limits:
          cpu: 1000m
          memory: 2Gi
      temporal-worker:
        enabled: true
        replicaCount: 1
        resources:
          requests:
            cpu: 500m
            memory: 1Gi
          limits:
            cpu: 1000m
            memory: 2Gi
      env:
        - name: SGP_ACCOUNT_ID
          value: "69c69407ee5d19e1dce57d57"
        - name: SGP_CLIENT_BASE_URL
          value: "http://egp-api-backend.egp.svc.cluster.local:80/public"
```

### Key fields

**`kubernetes.namespace`** determines where the agent's pods, services, and other resources are created.

**`auth.principal`** provides the `user_id` and `account_id` used to authenticate API calls. These must correspond to a valid SGP service account with permissions to create deployments.

**`helm_overrides`** lets you customize the container command, resource requests and limits, replica counts, Temporal worker configuration, and environment variables. Values specified here take precedence over the defaults in `manifest.yaml`. The `env` field uses array format (`[{name: X, value: Y}]`) for environment variables injected into the running container.

## Required secrets

The CI/CD workflows authenticate with the SGP API using secrets stored in your Gitea repository's Actions settings. You must configure the following secrets before the workflows can run.

| Secret           | Description                                                  |
| ---------------- | ------------------------------------------------------------ |
| `SGP_API_KEY`    | API key for authenticating with the SGP Build and Deploy API |
| `SGP_ACCOUNT_ID` | Your SGP account identifier                                  |

To add these secrets:

1. Navigate to your Gitea repository settings
2. Go to **Actions** > **Secrets**
3. Add `SGP_API_KEY` with your SGP API key value
4. Add `SGP_ACCOUNT_ID` with your account ID

<Note>
  The workflows use the in-cluster API base URL `http://egp-api-backend.egp.svc.cluster.local/public` by default. This is pre-configured in the workflow files and does not require a separate secret. If your cluster uses a different service address, update the `SGP_BASE_URL` variable in the workflow files directly.
</Note>

These repository-level secrets are different from the agent-level secrets defined in `manifest.yaml`. Repository secrets authenticate the CI/CD pipeline itself, while agent secrets (in the `credentials` section of `manifest.yaml`) are injected into the running agent container at runtime.

## Adding a new agent

To add a new agent to the CI/CD pipeline:

1. **Create the agent directory** under your team's path:
   ```bash theme={null}
   mkdir -p teams/my-team/agents/my-new-agent/src
   ```

2. **Add a `manifest.yaml`** with your agent's build context, name, and configuration. Use the example above as a starting point and update the `agent.name`, `build.context` paths, and `credentials` section for your agent.

3. **Add an `environments.yaml`** with your deployment target configuration. Set the `namespace`, `auth.principal`, and any `helm_overrides` for your environment.

4. **Add a `Dockerfile`** and `.dockerignore` at the paths referenced in your `manifest.yaml`. The Dockerfile should produce a container that runs your agent.

5. **Add your agent source code** under `src/` or wherever your Dockerfile expects it.

6. **Push to a feature branch** to trigger `cloud-build.yaml`. Verify that the build completes successfully in the Gitea Actions UI.

7. **Merge to main** to trigger `cloud-build-and-deploy-on-push.yaml`, which builds the agent and deploys it automatically using the configuration in `environments.yaml`.

8. **Verify the deployment** in the SGP dashboard. Navigate to your agent's detail page to confirm the deployment status is `healthy`.

<Warning>
  Ensure your `manifest.yaml` references only secrets that have already been created in SGP. A deployment will fail if it references a secret that does not exist. See [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets) for setup instructions.
</Warning>

## Next steps

* [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api): access builds and deployments programmatically
* [sgpctl CLI](/docs/v5/agents/agentex/cloud-build-and-deploy/sgpctl): build, deploy, and inspect agents from the terminal
* [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets): create and rotate credentials referenced in `manifest.yaml`


# Cloud Build and Deploy
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/overview

Build container images and deploy Agentex agents to SGP-managed Kubernetes clusters from the UI, CLI, CI/CD, or API.

## Overview

Cloud Build and Deploy is the SGP-managed pipeline for packaging your Agentex agent code into a container image and deploying it to a Kubernetes cluster. You push your agent source code, SGP builds a container image in the cloud, and deploys it to a managed cluster where it becomes available for traffic.

This removes the need to manage Docker registries, Helm charts, or Kubernetes resources directly. SGP handles image building, artifact storage, Helm releases, health checks, and traffic routing. You interact with the system through the SGP dashboard UI, the `sgpctl` CLI, CI/CD pipelines (Gitea Actions), or the Build and Deploy REST API.

Cloud Build and Deploy supports both long-lived production deployments and ephemeral preview deployments for branch and PR testing. Builds and deployments are tracked with status history, event timelines, and container logs in the SGP dashboard.

The system is built around two configuration files in your agent repository: `manifest.yaml` (what to build and how to deploy) and `environments.yaml` (where to deploy). These files give you declarative control over the build and deploy lifecycle without writing infrastructure code.

## How it works

The following diagram shows the end-to-end flow from pushing code to serving traffic:

```mermaid theme={null}
sequenceDiagram
    participant Dev as Developer
    participant CI as CI/CD (Gitea Actions)
    participant Build as SGP Cloud Build
    participant Deploy as SGP Deploy Service
    participant K8s as Managed K8s Cluster

    Dev->>CI: Push agent code
    CI->>CI: Package build context
    CI->>Build: POST /v5/builds
    Build->>Build: Build container image
    Build-->>CI: Build complete (image reference)
    CI->>Deploy: POST /v5/agentex/deployments
    Deploy->>K8s: Helm release with built image
    K8s->>K8s: Pods start, health checks pass
    K8s-->>Deploy: Agent healthy
    Deploy-->>Dev: Traffic routed to agent
```

The pipeline has two phases:

1. **Build phase**: Your CI pipeline (or a manual CLI/API call) sends the agent source code to the SGP Build Service via `POST /v5/builds`. SGP builds a container image using the instructions in your `manifest.yaml`, including the base image, build arguments, and any referenced secrets. The resulting image is stored in a managed container registry. You do not need to configure or authenticate with the registry yourself.

2. **Deploy phase**: A deployment is created via `POST /v5/agentex/deployments`. SGP generates a Helm release targeting the Kubernetes cluster and namespace specified in your `environments.yaml`. The deploy service monitors pod scheduling, image pulls, container startup, and health check responses. When all pods report healthy, SGP routes traffic to the new deployment and marks it as `healthy`.

By default, the CI/CD pipeline runs both phases automatically on merge to main. On feature branches, only the build phase runs. You can also trigger deployments manually from the UI, CLI, or API. You can redeploy a previous build without rebuilding.

<Info>
  A single build can have multiple deployment iterations (for example, if a deployment fails and you redeploy), but all deployments from a given build target the same environment.
</Info>

## Access methods

You can interact with Cloud Build and Deploy through four interfaces. Each supports the full build and deploy lifecycle.

| Interface                 | Best for                                         | Details                                                                                                                                                        |
| ------------------------- | ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **SGP Dashboard UI**      | Visual monitoring, manual deployments, debugging | Point-and-click access to builds, deployments, logs, and events. No setup required beyond dashboard access.                                                    |
| **CI/CD (Gitea Actions)** | Automated builds and deploys, team workflows     | Build on every push, build and deploy on merge to main. The recommended approach for production workflows.                                                     |
| **`sgpctl` CLI**          | Terminal workflows, scripting, debugging         | Command-line tool for creating builds, deploying, inspecting status, and tailing logs. Useful for ad-hoc operations and shell scripts.                         |
| **API and SDK**           | Programmatic access, custom integrations         | REST API and Python/TypeScript SDKs for full control over builds and deployments. Use when you need to integrate with custom tooling or orchestration systems. |

All four interfaces operate on the same underlying resources. A build started from the CLI appears in the dashboard UI, and a deployment created from CI/CD can be inspected via the API.

In practice, most teams use CI/CD for automated builds and deploys on main, and supplement with the CLI for debugging and the dashboard for monitoring.

## Typical workflow

A typical development cycle looks like this:

1. **Develop locally**: Write and test your agent code on your local machine. Use the Agentex local development tools to verify behavior before pushing.

2. **Push to a feature branch**: The CI/CD pipeline triggers a build-only workflow to validate your agent compiles and packages correctly.

3. **Merge to main**: On merge, the CI/CD pipeline builds the agent and deploys it automatically using the configuration in `environments.yaml`.

4. **Monitor**: The build and deployment appear on the agent detail page in the SGP dashboard. Build logs stream in real-time, and deployment events show rollout progress.

5. **Verify**: Check the deployment detail page for health status. For preview deployments (created separately via the UI or API/SDK), use the dedicated preview endpoint.

6. **Iterate**: If the deployment fails, use the events timeline and logs to diagnose the issue, fix the code, and push again.

## Prerequisites

Before using Cloud Build and Deploy, ensure the following are in place:

* **An Agentex agent repository**: Your agent code must be in a Gitea repository accessible to SGP. See the [Agentex agent types](/docs/v5/agents/agentex/overview) documentation if you have not yet created an agent.
* **A `manifest.yaml` file**: Must exist at the root of your agent repository. Defines how SGP builds and deploys your agent.
* **An `environments.yaml` file**: Defines the target environment (cluster, namespace, resource limits) for your deployments.
* **Account-level secrets (if needed)**: If your agent requires API keys or other credentials, these must be created in the SGP secrets store before your first deployment. See [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets).
* **SGP API access**: You need a valid SGP API key with permissions to create builds and deployments. This is configured in your CI/CD pipeline or CLI.

<Tip>
  If you do not yet have an agent repository, follow the [Agentex agent builder guide](/docs/v5/agents/agentex/types/agentic-agent) to scaffold a new agent project, then return here to set up the build and deploy pipeline.
</Tip>

## Key concepts

The following concepts are central to how Cloud Build and Deploy works.

### Build

A build represents a single container image build from your agent's source code. When you trigger a build, SGP packages the build context (your source files, `manifest.yaml`, and any referenced configuration), sends it to the build service, and produces a tagged container image. Each build is associated with a specific git commit and branch, providing full traceability from source to artifact.

Builds progress through the following statuses:

```mermaid theme={null}
stateDiagram-v2
    [*] --> queued
    queued --> running
    queued --> cancelled
    running --> success
    running --> failed
    running --> cancelled
    running --> timed_out
    running --> error
```

| Status      | Description                                             |
| ----------- | ------------------------------------------------------- |
| `queued`    | Build request accepted, waiting for capacity            |
| `running`   | Image build is in progress                              |
| `success`   | Image built and stored in the managed registry          |
| `failed`    | Build failed due to an error in your code or Dockerfile |
| `cancelled` | Build was cancelled by a user or automation             |
| `timed_out` | Build exceeded the maximum allowed duration             |
| `error`     | An internal error occurred during the build             |

### Deployment

A deployment is a Helm release of a built container image to a Kubernetes cluster. Each deployment creates or updates the Kubernetes resources needed to run your agent: pods, services, ingress rules, and any associated config maps or secrets. The target namespace and resource configuration come from your `environments.yaml`.

Deployments progress through the following statuses:

```mermaid theme={null}
stateDiagram-v2
    [*] --> pending
    pending --> in_progress
    pending --> cancelled
    in_progress --> healthy
    in_progress --> failed
    in_progress --> cancelled
```

| Status        | Description                                                      |
| ------------- | ---------------------------------------------------------------- |
| `pending`     | Deployment created, not yet started                              |
| `in_progress` | Helm release is being applied to the cluster                     |
| `healthy`     | Pods are running and passing health checks                       |
| `failed`      | Deployment failed (pod crash, resource limits, misconfiguration) |
| `cancelled`   | Deployment was cancelled before completion                       |

### Preview deployment

A preview deployment is an ephemeral deployment with an automatic TTL (time-to-live) teardown, designed for branch and PR testing. When you create a preview deployment, SGP provisions a dedicated endpoint where you can test the agent version built from your feature branch without affecting the production deployment.

Preview deployments are automatically cleaned up when:

* The configured TTL expires (default: 8 hours)
* You explicitly tear down the deployment

This makes preview deployments well-suited for code review workflows where reviewers need a live agent to test against. See [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments) for configuration details.

### Configuration files

Two YAML files in your agent repository control the build and deploy lifecycle:

* **`manifest.yaml`**: defines your agent's identity, build context, credentials, and deployment defaults. See the [CI/CD guide](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd#manifestyaml) for a full annotated reference.
* **`environments.yaml`**: defines the target environment (cluster, namespace, resource limits, Helm overrides). Read at deploy time, not build time, so you can change settings without rebuilding. See the [CI/CD guide](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd#environmentsyaml) for details.

Your `manifest.yaml` can reference account-level [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets) that are injected as environment variables at runtime.

## The SGP dashboard

The SGP dashboard provides a visual interface for monitoring builds and deployments. Key pages include:

* **Agents page**: lists all registered agents with their current deployment status. Access it from **Agents** in the left navigation.
* **Agent detail page**: shows build history, active deployment, and git metadata for a specific agent.
* **Build detail page**: shows all deployments created from a build, organized by type (production, preview, in-progress).
* **Deployment detail page**: provides the events timeline, container logs, status transitions, and configuration summary for a deployment.

See the [UI walkthrough](/docs/v5/agents/agentex/cloud-build-and-deploy/ui-walkthrough) for step-by-step flows through these pages.

## What's next

This overview covers the core concepts and the UI surfaces for monitoring builds and deployments. The remaining guides in this section walk through specific workflows in detail:

| Guide                                                                                     | Description                                                          |
| ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| [UI walkthrough](/docs/v5/agents/agentex/cloud-build-and-deploy/ui-walkthrough)           | Step-by-step flows for building and deploying from the SGP dashboard |
| [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd)          | Automated build and deploy pipelines triggered by git push           |
| [sgpctl CLI](/docs/v5/agents/agentex/cloud-build-and-deploy/sgpctl)                       | Build, deploy, and inspect agents from the terminal                  |
| [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api)                         | Programmatic access to builds and deployments via REST API and SDKs  |
| [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets)                         | Store and reference credentials securely                             |
| [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments) | Ephemeral branch deploys for PR testing                              |

If you are setting up Cloud Build and Deploy for the first time, start with the [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd) guide, which walks through `manifest.yaml` configuration and pipeline setup.

For terminal-based workflows, the [sgpctl CLI](/docs/v5/agents/agentex/cloud-build-and-deploy/sgpctl) guide covers installation, authentication, and common commands for building, deploying, and tailing logs.


# Preview Deployments
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments

Spin up short-lived preview deployments per branch or PR with automatic teardown.

## Overview

Preview deployments are ephemeral Helm releases designed for testing agent changes in isolation before promoting to production. Each preview gets its own unique Helm release name, so you can run multiple previews simultaneously (one per branch, PR, or experiment) without interfering with your production deployment or with each other.

Previews are always time-boxed. Every preview deployment has an `expires_at` timestamp, defaulting to 8 hours from creation if you do not specify one. When the TTL expires, a background cleanup process automatically tears down the deployment and releases its cluster resources. You can also tear down a preview manually at any time, or promote it to a permanent production deployment when you are satisfied with the results.

## How previews differ from production

|                        | Production                                | Preview                                                     |
| ---------------------- | ----------------------------------------- | ----------------------------------------------------------- |
| Helm release name      | Stable name (e.g. `customer-support-bot`) | Unique per-deploy (e.g. `customer-support-bot-pr-482-a3f9`) |
| Lifetime               | Permanent until replaced                  | Ephemeral, auto-teardown at `expires_at`                    |
| Default TTL            | None                                      | 8 hours                                                     |
| `preview_label`        | Not applicable                            | Optional grouping label (max 30 characters)                 |
| Simultaneous instances | One active production deployment          | Multiple previews can coexist                               |

The unique Helm release name is what makes simultaneous previews possible. Production deployments reuse a stable release name, so deploying a new version replaces the previous one. Preview deployments append a deployment ID suffix to the release name, giving each preview its own set of Kubernetes resources.

<Note>
  Preview deployments are supported through the SGP dashboard UI and the [API/SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api). The `sgpctl` CLI does not currently support creating preview deployments.
</Note>

## Creating a preview from the UI

From any successful build's detail page, click **Create deployment** (or use the chevron dropdown and select **Create preview deployment**). In the deploy modal, toggle **Preview deployment** to ON.

<Frame>
  <img alt="Deploy modal with preview toggle enabled, showing preview label field and TTL picker" />
</Frame>

When the preview toggle is enabled, two additional fields appear:

* **Preview label**: an optional free-text label (max 30 characters) for grouping related previews. Use your branch name, PR number, or any identifier that helps you find this preview later. In the screenshot, the label is set to `pr-495`.
* **TTL**: how long the preview should live before automatic teardown. The default is 8 hours.

Click **Deploy preview** to create the deployment. The preview appears in the Deployments list on the build detail page with a **Preview** badge and an "Expires in X hours" indicator.

## Creating a preview from the API

To create a preview deployment programmatically, set `preview` to `true` in your `POST /v5/agentex/deployments` request. You can optionally include a `preview_label` and a custom `expires_at` timestamp.

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST $SGP_CLIENT_BASE_URL/v5/agentex/deployments \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
    -H "Content-Type: application/json" \
    -d '{
      "manifest_file": "kind: Agent\nagent:\n  name: my-agent\n...",
      "environment_config": "schema_version: v1\nenvironments:\n  dev.aws:\n    environment: dev\n...",
      "build_id": "build_abc123",
      "preview": true,
      "preview_label": "pr-482",
      "expires_at": "2026-05-21T02:00:00Z"
    }'
  ```

  ```python Python theme={null}
  from datetime import datetime, timezone, timedelta
  from scale_gp_beta import SGPClient

  client = SGPClient(
      api_key="your-api-key",
      account_id="your-account-id",
  )

  deployment = client.deploy.create(
      manifest_file="kind: Agent\nagent:\n  name: my-agent\n...",
      environment_config="schema_version: v1\nenvironments:\n  dev.aws:\n    environment: dev\n...",
      build_id="build_abc123",
      preview=True,
      preview_label="pr-482",
      expires_at=(datetime.now(timezone.utc) + timedelta(hours=8)).isoformat(),
  )
  print(f"Preview created: {deployment.id} (expires: {deployment.expires_at})")
  ```

  ```typescript TypeScript theme={null}
  import SGPClient from 'scale-gp';

  const client = new SGPClient({
    apiKey: process.env.SGP_API_KEY,
    accountId: process.env.SGP_ACCOUNT_ID,
  });

  const expiresAt = new Date(Date.now() + 8 * 60 * 60 * 1000).toISOString();

  const deployment = await client.deploy.create({
    manifest_file: "kind: Agent\nagent:\n  name: my-agent\n...",
    environment_config: "schema_version: v1\nenvironments:\n  dev.aws:\n    environment: dev\n...",
    build_id: "build_abc123",
    preview: true,
    preview_label: "pr-482",
    expires_at: expiresAt,
  });
  console.log(`Preview created: ${deployment.id} (expires: ${deployment.expires_at})`);
  ```
</CodeGroup>

<Note>
  Preview deployments always have an `expires_at` value. If you omit it, the API defaults to 8 hours from the time of creation. The timestamp must be timezone-aware (include `Z` or a UTC offset like `+00:00`) and must be in the future.
</Note>

Validation rules for preview-specific fields:

| Field           | Rule                                                                                                                        |
| --------------- | --------------------------------------------------------------------------------------------------------------------------- |
| `preview_label` | Can only be set when `preview` is `true`. Maximum 30 characters. Non-unique; multiple deployments can share the same label. |
| `expires_at`    | Can only be set when `preview` is `true`. Must be an ISO 8601 timestamp with timezone. Must be in the future.               |

## Listing previews by label

The `preview_label` field is a non-unique grouping label. Multiple preview deployments can share the same label, which is useful when you redeploy the same branch multiple times. To find the latest preview for a given label, query the deployments list endpoint with the `preview_label` filter and `limit=1`.

<CodeGroup>
  ```bash curl theme={null}
  curl -s "$SGP_CLIENT_BASE_URL/v5/agentex/deployments?preview_label=pr-482&limit=1" \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID"
  ```

  ```python Python theme={null}
  deployments = client.deploy.list(
      preview_label="pr-482",
      limit=1,
  )
  if deployments:
      latest = deployments[0]
      print(f"Latest preview: {latest.id} (status: {latest.status})")
  ```

  ```typescript TypeScript theme={null}
  const deployments = await client.deploy.list({
    preview_label: "pr-482",
    limit: 1,
  });
  if (deployments.length > 0) {
    const latest = deployments[0];
    console.log(`Latest preview: ${latest.id} (status: ${latest.status})`);
  }
  ```
</CodeGroup>

<Tip>
  This pattern is useful in CI pipelines. After deploying a preview, your pipeline can poll this endpoint with the branch name as the label to check deployment health before running integration tests.
</Tip>

## Managing preview deployments

Every preview deployment row in the Deployments list has a three-dot actions menu with operations specific to previews.

<Frame>
  <img alt="Three-dot actions menu on a preview deployment showing View details, Copy deployment ID, Redeploy as preview, Promote to production, and Tear down preview" />
</Frame>

The available actions are:

### Promote to production

Select **Promote to production** to create a new permanent deployment with the same manifest and environment configuration as the preview.

<Frame>
  <img alt="Promote to production confirmation dialog" />
</Frame>

Promoting creates a new production deployment. The original preview deployment is not automatically torn down. It continues to run until its TTL expires or you tear it down manually. This lets you verify the production deployment is healthy before cleaning up the preview.

### Tear down preview

Select **Tear down preview** to immediately delete the deployment and release its cluster resources. A confirmation dialog appears before the teardown proceeds.

<Frame>
  <img alt="Tear down confirmation dialog" />
</Frame>

Once torn down, the deployment transitions to a terminal state and its Kubernetes resources are removed. This action cannot be undone. If you need the same configuration again, use **Redeploy as preview** on a previous deployment or create a new preview from the build detail page.

### Redeploy as preview

Select **Redeploy as preview** to open the deploy modal pre-filled with the preview's configuration, including the `preview_label` from the original deployment. This is a shortcut for redeploying the same branch after a code change without re-entering the configuration.

## TTL and expiration

Every preview deployment has an `expires_at` timestamp that controls its lifetime. When you create a preview without specifying `expires_at`, the API defaults to 8 hours from the time of creation.

In the SGP dashboard, preview deployments display an "Expires in X hours" badge so you can see at a glance how much time remains. As the TTL approaches, the badge counts down.

When the `expires_at` time is reached, a background cleanup process automatically tears down the preview deployment and releases its cluster resources. You do not need to take any action. Expired previews are cleaned up automatically.

If you need more time with a preview, create a new preview deployment with a longer TTL. You cannot extend the `expires_at` of an existing preview. The new preview can reuse the same `preview_label` so it remains easy to find via the API.

<Note>
  Automatic teardown is handled by a background process that runs periodically. There may be a short delay between the `expires_at` timestamp and the actual resource cleanup, but the preview will stop serving traffic at expiration.
</Note>

## CI/CD integration

Preview deployments work well in CI/CD pipelines for automated branch testing. A typical pattern is:

1. On push to a feature branch, build the agent image
2. Deploy the image as a preview with `preview_label` set to the branch name
3. Poll the deployments endpoint to confirm the preview is healthy
4. Run integration tests against the preview endpoint
5. On merge to main, deploy to production (the preview expires naturally or can be torn down)

Here is a simplified workflow snippet showing the deploy and verify steps:

```bash theme={null}
# Build the JSON payload safely so YAML newlines and quotes are escaped
PAYLOAD=$(jq -n \
  --rawfile manifest manifest.yaml \
  --rawfile env environments.yaml \
  --arg build_id "$BUILD_ID" \
  --arg label "$BRANCH_NAME" \
  '{manifest_file: $manifest, environment_config: $env, build_id: $build_id, preview: true, preview_label: $label}')

# After a successful build, deploy as preview
DEPLOY_RESPONSE=$(curl -s -X POST $SGP_CLIENT_BASE_URL/v5/agentex/deployments \
  -H "x-api-key: $SGP_API_KEY" \
  -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
  -H "Content-Type: application/json" \
  -d "$PAYLOAD")

DEPLOY_ID=$(echo "$DEPLOY_RESPONSE" | jq -r '.id')

# Poll until the preview is healthy or failed
while true; do
  STATUS=$(curl -s "$SGP_CLIENT_BASE_URL/v5/agentex/deployments/$DEPLOY_ID" \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
    | jq -r '.status')
  echo "Preview deploy status: $STATUS"
  case $STATUS in
    healthy) echo "Preview is live"; break ;;
    failed|cancelled) echo "Preview deploy failed"; exit 1 ;;
  esac
  sleep 10
done
```

<Tip>
  Set the `preview_label` to your branch name or PR number in CI. This makes it straightforward to look up the latest preview for any branch using the list endpoint with the label filter, and it keeps the SGP dashboard organized when multiple branches have active previews.
</Tip>

## Next steps

* [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api): full request and response schemas for preview deployment fields
* [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): workflow templates for automated builds
* [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets): store and reference credentials used during deployments


# Secrets
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/secrets

Store credentials as SGP Cloud Secrets and reference them from manifest.yaml to inject environment variables at runtime.

## Overview

Agents typically need credentials to function: API keys for LLM providers, database connection strings, authentication tokens for external services. SGP provides a two-layer system for handling these securely.

1. **SGP Cloud Secrets** are account-level key-value pairs stored in your cloud provider's secret manager. You create them once, and any agent in your account can reference them.
2. **Manifest credentials** are per-agent mappings defined in `manifest.yaml` that bind an SGP Cloud Secret to an environment variable injected into the agent's container at runtime.

You create secrets through the SGP dashboard or the Secrets API, then reference them by name in your agent's `manifest.yaml`. At deploy time, SGP syncs the secret value into a Kubernetes secret and mounts it as the specified environment variable in the agent's pod. Your agent code reads the environment variable as it would any other.

## How secrets flow

The following diagram traces a secret from creation through to your running agent code.

```mermaid theme={null}
flowchart LR
    A[Create secret via UI or API] --> B[Value stored in cloud secret manager]
    B --> C[manifest.yaml references secret_name]
    C --> D[Deploy triggered]
    D --> E[SGP syncs value to K8s secret]
    E --> F[Env var injected in pod]
    F --> G[Agent reads env var]
```

The value you provide when creating a secret is written directly to the cloud provider's secret store (such as Azure Key Vault). It is never stored in the SGP database and is never returned by any API call. At deploy time, the deploy service reads the value from the cloud secret store, creates or updates a Kubernetes secret in the target namespace, and mounts it as the environment variable specified in your `manifest.yaml`.

## Managing secrets in the UI

Navigate to **Assets > Secrets** in the SGP dashboard. The secrets table lists all SGP Cloud Secrets for your account.

<Frame>
  <img alt="Secrets table showing account-level secrets with key, description, created by, and modified columns" />
</Frame>

Each row displays:

* **Key**: the unique identifier for the secret (for example, `anthropic-api-key` or `redis-url-secret`)
* **Description**: an optional note explaining what the secret is used for
* **Created by**: the user who created the secret
* **Modified**: when the secret was last updated

From this table you can edit a secret (to update its value or description) or delete it.

## Creating a secret

Click **Create Secret** at the top of the secrets table. Fill in the required fields in the dialog.

<Frame>
  <img alt="Create Secret dialog with key, value, and description fields" />
</Frame>

| Field           | Required | Description                                                                                                                                                   |
| --------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Key**         | Yes      | A unique name for the secret. Must be lowercase alphanumeric characters, hyphens, or dots, with a maximum length of 253 characters (Kubernetes naming rules). |
| **Value**       | Yes      | The credential value. This is encrypted and stored in the cloud secret manager. It is never displayed again after creation.                                   |
| **Description** | No       | A human-readable note. Recommended so that other team members understand the secret's purpose.                                                                |

Click **Create Secret** to save. The new secret appears in the table immediately and is available for agents to reference.

<Note>
  Secret values are write-only. After creation, you can update the value but you cannot retrieve it. If you lose the original value, you must generate a new credential from the source system and update the secret.
</Note>

## Deleting a secret

Click the delete action on a secret's row. A confirmation dialog requires you to type the secret's key to proceed.

<Frame>
  <img alt="Delete confirmation dialog requiring the secret name to be typed for confirmation" />
</Frame>

<Warning>
  Deleting a secret is irreversible. Any agent whose `manifest.yaml` references the deleted secret will fail on its next deployment because the secret can no longer be resolved. Verify that no agents depend on a secret before deleting it.
</Warning>

## Secrets API

You can manage secrets programmatically through the SGP Cloud Secrets API. The following examples cover the most common operations.

### Create a secret

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST "$SGP_CLIENT_BASE_URL/v5/sgp/secrets" \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
    -H "Content-Type: application/json" \
    -d '{
      "key": "datadog-api-key",
      "value": "dd-api-xxxxxxxxxxxxxxxxxxxxxxxxxx",
      "description": "Datadog API key for agent telemetry"
    }'
  ```

  ```python Python theme={null}
  from scale_gp_beta import SGPClient

  client = SGPClient()

  secret = client.secrets.create(
      key="datadog-api-key",
      value="dd-api-xxxxxxxxxxxxxxxxxxxxxxxxxx",
      description="Datadog API key for agent telemetry",
  )
  print(secret.id, secret.key)
  ```

  ```typescript TypeScript theme={null}
  import SGPClient from 'scale-gp';

  const client = new SGPClient({
    apiKey: process.env.SGP_API_KEY,
    accountId: process.env.SGP_ACCOUNT_ID,
  });

  const secret = await client.secrets.create({
    key: "datadog-api-key",
    value: "dd-api-xxxxxxxxxxxxxxxxxxxxxxxxxx",
    description: "Datadog API key for agent telemetry",
  });
  console.log(secret.id, secret.key);
  ```
</CodeGroup>

The response includes the secret's `id`, `key`, `description`, and audit metadata. The `value` field is never returned.

```json theme={null}
{
  "id": "sec_abc123",
  "key": "datadog-api-key",
  "description": "Datadog API key for agent telemetry",
  "cloud_secret_path": "/sgp/secrets/datadog-api-key",
  "created_by": "usr_def456",
  "created_at": "2026-05-20T14:30:00Z",
  "updated_by": "usr_def456",
  "updated_at": "2026-05-20T14:30:00Z",
  "object": "sgp_cloud_secret"
}
```

### Update a secret

Use `PATCH /v5/sgp/secrets/{secret_id}` to rotate a value or update the description.

<CodeGroup>
  ```bash cURL theme={null}
  curl -X PATCH "$SGP_CLIENT_BASE_URL/v5/sgp/secrets/sec_abc123" \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
    -H "Content-Type: application/json" \
    -d '{
      "value": "dd-api-yyyyyyyyyyyyyyyyyyyyyyyyyy",
      "description": "Datadog API key for agent telemetry (rotated May 2026)"
    }'
  ```

  ```python Python theme={null}
  from scale_gp_beta import SGPClient

  client = SGPClient()

  secret = client.secrets.update(
      "sec_abc123",
      value="dd-api-yyyyyyyyyyyyyyyyyyyyyyyyyy",
      description="Datadog API key for agent telemetry (rotated May 2026)",
  )
  print(secret.key, secret.updated_at)
  ```

  ```typescript TypeScript theme={null}
  import SGPClient from 'scale-gp';

  const client = new SGPClient({
    apiKey: process.env.SGP_API_KEY,
    accountId: process.env.SGP_ACCOUNT_ID,
  });

  const secret = await client.secrets.update("sec_abc123", {
    value: "dd-api-yyyyyyyyyyyyyyyyyyyyyyyyyy",
    description: "Datadog API key for agent telemetry (rotated May 2026)",
  });
  console.log(secret.key, secret.updatedAt);
  ```
</CodeGroup>

### List secrets

<CodeGroup>
  ```bash cURL theme={null}
  curl "$SGP_CLIENT_BASE_URL/v5/sgp/secrets" \
    -H "x-api-key: $SGP_API_KEY" \
    -H "x-selected-account-id: $SGP_ACCOUNT_ID"
  ```

  ```python Python theme={null}
  from scale_gp_beta import SGPClient

  client = SGPClient()

  secrets = client.secrets.list()
  for secret in secrets:
      print(secret.key, secret.description)
  ```

  ```typescript TypeScript theme={null}
  import SGPClient from 'scale-gp';

  const client = new SGPClient({
    apiKey: process.env.SGP_API_KEY,
    accountId: process.env.SGP_ACCOUNT_ID,
  });

  const secrets = await client.secrets.list();
  for (const secret of secrets) {
    console.log(secret.key, secret.description);
  }
  ```
</CodeGroup>

For the full API schema including pagination, filtering, and delete operations, see the [SGP Cloud Secrets API reference](/reference/v5/sgp-cloud-secrets).

## Referencing secrets in manifest.yaml

Once a secret exists in the SGP secrets store, you reference it from your agent's `manifest.yaml` in the `agent.credentials` array. Each entry maps an SGP Cloud Secret to an environment variable that your agent code can read.

```yaml theme={null}
agent:
  credentials:
    - env_var_name: ANTHROPIC_API_KEY
      secret_name: anthropic-api-key
      secret_key: api-key
    - env_var_name: OPENAI_API_KEY
      secret_name: openai-api-key
      secret_key: api-key
    - env_var_name: DATABASE_URL
      secret_name: redis-url-secret
      secret_key: url
```

| Field          | Description                                                                                                                                   |
| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `env_var_name` | The environment variable name injected into the agent's container. Your code reads this with `os.environ["ANTHROPIC_API_KEY"]` or equivalent. |
| `secret_name`  | The `key` of the SGP Cloud Secret as it appears in the secrets table. Must match exactly.                                                     |
| `secret_key`   | The sub-key within the Kubernetes secret object. This is typically `api-key`, `token`, or `url`, depending on how the secret was provisioned. |

At deploy time, SGP resolves each `secret_name` to its corresponding cloud secret, syncs the value into a Kubernetes secret in the agent's namespace, and mounts the value at the specified `secret_key` as the environment variable `env_var_name` in the pod.

<Info>
  The `secret_key` field refers to the key within the Kubernetes secret data map, not a secondary lookup within the cloud secret store. Each SGP Cloud Secret maps to one Kubernetes secret, and `secret_key` selects which data entry within that Kubernetes secret to mount.
</Info>

## Rotating a secret

To rotate a credential:

1. Generate a new credential value from the source system (for example, your LLM provider's dashboard or your database admin panel).
2. Update the SGP Cloud Secret with the new value using the API (`PATCH /v5/sgp/secrets/{secret_id}`) or the **Edit** action in the secrets table UI.
3. Redeploy any agents that reference the rotated secret. The next deployment picks up the new value automatically.

Existing running pods continue to use the previous value until they are replaced by a new deployment. Updating a secret does not trigger automatic redeployment. You must explicitly redeploy each affected agent.

<Tip>
  To minimize downtime during rotation, update the secret value first, then redeploy agents one at a time. If the new credential is invalid, only the redeployed agent is affected, and you can roll back by redeploying the previous build.
</Tip>

## Security model

SGP Cloud Secrets are designed around a write-only, least-privilege model:

* **Write-only values**: secret values are never returned by any API call or displayed in the UI after creation. The SGP database stores only metadata (key, description, audit fields). The actual value lives exclusively in the cloud provider's secret manager.
* **Encrypted at rest**: values are stored using the cloud provider's native encryption (for example, Azure Key Vault with platform-managed keys).
* **Injected as Kubernetes secrets**: at deploy time, values are synced into Kubernetes secrets and mounted as environment variables. They are not written to disk or config maps in the cluster.
* **Role-based access**: only users with secret management permissions can create, update, or delete secrets. Read access to secret metadata (key and description) is available to users who can view the account's assets.
* **Audit trail**: every create, update, and delete operation is recorded with the acting user and timestamp.

## Next steps

* [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): full `manifest.yaml` reference including the `credentials` section
* [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api): programmatic access to builds, deployments, and secrets
* [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments): ephemeral deployments for branch and PR testing


# sgpctl CLI
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/sgpctl

Build, deploy, and observe Agentex agents from your terminal with sgpctl agentex-cloud.

## Overview

`sgpctl` is the command-line interface for managing Agentex cloud builds and deployments. It covers the same operations available through the SGP dashboard and the API/SDK, packaged for terminal workflows.

The CLI is organized under `sgpctl agentex-cloud` with two subcommand groups:

* **`build`**: submit cloud builds, list builds, stream build logs, and cancel running builds.
* **`deploy`**: submit deployments, list deployments, stream deploy logs, and watch rollouts in real time.

Every command outputs structured data (JSON or Rich tables) that you can pipe into other tools or inspect directly.

## Setup

### Install sgpctl

**Homebrew (macOS):**

```bash theme={null}
brew tap scaleapi/sgpctl https://github.com/scaleapi/sgpctl
brew install sgpctl
```

**Binary download (Linux/Windows):**

Download the latest binary for your platform from the [sgpctl GitHub releases](https://github.com/scaleapi/sgpctl/releases/latest).

**From the SGP monorepo:**

If you have the SGP repository cloned locally, you can build and install from source:

```bash theme={null}
cd services/sgpctl && poetry build && pipx install --force dist/sgpctl-*.whl
```

### Configure environment variables

All `sgpctl agentex-cloud` commands require three environment variables:

```bash theme={null}
export SGP_API_KEY="your-api-key"
export SGP_ACCOUNT_ID="your-account-id"
export SGP_BASE_URL="https://api.your-sgp-instance.scale.com"
```

<Note>
  All three environment variables are required. Commands will fail immediately if any of them is missing.
</Note>

You can add these exports to your shell profile (`~/.bashrc`, `~/.zshrc`) or load them from a `.env` file in your CI pipeline.

## Build commands

Build commands live under `sgpctl agentex-cloud build`. They let you submit new image builds, query build status, stream logs, and cancel running builds.

### Submit a build

Package your agent source code and submit it for a cloud build:

```bash theme={null}
sgpctl agentex-cloud build submit \
  --manifest ./manifest.yaml \
  --image-name my-agent \
  --tag v1.2.0
```

The `submit` command reads your `manifest.yaml`, packages the build context using `prepare_cloud_build_context()`, uploads the resulting tar.gz archive, and creates a build via `POST /v5/builds`.

On success, the command prints the build details as JSON:

```json theme={null}
{
  "build_id": "8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02",
  "agent_name": "my-agent",
  "status": "queued",
  "image_name": "my-agent",
  "image_tag": "v1.2.0"
}
```

**Options:**

| Flag                 | Description                                             |
| -------------------- | ------------------------------------------------------- |
| `--manifest`, `-m`   | Path to `manifest.yaml` (required)                      |
| `--image-name`, `-i` | Repository name for the built image (required)          |
| `--tag`, `-t`        | Image tag (default: `latest`)                           |
| `--agent-name`, `-n` | Agent name override (inferred from manifest if omitted) |
| `--build-arg`, `-b`  | Build argument as `KEY=VALUE` (repeatable)              |
| `--platform`, `-p`   | Target platform (e.g. `linux/amd64`)                    |

You can pass multiple build arguments by repeating the flag:

```bash theme={null}
sgpctl agentex-cloud build submit \
  --manifest ./manifest.yaml \
  --image-name my-agent \
  --tag v1.2.0 \
  --build-arg PYTHON_VERSION=3.11 \
  --build-arg INSTALL_DEV_DEPS=false
```

### Get build details

Retrieve the full details of a specific build:

```bash theme={null}
sgpctl agentex-cloud build get 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02
```

Returns the build object as JSON, including status, image URL, timestamps, and associated metadata.

### List builds

List recent builds, optionally filtered by agent name:

```bash theme={null}
sgpctl agentex-cloud build list --agent-name my-agent --limit 10
```

Example output:

| Build ID                               | Agent Name | Build Status | Image Name | Image Tag | Image URL                | Created At          |
| -------------------------------------- | ---------- | ------------ | ---------- | --------- | ------------------------ | ------------------- |
| `8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02` | my-agent   | success      | my-agent   | v1.2.0    | registry.sgp.scale.com/… | 2026-05-20 14:32:10 |
| `6d1b4e82-f3a9-4c8b-a2e5-9d7f1b3c8a04` | my-agent   | success      | my-agent   | v1.1.0    | registry.sgp.scale.com/… | 2026-05-18 09:15:43 |
| `2c7f9a31-d5e8-4a1f-b6c3-8e2d4f7a9b01` | my-agent   | failed       | my-agent   | v1.0.1    |                          | 2026-05-17 16:22:05 |

**Options:**

| Flag                 | Description                             |
| -------------------- | --------------------------------------- |
| `--agent-name`, `-n` | Filter by agent name                    |
| `--limit`, `-l`      | Maximum number of results (default: 20) |
| `--sort-order`       | Sort direction: `asc` or `desc`         |

### Stream build logs

Stream real-time build logs for a running or completed build:

```bash theme={null}
sgpctl agentex-cloud build logs 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02
```

Logs are delivered via Server-Sent Events (SSE) and print to stdout as they arrive:

```
[2026-05-20 14:32:15] Step 1/8 : FROM python:3.11-slim
[2026-05-20 14:32:16] ---> Using cache
[2026-05-20 14:32:16] Step 2/8 : WORKDIR /app
[2026-05-20 14:32:17] Step 3/8 : COPY requirements.txt .
[2026-05-20 14:32:18] Step 4/8 : RUN pip install -r requirements.txt
[2026-05-20 14:32:45] Step 5/8 : COPY . .
[2026-05-20 14:32:46] Step 6/8 : RUN python -m compileall .
[2026-05-20 14:32:48] Step 7/8 : EXPOSE 8080
[2026-05-20 14:32:48] Step 8/8 : CMD ["python", "main.py"]
[2026-05-20 14:32:50] Successfully built a1b2c3d4e5f6
[2026-05-20 14:32:52] Successfully tagged registry.sgp.scale.com/my-agent:v1.2.0
```

The command exits when the build completes or the timeout is reached.

| Flag        | Description                                     |
| ----------- | ----------------------------------------------- |
| `--timeout` | Maximum seconds to wait for logs (default: 300) |

### Cancel a build

Cancel a build that is queued or running:

```bash theme={null}
sgpctl agentex-cloud build cancel 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02
```

The build transitions to `cancelled` status. Builds that have already completed cannot be cancelled.

## Deploy commands

Deploy commands live under `sgpctl agentex-cloud deploy`. They let you submit deployments, check deployment status, stream logs, and watch rollouts.

### Submit a deployment

Deploy an agent using a completed build:

```bash theme={null}
sgpctl agentex-cloud deploy submit \
  --agent-path ./my-agent \
  --env production \
  --build-id 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02
```

The `submit` command reads `manifest.yaml` and `environments.yaml` from the directory specified by `--agent-path`, then creates a deployment via `POST /v5/agentex/deployments`.

Alternatively, specify an image directly instead of a build ID:

```bash theme={null}
sgpctl agentex-cloud deploy submit \
  --agent-path ./my-agent \
  --env staging \
  --image-name my-agent \
  --tag v1.2.0
```

**Options:**

| Flag                 | Description                                                                           |
| -------------------- | ------------------------------------------------------------------------------------- |
| `--agent-path`, `-a` | Path to agent directory containing `manifest.yaml` and `environments.yaml` (required) |
| `--env`, `-e`        | Environment key from `environments.yaml` (required)                                   |
| `--build-id`, `-b`   | Build ID to deploy                                                                    |
| `--image-name`, `-i` | Image repository name (alternative to `--build-id`)                                   |
| `--tag`, `-t`        | Image tag (used with `--image-name`)                                                  |
| `--watch`, `-w`      | Enter the live TUI dashboard after submitting                                         |

<Note>
  You must provide either `--build-id` or both `--image-name` and `--tag`. The command will reject the request if neither or both are specified.
</Note>

### Watch a deployment roll out

Pass `--watch` to `deploy submit` to enter a fullscreen Rich dashboard immediately after the deployment is created:

```bash theme={null}
sgpctl agentex-cloud deploy submit \
  --agent-path ./my-agent \
  --env production \
  --build-id 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 \
  --watch
```

The TUI dashboard shows three sections: a status header at the top (agent name, environment, deployment ID, current status), a Kubernetes events panel on the left, and a container logs panel on the right. Both panels update in real time as the deployment progresses.

The dashboard auto-refreshes and exits automatically when the deployment reaches a terminal state (`healthy` or `failed`). Press `q` or `Ctrl+C` to exit early.

### Get deployment details

Retrieve the full details and events for a specific deployment:

```bash theme={null}
sgpctl agentex-cloud deploy get 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05
```

Returns the deployment object as JSON, including status, events timeline, and configuration.

### List deployments

List recent deployments:

```bash theme={null}
sgpctl agentex-cloud deploy list --limit 10
```

Example output:

| ID                                     | Agent       | Status  | Namespace      | Created At          |
| -------------------------------------- | ----------- | ------- | -------------- | ------------------- |
| `4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05` | my-agent    | healthy | prod-agents    | 2026-05-20 14:33:00 |
| `9e2d5f81-a3c7-4b9e-d1f4-6c8a2e5b7d03` | my-agent    | healthy | staging-agents | 2026-05-19 11:20:15 |
| `1b8c3a62-f4d9-4e7c-a5b8-3d9f2c6e1a08` | support-bot | failed  | prod-agents    | 2026-05-19 09:45:33 |

**Options:**

| Flag            | Description                             |
| --------------- | --------------------------------------- |
| `--limit`, `-l` | Maximum number of results (default: 20) |

## Streaming logs

Both build and deploy commands support log streaming, but they use different transport mechanisms.

### Build logs (SSE)

Build logs stream via Server-Sent Events. The `build logs` command opens a persistent connection and prints each log line as it arrives:

```bash theme={null}
sgpctl agentex-cloud build logs 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 --timeout 600
```

The connection closes when the build completes or the timeout expires. This is useful for watching a build in progress from your terminal.

### Deploy logs (cursor-paged polling)

Deploy logs use cursor-based polling. The `deploy logs` command fetches new log lines at a configurable interval:

```bash theme={null}
sgpctl agentex-cloud deploy logs 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05
```

By default, the output renders in a Rich Live TUI panel that updates in place. For CI environments or piping to a file, use the `--plain` flag:

```bash theme={null}
sgpctl agentex-cloud deploy logs 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05 --plain
```

In plain mode, each log line is written to stdout as a single line with no formatting, making it suitable for CI log capture or `grep`:

```
2026-05-20T14:33:24Z Starting application...
2026-05-20T14:33:25Z Loading model weights...
2026-05-20T14:33:28Z Model loaded in 3.2s
2026-05-20T14:33:28Z Health check endpoint ready
2026-05-20T14:33:28Z Listening on 0.0.0.0:8080
```

**Options:**

| Flag              | Description                                    |
| ----------------- | ---------------------------------------------- |
| `--poll-interval` | Seconds between log fetches (default: 2)       |
| `--plain`         | Plain text output, no TUI (recommended for CI) |

## Common workflows

### Ship from laptop

Build an image and deploy it in one sequence:

```bash theme={null}
# Submit the build
sgpctl agentex-cloud build submit \
  --manifest ./manifest.yaml \
  --image-name my-agent \
  --tag v1.2.0

# Stream logs until the build completes
sgpctl agentex-cloud build logs 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 --timeout 600

# Deploy and watch the rollout
sgpctl agentex-cloud deploy submit \
  --agent-path ./my-agent \
  --env production \
  --build-id 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 \
  --watch
```

After the build completes, the deploy command with `--watch` gives you a live view of the rollout from start to finish.

### Check what is deployed

List active deployments to see which builds are currently serving traffic:

```bash theme={null}
sgpctl agentex-cloud deploy list
```

To get full details on a specific deployment, including the image tag and environment configuration:

```bash theme={null}
sgpctl agentex-cloud deploy get 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05
```

### Roll back to a previous build

Redeploy a known-good build by referencing its build ID:

```bash theme={null}
sgpctl agentex-cloud deploy submit \
  --agent-path ./my-agent \
  --env production \
  --build-id 6d1b4e82-f3a9-4c8b-a2e5-9d7f1b3c8a04 \
  --watch
```

This creates a new deployment using the image from the previous build. The existing deployment is replaced when the new one becomes healthy.

### Tail logs on a running deployment

Stream container logs from a deployment that is already running:

```bash theme={null}
sgpctl agentex-cloud deploy logs 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05
```

Use `--plain` to capture logs in a file:

```bash theme={null}
sgpctl agentex-cloud deploy logs 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05 --plain > deploy.log
```

## Next steps

* [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api): programmatic access to builds and deployments
* [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): automate builds triggered by git push
* [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets): store and reference credentials used during builds and deployments


# UI Walkthrough
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/ui-walkthrough

Step-by-step guide to building and deploying agents through the SGP web interface.

## Overview

This page walks through the Cloud Build and Deploy workflow in the SGP dashboard. The navigation flow is:

```
Agents grid → Agent detail → Build detail → Deploy modal → Deployment detail
```

**Prerequisites:**

* An SGP account with access to the Agents tab
* At least one Agentex agent registered to your account
* A `manifest.yaml` and `environments.yaml` prepared for your agent (see the [CI/CD guide](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd) for file format details)

## Viewing your agents

Open the **Agents** tab from the top navigation bar. This shows all Agentex agents registered to your account.

<Frame>
  <img alt="Agents grid showing registered Agentex agents with status badges and descriptions" />
</Frame>

Click any agent card to open its detail page.

## Agent detail and build history

The agent detail page has a metadata header at the top and the build history timeline below.

<Frame>
  <img alt="Agent detail page showing header metadata and build history timeline" />
</Frame>

The action buttons in the header link to the agent's live endpoint (**Open Agent**), its request traces (**View Traces**), and its performance profiler (**View Profiler**). Open Agent is only available when the agent has an active deployment.

The **Build History** lists every build for this agent, most recent first. The status badges map to:

| Status        | Meaning                                                     |
| ------------- | ----------------------------------------------------------- |
| **Building**  | Image build is in progress                                  |
| **Deploying** | Build succeeded, deployment is actively rolling out         |
| **Error**     | Build failed during the image build phase                   |
| **Inactive**  | Build succeeded but is not currently deployed               |
| **Success**   | Build completed successfully (visible on build detail page) |

Click **+ Add Build** to trigger a new cloud build from the UI.

<Note>
  Most teams use CI/CD pipelines for automated builds and reserve the UI button for ad-hoc builds during development.
</Note>

Click any build row to open its build detail page.

## Deploying a build

The build detail page shows build metadata, build logs, and all deployments created from this build.

<Frame>
  <img alt="Build detail page showing a successful build with no deployments and a Create deployment button" />
</Frame>

The **Build Logs** section is collapsible and includes a search bar for filtering output. Use the build duration in the header to distinguish early failures (missing Dockerfile, bad config) from late failures (dependency errors).

To deploy, click **Create deployment** in the Deployments section to open the deploy modal.

<Frame>
  <img alt="Deploy modal showing preview toggle, manifest file section, and environment variables section" />
</Frame>

The modal has two collapsible YAML sections: **Manifest File** (`manifest.yaml`) and **Environment Variables** (`environments.yaml`).

* On your first deployment, paste your `manifest.yaml` and `environments.yaml` content into the respective sections.
* On subsequent deployments, both sections pre-populate from the last deployment. You can deploy without expanding them if no changes are needed.
* Enable the **Preview deployment** toggle to create an ephemeral deployment with an automatic TTL instead of a production deployment. See [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments).

<Warning>
  Deployments that reference missing secrets will fail during the Helm release phase. Ensure secrets are created in the SGP secret store before deploying. See [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets).
</Warning>

## Monitoring a deployment

The build detail page lists all deployments from this build. Reload the page to see status updates.

<Frame>
  <img alt="Build detail page showing multiple deployments with mixed statuses" />
</Frame>

Production deployments show a gray badge; preview deployments show a purple badge with the PR number and an expiration countdown. Preview deployments are automatically torn down when the TTL expires.

Click any deployment row to inspect its event timeline and logs.

<Frame>
  <img alt="Deployment detail page showing metadata, deployment events timeline, and deploy logs section" />
</Frame>

The **Deployment Events** section shows the Kubernetes event sequence for the rollout. A successful deployment progresses through: Scheduled → Pulling → Pulled → Created → Started → ScalingReplicaSet → ReleaseSuccessful.

The **Deploy Logs** section shows container runtime output. If the deployment is healthy and the agent produces no stdout on startup, this section may show "No logs available."

### Diagnosing a failing deployment

A gap in the event timeline tells you where the problem is:

| Last event seen                              | Likely cause                                                              |
| -------------------------------------------- | ------------------------------------------------------------------------- |
| **Scheduled**                                | Image could not be pulled. Check the image reference and registry access. |
| **Pulling**                                  | Image pull is timing out. Large images or registry rate limits.           |
| **Started**                                  | Health check is not passing. Check Deploy Logs for runtime errors.        |
| **ScalingReplicaSet** (no ReleaseSuccessful) | Helm timeout or release hook issue.                                       |

<Tip>
  If a deployment stays in Deploying longer than expected, check Deployment Events. Missing events after Scheduled usually mean node resource constraints or image pull failures.
</Tip>

## Diagnosing a failed build

Failed builds show a **Failed** badge on the build detail page and an **Error** badge in the Build History list (same state, different labels).

<Frame>
  <img alt="Failed build detail page showing error status and build logs section" />
</Frame>

The **Create deployment** button is disabled for failed builds.

Check the **Build Logs** for error output. A short build duration (under 30 seconds) usually points to a configuration error (missing Dockerfile, invalid manifest) rather than a code error.

Common causes:

* **Dockerfile not found**: `manifest.yaml` references a path that doesn't exist in the build context
* **Dependency install failure**: `pip install`, `npm install`, etc. fails
* **Build context too large**: add a `.dockerignore` to exclude unnecessary files
* **Invalid manifest**: syntax errors or invalid field values in `manifest.yaml`
* **Missing secrets**: the manifest references a secret that doesn't exist
* **Base image unavailable**: the Dockerfile's base image can't be pulled
* **Timeout**: build exceeded the maximum duration

<Tip>
  If builds fail repeatedly, verify the agent builds locally first with `docker build`. This surfaces issues faster than iterating through the cloud pipeline.
</Tip>

## Next steps

* [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments): ephemeral branch deployments for PR testing
* [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets): store and reference credentials
* [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): automate builds triggered by git push
* [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api): programmatic access to builds and deployments


# Introduction to Agentex
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/overview

Learn about Agentex agent types and how to build them

Welcome to the Agentex client documentation. This guide will help you get started with building and interacting with Agentex agents.

## Quick Navigation

* 📘 **[Sync Agent Guide →](/docs/v5/agents/agentex/types/sync-agent)** - For simple, direct interactions
* 📗 **[Agentic (ACP) Agent Guide →](/docs/v5/agents/agentex/types/agentic-agent)** - For complex workflows and production apps

**Not sure which to use?** Learn more about [choosing the right agent type](https://agentex.sgp.scale.com/docs/development_guides/tutorials/).


# Agentic Agent
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/types/agentic-agent

How to interact with agentic (async) Agentex agents using the Agentex SDK

## Overview

Agentic agents use an **agentic ACP** architecture that:

* Supports both streaming and non-streaming responses
* Requires explicit task creation before sending events
* Ideal for complex workflows and long-running operations

### Key API Method: `send_event()`

For agentic (ACP) agents, you always use the **`send_event()`** method to communicate with the agent. This is different from sync agents, which use `send_message()` for direct communication.

```python theme={null}
# Agentic agents use send_event()
# 1. Create a task first (REQUIRED for agentic agents)
task = await client.agents.create_task(agent_id=agent.id, params={})

# 2. Send events to the task
await client.agents.send_event(
    agent_id=agent.id,
    params={
        "task_id": task.id,
        "content": {"type": "text", "author": "user", "content": "Hello!"}
    }
)
```

**Key Differences from Sync Agents:**

* **Task creation required** - Must create a task before sending events (sync agents don't need this)
* **Event-based communication** - Send events to tasks, not direct messages to agent names
* **Asynchronous processing** - Events are processed asynchronously, poll or stream for responses

The event-based model enables asynchronous processing, state management across multiple turns, and complex workflow orchestration.

### ACP Types: Base vs Temporal

Agentic agents can be implemented using two different backend types:

* **Base ACP** - Simple event-driven architecture, suitable for learning and simple use cases
* **Temporal** - Robust workflow engine for production, handles race conditions automatically

**Important:** From the client's perspective, both base ACP and Temporal agents use the **exact same API** (both use `send_event()`). The choice between base and Temporal is purely a server-side implementation detail that is transparent to client code. You can switch an agent from base to Temporal (or vice versa) without changing any client code.

## Setup

### Prerequisites

```bash theme={null}
pip install agentex-sdk
```

### Environment Variables

```bash theme={null}
export AGENTEX_API_KEY="your-api-key"
export AGENTEX_BASE_URL="agentex-base-url"
export SGP_ACCOUNT_ID="your-account-id"
```

### Initialize the Client

```python theme={null}
import asyncio
from agentex import AsyncAgentex
from agentex.types.text_content import TextContent

# Initialize the async Agentex client
client = AsyncAgentex(
    base_url=os.environ.get("AGENTEX_BASE_URL"),
    default_headers={
        "x-api-key": os.enviorn.get("AGENTEX_API_KEY"),
        "x-selected-account-id": os.environ.get("SGP_ACCOUNT_ID")
    }
)
```

## Basic Usage

### 1. Get Agent Information

```python theme={null}
from agentex.types import Agent

# List all agents
agents = await client.agents.list()
for agent in agents:
    print(f"Agent: {agent.name} - {agent.id} - {agent.acp_type}")

# Retrieve a specific agent by name
AGENT_NAME = "<your-agent-name>"  # Replace with your agent name
agent = await client.agents.retrieve_by_name(agent_name=AGENT_NAME)

print(f"Agent ID: {agent.id}")
print(f"Agent Type: {agent.acp_type}")
print(f"Description: {agent.description}")
```

### 2. Create a Task

Agentic agents require a task to be created before sending events:

```python theme={null}
import uuid

# Create a new task
create_task_response = await client.agents.create_task(
    agent_id=agent.id,
    params={
        "name": f"{uuid.uuid4()}-task",
        "params": {}
    }
)

task = create_task_response.result

print(f"Task ID: {task.id}")
print(f"Task Status: {task.status}")
```

### 3. Send an Event

```python theme={null}
# Create event content
event_content = TextContent(
    type="text",
    author="user",
    content="Hello! Can you explain what quantum computing is?"
)

# Send the event to the task
event_response = await client.agents.send_event(
    agent_id=agent.id,
    params={
        "task_id": task.id,
        "content": event_content
    }
)

event = event_response.result
print(f"Event ID: {event.id}")
print(f"Event created at: {event.created_at}")
```

### 4. Receive Messages (Polling)

Since agentic agents process events asynchronously, you need to poll for messages:

```python theme={null}
from datetime import datetime

has_received_response = False
messages = []
timeout = 30
start_time = datetime.now()

# Poll for messages
while not has_received_response and (datetime.now() - start_time).seconds < timeout:
    messages = await client.messages.list(task_id=task.id)

    for message in messages:
        content = message.content
        if content.author == "agent":
            has_received_response = True
            break

    await asyncio.sleep(1)

# Print received messages
print(f"\nReceived {len(messages)} messages:\n")
for message in messages:
    if message.content and message.content.type == "text":
        print(f"[{message.content.author}]: {message.content.content}")
        print("-" * 80)
```

## Streaming Messages

### Helper Function for Streaming

Use the built-in helper to subscribe to task messages when testing in a local environmet:

```python theme={null}
from agentex.lib.utils.dev_tools import subscribe_to_async_task_messages

async def send_and_stream_event(task_id: str, user_message: str):
    """Send an event and stream the agent's response."""

    # Send the event
    event_content = TextContent(
        type="text",
        author="user",
        content=user_message
    )

    await client.agents.send_event(
        agent_id=agent.id,
        params={
            "task_id": task_id,
            "content": event_content
        }
    )

    print(f"User: {user_message}\n")

    # Subscribe to task messages
    messages = await subscribe_to_async_task_messages(
        client=client,
        task_id=task_id,
        timeout=30,
        max_retries=10,
    )

    return messages

# Usage
await send_and_stream_event(
    task.id,
    "Can you tell me about machine learning?"
)
```

### Custom Streaming Implementation

For more control over streaming or for production environments, implement custom streaming logic:

```python theme={null}
import json

async def custom_send_stream_event(task_id: str, user_message: str):
    """Send an event and stream the agent's response with custom logic."""

    # Send the event
    event_content = TextContent(
        type="text",
        author="user",
        content=user_message
    )

    await client.agents.send_event(
        agent_id=agent.id,
        params={
            "task_id": task_id,
            "content": event_content
        }
    )

    print(f"User: {user_message}\n")
    print("Agent: ", end="", flush=True)

    # Stream events
    async with client.tasks.with_streaming_response.stream_events(
        task_id=task_id,
        timeout=30
    ) as stream:
        try:
            response_message = ""

            async for task_message_update in stream.iter_lines():
                if task_message_update.startswith("data: "):
                    # Remove 'data: ' prefix
                    task_message_update_json = task_message_update.strip()[6:]
                    task_message_update_data = json.loads(task_message_update_json)

                    if task_message_update_data.get("type") == "delta":
                        delta = task_message_update_data.get("delta", {})
                        if delta.get("type") == "text":
                            text_delta = delta.get("text_delta", "")
                            response_message += text_delta
                            print(text_delta, end="", flush=True)

                    elif task_message_update_data.get("type") == "done":
                        print("\n")
                        break

        except Exception as e:
            print(f"\nError streaming messages: {e}")

# Usage
await custom_send_stream_event(
    task.id,
    "Can you tell me about tennis?"
)
```

## Multi-Turn Conversations

Agentic agents automatically maintain conversation history:

```python theme={null}
# Create a new task for the conversation
conversation_task_id = str(uuid.uuid4())
create_task_response = await client.agents.create_task(
    agent_id=agent.id,
    id=conversation_task_id,
    params={}
)

conversation_task = create_task_response.result

# Define a multi-turn conversation
messages = [
    "Can you tell me about machine learning?",
    "What are some practical applications?",
    "How do I get started learning it?",
]

# Send each message and stream responses
for message in messages:
    print(f"Task ID: {conversation_task.id}")
    await custom_send_stream_event(conversation_task.id, message)
    print("=" * 80 + "\n")
```

### View Conversation History

Retrieve all messages from a task:

```python theme={null}
# Get all messages from the task
all_messages = await client.messages.list(task_id=conversation_task.id)

print("\n" + "=" * 80)
print("COMPLETE CONVERSATION HISTORY")
print("=" * 80 + "\n")

for idx, message in enumerate(all_messages, 1):
    if message.content and message.content.type == "text":
        author = message.content.author.upper()
        print(f"{idx}. [{author}]")
        print(f"{message.content.content}\n")
        print("-" * 80)
```

## Managing Tasks

### Check Task Status

```python theme={null}
# Retrieve task status
task_info = await client.tasks.retrieve(task_id=task.id)

print(f"Task ID: {task_info.id}")
print(f"Status: {task_info.status}")
print(f"Status Reason: {task_info.status_reason}")
print(f"Created At: {task_info.created_at}")
print(f"Updated At: {task_info.updated_at}")
```

### List All Tasks for an Agent

```python theme={null}
# List tasks
tasks = await client.tasks.list(agent_id=agent.id)

for task in tasks:
    print(f"Task {task.id}: {task.status}")
```

## State Management

Agentic agents can maintain state across events:

```python theme={null}
# List states for an agent
states = await client.states.list(agent_id=agent.id)

for state in states:
    print(f"State ID: {state.id}")
    print(f"Task ID: {state.task_id}")
    print(f"State Data: {state.state}")

```


# Sync Agent
Source: https://docs.gp.scale.com/docs/v5/agents/agentex/types/sync-agent

How to interact with synchronous (sync) Agentex agents using the Agentex SDK

## Overview

Sync agents use a **sync ACP** architecture that:

* Uses direct message-based communication via RPC

* Returns responses with the request (synchronously)

* Supports both streaming and non-streaming responses

### Key API Method: `send_message(), send_message_stream()`

For sync agents, you always use the **`send_message()`** or **`send_message_stream()`** method to communicate with the agent. This is different from agentic agents, which use `send_event()` for task-based communication.

```python theme={null}
# Sync agents use send_message() - no task creation needed!
response = client.agents.send_message(
    agent_name=AGENT_NAME,
    params={
        "content": {"type": "text", "author": "user", "content": "Hello!"}
    }
)
```

**Key Differences:**

* **No task creation required** - You can send messages directly to the agent by name
* **No event creation required** - You can send messages directly to the agent by name

## Setup

### Prerequisites

```bash theme={null}
pip install agentex-sdk
```

### Environment Variables

```bash theme={null}
export AGENTEX_API_KEY="your-api-key"
export AGENTEX_BASE_URL="http://agentex.agentex.azure.workspace.egp.scale.com"
export SGP_ACCOUNT_ID="your-account-id"
```

### Initialize the Client

```python theme={null}
from agentex import Agentex
from agentex.types import TextContent

# Initialize the Agentex client
client = Agentex(
    base_url=os.environ.get("AGENTEX_BASE_URL"),
    api_key="your-api-key",
    default_headers={
        "x-api-key": os.environ.get("AGENTEX_API_KEY"),
        "x-selected-account-id": os.environ.get("SGP_ACCOUNT_ID"),
    },
)

AGENT_NAME = "<your-agent-name>"  # Replace with your agent name
```

## Basic Usage

### Non-Streaming Response

Send a message and receive a complete response immediately as part of the response:

```python theme={null}
# Send a message without streaming
rpc_response = client.agents.send_message(
    agent_name=AGENT_NAME,
    params={
        "content": {
            "type": "text",
            "author": "user",
            "content": "Hello! Can you explain what machine learning is?",
        },
        "stream": False,
    },
)

# Extract and print the text content from the response
print("User: Hello! Can you explain what machine learning is?\n")
print("Agent:")
for task_message in rpc_response.result:
    content = task_message.content
    if isinstance(content, TextContent):
        print(content.content)
```

**Output:**

```
User: Hello! Can you explain what machine learning is?

Agent:
Machine learning is a subset of artificial intelligence that enables systems to learn and improve from experience without being explicitly programmed...
```

## Streaming Responses

For real-time responses, use streaming mode:

```python theme={null}
from agentex.types import TextDelta
from agentex.types.agent_rpc_result import (
    StreamTaskMessageDelta,
    StreamTaskMessageDone,
    StreamTaskMessageFull
)

print("User: Can you give me a simple example of supervised learning?\n")
print("Agent: ", end="", flush=True)

# Stream the response
for chunk in client.agents.send_message_stream(
    agent_name=AGENT_NAME,
    params={
        "content": {
            "type": "text",
            "author": "user",
            "content": "Can you give me a simple example of supervised learning?",
        },
    },
):
    # The result is a TaskMessageUpdate
    task_message_update = chunk.result

    # Print text deltas as they arrive
    if isinstance(task_message_update, StreamTaskMessageDelta):
        delta = task_message_update.delta
        if isinstance(delta, TextDelta):
            print(delta.text_delta, end="", flush=True)
    elif isinstance(task_message_update, StreamTaskMessageFull):
        content = task_message_update.content
        if isinstance(content, TextContent):
            print(content.content)

print("\n")
```


# Introduction to Agents
Source: https://docs.gp.scale.com/docs/v5/agents/overview

Gain an understanding of different agent types in SGP

## Agent Types

There are two main types of agents in SGP:

* [Agentex Agents](/docs/v5/agents/agentex/overview)
* [Agent Service Agents](/docs/v5/agents/agent-service/overview)

Agentex Agents are built in code and support streaming and non-streaming responses. This is the preferred way to build agents for production use and will have the most support moving forward.

Agent Service Agents are built inside of SGP using a YAML configuration. They are more limited in their capabilities and will be deprecated in the future.

## View agents

To view all available agents in SGP navigate to the [agents page](https://egp.dashboard.scale.com/beta/build).

On this page you can see a grid or table view of all agents for your account. Metadata like the agent type, status, description, and updated time is visible.

<Frame>
  <img alt="Agents grid view" />
</Frame>

Clicking on an agent will take you to the agent's detail page. You can also create an Agent Service Agent by clicking the **Create Agent** button.
Learn more about [Agent Service Agents Creation](/docs/v5/agents/agent-service/lifecycle/creating-agent-service-agents).


# Authentication
Source: https://docs.gp.scale.com/docs/v5/authentication



The next step is to get your API key and account ID, which are needed to make calls to the Scale GP API
and to create, modify resources in the right place.

First navigate to the [SGP Admin](https://app.sgp.scale.com/admin/api-key) and grab your API key.
If you do not have an account provisioned by your organization or Scale, please contact the relevant System
administrator.

<Frame>
  <img alt="Switch to Accounts and find the account ID." />
</Frame>

or via the UI by clicking on the "API Keys" tab in the admin dashboard.

<Frame>
  <img alt="API Key menu" />
</Frame>

Next, switch to [Accounts](https://app.sgp.scale.com/admin/accounts) and find the account ID for the
account that you want to work with. You can also create a new account. Make sure to copy this ID to use it when
testing your setup.

<Frame>
  <img alt="You can also create a new account." />
</Frame>

If you have an existing account but want to add Users to it, you can click into the account and add users.

<Frame>
  <img alt="Click into the account to add users." />
</Frame>

You can also change the role of a user by clicking on the user and changing the role.

<Frame>
  <img alt="Change the role of a user." />
</Frame>

You can also remove a user by clicking on the remove button in the actions column.

<Frame>
  <img alt="Remove a user." />
</Frame>

## Storing Your Credentials

For security and convenience when using the SDK, it's recommended to store your API key and account ID as environment variables. This prevents accidentally exposing your credentials in code repositories.

<CodeGroup>
  ```bash macOS/Linux theme={null}
  export SGP_API_KEY="your-api-key-here"
  export SGP_ACCOUNT_ID="your-account-id-here"
  ```

  ```bash Windows (PowerShell) theme={null}
  $env:SGP_API_KEY="your-api-key-here"
  $env:SGP_ACCOUNT_ID="your-account-id-here"
  ```

  ```bash Windows (CMD) theme={null}
  set SGP_API_KEY=your-api-key-here
  set SGP_ACCOUNT_ID=your-account-id-here
  ```
</CodeGroup>

The SDK will automatically use the `SGP_API_KEY` and `SGP_ACCOUNT_ID` environment variables if they are set. Otherwise, you can pass them to the `SGPClient` constructor.

```python Python theme={null}
import os
from scale_gp_beta import SGPClient

client = SGPClient(
    account_id="your-account-id",
    api_key="your-api-key",
)
```

The `SGPClient` constructor accepts the following parameters:

| Parameter     | Description                                                                                                          | Default                               |
| ------------- | -------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| `account_id`  | Your SGP account ID. Found in the [Accounts](https://app.sgp.scale.com/admin/accounts) tab of the admin dashboard.   | `SGP_ACCOUNT_ID` environment variable |
| `api_key`     | Your SGP API key. Found in the [API Keys](https://app.sgp.scale.com/admin/api-key) tab of the admin dashboard.       | `SGP_API_KEY` environment variable    |
| `environment` | The SGP environment to connect to. **Possible values:** `production-multitenant`, `staging`, `development`, `local`. | `production-multitenant`              |
| `base_url`    | Override the base URL with a custom endpoint. Takes precedence over `environment`.                                   | None                                  |

<Tip>
  To make the environment variables persist across sessions, add the export commands to your shell configuration file (e.g., `~/.bashrc`, `~/.zshrc`, or `~/.bash_profile`).
</Tip>


# Evaluation Group Dashboards
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/eval-group-dashboards

Compare metrics across multiple evaluations with evaluation group dashboards

<Frame>
  <img alt="Evaluation Group Dashboard" />
</Frame>

## What are Evaluation Group Dashboards?

Evaluation group dashboards let you aggregate and compare data across multiple evaluations in a single dashboard. Instead of viewing metrics for one evaluation at a time, you can visualize trends, compare performance, and track progress across an entire group of related evaluations.

Evaluation group dashboards support all the same widget types and query language as single-evaluation dashboards, with additional features for cross-evaluation analysis.

Watch the [evaluation group dashboard demo video](./overview#evaluation-group-dashboard) for a walkthrough.

<Info>
  Dashboards use an XOR constraint -- they belong to either a single evaluation OR an evaluation group, never both.
</Info>

## Creating an Evaluation Group Dashboard

### Via the UI

<Steps>
  <Step title="Open the Create Dashboard Modal">
    Navigate to the Dashboards page and click "New Dashboard"
  </Step>

  <Step title="Select Group">
    In the create dialog, select the "Groups" tab and choose the evaluation group you want to create a dashboard for.
  </Step>

  <Step title="Fill in Details">
    * **Name**: Give your dashboard a descriptive name
    * **Description**: Optional description explaining the dashboard's purpose
    * **Tags**: Optional tags for organization and filtering

    <Frame>
      <img alt="Evaluation Group Dashboard Create Modal" />
    </Frame>
  </Step>

  <Step title="Create">
    Click "Create" to save your dashboard. You can now add widgets to visualize your group data.
  </Step>
</Steps>

### Via the SDK

```python theme={null}
from scale_gp_beta import SGPClient

client = SGPClient(
    api_key="your-api-key",
    account_id="your-account-id",
    environment="development"
)

# Create dashboard for an evaluation group
dashboard = client.evaluation_dashboards.create(
    name="Cross-Evaluation Comparison",
    evaluation_group_id="eval-group-456",
    description="Compare performance across weekly evaluation runs",
    tags=["comparison", "weekly"]
)
```

## Querying Across Evaluations

Evaluation group dashboards extend the standard query language with fields for controlling which evaluations to include in computations.

### The `evaluation_ids` Field

Add `evaluation_ids` to a query to specify which evaluations in the group to include. If omitted, all evaluations in the group are used.

```json theme={null}
{
  "select": [
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "AVG",
        "column": "overall_score",
        "source": "data"
      },
      "alias": "avg_score"
    }
  ],
  "evaluation_ids": ["eval-abc", "eval-def", "eval-ghi"]
}
```

This computes the average score across only the three specified evaluations, rather than all evaluations in the group.

### Per-Aggregation `evaluation_ids`

Individual aggregation nodes can also specify their own `evaluation_ids`, which must be a subset of the query-level `evaluation_ids`. This allows you to compare metrics across different evaluation subsets within the same widget.

```json theme={null}
{
  "select": [
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "AVG",
        "column": "overall_score",
        "source": "data",
        "evaluation_ids": ["eval-abc"]
      },
      "alias": "eval_a_avg"
    },
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "AVG",
        "column": "overall_score",
        "source": "data",
        "evaluation_ids": ["eval-def"]
      },
      "alias": "eval_b_avg"
    }
  ],
  "evaluation_ids": ["eval-abc", "eval-def"]
}
```

<Warning>
  Per-aggregation `evaluation_ids` must be a subset of the query-level `evaluation_ids`. The system validates this constraint and will reject queries where a per-node set includes IDs not present at the query level.
</Warning>

### The `_evaluation_id` Column

A special `_evaluation_id` column is automatically available in evaluation group dashboard queries. This column contains the ID of the evaluation that each data row belongs to, allowing you to group or filter by evaluation source.

```json theme={null}
{
  "select": [
    {"expression": {"type": "COLUMN", "column": "_evaluation_id"}},
    {
      "expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"},
      "alias": "avg_score"
    },
    {
      "expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"},
      "alias": "item_count"
    }
  ],
  "groupBy": ["_evaluation_id"]
}
```

This produces a row per evaluation in the group, showing the average score and item count for each.

## Per-Evaluation Selection in Widgets

When creating widgets in the UI for an evaluation group dashboard, the widget creator allows you to select which evaluations to include per aggregation. This is particularly useful for:

* **Metric widgets**: Compare the same metric across specific evaluations
* **Chart widgets**: Compare multiple metrics across specific evaluations
* **Table widgets**: Include different evaluation subsets for different columns

The UI evaluation selector maps directly to the `evaluation_ids` field on aggregation nodes in the query.

## Auto-Recomputation on Group Changes

When evaluations are added to or removed from an evaluation group, all dashboard widgets for that group are automatically recomputed. This ensures your dashboards always reflect the current state of the group.

### How it Works

1. **Membership change detected**: When you add or remove evaluations from a group, the system triggers an asynchronous recomputation workflow
2. **Smart `evaluation_ids` updates**: Widgets whose `evaluation_ids` covered all previous group members are automatically expanded or contracted to reflect the new membership. For example, if a group had evaluations \[A, B] and you add C, widgets covering \[A, B] are updated to \[A, B, C]
3. **Results marked as pending**: Existing widget results are marked with `computation_status: "pending"` while recomputation runs
4. **Recomputation completes**: Each widget is recomputed with the updated evaluation data, and results are updated to `computation_status: "completed"`

<Tip>
  This allows for widgets to be updated as new evaluations are added to the group. Any widgets computed across all evaluations in the group will be updated to include the new evaluation.

  This is particularly useful for the timeseries widget, where each evaluation is plotted as a member of the x-axis.
</Tip>

## Example: Cross-Evaluation Comparison Dashboard

Here's a full example creating an evaluation group dashboard with multiple widget types.

### Step 1: Create the Dashboard

```python theme={null}
dashboard = client.evaluation_dashboards.create(
    name="Weekly Agent Performance",
    evaluation_group_id="eval-group-weekly",
    tags=["weekly-review"]
)
```

### Step 2: Add a Heading

```python theme={null}
client.evaluation_dashboards.widgets.create(
    dashboard_id=dashboard.id,
    title="Performance Overview",
    type="heading"
)
```

### Step 3: Add Metric Widgets per Evaluation

```python theme={null}
# Average score across all evaluations in the group
client.evaluation_dashboards.widgets.create(
    dashboard_id=dashboard.id,
    title="Overall Average Score",
    type="metric",
    query={
        "select": [
            {
                "expression": {
                    "type": "AGGREGATION",
                    "function": "AVG",
                    "column": "overall_score",
                    "source": "data"
                }
            }
        ]
    }
)
```

### Step 4: Add a Timeseries Showing Trends

```python theme={null}
client.evaluation_dashboards.widgets.create(
    dashboard_id=dashboard.id,
    title="Score Trend Over Evaluations",
    type="timeseries",
    query={
        "select": [
            {
                "expression": {
                    "type": "AGGREGATION",
                    "function": "AVG",
                    "column": "overall_score",
                    "source": "data"
                },
                "alias": "avg_score"
            },
            {
                "expression": {
                    "type": "AGGREGATION",
                    "function": "AVG",
                    "column": "accuracy_score",
                    "source": "data"
                },
                "alias": "avg_accuracy"
            }
        ]
    }
)
```

### Step 5: Add a Table Grouped by Evaluation

```python theme={null}
client.evaluation_dashboards.widgets.create(
    dashboard_id=dashboard.id,
    title="Per-Evaluation Breakdown",
    type="table",
    query={
        "select": [
            {"expression": {"type": "COLUMN", "column": "_evaluation_id"}},
            {
                "expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"},
                "alias": "items"
            },
            {
                "expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"},
                "alias": "avg_score"
            },
            {
                "expression": {
                    "type": "AGGREGATION",
                    "function": "PERCENTAGE",
                    "column": "*",
                    "params": {
                        "percentage_filters": {
                            "conditions": [{"column": "overall_score", "source": "data", "operator": ">=", "value": 85}]
                        }
                    }
                },
                "alias": "pass_rate"
            }
        ],
        "groupBy": ["_evaluation_id"]
    }
)
```

## Related Documentation

* [Evaluation Dashboards Overview](./overview) - Introduction to dashboards
* [Getting Started](./getting-started) - Create your first dashboard
* [Timeseries Widget](./widget-types/timeseries) - Ideal for group trend visualization
* [Query Language](./query-language) - Complete query syntax reference
* [API Reference](/reference/v5/evaluation-dashboards) - Programmatic dashboard management


# Latest Snapshots
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/eval-group-latest-snapshots

View the results from the most recent evaluation in an evaluation group

## What are Latest Snapshots?

A *latest snapshot* is a widget on an evaluation group dashboard that computes against only the most recently created evaluation in a group.

<Info>
  "Most recent" is defined as the evaluation in the group with the latest `created_at` timestamp -- **not** the order in which evaluations were added to the group.
</Info>

## When to use it

* **Snapshot tiles** at the top of a dashboard that need to read the last run's score.
* **Comparison views** where you want the latest run highlighted alongside trend visualizations of the full group.
* **Always-fresh KPI widgets** that should automatically refocus on a new evaluation as soon as it's added to the group.

## Where it applies

Latest snapshots are only available on widgets that issue a query.

| Widget type             | Latest Only available?                                          |
| ----------------------- | --------------------------------------------------------------- |
| Metric (Query Value)    | Yes                                                             |
| Bar                     | Yes                                                             |
| Histogram               | Yes                                                             |
| Donut                   | Yes                                                             |
| Table (grouped or flat) | Yes                                                             |
| Scatter                 | Yes                                                             |
| Timeseries              | Yes, but rarely useful -- collapses the chart to a single point |
| Markdown, Heading       | Not applicable                                                  |

The toggle is **only available on evaluation group dashboards**.

## Enable via the UI

Toggle **Latest only** in the widget modal when creating or editing a widget.

<Frame>
  <img alt="Latest only toggle in the Add Widget modal" />
</Frame>

## Enable via the SDK

Set `latest_only: true` on the query payload when creating or updating the widget. The flag is supported on both `SeriesQuery` (Bar / Histogram / Donut / Table / Scatter / Timeseries) and `MetricQuery` (Metric / Query Value).

```python theme={null}
# Metric widget that always shows the average score of the most recent run in the group
client.evaluation_dashboards.widgets.create(
    dashboard_id=dashboard.id,
    title="Latest Snapshot: Average Score",
    type="metric",
    query={
        "select": [
            {
                "expression": {
                    "type": "AGGREGATION",
                    "function": "AVG",
                    "column": "overall_score",
                    "source": "data",
                },
            }
        ],
        "latest_only": True,
    },
)
```

Omitting `latest_only` or setting it to `false` keeps the existing behavior of computing across the full group.


# Getting Started with Dashboards
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/getting-started

Step-by-step tutorial for creating your first evaluation dashboard

This guide walks you through creating your first evaluation dashboard from scratch, adding widgets, and organizing your layout. You can also watch the [single evaluation dashboard demo video](./overview#single-evaluation-dashboard) for a visual walkthrough.

## Using Sample Data

To follow along with this tutorial, you can use our sample evaluation dataset containing 40 agent evaluation items with realistic scores and metadata.
Upload the sample data to your account and create an evaluation with it.

**Download Sample Data:**

<a href="/assets/sample-data/sample-evaluation.csv">
  <Card title="Download sample-evaluation.csv" icon="download">
    Sample dataset with 40 evaluation items across 4 agents (GPT-4, Claude-3, Gemini-Pro, Llama-3)
  </Card>
</a>

**Data Structure:**

The sample data contains evaluation items with this structure:

```json theme={null}
{
  "id": "eval_001",
  "agent_name": "GPT-4-Turbo-Agent",
  "agent_version": "1.0",
  "judged_evaluation": {
    "overall_score": 87,
    "accuracy_score": 92,
    "relevance_score": 85,
    "coherence_score": 89,
    "helpfulness_score": 84,
    "fluency_score": 91
  },
  "timestamp": "2026-01-15T10:30:00Z",
  "task_type": "question_answering",
  "prompt_category": "technical",
  "response_length": 256,
  "model_temperature": 0.7
}
```

**Key Fields:**

* `agent_name`: Model being evaluated (GPT-4-Turbo-Agent, Claude-3-Sonnet-Agent, etc.)
* `*`: Nested scores (overall\_score, accuracy\_score, relevance\_score, coherence\_score, helpfulness\_score, fluency\_score)
* `task_type`: Type of task (question\_answering, summarization, code\_generation, analysis, translation, creative\_writing)
* `prompt_category`: Category (technical, general, business, language, creative)

**How to Use:**

1. Download the CSV file
2. Create a new evaluation via the API or SDK:

```python theme={null}
from scale_gp_beta import SGPClient
import csv

# Using api.dev-sgp.scale.com
client = SGPClient(
    api_key="your-api-key",
    account_id="your-account-id",
    environment="development"
)

# Load sample data
with open('sample-evaluation.csv', 'r') as f:
    sample_items = csv.reader(f)

# Create evaluation with sample data
evaluation = client.evaluations.create(
    name="Agent Performance Comparison",
    data=sample_items
)

print(f"Created evaluation: {evaluation.id}")
```

<Tip>
  The client uses the `environment` parameter to connect to different Scale GP deployments. Available options: `"production"`, `"production-multitenant"`, `"development"`, `"staging"`, `"local"`. For custom endpoints, use `base_url` instead.
</Tip>

3. Follow the rest of this guide to create dashboards and widgets using this evaluation

<Info>
  The examples throughout this guide reference fields from this sample dataset. If using your own data, adjust the column names accordingly.
</Info>

## Prerequisites

Before creating a dashboard, you need either:

* An existing evaluation with completed results, OR
* An evaluation group containing evaluations

If you don't have an evaluation yet, see [Next Gen Evaluation Getting Started](../next-gen-evaluation/getting-started) to create one.

## Step 1: Create a New Dashboard

### Via the UI

1. Navigate your version of SGP ([Dev SGP](https://app.dev-sgp.scale.com/))
2. Make sure the `evaluation-dashboards-enabled` feature flag is enabled for your account
   1. ([Instructions to enable the feature flag](https://scale.atlassian.net/wiki/x/C4EgLw))
3. Click the "Dashboards" tab
4. Click the "New Dashboard" button

<Frame>
  <img alt="New Dashboard Button" />
</Frame>

5. Fill in the dashboard details:
   * **Name**: Give your dashboard a descriptive name (e.g., "Model Performance Overview")
   * **Description**: Optional description explaining the dashboard's purpose
   * **Tags**: Optional tags for organization and filtering
   * **Evaluation / Evaluation Group**: Select the evaluation or evaluation group you want to create a dashboard for
   * **Template** *(optional)*: Select an existing single-evaluation dashboard to copy its widget layout
6. Click "Create" to save your dashboard

<Frame>
  <img alt="New Dashboard Form" />
</Frame>

### Via the SDK

```python theme={null}
from scale_gp_beta import SGPClient

client = SGPClient(
    api_key="your-api-key",
    account_id="your-account-id",
    environment="development"
)

# Create dashboard for a single evaluation
dashboard = client.evaluation_dashboards.create(
    name="Demo Dashboard",
    description="A demo dashboard for the demo evaluation",
    evaluation_id="eval-123",
    tags=["demo", "documentation"]
)

# Create dashboard from an existing template (single-evaluation dashboards only)
dashboard_from_template = client.evaluation_dashboards.create(
    name="Q2 Model Performance",
    evaluation_id="eval-456",
    template_dashboard_id="dash-template-abc"  # Copies widget layout from this dashboard
)

# Or create dashboard for an evaluation group
group_dashboard = client.evaluation_dashboards.create(
    name="Cross-Evaluation Comparison",
    evaluation_group_id="eval-group-456",
    tags=["comparison"]
)
```

## Step 2: Add Your First Widget (Metric)

Let's add a metric widget to display the average score across all evaluation items.

### Via the UI

1. From your dashboard page, click "Add Widget"
2. Select "Query Value" as the widget type
3. Configure the widget:
   * **Title**: "Average Score"
   * **Query**: Select the average of the "score" column
4. Click "Add"

<Frame>
  <img alt="Average Score Widget Form" />
</Frame>

<Frame>
  <img alt="Average Score Widget Result" />
</Frame>

<Tip>
  Widget results are automatically computed when you create or update a widget. The response includes both the widget configuration and the computed result.
</Tip>

### Via the API

```python theme={null}
# Add a metric widget showing average score
widget = client.evaluation_dashboards.widgets.create(
    dashboard_id=dashboard.id,
    title="Average Score",
    type="metric",
    query={
        "select": [
            {
                "expression": {
                    "type": "AGGREGATION",
                    "function": "AVG",
                    "column": "overall_score",
                    "source": "data"
                }
            }
        ]
    }
)

print(f"Computed result: {widget.result.computed_result}")
# Output: {'type': 'metric', 'data': 0.873}
```

## Step 3: Add a Chart Widget (Bar Chart)

Now let's add a bar chart to show score distribution across different models.

### Via the UI

1. Click "Add Widget" again
2. Select "Bar Chart" as the widget type
3. Configure the widget:
   * **Title**: "Score by Agent"
   * **Group By**: Select "agent\_name"
   * Under the Advanced Options
     * Add an aggregation, select "Average" on "overall\_score"
4. Click "Add"

<Frame>
  <img alt="Score by Agent Bar Chart Form" />
</Frame>

<Frame>
  <img alt="Score by Agent Bar Chart Result" />
</Frame>

### Via the SDK

```python theme={null}
# Add a bar chart widget showing average score by category
widget = client.evaluation_dashboards.widgets.create(
    dashboard_id=dashboard.id,
    title="Score by Agent",
    type="bar",
    query={
        "select": [
            {
                "expression": {
                    "type": "COLUMN",
                    "column": "agent_name",
                    "source": "data"
                }
            },
            {
                "expression": {
                    "type": "AGGREGATION",
                    "function": "AVG",
                    "column": "overall_score",
                    "source": "data"
                }
            }
        ],
        "groupBy": ["agent_name"]
    },
    config={
        "x_column": "agent_name"
    }
)
```

## Step 4: Add Section Headers

Use heading widgets to organize your dashboard into logical sections.

### Via the UI

1. Click "Add Widget"
2. Select "Heading" as the widget type
3. Configure the widget:
   * **Title**: "Graphs"
4. Click "Add"

<Frame>
  <img alt="Header Widget Form" />
</Frame>

<Frame>
  <img alt="Header Widget Result" />
</Frame>

### Via the SDK

```python theme={null}
# Add a heading widget
heading = client.evaluation_dashboards.widgets.create(
    dashboard_id=dashboard.id,
    title="Graphs",
    type="heading"
)
```

## Step 5: Organize and Configure Layout

### Reorder Widgets

Arrange widgets in your preferred order by dragging and dropping in the UI, or update the widget order via the API:

```python theme={null}
# Reorder widgets - first widget appears at top
client.evaluation_dashboards.update(
    dashboard_id=dashboard.id,
    widget_order=[heading.id, widget1.id, widget2.id]
)
```

<Info>
  For evaluation group dashboards, see the dedicated [Evaluation Group Dashboards](./eval-group-dashboards) guide for group-specific features like cross-evaluation queries, per-evaluation selection, and auto-recomputation.
</Info>

## Next Steps

* Learn about all available [Widget Types](./widget-types/overview)
* Compare across evaluations with [Evaluation Group Dashboards](./eval-group-dashboards)
* Capture [Latest Snapshots](./eval-group-latest-snapshots) of your evaluation group
* Master the [Query Language](./query-language) for advanced filtering and aggregations
* Explore [API Reference](/reference/v5/evaluation-dashboards) for programmatic query creation


# Introduction to Evaluation Dashboards
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/overview

Create custom dashboards to visualize and analyze evaluation metrics

## What are Evaluation Dashboards?

Evaluation Dashboards are customizable visualizations that allow you to analyze and monitor evaluation performance with widgets.
Dashboards are a collection of metric cards, charts, and tables that can be used to provide insights into evaluation results.

<Frame>
  <img alt="Evaluation Dashboards Overview" />
</Frame>

## Demo Videos

### Single Evaluation Dashboard

<Frame>
  <iframe />
</Frame>

### Evaluation Group Dashboard

<Frame>
  <iframe />
</Frame>

## Key Concepts

### Dashboards

A dashboard is a collection of widgets arranged in a custom layout. Each dashboard is scoped to either:

* **Single Evaluation**: Visualize metrics for one specific evaluation — see [demo video](#single-evaluation-dashboard)
* **Evaluation Group**: Compare metrics across multiple related evaluations — see [Evaluation Group Dashboards](./eval-group-dashboards) and [demo video](#evaluation-group-dashboard)

<Info>
  Dashboards use an XOR constraint - they belong to either a single evaluation OR an evaluation group, never both.
</Info>

### Widgets

Widgets are reusable visualization components that display computed metrics or data. Nine widget types are available:

* **Metric**: Display a single computed aggregation (e.g., "Average Score: 0.87")
* **Table**: Show aggregations in a table format, with grouping, sorting, and conditional formatting
* **Bar Chart**: Visualize data across a categorical dimension
* **Histogram**: Visualize data across a numerical dimension
* **Donut Chart**: Show proportional distribution of categories
* **Scatter Plot**: Explore correlations between two numerical dimensions
* **Timeseries**: Track metrics over time across evaluation runs
* **Markdown**: Add static content to the dashboard
* **Heading**: Organize dashboard sections

Learn more in the [Widget Types reference](./widget-types).

### Queries

For data-driven widgets (Metric, Table, Bar, Histogram, Donut, Scatter, Timeseries), you define a query using a SQL-like JSON object. Queries allow you to:

* Aggregate data using functions
  * `AVG`, `SUM`, `COUNT`, `MIN`, `MAX`, `STDDEV`, `VARIANCE`, `PERCENTILE`, `PERCENTAGE`, `COUNT`, `COUNT_DISTINCT`
* Filter results based on conditions
* Group by dimensions
* Sort and limit results

See the [Query Language guide](./query-language) for details.

### Widget Results

When you create or update a widget, the system automatically computes its results by executing the query against your evaluation data.
Results are stored in the database and can be accessed by the widget when it is rendered.

For evaluation group dashboards, widget results are also automatically recomputed when evaluations are added to or removed from the group. See [Evaluation Group Dashboards](./eval-group-dashboards) for details.

## Architecture & Data Model

Understanding how dashboards are structured in the database helps clarify the relationship between dashboards, widgets, and their computed results.

### Three Core Entities

**1. Dashboard**
A dashboard is a container that links to an evaluation or evaluation group and maintains an ordered list of widget IDs.

```json theme={null}
{
  "id": "dash-abc123",
  "name": "Model Performance Dashboard",
  "evaluation_id": "eval-xyz789",
  "widget_order": ["widget-1", "widget-2", "widget-3"]
}
```

The `widget_order` array defines both which widgets appear on the dashboard and their display order.

**2. Widget (Template)**
Widgets are reusable computation templates that define:

* What data to query (the query definition)
* How to display it (the configuration)
* The visualization type (metric, table, bar, etc.)

```json theme={null}
{
  "id": "widget-1",
  "title": "Average Score",
  "type": "metric",
  "query": {
    "select": [{"expression": {"type": "AGGREGATION", "function": "AVG", "column": "score"}}]
  },
  "config": {}
}
```

Widgets are stored independently from dashboards and can be reused across multiple dashboards by including the same widget ID in different `widget_order` arrays.
This enables the same dashboard template to be used for different evaluations.

**3. Widget Result (Computed Output)**
Widget results are the actual computed values produced by executing a widget's query against evaluation data.

```json theme={null}
{
  "id": "result-def456",
  "widget_id": "widget-1",
  "evaluation_id": "eval-xyz789",
  "computed_result": {"type": "metric", "data": 0.873},
  "computation_status": "completed",
  "computed_at": "2026-01-15T10:35:02Z"
}
```

Results are specific to both a widget and an evaluation, meaning the same widget produces different results when used on different evaluations.

### How They Work Together

1. **Dashboard Creation**: When you create a dashboard, you link it to an evaluation or evaluation group

2. **Adding Widgets**: When you add a widget to a dashboard:
   * A widget template is created (or reused if it already exists)
   * The widget's ID is appended to the dashboard's `widget_order`
   * The system automatically computes the widget result for the dashboard's evaluation data

3. **Display**: When rendering a dashboard:
   * The system fetches the dashboard and its ordered widget list
   * For each widget ID, it loads the widget definition (template)
   * It retrieves the corresponding widget result (computed output)
   * The UI renders widgets in order using the template definition and computed results

4. **Updates**: When you update a widget:
   * If the widget belongs to only one dashboard: The template is updated in place
   * If the widget is shared across multiple dashboards: A copy of the widget is created for your dashboard
   * Results are recomputed if the query or configuration changed

### Reusability Example

The same widget template can appear on multiple dashboards:

```
Widget "Average Score" (widget-1)
│── Dashboard A (evaluation-1)
│   └── Result: 0.873
│── Dashboard B (evaluation-2)
│   └── Result: 0.791
│── Dashboard C (evaluation-group-1)
    └── Result: 0.845
```

<Tip>
  This architecture enables efficient widget reuse while maintaining separate computed results for each evaluation context.
</Tip>

## Managing Widgets

### Editing Widgets

You can edit a widget's configuration, query, or title at any time:

**Via the UI:**

1. Navigate to your dashboard
2. Click the edit icon (pencil) on the widget you want to modify
3. Update the title, query, or configuration
4. Click "Save Changes"

**Via the SDK:**

```python theme={null}
updated_widget = client.evaluation_dashboards.widgets.update(
    dashboard_id=dashboard.id,
    widget_id=widget.id,
    title="Updated Widget Title",
    query={
        "select": [
            {
                "expression": {
                    "type": "AGGREGATION",
                    "function": "AVG",
                    "column": "overall_score",
                    "source": "data"
                }
            }
        ]
    }
)
```

<Info>
  **Widget Cloning Behavior**: If a widget is shared across multiple dashboards, editing it will automatically create a copy for your dashboard to prevent affecting other dashboards using the same widget.
</Info>

### Removing Widgets from a Dashboard

Remove widgets you no longer need from your dashboard:

**Via the UI:**

1. Navigate to your dashboard
2. Click the delete icon (trash) on the widget
3. Confirm the removal

**Via the SDK:**

```python theme={null}
client.evaluation_dashboards.widgets.remove(
    dashboard_id=dashboard.id,
    widget_id=widget.id
)
```

<Warning>
  Removing a widget from a dashboard only removes it from that dashboard's `widget_order`. The widget template itself remains in the system if it's used by other dashboards. To permanently delete a widget template, use the delete endpoint.
</Warning>

### Reordering Widgets

Organize your dashboard by rearranging widget order:

**Via the UI:**

1. Navigate to your dashboard
2. Click and drag widgets to reorder them
3. Changes are saved automatically

**Via the SDK:**

```python theme={null}
# Specify the desired order of widget IDs
client.evaluation_dashboards.update(
    dashboard_id=dashboard.id,
    widget_order=["widget-3", "widget-1", "widget-2"]  # New order
)
```

The first widget ID in the array appears at the top of the dashboard, and subsequent widgets follow in order.

## Duplicating Dashboards

Create a copy of an existing single-evaluation dashboard to reuse its widget configuration with a different evaluation.

<Warning>
  Dashboard duplication is only supported for **single-evaluation dashboards**. Evaluation group dashboards cannot be used as or created from templates.
</Warning>

### Use Cases for Duplication

* **Template Reuse**: Apply a dashboard template to multiple evaluations
* **Comparison**: Create identical dashboards for different evaluation runs
* **Iteration**: Start with a proven dashboard structure and customize it

### Creating from a Template

**Via the UI:**

1. Navigate to the Dashboards page
2. Go to the dashboard you want to duplicate
3. Click the three dots menu and select "Duplicate Dashboard"
4. Fill in the form — select the target evaluation and optionally choose a template dashboard to copy from
5. Click "Duplicate"

**Via the SDK:**

```python theme={null}
# Create a new dashboard from an existing template
new_dashboard = client.evaluation_dashboards.create(
    name="Q1 Model Performance",
    evaluation_id="eval-new-123",
    template_dashboard_id="dash-template-abc"  # Source dashboard to copy widgets from
)
```

When you duplicate a dashboard:

* The new dashboard copies the `widget_order` from the template
* All widgets are reused (same widget IDs)
* New widget results are computed for the new evaluation's data
* The template dashboard remains unchanged

<Tip>
  **Best Practice**: Create a "template" dashboard with your standard metrics and visualizations, then duplicate it for each new evaluation run to maintain consistency.
</Tip>

## Next Steps

* Ready to create your first dashboard? Watch the [single evaluation dashboard demo](#single-evaluation-dashboard) or follow the [Getting Started guide](./getting-started) for a step-by-step walkthrough.
* Want to compare across evaluations? Watch the [evaluation group dashboard demo](#evaluation-group-dashboard) or see the [Evaluation Group Dashboards](./eval-group-dashboards) guide.
* For detailed information on widget configuration, see the [Widget Types reference](./widget-types/overview).
* To learn the query language for data-driven widgets, read the [Query Language guide](./query-language).
* For programmatic query creation, read the [API Reference](/reference/v5/evaluation-dashboards).


# Query Language Reference
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/query-language

Deep dive into QueryAST for building powerful evaluation queries

The QueryAST (Query Abstract Syntax Tree) is a SQL-like query language for aggregating and filtering evaluation data in dashboard widgets. This guide covers the complete query syntax and advanced features.

## Query Basics

### When Are Queries Required?

Queries are required for data-driven widgets:

* **METRIC**: Requires a MetricQuery (single aggregation)
* **TABLE**: Requires a SeriesQuery (multiple rows/columns)
* **BAR**: Requires a SeriesQuery (grouped data)
* **HISTOGRAM**: Requires a SeriesQuery (distribution data)
* **DONUT**: Requires a SeriesQuery (grouped data for slices)
* **SCATTER**: Requires a SeriesQuery (two or more numeric columns)
* **TIMESERIES**: Requires a SeriesQuery (aggregations over time)

Queries are NOT needed for:

* **MARKDOWN**: Static content only
* **HEADING**: Text display only

### Data Sources

QueryAST can access two data sources for each evaluation item:

**`data.*` - User-defined data**

* Custom fields you provided when creating the evaluation
* Examples: `data.input`, `data.expected_output`, `data.category`, `data.score`

**`task_result_cache.*` - Task execution results**

* Results from evaluation tasks (chat completions, agent runs, etc.)
* Examples: `task_result_cache.llm_score`, `task_result_cache.latency_ms`, `task_result_cache.metadata.confidence`

<Info>
  Both data sources support nested fields using dot notation. For example, `data.metadata.model_version` or `task_result_cache.output.reasoning`.
</Info>

### Query Structure Overview

A query is a JSON object defining what data to select, how to aggregate it, and what filters to apply:

```json theme={null}
{
  "select": [...],      // Columns and aggregations to return
  "filter": {...},      // Optional WHERE clause conditions
  "groupBy": [...],     // Optional GROUP BY columns
  "orderBy": [...],     // Optional sorting
  "limit": 100          // Optional row limit
}
```

***

## Query Types

### MetricQuery

Returns a single scalar value. Used exclusively for METRIC widgets.

**Structure:**

```json theme={null}
{
  "select": [
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "AVG|SUM|COUNT|...",
        "column": "column_name",
        "source": "data|task_result_cache"
      },
      "alias": "optional_alias"
    }
  ],
  "filter": { /* optional */ }
}
```

**Constraints:**

* Select clause must have exactly one item
* That item must be an `AGGREGATION` (not a raw `COLUMN`)
* No `GROUP BY`, `ORDER BY`, or `LIMIT` allowed

#### Example: Average Score

**Example:**

```json theme={null}
{
  "select": [
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "AVG",
        "column": "score",
        "source": "data"
      },
      "alias": "avg_score"
    }
  ],
  "filter": {
    "conditions": [
      {
        "column": "category",
        "source": "data",
        "operator": "=",
        "value": "test"
      }
    ]
  }
}
```

**SQL Equivalent:**

```sql theme={null}
SELECT AVG(data->>'$.score') AS avg_score
FROM evaluation_items
WHERE data->>'$.category' = 'test'
```

### SeriesQuery

Returns multiple rows and columns. Used for `TABLE`, `BAR`, `HISTOGRAM`, `DONUT`, `SCATTER`, and `TIMESERIES` widgets.

**Structure:**

```json theme={null}
{
  "select": [
    // Mix of columns and aggregations
  ],
  "filter": { /* optional */ },
  "groupBy": [ /* optional column names */ ],
  "orderBy": [ /* optional sort specs */ ],
  "limit": 100 // optional
}
```

**Constraints:**

* Select clause must have at least one item
* Can mix `COLUMN` and `AGGREGATION` expressions
* When using `GROUP BY`, non-aggregated columns must be in the `groupBy` list

#### Example: Performance by Category

**Example:**

```json theme={null}
{
  "select": [
    {
      "expression": {
        "type": "COLUMN",
        "column": "category",
        "source": "data"
      }
    },
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "AVG",
        "column": "score",
        "source": "data"
      },
      "alias": "avg_score"
    }
  ],
  "groupBy": ["category"],
  "orderBy": [
    {
      "column": "avg_score",
      "direction": "DESC"
    }
  ],
  "limit": 10
}
```

**SQL Equivalent:**

```sql theme={null}
SELECT 
  data->>'$.category' AS category,
  AVG(data->>'$.score') AS avg_score
FROM evaluation_items
GROUP BY data->>'$.category'
ORDER BY avg_score DESC
LIMIT 10
```

***

## `evaluation_ids` (Evaluation Group Dashboards)

When building queries for [evaluation group dashboards](./eval-group-dashboards), you can control which evaluations in the group to include using the `evaluation_ids` field.

### Query-Level `evaluation_ids`

Add `evaluation_ids` at the top level of a query to scope it to specific evaluations in the group. If omitted, all evaluations in the group are included.

```json theme={null}
{
  "select": [
    {
      "expression": {"type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data"},
      "alias": "avg_score"
    }
  ],
  "evaluation_ids": ["eval-abc", "eval-def"]
}
```

### Per-Aggregation `evaluation_ids`

Individual aggregation expressions can specify their own `evaluation_ids` to narrow the scope further. These must be a subset of the query-level `evaluation_ids`.

```json theme={null}
{
  "select": [
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "AVG",
        "column": "score",
        "source": "data",
        "evaluation_ids": ["eval-abc"]
      },
      "alias": "eval_a_avg"
    },
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "AVG",
        "column": "score",
        "source": "data",
        "evaluation_ids": ["eval-def"]
      },
      "alias": "eval_b_avg"
    }
  ],
  "evaluation_ids": ["eval-abc", "eval-def"]
}
```

<Info>
  The special `_evaluation_id` column is automatically available in evaluation group dashboard queries. You can use it in `SELECT` and `GROUP BY` to break down data by evaluation source. See the [Evaluation Group Dashboards](./eval-group-dashboards) guide for details.
</Info>

***

## `SELECT` Clause

The `SELECT` clause defines what columns or aggregations to return.

### Column Expression

Reference a raw column from the data without aggregation.

```json theme={null}
{
  "expression": {
    "type": "COLUMN",
    "column": "category",
    "source": "data"
  },
  "alias": "category_name"  // Optional alias
}
```

**Fields:**

* `type`: Always "COLUMN"
* `column`: Name of the column to select
* `source`: "data" or "task\_result\_cache" (optional, defaults to "data")
* `alias`: Optional name for the result column (recommended for aggregation columns)

<Tip>
  Aliases are especially useful for aggregation columns -- they provide readable column headers in tables, disambiguate when multiple aggregations use the same column, and are required for meaningful labels in chart legends.
</Tip>

### Aggregation Expression

Apply an aggregation function to compute a derived value.

```json theme={null}
{
  "expression": {
    "type": "AGGREGATION",
    "function": "AVG",
    "column": "score",
    "source": "data",
    "params": {}  // Optional parameters for certain functions
  },
  "alias": "average_score"
}
```

**Fields:**

* `type`: Always "AGGREGATION"
* `function`: One of the supported aggregation functions (see below)
* `column`: Column to aggregate, or `"*"` for `COUNT(*)`
* `source`: "data" or "task\_result\_cache" (optional)
* `params`: Function-specific parameters (for `PERCENTILE` and `PERCENTAGE`)
* `alias`: Optional name for the result

***

## Aggregation Functions

### `COUNT`

Count the number of rows.

```json theme={null}
{
  "type": "AGGREGATION",
  "function": "COUNT",
  "column": "*"
}
```

**Use Cases:**

* Total number of items
* Items per category (with `GROUP BY`)
* Items matching filter criteria

### `SUM`

Sum all values in a column.

```json theme={null}
{
  "type": "AGGREGATION",
  "function": "SUM",
  "column": "cost",
  "source": "task_result_cache"
}
```

**Use Cases:**

* Total cost across all items
* Total tokens used
* Cumulative metrics

### `AVG`

Calculate the mean value.

```json theme={null}
{
  "type": "AGGREGATION",
  "function": "AVG",
  "column": "score",
  "source": "data"
}
```

**Use Cases:**

* Average score
* Mean latency
* Average confidence

### `MIN` / `MAX`

Find the minimum or maximum value.

```json theme={null}
{
  "type": "AGGREGATION",
  "function": "MAX",
  "column": "latency_ms",
  "source": "task_result_cache"
}
```

**Use Cases:**

* Worst-case latency
* Best score achieved
* Range calculations

### `STDDEV` / `VARIANCE`

Calculate standard deviation or variance.

```json theme={null}
{
  "type": "AGGREGATION",
  "function": "STDDEV",
  "column": "score",
  "source": "data"
}
```

**Use Cases:**

* Measure score consistency
* Identify high-variance categories
* Quality control metrics

### `PERCENTILE`

Calculate a specific percentile value.

```json theme={null}
{
  "type": "AGGREGATION",
  "function": "PERCENTILE",
  "column": "latency_ms",
  "source": "task_result_cache",
  "params": {
    "percentile": 95
  }
}
```

**Parameters:**

* `percentile`: Number between 0 and 100 (required)

**Use Cases:**

* P95/P99 latency tracking
* Tail performance analysis
* SLA monitoring

### `COUNT_DISTINCT`

Count unique values in a column.

```json theme={null}
{
  "type": "AGGREGATION",
  "function": "COUNT_DISTINCT",
  "column": "user_id",
  "source": "data"
}
```

**Use Cases:**

* Number of unique users
* Distinct categories present
* Cardinality checks

### `PERCENTAGE`

Calculate percentage of rows matching filter criteria.

```json theme={null}
{
  "type": "AGGREGATION",
  "function": "PERCENTAGE",
  "column": "*",
  "params": {
    "percentage_filters": {
      "conditions": [
        {
          "column": "score",
          "source": "data",
          "operator": ">=",
          "value": 0.7
        }
      ]
    }
  }
}
```

**Parameters:**

* `percentage_filters`: Filter object defining which rows to count (required)

**Use Cases:**

* Pass/fail rates
* Percentage above threshold
* Compliance metrics

***

## Filter Clause (`WHERE`)

Filter clauses narrow down which rows to include in computations.

### Basic Filter

Single condition:

```json theme={null}
{
  "filter": {
    "conditions": [
      {
        "column": "category",
        "source": "data",
        "operator": "=",
        "value": "quality"
      }
    ]
  }
}
```

### Multiple Conditions

Combine conditions with logical operators:

```json theme={null}
{
  "filter": {
    "conditions": [
      {
        "column": "score",
        "source": "data",
        "operator": ">=",
        "value": 0.5
      },
      {
        "column": "category",
        "source": "data",
        "operator": "=",
        "value": "production"
      }
    ],
    "logicalOperators": ["AND"]
  }
}
```

<Warning>
  The number of logical operators must be exactly `len(conditions) - 1`. For 3 conditions, you need 2 operators.
</Warning>

### Comparison Operators

| Operator   | Description         | Example Value             |
| ---------- | ------------------- | ------------------------- |
| `=`        | Equals              | `"test"` or `42`          |
| `!=`       | Not equals          | `"draft"`                 |
| `>`        | Greater than        | `0.5`                     |
| `<`        | Less than           | `100`                     |
| `>=`       | Greater or equal    | `0.7`                     |
| `<=`       | Less or equal       | `1.0`                     |
| `IN`       | In list             | `["cat1", "cat2"]`        |
| `NOT IN`   | Not in list         | `["archived", "deleted"]` |
| `LIKE`     | Substring match     | `"%error%"`               |
| `NOT LIKE` | Not substring match | `"%deprecated%"`          |

### Complex Filter Example

Combine multiple conditions with AND/OR logic:

```json theme={null}
{
  "filter": {
    "conditions": [
      {
        "column": "score",
        "source": "data",
        "operator": ">=",
        "value": 0.7
      },
      {
        "column": "category",
        "source": "data",
        "operator": "IN",
        "value": ["quality", "accuracy"]
      },
      {
        "column": "status",
        "source": "data",
        "operator": "!=",
        "value": "archived"
      }
    ],
    "logicalOperators": ["AND", "AND"]
  }
}
```

This evaluates as: `(score >= 0.7) AND (category IN ['quality', 'accuracy']) AND (status != 'archived')`

<Info>
  Logical operators are evaluated left-to-right without precedence. The expression `A AND B OR C` evaluates as `((A AND B) OR C)`.
</Info>

***

## `GROUP BY` Clause

Group rows by one or more columns before applying aggregations.

### Single Column Grouping

```json theme={null}
{
  "select": [
    {
      "expression": {"type": "COLUMN", "column": "category", "source": "data"}
    },
    {
      "expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"},
      "alias": "count"
    }
  ],
  "groupBy": ["category"]
}
```

**SQL Equivalent:**

```sql theme={null}
SELECT 
  data->>'$.category' AS category,
  COUNT(*) AS count
FROM evaluation_items
GROUP BY data->>'$.category'
```

### Multi-Column Grouping

Group by multiple dimensions:

```json theme={null}
{
  "select": [
    {
      "expression": {"type": "COLUMN", "column": "model", "source": "data"}
    },
    {
      "expression": {"type": "COLUMN", "column": "category", "source": "data"}
    },
    {
      "expression": {"type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data"},
      "alias": "avg_score"
    }
  ],
  "groupBy": ["model", "category"]
}
```

**SQL Equivalent:**

```sql theme={null}
SELECT 
  data->>'$.model' AS model,
  data->>'$.category' AS category,
  AVG(data->>'$.score') AS avg_score
FROM evaluation_items
GROUP BY data->>'$.model', data->>'$.category'
```

<Tip>
  When using `GROUP BY`, all non-aggregated columns in `SELECT` must appear in the `groupBy` array.
</Tip>

***

## `ORDER BY` Clause

Sort results by one or more columns.

### Basic Sorting

```json theme={null}
{
  "orderBy": [
    {
      "column": "avg_score",
      "direction": "DESC"
    }
  ]
}
```

**Directions:**

* `ASC`: Ascending (smallest to largest)
* `DESC`: Descending (largest to smallest)

### Multi-Column Sorting

Sort by multiple columns with different directions:

```json theme={null}
{
  "orderBy": [
    {
      "column": "category",
      "direction": "ASC"
    },
    {
      "column": "avg_score",
      "direction": "DESC"
    }
  ]
}
```

This sorts by category alphabetically, then by avg\_score descending within each category.

***

## `LIMIT` Clause

Restrict the number of rows returned.

```json theme={null}
{
  "limit": 20
}
```

**Use Cases:**

* Top-N queries (e.g., "Top 10 Items")
* Prevent overwhelming table displays
* Performance optimization for large datasets

**Example: Top 5 Categories**

```json theme={null}
{
  "select": [
    {"expression": {"type": "COLUMN", "column": "category", "source": "data"}},
    {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data"}, "alias": "avg_score"}
  ],
  "groupBy": ["category"],
  "orderBy": [{"column": "avg_score", "direction": "DESC"}],
  "limit": 5
}
```

***

## Advanced Examples

### Example 1: Multi-Dimensional Performance Table

Break down performance by model and category with multiple statistics.

```json theme={null}
{
  "select": [
    {
      "expression": {
        "type": "COLUMN",
        "column": "model",
        "source": "data"
      }
    },
    {
      "expression": {
        "type": "COLUMN",
        "column": "category",
        "source": "data"
      }
    },
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "COUNT",
        "column": "*"
      },
      "alias": "count"
    },
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "AVG",
        "column": "score",
        "source": "data"
      },
      "alias": "avg_score"
    },
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "STDDEV",
        "column": "score",
        "source": "data"
      },
      "alias": "stddev_score"
    },
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "MIN",
        "column": "score",
        "source": "data"
      },
      "alias": "min_score"
    },
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "MAX",
        "column": "score",
        "source": "data"
      },
      "alias": "max_score"
    }
  ],
  "groupBy": ["model", "category"],
  "orderBy": [
    {
      "column": "model",
      "direction": "ASC"
    },
    {
      "column": "avg_score",
      "direction": "DESC"
    }
  ]
}
```

### Example 2: Filtered Percentile Calculation

Calculate P95 score for only production items.

```json theme={null}
{
  "select": [
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "PERCENTILE",
        "column": "latency_ms",
        "source": "task_result_cache",
        "params": {
          "percentile": 95
        }
      },
      "alias": "p95_latency"
    }
  ],
  "filter": {
    "conditions": [
      {
        "column": "environment",
        "source": "data",
        "operator": "=",
        "value": "production"
      },
      {
        "column": "latency_ms",
        "source": "task_result_cache",
        "operator": ">",
        "value": 0
      }
    ],
    "logicalOperators": ["AND"]
  }
}
```

### Example 3: Complex Filter with IN and LIKE

Find items in specific categories containing certain text.

```json theme={null}
{
  "select": [
    {
      "expression": {"type": "COLUMN", "column": "input", "source": "data"}
    },
    {
      "expression": {"type": "COLUMN", "column": "category", "source": "data"}
    },
    {
      "expression": {"type": "COLUMN", "column": "score", "source": "data"}
    }
  ],
  "filter": {
    "conditions": [
      {
        "column": "category",
        "source": "data",
        "operator": "IN",
        "value": ["quality", "accuracy", "helpfulness"]
      },
      {
        "column": "input",
        "source": "data",
        "operator": "LIKE",
        "value": "%customer support%"
      }
    ],
    "logicalOperators": ["AND"]
  },
  "orderBy": [
    {
      "column": "score",
      "direction": "DESC"
    }
  ],
  "limit": 20
}
```

### Example 4: Pass/Fail Rate by Model

Calculate success rates grouped by model.

```json theme={null}
{
  "select": [
    {
      "expression": {"type": "COLUMN", "column": "model", "source": "data"}
    },
    {
      "expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"},
      "alias": "total"
    },
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "PERCENTAGE",
        "column": "*",
        "params": {
          "percentage_filters": {
            "conditions": [
              {
                "column": "score",
                "source": "data",
                "operator": ">=",
                "value": 0.7
              }
            ]
          }
        }
      },
      "alias": "pass_rate"
    }
  ],
  "groupBy": ["model"],
  "orderBy": [
    {
      "column": "pass_rate",
      "direction": "DESC"
    }
  ]
}
```

### Example 5: Nested Data Access

Query nested fields using dot notation.

```json theme={null}
{
  "select": [
    {
      "expression": {
        "type": "COLUMN",
        "column": "metadata.model_version",
        "source": "data"
      }
    },
    {
      "expression": {
        "type": "AGGREGATION",
        "function": "AVG",
        "column": "output.confidence",
        "source": "task_result_cache"
      },
      "alias": "avg_confidence"
    }
  ],
  "groupBy": ["metadata.model_version"]
}
```

This accesses:

* `data.metadata.model_version` - nested field in user data
* `task_result_cache.output.confidence` - nested field in task results

***

## Debugging Failed Queries

If your widget shows `computation_status: "failed"`, check:

1. **Column names**: Ensure columns exist in your evaluation data
2. **Source specification**: Verify correct source ("data" vs "task\_result\_cache")
3. **GROUP BY alignment**: All non-aggregated `SELECT` columns must be in `groupBy`
4. **Operator/value types**: Match operator to value type (IN requires list, LIKE requires string)
5. **Required params**: PERCENTILE needs percentile param, PERCENTAGE needs percentage\_filters

Check the `error_message` field in the widget result for specific error details.

***

## Next Steps

* Apply these query patterns in the [Widget Types reference](./widget-types/overview)
* See [Getting Started](./getting-started) for end-to-end examples
* Explore [API Reference](/reference/v5/evaluation-dashboards) for programmatic query creation


# Bar Chart Widget
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/bar

Visualize comparisons across categories using bar charts

<Info>
  **Using Sample Data**: The examples on this page use fields from our <a href="/assets/sample-data/sample-evaluation.csv">sample dataset</a>. Download it and create an evaluation with it to follow along with this tutorial.
</Info>

<Frame>
  <img alt="Bar Chart Widget Example" />
</Frame>

## When to Use

Use bar chart widgets to:

* Compare values across categories or groups
* Show rankings or relative performance
* Visualize aggregated metrics by dimension
* Make patterns and outliers immediately visible

## Configuration

### Required Fields

<ParamField type="string">
  Widget display name shown on the dashboard
</ParamField>

<ParamField type="string">
  Must be set to "bar"
</ParamField>

<ParamField type="object">
  SeriesQuery with `GROUP BY` for categorical breakdown
</ParamField>

### X-Axis Configuration (One Required)

You must provide **either** `x_column` **or** `x_column_group`, but not both:

<ParamField type="string">
  Single column name to use for x-axis categories. Use for simple bar charts grouping by one dimension.

  **Example:** `"agent_name"` - creates bars for each agent
</ParamField>

<ParamField type="string[]">
  Array of column names for multi-dimensional grouping. Use for complex bar charts that group by multiple dimensions.

  **Example:** `["agent_name", "task_type"]` - creates grouped bars combining agent and task type
</ParamField>

### Optional Config Fields

<ParamField type="boolean">
  Controls whether multiple series are stacked or displayed side-by-side.

  * `true` (default): Stack bars on top of each other (useful for showing totals)
  * `false`: Display bars side-by-side (better for comparing individual values)
</ParamField>

<ParamField type="boolean">
  When `true`, each bar's value is rendered directly on the chart so you don't have to hover to read it.

  Value labels apply to single-series charts only (charts with one aggregation). They are ignored on multi-series charts, which use a legend instead.
</ParamField>

### Query Requirements

Bar chart widgets require a **SeriesQuery** with:

* `groupBy` for categorical breakdown (required)
* Should have at least one aggregation for y-axis values
* Can include `filter` for row filtering

## Creating in the UI

<Steps>
  <Step title="Open Widget Creator">
    From your dashboard, click "Add Widget" and select "Bar Chart"
  </Step>

  <Step title="Enter Title">
    Give your chart a descriptive name (e.g., "Agent Name Accuracy Average")
  </Step>

  <Step title="Configure Query">
    <Frame>
      <img alt="Bar Chart Query Builder" />
    </Frame>

    * **Group By**: Select the categorical dimension - these will be the bars on the y-axis
    * **Aggregation**: Choose the metric to visualize (`AVG`, `SUM`, `COUNT`)
    * **Filter**: Add conditions to narrow the data (optional)
  </Step>

  <Step title="Show Values on Bars (Optional)">
    For single-series charts, toggle **Show values on bars** to render each bar's value directly on the chart. This sets `config.show_value_labels`.
  </Step>

  <Step title="Create Widget">
    Click "Add" to generate the bar chart

    <Frame>
      <img alt="Created Bar Chart Widget" />
    </Frame>
  </Step>
</Steps>

## Output Format

Bar chart widgets use the same series format as tables:

```json theme={null}
{
  "type": "series",
  "data": [
    {"category": "Quality", "avg_score": 0.89},
    {"category": "Accuracy", "avg_score": 0.85},
    {"category": "Helpfulness", "avg_score": 0.91}
  ]
}
```

The chart renders each object as a bar.

## Example Use Cases

### Use Case 1: Average Score by Agent

Compare performance across different agents.

<Frame>
  <img alt="Score by Agent Bar Chart" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Average Score by Agent",
      type="bar",
      query={
          "select": [
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "agent_name",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "overall_score",
                      "source": "data"
                  }
              }
          ],
          "groupBy": ["agent_name"]
      },
      config={
          "x_column": "agent_name"
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Average Score by Agent",
    "type": "bar",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}}
      ],
      "groupBy": ["agent_name"]
    },
    "config": {
      "x_column": "agent_name"
    }
  }
  ```
</CodeGroup>

***

### Use Case 2: Evaluation Count by Task Type

Show which task types have the most evaluations.

<Frame>
  <img alt="Count by Task Type Bar Chart" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Evaluations by Task Type",
      type="bar",
      query={
          "select": [
              {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}},
              {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
          ],
          "groupBy": ["task_type"]
      },
      config={"x_column": "task_type"}
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Evaluations by Task Type",
    "type": "bar",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
      ],
      "groupBy": ["task_type"]
    },
    "config": {
      "x_column": "task_type"
    }
  }
  ```
</CodeGroup>

***

### Use Case 3: High Performance Rate by Agent

Visualize percentage of evaluations scoring 85+ across agents.

<Frame>
  <img alt="High Performance Rate Bar Chart" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="High Performance Rate by Agent",
      type="bar",
      query={
          "select": [
              {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "PERCENTAGE",
                      "column": "overall_score",
                      "params": {
                          "percentage_filters": {
                              "conditions": [{
                                  "column": "overall_score",
                                  "source": "data",
                                  "operator": ">=",
                                  "value": 85
                              }]
                          }
                      }
                  }
              }
          ],
          "groupBy": ["agent_name"]
      },
      config={"x_column": "agent_name"}
  )
  ```

  ```json JSON theme={null}
  {
    "title": "High Performance Rate by Agent",
    "type": "bar",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
        {
          "expression": {
            "type": "AGGREGATION",
            "function": "PERCENTAGE",
            "column": "overall_score",
            "params": {
              "percentage_filters": {
                "conditions": [{
                  "column": "overall_score",
                  "source": "data",
                  "operator": ">=",
                  "value": 85
                }]
              }
            }
          }
        }
      ],
      "groupBy": ["agent_name"]
    },
    "config": {
      "x_column": "agent_name"
    }
  }
  ```
</CodeGroup>

***

### Use Case 4: Multi-Dimensional Grouping with x\_column\_group

Group by multiple dimensions to create side-by-side bar comparisons. This example shows average scores grouped by both agent and task type.

<Frame>
  <img alt="Multi-dimensional Grouped Bar Chart" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Score by Agent and Task Type",
      type="bar",
      query={
          "select": [
              {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
              {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}},
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "overall_score",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "relevance_score",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "accuracy_score",
                      "source": "data"
                  }
              }
          ],
          "groupBy": ["agent_name", "task_type"]
      },
      config={
          "x_column_group": ["agent_name", "task_type"],
          "stacked": False  # Display side-by-side for easier comparison
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Score by Agent and Task Type",
    "type": "bar",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
        {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}}
        {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "accuracy_score", "source": "data"}}
      ],
      "groupBy": ["agent_name", "task_type"]
    },
    "config": {
      "x_column_group": ["agent_name", "task_type"],
      "stacked": false
    }
  }
  ```
</CodeGroup>

## Stacked vs Unstacked Series

When you have multiple series (e.g., multiple aggregations or groupings), you can control how they're displayed:

### Stacked Bars (Default)

<Frame>
  <img alt="Stacked Bar Chart" />
</Frame>

Bars are stacked vertically, showing cumulative totals. Best for:

* Showing part-to-whole relationships
* Comparing total values across categories
* Visualizing composition over categories

```python theme={null}
config={"x_column": "agent_name", "stacked": True}  # Default
```

### Unstacked Bars (Side-by-Side)

<Frame>
  <img alt="Unstacked Bar Chart" />
</Frame>

Bars are displayed side-by-side, making individual values easier to compare. Best for:

* Comparing individual series values directly
* When exact values matter more than totals
* Multi-dimensional analysis (e.g., agent + task type)

```python theme={null}
config={"x_column": "agent_name", "stacked": False}
```

## Display Behavior

Bar charts apply the following formatting automatically. These behaviors are built into the rendering, so there are no extra config fields to set.

### Category labels

Category labels are shown next to each bar by default, so you can read values without hovering. Long labels are truncated to fit, and hovering a truncated label reveals its full text in a tooltip.

### Per-category colors

When a chart has a single series and more than one category (for example, an `AVG` grouped by `agent_name`), each bar is given a distinct color from a rotating palette. This makes individual categories easier to tell apart when there are many bars. Charts with multiple series keep one consistent color per series instead, so the series stay comparable across categories.

### Humanized column names

Raw column names are converted to readable Title Case in axis labels, the legend, and tooltips (for example, `experiment_param_value` becomes "Experiment Param Value" and `avgScore` becomes "Avg Score"). Aliases that already contain spaces are shown unchanged, so you can override the displayed text by aliasing a column in your query.

## Related Documentation

* [Table Widget](./table) - Tabular alternative with more detail
* [Histogram Widget](./histogram) - For numerical distributions
* [Query Language](../query-language) - `GROUP BY` and aggregation syntax
* [API Reference](/reference/v5/evaluation-dashboards) - Programmatic chart creation


# Donut Chart Widget
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/donut

Visualize categorical distributions as a donut/pie chart

<Info>
  **Using Sample Data**: The examples on this page use fields from our <a href="/assets/sample-data/sample-evaluation.csv">sample dataset</a>. Download it and create an evaluation with it to follow along with this tutorial.
</Info>

<Frame>
  <img alt="Donut Chart Widget Example" />
</Frame>

## When to Use

Use donut chart widgets to:

* Show proportional distribution of categories
* Visualize percentage breakdowns at a glance
* Compare relative sizes of groups
* Highlight dominant categories in your data

## Configuration

### Required Fields

<ParamField type="string">
  Widget display name shown on the dashboard
</ParamField>

<ParamField type="string">
  Must be set to "donut"
</ParamField>

<ParamField type="object">
  SeriesQuery with `GROUP BY` for categorical breakdown
</ParamField>

### X-Axis Configuration (One Required)

You must provide **either** `x_column` **or** `x_column_group`, but not both:

<ParamField type="string">
  Single column name to group by for slices. Each unique value becomes a slice.

  **Example:** `"task_type"` - creates slices for each task type
</ParamField>

<ParamField type="string[]">
  Array of column names for multi-dimensional grouping. Creates slices for each unique combination.

  **Example:** `["agent_name", "task_type"]` - creates slices for each agent-task combination
</ParamField>

### Query Requirements

Donut chart widgets require a **SeriesQuery** with:

* `groupBy` for categorical breakdown (required)
* At least one aggregation for slice values (`COUNT`, `SUM`, `AVG`, etc.)
* Can include `filter` to narrow the dataset

<Info>
  The donut chart automatically limits display to the **top 10 slices**. Any remaining categories are grouped into an "Other" slice.
</Info>

## Creating in the UI

<Steps>
  <Step title="Open Widget Creator">
    From your dashboard, click "Add Widget" and select "Donut Chart"
  </Step>

  <Step title="Enter Title">
    Give your chart a descriptive name (e.g., "Task Type Distribution")
  </Step>

  <Step title="Configure Query">
    <Frame>
      <img alt="Donut Chart Query Builder" />
    </Frame>

    * **Group By**: Select the categorical dimension - each unique value becomes a slice
    * **Aggregation**: Add a count aggregation to the query. This will be used to size the slices and is handled automatically in the UI.
    * **Filter**: Add conditions to narrow the data (optional)
  </Step>

  <Step title="Create Widget">
    Click "Add" to generate the donut chart

    <Frame>
      <img alt="Created Donut Chart Widget" />
    </Frame>
  </Step>
</Steps>

## Output Format

Donut chart widgets use the same series format as other chart types:

```json theme={null}
{
  "type": "series",
  "data": [
    {"task_type": "question_answering", "count": 12},
    {"task_type": "summarization", "count": 8},
    {"task_type": "code_generation", "count": 7},
    {"task_type": "analysis", "count": 6}
  ]
}
```

Each data point becomes a slice, with the aggregation value determining its proportional size.

## Example Use Cases

### Use Case 1: Distribution by Task Type

Show how evaluation items are distributed across task types.

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Task Type Distribution",
      type="donut",
      query={
          "select": [
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "task_type",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "COUNT",
                      "column": "*"
                  }
              }
          ],
          "groupBy": ["task_type"]
      },
      config={
          "x_column": "task_type"
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Task Type Distribution",
    "type": "donut",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
      ],
      "groupBy": ["task_type"]
    },
    "config": {
      "x_column": "task_type"
    }
  }
  ```
</CodeGroup>

***

### Use Case 2: Agent Workload Distribution

Visualize how many evaluation items each agent has processed.

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Agent Workload",
      type="donut",
      query={
          "select": [
              {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
              {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
          ],
          "groupBy": ["agent_name"]
      },
      config={"x_column": "agent_name"}
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Agent Workload",
    "type": "donut",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
      ],
      "groupBy": ["agent_name"]
    },
    "config": {
      "x_column": "agent_name"
    }
  }
  ```
</CodeGroup>

***

### Use Case 3: Prompt Category Breakdown with Filter

Show the distribution of prompt categories, filtered to only high-scoring items.

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="High-Scoring Items by Category",
      type="donut",
      query={
          "select": [
              {"expression": {"type": "COLUMN", "column": "prompt_category", "source": "data"}},
              {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
          ],
          "filter": {
              "conditions": [
                  {"column": "overall_score", "source": "data", "operator": ">=", "value": 85}
              ]
          },
          "groupBy": ["prompt_category"]
      },
      config={"x_column": "prompt_category"}
  )
  ```

  ```json JSON theme={null}
  {
    "title": "High-Scoring Items by Category",
    "type": "donut",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "prompt_category", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
      ],
      "filter": {
        "conditions": [
          {"column": "overall_score", "source": "data", "operator": ">=", "value": 85}
        ]
      },
      "groupBy": ["prompt_category"]
    },
    "config": {
      "x_column": "prompt_category"
    }
  }
  ```
</CodeGroup>

## Related Documentation

* [Bar Chart Widget](./bar) - For side-by-side categorical comparisons
* [Table Widget](./table) - Detailed numerical breakdowns
* [Query Language](../query-language) - `GROUP BY` and aggregation syntax
* [API Reference](/reference/v5/evaluation-dashboards) - Programmatic chart creation


# Heading Widget
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/heading

Organize dashboards with simple section headers

<Frame>
  <img alt="Heading Widget Example" />
</Frame>

## When to Use

Use heading widgets to:

* Divide dashboard into logical sections
* Label groups of related widgets

## Configuration

### Required Fields

<ParamField type="string">
  The heading text to display
</ParamField>

<ParamField type="string">
  Must be set to "heading"
</ParamField>

**No Query or Config Required**: Heading widgets simply display the title as a styled header.

## Creating in the UI

<Steps>
  <Step title="Open Widget Creator">
    From your dashboard, click "Add Widget" and select "Heading"
  </Step>

  <Step title="Enter Heading Text">
    <Frame>
      <img alt="Heading Widget Creator" />
    </Frame>

    Type the heading text in the title field (e.g., "Overview Metrics")
  </Step>

  <Step title="Create Widget">
    Click "Add" to add the heading to your dashboard

    <Frame>
      <img alt="Created Heading Widget" />
    </Frame>
  </Step>
</Steps>

## Heading vs Markdown

### Use Heading When:

* You only need a simple text header
* You want consistent dashboard styling
* You're organizing multiple short sections
* Simplicity is preferred

### Use Markdown When:

* You need formatted text beyond a header
* You want to include links, lists, or tables
* You're providing detailed context or instructions
* You need multiple paragraphs of content

## Related Documentation

* [Markdown Widget](./markdown) - For rich text content
* [All Widget Types](./overview) - Compare widget types
* [Getting Started](../getting-started) - Dashboard organization tips
* [API Reference](/reference/v5/evaluation-dashboards) - Programmatic heading creation


# Histogram Widget
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/histogram

Visualize data distributions across numerical ranges

<Info>
  **Using Sample Data**: The examples on this page use fields from our <a href="/assets/sample-data/sample-evaluation.csv">sample dataset</a>. Download it and create an evaluation with it to follow along with this tutorial.
</Info>

<Frame>
  <img alt="Histogram Widget Example" />
</Frame>

## When to Use

Use histogram widgets to:

* Show value distributions (groups data by distinct values)
* Visualize frequency counts or aggregated metrics
* Identify patterns in numerical data
* Display aggregated data in a distribution format

## Configuration

### Required Fields

<ParamField type="string">
  Widget display name shown on the dashboard
</ParamField>

<ParamField type="string">
  Must be set to "histogram"
</ParamField>

<ParamField type="object">
  SeriesQuery selecting the column
</ParamField>

### X-Axis Configuration (One Required)

You must provide **either** `x_column` **or** `x_column_group`, but not both:

<ParamField type="string">
  Single column name to group by for the x-axis. The histogram displays distinct values from this column.

  **Example:** `"overall_score"` - groups by each score value (75, 80, 85, etc.)
</ParamField>

<ParamField type="string[]">
  Array of column names for multi-dimensional grouping. Creates separate distributions for each combination.

  **Example:** `["agent_name"]` - creates separate distributions for each agent

  **Note:** Multi-dimensional grouping creates overlaid or stacked distributions for comparison.
</ParamField>

### Optional Config Fields

<ParamField type="boolean">
  Controls whether multiple distributions are stacked or overlaid.

  * `true` (default): Stack distributions on top of each other
  * `false`: Overlay distributions for easier comparison
</ParamField>

<ParamField type="boolean">
  When `true`, each bar's value is rendered directly on the chart so you don't have to hover to read it.

  Value labels apply to single-series histograms only (those with one aggregation). They are ignored on multi-series histograms, which use a legend instead.
</ParamField>

### Query Requirements

Histogram widgets require a **SeriesQuery** with:

* Select the column you want to group by (becomes the x-axis)
* Can include aggregations (`COUNT`, `SUM`, `AVG`, etc.) for the y-axis
* If no aggregation is specified, `COUNT` is used by default
* Can include `filter` to narrow the dataset

## Creating in the UI

<Steps>
  <Step title="Open Widget Creator">
    From your dashboard, click "Add Widget" and select "Histogram"
  </Step>

  <Step title="Enter Title">
    Give your histogram a descriptive name (e.g., "Score Distribution")
  </Step>

  <Step title="Select Column">
    Choose the column to group by (this will be the x-axis)
  </Step>

  <Step title="Add Aggregations (Optional)">
    <Frame>
      <img alt="Histogram Aggregation Builder" />
    </Frame>

    Add aggregations for the y-axis. If none specified, `COUNT` is used by default.
  </Step>

  <Step title="Add Filters (Optional)">
    <Frame>
      <img alt="Histogram Filter Builder" />
    </Frame>

    Optionally filter to specific subsets of data
  </Step>

  <Step title="Show Values on Bars (Optional)">
    For single-series histograms, toggle **Show values on bars** to render each bar's value directly on the chart. This sets `config.show_value_labels`.
  </Step>

  <Step title="Create Widget">
    Click "Add" to generate the histogram

    <Frame>
      <img alt="Created Histogram Widget" />
    </Frame>
  </Step>
</Steps>

## Output Format

Histogram widgets return grouped data:

```json theme={null}
{
  "type": "series",
  "data": [
    {"score": 75, "count": 12},
    {"score": 80, "count": 45},
    {"score": 85, "count": 89},
    {"score": 90, "count": 156},
    ...
  ]
}
```

Each data point represents a distinct value from the grouped column and its corresponding count or aggregated metric.

## Example Use Cases

### Use Case 1: Overall Score Distribution

Visualize how overall scores are distributed across all evaluations. This will by default show the count of evaluations for each score value.

<Frame>
  <img alt="Overall Score Distribution Histogram" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Overall Score Distribution",
      type="histogram",
      query={
          "select": [
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "overall_score",
                      "source": "data"
                  }
              },
              {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
          ]
          "groupBy": ["overall_score"]
      },
      config={
          "x_column": "overall_score"
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Overall Score Distribution",
    "type": "histogram",
    "query": {
      "select": [
        {
          "expression": {
            "type": "COLUMN",
            "column": "overall_score",
            "source": "data"
          }
        },
        {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
      ],
      "groupBy": ["overall_score"]
    },
    "config": {
      "x_column": "overall_score"
    }
  }
  ```
</CodeGroup>

***

### Use Case 2: Technical Task Score Distribution

Show distribution for evaluations in the technical category only.

<Frame>
  <img alt="Technical Task Distribution" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Technical Task Score Distribution",
      type="histogram",
      query={
          "select": [
              {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}},
              {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
          ],
          "filter": {
              "conditions": [
                  {"column": "prompt_category", "source": "data", "operator": "=", "value": "technical"}
              ]
          },
          "groupBy": ["overall_score"]
      },
      config={"x_column": "overall_score"}
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Technical Task Score Distribution",
    "type": "histogram",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}
      ],
      "filter": {
        "conditions": [
          {"column": "prompt_category", "source": "data", "operator": "=", "value": "technical"}
        ]
      },
      "groupBy": ["overall_score"]
    },
    "config": {
      "x_column": "overall_score"
    }
  }
  ```
</CodeGroup>

***

### Use Case 3: Comparing Distributions Across Categories

Create overlaid histograms to compare score distributions across different scores.

<Frame>
  <img alt="Grouped Histogram Comparing Agents" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Score Distribution by Score",
      type="histogram",
      query={
          "select": [
              {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}},
              {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "fluency_score", "source": "data"}},
              {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data"}}
          ],
          "groupBy": ["overall_score"]
      },
      config={
          "x_column": "overall_score",
          "stacked": True
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Score Distribution by Score",
    "type": "histogram",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "fluency_score", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data"}}
      ],
      "groupBy": ["overall_score"]
    },
    "config": {
      "x_column": "overall_score",
      "stacked": true
    }
  }
  ```
</CodeGroup>

## Stacked vs Overlaid Distributions

When comparing distributions across groups, you can control visualization style:

### Overlaid Distributions (Unstacked)

<Frame>
  <img alt="Overlaid Histogram" />
</Frame>

Multiple distributions shown with transparency, overlapping each other. Best for:

* Comparing distribution shapes directly
* Identifying differences in peaks and spread
* Seeing all distributions simultaneously

```python theme={null}
config={"x_column": "overall_score", "stacked": False}
```

### Stacked Distributions

<Frame>
  <img alt="Stacked Histogram" />
</Frame>

Distributions stacked vertically to show combined totals. Best for:

* Showing total counts across all groups
* Part-to-whole relationships
* When cumulative view is meaningful

```python theme={null}
config={"x_column": "overall_score", "stacked": True}  # Default
```

## Display Behavior

Histograms apply the following formatting automatically. These behaviors are built into the rendering, so there are no extra config fields to set.

### Per-category colors

When a histogram has a single series and more than one bucket, each bar is given a distinct color from a rotating palette, making individual buckets easier to tell apart. Histograms with multiple series keep one consistent color per series instead, so the series stay comparable across buckets.

### Humanized column names

Raw column names are converted to readable Title Case in axis labels, the legend, and tooltips (for example, `overall_score` becomes "Overall Score"). Aliases that already contain spaces are shown unchanged, so you can override the displayed text by aliasing a column in your query.

## Related Documentation

* [Bar Chart Widget](./bar) - For categorical distributions
* [Metric Widget](./metric) - Show percentiles and statistical measures
* [Table Widget](./table) - Detailed numerical breakdowns
* [Query Language](../query-language) - Filter and column syntax


# Markdown Widget
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/markdown

Add rich text documentation, instructions, and notes to dashboards

<Frame>
  <img alt="Markdown Widget Example" />
</Frame>

## When to Use

Use markdown widgets to:

* Document evaluation methodology
* Provide context for dashboard viewers
* Add instructions or guidelines
* Include links to related resources
* Create formatted notes and observations
* Define thresholds and alert criteria
* Credit data sources and team members

## Configuration

### Required Fields

<ParamField type="string">
  Widget title for reference (not displayed in rendered output)
</ParamField>

<ParamField type="string">
  Must be set to "markdown"
</ParamField>

<ParamField type="string">
  Markdown-formatted text to display
</ParamField>

### Supported Markdown Syntax

Markdown widgets support standard CommonMark syntax:

* **Headers**: `# H1`, `## H2`, `### H3`
* **Bold**: `**bold text**`
* **Italic**: `*italic text*`
* **Lists**: Bulleted (`- item`) and numbered (`1. item`)
* **Links**: `[text](url)`
* **Code**: Inline `` `code` `` and fenced code blocks
* **Blockquotes**: `> quote`
* **Horizontal rules**: `---`
* **Tables**: Standard markdown tables

## Creating in the UI

<Steps>
  <Step title="Open Widget Creator">
    From your dashboard, click "Add Widget" and select "Markdown"
  </Step>

  <Step title="Enter Title">
    Give your widget a reference name (e.g., "Dashboard Overview")
  </Step>

  <Step title="Write Content">
    <Frame>
      <img alt="Markdown Editor" />
    </Frame>

    Write or paste your markdown content in the editor. Most editors provide:
  </Step>

  <Step title="Preview">
    <Frame>
      <img alt="Markdown Preview" />
    </Frame>

    Check the rendered output to ensure formatting looks correct
  </Step>

  <Step title="Create Widget">
    Click "Add" to add to your dashboard

    <Frame>
      <img alt="Created Markdown Widget" />
    </Frame>
  </Step>
</Steps>

## Related Documentation

* [Heading Widget](./heading) - For simple section headers
* [All Widget Types](./overview) - Compare widget types
* [Getting Started](../getting-started) - Dashboard creation workflow
* [API Reference](/reference/v5/evaluation-dashboards) - Programmatic markdown widget creation


# Metric Widget
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/metric

Display single aggregation values like averages, counts, and percentiles

<Info>
  **Using Sample Data**: The examples on this page use fields from our <a href="/assets/sample-data/sample-evaluation.csv">sample dataset</a>. Download it and create an evaluation with it to follow along with this tutorial.
</Info>

<Frame>
  <img alt="Metric Widget Example" />
</Frame>

## When to Use

Use metric widgets to:

* Display top-level aggregation values (e.g., "Average Model Score: 0.87")
* Show summary statistics (e.g., "Total Items: 1,247")
* Highlight key thresholds (e.g., "95th Percentile Latency: 234ms")
* Create metric cards for at-a-glance monitoring
* Track pass rates and percentages

## Configuration

### Required Fields

<ParamField type="string">
  Widget display name shown on the dashboard
</ParamField>

<ParamField type="string">
  Must be set to "metric"
</ParamField>

<ParamField type="object">
  MetricQuery with exactly one aggregation function
</ParamField>

### Query Requirements

Metric widgets require a **MetricQuery** with these constraints:

* Must have exactly one item in the `select` clause
* That item must be an `AGGREGATION` expression (not a raw column)
* Can optionally include `filter` conditions
* Cannot use `groupBy`, `orderBy`, or `limit`

## Creating in the UI

<Steps>
  <Step title="Open Widget Creator">
    From your dashboard, click "Add Widget" and select "Query Value"
  </Step>

  <Step title="Enter Title">
    Give your metric a descriptive name (e.g., "Average Score")
  </Step>

  <Step title="Configure Query">
    <Frame>
      <img alt="Metric Query Builder" />
    </Frame>

    * **Column**: Select the field to aggregate
    * **Aggregation**: Choose the function
    * **Filter** (optional): Add conditions to narrow the data
  </Step>

  <Step title="Create Widget">
    Click "Add" to compute and display the result

    <Frame>
      <img alt="Created Metric Widget" />
    </Frame>
  </Step>
</Steps>

## Output Format

Metric widgets return a single scalar value:

```json theme={null}
{
  "type": "metric",
  "data": 0.873
}
```

The `data` field contains the computed number, which can be:

* A decimal number (0.873)
* An integer (1247)
* A percentage (0.85 representing 85%)

## Example Use Cases

### Use Case 1: Average Overall Score

Display the mean overall score across all evaluation items.

<Frame>
  <img alt="Average Overall Score Metric" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Average Overall Score",
      type="metric",
      query={
          "select": [
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "overall_score",
                      "source": "data"
                  }
              }
          ]
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Average Overall Score",
    "type": "metric",
    "query": {
      "select": [
        {
          "expression": {
            "type": "AGGREGATION",
            "function": "AVG",
            "column": "overall_score",
            "source": "data"
          }
        }
      ]
    }
  }
  ```
</CodeGroup>

***

### Use Case 2: Total Evaluation Count

Show how many evaluation items are in the dataset.

<Frame>
  <img alt="Total Evaluations Metric" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Total Evaluations",
      type="metric",
      query={
          "select": [
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "COUNT",
                      "column": "*"
                  }
              }
          ]
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Total Evaluations",
    "type": "metric",
    "query": {
      "select": [
        {
          "expression": {
            "type": "AGGREGATION",
            "function": "COUNT",
            "column": "*"
          }
        }
      ]
    }
  }
  ```
</CodeGroup>

***

### Use Case 3: High Performance Rate

Calculate percentage of evaluations scoring 85 or above.

<Frame>
  <img alt="High Performance Rate Metric" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="High Performance Rate",
      type="metric",
      query={
          "select": [
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "PERCENTAGE",
                      "column": "overall_score",
                      "params": {
                          "percentage_filters": {
                              "conditions": [
                                  {
                                      "column": "overall_score",
                                      "source": "data",
                                      "operator": ">=",
                                      "value": 85
                                  }
                              ]
                          }
                      }
                  }
              }
          ]
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "High Performance Rate",
    "type": "metric",
    "query": {
      "select": [
        {
          "expression": {
            "type": "AGGREGATION",
            "function": "PERCENTAGE",
            "column": "overall_score",
            "params": {
              "percentage_filters": {
                "conditions": [
                  {
                    "column": "overall_score",
                    "source": "data",
                    "operator": ">=",
                    "value": 85
                  }
                ]
              }
            }
          }
        }
      ]
    }
  }
  ```
</CodeGroup>

***

### Use Case 4: 95th Percentile Overall Score

Show the 95th percentile to understand top-end performance.

<Frame>
  <img alt="95th Percentile Overall Score Metric" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="95th Percentile Overall Score",
      type="metric",
      query={
          "select": [
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "PERCENTILE",
                      "column": "overall_score",
                      "source": "data",
                      "params": {
                          "percentile": 95
                      }
                  }
              }
          ]
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "95th Percentile Overall Score",
    "type": "metric",
    "query": {
      "select": [
        {
          "expression": {
            "type": "AGGREGATION",
            "function": "PERCENTILE",
            "column": "overall_score",
            "source": "data",
            "params": {
              "percentile": 95
            }
          }
        }
      ]
    }
  }
  ```
</CodeGroup>

***

### Use Case 5: Average Score for Technical Tasks

Calculate average score for only technical task types.

<Frame>
  <img alt="Technical Tasks Avg Score Metric" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Technical Tasks Avg Score",
      type="metric",
      query={
          "select": [
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "overall_score",
                      "source": "data"
                  }
              }
          ],
          "filter": {
              "conditions": [
                  {
                      "column": "prompt_category",
                      "source": "data",
                      "operator": "=",
                      "value": "technical"
                  }
              ]
          }
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Technical Tasks Avg Score",
    "type": "metric",
    "query": {
      "select": [
        {
          "expression": {
            "type": "AGGREGATION",
            "function": "AVG",
            "column": "overall_score",
            "source": "data"
          }
        }
      ],
      "filter": {
        "conditions": [
          {
            "column": "prompt_category",
            "source": "data",
            "operator": "=",
            "value": "technical"
          }
        ]
      }
    }
  }
  ```
</CodeGroup>

## Related Documentation

* [Query Language Reference](../query-language) - Full query syntax guide
* [All Widget Types](./overview) - Compare different widget types
* [Table Widget](./table) - For multi-row displays
* [API Reference](/reference/v5/evaluation-dashboards) - Explore programmatic widget creation


# Widget Types Overview
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/overview

Overview of all available widget types for evaluation dashboards

Dashboard widgets are visualization components that display evaluation data in different formats. This guide provides an overview of all available widget types.

## Available Widget Types

### Data-Driven Widgets

These widgets require queries and compute results from evaluation data:

<CardGroup>
  <Card title="Metric Widget" icon="hashtag" href="./metric">
    Display a single computed value like averages, counts, or percentiles
  </Card>

  <Card title="Table Widget" icon="table" href="./table">
    Show multi-dimensional data in rows and columns with grouping, sorting, and conditional formatting
  </Card>

  <Card title="Bar Chart" icon="chart-bar" href="./bar">
    Visualize comparisons across categories with bar charts
  </Card>

  <Card title="Histogram" icon="chart-simple" href="./histogram">
    Display data distributions across numerical ranges
  </Card>

  <Card title="Donut Chart" icon="chart-pie" href="./donut">
    Show proportional distribution of categories as a donut/pie chart
  </Card>

  <Card title="Scatter Plot" icon="chart-scatter" href="./scatter">
    Explore correlations between two numerical dimensions
  </Card>

  <Card title="Timeseries" icon="chart-line" href="./timeseries">
    Track metrics over time with line charts
  </Card>
</CardGroup>

### Static Content Widgets

These widgets display static content without queries:

<CardGroup>
  <Card title="Markdown Widget" icon="file-lines" href="./markdown">
    Add rich text documentation, instructions, and notes
  </Card>

  <Card title="Heading Widget" icon="heading" href="./heading">
    Organize dashboards with section headers
  </Card>
</CardGroup>

## Quick Reference

| Widget Type    | Purpose                        | Requires Query | Best For                                |
| -------------- | ------------------------------ | -------------- | --------------------------------------- |
| **Metric**     | Single aggregation value       | Yes            | Top-level metrics, summary stats        |
| **Table**      | Tabular data display           | Yes            | Detailed breakdowns, grouped data       |
| **Bar**        | Bar chart visualization        | Yes            | Categorical comparisons                 |
| **Histogram**  | Distribution visualization     | Yes            | Understanding distribution of data      |
| **Donut**      | Proportional distribution      | Yes            | Category breakdowns, percentage views   |
| **Scatter**    | X-Y relationship visualization | Yes            | Correlations, outlier detection         |
| **Timeseries** | Trends over time               | Yes            | Tracking metrics across evaluation runs |
| **Markdown**   | Rich text content              | No             | Documentation, instructions, notes      |
| **Heading**    | Section headers                | No             | Dashboard organization                  |

## Choosing the Right Widget Type

### Display a Single Number

Use a **Metric widget** when you need to show one computed value prominently:

* Overall average score
* Total item count
* Pass rate percentage
* 95th percentile latency

### Show Detailed Breakdowns

Use a **Table widget** when you need to display multiple dimensions and metrics:

* Performance by model and category
* Top 10 highest/lowest scoring items
* Multi-column statistical summaries

### Compare Categories

Use a **Bar Chart widget** to visualize how values differ across groups:

* Score by category
* Model performance comparison
* Items per status

### Visualize Distributions

Use a **Histogram widget** to understand how values are distributed:

* Score distribution
* Latency distribution
* Confidence level spread

### Show Proportional Breakdowns

Use a **Donut Chart widget** to visualize how categories are distributed:

* Task type distribution
* Agent workload share
* Prompt category breakdown

### Explore Correlations

Use a **Scatter Plot widget** to examine relationships between two metrics:

* Accuracy vs. relevance
* Score vs. response length
* Latency vs. quality

### Track Trends Over Time

Use a **Timeseries widget** to monitor metrics across evaluation runs:

* Score progression across evaluations
* Multi-metric trends in evaluation groups
* Performance tracking over time

### Add Documentation

Use **Markdown widgets** for context and instructions:

* Dashboard methodology
* Alert thresholds
* Team contact information
* Analysis guidelines

### Organize Sections

Use **Heading widgets** to structure your dashboard:

* "Overview Metrics"
* "Detailed Analysis"
* "Performance by Category"

## Widget Configuration Overview

All widgets share common fields:

**Required Fields:**

* `title`: Display name for the widget
* `type`: One of: metric, table, bar, histogram, donut, scatter, timeseries, markdown, heading

**Query Field (for data-driven widgets):**

* `query`: QueryAST object defining the computation

**Optional Fields:**

* `config`: Widget-specific display configuration

## Data Sources

Data-driven widgets can query two sources:

* **`data.*`**: User-defined evaluation data (your custom fields)
* **`task_result_cache.*`**: Task execution results

Both support nested fields using dot notation (e.g., `data.metadata.model_version`).

## Reusing Widgets

Widgets can be shared across multiple dashboards. When you include the same widget ID in different dashboards' `widget_order`, they all use the same widget definition but produce different results based on their evaluation context.

<Warning>
  When reusing widgets, make sure the shape of the data for each evaluation is the same. Otherwise, the widget will not be able to compute the result.
</Warning>

## Next Steps

Explore detailed documentation for each widget type:

* [Metric Widget](./metric) - Single value displays
* [Table Widget](./table) - Tabular data with conditional formatting
* [Bar Chart Widget](./bar) - Category comparisons
* [Histogram Widget](./histogram) - Distribution visualization
* [Donut Chart Widget](./donut) - Proportional distributions
* [Scatter Plot Widget](./scatter) - Correlation analysis
* [Timeseries Widget](./timeseries) - Trends over time
* [Markdown Widget](./markdown) - Rich text content
* [Heading Widget](./heading) - Section headers

Or continue learning:

* [Evaluation Group Dashboards](../eval-group-dashboards) - Compare across evaluations
* [Query Language](../query-language) - Build powerful queries
* [Getting Started](../getting-started) - Create your first dashboard
* [API Reference](/reference/v5/evaluation-dashboards) - Explore programmatic query creation


# Scatter Plot Widget
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/scatter

Visualize relationships between two numerical dimensions

<Info>
  **Using Sample Data**: The examples on this page use fields from our <a href="/assets/sample-data/sample-evaluation.csv">sample dataset</a>. Download it and create an evaluation with it to follow along with this tutorial.
</Info>

<Frame>
  <img alt="Scatter Plot Widget Example" />
</Frame>

## When to Use

Use scatter plot widgets to:

* Explore correlations between two numerical metrics
* Identify outliers in multi-dimensional data
* Visualize X-Y relationships with optional bubble sizing
* Spot clusters or patterns across evaluation items

## Configuration

### Required Fields

<ParamField type="string">
  Widget display name shown on the dashboard
</ParamField>

<ParamField type="string">
  Must be set to "scatter"
</ParamField>

<ParamField type="object">
  SeriesQuery selecting at least two numeric columns
</ParamField>

### X-Axis Configuration (One Required)

You must provide **either** `x_column` **or** `x_column_group`, but not both:

<ParamField type="string">
  Column name for the x-axis. Must be a numeric column.

  **Example:** `"accuracy_score"` - plots accuracy on the x-axis
</ParamField>

<ParamField type="string[]">
  Array of column names for multi-dimensional grouping on the x-axis.

  **Example:** `["agent_name", "accuracy_score"]`
</ParamField>

### Query Requirements

Scatter plot widgets require a **SeriesQuery** with:

* At least two numeric columns selected (one for x-axis, one for y-axis)
* Optional `GROUP BY` for creating grouped scatter points
* When grouped, aggregations determine the plotted values and dot size scales dynamically based on the count

<Info>
  Dot sizes are dynamically scaled based on the aggregation value, with the maximum size determined by the largest value in the dataset.
</Info>

## Creating in the UI

<Steps>
  <Step title="Open Widget Creator">
    From your dashboard, click "Add Widget" and select "Scatter Plot"
  </Step>

  <Step title="Enter Title">
    Give your chart a descriptive name (e.g., "Accuracy vs. Relevance")
  </Step>

  <Step title="Configure Axes">
    <Frame>
      <img alt="Scatter Plot Form" />
    </Frame>

    * **X-Axis**: Select the numeric column for the horizontal axis
    * **Y-Axis**: The remaining numeric columns become the y-axis values
    * **Group By**: Optionally group data points by a categorical dimension
    * **Filter**: Add conditions to narrow the data (optional)
  </Step>

  <Step title="Create Widget">
    Click "Add" to generate the scatter plot

    <Frame>
      <img alt="Created Scatter Plot Widget" />
    </Frame>
  </Step>
</Steps>

## Output Format

Scatter plot widgets return row data for each point:

```json theme={null}
{
  "type": "series",
  "data": [
    {"accuracy_score": 92, "relevance_score": 85, "count": 5},
    {"accuracy_score": 78, "relevance_score": 91, "count": 3},
    {"accuracy_score": 88, "relevance_score": 82, "count": 7}
  ]
}
```

Each data point represents a dot on the scatter plot. When grouped, the aggregation value determines both the position and the dot size.

## Example Use Cases

### Use Case 1: Accuracy vs. Relevance Scores

Explore the relationship between accuracy and relevance across evaluation items.

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Accuracy vs. Relevance",
      type="scatter",
      query={
          "select": [
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "accuracy_score",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "relevance_score",
                      "source": "data"
                  }
              }
          ]
      },
      config={
          "x_column": "accuracy_score"
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Accuracy vs. Relevance",
    "type": "scatter",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "accuracy_score", "source": "data"}},
        {"expression": {"type": "COLUMN", "column": "relevance_score", "source": "data"}}
      ]
    },
    "config": {
      "x_column": "accuracy_score"
    }
  }
  ```
</CodeGroup>

***

### Use Case 2: Overall Score vs. Response Length

Explore whether longer responses correlate with higher or lower scores.

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Overall Score vs. Response Length",
      type="scatter",
      query={
          "select": [
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "overall_score",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "response_length",
                      "source": "data"
                  }
              }
          ]
      },
      config={
          "x_column": "overall_score"
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Overall Score vs. Response Length",
    "type": "scatter",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}},
        {"expression": {"type": "COLUMN", "column": "response_length", "source": "data"}}
      ]
    },
    "config": {
      "x_column": "overall_score"
    }
  }
  ```
</CodeGroup>

## Related Documentation

* [Bar Chart Widget](./bar) - For categorical comparisons
* [Histogram Widget](./histogram) - For distribution visualization
* [Query Language](../query-language) - Filter and aggregation syntax
* [API Reference](/reference/v5/evaluation-dashboards) - Programmatic chart creation


# Table Widget
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/table

Display multi-dimensional data in rows and columns with grouping and sorting

<Info>
  **Using Sample Data**: The examples on this page use fields from our <a href="/assets/sample-data/sample-evaluation.csv">sample dataset</a>. Download it and create an evaluation with it to follow along with this tutorial.
</Info>

<Frame>
  <img alt="Table Widget Example" />
</Frame>

## When to Use

Use table widgets to:

* Show detailed breakdowns by multiple dimensions
* Display ranked lists (top/bottom performers)
* Present multi-column statistical summaries
* Create export-friendly data views
* Compare metrics across categories
* Show individual item details

## Configuration

### Required Fields

<ParamField type="string">
  Widget display name shown on the dashboard
</ParamField>

<ParamField type="string">
  Must be set to "table"
</ParamField>

<ParamField type="object">
  SeriesQuery returning multiple rows
</ParamField>

### Query Requirements

Table widgets require a **SeriesQuery** with:

* At least one item in the `select` clause
* Can mix `COLUMN` and `AGGREGATION` expressions
* Supports `filter` for row filtering
* Supports `groupBy` for dimensional breakdowns
* Supports `limit` for top-N results

<Info>
  When using `groupBy`, all non-aggregated columns in `select` must appear in the `groupBy` array.
</Info>

## Creating in the UI

<Steps>
  <Step title="Open Widget Creator">
    From your dashboard, click "Add Widget" and select "Table"
  </Step>

  <Step title="Turn off Grouping (Optional)">
    Decide whether to group by columns or not by toggling the "Group By" toggle. If you turn off grouping, you will not be able to aggregate data.

    <Info>
      When grouping is turned on, we will automatically output the count of the group.
    </Info>
  </Step>

  <Step title="Select Columns">
    If aggregating, specify which columns to group by, if not grouping select the columns you want to display.
  </Step>

  <Step title="Enter Title">
    Give your table a descriptive name (e.g., "Average Scores by Agent")
  </Step>

  <Step title="Add Aggregations (Optional)">
    <Frame>
      <img alt="Table Aggregation Builder" />
    </Frame>

    Add metrics to display:

    * **Columns**: Select fields from your data
    * **Aggregations**: Add computed metrics (AVG, COUNT, etc.)
  </Step>

  <Step title="Add Filters (Optional)">
    <Frame>
      <img alt="Table Filter Builder" />
    </Frame>

    Add conditions to narrow the data, these conditions will be applied to the data before it is aggregated.
  </Step>

  <Step title="Create Widget">
    Click "Add" to display the table

    <Frame>
      <img alt="Created Table Widget" />
    </Frame>
  </Step>
</Steps>

## Conditional Formatting

Apply color-coded formatting rules to table cells to highlight important values at a glance. Conditional formatting is available on grouped tables (tables with aggregations).

<Frame>
  <img alt="Table with Conditional Formatting" />
</Frame>

### Enabling in the UI

When creating or editing a grouped table widget, expand the "Formatting" section to add rules. Select the aggregation column to format, choose a rule type, and pick a color.

<Frame>
  <img alt="Conditional Formatting Form" />
</Frame>

### Rule Types

#### Threshold Rules

Apply formatting when a cell value meets a comparison condition.

| Field              | Type   | Description                                        |
| ------------------ | ------ | -------------------------------------------------- |
| `type`             | string | Must be `"threshold"`                              |
| `column`           | string | Aggregation column name to format                  |
| `operator`         | string | One of: `=`, `>`, `<`, `>=`, `<=`                  |
| `threshold`        | number | Value to compare against                           |
| `background_color` | string | One of: `green`, `yellow`, `red`, `purple`, `blue` |

#### Range Rules

Apply formatting when a cell value falls within a range (inclusive).

| Field              | Type   | Description                                        |
| ------------------ | ------ | -------------------------------------------------- |
| `type`             | string | Must be `"range"`                                  |
| `column`           | string | Aggregation column name to format                  |
| `min_threshold`    | number | Lower bound (inclusive)                            |
| `max_threshold`    | number | Upper bound (inclusive)                            |
| `background_color` | string | One of: `green`, `yellow`, `red`, `purple`, `blue` |

### SDK Example

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Agent Scores with Formatting",
      type="table",
      query={
          "select": [
              {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "overall_score",
                      "source": "data"
                  },
                  "alias": "avg_score"
              }
          ],
          "groupBy": ["agent_name"]
      },
      config={
          "conditional_formatting": [
              {
                  "type": "threshold",
                  "column": "avg_score",
                  "operator": ">=",
                  "threshold": 90,
                  "background_color": "green"
              },
              {
                  "type": "range",
                  "column": "avg_score",
                  "min_threshold": 70,
                  "max_threshold": 89,
                  "background_color": "yellow"
              },
              {
                  "type": "threshold",
                  "column": "avg_score",
                  "operator": "<",
                  "threshold": 70,
                  "background_color": "red"
              }
          ]
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Agent Scores with Formatting",
    "type": "table",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
        {
          "expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"},
          "alias": "avg_score"
        }
      ],
      "groupBy": ["agent_name"]
    },
    "config": {
      "conditional_formatting": [
        {
          "type": "threshold",
          "column": "avg_score",
          "operator": ">=",
          "threshold": 90,
          "background_color": "green"
        },
        {
          "type": "range",
          "column": "avg_score",
          "min_threshold": 70,
          "max_threshold": 89,
          "background_color": "yellow"
        },
        {
          "type": "threshold",
          "column": "avg_score",
          "operator": "<",
          "threshold": 70,
          "background_color": "red"
        }
      ]
    }
  }
  ```
</CodeGroup>

### Formatting Behavior

* Rules are evaluated in order -- the **first matching rule wins**
* Only applies to aggregation columns in grouped tables
* Available colors: `green`, `yellow`, `red`, `purple`, `blue`
* For percentage columns, formatting comparisons use the display value (0-100), not the raw decimal (0-1)

## Output Format

Table widgets return an array of row objects:

```json theme={null}
{
  "type": "series",
  "data": [
    {"category": "Quality", "count": 342, "avg_score": 0.89},
    {"category": "Accuracy", "count": 298, "avg_score": 0.85},
    {"category": "Helpfulness", "count": 411, "avg_score": 0.91}
  ]
}
```

Each object in the `data` array represents one row, with keys corresponding to column names.

## Example Use Cases

### Use Case 1: Agent Performance Breakdown

Show statistics grouped by agent with multiple metrics.

<Frame>
  <img alt="Agent Performance Table" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Performance by Agent",
      type="table",
      query={
          "select": [
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "agent_name",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "COUNT",
                      "column": "*"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "overall_score",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "MIN",
                      "column": "overall_score",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "MAX",
                      "column": "overall_score",
                      "source": "data"
                  }
              }
          ],
          "groupBy": ["agent_name"],
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Performance by Agent",
    "type": "table",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}},
        {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "MIN", "column": "overall_score", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "MAX", "column": "overall_score", "source": "data"}}
      ],
      "groupBy": ["agent_name"]
    }
  }
  ```
</CodeGroup>

***

### Use Case 2: Top 10 Highest-Scoring Evaluations

Display the best-performing individual evaluations without aggregation.

<Info>
  In the UI sort using the table header, this will not be reflected in the query.
</Info>

<Frame>
  <img alt="Top 10 Evaluations Table" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Top 10 Evaluations",
      type="table",
      query={
          "select": [
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "agent_name",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "task_type",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "overall_score",
                      "source": "data"
                  }
              }
          ],
          "orderBy": [
              {
                  "column": "overall_score",
                  "direction": "DESC"
              }
          ],
          "limit": 10
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Top 10 Evaluations",
    "type": "table",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
        {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}},
        {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}}
      ],
      "orderBy": [{"column": "overall_score", "direction": "DESC"}],
      "limit": 10
    }
  }
  ```
</CodeGroup>

***

### Use Case 3: Agent Comparison with Multiple Metrics

Compare agents across several statistical measures including accuracy and relevance.

<Frame>
  <img alt="Agent Comparison Table" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Agent Performance Comparison",
      type="table",
      query={
          "select": [
              {
                  "expression": {
                      "type": "COLUMN",
                      "column": "agent_name",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "COUNT",
                      "column": "*"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "overall_score",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "accuracy_score",
                      "source": "data"
                  }
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "relevance_score",
                      "source": "data"
                  }
              }
          ],
          "groupBy": ["agent_name"]
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Agent Performance Comparison",
    "type": "table",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}},
        {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "accuracy_score", "source": "data"}},
        {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data"}}
      ],
      "groupBy": ["agent_name"]
    }
  }
  ```
</CodeGroup>

***

### Use Case 4: Low-Scoring Evaluations Review

Show evaluations scoring below 80 that need attention.

<Frame>
  <img alt="Filtered Evaluations Table" />
</Frame>

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Low-Scoring Evaluations",
      type="table",
      query={
          "select": [
              {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
              {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}},
              {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}},
              {"expression": {"type": "COLUMN", "column": "accuracy_score", "source": "data"}}
          ],
          "filter": {
              "conditions": [
                  {
                      "column": "overall_score",
                      "source": "data",
                      "operator": "<",
                      "value": 80
                  }
              ]
          }
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Low-Scoring Evaluations",
    "type": "table",
    "query": {
      "select": [
        {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}},
        {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}},
        {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}},
        {"expression": {"type": "COLUMN", "column": "accuracy_score", "source": "data"}}
      ],
      "filter": {
        "conditions": [
          {
            "column": "overall_score",
            "source": "data",
            "operator": "<",
            "value": 80
          }
        ]
      }
    }
  }
  ```
</CodeGroup>

## Related Documentation

* [Query Language Reference](../query-language) - `GROUP BY` and `ORDER BY` syntax
* [Bar Chart Widget](./bar) - Visual alternative to tables
* [Metric Widget](./metric) - For single-value displays
* [API Reference](/reference/v5/evaluation-dashboards) - Programmatic table creation


# Timeseries Widget
Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/timeseries

Track metrics over time with line charts

<Warning>
  Timeseries widgets are only supported in evaluation group dashboards. They are not available when creating a dashboard for a single evaluation.
</Warning>

<Frame>
  <img alt="Timeseries Widget Example" />
</Frame>

## When to Use

Use timeseries widgets to:

* Track metric trends over time across evaluation runs
* Compare performance across evaluations in an evaluation group
* Monitor score progression across multiple evaluation iterations
* Visualize how metrics change as new evaluations are added to a group

## Configuration

### Required Fields

<ParamField type="string">
  Widget display name shown on the dashboard
</ParamField>

<ParamField type="string">
  Must be set to "timeseries"
</ParamField>

<ParamField type="object">
  SeriesQuery with at least one aggregation
</ParamField>

### Query Requirements

Timeseries widgets require a **SeriesQuery** with:

* At least one aggregation in the `select` clause
* The x-axis is determined automatically based on context:
  * **Evaluation group dashboards**: Uses the evaluation creation date as the x-axis, plotting one line per evaluation
* Can include `filter` to narrow the dataset
* Supports per-aggregation `evaluation_ids` for eval group dashboards (see below)

## Creating in the UI

<Steps>
  <Step title="Open Widget Creator">
    From your dashboard, click "Add Widget" and select "Timeseries"
  </Step>

  <Step title="Enter Title">
    Give your chart a descriptive name (e.g., "Average Score Over Time")
  </Step>

  <Step title="Configure Aggregations">
    <Frame>
      <img alt="Timeseries Form" />
    </Frame>

    Add one or more aggregations to track over time:

    * **Column**: Select the numeric field to aggregate
    * **Aggregation**: Choose the function (`AVG`, `SUM`, `COUNT`, etc.)
    * **Alias**: Name for the metric line (optional but recommended)
    * **Evaluations**: For eval group dashboards, select which evaluations to include per aggregation
  </Step>

  <Step title="Add Filters (Optional)">
    Add conditions to narrow the data before aggregation.
  </Step>

  <Step title="Create Widget">
    Click "Add" to generate the timeseries chart

    <Frame>
      <img alt="Created Timeseries Widget" />
    </Frame>
  </Step>
</Steps>

## Output Format

Timeseries widgets return data points for each time period:

```json theme={null}
{
  "type": "series",
  "data": [
    {"evaluation": "Eval Run 1", "avg_score": 0.82, "created_at": "2026-01-15"},
    {"evaluation": "Eval Run 2", "avg_score": 0.85, "created_at": "2026-01-22"},
    {"evaluation": "Eval Run 3", "avg_score": 0.89, "created_at": "2026-01-29"}
  ]
}
```

## Evaluation Group Context

In evaluation group dashboards, timeseries widgets have special behavior:

* **One line per metric**: Each metric in the query is plotted as a separate line on the chart
* **X-axis**: The evaluation creation date determines the position on the x-axis
* **Per-evaluation selection**: You can choose which evaluations to include per aggregation, allowing you to compare specific subsets
* **Auto-update**: When evaluations are added to or removed from the group, the timeseries automatically updates to reflect the new membership

### Example: Average Score Trend Across Evaluations

Track how average scores change across evaluation runs in a group.

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Average Score Trend",
      type="timeseries",
      query={
          "select": [
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "overall_score",
                      "source": "data"
                  },
                  "alias": "avg_overall_score"
              }
          ],
          "evaluation_ids": ["eval-1", "eval-2", "eval-3"]
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Average Score Trend",
    "type": "timeseries",
    "query": {
      "select": [
        {
          "expression": {
            "type": "AGGREGATION",
            "function": "AVG",
            "column": "overall_score",
            "source": "data"
          },
          "alias": "avg_overall_score"
        }
      ],
      "evaluation_ids": ["eval-1", "eval-2", "eval-3"]
    }
  }
  ```
</CodeGroup>

***

### Example: Multi-Metric Comparison Over Time

Track multiple metrics across evaluations to see how different scores evolve.

<CodeGroup>
  ```python Python theme={null}
  widget = client.evaluation_dashboards.widgets.create(
      dashboard_id=dashboard.id,
      title="Score Metrics Over Time",
      type="timeseries",
      query={
          "select": [
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "accuracy_score",
                      "source": "data"
                  },
                  "alias": "avg_accuracy"
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "relevance_score",
                      "source": "data"
                  },
                  "alias": "avg_relevance"
              },
              {
                  "expression": {
                      "type": "AGGREGATION",
                      "function": "AVG",
                      "column": "coherence_score",
                      "source": "data"
                  },
                  "alias": "avg_coherence"
              }
          ],
          "evaluation_ids": ["eval-1", "eval-2", "eval-3", "eval-4"]
      }
  )
  ```

  ```json JSON theme={null}
  {
    "title": "Score Metrics Over Time",
    "type": "timeseries",
    "query": {
      "select": [
        {
          "expression": {"type": "AGGREGATION", "function": "AVG", "column": "accuracy_score", "source": "data"},
          "alias": "avg_accuracy"
        },
        {
          "expression": {"type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data"},
          "alias": "avg_relevance"
        },
        {
          "expression": {"type": "AGGREGATION", "function": "AVG", "column": "coherence_score", "source": "data"},
          "alias": "avg_coherence"
        }
      ],
      "evaluation_ids": ["eval-1", "eval-2", "eval-3", "eval-4"]
    }
  }
  ```
</CodeGroup>

## Related Documentation

* [Evaluation Group Dashboards](../eval-group-dashboards) - Full guide on group dashboards
* [Bar Chart Widget](./bar) - For categorical comparisons
* [Metric Widget](./metric) - For single computed values
* [Query Language](../query-language) - Aggregation and filter syntax
* [API Reference](/reference/v5/evaluation-dashboards) - Programmatic chart creation


# Conditional Rubrics
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/conditional-rubrics

Enhance evaluation flexibility and complexity with per-row run conditions

## Overview

Rather than a task being applied uniformly across all rows, conditional rubrics allow for
rows to be pre-filtered on some `run_condition`.

Filtered out rows receive a `"skipped"` status. They appear as empty in the evaluation results table
and are excluded from evaluation dashboard calculations.

The following example filters out rows where the `subject` is not `'math'`.

<img alt="conditional-rubric-evaluation-results" />

Run conditions are currently supported via the SDK only for auto evaluations of type `guided_decoding`.

### Format

The `run_condition` object is a discriminated union on `op` with the following properties supported:

* Leaf types: `const`, `var` (item locator references)
* Comparison ops: `eq`, `ne`, `lt`, `lte`, `gt`, `gte`
* Set ops: `in`, `not_in`
* Logical ops: `and`, `or`, `not`, `is_null`, `is_not_null`

### Example

#### Simple `run_condition`

```
"run_condition":{
    "op": "eq",
    "left": { "op": "var", "path": "item.subject" },
    "right": { "op": "const", "value": "math" }
},
```

#### Nested `run_condition`

```
"run_condition": {
  "op": "or",
  "operands": [
    {
      "op": "eq",
      "left":  { "op": "var", "path": "item.Score" },
      "right": { "op": "const", "value": 5 }
    },
    {
      "op": "eq",
      "left":  { "op": "var", "path": "item.Score" },
      "right": { "op": "const", "value": 3 }
    }
  ]
}
```


# Quick Start - AgentOps SDK
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/getting-started

Getting started with the ScaleGP AgentOps SDK

You can interact with the API through HTTP requests from any language or via our official Python bindings (alpha)

## Authentication

Make sure to follow the previous steps described in [Authentication](../authentication) to obtain an API key and account ID.

## Installing the SGP SDK Client

To install the ScaleGP Python client, run the following command:

```python theme={null}
pip install scale-gp-beta
```

## Configuring the Client

The SGP client can be configured with your API key and account ID. Examples on how to use the SGP client can be found in [Recipes](/recipes). The simplest approach is to set environment variables:

```bash theme={null}
export SGP_API_KEY="your-api-key-here"
export SGP_ACCOUNT_ID="your-account-id-here"
```

Then instantiate the client:

```python theme={null}
from scale_gp_beta import SGPClient

client = SGPClient()
```

The SDK will automatically use the `SGP_API_KEY` and `SGP_ACCOUNT_ID` environment variables. You can also pass them explicitly:

```python theme={null}
from scale_gp_beta import SGPClient

client = SGPClient(
    account_id="your-account-id",
    api_key="your-api-key",
)
```

The `SGPClient` constructor accepts the following parameters:

| Parameter     | Description                                                                                                          | Default                               |
| ------------- | -------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| `account_id`  | Your SGP account ID. Found in the [Accounts](https://app.sgp.scale.com/admin/accounts) tab of the admin dashboard.   | `SGP_ACCOUNT_ID` environment variable |
| `api_key`     | Your SGP API key. Found in the [API Keys](https://app.sgp.scale.com/admin/api-key) tab of the admin dashboard.       | `SGP_API_KEY` environment variable    |
| `environment` | The SGP environment to connect to. **Possible values:** `production-multitenant`, `staging`, `development`, `local`. | `production-multitenant`              |
| `base_url`    | Override the base URL with a custom endpoint. Takes precedence over `environment`.                                   | None                                  |

### Connecting to Non-Production Environments

Use the `environment` parameter to connect to development or staging environments:

```python theme={null}
client = SGPClient(
    account_id="your-account-id",
    api_key="your-api-key",
    environment="development"
)
```

You can also override the URL directly via the `SGP_CLIENT_BASE_URL` environment variable or the `base_url` parameter (which takes precedence over `environment`).

<Warning>
  The Scale GP Python client is under active development, and many of the features in this document rely on the latest versions. To make sure you have the latest version, run `pip install scale-gp-beta --upgrade`.
</Warning>


# Auditing Evaluations
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/labeling-evaluations/auditing-evaluations

Guide for auditing contributor evaluations

## Overview

The platform supports auditing in order to enable users to review tasks that are completed by labelers.
Currently, the platform supports two levels of auditing. A user can only complete the first level of auditing after the task has been labeled.
A user can then only complete the second level of auditing after the first level of auditing has been completed.

## Auditing

<img alt="Task Table" />

The task table shows the status of all the tasks. To learn how to assign auditors to tasks, see [Managing Contributor Evaluations](/docs/v5/next-gen-evaluation/labeling-evaluations/managing-contributor-evaluations).
Users can assign themselves as auditors with manager permissions. But for each task, the person completing the task, the person doing the first level of audit, and the person doing the second level of audit must all be different people.

### Complete an Audit assigned to you

In order to complete an Audit that is assigned to you, click on the **Audit** button in the table of the corresponding task.
This will show the task with the auditing functionality.

<img alt="Audit Task UI" />

* **Mark as "Revision Needed"**: Request the contributor to revise their work
* **Fix Task**: Make corrections directly as the auditor
* **Approve**: Accept the task as completed correctly

If a user has marked the task for review, the audit view will also show that here. This is an example of what a task that has been flagged for review by the labler will look like for the auditor.

<img alt="Audit Flag for Review" />

There will be a section on the bottom right showing that the original labeler left a comment and flagged the task for review.


# Contributor and Auditor Metrics
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/labeling-evaluations/contributor-auditor-metrics

Understanding metrics for contributors and auditors

To see an aggregation of the performance of labelers and auditors on the platform, navigate to the **Contributor Metrics** section of annotation tasks.

<img alt="Contributor Metrics" />

This section provides deeper insights into the performance of the contributors that are labeling on this platform.


# Labeling Evaluations
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/labeling-evaluations/labeling-evaluations

Guide for labeling contributor evaluations

## Overview

This guide is designed for human annotators to understand how to label evaluations created through the human evaluations creation flow. For information on creating contributor evaluations, see [Contributor Evaluation](/docs/v5/next-gen-evaluation/ui-evaluations/contributor-evaluation).

### Start Labeling

<img alt="Start Labeling" />

To start labeling a task, click on the **Label** button next to a task that is assigned to your name.
If you don't have any assigned tasks but want to label a task, you can assign a task to yourself and then click the **Label** button.

### Labeling Tasks

<img alt="Labeling UI" />

When you open a task, you will see the information related to an evaluation run on the left side of the task screen.
On the right side, there will be a series of questions that you should answer as the annotator.

* **Save**: There is no save button. All tasks are auto saved when you answer questions. If you navigate away from a task and return to it, you will see answers you previously answerd.
* **Skip**: Skip. If you are unsure about how to answer this task, then you can utilize the Skip button to skip this task.
* **Submit**: Submit. When you are done with a task, use the Submit button to submit it. Submitting a task allows it to be audited by an auditor. You are allowed to edit a task after you submit if it has not yet been audited.
* **Flag for Review**: If you want to submit a task but have concerns about the task you want to flag to an auditor, toggle the flag for review button and leave a comment. The auditor will see your comment and see that you flagged this for review.


# Managing Contributor Evaluations
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/labeling-evaluations/managing-contributor-evaluations

Guide for managing contributor evaluations

## Overview

After setting up a [contributor evaluation](/docs/v5/next-gen-evaluation/ui-evaluations/contributor-evaluation), navigate to the **Annotation Tasks** section in the **Evaluations** tab to manage and monitor your evaluation tasks.

<img alt="Annotation Task Overview" />

## Assigning Tasks

Users can assign tasks to others either by going to the **Assigned To** column and selecting a user to assign each individual task to a user. Or they can use the checkbox and bulk assign tasks to a user.

### Individual Task Assignment

<img alt="Individual Task Assignment" />

### Bulk Assign Tasks

<img alt="Bulk Assign Tasks" />

## Assigning Auditors to Tasks

Users can also assign auditors to tasks by going to the **Audit Tasks** section. The platform allows for two levels of auditing. The first level of auditing can only be completed after the task is completed.
The second level of auditing can only be completed after the first level of auditing is completed.
For a single task, the user who labels must be different from the first level auditor. The second level auditor must be different from both the first level auditor and the person who did the labeling.

<img alt="Select Assign Auditors" />

<img alt="Assign Auditors" />


# Introduction to Next Gen Evals
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/overview

A simplified and extendable update to SGP's evaluation framework

## Overview

Evaluations have been overhauled in the upcoming iteration of SGP to decrease overhead while increasing capabilities for both UI and SDK users. The API is currently available in the latest version of SGP, and access to the new UI is available behind a feature flag, which can be enabled for an account and/or deployment on request.

At the most basic level, an evaluation now consists of:

1. **Data**: Unstructured, user defined data, optionally, in the form of a reusable dataset.
2. **Tasks**: A set of instructions to produce results based on said data. For example, a chat completion task based on an `input` field within the user defined data, or a human annotation task to produce the expected output. These tasks can be interdependent and composed to create as simple or complex of an evaluation that your use case requires.

## Creation

<Info>Examples in this guide use the [`scale-gp-beta`](https://pypi.org/project/scale-gp-beta/) package which runs exclusively on the V5 API.</Info>

### Simple evaluation creation via SDK

The following example illustrates how to create a simple evaluation through the SDK:

```python {8-9, 12-24} theme={null}
from scale_gp_beta import SGPClient

client = SGPClient(account_id=..., api_key=...)

my_evaluation = client.evaluations.create(
  name="My Example Evaluation",
  data=[
    { "input": "How does NextGen eval on SGP work?" },
    { "input": "How do I define tasks?" },
  ],
  tasks=[
    {
      "task_type": "chat_completion",
      "alias": "output",
      "configuration": {
        "model": "openai/gpt-4o",
        "messages": [
          {
            "role": "user",
            "content": "item.input"
          }
        ]
      }
    }
  ]
)
```

In this case, our `data` is made up of two items, each with an `input` field. A single task is specified to generate a chat completion from each data item.

### Evaluation with files creation via SDK

Files can be associated with evaluations for use in multimodal tasks or when your evaluation items need to reference uploaded content. First upload your files, then include them in the evaluation:

<Warning>
  Evaluations and datasets that contain files must be created via the API or SDK. This functionality is not currently supported in the UI.
</Warning>

```python theme={null}
from scale_gp_beta import SGPClient

client = SGPClient(account_id=..., api_key=...)

# Create evaluation with files
my_multimodal_evaluation = client.evaluations.create(
  name="My Multimodal Evaluation",
  data=[
    { "input_column": "This is the first row of the evaluation" },
    { "input_column": "This is the second row of the evaluation" },
  ],
  files=[
    {"file_column1": "sample_file_id_1", "file_column2": "sample_file_id_2"},
    {"file_column1": "sample_file_id_3", "file_column2": "sample_file_id_4"},
  ],
  tasks=[
    {
      "task_type":"contributor_evaluation.question",
      "alias":"Mispronunciation timestamps_61142007",
      "configuration":{
        "question_id":"61142007-bd18-48a4-b034-a0670a4a23f5",
        "queue_id":"default",
        "layout":{
            "direction":"column",
            "children":[
              {
                  "data":"item.input"
              },
              {
                  "data":"item.files.file_column1"
              }
            ]
        },
        "required":false
      }
    }
  ]
)
```

In this example, already uploaded files are referenced in the `files` array and they have a one to one mapping to the `data` array. This means that in the resultant evaluation, the first item from data and and the first item from files will collectively form the first row.

## Data

Each object within the `data` field can be thought of as a row in a table. We refer to these rows as items, and they can be retrieved like so:

```python theme={null}
client.evaluation_items.list(
  evaluation_id=my_evaluation.id
)
```

<Expandable title="example output">
  ```python {5,11} theme={null}
  [
    {
      "id": "0dc5611a-...",
      "object": "evaluation.item"
      "data": { "input": "How does NextGen eval on SGP work?" },
      ...
    },
    {
      "id": "1549a01e-...",
      "object": "evaluation.item"
      "data": { "input": "How do I define tasks?" },
      ...
    }
  ]
  ```
</Expandable>

Optionally, a dataset can be created for reusability and comparison across different evaluations:

```python {1-7, 11} theme={null}
my_dataset = client.datasets.create(
  name="My Example Dataset",
  data=[
    { "input": "How does NextGen eval on SGP work?" },
    { "input": "How do I define tasks?" }
  ]
)

my_evaluation = client.evaluations.create(
  name="My Example Evaluation",
  dataset_id=my_dataset.id
  tasks=[
    ...
  ]
)
```

## Tasks

Tasks can be thought of as instructions to produce "results" for each item within the evaluation, and are run asynchronously after evaluation creation. They contain 3 parts:

<ResponseField name="task_type" type="enum">
  The type of task to run (a list of supported types can be found [here](/reference/v5/evaluations/create-evaluation#body-tasks))
</ResponseField>

<ResponseField name="configuration" type="object">
  Used to specify `task_type` specific parameters, such as the `messages` field for a `chat_completion` task.

  <Accordion title="Referencing item data">
    A string following the format `"item.<field>"`, here on referred to as an `ItemLocator`, can be used within a configuration to indicate that the value should be pulled from the evaluation item at task execution time.

    For example, in the first code snippet we defined a `chat_completion` task that used an `ItemLocator` as the `content` field for a user message:

    ```python {6} theme={null}
    {
      "model": "openai/gpt-4o",
      "messages": [
        {
          "role": "user",
          "content": "item.input"
        }
      ]
    }
    ```

    When the task is ultimately executed, a chat completion request will be made for each evaluation item with the `content` field populated based on the item's `input` field.

    Data can also be referenced within a string by wrapping an `ItemLocator` in double curly braces:

    ```python {6} theme={null}
    {
      "model": "openai/gpt-4o",
      "messages": [
        {
          "role": "user",
          "content": "Answer the following question: {{item.input}}"
        }
      ]
    }
    ```
  </Accordion>
</ResponseField>

<ResponseField name="alias" type="string">
  Aliases are an optional way to specify the field name that will contain the task's result for each evaluation item. By default, this will be the `task_type`.
</ResponseField>


# Agent Evaluation
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/tasks/agent-evaluation

Leverage specialized agents to produce evaluation results.

## Agent-as-a-Judge

Use the `auto_evaluation.agent` task type for configuring auto evaluation tasks with specialized agents.

<Expandable title="configuration properties">
  <ResponseField name="model" type="string">
    The identifier of the model (in `<model_provider>/<model_name>` format) that the Agent will use to execute the task.
  </ResponseField>

  <ResponseField name="output_rules" type="string[]">
    A space to provide additional instructions to the agent.
  </ResponseField>

  <ResponseField name="output_type" type="string">
    If not provided, `output_type` is inferred from output rules.
  </ResponseField>

  <ResponseField name="temperature" type="float">
    For the `APEAgent` (Default Agent) you can configure a temperature
  </ResponseField>
</Expandable>

### Example Usage

The following illustrates a basic example in which an agent task is defined with an Instruction Following Agent to evaluate the adherence of a given response to a prompt.

```python theme={null}
client.evaluations.create(
  name="Example Instruction Following Evaluation,
  data=[
    {
      "prompt": "You must either describe the process of photosynthesis in exactly 20 words or explain why it’s important in a single sentence under 10 words — choose whichever option is more precise.",
      "response": "Photosynthesis converts sunlight, water, and carbon dioxide into glucose and oxygen, sustaining plant life and fueling ecosystems.",
    },
    ...
  ],
  tasks=[
    {
      "task_type": "auto_evaluation.agent",
      "alias": "instruction-following-agent",
      "configuration": {
        "name": "instruction-following-agent",
        "definition": "Perform comprehensive analysis of instruction following",
        "output_rules": [
            "Score adherence on a 1-5 scale", 
            "List specific violations and well-followed instructions"
        ],
        "output_type": "integer",
        "designated_to": {
                "agent_name": "IFAgent",
                "config": {
                    "model": "openai/o3-mini"
                }
            }
      }
    }
  ]
)
```


# Agentex Evaluation
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/tasks/agentex-evaluation

Generate outputs using external Agentex agents.

## Agentex Output

Use the `agentex_output` task type to generate outputs from an external Agentex agent for each item in your evaluation.

<Expandable title="configuration properties">
  <ResponseField name="agentex_agent_id" type="string">
    The ID of the Agentex agent to use for generating outputs.
  </ResponseField>

  <ResponseField name="input_column" type="string">
    The dataset column to use as input for the agent, specified as an ItemLocator (e.g., `"item.input"`). See [referencing item data](/docs/v5/next-gen-evaluation/overview#referencing-item-data).
  </ResponseField>

  <ResponseField name="include_traces" type="boolean">
    Whether to include trace data in the results. When enabled, execution spans from the agent are captured alongside the output.
  </ResponseField>
</Expandable>

### Example Usage

The following illustrates a complete example in which an Agentex agent is used to generate responses for each evaluation item.

```python theme={null}
from scale_gp_beta import SGPClient
import time

# Initialize the client
client = SGPClient(
    account_id="your-account-id",
    api_key="your-api-key"
)

# Create an evaluation with Agentex output task
evaluation = client.evaluations.create(
    name="My Agent Evaluation",
    data=[
        {"input": "What is the capital of France?"},
        {"input": "Explain quantum computing in simple terms."},
    ],
    tasks=[
        {
            "task_type": "agentex_output",
            "alias": "agent_response",
            "configuration": {
                "agentex_agent_id": "your-agent-id",
                "input_column": "item.input",
                "include_traces": True
            }
        }
    ]
)

print(f"Created evaluation: {evaluation.id}")

# Poll until complete
while True:
    eval_status = client.evaluations.retrieve(evaluation_id=evaluation.id)
    print(f"Status: {eval_status.status}")
    
    if eval_status.status == "completed":
        items = client.evaluation_items.list(evaluation_id=evaluation.id)
        
        for item in items:
            print(f"\nInput: {item.data['input']}")
            print(f"Output: {item.data['agent_response']['output']}")
            if item.data['agent_response'].get('trace'):
                print(f"Trace spans: {len(item.data['agent_response']['trace'])}")
        break
    
    time.sleep(5)
```

<Note>
  For Azure workspace deployments, use the `/api` path suffix in the base URL:

  ```python theme={null}
  from scale_gp_beta import SGPClient

  client = SGPClient(
      account_id="your-account-id",
      api_key="your-api-key",
      base_url="https://your-workspace.azure.workspace.egp.scale.com/api"
  )
  ```
</Note>

### Output Structure

The task produces the following output for each evaluation item:

**Successful response:**

```json theme={null}
{
  "output": "The agent's final text response",
  "trace": [...]  // Only if include_traces=True - list of execution spans
}
```

**Error response:**

```json theme={null}
{
  "output": {
    "error": "Error message",
    "error_type": "timeout | null_input | extraction_error | connection_error"
  }
}
```

### Combining with Other Tasks

Agentex outputs are generated before other evaluation tasks run. This allows you to chain the agent output with other tasks like auto evaluations or contributor evaluations.

```python theme={null}
evaluation = client.evaluations.create(
    name="Agentex with Judge Evaluation",
    data=[
        {
            "input": "What is the capital of France?",
            "expected_output": "Paris"
        },
        ...
    ],
    tasks=[
        # First, generate output from Agentex agent
        {
            "task_type": "agentex_output",
            "alias": "agent_response",
            "configuration": {
                "agentex_agent_id": "your-agent-id",
                "input_column": "item.input",
                "include_traces": False
            }
        },
        # Then, evaluate the output with an LLM judge
        {
            "task_type": "auto_evaluation.guided_decoding",
            "alias": "correctness",
            "configuration": {
                "model": "openai/gpt-4o",
                "prompt": """
                    Given the user's query: {{item.input}},
                    The agent's response was: {{item.agent_response.output}}
                    The expected response is: {{item.expected_output}}
                    Did the agent's response correctly answer the question?
                """,
                "choices": ["Yes", "No"]
            }
        }
    ]
)
```


# Auto Evaluation
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/tasks/auto-evaluation

Leverage LLM-based tasks to produce evaluation results.

## Guided Decoding

Use the `auto_evaluation.guided_decoding` task type for configuring auto evaluation tasks where the set of potential results are well defined.

<Expandable title="configuration properties">
  <ResponseField name="model" type="string">
    The identifier of the model (in `<model_provider>/<model_name>` format) to generate the auto evaluation result. Note this model must support some form of guided decoding (e.g. [OpenAI's response formatting](https://platform.openai.com/docs/guides/structured-outputs)) in order for results to be computed.
  </ResponseField>

  <ResponseField name="system_prompt" type="string">
    An optional system prompt to be sent as the first message of the chat completion request.
  </ResponseField>

  <ResponseField name="prompt" type="string">
    The user prompt containing the evaluation question and any relevant data from the evaluation item (see [referencing item data](/docs/v5/next-gen-evaluation/overview#referencing-item-data)).
  </ResponseField>

  <ResponseField name="run_condition" type="object">
    A condition that determines whether the model will be called for each row.
  </ResponseField>

  <ResponseField name="response_format" type="string[]">
    A list of responses for the model to return, each with a name and type configuration.
  </ResponseField>

  <ResponseField name="inference_args" type="object">
    Any additional properties to be included in the chat completion request to the model<br />(e.g. `{ "temperature": 0 }`).
  </ResponseField>
</Expandable>

### Example Usage

The following illustrates a basic example in which a guided decoding task is defined to evaluate the correctness of a generated output compared to the ground truth.

```python theme={null}
client.evaluations.create(
  name="Example Correctness Evaluation",
  data=[
    {
      "input": "What color is the sky?",
      "expected_output": "Blue",
      "generated_output": "The sky appears blue during ..."
    },
    ...
  ],
  tasks=[
    {
      "task_type": "auto_evaluation.guided_decoding",
      "alias": "correctness",
      "configuration": {
        "model": "openai/o3-mini",
        "prompt": """
          Given the user's query: {{item.input}},
          The agent's response was: {{item.generated_output}}
          The expected response is: {{item.expected_output}}
          Did the agent's response fully represent the expected response?
        """,
        "response_format":
          {
            "type":"object",
            "properties":
              {
                "question-response":
                {
                  "type":"string",
                  "enum":["yes","no"]
                }
              },
            "required":["question-response"]
          }
      }
    }
  ]
)
```

When defining a task, you can also customize the response format. For example, you can have the judge LLM provide a reason for the final score, which is often helpful. The following example demonstrates several options.

```
tasks = [
  {
    "task_type": "auto_evaluation.guided_decoding",
    "alias": "multi_response_option_judge",
    "configuration": {
      "model": "openai/gpt-4o",
      "prompt": "Evaluate this response...",
      "response_format": {
        "type": "object",
        "properties": {
          "is_helpful": {
            "type": "boolean",
            "description": "Whether the response is helpful"
          },
          "quality_score": {
            "type": "integer",
            "minimum": 1,
            "maximum": 5,
            "description": "Quality score from 1 to 5"
          },
          "accuracy_score": {
            "type": "number",
            "minimum": 0.0,
            "maximum": 1.0,
            "description": "Accuracy as a decimal"
          },
          "category": {
            "type": "string",
            "enum": ["excellent", "good", "fair", "poor"],
            "description": "Quality category"
          },
          "reasoning": {
            "type": "string",
            "description": "Explanation of the evaluation"
          }
        },
        "required": ["is_helpful", "quality_score", "accuracy_score", "category", "reasoning"]
      }
    }
  }
]
```


# Contributor Evaluation
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/tasks/contributor-evaluation

Defer evaluation tasks to human contributors.

## Question

Use the `contributor_evaluation.question` task type to ask contributors [questions](https://egp.dashboard.scale.com/evaluation/questions/questions) in the annotation UI.

<Expandable title="configuration properties">
  <ResponseField name="question_id" type="string">
    The ID of the question to include in the contributor annotation flow. Questions are most easily created through the UI [here](https://egp.dashboard.scale.com/evaluation/questions/questions/new).
  </ResponseField>

  <ResponseField name="layout" type="object">
    Controls the presentation of evaluation item data in the annotation UI.

    <Expandable title="child attributes">
      <ResponseField name="children" type="object[]">
        A list of elements to be rendered within the layout container. Can be either an individual `Component` object, or a nested `Container`.

        <Card title="">
          <Tabs>
            <Tab title="Component">
              <Expandable title="child attributes">
                <ResponseField name="data" type="ItemLocator">
                  A pointer to the data in each evaluation item to be displayed within the component.
                </ResponseField>

                <ResponseField name="label" type="string">
                  An optional display name to be used at the top of the component. Defaults to the field name from the `ItemLocator` used for `data`.
                </ResponseField>
              </Expandable>
            </Tab>

            <Tab title="Container">
              <Expandable title="child attributes">
                <ResponseField name="children" type="object[]">
                  A list of elements to be rendered within the layout container. Can be either an individual `Component` object, or a nested `Container`.
                </ResponseField>

                <ResponseField name="direction" type="enum<'row' | 'column'>">
                  The axis that children are placed in the container. Based on CSS [`flex-direction`](https://developer.mozilla.org/en-US/docs/Web/CSS/flex-direction).
                </ResponseField>
              </Expandable>
            </Tab>
          </Tabs>
        </Card>
      </ResponseField>

      <ResponseField name="direction" type="enum<'row' | 'column'>">
        The axis that children are placed in the container. Based on CSS [`flex-direction`](https://developer.mozilla.org/en-US/docs/Web/CSS/flex-direction).
      </ResponseField>
    </Expandable>
  </ResponseField>

  <ResponseField name="queue_id" type="string">
    An optional group identifier. Any `contributor_evaluation` tasks that share the same `queue_id` will be grouped together in the annotation UI.
  </ResponseField>
</Expandable>

### Example Usage

Assume we have a categorical question with ID `a0b25d45-7c6c-4480-9871-61ae0fd0b819` configured as such:

<img />

We can create an annotation queue as part of our evaluation to ask our contributors to answer this question in order to produce the same `correctness` result as the [guided decoding example](/docs/v5/next-gen-evaluation/tasks/auto-evaluation#example-usage):

```python theme={null}
client.evaluations.create(
  name="Example Correctness Evaluation",
  data=[
    {
      "input": "What color is the sky?",
      "expected_output": "Blue",
      "generated_output": "The sky appears blue during ..."
    },
    ...
  ],
  tasks=[
    {
      "task_type": "contributor_evaluation.question",
      "alias": "correctness",
      "configuration": {
        "question_id": "a0b25d45-7c6c-4480-9871-61ae0fd0b819",
        "layout": {
          "direction": "column",
          "children": [
            # Component:
            { "data": "item.input", "label": "User Query" },
            # Nested container:
            {
              "direction": "row",
              "children": [
                { "data": "item.expected_output", "label": "Expected Output" },
                { "data": "item.generated_output", "label": "Generated Output" }
              ]
            }
          ]
        },
        "queue_id": "default"
      }
    }
  ]
)
```


# Custom Function Evaluation
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/tasks/custom-function

Define your own Python scoring functions and run them as evaluation tasks.

## Overview

Use the `custom_function` task type to run your own Python scoring logic on each evaluation item, giving you full control over scoring beyond the built-in metrics. The function source code is extracted, sent to the API, and executed server-side in a sandboxed environment.

<Info>The [`scale-gp-beta`](https://pypi.org/project/scale-gp-beta/) SDK provides a `CustomFunction` helper that handles source extraction and serialization for you.</Info>

## SDK Helper

The `CustomFunction` class wraps a Python callable and provides methods to serialize it for the API and test it with a dry run.

```python theme={null}
from scale_gp_beta import SGPClient
from scale_gp_beta.lib import CustomFunction

client = SGPClient(account_id=..., api_key=...)

def string_similarity(expected: str, actual: str) -> float:
    import difflib
    return difflib.SequenceMatcher(None, expected, actual).ratio()

cf = CustomFunction(func=string_similarity)
```

<Warning>Any imports your function needs must be placed **inside the function body**. Module-level imports are not captured when the source is extracted and will not be available at execution time.</Warning>

<Expandable title="CustomFunction constructor parameters">
  <ResponseField name="func" type="Callable">
    The Python function to wrap. Must be a named function defined in a regular `.py` file — lambdas and built-in functions are not supported.
  </ResponseField>

  <ResponseField name="alias" type="string">
    Display name for the task result. Defaults to the function's `__name__`.
  </ResponseField>

  <ResponseField name="arg_mapping" type="Dict[str, str]">
    Maps function parameter names to dataset column names. By default, parameter names are matched directly to column names. Values are automatically prefixed with `item.` unless they already start with `item.` (to support nested paths like `item.nested.field`).
  </ResponseField>
</Expandable>

### Allowed Imports

Custom functions run in a restricted environment. Only these standard library modules are available:

`math`, `json`, `re`, `collections`, `itertools`, `functools`, `statistics`, `decimal`, `fractions`, `datetime`, `copy`, `textwrap`, `difflib`, `unicodedata`

### Return Type

Functions must return an `int` or `float`. Returning other types (including `bool`) will produce an error.

## Using `arg_mapping`

By default, function parameter names are matched to dataset column names. Use `arg_mapping` when your function parameters don't match the column names in your data:

```python theme={null}
def score(a: float, b: float) -> float:
    return a - b

cf = CustomFunction(
    func=score,
    alias="difference_score",
    arg_mapping={"a": "model_score", "b": "baseline_score"},
)
```

Values that already start with `item.` are passed through unchanged, which lets you reference nested fields:

```python theme={null}
cf = CustomFunction(
    func=score,
    arg_mapping={"a": "item.nested.model_score", "b": "baseline_score"},
)
# produces: {"a": "item.nested.model_score", "b": "item.baseline_score"}
```

## Dry Run

Test your function against sample data before creating a full evaluation. The dry run endpoint is synchronous — it executes your function inline and returns per-row results immediately.

```python theme={null}
result = cf.dry_run(
    client=client,
    sample_data=[
        {"expected": "Hello world", "actual": "Hello wrold"},
        {"expected": "The quick brown fox jumps over the lazy dog.", "actual": "The quick brown fox jumped over the lazy dog."},
    ],
)
print(result)
```

## Discovering Columns

If you already have an evaluation and want to see which columns are available for `arg_mapping`:

```python theme={null}
from scale_gp_beta.lib import get_evaluation_columns

columns = get_evaluation_columns(client, "eval_abc123")
for col in columns:
    print(col.field_name, col.data_type)
```

## Example Usage

The following illustrates a complete example creating an evaluation with a custom function task.

```python theme={null}
from scale_gp_beta import SGPClient
from scale_gp_beta.lib import CustomFunction

client = SGPClient(account_id=..., api_key=...)

def string_similarity(expected: str, actual: str) -> float:
    import difflib
    return difflib.SequenceMatcher(None, expected, actual).ratio()

cf_similarity = CustomFunction(func=string_similarity)

def length_ratio(expected: str, actual: str) -> float:
    if len(expected) == 0:
        return 0.0
    return len(actual) / len(expected)

cf_length = CustomFunction(func=length_ratio)

evaluation = client.evaluations.create(
    name="Custom Scoring Evaluation",
    data=[
        {"expected": "Hello world", "actual": "Hello wrold"},
        {"expected": "The quick brown fox jumps over the lazy dog.", "actual": "The quick brown fox jumped over the lazy dog."},
    ],
    tasks=[cf_similarity.serialize(), cf_length.serialize()],
)
```

You can also create evaluations using the raw task dict directly, without the SDK helper:

```python theme={null}
evaluation = client.evaluations.create(
    name="Custom Scoring Evaluation",
    data=[
        {"expected": "Hello world", "actual": "Hello wrold"},
        {"expected": "The quick brown fox jumps over the lazy dog.", "actual": "The quick brown fox jumped over the lazy dog."},
    ],
    tasks=[
        {
            "task_type": "custom_function",
            "alias": "string_similarity",
            "configuration": {
                "function_source": "def string_similarity(expected: str, actual: str) -> float:\n    import difflib\n    return difflib.SequenceMatcher(None, expected, actual).ratio()\n",
            }
        }
    ],
)
```

Poll for completion and inspect results:

```python theme={null}
import time

while True:
    evaluation = client.evaluations.retrieve(evaluation.id)
    if evaluation.status in ("completed", "failed"):
        break
    time.sleep(5)

items = client.evaluation_items.list(evaluation_id=evaluation.id)
for item in items:
    print(item.data)  # scores are stored under the task alias key, e.g. data["string_similarity"]
    if item.task_errors:
        print(item.task_errors)
```

Once the evaluation completes, you can also view the results — including per-item custom function scores — in the SGP UI.

## Constraints

| Constraint         | Detail                                                    |
| :----------------- | :-------------------------------------------------------- |
| Return type        | `int` or `float` only (not `bool`)                        |
| Imports            | Must be inside function body; only allowed stdlib modules |
| Function signature | No `*args`, `**kwargs`, async, or lambdas                 |
| Dry run timeout    | 10s per row, 30s total                                    |

<Expandable title="configuration properties">
  <ResponseField name="function_source" type="string">
    The full source code of the Python function to execute. The `CustomFunction` helper extracts this automatically via `inspect.getsource()`.
  </ResponseField>

  <ResponseField name="arg_mapping" type="object">
    A mapping from function parameter names to `ItemLocator` strings (e.g. `{"param": "item.column_name"}`). When omitted, parameter names are matched directly to column names.
  </ResponseField>
</Expandable>


# Agent as a Judge
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/ui-evaluations/agent-evaluation

Leverage specialized agents to produce evaluation results.

## Create Evaluation

### Navigate to the Evaluate tab and click "Create evaluation".

<img alt="create-evaluation" />

### Add Evaluation Details

Add in Evaluation name, description (optional), tags (optional), and select a dataset.

<img alt="create-evaluation" />

### Add an Agent Judge

An Agent Judge uses a systematic approach to evaluate your dataset. You can either select an existing agent or create a new one. In the dropdown, you will see all existing judges, both LLM and Agent.

<img alt="judge-dropdown" />

### Agent Judge Types

We offer two built-in agent judges, Default Agent and IF Agent.

<ul>
  <li> **Default Agent** - Used for general purpose evaluations </li>
  <li> **IF Agent** - Excels in systematically extracting instructions from a given prompt and assessing a response's adherence to each instruction. The IF Agent can identify implicit and explicit instructions, structure constraints, content requests, and behavioral requirements (tone, audience, vocabulary) </li>
</ul>

<img alt="select-judge-type" />

### Configure Agent Judge

Each Agent Judge has its own parameters that can be configured

<ul>
  <li> **Model** - The model that the Agent Judge uses to evaluate the dataset. </li>
  <li> **Output Column Name** - The name of the column the results of this judge will show up on your evaluation results. </li>
  <li> **Description** - A description of what the agent judge is evaluating. </li>
  <li> **Output Rules** - A space to provide additional instructions to the agent. </li>
  <li> **Output Type (optional)** - Text, integer, float, or boolean. If not provided, output type is inferred from output rules.</li>
</ul>

<img alt="agent-config" />

### Create Evaluation

Select the rows on the dataset you want to run the evaluation, and click **Create Evaluation**.

<img alt="create-agent-eval" />

## View Evaluation Results

If you navigate back to the **Evaluation** tab, you should be able to see the results of the evaluation.

### Data

The data page will have a column with the results of the Agent Judge

<img alt="agent-eval-results" />

### Overview

The overview page will have a graph with the visual representation of the evaluation result.

<img alt="agent-eval-overview" />


# Agentex Agent Output
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/ui-evaluations/agentex-evaluation

Generate evaluation outputs using external Agentex agents

## Overview

Agentex Agent Output allows you to generate outputs on an evaluation using an Agentex agent. This feature enables you to create an evaluation by selecting a dataset, then configure an Agentex agent to produce a new output column populated by the external agent's responses.

This is useful when you want to:

* Generate responses from your custom Agentex agents for evaluation
* Compare outputs from different agent configurations
* Test agent behavior across a variety of inputs in your dataset

## Create Evaluation

### Navigate to the Evaluate tab and click "Create evaluation".

<img alt="create-evaluation" />

### Add Evaluation Details

Add in Evaluation name, description (optional), tags (optional), and select a dataset.

<img alt="evaluation-details" />

### Add an Agentex Agent Output

Add an output and select "Agentex Output" from the output type options. This will configure an external Agentex agent to generate outputs for each row in your dataset.

<img alt="agentex-output-selection" />

### Configure Agentex Output

Configure the Agentex agent output with the following parameters:

<ul>
  <li> **Output Column Name (optional)** - The name of the header column where the agent's generated outputs will appear in your evaluation results. </li>
  <li> **Agentex Agent** - The Agentex agent you want to use to generate outputs. This dropdown will display all available Agentex agents in your account. Inactive agents will be greyed out. </li>
  <li> **Input Column** - The dataset column containing the input data to send to the agent for each row. </li>
  <li> **Include Traces** - Toggle to true to include a traces column in the output. </li>
</ul>

<Note>
  The input column schema (text or JSON) will be validated against the selected agent's expected schema. If the agent does not have a schema defined, validation will pass without error.
</Note>

<img alt="agentex-output-config" />

### Create Evaluation

Select the rows on the dataset you want to run the evaluation, and click **Create Evaluation**.

<img alt="create-evaluation-final" />

## View Evaluation Results

If you navigate back to the **Evaluation** tab, you should be able to see the results of the evaluation.

### Data

The data page will have a new column containing the outputs generated by your Agentex agent for each row.

<img alt="agentex-output-results-data" />

## Combining with Other Tasks

Agentex outputs are always generated first, before any other evaluation tasks run. This ensures the agent-generated outputs are available for subsequent tasks like LLM Judges or Contributor Evaluations to reference and evaluate. For example, you can:

1. Use an Agentex agent to generate outputs
2. Add an LLM Judge to evaluate the quality of those outputs
3. Add a Contributor Evaluation for human review

This allows you to build comprehensive evaluation pipelines that leverage both automated agents and human feedback.


# LLM as a Judge
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/ui-evaluations/auto-evaluation

Leverage LLM-based tasks to produce evaluation results.

## Create Evaluation

### Navigate to the Evaluate tab and click "Create evaluation".

<img alt="create-evaluation" />

### Add Evaluation Details

Add in Evaluation name, description (optional), tags (optional), and select a dataset.

<img alt="create-evaluation" />

### Add an LLM Judge

An LLM Judge prompts an LLM to answer questions to evaluate your dataset. You can either select an existing judge or you can configure a new one.

<img alt="create-evaluation" />

### Configure LLM Judge

An LLM Judge prompts an LLM to answer questions to evaluate your dataset.

<ul>
  <li> **Alias (Optional)** - The name of the column the results of this judge will show up on your evaluation results. </li>
  <li> **Model** - The model that the LLM Judge uses to evaluate the dataset. </li>
  <li> **System Prompt (Optional)** - A system prompt for an LLM judge is a set of instructions that defines the role and evaluation criteria for a large language model when it's used to assess, score, or make decisions about AI outputs, human responses, or other content. </li>
  <li> **Rubric** - A rubric for an LLM judge is a structured scoring framework that defines specific evaluation criteria, performance levels, and descriptors to ensure consistent and objective assessment of AI outputs or responses. </li>
  <li> **Response Options** - Constraints to what your Model can output. </li>
</ul>

<img alt="create-evaluation" />

### Create Evaluation

Select the rows on the dataset you want to run the evaluation, and click **Create Evaluation**.

<img alt="create-evaluation" />

## View Evaluation Results

If you navigate back to the **Evaluation** tab, you should be able to see the results of the evaluation.

### Data

The data page will have a column with the results of the LLM Judge

<img alt="create-evaluation" />

### Overview

The overview page will have a graph with the visual representation of the evaluation result.

<img alt="create-evaluation" />


# Contributor Evaluation
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/ui-evaluations/contributor-evaluation

Defer evaluation tasks to human contributors.

## Create Evaluation

### Navigate to the Evaluate tab and click "Create evaluation".

<img alt="create-evaluation" />

### Add Evaluation Details

Add in Evaluation name, description (optional), tags (optional), and select a dataset.

<img alt="create-evaluation" />

### Add Contributor Evaluation

Contributor Evaluation lets you set up an evaluation that is labeled by humans.

<img alt="create-evaluation" />

### Configure Contributor Evaluation

Configure the contributor Evaluation.

#### Annotation Layout configuration

The first step is to configure the data pieces that your contributors will see. You can select which columns you want the data to appear, and you can also configure how the data in the columns will show up.
<div> For example, for this configuration: </div>

<img alt="create-evaluation" />

<div> Annotators will see this layout when they label. </div>

<img alt="create-evaluation" />

#### Select Questions

Next, select the questions you want your contributor to answer. You can select a pre-existing question, or you can add a new one in this workflow.

<img alt="create-evaluation" />

<img alt="create-evaluation" />

<div> The questions that you select will show up in the right hand side of the data the contributors will see. </div>

<img alt="create-evaluation" />

### Create Evaluation

Select the rows on the dataset you want to run the evaluation, and click **Create Evaluation**.

<img alt="create-evaluation" />

### Annotate evaluations

When you create an annotation, you can now navigate to that evaluation and see the a tab for **Annotation Tasks**. You or assigned labelers can now start labeling these tasks.

<img alt="create-evaluation" />

## View Evaluation Results

### Completed annotations.

After all the tasks have been annotated, the tasks will show up as completed on the annotation tasks table.

<img alt="create-evaluation" />

### Data

The results will show up as a one column for each question on the data table.

<img alt="create-evaluation" />

### Overview

The results will also show up as a graph with the visual representation of the evaluation result on the overview page.

<img alt="create-evaluation" />


# Voice Evaluations
Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/voice-evaluations

End-to-end guide for creating evaluations with audio files

## Overview

Voice evaluations allow you to assess audio content through human annotation tasks. This guide walks you through the complete process of uploading audio files and creating evaluations with timestamp-based questions, such as identifying when mispronunciations occur.

<Info>Examples in this guide use the [`scale-gp-beta`](https://pypi.org/project/scale-gp-beta/) package which runs exclusively on the V5 API.</Info>

## Step 1: Upload Audio Files

Before creating a voice evaluation, you need to upload your audio files to the platform. There are two methods available:

### Method 1: Direct File Upload

Upload files directly from your local filesystem:

```python theme={null}
from scale_gp_beta import SGPClient

client = SGPClient(account_id=..., api_key=...)

# Upload a single audio file
audio_file_1 = client.files.create(
  file=open("audio_sample_1.mp3", "rb"),
  purpose="evaluation"
)

audio_file_2 = client.files.create(
  file=open("audio_sample_2.mp3", "rb"),
  purpose="evaluation"
)

# Store the file IDs for later use
file_id_1 = audio_file_1.id
file_id_2 = audio_file_2.id
```

See the [Upload File API reference](/reference/v5/files/upload-file) for more details.

### Method 2: Import from Cloud Storage

If your files are already stored in cloud storage (S3, GCS, etc.), you can import them in batch:

```python theme={null}
from scale_gp_beta import SGPClient

client = SGPClient(account_id=..., api_key=...)

# Import files from cloud storage
import_result = client.files.cloud_imports.create(
  files=[
    {
      "container": "my-s3-bucket",
      "filename": "audio_sample_1.mp3",
      "filepath": "evaluations/audio/audio_sample_1.mp3",
      "file_type": "audio/mpeg"
    },
    {
      "container": "my-s3-bucket",
      "filename": "audio_sample_2.mp3",
      "filepath": "evaluations/audio/audio_sample_2.mp3",
      "file_type": "audio/mpeg"
    }
  ]
)

# Extract file IDs from the import results
file_id_1 = import_result.results[0].file.id
file_id_2 = import_result.results[1].file.id
```

See the [Import Files API reference](/reference/v5/files/import-files-from-existing-blob-storage-in-cloud-batch-operation) for more details.

## Step 2: Create a Question

Create a timestamp-based question that annotators will use to mark specific points in the audio:

```python theme={null}
# Create a timestamp question for identifying mispronunciations
question = client.questions.create(
  type="timestamp",
  title="Mispronunciation",
  prompt="When did mispronunciation occur?",
  multi=True  # Set to False if only one timestamp is expected
)

# Store the question ID
question_id = question.id
```

<Note>
  Set `multi=True` if you want annotators to be able to mark multiple timestamps in a single audio file. For voice evaluations, common question types include:

  * **Mispronunciations**: When did the speaker mispronounce words?
  * **Disfluencies**: When did stuttering or filler words occur?
  * **Emotion changes**: When did the speaker's tone shift?
</Note>

## Step 3: Create the Voice Evaluation

Now combine the uploaded files and question into a complete evaluation:

<Warning>
  Evaluations and datasets that contain files must be created via the API or SDK. This functionality is not currently supported in the UI.
</Warning>

```python theme={null}
# Create the voice evaluation
voice_evaluation = client.evaluations.create(
  name="Voice Quality Evaluation - Mispronunciation Detection",
  description="Evaluate audio samples for mispronunciations",
  data=[
    { "audio_description": "Customer service call 1" },
    { "audio_description": "Customer service call 2" },
  ],
  files=[
    { "audio_file": file_id_1 },
    { "audio_file": file_id_2 },
  ],
  tasks=[
    {
      "task_type": "contributor_evaluation.question",
      "alias": "mispronunciation_timestamps",
      "configuration": {
        "question_id": question_id,
        "queue_id": "default",
        "layout": {
          "direction": "column",
          "children": [
            {
              "data": "item.audio_description"
            },
            {
              "data": "item.files.audio_file"
            }
          ]
        },
        "required": False
      }
    }
  ]
)

print(f"Evaluation created with ID: {voice_evaluation.id}")
```

### Understanding the Structure

* **`data`**: Each object represents metadata for one audio sample. The items in `data` have a one-to-one mapping with items in `files`.
* **`files`**: Each object contains file IDs that correspond to the uploaded audio files. The first item in `files` pairs with the first item in `data` to form the first evaluation row.
* **`tasks`**: Defines what annotators will do. Here, we use a `contributor_evaluation.question` task with a timestamp question.

See the [Create Evaluation API reference](/reference/v5/evaluations/create-evaluation) for more details.

## Next Steps

After creating your voice evaluation:

1. **Monitor Progress**: Check the evaluation status to see when annotation tasks are completed
2. **Annotate**: Contributors will annotate and answer the questions and tasks you have created
3. **Review Results**: Access the annotated timestamps through the evaluation items
4. **Analyze Data**: Export results to analyze patterns in voice quality

For more information on working with evaluations, see the [Next Gen Evaluation Overview](/docs/v5/next-gen-evaluation/overview).

## Example Annotation Flow

When contributors access the evaluation queue, they'll see a clean interface displaying the audio player along with any metadata you've included (such as call IDs or descriptions). The audio player allows them to play, pause, and scrub through the audio file. As they listen, they can use the timestamp controls to mark specific moments when events occur (e.g., mispronunciations). If `multi=True` was set, they can add multiple timestamps for each audio file. Once satisfied with their annotations, they submit their responses and move on to the next item in the queue.

<img alt="annotator-view-audio-file" />


# SGP Overview
Source: https://docs.gp.scale.com/docs/v5/overview/index

Overview of SGP

## Overview

The Scale GenAI Platform empowers modern enterprises to rapidly develop, test and deploy Generative AI applications for custom use cases, using their proprietary data assets.

The platform supports two distinct workflows related to Generative AI application development.
The first one is **Agent Building and Execution**, or the act of having an agent that runs.
The second one is **Agent Operations**, or the act of managing the agent, reviewing the data that it outputs, and improving that agent.

<img alt="overview-diagram" />

The ultimate goal of SGP is to enable teams of developers to be able to build and improve multiple agents specializing in multiple goals all on the same platform. SGP helps every Enterprise build and maintain a portfolio of agents.

## Agentex - Agent Building and Execution

### View Agents in the Platform

The agent builder view shows a list of all the agents that are currently being built or monitored in your account.

<img alt="agent-overview" />

Clicking into an agent allows users to interact with an agent, monitor the agent, and view evaluations performed by an agent.
Users can also see the deployment history of the agent.

<img alt="agent-overview" />

### Building Agents with Agentex

SGP's goal for building agents is to provide the most developer friendly framework and environment possible. At the core, building agents is just writing code.
Our goal is for all developers to continue developing in the environment they are most familiar with at the fastest speed possible.

#### Example Agent

This is an example of a medical assistant agent we built, a medical assistant that helps users find doctors at a specific hospital based on their symptoms. The agent diagnoses their symptom and then directs them to a doctor.

<img alt="agent-overview" />

#### Agentex Building Framework and Philosophy.

In today's world, pulling an agent from the internet is easy, most agent frameworks focus on helping users get started on writing an agent. The problem that SGP Agentex aims to solve is to help users quickly get from a locally developed agent into a fully functional production quality agent other people can use.
Backend - Agentex provides a simple CLI that enables users to run the agent locally.
Frontend - Agentex also provides the ability to run a local UI very quickly to allow users to test.
Users can then go to their local machine and run an agent.

<img alt="agent-overview" />

This experience enables a fast development cycle:

* User can test exactly what's going on.
* User can make alterations to the code and change the behavior.
* User can test again.

Agentex has a built a continuous deployment pipeline that takes changes from the agent and pushes them through a series of steps, and automatically deploys your agent for you.

<img alt="agent-overview-ci-cd-pipeline" />

After making changes, users will automaticaly be able to see on the platform the deployment history of the agent.

<img alt="agent-overview" />

### Interacting with Agents through Agentex

A user can interact with an agent by clicking "Open Agent" on the page of the specific Agent.

<img alt="agent-overview" />

This will take users to a **fully deployed, hosted version** of the agent developers and end users can interact with.

<img alt="agent-overview-deployed-agent" />

Since this is a live deployment, all interactions captured from this deployment will show up in our tracing interface (see below as part of Agent Operations) to assist developers with monitoring their application.

## AgentOps- Agent Monitoring and Improvements

### Tracing Agents with AgentOps

All agents built and deployed with Agentex (like in the example above) will have their interactions automatically show up in the tracing library.

<img alt="agent-overview-tracing-demo" />

The AgentOps Tracing Platform also has an SDK that allows users to upload their own traces to get the same experience.

Users can also see specific details on the execution of the agent in the tracing view.

<img alt="agent-overview-tracing-demo-details" />

Users can see the input, output, and all the steps the agent went through for the execution. Engineers can use this data to understand what tools the agent is calling, what steps the agents are going through for reasoning, and how the agent is performing and make adjustements to the agent.

<img alt="agent-overview-tracing-demo-details-output" />

To learn more about tracing, refer to the [tracing](/docs/v5/tracing/overview) section.

### Evaluating Agents with AgentOps

After identifying areas that can be improved from your agents, the AgentOps platform supports developers in running evaluations on their agents.

Users can create evaluation datasets through traces and then run evaluations on the platform.

<img alt="agent-overview-evaluations" />

The platform supports two types of evaluations:

* **LLM-as-a-Judge:** Have an LLM evaluate the dataset
* **Contributor Evaluation:** Have humans annotate the dataset

To learn more about evaluations, refer to the [evaluations](/docs/v5/next-gen-evaluation/overview) section.

### Summary

The combination of all these tools supports the agent development lifecycle.

1. Developing the local agent.
2. Review and deploy the local agent.
3. Test the deployed agent.
4. Monitor the deployed agent through traces.
5. Segment out evaluations from traces.

Enterprises that scale agents need to stop worrying about the overhead of development and focus on the business logic.
SGP is a platform designed to enable that.

<img alt="agent-overview-summary" />


# Built-in Tracing Providers
Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/agentex/built-in-providers

Auto-trace LLM calls and tool invocations using Agentex's built-in provider integrations for OpenAI Agents SDK, LiteLLM, and LangGraph.

Agentex provides three built-in tracing providers that automatically create spans for LLM calls and, in some cases, tool invocations. Each provider integrates with a different framework and has different tracing coverage.

<Warning>
  Built-in providers give you a baseline — auto-traced LLM calls and, in some cases, tool invocations — but truly useful traces require manual instrumentation by the agent implementer: turn-level spans, custom reasoning steps, structured input/output, and proper parent-child linking. Think of built-in providers as the foundation, not the finished product. See [Span Hierarchy & Best Practices](/docs/v5/tracing/integrations/agentex/span-hierarchy) for how to build on top of them.
</Warning>

## Comparison

| Provider                           | LLM Tracing | Tool Tracing | Framework             |
| ---------------------------------- | :---------: | :----------: | --------------------- |
| `TemporalTracingModelProvider`     |     Auto    |      No      | OpenAI Agents SDK     |
| `adk.providers.litellm`            |     Auto    |      No      | LiteLLM               |
| `create_langgraph_tracing_handler` |     Auto    |     Auto     | LangGraph / LangChain |

***

## Providers

<Tabs>
  <Tab title="OpenAI Agents SDK">
    ### How It Works

    For [Temporal-style agents](/docs/v5/agents/agentex/types/agentic-agent) using the OpenAI Agents SDK, LLM calls are auto-traced via `TemporalTracingModelProvider`. For non-Agentex usage of the OpenAI Agents SDK with the SGP tracing SDK directly, see the [OpenAI Agents Integration](/docs/v5/tracing/integrations/openai-agents) page.

    The tracing flow is:

    1. Your workflow sets `_task_id`, `_trace_id`, and `_parent_span_id` on the workflow instance
    2. `ContextWorkflowOutboundInterceptor` propagates these as Temporal headers to activities
    3. `ContextActivityInboundInterceptor` reads the headers and sets `ContextVar`s
    4. `TemporalTracingResponsesModel` wraps the base model and creates a span for each `get_response()` call

    ### Auto-Traced Span Structure

    Each LLM call produces a single span named `model_get_response`:

    ```python theme={null}
    # Span input:
    {
        "model": "gpt-4o",
        "has_system_instructions": True,
        "input_type": "list",
        "tools_count": 3,
        "handoffs_count": 0,
        "has_output_schema": False,
        "model_settings": {
            "temperature": 0.7,
            "max_tokens": 4096,
            "reasoning": {"effort": "high"}
        }
    }

    # Span output:
    {
        "new_items": [
            {"type": "message", "content": [{"type": "text", "text": "..."}]},
            {"type": "function_call", "name": "search", "arguments": "..."}
        ],
        "final_output": "The agent's text response..."
    }
    ```

    <Info>
      The span name is always `model_get_response`. The model name and settings appear in the span's input data.
    </Info>

    ### Streaming Hooks

    `TemporalStreamingHooks` streams tool lifecycle events to the UI in real time:

    ```python theme={null}
    from agentex.lib.core.temporal.plugins.openai_agents import TemporalStreamingHooks

    hooks = TemporalStreamingHooks(task_id=params.task.id)
    result = await Runner.run(agent, input_list, hooks=hooks)
    ```

    This streams:

    * `ToolRequestContent` when a tool starts (name, arguments)
    * `ToolResponseContent` when a tool finishes (name, result)
    * `TextContent` on agent handoffs

    ### Tracing Gap: Tool Calls

    <Warning>
      The OpenAI Agents SDK plugin does **not** create separate spans for tool executions. Tool calls appear inside the model span's `output.new_items` array as `function_call` entries, but they are not their own spans in the trace tree.

      If you need tool-level spans, create manual spans around your tool logic. See [Manual Spans](/docs/v5/tracing/integrations/agentex/span-hierarchy#manual-spans-for-custom-logic).
    </Warning>
  </Tab>

  <Tab title="LiteLLM">
    ### Available Methods

    The `adk.providers.litellm` module exposes four methods, all of which auto-create tracing spans:

    | Method                             | Streaming | Auto-send to UI |
    | ---------------------------------- | :-------: | :-------------: |
    | `chat_completion`                  |     No    |        No       |
    | `chat_completion_stream`           |    Yes    |        No       |
    | `chat_completion_auto_send`        |     No    |       Yes       |
    | `chat_completion_stream_auto_send` |    Yes    |       Yes       |

    The "auto-send" variants automatically create a `TaskMessage` that appears in the Agentex chat UI.

    ### Code Example

    ```python theme={null}
    from agentex.lib import adk

    response = await adk.providers.litellm.chat_completion_stream_auto_send(
        task_id=params.task.id,
        llm_config=LLMConfig(
            model="gpt-4o-mini",
            messages=state.messages,
            temperature=0.7,
        ),
        trace_id=params.task.id,
        parent_span_id=turn_span.id if turn_span else None,
    )
    ```

    ### Auto-Traced Span Structure

    Each method creates a single span named after the method (e.g., `chat_completion_stream_auto_send`):

    * **Input**: The full `LLMConfig`. model name, messages, temperature, and other settings
    * **Output**: The complete `Completion` object or `TaskMessage`
    * For streaming methods, chunks are aggregated into the final response before being set as span output

    ### Tracing Gap: Streaming Chunks

    <Info>
      Individual streaming chunks are not traced as separate spans. The LLM span captures the final aggregated response. If you need chunk-level visibility, you'll need custom instrumentation.
    </Info>
  </Tab>

  <Tab title="LangGraph">
    ### How It Works

    The LangGraph handler creates tracing spans for both LLM calls and tool calls by hooking into LangChain's callback system:

    ```python theme={null}
    from agentex.lib.adk._modules._langgraph_tracing import create_langgraph_tracing_handler

    handler = create_langgraph_tracing_handler(
        trace_id=task_id,
        parent_span_id=turn_span.id,
    )

    result = await graph.ainvoke(
        input_state,
        config={"callbacks": [handler]}
    )
    ```

    ### Span Naming

    | Call Type | Span Name          | `__span_type__` |
    | --------- | ------------------ | --------------- |
    | LLM call  | `llm:<model_name>` | `COMPLETION`    |
    | Tool call | `tool:<tool_name>` | `CUSTOM`        |

    ### What Gets Traced

    * **LLM calls**: `on_chat_model_start()` → `on_llm_end()` captures messages, model settings, output content, and tool calls
    * **Tool calls**: `on_tool_start()` → `on_tool_end()` captures tool name, input, and output

    <Info>
      This is the only built-in provider that creates separate spans for both LLM calls and tool calls automatically.
    </Info>

    ### Tracing Gap: Tool I/O Detail

    <Warning>
      The LangGraph handler captures tool name and string-serialized input/output only. Structured tool arguments and rich return types are coerced to strings. If you need full structured tool I/O, supplement with manual spans.
    </Warning>
  </Tab>
</Tabs>

***

## Choosing a Provider

<AccordionGroup>
  <Accordion title="I'm using OpenAI Agents SDK on Temporal">
    Use `TemporalTracingModelProvider` for automatic LLM call tracing. This is the default when you use the OpenAI Agents SDK plugin with Temporal workflows. No additional setup is needed for model call spans.

    For tool-level tracing, create manual spans around your tool functions using `adk.tracing.span()`. See [Manual Spans](/docs/v5/tracing/integrations/agentex/span-hierarchy#manual-spans-for-custom-logic) for examples.
  </Accordion>

  <Accordion title="I'm using LiteLLM directly">
    Use the `adk.providers.litellm` methods. All four methods automatically create tracing spans when you pass `trace_id` and `parent_span_id`.

    The `auto_send` variants are convenient for chat-style agents since they handle both tracing and UI message creation in one call.
  </Accordion>

  <Accordion title="I'm using LangGraph / LangChain">
    Use `create_langgraph_tracing_handler` registered as a callback. This gives you the most complete auto-tracing. Both LLM and tool calls get their own spans without any manual instrumentation.

    You still need to create turn-level parent spans manually to group operations by conversation turn.
  </Accordion>
</AccordionGroup>


# Multi-Agent Tracing
Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/agentex/multi-agent-tracing

Propagate trace context across orchestrator and sub-agents to build unified, cross-agent trace trees.

This page builds on the concepts in [Agentex Tracing Overview](./overview) and [Span Hierarchy & Best Practices](./span-hierarchy). Make sure you're familiar with creating spans and setting `parent_id` before diving into multi-agent propagation.

## The Problem

In a [multi-agent system](/docs/v5/agents/agentex/overview), each agent creates its own trace by default (`trace_id = task_id`, and each agent has its own task). The result is fragmented observability: the orchestrator's trace shows only its own spans, and each sub-agent's work lives in a separate, disconnected trace.

```mermaid theme={null}
graph TD
    subgraph Trace1["Trace: orchestrator_task_123"]
        _s1[ ]:::spacer
        _s1 ~~~ O1
        O1["turn:1"]
        O1 --> OL1["llm:gpt-4o"]
        O1 --> OD["dispatch sub-agents"]
    end

    subgraph Trace2["Trace: research_task_456"]
        _s2[ ]:::spacer
        _s2 ~~~ R1
        R1["turn:1"]
        R1 --> RL1["llm:o3"]
        R1 --> RT1["tool:web_search"]
    end

    subgraph Trace3["Trace: analysis_task_789"]
        _s3[ ]:::spacer
        _s3 ~~~ A1
        A1["turn:1"]
        A1 --> AL1["llm:o3"]
    end

    classDef spacer fill:none,stroke:none,color:transparent
    style Trace1 stroke:#e74c3c
    style Trace2 stroke:#3498db
    style Trace3 stroke:#2ecc71
```

You end up with three separate traces instead of one unified view of the entire workflow.

***

## The Solution: Trace ID Propagation

Pass the orchestrator's `trace_id` to every sub-agent so all spans land in a single trace tree. You also thread `parent_span_id` through each dispatch so the hierarchy forms correctly.

```mermaid theme={null}
graph TD
    subgraph Unified["Trace: orchestrator_task_123"]
        _s0[ ]:::spacer
        _s0 ~~~ O1
        O1["turn:1 · orchestrator"]
        O1 --> Sub1["subtask:research"]
        Sub1 --> RL1["llm:o3"]
        Sub1 --> RT1["tool:web_search"]
        O1 --> Sub2["subtask:analysis"]
        Sub2 --> AL1["llm:o3"]
        O1 --> OL1["llm:gpt-4o · synthesis"]
    end
    classDef spacer fill:none,stroke:none,color:transparent
```

Two things make this work:

1. **Same `trace_id` everywhere**: the orchestrator's `task_id` is used as the `trace_id` for all sub-agent spans
2. **Threaded `parent_span_id`**: each dispatch passes the current span's ID so sub-agent work nests correctly

***

## Context Propagation in Temporal

For Temporal-based agents, trace context flows automatically through the interceptor system:

```mermaid theme={null}
graph LR
    subgraph WF["Workflow Side"]
        WI["Workflow Instance"]
        CO["OutboundInterceptor"]
    end

    subgraph Headers["Temporal Transport"]
        H["Activity Headers"]
    end

    subgraph ACT["Activity Side"]
        CI["InboundInterceptor"]
        CV["ContextVars"]
        TM["TracingModel"]
    end

    WI --> CO
    CO --> H
    H --> CI
    CI --> CV
    CV --> TM
    TM --> S["Auto-traced span"]
```

The workflow instance exposes `self._task_id`, `self._trace_id`, and `self._parent_span_id`. The outbound interceptor copies these into Temporal activity headers (`context-task-id`, `context-trace-id`, `context-parent-span-id`). On the activity side, the inbound interceptor reads those headers into `ContextVar`s (`streaming_task_id`, `streaming_trace_id`, `streaming_parent_span_id`) that the tracing model picks up automatically.

### Setting Up Context in Your Workflow

```python theme={null}
from agentex.lib import adk
from temporalio import workflow

@workflow.defn
class MyAgentWorkflow:
    @workflow.run
    async def run(self, params):
        # These are read by ContextWorkflowOutboundInterceptor
        self._task_id = params.task_id
        self._trace_id = params.task_id  # trace_id = task_id

        # Create a turn-level span
        async with adk.tracing.span(
            trace_id=params.task_id, name="turn:1"
        ) as span:
            self._parent_span_id = span.span_id  # Auto-traced LLM calls nest under this

            # Any activity using TemporalTracingModelProvider
            # will automatically create child spans under this parent
            result = await workflow.execute_activity(
                invoke_model_activity, ...
            )
```

<Info>
  `ContextInterceptor` reads `self._task_id`, `self._trace_id`, and `self._parent_span_id` from the workflow instance. Setting these attributes is all you need. The interceptor handles header propagation automatically.
</Info>

***

## Multi-Agent Code Example

### Orchestrator Agent

The orchestrator creates spans for each sub-agent dispatch and passes its `trace_id` and `parent_span_id` through:

```python theme={null}
from agentex.lib import adk

async with adk.tracing.span(
    trace_id=orchestrator_task_id,
    name="turn:1",
    input={"sub_agents": ["research", "analysis", "synthesis"]},
) as orchestration_span:

    for sub_agent_name in sub_agents:
        async with adk.tracing.span(
            trace_id=orchestrator_task_id,
            parent_id=orchestration_span.id,
            name=f"subtask:{sub_agent_name}",
            input={"agent": sub_agent_name},
        ) as dispatch_span:

            # Pass trace context to the sub-agent
            await adk.tasks.create(
                agent_name=sub_agent_name,
                input=task_input,
                trace_id=orchestrator_task_id,
                parent_span_id=dispatch_span.id,
            )
```

### Sub-Agent (Child)

The sub-agent uses the **incoming `trace_id`** from the orchestrator instead of its own `task_id`:

```python theme={null}
@acp.on_task_event_send
async def handle_event_send(params: SendEventParams):
    # Use the propagated trace_id, NOT params.task.id
    trace_id = params.trace_id or params.task.id  # Fallback for standalone use

    async with adk.tracing.span(
        trace_id=trace_id,
        parent_id=params.parent_span_id,
        name=f"turn:{state.turn_number}",
        input=state.model_dump(),
    ) as span:
        response = await adk.providers.litellm.chat_completion_stream_auto_send(
            task_id=params.task.id,
            llm_config=llm_config,
            trace_id=trace_id,
            parent_span_id=span.id if span else None,
        )

        # Message to both parent and child task for visibility
        await adk.messages.create(
            task_id=params.task.id,
            trace_id=trace_id,
            parent_span_id=span.id if span else None,
            content=TextContent(author="agent", content=response_text),
        )
```

***

## Resulting Unified Trace Tree

With trace propagation, the entire multi-agent workflow appears under a single trace:

```mermaid theme={null}
graph TD
    T["Trace: orchestrator_task_123"]

    T --> T1["turn:1 · orchestrator"]
    T1 --> SR["subtask:research"]
    SR --> SRT1["turn:1 · research agent"]
    SRT1 --> SRL1["llm:gpt-4o"]
    SRT1 --> SRTool["tool:web_search"]
    SRT1 --> SRL2["llm:gpt-4o"]

    T1 --> SA["subtask:analysis"]
    SA --> SAT1["turn:1 · analysis agent"]
    SAT1 --> SAL1["llm:o3"]
    SAT1 --> SATool["tool:code_interpreter"]

    T1 --> SS["subtask:synthesis"]
    SS --> SST1["turn:1 · synthesis agent"]
    SST1 --> SSL1["llm:gpt-4o"]
```

***

## Verification

<Steps>
  <Step title="Run your orchestrator agent">
    Start the orchestrator: `agentex agents run --manifest manifest.yaml`
  </Step>

  <Step title="Trigger sub-agent dispatch">
    Send a task or message that causes the orchestrator to dispatch work to sub-agents.
  </Step>

  <Step title="Check spans via Agentex API">
    Query for all spans in the trace:

    ```bash theme={null}
    curl "https://your-agentex-url/spans?trace_id={orchestrator_task_id}"
    ```

    Verify that sub-agent spans share the orchestrator's `trace_id`.
  </Step>

  <Step title="Verify hierarchy">
    Check that each span's `parent_id` references an existing span's `id`. Sub-agent spans should nest under the orchestrator's dispatch spans.
  </Step>

  <Step title="Confirm in the SGP UI">
    Open the [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view), filter by the orchestrator's `task_id`, and confirm the unified tree shows orchestrator and sub-agent spans together.
  </Step>
</Steps>

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="Sub-agent spans appear in a separate trace">
    The sub-agent is using its own `task_id` as `trace_id` instead of the orchestrator's. Check that:

    * The orchestrator passes its `task_id` as `trace_id` when creating the sub-agent task
    * The sub-agent reads and uses the propagated `trace_id` (e.g., `params.trace_id or params.task.id`)
    * All `adk.tracing.span()` calls in the sub-agent use the propagated `trace_id`
  </Accordion>

  <Accordion title="Spans are flat instead of nested">
    Parent-child relationships are missing. Check that:

    * `parent_span_id` is passed through the dispatch (not just `trace_id`)
    * Sub-agent spans set `parent_id` to the received `parent_span_id`
    * Inside the sub-agent, child operations (LLM calls, tools) receive `parent_span_id=span.id` from the enclosing turn span
  </Accordion>

  <Accordion title="Auto-traced LLM spans are orphaned (no parent)">
    The workflow's `self._parent_span_id` was not set before executing the activity. Ensure:

    * `self._parent_span_id = span.id` is set **before** calling `workflow.execute_activity()`
    * The turn span is created and its `id` is captured before any activity execution
    * For Temporal workflows, the `ContextInterceptor` is registered in the worker's interceptor list
  </Accordion>
</AccordionGroup>


# Agentex Tracing Overview
Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/agentex/overview

Instrument Agentex agents with tracing to capture every turn, LLM call, and tool invocation as structured spans in the SGP Traces UI.

## What is Agentex Tracing?

Modern AI agents execute complex, multi-step workflows: reasoning, calling tools, making LLM requests, and coordinating with other agents. Without tracing, these workflows are opaque. When something goes wrong or runs slowly, you're left guessing.

[Agentex](/docs/v5/agents/agentex/overview) Tracing gives you deep visibility into your Temporal-style agents by capturing structured spans for every meaningful operation. You get:

* **One trace per task**: `trace_id` maps directly to your Agentex `task_id`
* **A structured span hierarchy**: turns, LLM calls, tool invocations, and custom logic all nest via `parent_id`
* **Tool and model I/O capture**: see exactly what went into and came out of each operation
* **Profiler-ready data**: consistent span naming enables aggregated performance analysis across all your traces

<Info>
  This guide covers tracing specific to Agentex agents. For general tracing concepts (spans, traces, the Traces UI), see [Introduction to Tracing](/docs/v5/tracing/overview).
</Info>

***

## Architecture

```mermaid theme={null}
graph LR
    subgraph Agent["Agent Code"]
        MS["adk.tracing.span()"]
        AL["TemporalTracingModelProvider"]
        AT["LangGraph Handler"]
    end

    subgraph Agentex["Agentex Backend"]
        API["Span API"]
        DB[("PostgreSQL")]
    end

    subgraph SGP["SGP Platform"]
        PROC["SGP Tracing Processor"]
        SGPDB[("SGP Backend")]
        UI["Traces UI"]
    end

    MS --> API
    AL --> API
    AT --> API
    API --> DB
    API --> PROC
    PROC --> SGPDB
    SGPDB --> UI
```

### Key Identity Relationships

| Concept | ID Field              | Typical Value                       |
| ------- | --------------------- | ----------------------------------- |
| Trace   | `trace_id`            | = `task_id` (the Agentex task UUID) |
| Span    | `id`                  | UUID4, auto-generated               |
| Parent  | `parent_id`           | `span.id` of the enclosing span     |
| Agent   | `data.__agent_name__` | Agent name from manifest            |
| Source  | `data.__source__`     | `"agentex"` (set by processor)      |

***

## Core Concepts

### trace\_id = task\_id

Every Agentex task has a unique ID. This same ID becomes the `trace_id` for all spans associated with that task. This 1:1 mapping means you can look up any task's full execution history by searching for its `task_id` in the Traces UI.

### Span Hierarchy

Spans form a tree via `parent_id` relationships. A turn-level span is the parent of the LLM calls and tool invocations that happen during that turn. This hierarchy is what the Traces UI renders as an expandable tree and Gantt chart.

### Span Data Model

```python theme={null}
class Span(BaseModel):
    id: str                       # UUID4 - unique span identifier
    name: str                     # Descriptive operation name
    trace_id: str                 # Groups spans into a trace (= task_id)
    start_time: datetime          # UTC timestamp when span began
    end_time: Optional[datetime]  # UTC timestamp when span ended
    parent_id: Optional[str]      # Parent span ID for hierarchy
    input: Optional[Dict]         # Operation input data
    output: Optional[Dict]        # Operation output data
    data: Optional[Dict]          # Metadata (__span_type__, __source__, etc.)
```

***

## Quick Start

<Steps>
  <Step title="Initialize Tracing in Your acp.py">
    Add the tracing initialization at the top level of your `acp.py` (or `workflow.py` for Temporal agents). This runs once when the agent process starts and configures both the Agentex and SGP tracing processors so spans appear in the SGP Traces UI.

    ```python theme={null}
    import os
    import logging

    from agentex.lib.core.tracing.processors.tracing_processor_manager import (
        add_tracing_processor_config,
    )
    from agentex.lib.types.tracing import SGPTracingProcessorConfig

    logger = logging.getLogger(__name__)

    # --- Tracing initialization (top-level, runs once at process start) ---

    SGP_API_KEY = os.getenv("SGP_API_KEY")
    SGP_ACCOUNT_ID = os.getenv("SGP_ACCOUNT_ID")
    SGP_BASE_URL = os.getenv("SGP_CLIENT_BASE_URL")

    if SGP_API_KEY and SGP_ACCOUNT_ID:
        add_tracing_processor_config(
            SGPTracingProcessorConfig(
                type="sgp",
                sgp_api_key=SGP_API_KEY,         # Your SGP API key
                sgp_account_id=SGP_ACCOUNT_ID,   # Your SGP account ID
                sgp_base_url=SGP_BASE_URL,       # e.g. "https://api.scale.com"
            )
        )

        # Optional: also initialize the SGP tracing SDK directly
        # (needed if you use scale_gp_beta.lib.tracing.create_span() anywhere)
        try:
            import scale_gp_beta.lib.tracing as tracing
            from scale_gp_beta import SGPClient

            client = SGPClient(
                api_key=SGP_API_KEY,
                account_id=SGP_ACCOUNT_ID,
                base_url=SGP_BASE_URL,
            )
            tracing.init(client=client)
            logger.info("SGP tracing initialized")
        except ImportError:
            logger.warning("scale_gp_beta not installed, skipping SGP SDK tracing init")
    else:
        logger.warning("SGP tracing disabled (missing SGP_API_KEY or SGP_ACCOUNT_ID)")
    ```

    The three required environment variables:

    | Env Var               | Description                                                 | Example                 |
    | --------------------- | ----------------------------------------------------------- | ----------------------- |
    | `SGP_API_KEY`         | Your SGP API key for authentication                         | `scl_...`               |
    | `SGP_ACCOUNT_ID`      | Your SGP account identifier                                 | `acc_...`               |
    | `SGP_CLIENT_BASE_URL` | The SGP API base URL. **Must be set**, there is no default. | `https://api.scale.com` |

    <Warning>
      `SGP_CLIENT_BASE_URL` has no default value. If it is not set, the tracing processor will not know where to send spans. Make sure this is configured in your agent's environment or secrets.
    </Warning>
  </Step>

  <Step title="Create a Turn Span">
    Wrap each conversation turn in a manual span. This groups all LLM calls and tool invocations for that turn.

    ```python theme={null}
    from agentex.lib import adk

    @acp.on_task_event_send
    async def handle_event_send(params: SendEventParams):
        state = ...  # get state
        state.turn_number += 1

        async with adk.tracing.span(
            trace_id=params.task.id,
            name=f"turn:{state.turn_number}",
            input=state.model_dump(),
        ) as span:
            # Everything inside this block nests under this turn span
            pass
    ```
  </Step>

  <Step title="Make a Traced LLM Call">
    Pass `trace_id` and `parent_span_id` to your LLM call. The built-in providers auto-create a child span.

    ```python theme={null}
        async with adk.tracing.span(
            trace_id=params.task.id,
            name=f"turn:{state.turn_number}",
            input=state.model_dump(),
        ) as span:
            response = await adk.providers.litellm.chat_completion_stream_auto_send(
                task_id=params.task.id,
                llm_config=LLMConfig(model="gpt-4o-mini", messages=state.messages),
                trace_id=params.task.id,
                parent_span_id=span.id if span else None,
            )

            if span:
                span.output = state.model_dump()
    ```
  </Step>

  <Step title="View in the Traces UI">
    Open the SGP Traces page, filter by your agent name, and click on the trace. You should see your turn span with the LLM call nested beneath it.

    <Frame>
      <img alt="Trace detail view showing a turn span with a child LLM span" />
    </Frame>
  </Step>
</Steps>

<Info>
  For a complete reference on the SGP Tracing SDK, see [Creating Traces and Spans](/docs/v5/tracing/sdk/creating-traces-spans).
</Info>

<Tip>
  The [Traces table](/docs/v5/tracing/tracing-ui/traces-table#controlling-the-content-preview) preview column extracts text from specific preferred keys in your span `input` and `output` (e.g., `title`, `query`, `content`). If the preview shows an unhelpful value, add a `title` key with the value you want displayed.
</Tip>

***

## The Two Tracing Systems

### Agentex Native Tracing (`adk.tracing`)

This is the primary system for Temporal agents. Spans are sent to the Agentex backend API.

1. `adk.tracing.span()` creates a span with a UUID4 id
2. On start: calls `POST /spans` to create the span in the Agentex database
3. On end: calls `PATCH /spans/{id}` to update with output and end\_time
4. If inside a Temporal workflow: routes through Temporal activities instead of direct HTTP

### SGP Tracing (`scale_gp_beta.lib.tracing`)

This is the SGP platform's native tracing SDK. You can use it directly or as a dual-write processor alongside Agentex tracing.

1. The SGP processor intercepts Agentex span events
2. Converts to SGP span format via `scale_gp_beta.lib.tracing.create_span()`
3. Adds metadata: `__source__: "agentex"`, `__agent_name__`, `__agent_id__`, `__acp_type__`
4. Flushes to the SGP backend via `upsert_batch()`

### Which Should You Use?

* **For Temporal agents**: Use `adk.tracing.span()`, which handles Temporal activity routing automatically
* **For standalone scripts**: Use `scale_gp_beta.lib.tracing.create_span()` directly
* **For dual-write**: Configure both processors (Agentex processor is the default; add the SGP processor for SGP UI visibility)

<Warning>
  If you only configure the Agentex processor (the default), spans will NOT appear in the SGP Traces UI. Add the SGP processor for dual-write visibility. See the Quick Start above.
</Warning>

***

## Next Steps

* Want to understand what's auto-traced for you? See the [Built-in Tracing Providers](./built-in-providers) guide for OpenAI Agents SDK, LiteLLM, and LangGraph coverage and gaps.
* For naming conventions, input/output patterns, and profiler-friendly trace structure, see [Span Hierarchy & Best Practices](./span-hierarchy).
* Building a multi-agent system? See [Multi-Agent Tracing](./multi-agent-tracing) to propagate trace context across orchestrator and sub-agents.
* For general tracing concepts and the SGP Tracing SDK, see [Introduction to Tracing](/docs/v5/tracing/overview) and [Creating Traces and Spans](/docs/v5/tracing/sdk/creating-traces-spans).
* To understand the Traces UI, see the [Traces Table](/docs/v5/tracing/tracing-ui/traces-table), [Profiler](/docs/v5/tracing/tracing-ui/profiler), and [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view) pages.
* New to Agentex? Start with the [Agentex Overview](/docs/v5/agents/agentex/overview) and [Agentex Agent Types](/docs/v5/agents/agentex/types/agentic-agent).


# Span Hierarchy & Best Practices
Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/agentex/span-hierarchy

Naming conventions, input/output patterns, and the ideal trace tree structure for Agentex agents.

Well-structured traces make the difference between useful observability and noisy data. This page covers how to name spans consistently, structure your trace tree, and populate span input/output so the [Profiler](/docs/v5/tracing/tracing-ui/profiler) and [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view) give you actionable insights.

***

## Naming Conventions

Use consistent, parseable span names. The [SGP Profiler](/docs/v5/tracing/tracing-ui/profiler) aggregates performance metrics by span name. Inconsistent naming splits your data into separate rows and makes bottleneck analysis unreliable.

| Span Type     | Name Pattern        | Example                                      |
| ------------- | ------------------- | -------------------------------------------- |
| Turn          | `turn:<N>`          | `turn:1`, `turn:2`                           |
| LLM Call      | `llm:<model>`       | `llm:gpt-4o`, `llm:claude-sonnet-4-20250514` |
| Tool Call     | `tool:<name>`       | `tool:web_search`, `tool:code_interpreter`   |
| Reasoning     | `reasoning:<step>`  | `reasoning:plan`, `reasoning:reflect`        |
| State Update  | `state:<operation>` | `state:update`, `state:create`               |
| Sub-task      | `subtask:<name>`    | `subtask:market_analysis`                    |
| Research Pass | `research_pass_<N>` | `research_pass_1`, `research_pass_2`         |

<Warning>
  The SGP Profiler aggregates metrics by span name. Mixing formats like `Turn 1` and `turn:1` will split your data into separate rows, making performance analysis unreliable. Pick a convention and stick with it.
</Warning>

***

## The Ideal Trace Tree

A well-structured multi-turn agent trace looks like this:

```mermaid theme={null}
graph TD
    T["Trace: task_abc123"]

    T --> T1["turn:1"]
    T1 --> T1L1["llm:gpt-4o ✦"]
    T1 --> T1Tool["tool:web_search ✦"]
    T1 --> T1L2["llm:gpt-4o ✦"]
    T1 --> T1S["state:update"]

    T --> T2["turn:2"]
    T2 --> T2R["reasoning:plan"]
    T2 --> T2Sub1["subtask:market_analysis"]
    T2Sub1 --> T2RP1["research_pass:1"]
    T2RP1 --> T2RP1L["llm:o3"]
    T2RP1 --> T2RP1T["tool:code_interpreter"]
    T2Sub1 --> T2RP2["research_pass:2"]
    T2RP2 --> T2RP2L["llm:o3"]
    T2 --> T2Sub2["subtask:competitor_review"]
    T2Sub2 --> T2CRP["research_pass:1"]
    T2CRP --> T2CRPL["llm:o3"]
    T2CRP --> T2CRPT["tool:web_search"]
    T2 --> T2S["state:update"]
```

*✦ = auto-traced by built-in providers*

Key principles:

* **Turn spans are the primary grouping unit**: every operation in a conversation turn nests under its turn span
* **Auto-traced spans nest automatically** when you pass `parent_span_id` to built-in providers
* **Sub-tasks and research passes** create intermediate grouping for complex multi-step operations
* **State updates** capture what changed at the end of a turn

***

## Span Input/Output Best Practices

Input and output appear in two places in the SGP UI: the [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view) span inspector panel and the [Profiler](/docs/v5/tracing/tracing-ui/profiler) span inspector. Well-structured I/O makes debugging dramatically faster.

<Tip>
  The [Traces table](/docs/v5/tracing/tracing-ui/traces-table#controlling-the-content-preview) preview column extracts text from specific preferred keys in your span `input` and `output` (e.g., `title`, `query`, `content`). If the preview shows an unhelpful value, add a `title` key with the value you want displayed.
</Tip>

<Frame>
  <img alt="Agentex profiler view showing aggregated span metrics" />
</Frame>

<AccordionGroup>
  <Accordion title="Turn spans">
    * **Input**: The user message or event that triggered this turn, plus current state
    * **Output**: The updated state after processing

    ```python theme={null}
    async with adk.tracing.span(
        trace_id=task_id,
        name=f"turn:{turn_number}",
        input={"message": user_message, "state": state.model_dump()},
    ) as span:
        # ... process turn ...
        if span:
            span.output = state.model_dump()
    ```

    Comparing input and output on a turn span lets you quickly see what changed during that turn.
  </Accordion>

  <Accordion title="LLM spans (auto-traced)">
    These are populated automatically by the [built-in providers](/docs/v5/tracing/integrations/agentex/built-in-providers):

    * **Input**: Model name, tools count, model settings, whether system instructions exist
    * **Output**: `new_items` (serialized response items), `final_output` (extracted text)
  </Accordion>

  <Accordion title="Tool spans">
    * **Input**: The tool arguments as a dict
    * **Output**: The tool result (string or structured data)

    ```python theme={null}
    async with adk.tracing.span(
        trace_id=task_id,
        name=f"tool:{tool_name}",
        parent_id=turn_span_id,
        input={"query": search_query, "max_results": 10},
    ) as span:
        result = await execute_tool(tool_name, search_query)
        if span:
            span.output = {"results": result, "count": len(result)}
    ```
  </Accordion>

  <Accordion title="Reasoning spans">
    * **Input**: The context or question being reasoned about
    * **Output**: The reasoning result, confidence score, and planned next steps

    ```python theme={null}
    async with adk.tracing.span(
        trace_id=task_id,
        name="reasoning:plan",
        parent_id=turn_span_id,
        input={"context": context, "question": user_question},
    ) as span:
        plan = await generate_plan(context, user_question)
        if span:
            span.output = {"plan": plan, "confidence": 0.95, "steps": 3}
    ```
  </Accordion>

  <Accordion title="State update spans">
    * **Input**: Previous state (or the delta being applied)
    * **Output**: The new state

    These are most useful when your state changes are complex and you want to track exactly what was modified.
  </Accordion>
</AccordionGroup>

***

## Manual Spans for Custom Logic

When [built-in providers](/docs/v5/tracing/integrations/agentex/built-in-providers) don't cover your use case (agent reasoning, data processing, custom tool execution), create manual spans.

### Using Agentex ADK (Temporal-aware)

Use `adk.tracing.span()` inside Temporal workflows and activities. It handles Temporal activity routing automatically.

```python theme={null}
from agentex.lib import adk

async with adk.tracing.span(
    trace_id=task_id,
    name="reasoning:reflect",
    parent_id=turn_span_id,
    input={"context": "...", "question": "..."},
    data={"__span_type__": "CUSTOM"},
) as span:
    result = await my_reasoning_function(...)
    if span:
        span.output = {"reasoning": result, "confidence": 0.95}
```

<Info>
  Always check `if span:` before setting attributes. The span is `None` when `trace_id` is falsy, which prevents errors when tracing is disabled.
</Info>

### Using SGP SDK Directly (non-Temporal)

For standalone scripts or non-Temporal agents, use the SGP tracing SDK directly:

```python theme={null}
import scale_gp_beta.lib.tracing as tracing

with tracing.create_span(
    name="research_task_activity_market_analysis",
    trace_id=task_id,
    parent_id=parent_span_id,
    input={"system_prompt": "...", "user_input": "...", "num_passes": 2},
) as span:
    result = await run_research(...)
    span.output = {"output_length": len(result), "success": True}
```

***

## Span Type Mapping to SGP

Set `__span_type__` in span `data` to control how the SGP UI categorizes the span:

| `__span_type__` | SGP `operation_type` | Use For                     |
| --------------- | -------------------- | --------------------------- |
| `STANDALONE`    | STANDALONE           | Default, generic operations |
| `COMPLETION`    | COMPLETION           | LLM / model calls           |
| `CUSTOM`        | CUSTOM               | Tool calls, custom logic    |

***

## Real-World Example: Multi-Level Research Agent

This example from a due diligence research agent shows effective multi-level span nesting:

```python theme={null}
@activity.defn
async def research_task_activity(
    task_info: TaskInfo,
    model: str,
    task_id: str,
    parent_span_id: str,
    num_passes: int = 2,
) -> ResearchOutput:
    span_name = f"research_task_activity_{task_info.section_title.lower().replace(' ', '_')}"

    # Level 1: Activity-level span
    with tracing.create_span(
        name=span_name,
        trace_id=task_id,
        parent_id=parent_span_id,
        input={
            "system_prompt": system_prompt,
            "user_input": user_input,
            "num_passes": num_passes,
        },
    ) as span:

        # Level 2: Per-pass spans
        for pass_num in range(num_passes):
            with tracing.create_span(
                name=f"research_pass_{pass_num + 1}",
                trace_id=task_id,
                parent_id=span.span_id,
                input={
                    "pass_num": pass_num + 1,
                    "is_refinement": pass_num > 0,
                    "message_count": len(messages),
                },
            ) as pass_span:

                # Level 3: Auto-traced LLM/tool spans via stream processor
                processor = AgentsStreamProcessor(
                    task_id,
                    pass_span.span_id,
                    f"{task_info.section_title} (pass {pass_num + 1})",
                )

                result = Runner.run_streamed(agent, messages)
                async for event in result.stream_events():
                    await processor.process_event(event)

                pass_span.output = {
                    "output_length": len(pass_output),
                    "success": True,
                }
```

This produces a clean, nested trace:

```
research_task_activity_market_analysis
├── research_pass_1
│   ├── llm:o3 (auto-traced)
│   ├── tool:web_search (auto-traced)
│   └── llm:o3 (auto-traced)
└── research_pass_2
    ├── llm:o3 (auto-traced)
    └── llm:o3 (auto-traced)
```

***

## Checklist

### Must-Do

* Set `trace_id = task_id` consistently across all spans
* Create turn-level spans wrapping each conversation turn
* Pass `parent_span_id` to all ADK methods and child operations
* Set `span.output` before the span closes (inside the context manager)
* Use descriptive, parseable span names (`turn:1`, `tool:search`, `llm:gpt-4o`)

### Should-Do

* Include relevant context in span `input`, not just raw data but structured metadata
* Set `__span_type__` in `data` for proper SGP categorization
* Handle errors gracefully by setting error output before the span closes
* Use nested spans for multi-step operations (research passes, sub-tasks)

### Avoid

* Don't create flat traces where everything sits at the root level. Use `parent_id` hierarchy
* Don't put massive payloads in input/output. Summarize or truncate large data
* Don't forget to check `if span:` before setting attributes (span is `None` when `trace_id` is falsy)
* Don't create spans without `trace_id`. They'll be orphaned and invisible in the UI


# OpenAI Agents Integration
Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/openai-agents

Automatically trace openai-agents workflows with the SGP processor.

The SGP Tracing SDK provides an integration for the `openai-agents` library via the `OpenAITracingSGPProcessor`. This processor automatically captures detailed traces from your AI agent runs, translating events from the agent's lifecycle into SGP traces and spans.

This provides deep visibility into your agent's execution, including tool calls and model invocations, without requiring manual instrumentation.

<Warning>
  This integration requires the [`openai-agents`](https://pypi.org/project/openai-agents/) package. The processor will raise an `ImportError` if the package is not installed.
</Warning>

***

## How it Works

The `OpenAITracingSGPProcessor` hooks into the `openai-agents` tracing system. You register it once, and it listens for lifecycle events, handling the translation to SGP tracing objects automatically.

* **Trace Management**:
  * When an agent run starts, `on_trace_start` creates a corresponding SGP trace with the type `AGENT_WORKFLOW`. The root span of this trace is stored internally.
  * When the run ends, `on_trace_end` finds the stored root span and marks it as complete.

* **Span Management**:
  * As the agent executes, `on_span_start` fires for each new operation. It creates a new SGP span, linking it to the correct parent (either another span or the root trace).
  * It uses the `openai_span_type_map` helper to convert the agent's span type (e.g., "function", "generation") into an SGP `SpanType` (e.g., "CODE\_EXECUTION", "COMPLETION").
  * When the operation finishes, `on_span_end` populates the span's `input`, `output`, and `metadata` using helper functions that parse the data from the agent. If the agent reports an error, it's recorded on the span.

***

## Usage Example

To use the processor, you must initialize SGP tracing and then register an instance of `OpenAITracingSGPProcessor` with the `openai-agents` library.

```python theme={null}
import os
import asyncio
import requests
from bs4 import BeautifulSoup
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
from agents import Agent, Runner, function_tool, set_trace_processors
from datetime import date

import scale_gp_beta.lib.tracing as tracing
from scale_gp_beta import SGPClient
from scale_gp_beta.lib.tracing.integrations import OpenAITracingSGPProcessor


@function_tool
def google_search(query: str) -> list[str]:
    scoped_span = tracing.current_span()
    try:
        api_key, cse_id = os.environ["GOOGLE_API_KEY"], os.environ["GOOGLE_CSE_ID"]
        with tracing.create_span("build_custom_search"):
            service = build("customsearch", "v1", developerKey=api_key)

        with tracing.create_span("execute_custom_search", input={"query": query}) as span:
            res = service.cse().list(q=query, cx=cse_id, num=5).execute()
            span.output = {"response": res}
        urls = [item['link'] for item in res.get('items', [])]

        return urls
    except HttpError as e:
        error_message = f"An error occurred with Google Search: {e}"
        scoped_span.set_error(error_message) if scoped_span else None
        return [error_message]
    except Exception as e:
        error_message = f"An unexpected error occurred: {e}"
        scoped_span.set_error(error_message) if scoped_span else None
        return [error_message]


@function_tool
def get_article_content(url: str) -> str:
    scoped_span = tracing.current_span()
    try:
        headers = {
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36'
        }
        with tracing.create_span("GET_article_content", input={"url": url}, span_type="FILE_RETRIEVAL") as span:
            response = requests.get(url, headers=headers, timeout=10)
            response.raise_for_status()
            span.output = {"text": response.text, "code": response.status_code}

        with tracing.create_span("parse_article_content", span_type="DATA_MANIPULATION", input={"text": response.text}) as span:
            soup = BeautifulSoup(response.text, 'html.parser')
            paragraphs = soup.find_all('p')
            article_text = "\n".join([p.get_text() for p in paragraphs])
            truncated_text = (article_text[:4000] + '...') if len(article_text) > 4000 else article_text
            span.output = {"truncated_text": truncated_text}
        return truncated_text

    except requests.RequestException as e:
        error_message = f"An error occurred while fetching the URL: {e}"
        scoped_span.set_error(error_message) if scoped_span else None
        return error_message
    except Exception as e:
        error_message = f"An unexpected error occurred during content extraction: {e}"
        scoped_span.set_error(error_message) if scoped_span else None
        return error_message


async def main() -> None:
    today_str = f"Today's date is {date.today().strftime("%A, %B %d, %Y")}."

    news_agent = Agent(
        name="Personalized News Digest Agent",
        instructions=(
            f"{today_str}. You are a research assistant. Your goal is to provide a concise summary of recent news based on the user's request. "
            "First, use the `Google Search` tool to find relevant articles. "
            "Next, use the `get_article_content` tool on the most promising URLs to read them. "
            "Finally, synthesize the information from the articles into a coherent summary for the user."
        ),
        model="gpt-4-turbo",
        tools=[google_search, get_article_content],
    )

    user_prompt = input("Please enter your news request: ")
    result = await Runner.run(news_agent, user_prompt)
    print(result.final_output)

if __name__ == "__main__":
    SGP_API_KEY, SGP_ACCOUNT_ID = "XXX", "XXX"
    os.environ["OPENAI_API_KEY"], os.environ["GOOGLE_API_KEY"], os.environ["GOOGLE_CSE_ID"] = "XXX", "XXX", "XXX"

    client = SGPClient(api_key=SGP_API_KEY, account_id=SGP_ACCOUNT_ID)

    tracing.init(client)
    set_trace_processors([OpenAITracingSGPProcessor()])

    asyncio.run(main())
```


# OpenTelemetry Integration
Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/opentelemetry

Instrument your agents with standard OpenTelemetry integrations and send traces to SGP

<Warning>
  **Experimental Feature**: This integration is experimental and can change at any point. The OpenTelemetry GenAI spec itself is evolving rapidly, which may affect this integration. This service might not be available in all variations of SGP.
</Warning>

The OpenTelemetry integration allows your agents to be instrumented with standard OpenTelemetry integrations for your framework. By passing a few headers (your SGP API key and account ID), traces are automatically sent to SGP for observability.

For native SDK integration instead, see [Tracing SDK Initialization](/docs/v5/tracing/sdk/initialization).

***

## How It Works

This experimental forwarder service acts as a proxy between OpenTelemetry-instrumented applications and SGP's tracing system. When you send OTLP traces to the SGP OpenTelemetry endpoint, the service automatically converts them into SGP's internal tracing format.

The conversion process maps OpenTelemetry's [GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) to SGP metadata (such as model names, token counts, and provider information), auto-detects frameworks like Pydantic AI to extract input and output data, and infers operation types including completions, retrieval, reranking, and other operations.

For detailed information on GenAI semantic conventions, refer to the [OpenTelemetry GenAI attributes specification](https://opentelemetry.io/docs/specs/semconv/registry/attributes/gen-ai/).

***

## Prerequisites

To use the OpenTelemetry integration, you'll need:

1. **SGP API credentials**: Your API key and account ID
2. **OpenTelemetry SDK**: Install the appropriate SDK for your language
3. **OTLP endpoint**: `https://opentelemetry.<sgp-url-placeholder>`

**Configuration:**

```bash theme={null}
export OTEL_EXPORTER_OTLP_ENDPOINT="https://opentelemetry.<sgp-url-placeholder>"
export OTEL_EXPORTER_OTLP_HEADERS="x-sgp-api-key=YOUR_API_KEY,x-sgp-account-id=YOUR_ACCOUNT_ID"
```

***

## Usage Examples

These examples show how to use ecosystem integrations with SGP's OpenTelemetry forwarder.

### OpenAI with OpenLLMetry

[OpenLLMetry](https://github.com/traceloop/openllmetry) provides automatic instrumentation for OpenAI and other LLM providers:

```python {6-7, 10-14} Python theme={null}
import os
from openai import OpenAI
from traceloop.sdk import Traceloop

# Set SGP credentials
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://opentelemetry.<sgp-url-placeholder>"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "x-sgp-api-key=YOUR_KEY,x-sgp-account-id=YOUR_ACCOUNT_ID"

# Initialize Traceloop - automatically instruments OpenAI
Traceloop.init(
    app_name="my-app",
    api_endpoint=os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"],
    disable_batch=False,
)

# Use OpenAI normally - traces are automatically sent to SGP
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
```

OpenLLMetry automatically captures model names, token usage, prompts, completions, function calls, and performance metrics.

### OpenAI Agents SDK with Logfire

[Logfire can instrument the OpenAI Agents SDK](https://logfire.pydantic.dev/docs/integrations/llms/openai/#openai-agents) for comprehensive observability:

```python {6-7, 10-11} Python theme={null}
import os
import logfire
from agents import Agent, Runner

# Set SGP credentials
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://opentelemetry.<sgp-url-placeholder>"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "x-sgp-api-key=YOUR_KEY,x-sgp-account-id=YOUR_ACCOUNT_ID"

# Configure Logfire to use custom OTLP endpoint (respects OTEL env vars)
logfire.configure(send_to_logfire=False)
logfire.instrument_openai_agents()

# Create and run agent - traces are automatically sent to SGP
agent = Agent(name="assistant", instructions="You are a helpful assistant")
result = Runner.run_sync(agent, "Write a haiku about recursion in programming.")
```

### Pydantic AI with Logfire

[Pydantic AI integrates with Logfire](https://ai.pydantic.dev/logfire/#setting-opentelemetry-sdk-providers) for observability. You can redirect Logfire traces to SGP by configuring the OpenTelemetry SDK providers:

```python {6-7, 10-11} Python theme={null}
import os
import logfire
from pydantic_ai import Agent

# Set SGP credentials
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://opentelemetry.<sgp-url-placeholder>"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "x-sgp-api-key=YOUR_KEY,x-sgp-account-id=YOUR_ACCOUNT_ID"

# Configure Logfire to use custom OTLP endpoint (respects OTEL env vars)
logfire.configure(send_to_logfire=False)
logfire.instrument_pydantic_ai()

# Create and run agent - traces are automatically sent to SGP
agent = Agent("openai:gpt-4o-mini")
result = agent.run_sync("What is the capital of France?")
```

### LangChain with LangSmith

[LangChain's observability](https://docs.langchain.com/langsmith/observability-quickstart#3-define-your-application) can be configured to export traces via OpenTelemetry. Note that LangChain requires the full `/v1/traces` path in the endpoint URL:

```python {6,8-9} Python theme={null}
import os
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

# Enable LangSmith OpenTelemetry export to SGP
os.environ["LANGSMITH_OTEL_ENABLED"] = "true"
# Note: LangChain requires the full /v1/traces path in the endpoint
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://opentelemetry.<sgp-url-placeholder>/v1/traces"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "x-sgp-api-key=YOUR_KEY,x-sgp-account-id=YOUR_ACCOUNT_ID"

# Use LangChain normally - traces are automatically sent to SGP
prompt = ChatPromptTemplate.from_template("Tell me a joke about {topic}")
model = ChatOpenAI(model="gpt-4o-mini")
chain = prompt | model
result = chain.invoke({"topic": "programming"})
```

***

## Viewing Traces

After sending traces, view them in the SGP UI:

1. Navigate to your application in the SGP dashboard
2. Go to the Traces tab
3. Filter and drill down into your OTLP-generated traces

Traces will appear with the operation types and metadata automatically extracted from your OTLP spans.

For more information about SGP tracing features and concepts, see the [Tracing Overview](/docs/v5/tracing/overview).


# Temporal Integration
Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/temporal

Emit SGP traces from Temporal workflows and activities without breaking workflow determinism.

<Info>
  If you're using **agentex agents on Temporal**, you don't need this page — agentex provides its own SGP tracing integration through its tracing processors. This guide is for users writing their own Temporal workflows or activities that emit SGP spans directly.
</Info>

[Temporal](https://temporal.io/) workflows are deterministic: workflow code is replayed from history on every worker, so it cannot perform network IO, generate non-deterministic UUIDs, or read system time directly. This makes integrating any tracing SDK with Temporal a question of **where** the tracing calls live.

Activities, by contrast, are regular Python functions running on a worker. They are free to do network IO, so the SGP Tracing SDK works inside an activity exactly like in any other process.

This guide covers the two patterns that come up in practice.

## Pattern 1: Trace at the activity boundary (recommended)

Use this when "one activity = one span" gives you the granularity you need. It is the simplest and matches how most Temporal applications are observed.

* Activities call `tracing.create_span(...)` directly using the SDK's context manager.
* The caller of the workflow creates the parent span and passes `trace_id` and `parent_span_id` as workflow input.
* The workflow forwards those IDs to each activity. Each activity span is rooted under the caller's parent.

```python theme={null}
from datetime import timedelta
import scale_gp_beta.lib.tracing as tracing
from scale_gp_beta import SGPClient
from temporalio import activity, workflow
from temporalio.client import Client
from temporalio.worker import Worker

@activity.defn
async def fetch_data_activity(url: str, trace_id: str, parent_span_id: str) -> dict:
    with tracing.create_span(
        name="fetch_data",
        trace_id=trace_id,
        parent_id=parent_span_id,
        input={"url": url},
    ) as span:
        # ... actual work ...
        result = {"status": "ok", "url": url}
        span.output = result
        return result

@workflow.defn
class MyWorkflow:
    @workflow.run
    async def run(self, urls: list[str], trace_id: str, parent_span_id: str) -> list[dict]:
        results = []
        for url in urls:
            result = await workflow.execute_activity(
                fetch_data_activity,
                args=(url, trace_id, parent_span_id),
                start_to_close_timeout=timedelta(seconds=30),
            )
            results.append(result)
        return results

async def run_worker():
    tracing.init(SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID"))
    client = await Client.connect("localhost:7233")
    worker = Worker(
        client,
        task_queue="my-queue",
        workflows=[MyWorkflow],
        activities=[fetch_data_activity],
    )
    await worker.run()
```

The caller starts the workflow inside an existing span, passing the IDs through workflow input:

```python theme={null}
import scale_gp_beta.lib.tracing as tracing
from scale_gp_beta import SGPClient
from temporalio.client import Client

# Initialize the SDK once at the caller process entry point. The caller and
# the worker run in different processes, each with its own queue manager, so
# tracing.init() is required in both.
tracing.init(SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID"))

async def kick_off():
    client = await Client.connect("localhost:7233")
    with tracing.create_trace("my_pipeline") as trace:
        with tracing.create_span("orchestrate_pipeline") as parent:
            await client.execute_workflow(
                MyWorkflow.run,
                args=[["https://a.example", "https://b.example"], trace.trace_id, parent.span_id],
                id="my-workflow-1",
                task_queue="my-queue",
            )
```

<Warning>
  Use `workflow.uuid4()` (Temporal's deterministic UUID helper), not `uuid.uuid4()`, when generating IDs inside workflow code. Standard library `uuid.uuid4()` is non-deterministic and will break workflow replay.
</Warning>

<Info>
  The SDK's background queue worker is a daemon thread inside the **activity worker** process, not the workflow. It is fine to use the default `tracing.init()` configuration. The queue flushes on size or cadence triggers and on worker shutdown.
</Info>

## Pattern 2: Buffer in workflow, flush via activity (advanced)

Use this when you want span boundaries that mirror workflow steps rather than activity invocations, or when activity calls are too coarse-grained to capture the hierarchy you want.

The shape:

* The workflow buffers lightweight span dicts (the SDK's `Item` `TypedDict`) in workflow state. No network IO, no SDK calls.
* A dedicated `flush_spans` activity receives the buffered list and writes them via `client.spans.upsert_batch(items=...)`.
* The flush activity runs:
  * when the buffer hits a size threshold,
  * on a periodic timer,
  * before the workflow completes or calls `continue_as_new`.

```python theme={null}
from datetime import timedelta
from scale_gp_beta import AsyncSGPClient
from scale_gp_beta.types.span_upsert_batch_params import Item as SpanRequest
from temporalio import activity, workflow
from temporalio.common import RetryPolicy

# Module-level client so we reuse the connection pool across activity invocations.
_client = AsyncSGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID")

# 4xx-style errors that should not retry: bad input, auth failure, validation.
# 5xx, rate limits, and transport errors are retried by Temporal per the policy below.
_NON_RETRYABLE = [
    "BadRequestError",
    "AuthenticationError",
    "PermissionDeniedError",
    "NotFoundError",
    "UnprocessableEntityError",
]

@activity.defn
async def flush_spans_activity(spans: list[SpanRequest]) -> None:
    # No tracing.init() needed here: this activity bypasses the SDK queue and
    # writes directly via upsert_batch. tracing.init() is only required if you
    # call tracing.create_span(...) inside an activity (Pattern 1).
    await _client.spans.upsert_batch(items=spans)

@workflow.defn
class BufferedTracingWorkflow:
    def __init__(self) -> None:
        self._buffer: list[SpanRequest] = []

    def _enqueue(self, span: SpanRequest) -> None:
        self._buffer.append(span)

    async def _maybe_flush(self, force: bool = False) -> None:
        if not self._buffer:
            return
        if not force and len(self._buffer) < 200:
            return
        spans_to_send = self._buffer
        self._buffer = []
        await workflow.execute_activity(
            flush_spans_activity,
            args=(spans_to_send,),
            start_to_close_timeout=timedelta(seconds=30),
            schedule_to_close_timeout=timedelta(minutes=10),
            retry_policy=RetryPolicy(
                initial_interval=timedelta(seconds=1),
                maximum_interval=timedelta(seconds=30),
                maximum_attempts=5,
                non_retryable_error_types=_NON_RETRYABLE,
            ),
        )

    @workflow.run
    async def run(self, items: list[str]) -> None:
        trace_id = workflow.uuid4().hex
        root_span_id = workflow.uuid4().hex
        root: SpanRequest = {
            "id": root_span_id,
            "trace_id": trace_id,
            "name": "workflow_root",
            "start_timestamp": workflow.now().isoformat(),
        }
        self._enqueue(root)

        for item in items:
            child: SpanRequest = {
                "id": workflow.uuid4().hex,
                "trace_id": trace_id,
                "parent_id": root_span_id,
                "name": f"process_{item}",
                "start_timestamp": workflow.now().isoformat(),
            }
            # Real work goes through an activity:
            # await workflow.execute_activity(do_work, item, start_to_close_timeout=...)
            child["end_timestamp"] = workflow.now().isoformat()
            self._enqueue(child)
            await self._maybe_flush()

        # Set root's end_timestamp and re-enqueue so the upsert lands. The
        # upsert_batch endpoint is keyed on `id`, so a second send with
        # end_timestamp set just updates the existing row. This handles the
        # case where root was already flushed mid-loop (buffer hit 200) and
        # the in-memory mutation alone would be lost.
        root["end_timestamp"] = workflow.now().isoformat()
        self._enqueue(root)
        await self._maybe_flush(force=True)
```

<Info>
  `Item` is a `TypedDict`, so values are plain `dict` at runtime. That means no `asdict(...)` conversion before the flush activity, and Temporal's default data converter serializes them as JSON cleanly.
</Info>

<Warning>
  A non-retryable failure (any error class listed in `non_retryable_error_types`) drops the buffered batch. The data is gone — there is no second chance for those spans. Choose the non-retryable list deliberately: list only errors where retrying truly cannot succeed (bad input, auth) and let everything else retry.
</Warning>

### Why `upsert_batch` and not `create`

* **Idempotency.** Temporal retries activities on transient failures. `upsert_batch` is keyed on the client-supplied span `id`, so a retry that partially succeeded will re-converge to the right state. `client.spans.create` is not safe under retry because a partial success leaves orphan rows.
* **Throughput.** One HTTP call carries many spans. The batch endpoint accepts up to 1000 items per request.

<Info>
  Keep batches at 200 to 500 items per flush in practice. That keeps payloads under 1 MB and HTTP timeouts comfortable, and gives you headroom under the 1000-item server cap.
</Info>

## Span integrity discipline

The server rejects the whole batch if any child's `trace_id` does not match its parent's. This is easy to trip if parent IDs are minted in workflow code but child IDs are minted in an activity without the trace context piped through. Always pass `trace_id` (and `parent_span_id` where relevant) explicitly when crossing the workflow/activity boundary.

## Distributed tracing across workflows

If one workflow starts another (child workflow, or an external `start_workflow` call), propagate `trace_id` and the originating span's `id` via workflow input or memo, and use them in the child workflow as `parent_id` for its root span. The same propagation rule from the [Distributed Tracing](/docs/v5/tracing/sdk/multi-process) page applies; Temporal's workflow-input mechanism is a clean place to carry it.


# Introduction to Tracing
Source: https://docs.gp.scale.com/docs/v5/tracing/overview

Gain deep visibility into your workflows with tracing.

## Overview

Modern applications, especially those leveraging AI agents and complex tool use, are distributed and often opaque. Understanding the flow of data, performance bottlenecks, and error propagation across different services and functions is critical for debugging, optimization, and ensuring reliability.

Our Tracing capabilities provide **deep observability** into your workflows by allowing you to generate, collect, and visualize **spans** and **traces**. This system is designed to significantly decrease the overhead of understanding complex operations while increasing your ability to debug and monitor.

At its core, Tracing now consists of:

1. **Spans**: Individual units of work within your application.
2. **Traces**: Collections of interconnected spans representing a complete workflow.
3. **Dedicated Traces Page**: A new UI for visualizing, filtering, and drilling down into your tracing data.
4. **Flexible SDK & APIs**: Tools to instrument your Python applications and integrate tracing directly into your code.

## Traces & Spans Concepts

To effectively use tracing, it's helpful to understand the fundamental concepts:

* **Span:** A span represents a single **unit-of-work** within a trace, such as a function call, a database query, or an external API request. It captures crucial information about that operation, including:
  * `name`: A descriptive name for the operation.
  * `input` & `output`: Data flowing into and out of the operation.
  * `trace_id`: The identifier of the overall workflow this span belongs to.
  * `parent_id`: The identifier of the span that directly invoked this operation, allowing us to build a nested hierarchy.
  * `start_time` & `end_time`: Timestamps for the duration of the operation.
  * `metadata`: Arbitrary key-value pairs for additional context or custom annotations.
  * `status`: Indicates if the span completed successfully or with an error.
  * `type`: Optionally give the span a type, to better filtering and visualization

* **Trace:** A trace is a **collection of spans** that together describe a complete, end-to-end workflow, such as a user's web request, an AI agent's complex decision-making process, or a batch processing job.
  * Every trace begins with a single **"root-span"**, which represents the starting point of that entire workflow.
  * Server-side, we primarily record and visualize spans. The "trace" resource itself is implicitly derived from the relationships and common `trace_id` among spans, with the root span providing the top-level details for the workflow.

For a deeper dive into the standardized tracing model, refer to the OpenTelemetry specification on Traces & Spans: [https://opentelemetry.io/docs/specs/otel/overview/#tracing-signal](https://opentelemetry.io/docs/specs/otel/overview/#tracing-signal).

## Traces Page in the UI

The new dedicated Traces page in the SGP UI provides powerful capabilities to monitor, filter, and drill down into your trace data:

* **High-Level Monitoring:** Get an immediate overview of your application's workflows.
* **Filtering:** Use basic filters now, with more in-depth filtering capabilities coming soon. You'll be able to search on:
  * **Static fields**: Like a specific trace ID or a span type.
  * **Dynamic fields**: Custom key-value pairs found in your span's metadata.
* **Trace Detail View:** Clicking on any trace reveals a detailed breakdown, allowing you to:
  * View run-time statistics for each span.
  * Inspect input, output, and metadata associated with each span.
  * Trigger thread-forking (for SGP native traces) for advanced debugging.

## Getting Started with Tracing

<Info>Examples in this guide use the [`scale-gp-beta`](https://pypi.org/project/scale-gp-beta/) package, which runs exclusively on the V5 API.</Info>

The SGP Tracing SDK provides a convenient and ergonomic way to instrument your Python applications. The Python SDK is auto-generated using Stainless with a custom code patch for ergonomic tracing support.

### Initialization

Before you can create any traces or spans, you should initialize the tracing SDK with your `SGPClient`.
It's best practice to do this once at your application's entry point.
It is not strictly required to call `tracing.init()`, the SDK will attempt to create a default client using default ENV variables, if not set we log a warning and will not send tracing data.

```python theme={null}
import scale_gp_beta.lib.tracing as tracing
from scale_gp_beta import SGPClient

# Initialize with your API Key and Account ID
client = SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID")
tracing.init(client=client)
```

The tracing SDK uses the `SGPClient` for all data export requests. This means you can leverage existing client configurations, such as `base_url` or `timeout` settings, by passing a pre-configured client to `tracing.init()`.

### Creating Your First Trace & Spans

The most straightforward and recommended way to create traces and spans is by using them as context managers (`with` statements). This ensures that spans are automatically started, ended, and errors are captured, simplifying your instrumentation.

### Define a Workflow as a Trace

Use `tracing.create_trace()` as a context manager to define a new end-to-end workflow. This automatically creates the root span for your trace.

```python theme={null}
import scale_gp_beta.lib.tracing as tracing

def my_application_flow():
    with tracing.create_trace(name="my_customer_onboarding_process", metadata={"customer_tier": "premium"}):
        print("Starting customer onboarding...")
        # ... logic for onboarding
        print("Customer onboarding complete.")
```

### Instrument Units of Work as Spans

Inside a `create_trace` block, use `tracing.create_span()` as a context manager. These spans will automatically inherit the `trace_id` from the current trace and will be nested under the currently active span (or the root span if no other span is active).

<Warning>If you are working in a distributed setting, please see [Distributed Tracing](/docs/v5/tracing/sdk/multi-process) </Warning>

### Quick Start Examples

For quick runnable examples, see the [examples/tracing](https://github.com/scaleapi/sgp-python-beta/tree/main/examples/tracing) directory in the repository.

## Traces API Endpoints

For direct interaction with span data via the API, beyond the SDK, you can use the following endpoints:

* [Create a Span](/reference/v5/spans/create-span)
* [Get Span](/reference/v5/spans/get-span)
* [Update Span](/reference/v5/spans/update-span)
* [Batch Create Spans](/reference/v5/spans/create-spans-in-batch)
* [Batch Upsert Spans](/reference/v5/spans/upsert-spans-in-batch)
* [Search Spans](/reference/v5/spans/search-and-list-spans)


# Configuration Options
Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/configuration-options

Configure the SGP Tracing SDK's behavior.

The SGP Tracing SDK allows you to customize its behavior through a combination of environment variables and parameters passed during SDK initialization.

## Environment Variables

These environment variables provide a quick way to configure common settings, especially for deployment environments.

| Environment Variable    | Description                                                                                                                                                                                                                                             |
| :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `DISABLE_SCALE_TRACING` | If set to any value (e.g., `true`, `1`), tracing data will **not** be exported. In this mode, the SDK returns "No-Op" variants of Trace and Span objects, which allow your code to run without errors but perform no actual data collection or sending. |
| `SGP_API_KEY`           | Your SGP API Key. If set, the SDK can automatically initialize without needing to pass an `SGPClient` explicitly to `tracing.init()`.                                                                                                                   |
| `SGP_ACCOUNT_ID`        | Your SGP Account ID. If set, the SDK can automatically initialize without needing to pass an `SGPClient` explicitly to `tracing.init()`.                                                                                                                |
| `SGP_CLIENT_BASE_URL`   | Custom base URL for API requests. If set, this overrides the default environment URL. Mutually exclusive with the `environment` parameter.                                                                                                              |

## Programmatic Configuration

The `tracing.init()` function, typically called at your application's entry point, accepts various parameters to configure the tracing behavior. Programmatic settings take precedence over environment variables.

```python theme={null}
import scale_gp_beta.lib.tracing as tracing
from scale_gp_beta import SGPClient

# Example of a comprehensive initialization
tracing.init(
    client=SGPClient(
        api_key="YOUR_API_KEY",
        account_id="YOUR_ACCOUNT_ID",
        base_url="SGP_CLIENT_BASE_URL",
        timeout=30.0
    ),
    disabled=False, # Explicitly enable or disable tracing (overrides ENV var if False)
)
```

### `client`

An initialized `SGPClient` instance. If provided, the SDK will use this client for all tracing data export requests. This allows you to configure HTTP settings (like `base_url`, `timeout`, `max_retries` for the client itself) that will also apply to your tracing data.

### `disabled`

* Set to `True` to explicitly disable tracing. When disabled, `create_trace()` and `create_span()` will return "No-Op" objects that perform no data collection or export.
* Set to `False` to explicitly enable tracing.
* Defaults to `False` if not specified, unless `DISABLE_SCALE_TRACING` environment variable is set.


# Creating Traces and Spans
Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/creating-traces-spans

Instrument your Python code with the Tracing SDK.

The SGP Tracing SDK offers flexible ways to instrument your application by creating traces and spans. You can choose between two main approaches: using **context managers** for automatic lifecycle management, or **explicit control** for manual start/end handling.

## Context Managers (Recommended)

The most straightforward and robust way to create traces and spans is by using them as Python context managers (`with` statements). This approach automatically handles span start and end times, associates spans with the correct trace context, and captures exceptions, significantly reducing boilerplate and potential errors.

### Creating a Trace with a Root Span

Every trace begins with a root span. Use `tracing.create_trace()` as a context manager to define a new top-level workflow. This automatically creates the root span for your trace, setting the context for all child spans created within its block.

```python {1, 5} theme={null}
import scale_gp_beta.lib.tracing as tracing

def my_workflow():
    # This creates a new trace and its root span
    with tracing.create_trace(name="my_application_workflow", metadata={"env": "production"}):
        print("Starting my application workflow...")
        # ... your main workflow logic goes here ...
        print("Application workflow completed.")
```

### Creating Spans within a Trace

Inside a `tracing.create_trace` block, use `tracing.create_span()` as a context manager. These spans will automatically:

* Inherit the `trace_id` from the current trace.
* Be associated as a child of the currently active span (if another `create_span` is active) or the root span.

```python {2, 6, 18, 20} theme={null}
import time
import scale_gp_beta.lib.tracing as tracing

def fibonacci(curr: int) -> int:
    # This span becomes a child of the current active span (e.g., "main_execution")
    with tracing.create_span("fibonacci_calculation", input={"curr": curr}) as span:
        time.sleep(0.05) # Simulate some work
        if curr < 2:
            span.output = {"res": curr}
            return curr

        res = fibonacci(curr - 1) + fibonacci(curr - 2)
        span.output = {"res": res}
        return res

def main_traced_example():
    # This creates the main trace and its root span
    with tracing.create_trace("my_fibonacci_trace"):
        # This span will be a child of the "my_fibonacci_trace" root span
        with tracing.create_span("main_execution", metadata={"version": "1.0"}) as main_span:
            fib_result = fibonacci(5)
            main_span.output = {"final_fib_result": fib_result}
            print(f"Fibonacci(5) = {fib_result}")
```

<Tip>
  The [Traces table](/docs/v5/tracing/tracing-ui/traces-table#controlling-the-content-preview) preview column extracts text from specific preferred keys in your span `input` and `output` (e.g., `title`, `query`, `content`). If the preview shows an unhelpful value, add a `title` key with the value you want displayed.
</Tip>

## Explicit Control

For advanced scenarios where context managers are not suitable (e.g., integrating with existing systems that manage context, or reporting historical data), you can manually manage the lifecycle of your spans. This approach requires more diligence to ensure `start()` and `end()` are always called, and consistency is maintained across your trace.

### Manually Managing Spans (without implicit context)

You can create `Span` objects and explicitly control their `trace_id` and `parent_id` for fine-grained hierarchy management. This is particularly useful when you need to bridge tracing across different processes, services, or when a context manager is not practical.

Remember to manually call `span.start()` to begin the span's lifecycle and `span.end()` to complete it. For error handling, you'll need to wrap your logic in a `try...finally` block.

```python {5,15-16,20,23,26,35-36,41} theme={null}
import uuid
import time
import random
from typing import Any, Dict
import scale_gp_beta.lib.tracing as tracing

class MockDatabase:
    def __init__(self) -> None:
        self._data = {
            "SELECT * FROM users WHERE id = 1;": {"id": 1, "name": "Alice"},
            "SELECT * FROM users WHERE id = 2;": {"id": 2, "name": "Bob"},
        }
    def execute_query(self, query: str, trace_id: str) -> Dict[str, Any]:
        # Manually create, start, set output, and end the span
        db_span = tracing.create_span("db_query", input={"query": query}, trace_id=trace_id)
        db_span.start() # Manually start the span
        try:
            time.sleep(random.uniform(0.1, 0.3)) # Simulate delay
            result = self._data.get(query, {})
            db_span.output = {"result": result}
            return result
        finally:
            db_span.end() # Manually end the span

def get_user_from_db_explicit(db: MockDatabase, user_id: int, trace_id: str) -> Dict[str, Any]:
    with tracing.create_span("get_user_from_db", input={"user_id": user_id}, trace_id=trace_id):
        query = f"SELECT * FROM users WHERE id = {user_id};"
        return db.execute_query(query, trace_id)

def main_explicit_control_example():
    db = MockDatabase()
    my_trace_id = str(uuid.uuid4())

    # Manually create a root span for the trace, providing the trace_id explicitly
    main_span = tracing.create_span("main_explicit_call", metadata={"env": "local"}, trace_id=my_trace_id)
    main_span.start() # Manually start the main span
    try:
        user = get_user_from_db_explicit(db, 1, my_trace_id)
        print(f"Retrieved user: {user.get('name')}")
    finally:
        main_span.end() # Manually end the main span
```

### Exporting Historical or Pre-defined Tracing Data

You can pre-define `start_time`, `end_time`, `span_id`, `parent_id`, and `trace_id` when creating a span. This is useful for reporting historical data, replaying events, or reconstructing traces from external sources. After setting these attributes, call `span.flush()` to send the data.

```python theme={null}
import uuid
from datetime import datetime, timezone, timedelta
import scale_gp_beta.lib.tracing as tracing

parent_span_id = str(uuid.uuid4())
trace_id = str(uuid.uuid4())
child_span_id = str(uuid.uuid4())

now = datetime.now(timezone.utc)

# Parent Span: Create and configure a span with specific IDs and timestamps.
parent_span = tracing.create_span(
    "my_parent_span_name",
    input={"test": "input"},
    output={"test": "output"},
    metadata={"test": "metadata"},
    span_id=parent_span_id,
    trace_id=trace_id,
)
parent_span.start_time = (now - timedelta(minutes=10)).isoformat()
parent_span.end_time = now.isoformat()
parent_span.flush(blocking=True) # Send this span immediately

# Child Span: Link to the parent and trace using explicit IDs and timestamps.
child_span = tracing.create_span(
    "my_child_span_name",
    input={"test": "another input"},
    output={"test": "another output"},
    metadata={"test": "another metadata"},
    span_id=child_span_id,
    trace_id=trace_id,
    parent_id=parent_span_id, # Link to the parent span
)
child_span.start_time = (now - timedelta(minutes=6)).isoformat()
child_span.end_time = (now - timedelta(minutes=2)).isoformat()
child_span.flush() # Send this span immediately (defaults to blocking=True)
```

<Warning>
  `span.flush()` by default blocks the main thread until the request has finished. For non-blocking behavior, use `span.flush(blocking=False)` to enqueue the request for the background worker. This is generally recommended for performance-sensitive applications.
</Warning>


# Flushing Tracing Data
Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/flushing-data

Understand how and when to manually send buffered tracing data to SGP.

By default, tracing data (spans) are collected in an in-memory queue and sent to the SGP platform **asynchronously in batches** by a background worker. This batching mechanism is designed for efficiency and minimal performance impact on your application.

However, there are specific scenarios where you might need to force an immediate export of tracing data:

* **Critical workflow points:** To ensure data is persisted before sensitive operations or transitions in a distributed workflow.
* **Debugging:** To see tracing data appear on the Traces page immediately during development.

## How to Flush Tracing Data

The SDK provides methods to manually trigger data export:

### Flushing the entire queue

Use `tracing.flush_queue()` to force a synchronous export of *all* spans currently buffered in the SDK's internal queue. This method will block the calling thread until all pending spans have been sent.

```python {12} theme={null}
import scale_gp_beta.lib.tracing as tracing
import time

# Simulate creating some spans
with tracing.create_trace("short_lived_process_trace"):
    with tracing.create_span("task_alpha"):
        time.sleep(0.1)
    with tracing.create_span("task_beta"):
        time.sleep(0.1)

# Force all pending spans to be sent before the application exits
tracing.flush_queue()
print("All pending spans have been flushed to SGP.")

# In a multi-worker or distributed application, you might use this
# before enqueuing a job to ensure parent trace data is sent first.
```

### Flushing individual spans or traces

You can also explicitly flush an individual `Span` object. When you call `span.flush()`, that specific span will be sent. For `Trace` objects, flushing (`trace.flush()`) will send the root span.

```python theme={null}
import uuid
from datetime import datetime, timezone, timedelta
import scale_gp_beta.lib.tracing as tracing

now = datetime.now(timezone.utc)

# Example: Flushing a single span immediately after creation
single_span = tracing.create_span("critical_single_operation", trace_id=str(uuid.uuid4()))
single_span.start_time = now.isoformat()
single_span.end_time = (now + timedelta(seconds=1)).isoformat()
single_span.output = {"status": "completed"}
single_span.flush() # This span is sent to the backend immediately

# Example: Flushing a trace's root span
my_trace = tracing.create_trace(name="important_trace")
my_trace.root_span.start_time = now.isoformat()
# ... perform operations within the trace ...
my_trace.root_span.end_time = (now + timedelta(seconds=5)).isoformat()
my_trace.flush()

```

<Warning title="Blocking Behavior of `flush()`">
  By default, `span.flush()` and `trace.flush()` are **blocking** operations. This means the calling thread will pause until the data is successfully sent to the SGP platform.

  For non-blocking behavior (recommended in performance-sensitive applications), pass `blocking=False`:

  ```python theme={null}
  my_span = tracing.create_span("my_async_span", trace_id="...")
  my_span.start()
  # ... do work ...
  my_span.end()
  my_span.flush(blocking=False) # Enqueue for background worker, don't block
  ```
</Warning>

### Automatic Shutdown Flush

You typically do not need to manually call `flush_queue()` at the very end of your program. When the SDK's background worker is shut down (e.g., when your Python application exits normally), it will automatically attempt to flush any remaining tracing data in the queue before exiting.


# Helper Methods
Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/helper-methods

Access the currently active span and trace in your application's execution context.

The SGP Tracing SDK leverages Python's `contextvars` to automatically manage the active span and trace within your application's execution flow. This allows you to easily retrieve information about the current operation without explicitly passing objects around.

You can retrieve the currently active span or trace in the execution context using the following helper methods:

### `tracing.current_span()`

Returns the `Span` instance representing the operation currently active in the local execution context. This is typically the innermost span created by `tracing.create_span()`.

### `tracing.current_trace()`

Returns the `Trace` instance representing the overall workflow currently active in the local execution context. This will be the trace established by `tracing.create_trace()`.

### Example Usage

This example demonstrates how `current_span()` and `current_trace()` can be used to inspect the active context within nested operations:

```python {1,4-5} theme={null}
import scale_gp_beta.lib.tracing as tracing

def process_data_step():
    current_span_instance = tracing.current_span()
    current_trace_instance = tracing.current_trace()

    if current_span_instance:
        print(f"Current Span: {current_span_instance.name} (ID: {current_span_instance.span_id})")
    if current_trace_instance:
        print(f"Current Trace: (ID: {current_trace_instance.trace_id})")

    print("Processing data...")

def main_workflow_with_helpers():
    with tracing.create_trace("my_main_workflow"):
        print("Main workflow started.")
        with tracing.create_span("data_processing"):
            process_data_step()
```

### When are these methods useful?

* **Debugging:** Quickly inspect the current tracing context during development or when logging.
* **Contextual Logging:** Enrich log messages with `span_id` or `trace_id` without needing to pass them explicitly.
* **Conditional Logic:** Implement logic that depends on the existence or attributes of the current span/trace.

These methods are primarily for introspection and should not be used to create or manage the lifecycle of spans, which should be done via `create_span()` and `create_trace()`.


# Initialization
Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/initialization

Set up and initialize the Scale GP Tracing SDK in your Python applications.

The SGP Tracing SDK provides a convenient way to instrument your Python applications with tracing capabilities, allowing you to generate, manage, and send spans to the Scale GP platform. This enables detailed monitoring and debugging of your workflows.

<Info>Python SDK tracing is currently only available in the Beta SDK.</Info>

## Installation

First, install the `scale-gp-beta` package using pip:

```sh theme={null}
pip install --pre scale-gp-beta
```

## Initialisation

Before you can create any traces or spans, you must initialize the tracing SDK. It's best practice to do this once at your application's entry point.

You have two primary options for initialization:

### Initialize with an `SGPClient`

This method gives you full control over the `SGPClient`'s configuration (e.g., `base_url`, `timeout`).

```Python theme={null}
import scale_gp_beta.lib.tracing as tracing
from scale_gp_beta import SGPClient

# Replace with your actual API Key and Account ID
client = SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID")
tracing.init(client=client)
```

### Initialize using Environment Variables

If you have set the `SGP_API_KEY` and `SGP_ACCOUNT_ID` environment variables, you can omit passing a client. The SDK will automatically attempt to create a default `SGPClient` for you.

```bash theme={null}
export SGP_API_KEY="sk-your-api-key"
export SGP_ACCOUNT_ID="your-account-id"
```

Then, in your Python code:

```python theme={null}
import scale_gp_beta.lib.tracing as tracing

# The SDK will automatically find SGP_API_KEY and SGP_ACCOUNT_ID
# from your environment variables.
tracing.init()
```

## Disabling Tracing

You can control whether tracing data is collected and exported. This is useful for development, testing, or specific environments where tracing is not desired.

### Environment Variable

```bash theme={null}
export DISABLE_SCALE_TRACING=true
```

### Programmatically

Pass the `disabled=True` parameter directly to the `init()` function. This will override the environment variable setting.

```python theme={null}
import scale_gp_beta.lib.tracing as tracing
from scale_gp_beta import SGPClient

# Tracing will be disabled, regardless of environment variables
tracing.init(client=SGPClient(api_key="...", account_id="..."), disabled=True)
```


# Distributed Tracing
Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/multi-process

Implement tracing across multiple processes and workers in your Python app.

Distributed applications, especially those involving multiple Python processes, threads, or asynchronous workers, introduce unique challenges for tracing. While the SGP Tracing SDK automatically manages span context within a single Python process (using `contextvars`), this automatic propagation **does not extend across process boundaries or when work is explicitly dispatched to new, independent execution contexts.**

<Info>
  The SGP backend no longer expects spans to arrive in-order. This means there is a chance that child spans may become orphans if their respective parent span fails to send.
</Info>

This guide outlines key strategies for effective tracing in multi-process and multi-worker environments.

## Understanding Context Propagation

When using context managers (`with tracing.create_span(...)`), the SDK automatically sets the current span and trace in a context-local variable. However, when you:

* Spawn a new Python process (e.g., using `multiprocessing`).
* Enqueue a task to a background job queue (e.g., Celery, RQ).
* Dispatch work to a separate thread pool where contextvars might not propagate by default (though `threading` typically handles this better than `multiprocessing`).

The new execution context will **not** automatically inherit the `trace_id` or `parent_id` from the originating process. You must explicitly pass this context.

## Strategies for Distributed Tracing

### One Trace Per Worker (Simplest)

For parallel work where strict hierarchical linking of *every* operation across workers isn't necessary, the easiest approach is to create an independent trace for each worker or process.

You can then use a `group_id` to logically link these independent traces together, allowing you to see all related activity in the Traces page, even if they don't form a single, continuous trace hierarchy. This is ideal for scenarios where workers process independent units of work concurrently.

```python theme={null}
import uuid
import time
import multiprocessing
import scale_gp_beta.lib.tracing as tracing
from scale_gp_beta import SGPClient

def process_item_in_worker(item_id: int, shared_group_id: str):
    # Each worker creates its own independent trace, ensure there is a working client with init or by setting ENV Vars
    with tracing.create_trace(name=f"worker_process_for_item_{item_id}", group_id=shared_group_id):
        with tracing.create_span("item_processing_task"):
            time.sleep(0.5)
            print(f"Worker {multiprocessing.current_process().name}: Processed item {item_id}")

def main_multi_process_example():
    batch_group_id = str(uuid.uuid4())

    processes = []
    for i in range(3):
        p = multiprocessing.Process(target=process_item_in_worker, args=(i, batch_group_id), name=f"Worker-{i}")
        processes.append(p)
        p.start()

    for p in processes:
        p.join()

if __name__ == "__main__":
    tracing.init(SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID"))
    main_multi_process_example()
```

### Extending a Trace Across Workers

If you need to maintain a single, continuous trace hierarchy where operations performed in separate workers are direct children of a span in the main process (or another worker), you must manually propagate the tracing context.

This involves:

1. **Retrieving** the `group_id`, `trace_id` and `span_id` of the parent span in the originating process.
2. **Passing** these IDs to the new worker/process (e.g., as function arguments or message queue payload fields).
3. **Using** these explicit IDs when creating new spans in the worker, ensuring they are correctly linked as children.

Be careful about setting the `Scope` object directly in your worker process as this scope may well be shared across jobs leading to incorrect tracing and leaking of tracing data.

#### Example: Passing Context via Function Arguments

```python theme={null}
import time
import multiprocessing
import scale_gp_beta.lib.tracing as tracing
from scale_gp_beta import SGPClient


def perform_subtask_in_worker(task_name: str, trace_id: str, parent_span_id: str):
    # Initialize tracing in the worker process. Can skip if you have SGP_API_KEY and SGP_ACCOUNT_ID set
    # tracing.init(SGPClient(api_key="...", account_id="..."))

    with tracing.create_span(
        name=f"worker_subtask_{task_name}",
        trace_id=trace_id,
        parent_id=parent_span_id,
        metadata={"worker_id": multiprocessing.current_process().name}
    ) as worker_span:
        print(f"Worker {multiprocessing.current_process().name}: Performing {task_name}")
        time.sleep(0.3)
        worker_span.output = {"status": "completed", "result": f"Task {task_name} finished"}

def main_distributed_trace_example():
    with tracing.create_trace("main_orchestrator_trace") as trace:
        processes = []
        for i in range(2):
            p = multiprocessing.Process(
                target=perform_subtask_in_worker,
                args=(f"subtask_{i+1}", trace.trace_id, trace.span_id),
                name=f"ChildWorker-{i+1}"
            )
            processes.append(p)
            p.start()

        for p in processes:
            p.join()

if __name__ == "__main__":
    tracing.init(SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID"))
    main_distributed_trace_example()
```

#### Important Considerations

* **Worker Initialization:** Each independent Python process (e.g., a new `multiprocessing.Process`) will have its own tracing queue manager. Ensure `tracing.init()` is called **within each worker's entry point** if you expect it to send tracing data. This typically means calling `tracing.init()` at the start of the function that the worker executes.
* **Error Handling:** In distributed systems, be diligent with error handling and ensure spans are ended correctly, even if an exception occurs. Context managers (`with tracing.create_span(...)`) handle this automatically within their scope.


# Profiler
Source: https://docs.gp.scale.com/docs/v5/tracing/tracing-ui/profiler

Analyze aggregated performance metrics and identify bottlenecks across your traces.

## Profiler

The Profiler provides an aggregated overview of where your agent spends its time, helping you pinpoint which spans are consistently causing delays across many executions. Instead of analyzing individual traces one by one, the Profiler shows performance metrics aggregated from all traces matching your filters, making it easy to identify systemic bottlenecks.

### Agent Selection

When you first open the Profiler tab, you'll see a list of agents to analyze. Select an agent to view its performance profile.

<img alt="agent-selection" />

### Performance Metrics

Once an agent is selected, the profiler displays key performance metrics at the top:

* **Average Duration**: The mean execution time across all traces for this agent.
* **Total Spans**: The total number of spans recorded.
* **P95 Latency**: The 95th percentile latency, showing worst-case performance for most requests.
* **Unique Spans**: The count of distinct span types in the traces.
* **View Agent Button**: Takes you to the Agents page to view the agent details.

<img alt="profiler-tab" />

Clicking "View Agent" navigates you to the Agents page where you can see additional details about the agent:

<img alt="agent-page" />

### Aggregated Span Table

The span table is the core of the Profiler. Instead of listing individual traces, it lists every unique span name pertaining to the selected agent (e.g., `generate_assistant_message`, `followup_search_state`) and shows performance metrics aggregated from all traces.

Key columns include:

* **Span Name**: The name of the operation.
* **Average Duration**: The average time this span takes to complete.
* **Total Time (%)**: The percentage of total execution time across all traces consumed by this span. Sorting by this column is the most effective way to find major bottlenecks.
* **P50 Latency**: The median execution time, representing typical performance.
* **P95 Latency**: The 95th percentile latency, crucial for understanding worst-case performance and identifying outliers.
* **Frequency (Avg. per Trace)**: How many times this span is called on average in a single execution.

Click any row to view detailed metrics in the Span Inspector.

### Aggregate Flame Graph

The flame graph provides a visual representation of aggregated span performance:

* Each horizontal bar represents a span, with width proportional to its Total Time (%).
* The visualization provides an immediate, intuitive map of your application's biggest latency contributors.
* Spans are stacked vertically to show parent-child relationships.
* Color coding helps distinguish different span types.
* Click on any bar in the flame graph to open the Span Inspector for detailed metrics.

### Span Inspector

When you click a row in the span table or a bar in the flame graph, the Inspector panel slides out with a deep dive on that specific span.

<img alt="span-inspector" />

**Detailed Stats**: The inspector shows all key metrics for the selected span:

* **% Execution Time**: The percentage of total execution time consumed by this span across all traces.
* **Average Duration**: Mean execution time for this span.
* **Min**: The fastest execution time observed.
* **P50 (Median)**: The median execution time, representing typical performance.
* **P95**: The 95th percentile execution time.
* **P99**: The 99th percentile execution time.
* **Max**: The slowest execution time observed.

**Latency Distribution**: A histogram showing the spread of durations for this span. This visualization helps you understand if latency is consistent or has a long tail of slow outliers.

**Example Traces**: Critical links back to the Traces page for deeper investigation:

* **View Slowest Traces**: Takes you to the Traces page, pre-filtered to show executions where this specific span was in the 95th percentile of slowness.
* **View Recent Traces**: Links to the most recent traces that include this span.

## Example Workflow

### Debugging Inconsistent Agent Performance

**The Problem**: A developer notices that the "Deep Research V3" agent's duration varies wildly. Some traces are fast, but others (like a 117.7s execution) are painfully slow. They don't know where to start looking.

**Step 1: Navigate to Profiler**

From the Traces view, they click the Profiler tab and select the "Deep Research V3" agent to analyze performance over the last 7 days.

**Step 2: Identify the Bottleneck**

They sort the Aggregated Span Table by **Total Time (%)** to find major bottlenecks. They immediately see that `final_report_state` is at the top, consuming 38% of all agent execution time. They also notice its P95 latency is over 2x its P50 median (85.6s vs 35.5s), confirming it's a source of inconsistent performance.

**Step 3: Inspect the Span**

They click the `final_report_state` row in the span table. The Span Inspector panel slides out, showing a histogram with a long tail. The detailed statistics confirm the outlier problem—some executions are extremely slow while most are reasonable.

**Step 4: Find Specific Examples**

In the Span Inspector, they click "View Slowest Traces" to see the actual problem cases.

**Step 5: Debug**

They are taken back to the Traces page, but it's pre-filtered to show only the 5% of traces where `final_report_state` was slowest. They click the top trace in this list, which brings them to the detailed waterfall view. Now, instead of guessing, they know to focus their investigation on the `final_report_state` span and its children to solve the systemic issue.

## Common Use Cases

### Identifying Performance Bottlenecks

Use the Profiler to identify which spans are consuming the most time across your traces:

1. Select an agent to view its aggregated performance profile.
2. Sort the Aggregated Span Table by **Total Time (%)** to find major bottlenecks.
3. Examine the aggregate flame graph to visualize which operations take the longest—width represents each span's contribution to total execution time.
4. Look for spans with high P95 or P99 latencies relative to their P50 median, indicating inconsistent performance.
5. Click on slow spans to view detailed statistics and latency distribution in the Span Inspector.
6. Use "View Slowest Traces" to investigate specific instances of poor performance.

### Comparing Agent Performance

Monitor and compare performance across different agents:

1. Switch between agents in the Agent Selection view.
2. Compare average durations and P95 latencies across agents.
3. Identify which agents or workflows need optimization.
4. Use the flame graph to understand differences in execution patterns.

### Monitoring Performance Over Time

Track performance trends and identify regressions:

1. Regularly check P95 and P99 latencies for critical spans.
2. Compare current metrics against historical baselines.
3. Investigate spikes in execution time by viewing recent traces containing affected spans.
4. Use span-level metrics to validate the impact of optimizations.


# Trace Detail View
Source: https://docs.gp.scale.com/docs/v5/tracing/tracing-ui/trace-detail-view

Inspect individual traces with comprehensive span-level details and visualizations.

## Trace Detail View

Clicking on a trace opens the detail view, which provides a comprehensive breakdown of the entire workflow.

<img alt="trace-detail-view" />

### Navigation Controls

At the top of the trace detail view, you'll find several navigation and viewing options:

* **Previous/Next Trace**: Navigate between traces in your current filtered results without returning to the main table.
* **Download Trace**: Export the complete trace data for offline analysis or sharing.
* **Gantt Chart View**: Toggle to a Gantt chart visualization to see span timing and dependencies.
* **Search**: Search within the trace for specific spans or content.

### Data Display Options

The detail view provides flexible viewing options for trace and span data:

* **Pretty/Raw Toggle**: Switch between formatted (Pretty) and raw (Raw) views of the data for easier reading or debugging.
* **Format Selection**: Toggle between JSON and YAML formats to view the data in your preferred structure.
* **Edit Mode**: Click the **Edit** button to modify span data for thread forking (available when viewing span details). See the [Thread Forking](#thread-forking) section for more details.
* **Expand View**: Open a full-screen view of the data for detailed inspection.

### Trace Visualization Graph

The trace graph displays a hierarchical visualization of all spans within the trace:

* **Hierarchical Layout**: Spans are arranged in a tree structure on the left showing parent-child relationships, with each span displaying its name and duration.
* **Timeline View**: Each span is positioned horizontally based on its start time and duration, making it easy to identify sequential vs. parallel operations.
* **Interactive Navigation**: Click on any span in the graph to view its detailed information in the side panel.

### Span Details Panel

When you select a span from the graph, the details panel on the right reveals comprehensive information:

* **Input**: The data that was passed into the operation, helping you understand what the span received.
* **Output**: The data produced by the operation, showing the result of the span's execution.
* **Initial State**: The starting state or context for the span, particularly useful for agent workflows.
* **Metadata**: Custom key-value pairs attached to the span for additional context.
* **Span Attributes**: All span properties including trace ID, parent ID, and span type.
* **Assessments**: Click the **Assessments** button to add evaluations or annotations to the span. You can create assessments to track quality, correctness, or other criteria for specific spans in your traces.

### Performance Analysis

The trace detail view helps identify performance issues:

* **Duration Comparison**: Quickly spot spans that take disproportionately long compared to siblings.
* **Critical Path**: Identify the sequence of spans that determines the overall trace duration.
* **Parallel Execution**: Visualize which operations run concurrently vs. sequentially.

## Thread Forking

<Info>Available for traces generated by SGP native applications.</Info>

For traces created by SGP's native agent system, you can trigger thread forking directly from the trace detail view. This advanced debugging feature allows you to:

* Resume execution from a specific point in the trace.
* Test different execution paths without rerunning the entire workflow.
* Debug complex decision-making logic in AI agents.

To fork a trace:

1. Select a span in the trace graph where you want to resume execution.
2. In the span details panel, click the **Edit** button to modify the input, output, or initial state.
3. Make your desired changes to test different scenarios.
4. Click **Save and Execute** to create a new execution thread from that point forward.

This creates a new trace with your modified parameters, allowing you to experiment with different inputs or states without affecting the original trace.

<img alt="forking" />

<img alt="forking-example" />

## Common Use Cases

### Debugging Individual Trace Failures

When investigating a specific failed trace:

1. Open the trace detail view from the Traces table.
2. Use the trace visualization graph to identify which span failed.
3. Inspect the failed span's input, output, and metadata to understand what went wrong.
4. Review parent spans to see what data was passed to the failed operation.
5. Check sibling spans to see if related operations also encountered issues.

### Understanding Execution Flow

To understand how a complex workflow executes:

1. Open the trace detail view for a representative trace.
2. Use the hierarchical layout to see the parent-child relationships between spans.
3. Examine the timeline view to identify sequential vs. parallel operations.
4. Review input and output for each span to understand data flow through the system.
5. Use the search function to quickly locate specific operations within the trace.

### Analyzing Slow Traces

When investigating performance issues in a specific trace:

1. Open the slow trace from the Traces table (sorted by duration).
2. Use the trace visualization to identify the critical path—the sequence of spans that determines total duration.
3. Look for spans with unusually long durations compared to typical executions.
4. Examine span metadata for context about why operations took longer than expected.
5. Use the Gantt Chart view to see timing and dependencies more clearly.

### Testing Alternative Execution Paths

For SGP native applications, use thread forking to experiment with different execution scenarios. See the [Thread Forking](#thread-forking) section above for detailed instructions on how to modify span data and create alternative execution paths.


# Traces
Source: https://docs.gp.scale.com/docs/v5/tracing/tracing-ui/traces-table

Monitor and analyze your application traces through the dedicated Traces page.

## Overview

The Traces page provides a comprehensive interface for monitoring, filtering, and analyzing trace data from your instrumented applications. This dedicated UI gives you visibility into your workflows, allowing you to identify performance bottlenecks, debug errors, and understand the execution flow of complex operations.

At the top right of the page, you'll find a date range selector that allows you to filter traces by time period. Adjusting this date range updates both the overview metrics graphs and the traces table to show only data from your selected timeframe. You can also toggle auto-refresh to automatically update the view with new traces as they arrive.

<img alt="tracing" />

## Overview Metrics

At the top of the Traces page, you'll find three interactive graphs that provide high-level insights into your trace data over time. You can switch between different views by clicking the tabs:

### Total Requests

The Total Requests view shows the volume of traces over time, giving you a quick overview of activity levels. Hover over any point in the graph to see the exact count of traces for that time period. This helps you identify traffic patterns, usage spikes, or periods of low activity.

<img alt="total-requests" />

### Latency

The Latency view displays performance metrics across your traces, showing p50 (median), p75, and p95 percentile response times. This multi-line graph helps you understand not just average performance, but also worst-case scenarios and performance consistency. Spikes in p95 latency can indicate performance issues affecting a subset of requests.

<img alt="latency" />

### Errors

The Errors view tracks the rate of failed traces over time, allowing you to quickly spot error patterns or incidents. Use this view to correlate error spikes with deployments.

<img alt="errors" />

## Traces Table

The main view displays a table of all traces collected from your applications. Each row represents a single trace (workflow), showing key information at a glance:

* **Content**: A preview of the trace's input and output data, giving you a quick glimpse of what the workflow processed.
* **Trace Name**: The descriptive name identifying the trace.
* **Timestamp**: When the trace began execution.
* **Duration**: The total time taken for the entire workflow to complete.
* **Type**: The span type for categorization.

Click on any trace row to open the detailed trace view and inspect individual spans.

<img alt="traces-table" />

### Controlling the Content Preview

The **Content** column extracts a text preview from the root span's `input` and `output` fields. The preview system checks specific [preferred keys](https://github.com/scaleapi/scaleapi/blob/31a6bbe24a3db84f3c032714d03baf2e085fdb57/packages/egp-api-backend/egp_api_backend/server/api/apps/public/routers/spans/spans_models.py#L299-L313) in priority order before falling back to the first available value.

**Input preview** priority:

`output_text`, `output-text`, `user_message`, `query`, `serp_query`, `content`, `message`, `prompt`, `title`, `value`, `result`, `text`, `raw_output`

**Output preview** priority:

`summary_text`, `content_text`, `summary`, `content` → then the same general key list above

If your input or output doesn't use one of these keys, the preview may display an unhelpful value (e.g., showing a numeric field like `age` instead of a meaningful identifier).

To control what appears in the preview, add a key from the list above to your span's `input` and/or `output` with the value you want displayed:

```python theme={null}
with tracing.create_trace(name="patient_intake"):
    with tracing.create_span(
        "process_record",
        input={
            "title": "MRN-20394",  # This will appear in the trace list preview
            "mrn": "MRN-20394",
            "age": 58,
            "department": "cardiology",
        },
    ) as span:
        result = process(span.input)
        span.output = {
            "summary": "Intake complete — MRN-20394",  # Preview for the output column
            "status": "processed",
            "risk_score": 0.72,
        }
```

<Tip>
  You don't need to remove your existing keys. Simply add a perferred key alongside your other fields — the preview system will prioritize it while the full data remains accessible in the [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view).
</Tip>

### Working with Selected Traces

You can select traces from the table using the checkboxes on the left of each trace row. Once you've selected traces, additional options become available:

* **Add to Dataset**: Create a new dataset from the selected traces for evaluations.
* **Download**: Export the selected traces for offline analysis or sharing.

<img alt="add-to-dataset" />

<img alt="dataset" />

## Searching Traces

The search bar above the Traces table lets you find traces by content. The query runs across each span's `input` and `output` fields. If the query is an exact match for a `trace_id`, the corresponding trace is returned directly — useful for jumping to a specific trace from a log line, error report, or shared link.

## Filtering Traces

Use the filter controls on the left side of the page to narrow down the traces displayed in the table. You can use the search box at the top of the filters panel to quickly find specific filter options by typing keywords.

Available filters include:

#### Duration

Filter traces by their execution time using either a min-to-max range input or an interactive slider. This helps you quickly identify slow workflows or find traces within a specific performance window.

#### Span Scope

Filter traces based on the scope of spans you want to examine. You can choose between **Root Spans** (only the top-level spans of each trace) or **All Spans** (every span within the traces).

#### Status

Filter by completion status to quickly identify successful traces or those that encountered errors during execution.

#### Application

Filter traces by the application that generated them, useful when monitoring multiple applications or services.

#### Span Name

Filter by specific span names to find traces containing particular operations or function calls.

#### Span Type

Filter by span type to view traces of a particular category, such as LLM calls, database queries, or API requests.

#### Agent Name

Filter traces generated by specific agents, helping you monitor individual agent performance and behavior.

#### Agent ID

Filter by unique agent identifiers for precise agent-level filtering.

#### ACP Type

Filter traces by ACP (Agent Control Protocol) type for agent-specific workflows.

#### Assessment

Filter traces based on assessment criteria or evaluation results.

<img alt="filters" />

## Common Use Cases

### Debugging Errors

When a workflow fails, use the Traces page to:

1. Filter by status to show only error traces.
2. Open the trace detail view to see exactly which span failed.
3. Inspect the error span's input, output, and metadata to understand the root cause.
4. Review parent spans to see what led to the failure.

### Performance Optimization

To identify performance bottlenecks:

1. Sort traces by duration to find slow workflows.
2. Open the trace graph to visualize the execution timeline.
3. Identify spans with long durations that could be optimized.
4. Look for sequential operations that could be parallelized.

### Monitoring Workflows

For ongoing monitoring of your applications:

1. Use the time range filter to view recent traces.
2. Monitor the success/error ratio in the table.
3. Set up custom metadata filters to track specific features or user segments.
4. Review trace patterns to ensure expected execution flows.


# Chat Completions
Source: https://docs.gp.scale.com/reference/v5/inference/chat-completions

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/chat/completions



# Completions
Source: https://docs.gp.scale.com/reference/v5/inference/completions

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/completions



# Generic Inference
Source: https://docs.gp.scale.com/reference/v5/inference/generic-inference

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/inference



# List Chat Completion Models
Source: https://docs.gp.scale.com/reference/v5/inference/list-chat-completion-models

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/chat/completions/models



# Responses
Source: https://docs.gp.scale.com/reference/v5/inference/responses

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/responses



# Archive Question
Source: https://docs.gp.scale.com/reference/v5/questions/archive-question

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/questions/{question_id}
Archive a question by setting archived_at in the db



# Create Question
Source: https://docs.gp.scale.com/reference/v5/questions/create-question

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/questions



# Get Question
Source: https://docs.gp.scale.com/reference/v5/questions/get-question

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/questions/{question_id}



# List Questions
Source: https://docs.gp.scale.com/reference/v5/questions/list-questions

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/questions



# Restore Archived Question
Source: https://docs.gp.scale.com/reference/v5/questions/restore-archived-question

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/questions/{question_id}/restore



# Update Question
Source: https://docs.gp.scale.com/reference/v5/questions/update-question

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/questions/{question_id}



# Add Criterion to Rubric
Source: https://docs.gp.scale.com/reference/v5/rubrics/add-criterion-to-rubric

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/rubrics/{rubric_id}/criteria



# Archive Rubric
Source: https://docs.gp.scale.com/reference/v5/rubrics/archive-rubric

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/rubrics/{rubric_id}



# Create Rubric
Source: https://docs.gp.scale.com/reference/v5/rubrics/create-rubric

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/rubrics



# Get Rubric
Source: https://docs.gp.scale.com/reference/v5/rubrics/get-rubric

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/rubrics/{rubric_id}



# List Criterion Versions
Source: https://docs.gp.scale.com/reference/v5/rubrics/list-criterion-versions

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/rubrics/{rubric_id}/criteria/{rubric_criteria_id}/versions



# List Rubrics
Source: https://docs.gp.scale.com/reference/v5/rubrics/list-rubrics

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/rubrics



# Update Criterion
Source: https://docs.gp.scale.com/reference/v5/rubrics/update-criterion

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/rubrics/{rubric_id}/criteria/{rubric_criteria_id}



# Update or Restore Rubric
Source: https://docs.gp.scale.com/reference/v5/rubrics/update-or-restore-rubric

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/rubrics/{rubric_id}



# Configure Vector Store
Source: https://docs.gp.scale.com/reference/v5/vector-stores/configure-vector-store

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/vector-stores/{vector_store_name}/configure
Update the indexed metadata fields configuration for a vector store.

Only indexed metadata fields can be used for filtering during query, list, and count operations.
Non-indexed fields cannot be filtered on.

**Field Types:** Only STRING, NUMBER, and BOOLEAN fields can be indexed (maximum 20 fields). OBJECT and LIST
types are stored but cannot be indexed for filtering purposes.

**Adding Fields:** New indexed fields can be added at any time. Existing documents containing those fields
will have their metadata automatically indexed.

**Removing Fields:** Indexed fields cannot be removed once added. Each indexed field increases write
latency and storage overhead, so only index fields you actively filter on.

**Note:** The `name` and `embedding_config` are immutable after creation.



# Create Vector Store
Source: https://docs.gp.scale.com/reference/v5/vector-stores/create-vector-store

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/vector-stores/create
Create a new vector store for storing and querying document embeddings.

The vector store name must be unique within your account and follow naming conventions (3-63 characters,
alphanumeric with hyphens/underscores). Once created, the embedding configuration and dimensions
are immutable and cannot be changed. To use a different model, you must create a new vector store.

**Embedding Configuration:** Provide `embedding_config` (for base or custom model deployments),
`embedding_model` (shorthand for a base model), or `dimensions` only (raw embeddings).
- With `embedding_config` or `embedding_model`: dimensions are auto-derived, and documents can be
  upserted with text content (auto-embedded) or with pre-computed embeddings.
- With `dimensions` only: the store accepts only pre-computed embeddings. Semantic/hybrid queries
  are not supported (lexical search only).

**Indexed Fields:** Optionally specify metadata fields to index at creation time. Only indexed fields
can be used for filtering -- indexing is required, not just a performance optimization. Additional indexed
fields can be added later using the configure endpoint, but cannot be removed once added. Keep in mind
that each indexed field increases write latency and storage overhead, so only index fields you actively filter on.



# Delete Vectors
Source: https://docs.gp.scale.com/reference/v5/vector-stores/delete-vectors

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/vector-stores/{vector_store_name}/delete
Delete documents from a vector store by document IDs or metadata filter criteria.

**Delete by IDs:** Provide an array of document IDs to delete specific documents. Non-existent documents
are silently skipped.

**Delete by Filter:** Use metadata filters to delete all documents matching the specified criteria (e.g.,
delete all documents where `status: "archived"`). The filter must specify at least one condition and cannot
be empty. To delete all documents, use the drop endpoint instead.

**Filter Operators:** Supports MongoDB-style operators including equality (`{"field": "value"}`),
comparison (`$gt`, `$gte`, `$lt`, `$lte`, `$eq`, `$ne`), logical (`$and`, `$or`, `$not`),
and membership (`$in`, `$nin`). Only indexed metadata fields can be used for filtering.

**Best Practice:** Use the count endpoint with the same filter to preview the number of documents that
will be deleted before executing the deletion operation.



# Drop Vector Store
Source: https://docs.gp.scale.com/reference/v5/vector-stores/drop-vector-store

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/vector-stores/{vector_store_name}/drop
Permanently delete a vector store and all its contents.

**⚠️ WARNING:** This is a destructive operation that cannot be undone. All documents, embeddings, metadata,
and index configurations will be permanently deleted. Data recovery is not possible after deletion.



# Get Vector Store
Source: https://docs.gp.scale.com/reference/v5/vector-stores/get-vector-store

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/vector-stores/{vector_store_name}
Retrieve detailed configuration and metadata for a specific vector store.

Returns the store's embedding model, dimensions, indexed metadata field definitions,
creation timestamp, and last update timestamp. Use this to verify store settings before
performing operations or to display store information in your application.



# List Vector Stores
Source: https://docs.gp.scale.com/reference/v5/vector-stores/list-vector-stores

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/vector-stores
List all vector stores in your account with pagination.

Returns vector stores sorted by creation date (newest first). Each store includes its configuration,
embedding model, dimensions, indexed fields, and timestamps.



# Upsert Vectors
Source: https://docs.gp.scale.com/reference/v5/vector-stores/upsert-vectors

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/vector-stores/{vector_store_name}/upsert
Insert new documents or update existing documents in a vector store.

**Upsert Behavior:** If a document ID already exists, it will be completely replaced with the new content
and metadata. The previous document's text, embedding, and all metadata fields are discarded. If the ID
does not exist, a new document is created.

**Document Content:** Each document supports several modes:
- `content` only: text is automatically embedded using the store's configured model.
- `embedding` only: pre-computed embedding vector is used directly. Dimension must match the store's configuration.
- Both `content` and `embedding`: the pre-computed embedding is stored and text is kept for retrieval/search.
- Neither (metadata-only): only metadata is updated on an existing document without re-embedding.
  If the document does not exist, it will appear as a failure in the batch response.

A store created without an embedding model (dimensions-only) only accepts documents with pre-computed `embedding`.

**Batch Operations:** This endpoint supports batch operations with partial success handling and mixed
document types (some with raw embeddings, some with content) in the same call.

**Metadata:** Supports nested metadata with string, number, boolean, object, and array types. Null values
are not permitted—omit the field or use an empty string instead.



# Create Agent Config
Source: https://docs.gp.scale.com/reference/v5/agent-configs/create-agent-config

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/agent_configs



# Delete Agent Config
Source: https://docs.gp.scale.com/reference/v5/agent-configs/delete-agent-config

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/agent_configs/{agent_config_id}



# Get Agent Config
Source: https://docs.gp.scale.com/reference/v5/agent-configs/get-agent-config

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/agent_configs/{agent_config_id}



# List Agent Configs
Source: https://docs.gp.scale.com/reference/v5/agent-configs/list-agent-configs

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/agent_configs



# List MCP Tool Names
Source: https://docs.gp.scale.com/reference/v5/agent-configs/list-mcp-tool-names

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/agent_configs/mcp_tools
AllowedTool values that route to MCP servers (vs harness-provided tools). Exposed primarily so the generated SDK includes the `McpTool` enum type — the frontend asserts its own MCP-tool classifier against this enum to prevent drift.



# Update Agent Config
Source: https://docs.gp.scale.com/reference/v5/agent-configs/update-agent-config

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/agent_configs/{agent_config_id}



# Create agent evaluation
Source: https://docs.gp.scale.com/reference/v5/agent-evaluations/create-agent-evaluation

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/agent-evaluations
Register a new two-plane agent evaluation and return its initial record.

This is the control-plane entry point for the two-plane agent-eval system and is
gated by the `SGP_TWO_PLANE_AGENT_EVALS_ENABLED` feature flag. The request is
metadata-only: it must reference data that already lives in the data plane via
`data_plane_dataset_id` and must supply at least one task and a
`projection_policy_id`. Inline raw evaluation data is rejected — if the payload
contains any forbidden raw-data field (e.g. `prompt`, `output`, `response`,
`dataset`, `rows`, `traces`, `credentials`), validation fails before the request
is forwarded. Use this instead of embedding evaluation rows in the request.

The call is proxied to the agent-eval control service; if that service is
unreachable the endpoint returns 503. On success the returned record includes a
`run_id`, a `status`, and a `progress` block (per-workflow status and item
completion counts) that reflect the run as it executes over time — poll the get
or results endpoints to observe completion rather than expecting a finished run
on this response.



# Get agent evaluation
Source: https://docs.gp.scale.com/reference/v5/agent-evaluations/get-agent-evaluation

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/agent-evaluations/{evaluation_id}
Fetch a single agent evaluation's record and live progress by id.

Returns the full evaluation record including its configured `tasks`, current
`status`, and `progress` (per-workflow status and item completion counts) — use
this to poll the run's state. For computed aggregate metrics of a finished or
in-flight run (scores, latency, error breakdown), use the
`/v5/agent-evaluations/{evaluation_id}/results` endpoint instead. Gated by the
`SGP_TWO_PLANE_AGENT_EVALS_ENABLED` feature flag and requires read access. The
request is proxied to the agent-eval control service, which returns 503 if it is
unreachable.



# Get agent evaluation results
Source: https://docs.gp.scale.com/reference/v5/agent-evaluations/get-agent-evaluation-results

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/agent-evaluations/{evaluation_id}/results
Fetch the computed result summary projection for an agent evaluation.

Returns the precomputed `evaluation-run-summary` projection for the run:
aggregate metrics (`mean_score`, `p95_latency_ms`, `error_category_counts`),
completion counts, and the projection's provenance (`policy_id`,
`source_data_plane_version`). Because these values are served from a projection
rather than computed live, the response carries freshness metadata (`synced_at`,
`freshness_seconds`) indicating how stale the summary is — use it to decide
whether to wait and re-poll. For the evaluation's configuration and raw progress
rather than computed metrics, use `GET /v5/agent-evaluations/{evaluation_id}`.
Gated by the `SGP_TWO_PLANE_AGENT_EVALS_ENABLED` feature flag and requires read
access; the request is proxied to the agent-eval control service, which returns
503 if it is unreachable.



# List agent evaluations
Source: https://docs.gp.scale.com/reference/v5/agent-evaluations/list-agent-evaluations

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/agent-evaluations
List agent evaluations for the caller's account, with cursor pagination.

Returns a page of evaluation records under `data` along with a `next_cursor`;
pass the cursor back to page through results. The request's query string is
forwarded verbatim to the agent-eval control service, so any pagination or
filtering parameters it supports are honored. Gated by the
`SGP_TWO_PLANE_AGENT_EVALS_ENABLED` feature flag and requires read access.
Returns 503 if the control service is unreachable.



# Cancel Build
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/cancel-build

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/builds/{build_id}/cancel
Cancel a pending or running build.

Returns 202 Accepted — the actual cancellation happens asynchronously
via a Temporal workflow.



# Get Build
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/get-build

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/builds/{build_id}
Get a build by ID, including current lifecycle status.



# List Builds
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/list-builds

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/builds



# List Undeployed Agents
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/list-undeployed-agents

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/builds/undeployed
List agents that exist only as cloud builds with no healthy deployment.

Returns distinct agent names from builds that have never had a deploy
reach 'healthy' status, along with each agent's latest build info and
total build count. Useful for surfacing agents that have been built but
not yet deployed.



# Stream Build Logs
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/stream-build-logs

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/builds/{build_id}/logs
Stream build logs via Server-Sent Events (SSE).

Returns a streaming response with content-type text/event-stream.
Each log line is sent as an SSE data event.



# Submit Build
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/submit-build

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/builds
Submit a container image build.

Upload a tar.gz archive containing the build context (Dockerfile and any files
needed for the build) along with image name, tag, and optional build arguments.

Maximum file size: 500MB



# Create Credential
Source: https://docs.gp.scale.com/reference/v5/credentials/create-credential

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/credentials
Create a new credential for storing sensitive data like API keys, tokens, or other secrets.



# Delete Credential
Source: https://docs.gp.scale.com/reference/v5/credentials/delete-credential

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/credentials/{credential_id}
Permanently delete a credential and all its associated data.



# Get Credential by ID
Source: https://docs.gp.scale.com/reference/v5/credentials/get-credential-by-id

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/credentials/{credential_id}
Retrieve a specific credential by its unique identifier.



# Get Credential by Name
Source: https://docs.gp.scale.com/reference/v5/credentials/get-credential-by-name

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/credentials/name/{credential_name}
Retrieve a specific credential by its name.



# Get Credential Secret by ID
Source: https://docs.gp.scale.com/reference/v5/credentials/get-credential-secret-by-id

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/credentials/{credential_id}/secret
Retrieve the plaintext payload of a credential by its ID.



# Get Credential Secret by Name
Source: https://docs.gp.scale.com/reference/v5/credentials/get-credential-secret-by-name

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/credentials/name/{credential_name}/secret
Retrieve the plaintext payload of a credential by its name.



# List Credentials
Source: https://docs.gp.scale.com/reference/v5/credentials/list-credentials

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/credentials
Retrieve a paginated list of all credentials for the current account with optional name filtering.



# Update Credential
Source: https://docs.gp.scale.com/reference/v5/credentials/update-credential

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/credentials/{credential_id}
Update an existing credential's properties including name, description, type, payload, and metadata.



# Batch Create Dataset Items
Source: https://docs.gp.scale.com/reference/v5/dataset-items/batch-create-dataset-items

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/dataset-items/batch



# Delete Dataset Item
Source: https://docs.gp.scale.com/reference/v5/dataset-items/delete-dataset-item

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/dataset-items/{dataset_item_id}



# Get Dataset Item
Source: https://docs.gp.scale.com/reference/v5/dataset-items/get-dataset-item

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/dataset-items/{dataset_item_id}



# List Dataset Items
Source: https://docs.gp.scale.com/reference/v5/dataset-items/list-dataset-items

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/dataset-items



# Update Dataset Item
Source: https://docs.gp.scale.com/reference/v5/dataset-items/update-dataset-item

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/dataset-items/{dataset_item_id}



# Create Dataset
Source: https://docs.gp.scale.com/reference/v5/datasets/create-dataset

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/datasets



# Delete Dataset
Source: https://docs.gp.scale.com/reference/v5/datasets/delete-dataset

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/datasets/{dataset_id}



# Get Dataset
Source: https://docs.gp.scale.com/reference/v5/datasets/get-dataset

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/datasets/{dataset_id}



# List Datasets
Source: https://docs.gp.scale.com/reference/v5/datasets/list-datasets

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/datasets



# Update or Restore Dataset
Source: https://docs.gp.scale.com/reference/v5/datasets/update-or-restore-dataset

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/datasets/{dataset_id}



# Create Evaluation Chart
Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/create-evaluation-chart

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/evaluation-charts
Create a new evaluation chart for an evaluation or evaluation group



# Delete Evaluation Chart
Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/delete-evaluation-chart

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/evaluation-charts/{chart_id}
Delete an evaluation chart and its columns



# Get Evaluation Chart
Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/get-evaluation-chart

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluation-charts/{chart_id}
Get a single evaluation chart by ID



# List Evaluation Charts
Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/list-evaluation-charts

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluation-charts
List charts filtered by evaluation_id or evaluation_group_id



# Patch Evaluation Chart
Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/patch-evaluation-chart

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/evaluation-charts/{chart_id}
Partially update chart metadata (title, type). Columns cannot be modified via PATCH.



# Replace Evaluation Chart
Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/replace-evaluation-chart

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json put /v5/evaluation-charts/{chart_id}
Replace chart metadata and/or columns. Columns are replaced if provided.



# Add Widget to Dashboard
Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/add-widget-to-dashboard

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/evaluation-dashboards/{dashboard_id}/widgets
Create a new widget, append it to the dashboard, and compute its result in one call.

The widget is persisted as its own entity, its ID is appended to the dashboard's
`widget_order`, and — unless it is a `markdown` or `heading` widget — its result is
computed synchronously against the dashboard's evaluation (or evaluation-group) data
before the response returns. Validation is type-specific: `markdown` requires
`config.content`, `bar`/`histogram`/`donut`/`scatter` require exactly one of
`config.x_column` or `config.x_column_group`, `table` configs with conditional
formatting are schema-validated, and all other chart types require a `query`. The
dashboard must exist and not be archived. A computation failure does not fail the
request: the widget is still created and returned with a result whose
`computation_status` is `failed` and an `error_message` set, while `markdown` and
`heading` widgets return a null result.



# Create Evaluation Dashboard
Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/create-evaluation-dashboard

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/evaluation-dashboards
Create a dashboard bound to exactly one evaluation or evaluation group.

The request must set exactly one of `evaluation_id` or `evaluation_group_id`
(enforced as XOR) — supplying both or neither is rejected, and the referenced
evaluation or group must already exist in the caller's account or the call fails
with a not-found error. If `template_dashboard_id` is provided, the new dashboard
copies that template's `widget_order` and synchronously computes a result for each
of those widgets against the new dashboard's evaluation data before returning. Any
`widget_order` supplied is validated to contain only existing, non-duplicate widget
IDs. The dashboard is created and returned immediately; individual widgets are added
afterward through the widget sub-endpoints.



# Delete Evaluation Dashboard
Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/delete-evaluation-dashboard

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/evaluation-dashboards/{dashboard_id}
Soft-delete a dashboard by setting its archived timestamp.

The dashboard row is retained and marked archived rather than physically removed, so
it stops appearing in default listings but can still be fetched with
`include_archived=true`; the archived dashboard is returned by this call. Associated
widgets and widget results are not deleted or detached. Returns a not-found error if
the dashboard does not exist in the caller's account.



# Get Evaluation Dashboard
Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/get-evaluation-dashboard

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluation-dashboards/{dashboard_id}
Fetch a single dashboard by ID within the caller's account.

By default only the dashboard's own fields are returned. Pass `views=widgets`
and/or `views=widget_results` to eagerly load the dashboard's widgets and their
last-computed results as nested relationships; without those views the `widgets`
and `widget_results` fields are omitted from the response entirely. Set
`include_archived=true` to retrieve a soft-deleted dashboard. Returns a not-found
error if no matching dashboard exists in the account.



# List Evaluation Dashboards
Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/list-evaluation-dashboards

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluation-dashboards
List dashboards in the caller's account, paginated, with optional filters.

Filter to a single evaluation with `evaluation_id` or to a group with
`evaluation_group_id`; passing both is rejected, since a dashboard is bound to one
or the other. `tags` matches case-insensitively (values are lowercased before
lookup), `created_by_ids` filters by creator identity, and `search` matches the
dashboard name and tags. Archived dashboards are excluded unless `include_archived`
is true. The returned items carry only the dashboards' own fields — widgets and
widget results are never embedded here; use the get-by-id endpoint with the
`widgets`/`widget_results` views to load those.



# Patch Evaluation Dashboard
Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/patch-evaluation-dashboard

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/evaluation-dashboards/{dashboard_id}
Partially update a dashboard's `name`, `description`, `tags`, or `widget_order`.

Only these mutable fields can change — the dashboard's bound evaluation or group is
fixed at creation and cannot be reassigned here, and unset fields are left untouched
(partial-update semantics). Reorder existing widgets by sending a new `widget_order`,
whose entries are validated to be existing, non-duplicate widget IDs before the
update is applied; creating and attaching new widgets is done through the widget
sub-endpoints, not by editing `widget_order`. Archived dashboards cannot be updated
and return a not-found error.



# Remove Widget from Dashboard
Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/remove-widget-from-dashboard

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/evaluation-dashboards/{dashboard_id}/widgets/{widget_id}
Detach a widget from this dashboard without deleting the widget itself.

This removes the widget's ID from the dashboard's `widget_order` only; the
underlying widget entity and any computed widget results are left intact, so a widget
shared with other dashboards continues to work there. The widget must currently be in
this dashboard's `widget_order`, otherwise a not-found error is returned. Responds
with 204 No Content on success.



# Update Dashboard Widget
Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/update-dashboard-widget

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/evaluation-dashboards/{dashboard_id}/widgets/{widget_id}
Update a widget's fields within this dashboard, using copy-on-write for shared widgets.

If the widget belongs only to this dashboard it is updated in place; if it is
referenced by more than one dashboard, a new widget is created with the updates
applied and swapped into this dashboard's `widget_order`, leaving the other
dashboards' copy untouched. The widget must already be in this dashboard's
`widget_order`, otherwise the call is rejected. The result is recomputed
synchronously when the `query` changes or when the widget was cloned; otherwise the
existing cached result is returned. For `table` widgets, conditional-formatting
column references are re-resolved against the (possibly updated) query on each save.
The dashboard must exist and not be archived.



# Archive Evaluation Group
Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/archive-evaluation-group

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/evaluation-groups/{group_id}
Archive (soft-delete) an evaluation group and its dependent records.

This is a soft delete: the group is marked deleted rather than removed, so it no
longer appears in default list/get results but can still be fetched with the
include-deleted flags. The same delete timestamp cascades as a soft delete to the
group's members, row identifiers, and its dashboard charts and their chart columns;
the member evaluations themselves are not affected. The archived group is returned in
the response.



# Create Evaluation Group
Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/create-evaluation-group

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/evaluation-groups
Create a new evaluation group bundling one or more existing evaluations.

The request must list at least one evaluation ID, and duplicate IDs are rejected.
Every listed evaluation is validated for existence and account access before the
group is created; an unknown or inaccessible evaluation ID fails the whole request
with a 400 rather than creating a partial group. The named evaluations are added as
group members in the same transaction, and any supplied row_identifiers (an
evaluation_id-to-column-name mapping used for cross-dataset joins) are persisted
alongside them, ignoring entries for evaluation IDs not in the group. The returned
group includes its members enriched with each evaluation's name, tags, and creation
time. Unlike the PUT and PATCH update endpoints, creation does not trigger any
dashboard-widget recomputation.



# Get Evaluation Group
Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/get-evaluation-group

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluation-groups/{group_id}
Fetch a single evaluation group by its ID.

By default only non-archived groups are returned; set include_deleted to also resolve
a soft-deleted group. The views parameter optionally expands the response with the
group's members and/or row_identifiers, and when members are loaded they are enriched
with each member evaluation's name, tags, and creation time. Use the schema endpoint
instead when you need the per-evaluation column schemas of the group's members.



# Get Evaluation Group Schema
Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/get-evaluation-group-schema

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluation-groups/{group_id}/schema
Return a separate column schema for each active member evaluation of the group.

Rather than a single merged schema, the response holds one schema entry per active
member evaluation (each with its field list, total item count, and sampling info),
which lets a caller filter columns down to a chosen subset of the group's evaluations.
Schemas are computed from the member evaluations' items; include_archived controls
whether archived items are counted in that analysis. A group with no active members
returns an empty schema list. This differs from the plain get endpoint, which returns
group metadata and members but not their column schemas.



# List Evaluation Groups
Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/list-evaluation-groups

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluation-groups
List the calling account's evaluation groups with optional filters.

Results are paginated and scoped to the authenticated account. The optional filters
combine with AND semantics: name does a case-insensitive partial (substring) match on
the group name, tags returns only groups whose tag list contains all of the supplied
tags, and evaluation_id returns only groups that have the given evaluation as an
active member. By default archived (soft-deleted) groups are excluded; set
include_deleted to include them. The views parameter optionally expands each group
with its members and/or row_identifiers, and returned members are enriched with each
evaluation's name, tags, and creation time.



# Patch Evaluation Group
Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/patch-evaluation-group

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/evaluation-groups/{group_id}
Partially update an evaluation group's attributes.

Only the fields present in the request are changed; unset fields are left untouched.
Unlike the PUT endpoint, metadata is merged into the existing metadata key-by-key
rather than replaced wholesale, and group membership (evaluation_ids) cannot be
changed here -- use PUT to replace members. At least one field must be supplied or the
request fails with a 400. Supplying row_identifiers replaces the group's entire row
identifier set (an evaluation_id-to-column-name mapping for cross-dataset joins);
passing an empty mapping clears it.



# Update Evaluation Group
Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/update-evaluation-group

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json put /v5/evaluation-groups/{group_id}
Replace an evaluation group's attributes and, optionally, its membership.

This is a full replacement of the group's attributes: every attribute field (name,
description, tags, metadata) is written from the request, so omitting an optional
field clears it rather than leaving it unchanged. Use PATCH instead to update only
selected fields and to merge (rather than overwrite) metadata. Membership is replaced
only when evaluation_ids is supplied: the resulting members become exactly that set
(added/removed by diff against the current members), it must contain at least one ID
with no duplicates, and every ID is validated for existence and account access or the
request fails with a 400. When membership actually changes, a best-effort Temporal
workflow is started to recompute the group's dashboard widgets; this is fire-and-forget
and its failure does not fail the request. Removing members hard-deletes the
corresponding chart columns and any charts left with no columns.



# Export evaluation items to a file
Source: https://docs.gp.scale.com/reference/v5/evaluation-items/export-evaluation-items-to-a-file

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/evaluation-items/export
Export all evaluation items for a single evaluation as a downloadable file.

The evaluation is specified by `evaluation_id` in the request body. `export_format`
selects CSV, JSON, or JSONL; for CSV the per-item `data` and `files` fields are
flattened into individual columns and metric-like result columns are expanded.
`export_method` controls delivery: `direct` returns the file contents inline in the
response, while `signed_url` uploads the file to object storage and returns a
pre-signed download URL. Requesting `signed_url` in an environment where object
storage is not configured fails with a 501, so use `direct` there instead. Set
`include_archived=true` to include archived items in the export. This endpoint reads
items only and does not modify the evaluation.



# Get a single evaluation item
Source: https://docs.gp.scale.com/reference/v5/evaluation-items/get-a-single-evaluation-item

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluation-items/{evaluation_item_id}
Retrieve a single evaluation item by its ID within the caller's account.

By default only non-archived items are returned; pass `include_archived=true` to
also retrieve an item that has been archived. The response merges the item's cached
task results into its `data` field and exposes a `task_errors` map keyed by task
alias, so a task that failed on this item surfaces as an entry there rather than as a
request error. Use this to inspect one item's input data and per-task results; to page
through many items, use the list endpoint instead. The request fails if no item with
the given ID exists in the caller's account.



# List and filter evaluation items
Source: https://docs.gp.scale.com/reference/v5/evaluation-items/list-and-filter-evaluation-items

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluation-items
Return a paginated list of evaluation items belonging to the caller's account.

Pass `evaluation_id` to restrict the results to a single evaluation's items. The
`completion_status` filter selects items by whether any of their tasks errored:
`failed` returns only items that have task errors, `passed` returns only items with no
task errors, and `all` (or omitting the parameter) returns every item. Archived items
are excluded unless `include_archived=true`. Each returned item has its cached task
results merged into `data` and its errors exposed in `task_errors`, identically to the
single-item endpoint.



# Add Test Criteria to Evaluation
Source: https://docs.gp.scale.com/reference/v5/evaluations/add-test-criteria-to-evaluation

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/evaluations/{evaluation_id}/tasks
Add a new test criteria to an existing evaluation.

Narrowed to contributor question tasks (`contributor_evaluation.question`); other task types
must be configured when the evaluation is first created and are rejected here. The request is
also rejected if the evaluation is archived, if a test criteria with the same alias already
exists, or if any contributor annotation task for the evaluation has already been claimed or
completed. Because only contributor question tasks are accepted, the added criteria is applied
synchronously and contributors answer it against the evaluation's existing items — no async job
or Temporal workflow is started.



# Archive Evaluation
Source: https://docs.gp.scale.com/reference/v5/evaluations/archive-evaluation

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/evaluations/{evaluation_id}
Archive (soft-delete) an evaluation.

Sets the evaluation's archived timestamp rather than permanently deleting it, and cascades the
archive to the evaluation's items and dashboards while removing it from any evaluation groups.
When two-plane evaluations are enabled the cascade only archives dashboards and removes group
membership — the evaluation's items are not archived. The evaluation can later be brought back
with a restore request to the update endpoint.



# Create Evaluation
Source: https://docs.gp.scale.com/reference/v5/evaluations/create-evaluation

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/evaluations
Create an evaluation together with its items, optionally running test criteria against them.

Accepts three request shapes: standalone (inline `data`), from an existing dataset
(`dataset_id` with optional per-item references), or with a new reusable dataset created inline
from `data`. When the evaluation includes tasks that require execution (for example an LLM judge
or custom function), an async job and a Temporal workflow are started and the evaluation is
returned immediately with status `running`; task results and `error_count` populate
asynchronously. When it includes only contributor tasks, taxonomy-only input, or no tasks, no
workflow runs and it is returned with status `completed`. Optional `tasks`, `metadata`, `tags`,
and `taxonomy_params` are persisted alongside the evaluation and its items.



# Filter Evaluations
Source: https://docs.gp.scale.com/reference/v5/evaluations/filter-evaluations

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/evaluations/filter
Filter evaluations by metadata, status, and tags.

Accepts up to 10 filters combined with AND logic, each comparing a key against a value with an
operator (`==`, `!=`, `>=`, `<=`, `IN`, `NOT_IN`). Filter on metadata keys returned by the
metadata-keys endpoint, plus the built-in `status` and `tag` keys. Archived evaluations are
excluded unless `include_archived` is set, and the `tasks` view includes task configurations in
each result. Use this for metadata or status filtering; for simple name or tag lookups the list
endpoint is sufficient.



# Get Available Evaluation Statuses
Source: https://docs.gp.scale.com/reference/v5/evaluations/get-available-evaluation-statuses

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluations/statuses
List the possible evaluation status values.

Returns every value of the evaluation status enumeration, suitable for populating a status
filter in clients.



# Get Available Metadata Keys
Source: https://docs.gp.scale.com/reference/v5/evaluations/get-available-metadata-keys

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluations/metadata-keys
List the metadata keys available for filtering evaluations.

Returns the distinct metadata keys present on the account's evaluations, plus the built-in
`status` and `tag` keys, sorted alphabetically. Use the returned keys to build requests to the
evaluations filter endpoint.



# Get Evaluation
Source: https://docs.gp.scale.com/reference/v5/evaluations/get-evaluation

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluations/{evaluation_id}
Retrieve a single evaluation by ID.

Returns the evaluation with its datasets, async-job progress, metadata, and task-error count.
Archived evaluations are excluded unless `include_archived` is set. Pass the `tasks` view to
include the evaluation's task configurations in the response.



# Get Evaluation Data Schema
Source: https://docs.gp.scale.com/reference/v5/evaluations/get-evaluation-data-schema

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluations/{evaluation_id}/schema
Describe the data schema of an evaluation's items.

Inspects the item `data` and task-result fields and returns each discovered field with its
flattened key path, JSON type, source, and the number of items containing it, ordered
alphabetically by field name. For large evaluations the schema may be inferred from a sample of
items, in which case `is_sampled` is set and `sample_size` reports how many were analyzed. Set
`include_archived` to include archived items in the analysis.



# Get Evaluation Taxonomy
Source: https://docs.gp.scale.com/reference/v5/evaluations/get-evaluation-taxonomy

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluations/{evaluation_id}/taxonomy
Get the taxonomy JSON for an evaluation's contributor question tasks.

Returns the raw taxonomy document stored for the evaluation. Responds with a not-found error if
the evaluation has no taxonomy.



# List Evaluations
Source: https://docs.gp.scale.com/reference/v5/evaluations/list-evaluations

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/evaluations
List evaluations for the account, with pagination.

Supports filtering by case-insensitive name substring and by tags; archived
evaluations are excluded unless `include_archived` is set. Pass the `tasks` view to include each
evaluation's task configurations in the response. Use this for simple name or tag lookups;
to filter on metadata key-value pairs or status, use the filter endpoint instead.



# Update or Restore Evaluation
Source: https://docs.gp.scale.com/reference/v5/evaluations/update-or-restore-evaluation

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/evaluations/{evaluation_id}
Update an evaluation's mutable fields, or restore it from the archive.

The action is selected by the request body: a restore request un-archives the evaluation and
cascades the restore to its items and dashboards, while any other body applies a partial update
to fields such as name, description, tags, and metadata (metadata is applied as an RFC 7396
merge patch). Updating an already-archived evaluation is rejected — restore it first. The
evaluation row is locked for the duration of the write to avoid concurrent-update races.



# Update Test Criteria Configuration
Source: https://docs.gp.scale.com/reference/v5/evaluations/update-test-criteria-configuration

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/evaluations/{evaluation_id}/tasks/{alias}
Replace the full configuration of a single test criteria, identified by its alias.

The alias must match an existing test criteria on the evaluation, and the replacement
configuration is validated against the evaluation's current items before being applied. The
request is rejected if the evaluation is archived, if no test criteria matches the alias, or if
any contributor annotation task for the evaluation has already been claimed or completed — at
that point labelers are in-flight and mutating the task definition would corrupt their work.



# Delete File
Source: https://docs.gp.scale.com/reference/v5/files/delete-file

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/files/{file_id}



# Get File
Source: https://docs.gp.scale.com/reference/v5/files/get-file

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/files/{file_id}



# Get File Content
Source: https://docs.gp.scale.com/reference/v5/files/get-file-content

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/files/{file_id}/content



# Import files from existing blob storage in cloud (batch operation)
Source: https://docs.gp.scale.com/reference/v5/files/import-files-from-existing-blob-storage-in-cloud-batch-operation

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/files/cloud_imports



# List Files
Source: https://docs.gp.scale.com/reference/v5/files/list-files

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/files



# Update File
Source: https://docs.gp.scale.com/reference/v5/files/update-file

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/files/{file_id}



# Upload File
Source: https://docs.gp.scale.com/reference/v5/files/upload-file

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/files



# Create Model
Source: https://docs.gp.scale.com/reference/v5/models/create-model

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/models



# Delete Model
Source: https://docs.gp.scale.com/reference/v5/models/delete-model

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/models/{model_id}



# Get Model
Source: https://docs.gp.scale.com/reference/v5/models/get-model

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/models/{model_id}



# List Models
Source: https://docs.gp.scale.com/reference/v5/models/list-models

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/models



# Update Model
Source: https://docs.gp.scale.com/reference/v5/models/update-model

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/models/{model_id}



# Create Span Assessment
Source: https://docs.gp.scale.com/reference/v5/span-assessment/create-span-assessment

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/span-assessments
Create new assessment for a span (comment, rating, approval, rubric, overwrite, or metadata)



# Delete Span Assessment
Source: https://docs.gp.scale.com/reference/v5/span-assessment/delete-span-assessment

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/span-assessments/{span_assessment_id}
Delete assessment



# Get Span Assessment
Source: https://docs.gp.scale.com/reference/v5/span-assessment/get-span-assessment

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/span-assessments/{span_assessment_id}
Get an assessment by ID



# List Span Assessments
Source: https://docs.gp.scale.com/reference/v5/span-assessment/list-span-assessments

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/span-assessments
Get all assessments for a specific span or trace, optionally filtered by assessment type



# Update Span Assessment
Source: https://docs.gp.scale.com/reference/v5/span-assessment/update-span-assessment

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/span-assessments/{span_assessment_id}
Update existing assessment (only by the original creator)



# Create Span
Source: https://docs.gp.scale.com/reference/v5/spans/create-span

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/spans



# Create Spans in Batch
Source: https://docs.gp.scale.com/reference/v5/spans/create-spans-in-batch

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/spans/batch



# Export spans
Source: https://docs.gp.scale.com/reference/v5/spans/export-spans

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/spans/export



# Get aggregated metrics grouped by span name
Source: https://docs.gp.scale.com/reference/v5/spans/get-aggregated-metrics-grouped-by-span-name

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/spans/metrics/by-span



# Get bucketed metrics for spans
Source: https://docs.gp.scale.com/reference/v5/spans/get-bucketed-metrics-for-spans

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/spans/metrics



# Get Span
Source: https://docs.gp.scale.com/reference/v5/spans/get-span

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/spans/{span_id}



# Search and list spans
Source: https://docs.gp.scale.com/reference/v5/spans/search-and-list-spans

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/spans/search



# Update Span
Source: https://docs.gp.scale.com/reference/v5/spans/update-span

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/spans/{span_id}



# Upsert Spans in Batch
Source: https://docs.gp.scale.com/reference/v5/spans/upsert-spans-in-batch

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json put /v5/spans/batch



# Count Vectors
Source: https://docs.gp.scale.com/reference/v5/vector-stores/count-vectors

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/vector-stores/{vector_store_name}/count
Count documents in a vector store, optionally filtered by metadata.

**Use Cases:**
- Monitor vector store size and growth over time
- Preview the number of documents matching a filter before deletion
- Validate data ingestion by comparing expected versus actual document counts
- Analyze document distribution across metadata categories

**Filtering:** Apply the same metadata filter syntax as delete and list operations. Only indexed fields
can be used for filtering. An empty filter counts all documents in the store.



# Get Vector
Source: https://docs.gp.scale.com/reference/v5/vector-stores/get-vector

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/vector-stores/{vector_store_name}/vectors/{vector_id}
Retrieve a single document by its unique ID.

Returns the document's full content, metadata, and optionally its embedding vector. Use this endpoint
for direct lookups when the exact document ID is known. For content similarity search,
use the query endpoint.



# List Vectors
Source: https://docs.gp.scale.com/reference/v5/vector-stores/list-vectors

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/vector-stores/{vector_store_name}/vectors
List documents in a vector store with cursor-based pagination.

**Use Cases:** Browse documents, export content, audit stored data, or retrieve documents by metadata
without semantic search.

**Ordering:** Documents are returned in storage order (insertion order), not ranked by similarity.
For similarity-based retrieval, use the query endpoint.

**Filtering:** Apply metadata filters to narrow results to specific subsets (e.g., all documents
where `category: "research"`). Only indexed fields can be used for filtering.

**Pagination:** Uses cursor-based pagination for efficient traversal of large datasets. Pass the
`next_cursor` from each response as `starting_after` to retrieve the next page, or `prev_cursor`
as `ending_before` to retrieve the previous page. A null cursor indicates no further pages exist.

**Embedding Vectors:** Setting `include_vectors=true` includes the full embedding vector arrays in
the response. This significantly increases payload size and reduces the maximum page size from 1000 to
100 documents. Enable only when raw vectors are required for external processing.



# Query Vectors
Source: https://docs.gp.scale.com/reference/v5/vector-stores/query-vectors

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/vector-stores/{vector_store_name}/query
Query documents using similarity search with optional reranking.

Primary endpoint for semantic search, question-answering, and RAG (Retrieval-Augmented Generation)
applications. Returns documents ranked by relevance to the query text with similarity scores.

**Query Types:**
- `semantic` (default): Approximate nearest-neighbor search using HNSW over cosine similarity of document embeddings. Optimal for question-answering, conceptual search,
  and finding semantically related content without requiring exact keyword matches.
- `lexical`: Keyword-based text search (BM25 algorithm). Optimal for exact phrase matching, proper nouns,
  and scenarios where keyword presence is more important than semantic similarity.
- `hybrid`: Combines semantic and lexical approaches with weighted scoring. Provides maximum recall by
  identifying documents matching either semantically or lexically.

**Metadata Filtering:** Narrow the search scope by applying metadata filters (e.g., search only documents
where `category: "technical"`). Only indexed fields can be used for filtering.
Filters are applied before similarity search for optimal efficiency.

**Reranking (Advanced):** Optionally enhance result quality using a cross-encoder reranking model.
The reranker rescores the initial results using a more sophisticated model that evaluates the complete
query-document pair (not solely embeddings). This adds 100-500ms latency but significantly improves
precision for high-stakes applications.

**Reranking Strategy:** Set `top_k` higher than the desired final count (e.g., 50) to retrieve more
candidates from the initial search. Then configure `rerank_top_n` to the desired final count (e.g., 10)
to return only the most relevant documents after reranking. This two-stage approach maximizes both recall
and precision.

**Performance Metrics:** The response includes detailed timing breakdowns (embedding generation time,
index query time, reranking time) to facilitate search pipeline optimization and latency analysis.

**Similarity Scores:** Each result includes a `score` field indicating relevance. Higher scores indicate
greater relevance. Score ranges and semantics vary by query type (semantic scores use cosine similarity,
lexical scores use BM25, hybrid scores combine both approaches).



# Capabilities Changelog
Source: https://docs.gp.scale.com/docs/capabilities/changelog



This page tracks updates and additions to Scale's Capabilities documentation.

***

## Latest Updates

### How to run workflows using a Script

**Added:** May 2026

New tutorial for starting saved Workflows runs from your own code using the Compass HTTP API—authentication, run-and-wait, session polling, workflow variables, and structured error handling.

#### New documentation page

**[How to run workflows using a Script](/docs/capabilities/workflows/run-workflows-using-a-script)**

* Call `POST /v1/workflow/run` and retrieve results with `GET /v1/sessions/:sessionId/result`
* Optional polling with `GET /v1/sessions/:sessionId` when you do not want a long-lived request
* Pass string variables that override workflow defaults and populate `{{variable_name}}` placeholders in cards
* Reference for session and card status values and common HTTP error responses

***

### Dex Documentation Refresh

**Updated:** April 2026

Refreshed the Dex documentation to match the current async-only SDK and the latest document-understanding doc structure.

#### What's Updated

* Removed stale `DexSyncClient` guidance from Dex docs
* Corrected SDK method names and examples to match the current Python SDK
* Updated runtime guidance to Python 3.10+ and current 0.4.x SDK usage
* Clarified wrapper-object access patterns and placed `get_download_url()` under `DexFile` in the SDK reference

#### Updated Pages

* [Getting Started with Dex](/docs/capabilities/document-understanding/getting-started-with-dex)
* [Dex SDK API Reference](/docs/capabilities/document-understanding/dex-sdk-api-reference)
* [Best Practices](/docs/capabilities/document-understanding/best-practices)
* [Introduction to Dex](/docs/capabilities/document-understanding/introduction-to-dex)

***

### Document Understanding Workflows

**Added:** April 2026

Workflows now supports document parsing & extraction using Dex directly within workflows. This integration enables you to process unstructured documents, extract structured data, and use the results in downstream workflow operations.

#### What's New

**New Documentation Page:**

**[Guide to Document Understanding with Dex in Workflows](/docs/capabilities/workflows/dex-parsing-in-workflows)**

* Step-by-step tutorial for building document understanding in workflows
* Connect to cloud file storage (Amazon S3, Azure Blob, Google Cloud Storage)
* Filter and select documents for processing
* Parse documents with configurable Dex engines (Reducto)
* Automatically create and manage Dex projects
* Extract structured data from parsed documents
* Push parsed data to Knowledge Base for semantic search and RAG
* Use output data in subsequent workflow cards

#### Key Features

**Dex Card:**

* **Project Management**: Automatically create Dex projects or reuse existing ones
* **Multiple Parse Engines**: Support for Reducto and other parsing engines
* **Flexible Configuration**: Configurable chunking methods and chunk sizes
* **Structured Extraction**: Pulls targeted data from unstructured documents with citation & confidence scores
* **Knowledge Base Integration**: Push parsed results to vector stores for semantic search
* **Output Control**: Choose between detailed parsed content or metadata-only outputs

**Data Connector for Cloud File Storages:**

* Connect to Amazon S3, Azure Blob Storage, and Google Cloud Storage
* Browse and select files from registered cloud storage connections

**Workflow Integration:**

* Access parsed content in downstream cards using `{{parsedContent}}` variable
* Call agents to analyze, categorize, or transform parsed documents
* Export parsed data to SGP Datasets or Evaluations or SGP Knowledge Base
* Monitor parsing status and track results

#### Some Use Cases

* **Resume Parsing**: Extract structured data from resumes and categorize candidates
* **Document Analysis**: Parse contracts, invoices, or reports and extract key information
* **Content Extraction**: Convert unstructured documents into structured datasets for agent training or analysis
* **Semantic Search**: Build Knowledge Base indexes from document collections

***

### Automated Evaluation Workflows

**Added:** January 2026

Workflows is an AI-powered low-code agentic workflow orchestrator that enables enterprise operators to import data, transform it, call LLMs or Agents, analyze or evaluate the output, and automate the entire workflow—all in one interface.

#### What's New

**Two updated documentation pages:**

1. **Updated: [Introduction to Workflows](/docs/capabilities/workflows/introduction-to-workflows)**
   * Key features and available card types
   * Scheduling and monitoring workflows
   * Roadmap for H1 2026

2. **New: [Guide to Automating Evals on Workflows](/docs/capabilities/workflows/evals-in-workflows)**
   * Step-by-step tutorial for building automated evaluation workflows
   * Connecting to data sources (Snowflake, Postgres, MongoDB, MS SQL)
   * Calling Agentex agents for completions
   * Joining ground truth datasets from other workflows
   * Configuring LLM as Judge evaluations
   * Exporting results to Evaluations and Dashboards
   * Scheduling automated evaluation runs

#### Updates to Key Features

**Data Source Connections:** Query data directly from sources like `Snowflake`, `Postgres`, `MongoDB`, or `MS SQL`

**Workflow Building Cards:**

* `Call Agent`: Generate outputs from your Agentex agents with configurable timeout, retries, and response caching
* `Join Workflow`: Combine datasets from multiple workflows with Left, Right, Inner, or Outer joins
* `LLM as Judge`: Evaluate agent outputs with customizable rubrics and judge configurations

**SGP Integrations:**

* Access sources configured in SGP `Data Sources` tab in your workflows
* Export datasets directly to SGP `Datasets` tab
* Export evaluation results directly to SGP `Evaluations` tab

**Notifications for automations:** Receive Slack notifications for failed or passed runs

#### How to see the features

Workflows is available as the `Workflows` tab in SGP. Navigate to **SGP > Workflows > New Workflow**. Contact the Scale team if you don't see it on your SGP instance.

<Note>Workflows instances deployed in customer VPCs may have a limited set of available cards. Contact the Workflows team to activate or disable cards based on your requirements.</Note>

***

### Dex SDK: Version 0.4.0

**Released:** February 2025

Major improvements to the Dex SDK introducing unified entity operations, pagination, filtering, sorting, and enhanced async job support with SGP tracing integration.

#### 🔄 Unified Entity Operations

**New Feature**: Consistent `get` and `list` operations are now available for all entities within a project.

**What's New:**

* **Unified API**: All entities (files, jobs, parse\_results, extractions, vector\_stores) now support standardized `get` and `list` operations
* **Project-scoped operations**: Access all entity types through your project instance
* **Consistent patterns**: Same approach works across all entity types

**Example:**

```python theme={null}
# List operations available for all entities
files = await project.list_files()
jobs = await project.list_jobs()
parse_results = await project.list_parse_results()
extractions = await project.list_extractions()
vector_stores = await project.list_vector_stores()
```

#### 📄 Pagination, Filtering, and Sorting

**New Feature**: Token-based pagination with filtering and sorting for improved performance and discoverability.

**What's New:**

* **Token-based pagination**: Set page size and use continuation tokens to navigate through results
* **Entity-specific filters**: Available for all entities with dedicated filter types
* **Flexible sorting**: Sort results by any field in ascending or descending order
* **Performance improvements**: Process large datasets more efficiently

**Pagination:**

```python theme={null}
from dex_sdk.types import PaginationParams

pagination_params = PaginationParams(
    page_size=10,
    sort_by="created_at",
    sort_order="desc",
    continuation_token=None
)

files = await project.list_files(pagination_params=pagination_params)
```

**Filtering:**

```python theme={null}
from dex_sdk.types import FileListFilter, JobListFilter, ParseResultListFilter, ExtractionListFilter, VectorStoreListFilter
from datetime import datetime, timedelta

# Filter for entities created in the last day
file_filter = FileListFilter(created_at_start=datetime.now() - timedelta(days=1))
job_filter = JobListFilter(created_at_start=datetime.now() - timedelta(days=1))
parse_result_filter = ParseResultListFilter(created_at_start=datetime.now() - timedelta(days=1))
extraction_filter = ExtractionListFilter(created_at_start=datetime.now() - timedelta(days=1))
vector_store_filter = VectorStoreListFilter(created_at_start=datetime.now() - timedelta(days=1))

# Apply filters
files = await project.list_files(pagination_params=pagination_params, filter=file_filter)
jobs = await project.list_jobs(pagination_params=pagination_params, filter=job_filter)
parse_results = await project.list_parse_results(pagination_params=pagination_params, filter=parse_result_filter)
extractions = await project.list_extractions(pagination_params=pagination_params, filter=extraction_filter)
vector_stores = await project.list_vector_stores(pagination_params=pagination_params, filter=vector_store_filter)
```

#### 🔧 Enhanced Type Support

**Improved**: Better type safety with conditional return types based on pagination parameters.

**What's New:**

* **Type overrides**: SDK provides different types for paginated vs non-paginated responses
* **Backward compatibility**: Existing code continues to work without changes
* **Better IDE support**: Enhanced autocomplete and type checking

#### ⚡ Async Job Improvements

**New Feature**: Enhanced async job support with SGP tracing integration for better monitoring and debugging.

**What's New:**

* **Job monitoring**: Easily track job status and progress
* **Throttling support**: Control job throughput to match your needs
* **SGP trace integration**: Jobs are now connected to SGP traces for end-to-end observability
* **Trace retrieval**: Fetch complete trace data for any job

**Example:**

```python theme={null}
from dex_sdk.types import ReductoParseJobParams, ReductoParseEngineOptions, ReductoChunkingOptions, ReductoChunkingMethod, ParseEngine, JobStatus
import asyncio

# Start a parse job
parse_job = await project.start_parse_job(
    dex_file=dex_file,
    parameters=ReductoParseJobParams(
        engine=ParseEngine.REDUCTO,
        options=ReductoParseEngineOptions(
            chunking=ReductoChunkingOptions(
                chunk_mode=ReductoChunkingMethod.VARIABLE,
            )
        ),
    ),
)

# Monitor job progress
while parse_job.data.status != JobStatus.SUCCEEDED and parse_job.data.status != JobStatus.FAILED:
    await asyncio.sleep(1)
    await parse_job.refresh()

# Get the result
parse_result = await parse_job.get_result()

# Retrieve SGP traces for the job
from scale_gp_beta import SGPClient

sgp_client = SGPClient(...)
spans = [span for span in sgp_client.spans.search(
    sort_by="created_at",
    sort_order="desc",
    extra_metadata={"job_id": "job_parse_eb62b57e8c4940028b82c559903ed003"},
    parents_only=True,
)]

trace_id = spans[0].trace_id
spans = [span for span in sgp_client.spans.search(trace_ids=[trace_id])]
```

**Benefits:**

* **Better observability**: Track your jobs through SGP's tracing infrastructure
* **Easier debugging**: Access detailed execution traces for failed jobs
* **Performance monitoring**: Analyze job performance and identify bottlenecks
* **Request correlation**: Connect job execution to API requests and traces

#### How to Update

**1. Install/Update SDK:**

Use your configured CodeArtifact credentials to upgrade the Dex SDK to version 0.4.0 or higher.

**2. Explore New Features:**

* Add pagination to large list operations for better performance
* Use filters to find specific entities by creation time or other criteria
* Integrate SGP tracing to monitor your async jobs
* Leverage enhanced type support for better IDE experience

**3. Backward Compatibility:**

Version 0.4.0 is fully backward compatible. Your existing code will continue to work without modifications.

***

### Dex Document Understanding: Version 0.3.2

**Released:** November 2025

Significant updates to the Dex document understanding service and SDK, including authentication changes, data retention policies, and comprehensive documentation improvements.

#### 🔐 Authentication Changes

**Breaking Change**: The authentication method for Dex has been updated to improve security and simplify the API.

**What Changed:**

* **SGP credentials are now passed directly to `DexClient`** instead of being stored in project configurations
* Every API request is now authenticated using your SGP API key and account ID
* The `ProjectCredentials` and `SGPCredentials` types are deprecated and will be removed in a future version

**Migration Required:**

Old way (deprecated):

```python theme={null}
from dex_sdk import DexClient
from dex_sdk.types import ProjectCredentials, SGPCredentials

dex_client = DexClient(base_url="https://dex.sgp.scale.com")

project = await dex_client.create_project(
    name="My Project",
    credentials=ProjectCredentials(
        sgp=SGPCredentials(
            account_id=os.getenv("SGP_ACCOUNT_ID"),
            api_key=os.getenv("SGP_API_KEY"),
        ),
    ),
)
```

New way (version 0.3.2+):

```python theme={null}
import os
from dex_sdk import DexClient

# Pass credentials when creating the client
dex_client = DexClient(
    base_url="https://dex.sgp.scale.com",
    api_key=os.getenv("SGP_API_KEY"),
    account_id=os.getenv("SGP_ACCOUNT_ID"),
)

# Create project without credentials parameter
project = await dex_client.create_project(
    name="My Project",
)
```

**Benefits:**

* **Enhanced Security**: SGP credentials are no longer stored in the Dex database
* **Simpler API**: Credentials are set once at client initialization
* **Consistent Authentication**: Every request is authenticated the same way

#### 🗄️ Data Retention Policies

**New Feature**: Dex now supports configurable data retention policies for automatic lifecycle management of files and processing artifacts.

**What's New:**

* **Automatic cleanup**: Set retention periods for files and result artifacts to automatically delete data after a specified time
* **Flexible configuration**: Configure different retention periods for files vs. processing artifacts
* **Project-level control**: Retention policies are configured per project and can be updated at any time

**Usage:**

Configure retention when creating or updating a project:

```python theme={null}
from datetime import timedelta
from dex_sdk.types import ProjectConfiguration, RetentionPolicy

project = await dex_client.create_project(
    name="My Project",
    configuration=ProjectConfiguration(
        retention=RetentionPolicy(
            files=timedelta(days=30),           # Files expire after 30 days
            result_artifacts=timedelta(days=7),  # Parse/extract results expire after 7 days
        )
    )
)
```

**Use Cases:**

* **Compliance**: Meet regulatory requirements (GDPR, HIPAA) by enforcing data retention limits
* **Cost Management**: Reduce storage costs by automatically cleaning up old files and artifacts
* **Security**: Minimize data exposure by limiting how long sensitive documents are stored

See the [Getting Started with Dex](/docs/capabilities/document-understanding/getting-started-with-dex#configuring-data-retention) guide for detailed examples.

#### 📚 Documentation Improvements

**Major Update**: Comprehensive expansion of the Dex SDK documentation with detailed type information and practical examples.

**API Reference Enhancements:**

* **Common Types Section**: Added comprehensive documentation for all core data models organized by category:
  * **Project Types**: `ProjectEntity`, `ProjectStatus`, `ProjectConfiguration`, `RetentionPolicy`
  * **File Types**: `FileEntity`, `FileStatus`, `FileDownloadURL`
  * **Job Types**: `JobEntity`, `JobOperationType`, `JobStatus`
  * **Parse Types**: `ParseResultEntity`, `ParseResultMetadata`, `ParseChunk`, `ParseBlock`, `BoundingBox`, `ParseEngine`, `ReductoChunkingMethod`
  * **Extraction Types**: `ExtractionEntity`, `ExtractionResult`, `ExtractedField`, `ExtractionCitation`, `ExtractionParameters`, `UsageInfo`
  * **Vector Store Types**: `VectorStoreEntity`, `VectorStoreEngines`, `VectorStoreChunk`, `SearchConfig`
* **Parse Job Parameters**: New section documenting parsing configuration options

**Enhanced Examples:**

* **Working with Parse Results**: How to access parse metadata, iterate through chunks and blocks, access bounding boxes and confidence scores
* **Working with Extraction Results**: How to access extracted field values, citations, confidence scores, and token usage information

**Updated Documentation:**

* **Introduction**: Updated File Management section to mention data retention policies
* **Getting Started Guide**: Added "Configuring Data Retention" section with practical examples
* **API Reference**: Enhanced type documentation and examples

#### 🔧 Type System Updates

**ParseResult Naming Changes:**

The ParseResult classes have been harmonized for consistency:

* **Request objects** now end with `*Request` (e.g., `CustomParseResultRequest`)
* **Entity objects** now end with `*Entity` (e.g., `ParseResultEntity`)

Migration:

```python theme={null}
# Old import (deprecated)
from dex_sdk.types import CustomParseResult

# New import
from dex_sdk.types import CustomParseResultRequest
```

**Improved Type Safety:**

* Fixed typing inconsistencies for entities returned by the DEX API
* Better type hints for all SDK methods
* Enhanced autocomplete support in IDEs

#### How to Update

**1. Install/Update SDK:**

Use your configured CodeArtifact credentials to upgrade the Dex SDK to version 0.3.2 or higher.

**2. Update Your Code:**

1. Add `api_key` and `account_id` parameters to `DexClient()` initialization
2. Remove `credentials` parameter from `create_project()` calls
3. Remove imports of `ProjectCredentials` and `SGPCredentials`

**3. Test Your Integration:**

Run your scripts to ensure they work with the new authentication method.

**MCP Integration Status:**

Authentication via Model Context Protocol (MCP) is still in progress. If you encounter issues, please reach out to the Dex team at `#dex-help`.

***

### New Capability: IRIS OCR

**Added:** October 2024

IRIS is Scale's OCR capability that transforms document images and PDFs into structured text through an intelligent multi-stage pipeline.

#### What's New

**Two new documentation pages:**

1. **[Introduction to IRIS](/docs/capabilities/ocr/introduction-to-iris)**
   * Overview of IRIS OCR capability
   * Three-stage pipeline architecture (layout detection, OCR processing, assembly)
   * 15+ supported OCR models including open-source and vision-language models
   * Multi-language support with specialized Arabic models
   * Common use cases and key advantages

2. **[Getting Started with IRIS](/docs/capabilities/ocr/getting-started-with-iris)**
   * Comprehensive guide to using IRIS through Dex SDK
   * Prerequisites and setup instructions
   * Parsing PDFs and images with complete examples
   * Configuration options for parse engine
   * Understanding parse results and chunk structure
   * File management and error handling
   * Batch processing examples
   * Multi-language support details
   * Best practices for production use
   * Performance considerations and optimization tips

#### Key Features

* **Layout-Aware Processing**: Automatically detects text, tables, and images before OCR
* **Multiple OCR Engines**: Choose from Tesseract, EasyOCR, PaddleOCR, Surya, GPT-4o, Gemini, and more
* **Table-Specific Processing**: Specialized models optimized for accurate table extraction
* **Multi-Language Support**: Process documents in 35+ languages including Arabic
* **Dex Integration**: Seamless integration with Dex's document understanding platform
* **Async Processing**: Non-blocking parse jobs with project-based organization

#### How to Access

IRIS is available through the Dex SDK as a parse engine option:

```python theme={null}
from dex_core.models.parse_job import IrisParseEngineOptions, IrisParseJobParams

parse_job = await dex_file.start_parse_job(
    IrisParseJobParams(options=IrisParseEngineOptions())
)
```

***

## Configuration Updates

### Navigation Structure Improvements

**Updated:** October 2024

* Added explicit V5 (beta) version tags to all Capabilities navigation groups
* Ensures proper scoping of Capabilities documentation to V5
* Improved navigation organization for better user experience

**Affected Sections:**

* Getting Started
* Document Understanding
* OCR
* Workflows

***

## Support and Feedback

### Workflows-Specific Questions

For questions or issues related to SGP Workflows:

* **Slack**: `#compass`
* **Documentation**: [Introduction to Workflows](/docs/capabilities/workflows/introduction-to-workflows)
* **Tutorial**: [Guide to Automating Evals on Workflows](/docs/capabilities/workflows/evals-in-workflows)

### Dex-Specific Questions

For questions or issues related to Dex:

* **Slack**: `#dex-help`
* **Documentation**: [Getting Started with Dex](/docs/capabilities/document-understanding/getting-started-with-dex)
* **API Reference**: [Dex SDK API Reference](/docs/capabilities/document-understanding/dex-sdk-api-reference)

### General Capabilities Documentation

Have suggestions for improving our Capabilities documentation? Please contact the Scale AI team or submit feedback through your account dashboard.


# Best Practices
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/best-practices

Quick start, optimization tips, data retention, and common patterns for Dex document understanding.

## 30-Second Quick Start

```python theme={null}
import os
import asyncio
from pydantic import BaseModel, Field
from dex_sdk import DexClient
from dex_sdk.types import (ParseEngine, IrisParseJobParams, IrisParseEngineOptions)

class Schema(BaseModel):
    field: str = Field(description="Description")

async def main():
    client = DexClient(base_url="https://dex.sgp.scale.com", api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"))
    project = await client.create_project(name="My Project")
    file = await project.upload_file("doc.pdf")
    parsed = await file.parse(
        IrisParseJobParams(
            engine=ParseEngine.IRIS,
            options=IrisParseEngineOptions(text_ocr="openai/gpt-5.4", layout="whole_page"
            ),
        )
    )
    result = await parsed.extract(extraction_schema=Schema, user_prompt="Extract data", model="openai/gpt-5.4")
    print(result.result.data)

asyncio.run(main())
```

***

## Async Client Pattern

The current Dex SDK is async-only. Prefer `async with DexClient(...)` when you are making multiple calls so the underlying HTTP connection can be reused:

```python theme={null}
from dex_sdk import DexClient

async with DexClient(
    base_url="https://dex.sgp.scale.com",
    api_key=os.getenv("SGP_API_KEY"),
    account_id=os.getenv("SGP_ACCOUNT_ID"),
) as client:
    project = await client.create_project(name="My Project")
```

***

## Data Retention Policies

Configure automatic data lifecycle management for compliance and cost optimization.

### Setting Retention Policies

```python theme={null}
from datetime import timedelta
from dex_sdk.types import ProjectConfiguration, ProjectUpdateRequest, RetentionPolicy

# Create project with retention policy
project = await dex_client.create_project(
    name="Compliant Project",
    configuration=ProjectConfiguration(
        retention=RetentionPolicy(
            files=timedelta(days=30),           # Files expire after 30 days
            result_artifacts=timedelta(days=7),  # Parse/extract results expire after 7 days
        )
    )
)
```

### Updating Retention Policies

```python theme={null}
await dex_client.update_project(
    project_id=project.id,
    project_data=ProjectUpdateRequest(
        configuration=ProjectConfiguration(
            retention=RetentionPolicy(
                files=timedelta(days=90),        # Extend to 90 days
                result_artifacts=timedelta(days=30),  # Extend to 30 days
            )
        )
    )
)
```

### Retention Policy Use Cases

**Compliance (GDPR, HIPAA):**

```python theme={null}
retention=RetentionPolicy(
    files=timedelta(days=2555),  # 7 years
    result_artifacts=timedelta(days=365),  # 1 year
)
```

**Cost Optimization:**

```python theme={null}
retention=RetentionPolicy(
    files=timedelta(days=30),    # Raw files for 1 month
    result_artifacts=timedelta(days=7),   # Results for 1 week
)
```

**Security (Minimize Exposure):**

```python theme={null}
retention=RetentionPolicy(
    files=timedelta(days=1),     # Delete files after 1 day
    result_artifacts=timedelta(hours=24),  # Delete results after 24 hours
)
```

***

## Optimizing Extraction Accuracy

1. **Write Clear Prompts**
   ```python theme={null}
   # Good: Specific and detailed
   user_prompt = """Extract the following fields:
   - Invoice number: Find in top-right corner or header
   - Total amount: The final amount including tax
   - Date: Invoice date, not payment date"""

   # Bad: Vague
   user_prompt = "Extract invoice info"
   ```

2. **Design Good Schemas**
   ```python theme={null}
   # Good: Descriptive fields with clear types
   class InvoiceData(BaseModel):
       invoice_number: str = Field(description="The invoice number (format: INV-XXXXX)")
       total_amount: float = Field(description="Total amount in USD, including tax")
       invoice_date: str = Field(description="Invoice date in YYYY-MM-DD format")

   # Bad: No descriptions
   class InvoiceData(BaseModel):
       number: str
       amount: float
       date: str
   ```

3. **Enable Citations and Confidence**
   * Always set `generate_citations=True` for debugging
   * Use `generate_confidence=True` to filter low-confidence results

4. **Use Vector Stores for Large Documents**
   * Documents > 50 pages benefit from RAG-enhanced extraction
   * Vector stores improve accuracy for cross-document queries

***

## Getting SGP Traces for Extraction

Use `start_extract_job` instead of `extract` to get async jobs that are linked to SGP traces. Search for traces by job ID to debug extraction latency, token usage, or failures.

```python theme={null}
import os
from scale_gp_beta import SGPClient
from dex_sdk.types import ExtractionParameters

# Start extract job (returns immediately)
extract_job = await parse_result.start_extract_job(
    ExtractionParameters(
        model="openai/gpt-5.4",
        extraction_schema=MySchema.model_json_schema(),
        user_prompt="Extract the data",
    )
)

# Wait for completion
result = await extract_job.get_result()

# Retrieve SGP traces for the extract job
sgp_client = SGPClient(
    api_key=os.getenv("SGP_API_KEY"),
    account_id=os.getenv("SGP_ACCOUNT_ID"),
)

spans = list(sgp_client.spans.search(
    sort_by="created_at",
    sort_order="desc",
    extra_metadata={"job_id": extract_job.data.id},
    parents_only=True,
))

if spans:
    trace_id = spans[0].trace_id
    all_spans = list(sgp_client.spans.search(trace_ids=[trace_id]))
    for span in all_spans:
        print(f"Span: {span.name}, Duration: {span.duration_ms}ms")
```

***

## Common Workflows

**Single Document Processing:**

```
Upload → Parse → Extract → Review Results
```

**Multi-Document Analysis:**

```
Upload All → Parse All → Create Vector Store → Add to Store → Extract → Aggregate
```

**Iterative Refinement:**

```
Parse → Review → Adjust Chunking/Engine → Re-parse → Extract
```

**Production Pipeline:**

```
Upload → Parse → Store in Vector DB → Extract on Demand → Cache Results
```

***

## Performance Optimization

### Reduce Latency

1. **Use appropriate chunking**: Smaller chunks = faster parsing
2. **Limit OCR scope**: Only process pages you need
3. **Batch operations**: Process multiple files in parallel
4. **Cache parse results**: Reuse parsed documents for multiple extractions

### Reduce Costs

1. **Set retention policies**: Auto-delete old data
2. **Choose right model**: Use smaller models when possible
3. **Optimize prompts**: Shorter prompts = lower token costs
4. **Filter before extraction**: Use vector search to find relevant chunks first

***

## Common Errors & Quick Fixes

| Error                 | Cause                           | Quick Fix                                         |
| --------------------- | ------------------------------- | ------------------------------------------------- |
| `AuthenticationError` | Missing/invalid credentials     | Check `SGP_API_KEY` and `SGP_ACCOUNT_ID` env vars |
| `FileUploadError`     | Unsupported format or too large | Check file type, reduce size                      |
| `ParsingError`        | OCR failure                     | Try different engine or check document quality    |
| `ExtractionError`     | Invalid schema                  | Validate Pydantic model, check field types        |
| `ConnectionError`     | Network issues                  | Check internet connection, verify base URL        |
| `RateLimitError`      | Too many requests               | Implement backoff/retry, reduce concurrency       |

***

## Access Response Data

Remember: SDK methods return **wrapper objects**. Use `.data` for full entity fields, and use convenience properties like `.id` when available:

```python theme={null}
# Correct
project = await client.create_project(name="Test")
project_id = project.data.id
project_name = project.data.name

# Also valid
project_id = project.id
```

**Common Response Attributes:**

```python theme={null}
project.data.id                                 # Project ID
project.data.name                               # Project name
project.data.created_at                         # Creation timestamp

dex_file.data.id                                # File ID
dex_file.data.filename                          # Filename
dex_file.data.size_bytes                        # File size

parse_result.data.id                            # Parse result ID
parse_result.data.engine                        # Engine used
parse_result.data.parse_metadata.pages_processed  # Pages count

extract_result.result.data                      # Extracted data dict
extract_result.result.usage_info.total_tokens   # Token usage
```

***

## Type Import Quick Lookup

| Type                        | Import From     | Used For               |
| --------------------------- | --------------- | ---------------------- |
| `DexClient`                 | `dex_sdk`       | Client initialization  |
| `ParseEngine`               | `dex_sdk.types` | OCR engine selection   |
| `IrisParseJobParams`        | `dex_sdk.types` | Iris configuration     |
| `ReductoParseJobParams`     | `dex_sdk.types` | Reducto configuration  |
| `IrisParseEngineOptions`    | `dex_sdk.types` | Iris parser options    |
| `ReductoParseEngineOptions` | `dex_sdk.types` | Parser options         |
| `ExtractionParameters`      | `dex_sdk.types` | Extraction config      |
| `VectorStoreEngines`        | `dex_sdk.types` | Vector store engines   |
| `VectorStoreSearchResult`   | `dex_sdk.types` | Search results         |
| `ProjectConfiguration`      | `dex_sdk.types` | Project config         |
| `RetentionPolicy`           | `dex_sdk.types` | Data retention         |
| `PaginationParams`          | `dex_sdk.types` | Pagination config      |
| `FileListFilter`            | `dex_sdk.types` | File filtering         |
| `JobListFilter`             | `dex_sdk.types` | Job filtering          |
| `ParseResultListFilter`     | `dex_sdk.types` | Parse result filtering |
| `ExtractionListFilter`      | `dex_sdk.types` | Extraction filtering   |
| `VectorStoreListFilter`     | `dex_sdk.types` | Vector store filtering |
| `JobStatus`                 | `dex_sdk.types` | Job status enum        |
| `ReductoChunkingMethod`     | `dex_sdk.types` | Chunking method enum   |
| `ReductoChunkingOptions`    | `dex_sdk.types` | Chunking config        |

**Response Types** (accessed via `.data`, no import needed):

* `ProjectEntity`, `FileEntity`, `ParseResultEntity`, `ExtractionEntity`, `VectorStoreEntity`

***

## Next Steps

* **[Getting Started](getting-started-with-dex)**: Step-by-step tutorial
* **[File Management](file-management)**: Upload and manage files
* **[API Reference](dex-sdk-api-reference)**: Complete SDK documentation
* **[Troubleshooting](troubleshooting)**: Detailed error solutions


# Chunking
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/chunking

Choose chunking strategies for parsed documents, Dex rechunking or Reducto parse-time chunking.

Dex provides flexible chunking for parsed documents. You can apply chunking in two ways: **Dex rechunking** (parse once, then rechunk with different strategies) or **engine-specific chunking** during the initial parse (Reducto only).

***

## IRIS v2 Chunk Structure

IRIS v2 always returns a `chunks` array in the parse result. `chunk_mode: disabled` is a Reducto-only concept (soon to be deprecated) that does not exist in IRIS v2.
Chunk count and size are driven by **how IRIS segments each page**, not by a separate chunking enum.

| Control                           | Values                                                                                               | Effect on chunks                                                 |
| --------------------------------- | ---------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- |
| `layout`                          | `"rt_detr_bce"` (default)                                                                            | One chunk per detected layout region (text, table, image, etc.)  |
| `layout`                          | `"whole_page"`                                                                                       | One region per page → **one chunk per page**                     |
| `e2e_ocr` + `e2e_response_parser` | e.g. DeepSeek OCR2                                                                                   | Full-page e2e OCR; parser defines region/box structure           |
| Layout filters                    | `confidence_threshold`, `containment_threshold`, `strict_containment_filter`, table/image thresholds | Fewer or more regions → fewer or more chunks                     |
| `img_method`                      | `"description"`, `"base64"`, `"skip"`                                                                | Whether image regions become chunks (`skip` omits image regions) |

These options live on `IrisParseEngineOptions` inside `IrisParseJobParams`. They are **not** the same as Dex’s four rechunk strategies below—they control IRIS’s native output only.

### Parse with IRIS v2

```python theme={null}
from dex_sdk.types import (
    ParseEngine,
    IrisParseJobParams,
    IrisParseEngineOptions,
)
parse_result = await dex_file.parse(
    IrisParseJobParams(
        engine=ParseEngine.IRIS,
        options=IrisParseEngineOptions(
            layout="rt_detr_bce",   # default: region-level chunks
            # layout="whole_page",  # alternative: one chunk per page
            text_ocr="openai/gpt-5.4",
        ),
    )
)
```

## Dex Chunking Strategies

Dex offers four chunking strategies that work on any parse result—including IRIS v2 via rechunking.
Use these when you need consistent, configurable chunk boundaries across documents or for embeddings/RAG

| Strategy     | Description                                                          | Best For                                      | Embedding Suitability |
| ------------ | -------------------------------------------------------------------- | --------------------------------------------- | --------------------- |
| `token_size` | Splits by token count using a tokenizer (e.g., tiktoken)             | LLM APIs with token limits, cost optimization | Excellent             |
| `recursive`  | Recursively splits using separators (paragraphs → sentences → words) | Articles, documentation, RAG systems          | Excellent             |
| `by_page`    | Splits by page boundaries, grouping complete pages                   | Legal documents, forms, reports               | May be large          |
| `by_section` | Splits by section headers (e.g., markdown #, ##, ###)                | Technical manuals, wikis, academic papers     | Good                  |

***

## Parse Once, Post-chunking as needed

With IRIS 2, you parse without Dex rechunking (omit chunking\_options), then apply Dex strategies on the parse result.
This is the recommended pattern: one parse, many chunking experiments.

```python theme={null}
from dex_sdk.types import IrisParseJobParams, IrisParseEngineOptions, ParseEngine
from dex_core.models.chunking import (
    TokenSizeChunkingOptions,
    RecursiveChunkingOptions,
    PageChunkingOptions,
    SectionChunkingOptions,
)
# Step 1: Parse with IRIS v2 (native region- or page-level chunks)
parse_result = await dex_file.parse(
    IrisParseJobParams(
        engine=ParseEngine.IRIS,
        options=IrisParseEngineOptions(
            layout="whole_page",  # optional: coarser native chunks
        ),
    )
)

# Step 2: Rechunk with Dex strategies
# Token-based (for LLM APIs, embedding models)
rechunked = await parse_result.rechunk(
    TokenSizeChunkingOptions(
        chunk_size=512,
        chunk_overlap=50,
        encoding_name="cl100k_base",
    )
)

# Recursive (preserves paragraphs and sentences)
rechunked = await parse_result.rechunk(
    RecursiveChunkingOptions(
        chunk_size=1000,
        chunk_overlap=200,
    )
)

# Page-based (legal docs, forms)
rechunked = await parse_result.rechunk(
    PageChunkingOptions(pages_per_chunk=1)
)

# Section-based (structured documents with headers)
rechunked = await parse_result.rechunk(
    SectionChunkingOptions(
        section_headers=None,  # Auto-detect from block types
        include_header_in_chunk=True,
    )
)
```

***

## Async Rechunking

For long-running documents, start the rechunk job and poll for completion:

```python theme={null}
# Start rechunk job (returns immediately)
job = await parse_result.start_rechunk_job(
    TokenSizeChunkingOptions(chunk_size=512, chunk_overlap=50)
)

# Wait for completion and get rechunked result
rechunked = await job.get_rechunked_result()
```

***

## Chunking Decision Tree

**IRIS v2 native (`IrisParseEngineOptions`):**

* **Many small, layout-aware chunks** → `layout="rt_detr_bce"` (default)
* **One chunk per page** → `layout="whole_page"`
* **Full-page e2e model** → `e2e_ocr` + `e2e_response_parser`
* **Fewer regions (fewer chunks)** → raise confidence thresholds or tighten containment filters

**Dex rechunk (after IRIS v2 parse):**

**Use `token_size` when:**

* Working with LLM APIs that have token limits
* Embedding models with specific token limits
* Cost optimization

**Use `recursive` when:**

* General document chunking for RAG
* Preserving paragraphs and sentences
* Articles, blog posts, documentation

**Use `by_page` when:**

* Legal documents, forms, reports
* Page references matter
* Page structure should be preserved

**Use `by_section` when:**

* Documents have clear section headers
* Technical manuals, wikis, academic papers
* Semantic coherence within topics

***

## Reducto Chunking Parse-Time - (Legacy)

When using the **Reducto** parse engine (soon to be deprecated), you can chunk during the initial parse instead of rechunking.
Reducto's methods are layout-aware and use document structure.

| Method             | Chunk Size                 | Best For                       | Embedding    | Location Tracking |
| ------------------ | -------------------------- | ------------------------------ | ------------ | ----------------- |
| **VARIABLE**       | Auto (optimal)             | General use, embeddings        | Excellent    | Good              |
| **BLOCK**          | Small (\~100-500 chars)    | Precise locations, UI overlays | Too small    | Excellent         |
| **SECTION**        | Medium (\~1000-3000 chars) | Structured documents           | Good         | Good              |
| **PAGE**           | Large (full page)          | Page-oriented docs             | May be large | Excellent         |
| **PAGE\_SECTIONS** | Medium-Large               | Hybrid needs                   | Good         | Good              |
| **DISABLED**       | Very large (entire doc)    | Special cases                  | Too large    | Excellent         |

**Recommendation:** Use `VARIABLE` for most cases, especially with embeddings.

```python theme={null}
# (Legacy) Reducto parse-time chunking
from dex_sdk.types import (
    ParseEngine,
    ReductoParseJobParams,
    ReductoChunkingMethod,
    ReductoChunkingOptions,
    ReductoParseEngineOptions,
)

# Reducto variable chunking (parse-time)
parse_result = await dex_file.parse(
    ReductoParseJobParams(
        engine=ParseEngine.REDUCTO,
        options=ReductoParseEngineOptions(
            chunking=ReductoChunkingOptions(
                chunk_mode=ReductoChunkingMethod.VARIABLE,
                chunk_size=None,
            )
        ),
    )
)

# Reducto block chunking for precise location tracking
parse_result = await dex_file.parse(
    ReductoParseJobParams(
        engine=ParseEngine.REDUCTO,
        options=ReductoParseEngineOptions(
            chunking=ReductoChunkingOptions(chunk_mode=ReductoChunkingMethod.BLOCK)
        ),
    )
)
```

### Pattern: Retry with Different Chunking

```python theme={null}
# (Legacy) Reducto retry pattern
# Try 1: Variable chunking
result1 = await file.parse(
    ReductoParseJobParams(
        engine=ParseEngine.REDUCTO,
        options=ReductoParseEngineOptions(
            chunking=ReductoChunkingOptions(
                chunk_mode=ReductoChunkingMethod.VARIABLE,
            )
        ),
    )
)

# If not satisfactory, try block chunking
result2 = await file.parse(
    ReductoParseJobParams(
        engine=ParseEngine.REDUCTO,
        options=ReductoParseEngineOptions(
            chunking=ReductoChunkingOptions(
                chunk_mode=ReductoChunkingMethod.BLOCK,
            )
        ),
    )
)
```

***

### Reducto-only decision tree (Legacy)

**Use Reducto `VARIABLE` when:**

* Still on Reducto and want layout-aware chunking at parse time
* General document processing with parser-optimal chunk sizes

**Use Reducto `BLOCK` when:**

* Need precise bounding box information
* Building UI overlays on documents
* Not using for embeddings

**Use Reducto `DISABLED` when:**

* You plan to rechunk with Dex strategies (same pattern as IRIS v2)

```
What's your primary goal? (Reducto only)
├─ Embeddings/Semantic Search → VARIABLE
├─ Precise bounding boxes/UI → BLOCK
├─ Page-by-page processing → PAGE
└─ Structured document navigation → SECTION
```

***

## Next Steps

* **[Vector Stores](vector-stores)**: Add chunks to vector stores for semantic search
* **[Extract](extract)**: Extract structured data from parse results
* **[Parse](parse)**: Parse engine options and configuration (IRIS v2)


# Extract
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/extract

Extract structured data from parse results or vector stores using LLM-powered extraction.

Extract structured data from documents using Pydantic schemas and LLM models. Extract from single parse results or use [vector stores](vector-stores#rag-enhanced-extraction) for RAG-enhanced extraction on large documents.

***

## Extract from Parse Result

```python theme={null}
extract_result = await parse_result.extract(
    extraction_schema=MySchema,
    user_prompt="Extract the data",
    model="openai/gpt-5.4",
    generate_citations=True,
    generate_confidence=True,
)
```

***

## Extract from Vector Store

For large documents or multi-file analysis, use [RAG-enhanced extraction](vector-stores#rag-enhanced-extraction) from a vector store. This improves accuracy by retrieving relevant chunks before extraction.

```python theme={null}
result = await vector_store.extract(
    extraction_schema=MySchema,
    user_prompt="Extract from all documents",
    model="openai/gpt-5.4",
)
```

***

## Extract from File (Direct)

Extract directly from an uploaded file, skipping the parse step.

**When to use:** structured extraction from images or small documents (≈1–5 pages) where visual structure is vital for accurate extraction — passports, ID cards, forms, invoices, and similar. For text-dense or large documents, prefer parse-then-extract.

```python theme={null}
result = await file.extract(
    extraction_schema=MySchema,
    user_prompt="Extract the fields",
    model="openai/gpt-5.4",
    generate_confidence=True,
)
```

Supports PDF, image (PNG, JPG, WebP), and office (DOCX, DOC, PPTX, PPT) files. Note that, citations are not yet supported in this mode.

***

## Extract with Citations

```python theme={null}
result = await parse_result.extract(
    extraction_schema=Schema,
    user_prompt="Extract data",
    model="openai/gpt-5.4",
    generate_citations=True,  # Get source locations
)

for field_name, field in result.result.data.items():
    if field.citations:
        print(f"{field_name} found on page {field.citations[0].page}")
```

***

## Filter Low Confidence Results

```python theme={null}
result = await parse_result.extract(
    extraction_schema=Schema,
    user_prompt="Extract data",
    model="openai/gpt-5.4",
    generate_confidence=True,
)

high_confidence = {
    k: v for k, v in result.result.data.items()
    if v.confidence and v.confidence > 0.8
}
```

***

## Batch Extraction

Start extract jobs for multiple documents, then monitor and collect results. This pattern gives you control over job status and lets you handle failures per document.

```python theme={null}
import asyncio

# Start extract jobs for all parse results (returns immediately)
extract_parameters = ExtractionParameters(
    model="openai/gpt-5.4",
    extraction_schema=InvoiceData.model_json_schema(),
    user_prompt="Extract invoice details.",
    generate_citations=True,
    generate_confidence=True,
)

extract_jobs = [
    await parse_result.start_extract_job(extract_parameters)
    for parse_result in parse_results
]

# Monitor jobs until all complete
while any(
    job.data.status not in [JobStatus.SUCCEEDED, JobStatus.FAILED]
    for job in extract_jobs
):
    await asyncio.sleep(1)
    await asyncio.gather(*[job.refresh() for job in extract_jobs])

# Collect results from successful jobs
extractions = []
for i, job in enumerate(extract_jobs):
    if job.data.status == JobStatus.SUCCEEDED:
        extraction = await job.get_result()
        extractions.append(extraction)
        print(f"Document {i+1}: {extraction.result.data}")
    else:
        print(f"Document {i+1} failed: {job.data.error}")
```

To wait for all jobs and collect results in one call, use `asyncio.gather` with `get_result`:

```python theme={null}
# Start all jobs in parallel
extract_jobs = await asyncio.gather(*[
    parse_result.start_extract_job(extract_parameters)
    for parse_result in parse_results
])

# Wait for all to complete and collect results
extractions = await asyncio.gather(*[job.get_result() for job in extract_jobs])

for i, extraction in enumerate(extractions):
    print(f"Document {i+1}: {extraction.result.data}")
```

***

## Appendix: Essential Imports

```python theme={null}
from pydantic import BaseModel, Field
from dex_sdk.types import ExtractionParameters, JobStatus
```

***

## Next Steps

* **[Vector Stores](vector-stores)**: RAG-enhanced extraction for large documents
* **[Best Practices](best-practices)**: Optimize extraction accuracy
* **[API Reference](dex-sdk-api-reference)**: Complete SDK documentation


# File Management
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/file-management

Upload, list, and manage files in Dex projects with pagination and filtering.

Manage files, projects, and collections in Dex. The same pagination and filtering patterns apply to jobs, parse results, extractions, and vector stores.

***

## Create Project

```python theme={null}
import os
from dex_sdk import DexClient

dex_client = DexClient(
    base_url="https://dex.sgp.scale.com",
    api_key=os.getenv("SGP_API_KEY"),
    account_id=os.getenv("SGP_ACCOUNT_ID"),
)

project = await dex_client.create_project(name="My Project")
```

### Create Project with Retention

```python theme={null}
project = await dex_client.create_project(
    name="My Project",
    configuration=ProjectConfiguration(
        retention=RetentionPolicy(
            files=timedelta(days=30),
            result_artifacts=timedelta(days=7),
        )
    )
)
```

***

## Upload File

```python theme={null}
dex_file = await project.upload_file("document.pdf")
```

### Process Multiple Files

```python theme={null}
import asyncio

files_to_upload = ["doc1.pdf", "doc2.pdf", "doc3.pdf"]
dex_files = await asyncio.gather(*[project.upload_file(f) for f in files_to_upload])
print(f"Uploaded {len(dex_files)} files")
```

***

## Pagination and Filtering

**New in v0.4.0**: Efficiently manage large collections with pagination and filtering.

### Listing Files with Pagination

```python theme={null}
pagination_params = PaginationParams(
    page_size=50,
    sort_by="created_at",
    sort_order="desc",
    continuation_token=None
)

result = await project.list_files(pagination_params=pagination_params)

# Access items
for file in result.items:
    print(f"File: {file.filename} ({file.created_at})")

# Paginate through remaining results
while result.next_token:
    pagination_params.continuation_token = result.next_token
    result = await project.list_files(pagination_params=pagination_params)
    # Process result.items
```

### Filtering by Creation Time

```python theme={null}
from datetime import datetime, timedelta

# Filter files created in the last 24 hours
file_filter = FileListFilter(
    created_at_start=datetime.now() - timedelta(days=1)
)
recent_files = await project.list_files(filter=file_filter)

# Combine pagination and filtering for files
pagination_params = PaginationParams(page_size=20, sort_by="created_at", sort_order="desc")
file_filter = FileListFilter(created_at_start=datetime.now() - timedelta(days=30))
recent_files = await project.list_files(
    pagination_params=pagination_params,
    filter=file_filter
)
```

### Available List Operations

All entity types support unified `list_*` operations with pagination and filtering:

* `list_files(pagination_params, filter)` - List uploaded files
* `list_jobs(pagination_params, filter)` - List all jobs
* `list_parse_results(pagination_params, filter)` - List parse results
* `list_extractions(pagination_params, filter)` - List extractions
* `list_vector_stores(pagination_params, filter)` - List vector stores

***

## Supported File Types

Dex supports a wide variety of document formats:

### Images

PNG, JPEG/JPG, GIF, BMP, TIFF, PCX, PPM, APNG, PSD, CUR, DCX, FTEX, PIXAR, HEIC

### Documents

PDF, DOCX, DOC, DOTX, WPD, TXT, RTF, PPTX, PPT

### Spreadsheets

CSV, XLSX, XLSM, XLS, XLTX, XLTM, QPW

<Note>
  For best results with spreadsheets, use XLSX format. CSV files are processed as-is without layout analysis.
</Note>

***

## Access Response Data

SDK methods return **wrapper objects**; access data via `.data`:

```python theme={null}
# Correct
project_id = project.data.id
project_name = project.data.name

dex_file.data.id
dex_file.data.filename
dex_file.data.size_bytes
```

***

## Appendix: Essential Imports

```python theme={null}
from dex_sdk.types import (
    PaginationParams,
    FileListFilter,
    ProjectConfiguration,
    RetentionPolicy,
)
from datetime import timedelta
```

***

## Next Steps

* **[Parse](parse)**: Parse documents with IRIS or Reducto
* **[Vector Stores](vector-stores)**: Create vector stores for semantic search
* **[API Reference](dex-sdk-api-reference)**: Complete SDK documentation


# Getting Started with Dex
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/getting-started-with-dex



This guide will help you get started with Scale's Dex service - a document understanding capability that extracts accurate, structured information from unstructured documents.

<h2>Overview</h2>

Dex is Scale's document understanding capability that provides composable primitives for:

* **File Management** - Secure file upload, storage, and retrieval
* **Document Parsing** - Convert any document (PDFs, DOCX, images, etc.) into structured JSON format with multiple OCR engines
* **Vector Stores** - Embed, index and search parsed document corpora
* **Data Extraction** - Extract specific information using custom schemas, prompts, and RAG-enhanced context
* **Project Management** - Organize and isolate data with proper credential management and authorization

<h2>Prerequisites</h2>

Before using Dex, ensure you have:

* ✅ A valid Scale account with SGP (Scale General Platform) access
* ✅ Your SGP account ID and API key set as environment variables:

```bash theme={null}
export SGP_ACCOUNT_ID="your_account_id"
export SGP_API_KEY="your_api_key"
```

* ✅ Python 3.10+ installed
* ✅ Dex SDK installed (see Installation section)

<h2>Installation</h2>

### Install Dex SDK from CodeArtifact

With access to Scale CodeArtifact, install the Dex SDK (version 0.4.0 or higher recommended) using your configured CodeArtifact credentials.

This will install all required dependencies including Pydantic.

> **Note**: Current examples in this guide were validated against the 0.4.x SDK line, which uses SGP API key and account ID authentication. Version 0.4.0 also introduced pagination, filtering, and enhanced async job support. See the [Changelog](/docs/capabilities/changelog) for details.

<h2>Quick Start</h2>

### 1. Initialize Dex Client

```python theme={null}
import os
from pydantic import BaseModel, Field
from dex_sdk import DexClient
from dex_sdk.types import ParseEngine, IrisParseJobParams, IrisParseEngineOptions

# Initialize the Dex client with SGP credentials
dex_client = DexClient(
    base_url="https://dex.sgp.scale.com",
    api_key=os.getenv("SGP_API_KEY"),
    account_id=os.getenv("SGP_ACCOUNT_ID"),
)
```

### 2. Create a Project

Projects isolate your data and credentials for tracing, billing, and SGP model calls. Every operation is tied to a project.

```python theme={null}
# Create a project
project = await dex_client.create_project(name="My Dex Project")
print(f"Created project: {project.data.id}")
```

> **Tip**: Keep one project per use case or group of related files for clean traceability.

### 3. Upload a Document

Upload your document to the project. Dex supports PDFs, images, spreadsheets, and more.

```python theme={null}
# Upload a file to the project
dex_file = await project.upload_file("path/to/your/document.pdf")
print(f"Uploaded: {dex_file.data.filename} → {dex_file.data.id}")
```

**Supported file types:** PDF, PNG, JPEG, DOCX, XLSX, CSV, and many more. See [File Management](file-management#supported-file-types) for the complete list.

### 4. Parse the Document

Parse converts your document into a structured format with text, tables, and layout information.

```python theme={null}
# Parse the document with Iris V2
parse_result = await dex_file.parse(
    IrisParseJobParams(
        engine=ParseEngine.IRIS,
        options=IrisParseEngineOptions(
            layout="rt_detr_bce",
            text_ocr="openai/gpt-5.4",
            table_ocr="openai/gpt-5.4"
        ),
    )
)
print(f"Parsed: {parse_result.data.parse_metadata.pages_processed} pages")
```

> **Note**: Parsing is asynchronous. The SDK automatically polls for completion.

### 5. Extract Structured Data

Define a schema and extract specific information from your document.

```python theme={null}
# Define your extraction schema using Pydantic
class InvoiceData(BaseModel):
    """Schema for invoice extraction"""
    invoice_number: str = Field(description="The invoice number")
    total_amount: float = Field(description="Total amount in dollars")
    date: str = Field(description="Invoice date in YYYY-MM-DD format")
    vendor_name: str = Field(description="Name of the vendor")

# Extract data with a clear prompt
extract_result = await parse_result.extract(
    extraction_schema=InvoiceData,
    user_prompt="Extract invoice details from this document.",
    model="openai/gpt-5.4",
    generate_citations=True,
    generate_confidence=True,
)

# Access the extracted data
result = extract_result.result
for field_name, field in result.data.items():
    print(f"{field_name}: {field.value} (confidence: {field.confidence:.2f})")
```

***

<h2>Complete Example</h2>

Here's a complete working example you can copy and run:

```python theme={null}
import os
import asyncio
from pydantic import BaseModel, Field
from dex_sdk import DexClient
from dex_sdk.types import ParseEngine, IrisParseJobParams, IrisParseEngineOptions

class InvoiceData(BaseModel):
    invoice_number: str = Field(description="The invoice number")
    total_amount: float = Field(description="Total amount in dollars")

async def main():
    # 1. Initialize client
    dex_client = DexClient(
        base_url="https://dex.sgp.scale.com",
        api_key=os.getenv("SGP_API_KEY"),
        account_id=os.getenv("SGP_ACCOUNT_ID"),
    )

    # 2. Create project
    project = await dex_client.create_project(name="Invoice Processing")

    # 3. Upload document
    dex_file = await project.upload_file("invoice.pdf")

    # 4. Parse document
    parse_result = await dex_file.parse(
        IrisParseJobParams(
            engine=ParseEngine.IRIS,
            options=IrisParseEngineOptions(
                layout="rt_detr_bce",
                text_ocr="openai/gpt-5.4",
                table_ocr="openai/gpt-5.4"
            ),
        )
    )

    # 5. Extract data
    extract_result = await parse_result.extract(
        extraction_schema=InvoiceData,
        user_prompt="Extract invoice number and total amount.",
        model="openai/gpt-5.4",
        generate_citations=True,
    )

    print(extract_result.result.data)

# Run the example
asyncio.run(main())
```

***

<h2>Next Steps</h2>

Now that you've completed the basics, explore these topics:

### Learn More

* **[File Management](file-management)**: Upload, pagination, and supported file types
* **[Parse](parse)**: Parse engines and async job monitoring
* **[Chunking](chunking)**: Chunking strategies for documents
* **[Vector Stores](vector-stores)**: Semantic search and RAG-enhanced extraction
* **[Extract](extract)**: Batch extraction and extraction patterns
* **[Best Practices](best-practices)**: Quick start, retention policies, and optimization

### Deep Dive into the API

* **[API Reference](dex-sdk-api-reference)**: Complete SDK documentation with all methods and types
* **[Troubleshooting Guide](troubleshooting)**: Common issues and solutions
* **[Changelog](/docs/capabilities/changelog)**: Latest updates and breaking changes

### Additional Resources

* **REST API**: For non-Python integrations, see the [API Reference](api-reference)
* **Support**: Questions? Ask in Slack channel `#dex-help`
* **Examples**: More examples in the [Introduction guide](introduction-to-dex#common-use-cases)

***

<h2>Common Questions</h2>

**Q: How do I process multiple documents?**
A: Upload multiple files to the same project, parse each one, then optionally use vector stores for cross-document search. See [Extract](extract#batch-extraction).

**Q: Does the SDK provide a synchronous client?**
A: The current SDK is async-only. Use `DexClient` with `await`, or `async with DexClient(...)` to reuse a single HTTP connection across multiple calls.

**Q: How do I configure data retention policies?**
A: Set retention policies when creating a project. See [Best Practices](best-practices#data-retention-policies).

**Q: What OCR engines are available?**
A: Reducto (production-ready) and Iris (experimental, for custom needs). See **[When to choose Iris?](/docs/capabilities/ocr/when-to-choose-iris)**.

**Q: How do I list files with pagination?** (New in v0.4.0)
A: Use `PaginationParams` with `list_files()` to control page size and sorting. See [API Reference](dex-sdk-api-reference#file-operations).

**Q: How do I monitor async jobs?** (New in v0.4.0)
A: Use `start_parse_job()` for better control and access to SGP traces. See [API Reference](dex-sdk-api-reference#job-monitoring-and-sgp-tracing).


# Indexing
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/indexing

Build a hierarchical index over your project's files for fast BM25 search and tree navigation.

An index organizes your project's files into a searchable tree. Leaf nodes (level 0) represent individual files, and higher-level nodes (level 1+) hold generated summaries that group related files—like folders. You can run fast lexical (BM25) search over the files, or navigate the hierarchy level by level.

<Note>
  Indexing is a newly released Dex capability. The API may change in future SDK versions.
</Note>

The index lifecycle is: **create → add/remove files (async) → search/navigate (sync) → delete**.

***

## Index vs. Vector Store

Indexes and [vector stores](vector-stores) are both retrieval primitives, but they serve different access patterns:

|               | Index                                                               | Vector Store                                    |
| ------------- | ------------------------------------------------------------------- | ----------------------------------------------- |
| **Search**    | Lexical BM25 over file summaries                                    | Semantic, lexical, or hybrid over chunks        |
| **Structure** | Hierarchical tree you can navigate                                  | Flat collection of chunks                       |
| **Best for**  | Finding and browsing relevant files in a corpus; agentic navigation | Retrieving passages for RAG-enhanced extraction |

Both can be combined as inputs to a [research](research) job.

***

## Creating an Index

Indexing currently supports only the file-system index engine (`FileSystemIndexCreateRequest`). This engine builds the tree from the folder structure of your Dex project: folders are inferred from each file's name, treating everything before the last `/` as the folder path. For example, a file named `finance/2016/report.pdf` becomes a leaf under a `finance/2016` folder, with a parent `finance` folder above it. Folder nodes are summarized using the configured `model` and prompts.

```python theme={null}
from dex_sdk import FileSystemIndexCreateRequest

index = await project.create_index(
    FileSystemIndexCreateRequest(
        name="financial-reports",
        model="openai/gpt-5.2",  # used to generate folder/file summaries
        file_summarization_prompt="<prompt for summarizing each file>",
        folder_summarization_prompt="<prompt for summarizing each folder>",
    )
)

print("Status:", index.status)   # PENDING, READY, UPDATING, or FAILED
print("Nodes:", len(index.nodes))
```

***

## Adding Files

`add_files` starts an async job and blocks until it completes. Adding files is best-effort per file, so the result reports which files succeeded and which failed.

```python theme={null}
result = await index.add_files(
    file_ids=["file_1", "file_2", "file_3"]
)

print("Succeeded:", result.succeeded_file_ids)
for failed in result.failed_files:
    print(f"Failed {failed.file_id}: {failed.error}")

# Retry failed files
result = await index.add_files(
    file_ids=[failed.file_id for failed in result.failed_files]
)
```

To start the job without blocking, use `start_add_files_job`, which returns immediately so you can poll the job yourself:

```python theme={null}
job = await index.start_add_files_job(file_ids=["file_1", "file_2"])
print("Job started:", job.id)
```

***

## Searching

`search` runs BM25 over the leaf nodes (files) and returns the most relevant matches.

```python theme={null}
response = await index.search(query="quarterly revenue", top_k=5)

for node in response.nodes:
    print(f"[level {node.level}] {node.id}: {node.text[:80]}")
```

Restrict the search to a subtree by passing `current_node_id`:

```python theme={null}
response = await index.search(
    query="quarterly revenue",
    top_k=5,
    current_node_id="<node_id>",  # search only within this node's subtree
)
```

***

## Navigating the Hierarchy

`navigate` ranks the children of a node so you can walk the tree from broad summaries down to individual files. Pass `current_node_id=None` to start from the root.

```python theme={null}
response = await index.navigate(query="finance", current_node_id=None, top_k=5)

for node in response.nodes:
    print(f"[level {node.level}] {node.id}: {node.text[:80]}")
```

***

## Removing Files and Deleting

Removing files is all-or-nothing: on success, every requested file is removed from the index.

```python theme={null}
# Remove files (blocks until the job finishes)
await index.remove_files(file_ids=["file_1", "file_2"])

# Re-fetch the latest index state from the API
await index.refresh()

# Permanently delete the index and its storage
await index.delete()
```

***

## Listing and Retrieving Indexes

```python theme={null}
# List all indexes in the project
indexes = await project.list_indexes()

# Fetch a single index by ID (with hydrated nodes)
index = await project.get_index_by_id("index_123")
```

***

## Appendix: Essential Imports

```python theme={null}
from dex_sdk import DexClient, FileSystemIndexCreateRequest
```

***

## Next Steps

* **[Research](research)**: Run agentic research jobs that use indexes as input
* **[Vector Stores](vector-stores)**: Semantic and hybrid search for RAG-enhanced extraction
* **[Extract](extract)**: Extract structured data from parse results or vector stores


# Introduction to Dex
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/introduction-to-dex



**Dex is Scale's document understanding service that transforms unstructured documents into actionable, structured data.** It is a comprehensive platform that combines **advanced OCR, natural language processing, and machine learning** to extract meaningful information from PDFs, images, spreadsheets, and more.

***

## Why Use Dex?

Around 80-90% of enterprise data lives within unstructured formats such as PDFs and DOCX files. Dex solves the most common challenges of programmatic document processing:

* **Format Diversity:** Process any document type with a single API—business reports, financial documents, legal contracts, healthcare records, and more.
* **Unstructured Data:** Convert complex layouts into structured JSON with semantic understanding, including text, tables, charts, and infographics.
* **Quality Variations:** Handle scanned, handwritten, and low-quality documents with high accuracy across multiple languages.
* **Scalability:** Process thousands of documents efficiently with built-in scalable infrastructure.
* **Flexibility:** Choose from multiple OCR engines and customize extraction with your own tools and workflows.

***

## Core Primitives

Dex is designed as a capability rather than a standalone product, centered around composable primitives that can be used, extended, and combined:

### File Management

Upload, retrieve, and securely store confidential documents. Supports persistent storage with metadata tracking, secure access patterns, and configurable data retention policies for automatic lifecycle management.

### Parse

Convert documents into machine-readable formats using multiple OCR engines. Dex extracts:

* **Plain text** in multiple languages (English, Spanish, Arabic, German, and more)
* **Tables** including small and large tabular data (up to 500+ rows)
* **Checkboxes** for form processing
* **Images and figures** with bounding box information
* **Charts** for data visualization analysis

### Vector Stores

Vectorize and index parsed documents for semantic search and retrieval. Vector stores enable:

* **Semantic search** over document chunks with embedding-based similarity
* **Context management** for multi-file processing and large documents
* **Regex search** for pattern-based extraction (dates, IDs, emails, etc.)
* **Document summarization** for quick overview generation

### Extract

Extract structured data from parse results or document collections using:

* **Custom schemas** defined with Pydantic models
* **Natural language prompts** to guide extraction
* **Citations** that link extracted data to source locations
* **Confidence scores** for quality assessment
* **RAG-enhanced extraction** using vector store context
* **Agentic extraction** with custom MCP tools for advanced workflows

***

## Ways to Interact with Dex

Dex provides multiple interfaces to support different use cases:

* **REST API:** OpenAPI-documented endpoints for direct integration
* **Python SDK:** High-level async wrapper for rapid development
* **MCP Server:** Model Context Protocol integration for agent-based workflows (coming soon...)

***

## Common Use Cases

* **Financial Services:** Automate invoice processing, tax document analysis, and financial report extraction.
* **Healthcare:** Extract patient information from medical records, insurance claims, and healthcare forms.
* **Legal:** Analyze contracts, process discovery documents, and extract key clauses and obligations.
* **Business Operations:** Process HR documents, supply chain orders, customer service tickets, and business reports.

***

## Understanding Industry Document Challenges

Different industries face unique document processing challenges based on their document types and layouts. For a comprehensive overview of typical document formats and layout challenges across finance, healthcare, insurance, and legal sectors, see [Industry Document Types and Layout Challenges](https://llms.reducto.ai/industry-document-types-and-layout-challenges).

This guide covers:

* **Finance:** SEC filings, research reports, and financial statements with multi-column layouts, complex footnotes, and embedded visualizations
* **Healthcare:** Medical records and clinical documentation with handwritten elements, scanned materials, and variable form structures
* **Insurance:** Claims forms (CMS-1500, UB-04) combining typed prompts with handwritten responses on poor-quality scans
* **Legal:** Contracts and court filings requiring hierarchical structure preservation through complex sections and redlined annotations

Understanding these document-specific challenges can help you optimize your Dex configuration for better extraction accuracy and results.

***

## Language Support

Dex supports multi-language document processing with good support for germanic languages. For non-germanic, there are 35 languages including but not limited to:
Afrikaans: 🇿🇦 - Albanian: 🇦🇱 - Arabic: 🇸🇦 - Armenian: 🇦🇲 - Belarusian: 🇧🇾 - Bengali: 🇧🇩 - Bulgarian: 🇧🇬 - Catalan: 🇪🇸 - Chinese: 🇨🇳 - Croatian: 🇭🇷 - Czech: 🇨🇿 - Danish: 🇩🇰 - Dutch: 🇳🇱 - English: 🇬🇧 - Estonian: 🇪🇪 - Filipino: 🇵🇭 - Finnish: 🇫🇮 - French: 🇫🇷 - German: 🇩🇪 - Greek: 🇬🇷 - Gujarati: 🇮🇳 - Hebrew: 🇮🇱 - Hindi: 🇮🇳 - Hungarian: 🇭🇺 - Icelandic: 🇮🇸 - Indonesian: 🇮🇩 - Italian: 🇮🇹 - Japanese: 🇯🇵 - Kannada: 🇮🇳 - Khmer: 🇰🇭 - Korean: 🇰🇷 - Lao: 🇱🇦 - Latvian: 🇱🇻 - Lithuanian: 🇱🇹 - Macedonian: 🇲🇰 - Malay: 🇲🇾 - Malayalam: 🇮🇳 - Marathi: 🇮🇳 - Nepali: 🇳🇵 - Norwegian: 🇳🇴 - Persian: 🇮🇷 - Polish: 🇵🇱 - Portuguese: 🇵🇹 - Punjabi: 🇮🇳 - Romanian: 🇷🇴 - Russian: 🇷🇺 - Serbian: 🇷🇸 - Slovak: 🇸🇰 - Slovenian: 🇸🇮 - Spanish: 🇪🇸 - Swedish: 🇸🇪 - Tagalog: 🇵🇭 - Tamil: 🇮🇳 - Telugu: 🇮🇳 - Thai: 🇹🇭 - Turkish: 🇹🇷 - Ukrainian: 🇺🇦 - Vietnamese: 🇻🇳 - Yiddish: 🇮🇱

***

## Key Features

### Citations and Traceability

Every extracted field can be associated with its source location (page number, bounding box, text snippet), enabling auditability and human review.

### Confidence Scoring

Assigns confidence scores to extracted fields based on model outputs, helping you filter and prioritize results for downstream review.

### Flexible OCR Engine Support

Choose from multiple OCR engines:

* **Iris**: See **[When to choose Iris?](/docs/capabilities/ocr/when-to-choose-iris)**.
* **Reducto**: see documentation [here](https://docs.reducto.ai/overview)

### Data Lifecycle Management

Configurable retention policies automatically manage the lifecycle of files and processing artifacts, helping you meet compliance requirements and optimize storage costs.

***

## Infrastructure and Deployment

### Core Components

Dex is deployed as a Kubernetes service with the following components:

**Application Pods:**

* **API Pod**: Handles REST API requests for document processing
* **Temporal Worker Pod**: Processes document understanding workflows
* Both use the same Docker image with different entry points

**Infrastructure Services:**

* **Temporal Server**: Workflow orchestration (v1.25.0)
* **Postgres Database**: Application data and Temporal state (Postgres 17)
* **Object Storage**: S3, MinIO, or Azure Blob Storage for document artifacts

**Deployment:**

* Deployed via Helm charts on Kubernetes
* Supported on AWS, Azure, and GCP
* Requires namespace and service configuration

### GPU and Hardware Requirements

GPU requirements depend on which OCR engine and extraction models you use.

**OCR Engine Options:**

**Cloud Provider OCR (No provisioned GPU):**

* AWS Textract
* Azure Vision Read
* GCP Vision API

**Iris:**

* For custom OCR needs with 15+ configurable models. See **[When to choose Iris?](/docs/capabilities/ocr/when-to-choose-iris)**.

**Reducto Local OCR (Soon to be deprecated):**

* **Small (CPU)**: No GPU. Germanic languages. \~3s/page latency.
* **Medium**: A10G GPU. Germanic languages.
* **Large**: 8xH100 GPUs. 40+ languages (no Arabic RTL, degraded CJK).

**Extraction, Key-Value, Tables:**

* Azure OpenAI or Claude via Bedrock (no GPU, recommended)
* Hosted H100 (slower auto-scaling)

### Recommended Configuration

**For minimal GPU requirements:**

* OCR: Reducto Small (CPU) or Azure Vision Read
* Extraction: Azure OpenAI or Claude via AWS Bedrock
* Infrastructure: Kubernetes cluster with Postgres and object storage

***

## Getting Started

To begin using Dex, you'll need a Scale account with SGP access.

**Quick Links:**

* **[Getting Started Guide](/docs/capabilities/document-understanding/getting-started-with-dex)**: Step-by-step tutorial for your first extraction
* **[File Management](/docs/capabilities/document-understanding/file-management)**: Upload, pagination, and file types
* **[Parse](/docs/capabilities/document-understanding/parse)**: Parse engines and async jobs
* **[Chunking](/docs/capabilities/document-understanding/chunking)**: Chunking strategies
* **[Vector Stores](/docs/capabilities/document-understanding/vector-stores)**: Semantic search and RAG
* **[Extract](/docs/capabilities/document-understanding/extract)**: Data extraction and batch processing
* **[Best Practices](/docs/capabilities/document-understanding/best-practices)**: Quick start and optimization
* **[API Reference](/docs/capabilities/document-understanding/dex-sdk-api-reference)**: Complete SDK documentation

Dex makes document understanding accessible to developers while delivering the power and accuracy required for production applications.


# Parse
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/parse

Parse documents with Iris, monitor async jobs, and handle multi-language documents.

Convert documents to structured format for extraction. Dex supports multiple parse engines and async job monitoring.

***

## Parse Document (Default)

```python theme={null}
parse_result = await dex_file.parse(
    IrisParseJobParams(
        engine=ParseEngine.IRIS,
        options=IrisParseEngineOptions(
            layout="rt_detr_bce",
            text_ocr="openai/gpt-5.4",
            table_ocr="openai/gpt-5.4",
            confidence_threshold=0.5,
            img_method="description",
        ),
    )
)
```

***

### Which OCR Engine?

See **[When to choose Iris?](/docs/capabilities/ocr/when-to-choose-iris)** for custom OCR needs.

***

## Async Job Monitoring

**New in v0.4.0**: Use `start_parse_job` for better control over async operations and access to SGP traces.

### Monitor Parse Job

```python theme={null}
import asyncio
from dex_sdk.types import (
    ParseEngine,
    IrisParseJobParams,
    IrisParseEngineOptions,
)

# Start a parse job (returns immediately)
parse_job = await project.start_parse_job(
    dex_file=dex_file,
    parameters=IrisParseJobParams(
        engine=ParseEngine.IRIS,
        options=IrisParseEngineOptions(
            layout="rt_detr_bce"
        )
    )
)

# Monitor job progress
while parse_job.data.status not in [JobStatus.SUCCEEDED, JobStatus.FAILED]:
    await asyncio.sleep(1)
    await parse_job.refresh()
    print(f"Job status: {parse_job.data.status}")

# Get result
if parse_job.data.status == JobStatus.SUCCEEDED:
    parse_result = await parse_job.get_result()
    print("Parse completed successfully")
else:
    print(f"Parse failed: {parse_job.data.error_message}")
```

### Retrieving SGP Traces for Debugging

```python theme={null}
from scale_gp_beta import SGPClient

sgp_client = SGPClient(
    api_key=os.getenv("SGP_API_KEY"),
    account_id=os.getenv("SGP_ACCOUNT_ID"),
)

# Search for job traces
spans = list(sgp_client.spans.search(
    sort_by="created_at",
    sort_order="desc",
    extra_metadata={"job_id": parse_job.data.id},
    parents_only=True,
))

if spans:
    trace_id = spans[0].trace_id
    all_spans = list(sgp_client.spans.search(trace_ids=[trace_id]))
    for span in all_spans:
        print(f"Span: {span.name}, Duration: {span.duration_ms}ms")
```

***

## Process Multiple Files

```python theme={null}
import asyncio
from dex_sdk.types import (
    ParseEngine,
    IrisParseJobParams,
    IrisParseEngineOptions,
)

# Parse all files in parallel
parse_tasks = [
    dex_file.parse(
        IrisParseJobParams(
            engine=ParseEngine.IRIS,
            options=IrisParseEngineOptions(
                layout="rt_detr_bce",  # native region-level chunks (IRIS v2 default)
            ),
        )
    )
    for dex_file in dex_files
]
parse_results = await asyncio.gather(*parse_tasks)
print(f"Parsed {len(parse_results)} documents")
```

***

## Multi-Language Support

Dex supports 35+ languages with automatic language detection.

### Supported Languages

Germanic languages have excellent support. Additional 35+ languages include:

* European: Spanish, French, German, Italian, Portuguese, Dutch, Swedish, Norwegian, Danish, Polish, Russian, Ukrainian
* Asian: Chinese, Japanese, Korean, Thai, Vietnamese, Khmer, Lao
* Middle Eastern: Arabic, Hebrew, Persian, Turkish
* Indian: Hindi, Bengali, Tamil, Telugu, Malayalam, Kannada, Gujarati, Marathi, Punjabi
* And many more...

See the [Introduction guide](introduction-to-dex#language-support) for the complete list.

***

## Next Steps

* **[Chunking](chunking)**: Choose chunking strategies for your documents
* **[Extract](extract)**: Extract structured data from parse results
* **[Vector Stores](vector-stores)**: Use vector stores for large documents


# Research
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/research

Run agentic research jobs over parse results, vector stores, and indexes to answer complex questions with citations.

Research runs an agentic job that answers complex questions over your documents. It decomposes the task, searches and cites your sources, aggregates findings, and validates the output. Inputs are scoped to [parse results](parse), [vector stores](vector-stores), and/or [indexes](indexing), and results come back as structured fields—or a markdown report—each backed by citations and usage metrics.

<Note>
  Research is a newly released Dex capability. The API may change in future SDK versions.
</Note>

***

## Prerequisites

A research job needs at least one source to search. Before running a job, prepare one or more of:

* **[Parse results](parse)** — parsed documents in your project
* **A [vector store](vector-stores)** — for semantic/hybrid chunk search (at most one per job)
* **An [index](indexing)** — for locating relevant files in a corpus or files which are not ingested in the vector store (at most one per job)

***

## Running a Research Job

Build a `ResearchJobRequest` describing the inputs and the task, start the job, then poll for the result.

```python theme={null}
from dex_sdk import ResearchInputs, ResearchJobRequest, ResearchTask

req = ResearchJobRequest(
    inputs=ResearchInputs(
        vector_stores=[vector_store.id],
        indexes=[index.id],
    ),
    task=ResearchTask(
        description="Between 'UK - DP' and 'DACH - DP', which had higher cash income in FY 2016?",
        output_schema=MyOutputSchema.model_json_schema(),
    ),
    model="openai/gpt-5.2",
)

# Start the job
job = await project.start_research_job(req)
print("Research job:", job.id)

# Poll to completion and get the typed result
res = await job.get_result(max_attempts=1000)
```

***

## Reading Results

The result's `data` maps each output field to a value, confidence, and citations. Citations are a discriminated union—use `citation.kind` to tell block citations (from PDFs) from spreadsheet citations.

```python theme={null}
for name, field in res.data.items():
    print(f"{name}: {field.value} (confidence={field.confidence})")
    for citation in field.citations:
        if citation.kind == "block":
            print(f"  [block] file={citation.file_id} page={citation.page} bbox={citation.bbox}")
        else:  # "spreadsheet"
            print(f"  [spreadsheet] file={citation.file_id} sheet={citation.sheet} cell={citation.location}")

# Other details
print("Total tokens used:", res.usage_info.total_tokens)
print("Research trace:", res.trace)
```

***

## Structured vs. Report Output

Choose the output shape on the `ResearchTask`:

* **Structured** — set `output_schema` to a JSON Schema. Results populate `res.data` with one entry per schema field.
* **Report** — set `report_schema` to a markdown template. Access the rendered report with the `res.report` shortcut.

```python theme={null}
task = ResearchTask(
    description="Summarize FY2016 segment performance.",
    report_schema="## Summary\n{summary}\n\n## Key figures\n{figures}",
)
# ...after the job completes:
print(res.report)  # markdown string, or None for structured runs
```

***

## Advanced Configuration

For finer control, configure tools and per-stage behavior.

**Per-tool config** — enable/disable tools and pass tool-specific options (e.g. `top_k`, filters, rerank config):

```python theme={null}
from dex_sdk.types import ResearchToolConfigEntry

req = ResearchJobRequest(
    inputs=ResearchInputs(vector_stores=[vector_store.id], indexes=[index.id]),
    task=ResearchTask(description="..."),
    model="openai/gpt-5.2",
    tools={
        # Vector store search tools: similarity_search, lexical_search, hybrid_search.
        # Index search tools: search_index, navigate_index.
        "similarity_search": ResearchToolConfigEntry(
            enabled=True,
            config={"top_k": 10},
        ),
        "lexical_search": ResearchToolConfigEntry(enabled=False, config={}),
    },
)
```

**Per-stage overrides** — override the prompt, model, temperature, or reasoning effort for any stage of the agentic pipeline (decomposition, subtasks, aggregation, output validation):

```python theme={null}
from dex_sdk.types import (
    ResearchAdvanced,
    ResearchAdvancedStep,
    ResearchReasoningBlock,
)

req.advanced = ResearchAdvanced(
    aggregation=ResearchAdvancedStep(
        prompt="<override prompt for the aggregation step>",
        model="openai/gpt-5.4",
        temperature=0.0,
        reasoning=ResearchReasoningBlock(effort="high"),
    ),
)
```

Additional flags on the request:

* `enable_subtable_citations` — cite specific table cells/ranges rather than whole tables.

***

## Listing Research Jobs

```python theme={null}
jobs = await project.list_research_jobs()
for job in jobs:
    print(job.id)
```

***

## Appendix: Essential Imports

```python theme={null}
from dex_sdk import DexClient, ResearchInputs, ResearchJobRequest, ResearchTask
```

The advanced configuration types (`ResearchToolConfigEntry`, `ResearchAdvanced`, `ResearchAdvancedStep`, `ResearchReasoningBlock`, and the result types `ResearchResult`, `BlockCitation`, `SpreadsheetCitation`) are re-exported from `dex_sdk.types`.

***

## Next Steps

* **[Indexing](indexing)**: Build a hierarchical index to use as a research input
* **[Vector Stores](vector-stores)**: Semantic and hybrid search for RAG-enhanced extraction
* **[Extract](extract)**: Extract structured data from a single parse result or vector store


# Vector Stores
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/vector-stores

Create vector stores for semantic search and RAG-enhanced extraction from large or multi-file documents.

Vector stores enable semantic search and RAG-enhanced extraction for large documents or multi-file processing.

***

## When to Use Vector Stores?

```
Is your document > 50 pages?
├─ Yes → Use vector store
└─ No → Do you have multiple documents to query together?
   ├─ Yes → Use vector store
   └─ No → Extract directly from parse result
```

***

## Vector Store Engines

Dex supports two vector store engines. Prefer `SGP_VECTOR_STORE` for new implementations.

| Engine               | Best For                                                  | Embedding Options                    | Search Features                                      |
| -------------------- | --------------------------------------------------------- | ------------------------------------ | ---------------------------------------------------- |
| `SGP_VECTOR_STORE`   | **Recommended.** Document RAG, advanced search, reranking | Base embedding models                | Semantic, lexical, hybrid search; optional reranking |
| `SGP_KNOWLEDGE_BASE` | Legacy only                                               | Base models or Models API deployment | Semantic search                                      |

<Warning>
  You must update your SGP platform to support the SGP Vector Store engine. If `SGP_VECTOR_STORE` is not available, use `SGP_KNOWLEDGE_BASE` as a fallback.
</Warning>

### SGP Vector Store — Recommended

The SGP Vector Store engine is the preferred vector store implementation. Use it for:

* Document RAG workflows within Dex
* Lexical (keyword) or hybrid search in addition to semantic search
* Reranking search results for higher relevance

**Embedding configuration:** Base embedding models only (e.g., `openai/text-embedding-3-large`, `sentence-transformers/all-mpnet-base-v2`).

**Search features:**

* **Query types:** `semantic`, `lexical`, or `hybrid`
* **Reranking:** Optional `rerank_model` and `rerank_top_k` for improved result ordering

### SGP Knowledge Base — Legacy

The SGP Knowledge Base engine is SGP's legacy vector store implementation. Its use is discouraged for new projects. Consider it only when:

* You need integration with SGP data connectors for ingesting from external sources
* You must use a custom embedding model deployed via the Models API (`embedding_type="models_api"`)
* Your SGP platform does not yet support SGP Vector Store

**Embedding configuration:**

* **Base models** (`embedding_type="base"`): Use pre-configured embedding models such as `openai/text-embedding-3-large`
* **Models API** (`embedding_type="models_api"`): Use a custom model deployment via `model_deployment_id`

**Search:** Semantic search only. Reranking and query type options are not supported.

<Note>
  `create_vector_store` returns `DexSGPKnowledgeBase` for `SGP_KNOWLEDGE_BASE` and `DexSGPVectorStore` for `SGP_VECTOR_STORE`. Use `DexSGPVectorStore.get_by_id` only for SGP Vector Store IDs—it will raise if given a Knowledge Base ID.
</Note>

***

## Creating a Vector Store

```python theme={null}
from dex_sdk.types import VectorStoreEngines

# Create an SGP Vector Store (recommended)
vs_store = await project.create_vector_store(
    name="Financial Documents Store",
    engine=VectorStoreEngines.SGP_VECTOR_STORE,
    embedding_model="openai/text-embedding-3-large",
)

# Or create a Knowledge Base vector store (legacy; for data connector integration)
kb_store = await project.create_vector_store(
    name="Financial Documents Store",
    engine=VectorStoreEngines.SGP_KNOWLEDGE_BASE,
    embedding_model="openai/text-embedding-3-large",
)
```

***

## Adding Documents to Vector Store

```python theme={null}
# Add parsed files to the vector store
await vector_store.add_parse_results([parse_result.id])
print("Parse results added to vector store")
```

***

## Semantic Search

```python theme={null}
# Perform semantic search across all documents
search_results = await vector_store.search(
    query="What is the total taxable income?",
    top_k=5,
)

# Search within a specific file
file_search_results = await vector_store.search_in_file(
    file_id=dex_file.id,
    query="What is the total taxable income?",
    top_k=5,
    filters=None,  # Optional filters like {"file_id": "file_123"}
)

# Access search results
for chunk in search_results.chunks:
    print(f"Score: {chunk.score:.2f}")
    print(f"Content: {chunk.content[:100]}...")
    print(f"File: {chunk.file_id}, Page: {chunk.blocks[0].page_number if chunk.blocks else 'N/A'}")

    # Access additional metadata from the chunk
    if chunk.metadata:
        print(f"Metadata: {chunk.metadata}")
```

SGP Vector Store only: use `query_type` and reranking for improved results:

```python theme={null}
# Hybrid search with reranking (SGP Vector Store only)
search_results = await vs_store.search(
    query="What is the total taxable income?",
    top_k=10,
    query_type="hybrid",
    rerank_model="cohere.rerank-english-v3.0",
    rerank_top_k=5,
)
```

***

## Custom Metadata Schema

You can define custom metadata fields on your vector store and attach metadata to each parse result. This metadata is indexed and filterable when searching, so you can narrow results by document attributes such as department, document type, or priority.

**1. Define the schema when creating the vector store**

Specify `vector_store_metadata_schema` as a dict mapping field names to types: `"string"`, `"int"`, `"double"`, or `"boolean"`.

```python theme={null}
vector_store = await project.create_vector_store(
    name="Documents by Department",
    engine=VectorStoreEngines.SGP_VECTOR_STORE,
    embedding_model="openai/text-embedding-3-large",
    vector_store_metadata_schema={
        "department": "string",
        "priority": "int",
        "is_public": "boolean",
    },
)
```

**2. Attach metadata when parsing**

Pass `vector_store_metadata` in your parse job parameters. The metadata is stored with the parse result and indexed when you add it to the vector store.

```python theme={null}
from dex_sdk.types import ParseEngine, IrisParseJobParams, IrisParseEngineOptions

parse_result = await dex_file.parse(
    IrisParseJobParams(
        engine=ParseEngine.IRIS,
        options=IrisParseEngineOptions(
            layout="rt_detr_bce",
            text_ocr="openai/gpt-5.4",
            table_ocr="openai/gpt-5.4",
            confidence_threshold=0.5,
            img_method="description",
        ),
        vector_store_metadata={
            "department": "legal",
            "priority": 1,
            "is_public": False,
        },
    )
)
await vector_store.add_parse_results([parse_result.id])
```

For custom parse results, pass `vector_store_metadata` when creating the parse result via `DexParseResult.from_custom_results`.

**3. Filter on custom metadata when searching**

Use the `filters` parameter in search. Filter format depends on the engine:

| Operation                  | SGP Vector Store | SGP Knowledge Base | Example                                                                           |
| -------------------------- | ---------------- | ------------------ | --------------------------------------------------------------------------------- |
| Equal                      | `$eq`            | `eq`               | `{"department": {"$eq": "legal"}}`                                                |
| Not equal                  | `$ne`            | `ne`               | `{"department": {"$ne": "archived"}}`                                             |
| Greater than               | `$gt`            | `gt`               | `{"priority": {"$gt": 1}}`                                                        |
| Greater than or equal      | `$gte`           | `gte`              | `{"priority": {"$gte": 1}}`                                                       |
| Less than                  | `$lt`            | `lt`               | `{"priority": {"$lt": 5}}`                                                        |
| Less than or equal         | `$lte`           | `lte`              | `{"priority": {"$lte": 5}}`                                                       |
| In (list of values)        | `$in`            | `in`               | `{"department": {"$in": ["legal", "finance"]}}`                                   |
| Not in (list of values)    | `$nin`           | `nin`              | `{"department": {"$nin": ["archived"]}}`                                          |
| And (combine conditions)   | `$and`           | `and`              | `{"$and": [{"department": {"$eq": "legal"}}, {"priority": {"$gte": 1}}]}`         |
| Or (any condition matches) | `$or`            | `or`               | `{"$or": [{"department": {"$eq": "legal"}}, {"department": {"$eq": "finance"}}]}` |
| Not (negate condition)     | `$not`           | `not`              | `{"department": {"$not": {"$eq": "archived"}}}`                                   |

```python theme={null}
# Filter by department (SGP Vector Store)
search_results = await vector_store.search(
    query="contract terms",
    top_k=10,
    filters={"department": {"$eq": "legal"}},
)

# Filter by multiple values (SGP Vector Store)
search_results = await vector_store.search(
    query="quarterly report",
    top_k=10,
    filters={"department": {"$in": ["legal", "finance"]}},
)

# Filter by numeric value (SGP Vector Store)
search_results = await vector_store.search(
    query="policy",
    top_k=10,
    filters={"priority": {"$gte": 1}},
)

# Same filter on SGP Knowledge Base — operators have no $ prefix
search_results = await knowledge_base.search(
    query="contract terms",
    top_k=10,
    filters={"department": {"eq": "legal"}},
)
```

<Note>
  The schema must be defined when creating the vector store. Fields in `vector_store_metadata` that are not in `vector_store_metadata_schema` are not indexed and cannot be used for filtering.
</Note>

***

## RAG-Enhanced Extraction

Extract data using vector store context for improved accuracy on large documents:

```python theme={null}
# Extract from vector store with RAG context
extract_result = await vector_store.extract(
    extraction_schema=FinancialData,
    user_prompt="""Extract financial data from all documents in this store.
    Focus on quarterly income statements.""",
    model="openai/gpt-5.4",
    generate_citations=True,
    generate_confidence=True,
)

# Access extracted data with citations
result = extract_result.result
for field_name, field in result.data.items():
    print(f"{field_name}: {field.value} (confidence: {field.confidence:.2f})")
    if field.citations:
        for cite in field.citations:
            print(f"  → From {cite.file_id}, page {cite.page}")
```

### Pattern: RAG for Large Documents

```python theme={null}
# 1. Create vector store
vs = await project.create_vector_store(
    name="Store",
    engine=VectorStoreEngines.SGP_VECTOR_STORE,
    embedding_model="openai/text-embedding-3-large",
)

# 2. Add documents
await vs.add_parse_results([parse_result.id])

# 3. Extract with context
result = await vs.extract(
    extraction_schema=Schema,
    user_prompt="Extract from all docs",
    model="openai/gpt-5.4",
)
```

***

## Next Steps

* **[Extract](extract)**: Extract structured data from parse results or vector stores
* **[Chunking](chunking)**: Optimize chunking for vector store embeddings
* **[API Reference](dex-sdk-api-reference)**: Complete SDK documentation


# SGP Capabilities Overview
Source: https://docs.gp.scale.com/docs/capabilities/overview



SGP provides a comprehensive suite of capabilities for building, deploying, and managing AI applications at scale. Our platform combines powerful tools for development, evaluation, and production deployment.

### 📄 Document Understanding

Advanced document processing and information extraction capabilities. [Learn more about Document Understanding →](/docs/capabilities/document-understanding/introduction-to-dex)

**Key Features:**

* **Universal Format Support**: PDFs, images, spreadsheets, presentations
* **Intelligent Parsing**: Convert documents to structured JSON
* **Custom Extraction**: Define schemas for specific data needs
* **High Accuracy OCR**: Advanced text recognition and extraction
* **Project Management**: Secure, isolated document processing

### ⚙️ Workflows

No-code/low-code durable workflow orchestration platform that enables enterprise operators to import data, transform it, call LLMs or Agents, analyze or evaluate the output, and automate the entire workflow. [Learn more about Workflows →](/docs/capabilities/workflows/introduction-to-workflows)

**Key Features:**

* **Importing data**: Read data from various sources like agent traces, CSVs, blob storages, or queryable data sources.
* **Transforming data**: Transform your data using custom `Python` code or simple functions. Extract structured data from PDFs & documents.
* **Call LLMs or Agents**: Generate output and traces against input from each row of your dataset using LLMs or `Agentex` agents.
* **Evaluate data**: Evaluate your outputs using `LLM as Judge` and access the evaluation anytime on SGP.
* **Visualize data**: Generate visualizations like charts & build custom dashboards.
* **Export the data**: Export the data as a CSV or SGP dataset or SGP Evaluation.
* **Automate & monitor**: Schedule the workflow to run periodically. Execute a workflow via an API call. Monitor logs & get alerts.

### Training

Train custom models on your own data without managing cloud infrastructure. Submit training jobs through a single API and Train handles the rest, whether your workload runs on GCP, AWS, or Azure. [Learn more about Training](/docs/capabilities/training/introduction-to-train)

**Key Features:**

* **Any cloud backend**: Run jobs on Vertex AI, SageMaker, or Azure ML with a single API
* **Bring your own image**: Push a Docker image once and reference it in any job
* **Portable data access**: Mount GCS, S3, or Azure Blob Storage into your container using consistent environment variables
* **Simple job lifecycle**: Track progress from submission to completion with a consistent status model across all backends


# Create Api Key
Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/create-api-key

https://agentex.dev-sgp.scale.com/openapi.json post /agent_api_keys



# Create Webhook Trigger
Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/create-webhook-trigger

https://agentex.dev-sgp.scale.com/openapi.json post /agent_api_keys/webhook-trigger
Wire a webhook trigger in one call.

Registers the source's signature-verification key (github/slack) for the agent and
returns the ready-to-paste forward webhook URL plus the signing secret (shown once).
The webhook then flows through the existing /agents/forward ingress, which verifies
the signature against this key. Bundles the existing key-create + URL composition so
a UI (or a curl) can set up a trigger without two steps.



# Delete API key by ID
Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/delete-api-key-by-id

https://agentex.dev-sgp.scale.com/openapi.json delete /agent_api_keys/{id}
Delete API key by ID.



# Delete API key by name
Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/delete-api-key-by-name

https://agentex.dev-sgp.scale.com/openapi.json delete /agent_api_keys/name/{api_key_name}
Delete API key by name.



# List API keys for an agent ID
Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/list-api-keys-for-an-agent-id

https://agentex.dev-sgp.scale.com/openapi.json get /agent_api_keys
List API keys for an agent ID.



# Return named API key for the agent ID
Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/return-named-api-key-for-the-agent-id

https://agentex.dev-sgp.scale.com/openapi.json get /agent_api_keys/name/{name}
Return named API key for the agent ID.



# Return the API key by ID
Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/return-the-api-key-by-id

https://agentex.dev-sgp.scale.com/openapi.json get /agent_api_keys/{id}
Return API key by ID.



# Get Agent Task Tracker
Source: https://docs.gp.scale.com/reference/agentex/agent-task-tracker/get-agent-task-tracker

https://agentex.dev-sgp.scale.com/openapi.json get /tracker/{tracker_id}
Get agent task tracker by tracker ID



# List Agent Task Trackers
Source: https://docs.gp.scale.com/reference/agentex/agent-task-tracker/list-agent-task-trackers

https://agentex.dev-sgp.scale.com/openapi.json get /tracker
List all agent task trackers, optionally filtered by query parameters.



# Update Agent Task Tracker
Source: https://docs.gp.scale.com/reference/agentex/agent-task-tracker/update-agent-task-tracker

https://agentex.dev-sgp.scale.com/openapi.json put /tracker/{tracker_id}
Update agent task tracker by tracker ID



# Delete Agent by ID
Source: https://docs.gp.scale.com/reference/agentex/agents/delete-agent-by-id

https://agentex.dev-sgp.scale.com/openapi.json delete /agents/{agent_id}
Delete an agent by its unique ID.



# Delete Agent by Name
Source: https://docs.gp.scale.com/reference/agentex/agents/delete-agent-by-name

https://agentex.dev-sgp.scale.com/openapi.json delete /agents/name/{agent_name}
Delete an agent by its unique name.



# Forward GET request to agent by name
Source: https://docs.gp.scale.com/reference/agentex/agents/forward-get-request-to-agent-by-name

https://agentex.dev-sgp.scale.com/openapi.json get /agents/forward/name/{agent_name}/{path}
Forward a GET request to an agent by its name.



# Forward POST request to agent by name
Source: https://docs.gp.scale.com/reference/agentex/agents/forward-post-request-to-agent-by-name

https://agentex.dev-sgp.scale.com/openapi.json post /agents/forward/name/{agent_name}/{path}
Forward a POST request to an agent by its name.



# Get Agent by ID
Source: https://docs.gp.scale.com/reference/agentex/agents/get-agent-by-id

https://agentex.dev-sgp.scale.com/openapi.json get /agents/{agent_id}
Get an agent by its unique ID.



# Get Agent by Name
Source: https://docs.gp.scale.com/reference/agentex/agents/get-agent-by-name

https://agentex.dev-sgp.scale.com/openapi.json get /agents/name/{agent_name}
Get an agent by its unique name.



# Handle Agent RPC by ID
Source: https://docs.gp.scale.com/reference/agentex/agents/handle-agent-rpc-by-id

https://agentex.dev-sgp.scale.com/openapi.json post /agents/{agent_id}/rpc
Handle JSON-RPC requests for an agent by its unique ID.



# Handle Agent RPC by Name
Source: https://docs.gp.scale.com/reference/agentex/agents/handle-agent-rpc-by-name

https://agentex.dev-sgp.scale.com/openapi.json post /agents/name/{agent_name}/rpc
Handle JSON-RPC requests for an agent by its unique name.



# List Agents
Source: https://docs.gp.scale.com/reference/agentex/agents/list-agents

https://agentex.dev-sgp.scale.com/openapi.json get /agents
List all registered agents, optionally filtered by query parameters.



# Register Agent
Source: https://docs.gp.scale.com/reference/agentex/agents/register-agent

https://agentex.dev-sgp.scale.com/openapi.json post /agents/register
Register a new agent or update an existing one.



# Register Build
Source: https://docs.gp.scale.com/reference/agentex/agents/register-build

https://agentex.dev-sgp.scale.com/openapi.json post /agents/register-build
Register an agent at build time, before it is deployed, so it can be permissioned and shared prior to deploy. Idempotent by name.



# Delete Thread
Source: https://docs.gp.scale.com/reference/agentex/checkpoints/delete-thread

https://agentex.dev-sgp.scale.com/openapi.json post /checkpoints/delete-thread



# Get Checkpoint Tuple
Source: https://docs.gp.scale.com/reference/agentex/checkpoints/get-checkpoint-tuple

https://agentex.dev-sgp.scale.com/openapi.json post /checkpoints/get-tuple



# List Checkpoints
Source: https://docs.gp.scale.com/reference/agentex/checkpoints/list-checkpoints

https://agentex.dev-sgp.scale.com/openapi.json post /checkpoints/list



# Put Checkpoint
Source: https://docs.gp.scale.com/reference/agentex/checkpoints/put-checkpoint

https://agentex.dev-sgp.scale.com/openapi.json post /checkpoints/put



# Put Writes
Source: https://docs.gp.scale.com/reference/agentex/checkpoints/put-writes

https://agentex.dev-sgp.scale.com/openapi.json post /checkpoints/put-writes



# Get Deployment by ID
Source: https://docs.gp.scale.com/reference/agentex/deployment-history/get-deployment-by-id

https://agentex.dev-sgp.scale.com/openapi.json get /deployment-history/{deployment_id}
Get a deployment record by its unique ID.



# List Deployments for an agent
Source: https://docs.gp.scale.com/reference/agentex/deployment-history/list-deployments-for-an-agent

https://agentex.dev-sgp.scale.com/openapi.json get /deployment-history
List deployment history for an agent.



# Create Deployment
Source: https://docs.gp.scale.com/reference/agentex/deployments/create-deployment

https://agentex.dev-sgp.scale.com/openapi.json post /agents/{agent_id}/deployments
Create a new deployment record in PENDING status.



# Delete Deployment
Source: https://docs.gp.scale.com/reference/agentex/deployments/delete-deployment

https://agentex.dev-sgp.scale.com/openapi.json delete /agents/{agent_id}/deployments/{deployment_id}
Delete a non-production deployment.



# Get Deployment
Source: https://docs.gp.scale.com/reference/agentex/deployments/get-deployment

https://agentex.dev-sgp.scale.com/openapi.json get /agents/{agent_id}/deployments/{deployment_id}
Get a specific deployment by ID.



# List Deployments
Source: https://docs.gp.scale.com/reference/agentex/deployments/list-deployments

https://agentex.dev-sgp.scale.com/openapi.json get /agents/{agent_id}/deployments
List deployments for an agent, newest first.



# Preview RPC
Source: https://docs.gp.scale.com/reference/agentex/deployments/preview-rpc

https://agentex.dev-sgp.scale.com/openapi.json post /agents/{agent_id}/deployments/{deployment_id}/rpc
Send an RPC request to a specific deployment (for preview testing).



# Promote Deployment
Source: https://docs.gp.scale.com/reference/agentex/deployments/promote-deployment

https://agentex.dev-sgp.scale.com/openapi.json post /agents/{agent_id}/deployments/{deployment_id}/promote
Promote a deployment to production with atomic cutover.



# Get Event
Source: https://docs.gp.scale.com/reference/agentex/events/get-event

https://agentex.dev-sgp.scale.com/openapi.json get /events/{event_id}



# List Events
Source: https://docs.gp.scale.com/reference/agentex/events/list-events

https://agentex.dev-sgp.scale.com/openapi.json get /events
List events for a specific task and agent.

Optionally filter for events after a specific sequence ID.
Results are ordered by sequence_id.



# Batch Create Messages
Source: https://docs.gp.scale.com/reference/agentex/messages/batch-create-messages

https://agentex.dev-sgp.scale.com/openapi.json post /messages/batch



# Batch Update Messages
Source: https://docs.gp.scale.com/reference/agentex/messages/batch-update-messages

https://agentex.dev-sgp.scale.com/openapi.json put /messages/batch



# Create Message
Source: https://docs.gp.scale.com/reference/agentex/messages/create-message

https://agentex.dev-sgp.scale.com/openapi.json post /messages



# Get Message
Source: https://docs.gp.scale.com/reference/agentex/messages/get-message

https://agentex.dev-sgp.scale.com/openapi.json get /messages/{message_id}



# List Messages
Source: https://docs.gp.scale.com/reference/agentex/messages/list-messages

https://agentex.dev-sgp.scale.com/openapi.json get /messages
List messages for a task with offset-based pagination.

For cursor-based pagination with infinite scroll support, use /messages/paginated.



# List Messages Paginated
Source: https://docs.gp.scale.com/reference/agentex/messages/list-messages-paginated

https://agentex.dev-sgp.scale.com/openapi.json get /messages/paginated
List messages for a task with cursor-based pagination.

This endpoint is designed for infinite scroll UIs where new messages may arrive
while paginating through older ones.

Args:
    task_id: The task ID to filter messages by
    limit: Maximum number of messages to return (default: 50)
    cursor: Opaque cursor string for pagination. Pass the `next_cursor` from
            a previous response to get the next page.
    direction: Pagination direction - "older" to get older messages (default),
               "newer" to get newer messages.

Returns:
    PaginatedMessagesResponse with:
    - data: List of messages (newest first when direction="older")
    - next_cursor: Cursor for fetching the next page (null if no more pages)
    - has_more: Whether there are more messages to fetch

Example:
    First request: GET /messages/paginated?task_id=xxx&limit=50
    Next page: GET /messages/paginated?task_id=xxx&limit=50&cursor=<next_cursor>



# Update Message
Source: https://docs.gp.scale.com/reference/agentex/messages/update-message

https://agentex.dev-sgp.scale.com/openapi.json put /messages/{message_id}



# Create Schedule
Source: https://docs.gp.scale.com/reference/agentex/schedules/create-schedule

https://agentex.dev-sgp.scale.com/openapi.json post /agents/{agent_id}/schedules
Create a new schedule for recurring workflow execution for an agent.



# Delete Schedule
Source: https://docs.gp.scale.com/reference/agentex/schedules/delete-schedule

https://agentex.dev-sgp.scale.com/openapi.json delete /agents/{agent_id}/schedules/{schedule_name}
Delete a schedule permanently.



# Get Schedule
Source: https://docs.gp.scale.com/reference/agentex/schedules/get-schedule

https://agentex.dev-sgp.scale.com/openapi.json get /agents/{agent_id}/schedules/{schedule_name}
Get details of a schedule by its name.



# List Agent Schedules
Source: https://docs.gp.scale.com/reference/agentex/schedules/list-agent-schedules

https://agentex.dev-sgp.scale.com/openapi.json get /agents/{agent_id}/schedules
List all schedules for an agent.



# Pause Schedule
Source: https://docs.gp.scale.com/reference/agentex/schedules/pause-schedule

https://agentex.dev-sgp.scale.com/openapi.json post /agents/{agent_id}/schedules/{schedule_name}/pause
Pause a schedule to stop it from executing.



# Trigger Schedule
Source: https://docs.gp.scale.com/reference/agentex/schedules/trigger-schedule

https://agentex.dev-sgp.scale.com/openapi.json post /agents/{agent_id}/schedules/{schedule_name}/trigger
Trigger a schedule to run immediately, regardless of its regular schedule.



# Unpause Schedule
Source: https://docs.gp.scale.com/reference/agentex/schedules/unpause-schedule

https://agentex.dev-sgp.scale.com/openapi.json post /agents/{agent_id}/schedules/{schedule_name}/unpause
Unpause/resume a schedule to allow it to execute again.



# Create Span
Source: https://docs.gp.scale.com/reference/agentex/spans/create-span

https://agentex.dev-sgp.scale.com/openapi.json post /spans
Create a new span with the provided parameters



# Get Span
Source: https://docs.gp.scale.com/reference/agentex/spans/get-span

https://agentex.dev-sgp.scale.com/openapi.json get /spans/{span_id}
Get a span by ID



# List Spans
Source: https://docs.gp.scale.com/reference/agentex/spans/list-spans

https://agentex.dev-sgp.scale.com/openapi.json get /spans
List spans, optionally filtered by trace_id and/or task_id



# Partial Update Span
Source: https://docs.gp.scale.com/reference/agentex/spans/partial-update-span

https://agentex.dev-sgp.scale.com/openapi.json patch /spans/{span_id}
Update a span with the provided output data and mark it as complete



# Create Task State
Source: https://docs.gp.scale.com/reference/agentex/states/create-task-state

https://agentex.dev-sgp.scale.com/openapi.json post /states



# Delete Task State
Source: https://docs.gp.scale.com/reference/agentex/states/delete-task-state

https://agentex.dev-sgp.scale.com/openapi.json delete /states/{state_id}



# Get State by State ID
Source: https://docs.gp.scale.com/reference/agentex/states/get-state-by-state-id

https://agentex.dev-sgp.scale.com/openapi.json get /states/{state_id}
Get a state by its unique state ID.



# List States
Source: https://docs.gp.scale.com/reference/agentex/states/list-states

https://agentex.dev-sgp.scale.com/openapi.json get /states
List all states, optionally filtered by query parameters.



# Update Task State
Source: https://docs.gp.scale.com/reference/agentex/states/update-task-state

https://agentex.dev-sgp.scale.com/openapi.json put /states/{state_id}



# Clean Task
Source: https://docs.gp.scale.com/reference/agentex/task-retention/clean-task

https://agentex.dev-sgp.scale.com/openapi.json post /tasks/{task_id}/clean
Delete content-bearing rows for a stale task.

Refuses on active tasks, in-flight workflows, or unprocessed events
regardless of `force`. The `force=true` flag only bypasses the
idle-threshold check.



# Export Task
Source: https://docs.gp.scale.com/reference/agentex/task-retention/export-task

https://agentex.dev-sgp.scale.com/openapi.json get /tasks/{task_id}/export
Build a self-contained snapshot of a task's content surfaces.

Returns the exact payload format that POST /rehydrate accepts, so
export → clean → rehydrate is a round-trip-equivalent operation.



# Export Task To Url
Source: https://docs.gp.scale.com/reference/agentex/task-retention/export-task-to-url

https://agentex.dev-sgp.scale.com/openapi.json post /tasks/{task_id}/export
Build the task snapshot and PUT it to a caller-supplied presigned URL.

Use this when the snapshot is too large for a JSON response body (long
conversations, deep reasoning content, many attachments). The upload URL
must be https and resolve to a public address — see SSRF guard.



# Rehydrate Task
Source: https://docs.gp.scale.com/reference/agentex/task-retention/rehydrate-task

https://agentex.dev-sgp.scale.com/openapi.json post /tasks/{task_id}/rehydrate
Restore content-bearing rows from a snapshot.

Two modes:
- Inline: caller provides messages and task_states in the request body.
- URL: caller provides snapshot_url; Agentex downloads and parses it.

Refuses if the task isn't currently in a cleaned state, or if any supplied
message/state ID already exists in Mongo (catches double-rehydrate).



# Cancel Task
Source: https://docs.gp.scale.com/reference/agentex/tasks/cancel-task

https://agentex.dev-sgp.scale.com/openapi.json post /tasks/{task_id}/cancel
Mark a running task as canceled.



# Complete Task
Source: https://docs.gp.scale.com/reference/agentex/tasks/complete-task

https://agentex.dev-sgp.scale.com/openapi.json post /tasks/{task_id}/complete
Mark a running task as completed.



# Delete Task by ID
Source: https://docs.gp.scale.com/reference/agentex/tasks/delete-task-by-id

https://agentex.dev-sgp.scale.com/openapi.json delete /tasks/{task_id}
Delete a task by its unique ID.



# Delete Task by Name
Source: https://docs.gp.scale.com/reference/agentex/tasks/delete-task-by-name

https://agentex.dev-sgp.scale.com/openapi.json delete /tasks/name/{task_name}
Delete a task by its unique name.



# Fail Task
Source: https://docs.gp.scale.com/reference/agentex/tasks/fail-task

https://agentex.dev-sgp.scale.com/openapi.json post /tasks/{task_id}/fail
Mark a running task as failed.



# Get Task by ID
Source: https://docs.gp.scale.com/reference/agentex/tasks/get-task-by-id

https://agentex.dev-sgp.scale.com/openapi.json get /tasks/{task_id}
Get a task by its unique ID.



# Get Task by Name
Source: https://docs.gp.scale.com/reference/agentex/tasks/get-task-by-name

https://agentex.dev-sgp.scale.com/openapi.json get /tasks/name/{task_name}
Get a task by its unique name.



# List Tasks
Source: https://docs.gp.scale.com/reference/agentex/tasks/list-tasks

https://agentex.dev-sgp.scale.com/openapi.json get /tasks
List all tasks.



# Query Task Workflow
Source: https://docs.gp.scale.com/reference/agentex/tasks/query-task-workflow

https://agentex.dev-sgp.scale.com/openapi.json get /tasks/{task_id}/query/{query_name}
Query a Temporal workflow associated with a task for its current state.



# Stream Task Events by ID
Source: https://docs.gp.scale.com/reference/agentex/tasks/stream-task-events-by-id

https://agentex.dev-sgp.scale.com/openapi.json get /tasks/{task_id}/stream
Stream events for a task by its unique ID.



# Stream Task Events by Name
Source: https://docs.gp.scale.com/reference/agentex/tasks/stream-task-events-by-name

https://agentex.dev-sgp.scale.com/openapi.json get /tasks/name/{task_name}/stream
Stream events for a task by its unique name.



# Terminate Task
Source: https://docs.gp.scale.com/reference/agentex/tasks/terminate-task

https://agentex.dev-sgp.scale.com/openapi.json post /tasks/{task_id}/terminate
Mark a running task as terminated.



# Timeout Task
Source: https://docs.gp.scale.com/reference/agentex/tasks/timeout-task

https://agentex.dev-sgp.scale.com/openapi.json post /tasks/{task_id}/timeout
Mark a running task as timed out.



# Update Task by ID
Source: https://docs.gp.scale.com/reference/agentex/tasks/update-task-by-id

https://agentex.dev-sgp.scale.com/openapi.json put /tasks/{task_id}
Update mutable fields for a task by its unique ID.



# Update Task by Name
Source: https://docs.gp.scale.com/reference/agentex/tasks/update-task-by-name

https://agentex.dev-sgp.scale.com/openapi.json put /tasks/name/{task_name}
Update mutable fields for a task by its unique Name.



# Create Deployment
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-deploy/create-deployment

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/agentex/deployments
Create a new deployment.

Submits a deployment request. The deployment will:
1. Parse and merge configuration files
2. Create a Kubernetes Job running helm install
3. Wait for deployment completion
4. Return results



# Delete Deployment
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-deploy/delete-deployment

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/agentex/deployments/{deployment_id}
Soft-delete a deployment. Marks it as deleted and triggers K8s resource cleanup.



# Get Deployment
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-deploy/get-deployment

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/agentex/deployments/{deployment_id}
Get a deployment by its ID.



# Get Deployment Logs
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-deploy/get-deployment-logs

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/agentex/deployments/{deployment_id}/logs
Get structured log lines for a deployment with cursor-based pagination.

The CLI can poll this endpoint to stream logs incrementally:
1. First call: no cursor
2. Subsequent calls: cursor=next_cursor from the previous response
3. Stop polling when the deployment reaches a terminal status



# List Deployments
Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-deploy/list-deployments

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/agentex/deployments
List all deployments with pagination and optional filters.



# Create Secret
Source: https://docs.gp.scale.com/reference/v5/sgp-cloud-secrets/create-secret

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json post /v5/sgp/secrets
Create an account-level secret.

The secret value is stored in the cloud provider's secret store.
SGP only stores metadata (key name, description, audit info).
The value is never returned by any API.
Returns 409 if a secret with the same key already exists.



# Delete Secret
Source: https://docs.gp.scale.com/reference/v5/sgp-cloud-secrets/delete-secret

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json delete /v5/sgp/secrets/{secret_id}
Delete a secret from both the cloud provider and SGP metadata.



# Get Secret
Source: https://docs.gp.scale.com/reference/v5/sgp-cloud-secrets/get-secret

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/sgp/secrets/{secret_id}
Get a single secret's metadata by ID. The value is never returned.



# List Secrets
Source: https://docs.gp.scale.com/reference/v5/sgp-cloud-secrets/list-secrets

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json get /v5/sgp/secrets
List secret metadata for the account. Values are never returned.



# Update Secret
Source: https://docs.gp.scale.com/reference/v5/sgp-cloud-secrets/update-secret

https://api.dev-sgp.scale.com/openapi-versions/v5/openapi.json patch /v5/sgp/secrets/{secret_id}
Update an existing secret's description and/or value.

If value is provided, the cloud provider secret is updated.
The secret value is never returned by any API.



# API Reference
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/api-reference

REST API reference for Document Understanding (Dex) — projects, files, parse, extract, and vector stores.

The Document Understanding API provides REST endpoints for document processing. Use the API for non-Python integrations or when you need direct control over HTTP requests.

Browse the full REST API in the **[Dex API](/reference/dex)** tab. Key endpoint groups:

| Group             | Description                                     |
| ----------------- | ----------------------------------------------- |
| **Projects**      | Create, list, and manage projects               |
| **Files**         | Upload files, get download URLs, stream content |
| **Parse**         | Create parse jobs, list results, rechunk        |
| **Extract**       | Create extract jobs, list extractions           |
| **Vector Stores** | Create stores, add files, search                |
| **Jobs**          | Get job status and monitor progress             |

***

## Authentication

All requests require:

* **`x-api-key`**: Your SGP API key
* **`x-selected-account-id`**: Your SGP account ID

```bash theme={null}
curl -X GET "https://dex.sgp.scale.com/v1/projects" \
  -H "x-api-key: $SGP_API_KEY" \
  -H "x-selected-account-id: $SGP_ACCOUNT_ID"
```

***

## Base URL

| Environment | Base URL                    |
| ----------- | --------------------------- |
| Production  | `https://dex.sgp.scale.com` |

***

## Next Steps

* **[Getting Started](getting-started-with-dex)**: Step-by-step tutorial with the Python SDK
* **[Dex SDK API Reference](dex-sdk-api-reference)**: Python SDK methods and types
* **[Best Practices](best-practices)**: Optimization and common patterns


# Dex SDK API Reference
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/dex-sdk-api-reference



This reference documents the Python SDK methods for Scale's Dex document understanding capability.

## DexClient

The main client for interacting with the Dex service.

### Project Management

* `create_project(name, configuration)` - Create a new project with optional configuration
* `list_projects()` - List all accessible projects
* `get_project(project_id)` - Retrieve a specific project
* `update_project(project_id, project_data)` - Update project name, configuration, or status

**Example:**

```python theme={null}
import os
from datetime import timedelta
from dex_sdk import DexClient
from dex_sdk.types import ProjectConfiguration, RetentionPolicy

# Initialize client with SGP credentials
dex_client = DexClient(
    base_url="https://dex.sgp.scale.com",
    api_key=os.getenv("SGP_API_KEY"),
    account_id=os.getenv("SGP_ACCOUNT_ID"),
)

# Create project (credentials are passed via client initialization)
project = await dex_client.create_project(
    name="My Project",
)

# Create project with data retention policy
project = await dex_client.create_project(
    name="My Compliant Project",
    configuration=ProjectConfiguration(
        retention=RetentionPolicy(
            files=timedelta(days=30),
            result_artifacts=timedelta(days=7),
        )
    )
)

# Update project configuration
from dex_sdk.types import ProjectUpdateRequest

await dex_client.update_project(
    project_id=project.id,
    project_data=ProjectUpdateRequest(
        configuration=ProjectConfiguration(
            retention=RetentionPolicy(
                files=timedelta(days=90),
                result_artifacts=timedelta(days=30),
            )
        )
    ),
)
```

***

## Project

Represents a Dex project with isolated data and credentials.

### File Operations

* `upload_file(file_path)` - Upload a document to the project
* `list_files(pagination_params, filter)` - List all uploaded files with optional pagination and filtering
* `get_file_by_id(file_id)` - Get file metadata

**Example:**

```python theme={null}
# Upload a file
dex_file = await project.upload_file("path/to/document.pdf")

# List all files (simple)
files = await project.list_files()

# List files with pagination
from dex_sdk.types import PaginationParams

pagination_params = PaginationParams(
    page_size=10,
    sort_by="created_at",
    sort_order="desc",
    continuation_token=None
)
files = await project.list_files(pagination_params=pagination_params)

# List files with filtering
from dex_sdk.types import FileListFilter
from datetime import datetime, timedelta

file_filter = FileListFilter(created_at_start=datetime.now() - timedelta(days=1))
files = await project.list_files(pagination_params=pagination_params, filter=file_filter)

# Get a temporary download URL for an uploaded file
download_url = await dex_file.get_download_url()
```

### Job Operations

* `list_jobs(pagination_params, filter)` - List all jobs in the project with optional pagination and filtering
* `get_job(job_id)` - Get job details and status

**Example:**

```python theme={null}
# List all jobs (simple)
jobs = await project.list_jobs()

# List jobs with pagination and filtering
from dex_sdk.types import PaginationParams, JobListFilter
from datetime import datetime, timedelta

pagination_params = PaginationParams(
    page_size=10,
    sort_by="created_at",
    sort_order="desc",
)
job_filter = JobListFilter(created_at_start=datetime.now() - timedelta(days=1))
jobs = await project.list_jobs(pagination_params=pagination_params, filter=job_filter)
```

### Parse Result Operations

* `list_parse_results(pagination_params, filter)` - List all parse results with optional pagination and filtering
* `get_parse_result(parse_result_id)` - Get parse result details

**Example:**

```python theme={null}
# List parse results with filtering
from dex_sdk.types import ParseResultListFilter

parse_result_filter = ParseResultListFilter(created_at_start=datetime.now() - timedelta(days=7))
parse_results = await project.list_parse_results(filter=parse_result_filter)
```

### Extraction Operations

* `list_extractions(pagination_params, filter)` - List all extractions with optional pagination and filtering
* `get_extraction(extraction_id)` - Get extraction details

**Example:**

```python theme={null}
# List extractions with filtering
from dex_sdk.types import ExtractionListFilter

extraction_filter = ExtractionListFilter(created_at_start=datetime.now() - timedelta(days=7))
extractions = await project.list_extractions(filter=extraction_filter)
```

### Vector Store Operations

* `create_vector_store(name, engine, embedding_model)` - Create a vector store with SGP Knowledge Base engine
* `list_vector_stores(pagination_params, filter)` - List all vector stores with optional pagination and filtering
* `get_vector_store_by_id(vector_store_id)` - Get vector store details
* `delete()` - Delete a vector store from a `DexVectorStore` instance

**Example:**

```python theme={null}
from dex_sdk.types import VectorStoreEngines

vector_store = await project.create_vector_store(
    name="My Vector Store",
    engine=VectorStoreEngines.SGP_KNOWLEDGE_BASE,
    embedding_model="openai/text-embedding-3-large",
)

# List vector stores with filtering
from dex_sdk.types import VectorStoreListFilter

vector_store_filter = VectorStoreListFilter(created_at_start=datetime.now() - timedelta(days=7))
vector_stores = await project.list_vector_stores(filter=vector_store_filter)

same_vector_store = await project.get_vector_store_by_id(vector_store.id)
await same_vector_store.delete()
```

***

## DexFile

Represents an uploaded file in Dex.

### Parsing

* `parse(params)` - Parse document to structured format (automatically polls for completion)
* `start_parse_job(params)` - Start a parse job and return immediately (recommended for async workflows)
* `get_download_url()` - Get a temporary download URL for the uploaded file

**Example:**

```python theme={null}
from dex_sdk.types import (
    ParseEngine,
    IrisParseJobParams,
    IrisParseEngineOptions,
)
iris_params = IrisParseJobParams(
    engine=ParseEngine.IRIS,
    options=IrisParseEngineOptions(
        # default layout="rt_detr_bce" → region-level chunks (closest to layout-aware parsing)
        # layout="whole_page",  # optional: one chunk per page
        # text_ocr="openai/gpt-5.4",  # optional
    ),
)
# Standard parsing (auto-polls for completion)
parse_result = await dex_file.parse(iris_params)  # max_attempts=100 is optional

# Async job workflow (recommended for better control)
parse_job = await dex_file.start_parse_job(iris_params)
parse_result = await parse_job.get_result()
# See Job Monitoring section for how to track job progress
```

### Working with Parse Results

After parsing, you can access the structured content including chunks and blocks.

**Example:**

```python theme={null}
# Parse the file
parse_result = await dex_file.parse(parse_params)

# Access metadata
metadata = parse_result.data.parse_metadata
print(f"Source: {metadata.filename} ({metadata.pages_processed} pages, engine: {parse_result.data.engine})")

# Access content chunks
for i, chunk in enumerate(parse_result.data.content.chunks):
    print(f"\nChunk {i}: {chunk.content[:100]}... ({len(chunk.blocks)} blocks)")

    for block in chunk.blocks:
        print(f"  [{block.type}] Page {block.page_number}, "
              f"confidence: {block.confidence:.2f}, "
              f"pos: ({block.bbox.left:.2f}, {block.bbox.top:.2f})")
```

***

## ParseResult

Represents the result of a document parsing operation.

### Extraction

* `extract(extraction_schema, user_prompt, model, generate_citations, generate_confidence)` - Extract structured data with user prompt, schema, model, and options

**Parameters:**

* `extraction_schema` (BaseModel): Pydantic model class for extraction (pass the class directly, not `model_json_schema()`)
* `user_prompt` (str): Natural language instructions for extraction
* `model` (str): LLM model to use (e.g., "openai/gpt-5.4")
* `generate_citations` (bool): Include source citations in results
* `generate_confidence` (bool): Include confidence scores in results

**Example:**

```python theme={null}
from pydantic import BaseModel, Field

class InvoiceData(BaseModel):
    invoice_number: str = Field(description="The invoice number")
    total_amount: float = Field(description="Total amount in dollars")
    date: str = Field(description="Invoice date")

extract_result = await parse_result.extract(
    extraction_schema=InvoiceData,
    user_prompt="Extract invoice details from this document.",
    model="openai/gpt-5.4",
    generate_citations=True,
    generate_confidence=True,
)
```

### Working with Extraction Results

After extraction, you can access the structured data, citations, and confidence scores.

**Example:**

```python theme={null}
# Extract data
extract_result = await parse_result.extract(
    extraction_schema=InvoiceData,
    user_prompt="Extract invoice details from this document.",
    model="openai/gpt-5.4",
    generate_citations=True,
    generate_confidence=True,
)

# Access the extraction result
result = extract_result.result

# Access structured data
for field_name, field in result.data.items():
    print(f"{field_name}: {field.value} (confidence: {field.confidence:.2f})")

    if field.citations:
        for cite in field.citations:
            loc = f", pos: ({cite.bbox.left:.2f}, {cite.bbox.top:.2f})" if cite.bbox else ""
            print(f"  → Page {cite.page}: {cite.content[:50]}...{loc}")

# Access usage information
if result.usage_info:
    usage = result.usage_info
    print(f"\nTokens: {usage.total_tokens} total ({usage.prompt_tokens} prompt + {usage.completion_tokens} completion)")
```

***

## VectorStore

Represents a vector store for semantic search and RAG-enhanced extraction.

### Indexing

* `add_parse_results(parse_result_ids)` - Add parsed documents to vector store by parse result IDs

**Example:**

```python theme={null}
# Add parsed documents to vector store
await vector_store.add_parse_results([parse_result.id])
```

### Search

* `search(query, top_k, filters)` - Semantic search across all documents in the vector store
* `search_in_file(file_id, query, top_k, filters)` - Search within a specific file with optional filters

**Example:**

```python theme={null}
# Search across all documents
results = await vector_store.search(
    query="What is the total revenue?",
    top_k=5,
)

# Search within a specific file
file_results = await vector_store.search_in_file(
    file_id=dex_file.id,
    query="What is the total revenue?",
    top_k=5,
    filters=None,
)
```

### Extraction

* `extract(extraction_schema, user_prompt, model, generate_citations, generate_confidence)` - Extract structured data from entire vector store with RAG context

**Example:**

```python theme={null}
# Extract from vector store with RAG context
extract_result = await vector_store.extract(
    extraction_schema=FinancialData,
    user_prompt="Extract financial summary from all documents.",
    model="openai/gpt-5.4",
    generate_citations=True,
    generate_confidence=True,
)
```

***

## Job Monitoring and SGP Tracing

**New in v0.4.0**: Enhanced async job support with SGP tracing integration for better observability and debugging.

### Async Job Workflow

Use `start_parse_job()` for better control over async operations and access to SGP traces.

**Example:**

```python theme={null}
from dex_sdk.types import (
    IrisParseJobParams,
    IrisParseEngineOptions,
    ParseEngine,
    JobStatus,
)
import asyncio

iris_params = IrisParseJobParams(
    engine=ParseEngine.IRIS,
    options=IrisParseEngineOptions(
        # layout="rt_detr_bce" is default (region-level chunks)
        # layout="whole_page",  # one chunk per page
    ),
)
# Start a parse job
parse_job = await project.start_parse_job(
    dex_file=dex_file,
    parameters=iris_params
)

# Monitor job progress
while parse_job.data.status != JobStatus.SUCCEEDED and parse_job.data.status != JobStatus.FAILED:
    await asyncio.sleep(1)
    await parse_job.refresh()

# Get the result
parse_result = await parse_job.get_result()
```

### Retrieving SGP Traces

Jobs are now connected to SGP traces for end-to-end observability. You can retrieve complete trace data for any job.

**Example:**

```python theme={null}
from scale_gp_beta import SGPClient

# Initialize SGP client
sgp_client = SGPClient(
    api_key=os.getenv("SGP_API_KEY"),
    account_id=os.getenv("SGP_ACCOUNT_ID"),
)

# Search for spans by job ID
spans = [span for span in sgp_client.spans.search(
    sort_by="created_at",
    sort_order="desc",
    extra_metadata={"job_id": parse_job.data.id},
    parents_only=True,
)]

# Get full trace
if spans:
    trace_id = spans[0].trace_id
    all_spans = [span for span in sgp_client.spans.search(trace_ids=[trace_id])]

    # Analyze trace data
    for span in all_spans:
        print(f"Span: {span.name}, Duration: {span.duration_ms}ms")
```

**Benefits:**

* **Better observability**: Track jobs through SGP's tracing infrastructure
* **Easier debugging**: Access detailed execution traces for failed jobs
* **Performance monitoring**: Analyze job performance and identify bottlenecks
* **Request correlation**: Connect job execution to API requests and traces

***

## Parse Job Parameters

When parsing documents, you can specify different engines and options to customize the parsing behavior.

### Iris Parse Parameters

**IrisParseJobParams** - Parameters for the Iris OCR engine (Default engine).

**Fields:**

* `engine` (ParseEngine): Set to `ParseEngine.IRIS`
* `options` (IrisParseEngineOptions): Parsing options
* `vector_store_metadata` (dict | None): Metadata to populate into the vector store for filtering when searching. Keys must match the `vector_store_metadata_schema` of the target vector store.

**IrisParseEngineOptions:**

| Field                        | Type                                                | Default (if omitted) | Description                                                                                       |          |        |                                  |
| ---------------------------- | --------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------- | -------- | ------ | -------------------------------- |
| `layout`                     | `"rt_detr_bce"` \| `"whole_page"` \| `null`         | `"rt_detr_bce"`      | Layout detector. `rt_detr_bce` → one chunk per detected region; `whole_page` → one chunk per page |          |        |                                  |
| `text_ocr`                   | `str` \| `null`                                     | `"openai/gpt-5.4"`   | SGP model ID for text regions                                                                     |          |        |                                  |
| `table_ocr`                  | `str` \| `null`                                     | `"openai/gpt-5.4"`   | SGP model ID for table regions                                                                    |          |        |                                  |
| `text_prompt`                | `str` \| `null`                                     | built-in default     | User prompt for text extraction (VLMs)                                                            |          |        |                                  |
| `text_system_prompt`         | `str` \| `null`                                     | built-in default     | System prompt for text extraction (VLMs)                                                          |          |        |                                  |
| `table_prompt`               | `str` \| `null`                                     | built-in default     | User prompt for table extraction (VLMs)                                                           |          |        |                                  |
| `model_parameters`           | `dict` \| `null`                                    | —                    | Extra LLM inference args (e.g. `max_tokens`, `temperature`, `reasoning_effort` \["low"            | "medium" | "high" | null]); merged into SGP requests |
| `confidence_threshold`       | `float` \| `null`                                   | `0.4`                | Min confidence for **text** layout boxes                                                          |          |        |                                  |
| `table_confidence_threshold` | `float` \| `null`                                   | `0.5`                | Min confidence for **table** layout boxes                                                         |          |        |                                  |
| `image_confidence_threshold` | `float` \| `null`                                   | `0.5`                | Min confidence for **image** layout boxes                                                         |          |        |                                  |
| `containment_threshold`      | `float` \| `null`                                   | `0.8`                | Drop nested boxes when the smaller box is this fraction inside a larger one                       |          |        |                                  |
| `strict_containment_filter`  | `bool` \| `null`                                    | `true`               | If `true`, drop any contained box; if `false`, only drop same-label overlaps                      |          |        |                                  |
| `img_method`                 | `"description"` \| `"base64"` \| `"skip"` \| `null` | `"description"`      | How image regions are emitted: LLM description, inline base64, or omit                            |          |        |                                  |
| `img_description_prompt`     | `str` \| `null`                                     | built-in default     | Prompt when `img_method="description"`                                                            |          |        |                                  |
| `image_description_model`    | `str` \| `null`                                     | `"openai/gpt-5.4"`   | SGP model for image descriptions                                                                  |          |        |                                  |

**Example:**

```python theme={null}
from dex_sdk.types import (
    ParseEngine,
    IrisParseJobParams,
    IrisParseEngineOptions,
)
parse_result = await dex_file.parse(
    IrisParseJobParams(
        engine=ParseEngine.IRIS,
        options=IrisParseEngineOptions(
            layout="rt_detr_bce",       # or "whole_page"
            text_ocr="openai/gpt-5.4",
            confidence_threshold=0.5,
        ),
    )
)
```

### Reducto Parse Parameters (Legacy)

**ReductoParseJobParams** - Parameters for the Reducto OCR engine.

**Best for:** English and Latin-script documents (Spanish, French, German, Italian, Portuguese, etc.) with tables, figures, and complex layouts.

**Fields:**

* `engine` (ParseEngine): Set to `ParseEngine.REDUCTO`
* `options` (ReductoParseEngineOptions): Parsing options
* `advanced_options` (dict): Advanced options for fine-tuning
* `experimental_options` (dict): Experimental features
* `priority` (bool): Whether to prioritize this job (default: False)
* `vector_store_metadata` (dict | None): Metadata to populate into the vector store for filtering when searching. Keys must match the `vector_store_metadata_schema` of the target vector store.

**ReductoParseEngineOptions:**

* `chunking` (ReductoChunkingOptions | None): Chunking configuration

**ReductoChunkingOptions:**

* `chunk_mode` (ReductoChunkingMethod): Chunking method (default: `VARIABLE`)
  * `DISABLED`: No chunking
  * `BLOCK`: Block-level chunks
  * `PAGE`: Page-level chunks
  * `PAGE_SECTIONS`: Page sections
  * `SECTION`: Section-level chunks
  * `VARIABLE`: Variable-size chunks based on content
* `chunk_size` (int | None): Custom chunk size

**Example:**

```python theme={null}
# (Legacy) Reducto parse parameters
from dex_sdk.types import (
    ParseEngine,
    ReductoParseJobParams,
    ReductoChunkingMethod,
    ReductoChunkingOptions,
    ReductoParseEngineOptions,
)

parse_params = ReductoParseJobParams(
    engine=ParseEngine.REDUCTO,
    options=ReductoParseEngineOptions(
        chunking=ReductoChunkingOptions(
            chunk_mode=ReductoChunkingMethod.VARIABLE,
            chunk_size=None,
        )
    ),
    priority=False,
)

parse_result = await dex_file.parse(parse_params)
```

***

## Common Types

This section documents the core data models and types used throughout the Dex SDK.

### Type Categories

**Importable Types** - Types you can import from `dex_sdk.types` to configure your requests:

* Configuration types (ProjectConfiguration, RetentionPolicy)
* Pagination types (PaginationParams, FileListFilter, JobListFilter, ParseResultListFilter, ExtractionListFilter, VectorStoreListFilter) - **New in v0.4.0**
* Parse parameter types (IrisParseJobParams, ReductoParseJobParams, etc.)
* Enum types (ParseEngine, ReductoChunkingMethod, VectorStoreEngines, JobStatus) - **JobStatus new in v0.4.0**

**Response Types** - Types returned by the SDK, accessible via the `.data` attribute on wrapper objects:

* When you call SDK methods, you get wrapper objects (DexProject, DexFile, DexParseResult, etc.)
* Access the underlying data via `.data`: `project.data.id`, `file.data.filename`
* These entities are automatically validated but don't need to be imported

***

### Configuration Types

#### ProjectConfiguration

Configuration options for a Dex project.

**Import:** `from dex_sdk.types import ProjectConfiguration`

**Fields:**

* `retention` (RetentionPolicy | None): Data retention policy for the project

**Example:**

```python theme={null}
from datetime import timedelta
from dex_sdk.types import ProjectConfiguration, RetentionPolicy

config = ProjectConfiguration(
    retention=RetentionPolicy(
        files=timedelta(days=30),
        result_artifacts=timedelta(days=7),
    )
)
```

#### RetentionPolicy

Defines data retention periods for automatic cleanup of files and processing artifacts.

**Import:** `from dex_sdk.types import RetentionPolicy`

**Fields:**

* `files` (timedelta | None): Retention period for uploaded files. Files older than this period are automatically deleted. If `None`, files are retained indefinitely.
* `result_artifacts` (timedelta | None): Retention period for parse results, extraction results, and job artifacts. If `None`, artifacts are retained indefinitely.

**Example:**

```python theme={null}
from datetime import timedelta
from dex_sdk.types import RetentionPolicy

# 30-day file retention, 7-day artifact retention
policy = RetentionPolicy(
    files=timedelta(days=30),
    result_artifacts=timedelta(days=7),
)

# Keep files indefinitely, but clean up artifacts after 14 days
policy = RetentionPolicy(
    files=None,
    result_artifacts=timedelta(days=14),
)
```

**Use Cases:**

* **Compliance**: Meet regulatory requirements (GDPR, HIPAA, etc.)
* **Cost Management**: Automatically clean up old data to reduce storage costs
* **Security**: Limit exposure of sensitive documents by enforcing retention limits

> **Note**: The retention period is calculated from the creation time of the file or artifact. Retention policies can be updated at any time using `update_project()`.

#### PaginationParams

**New in v0.4.0**: Parameters for paginated list operations.

**Import:** `from dex_sdk.types import PaginationParams`

**Fields:**

* `page_size` (int | None): Number of items to return per page (default: 50, max: 100)
* `sort_by` (str | None): Field name to sort by (e.g., `"created_at"`)
* `sort_order` (str | None): Sort order, either `"asc"` or `"desc"` (default: `"desc"`)
* `continuation_token` (str | None): Token for fetching next/previous page

**Example:**

```python theme={null}
from dex_sdk.types import PaginationParams

pagination_params = PaginationParams(
    page_size=10,
    sort_by="created_at",
    sort_order="desc",
    continuation_token=None  # Set to next_token from previous response for pagination
)

files = await project.list_files(pagination_params=pagination_params)
```

#### List Filter Types

**New in v0.4.0**: Filter types for list operations on different entity types.

**Import:** `from dex_sdk.types import FileListFilter, JobListFilter, ParseResultListFilter, ExtractionListFilter, VectorStoreListFilter`

**Common Fields:**

* `created_at_start` (datetime | None): Filter for entities created after this time
* `created_at_end` (datetime | None): Filter for entities created before this time

**Example:**

```python theme={null}
from dex_sdk.types import FileListFilter, JobListFilter
from datetime import datetime, timedelta

# Filter files created in the last 24 hours
file_filter = FileListFilter(
    created_at_start=datetime.now() - timedelta(days=1)
)
files = await project.list_files(filter=file_filter)

# Filter jobs created in the last week
job_filter = JobListFilter(
    created_at_start=datetime.now() - timedelta(days=7)
)
jobs = await project.list_jobs(filter=job_filter)
```

**Available Filter Types:**

* `FileListFilter` - Filter uploaded files
* `JobListFilter` - Filter jobs
* `ParseResultListFilter` - Filter parse results
* `ExtractionListFilter` - Filter extractions
* `VectorStoreListFilter` - Filter vector stores

#### ExtractionParameters

Parameters for extraction operations.

**Import:** `from dex_sdk.types import ExtractionParameters`

**Fields:**

* `model` (str): LLM model to use (e.g., `"openai/gpt-5.4"`)
* `model_kwargs` (dict | None): Additional kwargs for the LLM model
* `extraction_schema` (dict): JSON schema defining the desired output structure
* `system_prompt` (str | None): High-level instructions for the extraction model
* `user_prompt` (str | None): Specific hints about the current document
* `generate_citations` (bool): Whether to return bounding boxes for extracted values (default: True)
* `generate_confidence` (bool): Whether to return confidence scores (default: True)

***

### Parse Configuration Types

#### ParseEngine

Enum of available OCR engines.

**Import:** `from dex_sdk.types import ParseEngine`

**Values:**

* `IRIS` = "iris"
* `REDUCTO` = "reducto"
* `CUSTOM` = "custom"

#### ReductoParseJobParams (Deprecated)

Parameters for the Reducto OCR engine.

**Import:** `from dex_sdk.types import ReductoParseJobParams`

See the [Parse Job Parameters](#parse-job-parameters) section for detailed usage.

#### IrisParseJobParams

Parameters for the Iris OCR engine.

**Import:** `from dex_sdk.types import IrisParseJobParams`

See the [Parse Job Parameters](#parse-job-parameters) section for detailed usage.

#### ReductoChunkingMethod (Deprecated)

Enum of chunking methods for Reducto parser.

**Import:** `from dex_sdk.types import ReductoChunkingMethod`

**Values:**

* `DISABLED` = "disabled"
* `BLOCK` = "block"
* `PAGE` = "page"
* `PAGE_SECTIONS` = "page\_sections"
* `SECTION` = "section"
* `VARIABLE` = "variable"

#### ReductoChunkingOptions (Deprecated)

Chunking configuration for Reducto parser.

**Import:** `from dex_sdk.types import ReductoChunkingOptions`

**Fields:**

* `chunk_mode` (ReductoChunkingMethod): Chunking method
* `chunk_size` (int | None): Custom chunk size

#### ReductoParseEngineOptions (Deprecated)

Options for Reducto parser.

**Import:** `from dex_sdk.types import ReductoParseEngineOptions`

**Fields:**

* `chunking` (ReductoChunkingOptions | None): Chunking configuration

#### IrisParseEngineOptions

Options for Iris parser.

**Import:** `from dex_sdk.types import IrisParseEngineOptions`

**Fields:**

* `layout` (str | None): Layout detection model
* `text_ocr` (str | None): Text OCR model
* `table_ocr` (str | None): Table OCR model
* `text_prompt` (str | None): Custom prompt for text extraction
* `table_prompt` (str | None): Custom prompt for table extraction
* `left_to_right` (bool | None): Sort regions left-to-right
* `confidence_threshold` (float | None): Minimum confidence threshold
* `containment_threshold` (float | None): Containment threshold for filtering

***

### Vector Store Types

#### VectorStoreEngines

Enum of available vector store engines.

**Import:** `from dex_sdk.types import VectorStoreEngines`

**Values:**

* `SGP_KNOWLEDGE_BASE` = "sgp\_knowledge\_base"

#### VectorStoreSearchResult

Result from vector store search operations containing matching chunks.

**Import:** `from dex_sdk.types import VectorStoreSearchResult`

**Fields:**

* `chunks` (list\[VectorStoreChunk]): List of matching chunks with relevance scores

**Example:**

```python theme={null}
# Perform search
search_results = await vector_store.search(
    query="What is the total revenue?",
    top_k=5,
)

# Access chunks
for chunk in search_results.chunks:
    print(f"Score: {chunk.score:.2f}")
    print(f"Content: {chunk.content[:100]}...")
    print(f"File ID: {chunk.file_id}")
```

#### VectorStoreChunk

Represents a single chunk returned from vector store search operations.

**Fields:**

* `content` (str): Text content of the chunk
* `score` (float): Relevance score for the search query
* `file_id` (str | None): ID of the file this chunk belongs to
* `parse_result_id` (str | None): ID of the parse result this chunk belongs to
* `metadata` (dict\[str, Any] | None): Additional metadata from the chunk, which may include information like chunk indices, embeddings metadata, or other custom fields added during indexing
* `blocks` (list): List of block objects with layout information

**Example:**

```python theme={null}
# Access chunk details
for chunk in search_results.chunks:
    print(f"Score: {chunk.score:.2f}")
    print(f"Content: {chunk.content[:100]}...")
    print(f"File: {chunk.file_id}, Parse Result: {chunk.parse_result_id}")

    # Access additional metadata
    if chunk.metadata:
        print(f"Chunk metadata: {chunk.metadata}")

    # Access block layout information
    for block in chunk.blocks:
        print(f"  Page {block.page_number}, type: {block.type}")
```

***

### Response Entity Types

These types are returned by SDK methods and accessed via the `.data` attribute on wrapper objects. You typically don't need to import these directly.

#### Working with Response Data

When you call SDK methods, you receive wrapper objects with a `.data` attribute:

```python theme={null}
# Create a project
project = await dex_client.create_project(name="My Project")
print(f"Project: {project.data.name} ({project.data.id}) created at {project.data.created_at}")

# Upload a file
dex_file = await project.upload_file("document.pdf")
print(f"File: {dex_file.data.filename} ({dex_file.data.size_bytes} bytes) → {dex_file.data.id}")

# Parse a file
parse_result = await dex_file.parse(parse_params)
metadata = parse_result.data.parse_metadata
print(f"Parsed: {metadata.pages_processed} pages with {parse_result.data.engine} → {parse_result.data.id}")
```

#### Common Response Entity Fields

**ProjectEntity** (accessed via `project.data`):

* `id` (str): Project ID with `proj_` prefix
* `name` (str): Project readable name
* `status` (str): Project status (`"active"` or `"archived"`)
* `configuration` (ProjectConfiguration | None): Project configuration
* `created_at` (datetime): When the project was created
* `archived_at` (datetime | None): When the project was archived

**FileEntity** (accessed via `dex_file.data`):

* `id` (str): File ID with `file_` prefix
* `project_id` (str): Project ID that the file belongs to
* `filename` (str): Original filename
* `size_bytes` (int): File size in bytes
* `mime_type` (str): MIME type of the file
* `status` (str): Current file status
* `created_at` (datetime): When the file was uploaded

**ParseResultEntity** (accessed via `parse_result.data`):

* `id` (str): Parse result ID with `pres_` prefix
* `project_id` (str): Project ID
* `source_document_id` (str): Source document ID that was parsed
* `engine` (str): Engine used for parsing
* `parse_metadata` (object): Metadata including `filename`, `pages_processed`
* `content` (object): Parsed content with `chunks`
* `created_at` (datetime): When the parse result was created

**ExtractionEntity** (accessed via `extract_result` or in extraction results):

* `id` (str): Extraction result ID
* `source_id` (str): Source ID that was extracted from
* `result` (object): The extraction result with `data` and `usage_info`
* `parameters` (ExtractionParameters): Parameters used for extraction
* `created_at` (datetime): When the extraction was completed
* `processing_time_ms` (int | None): Processing time in milliseconds

**VectorStoreEntity** (accessed via `vector_store.data`):

* `id` (str): Vector store ID with `vs_` prefix
* `project_id` (str): Project ID
* `name` (str): Name of the vector store
* `engine` (str): Engine used for vector store
* `created_at` (datetime): When the vector store was created

***

### Deprecated Types

The following types are deprecated as of version 0.3.2 and should no longer be used:

* **ProjectCredentials** - No longer used; credentials are passed to `DexClient` constructor
* **SGPCredentials** - No longer used; credentials are passed to `DexClient` constructor

See the [Changelog](/docs/capabilities/changelog#authentication-changes) for migration instructions.

***

## Error Handling

The SDK raises exceptions for various error conditions. For detailed troubleshooting guidance, see the **[Troubleshooting Guide](troubleshooting)**.

```python theme={null}
from dex_sdk.exceptions import DexException

try:
    parse_result = await dex_file.parse(...)
except DexException as e:
    print(f"Error: {e}")
```

***

## Async/Await Pattern

The Dex SDK is fully async. Use `await` with all SDK methods:

```python theme={null}
import asyncio
import os

async def main():
    # Initialize client with credentials
    dex_client = DexClient(
        base_url="https://dex.sgp.scale.com",
        api_key=os.getenv("SGP_API_KEY"),
        account_id=os.getenv("SGP_ACCOUNT_ID"),
    )

    project = await dex_client.create_project(name="My Project")
    dex_file = await project.upload_file("document.pdf")
    parse_result = await dex_file.parse(...)
    extract_result = await parse_result.extract(...)

# Run in Jupyter/IPython
await main()

# Run in regular Python script
asyncio.run(main())
```

***

## See Also

* **[Getting Started Guide](getting-started-with-dex)**: Quick start tutorial for beginners
* **[File Management](file-management)**: Upload, pagination, and file types
* **[Parse](parse)**: Parse engines and async job monitoring
* **[Chunking](chunking)**: Chunking strategies
* **[Vector Stores](vector-stores)**: Semantic search and RAG-enhanced extraction
* **[Extract](extract)**: Data extraction and batch processing
* **[Best Practices](best-practices)**: Quick start, retention policies, and optimization
* **[Introduction to Dex](introduction-to-dex)**: Core concepts and architecture overview
* **[Troubleshooting Guide](troubleshooting)**: Common issues and solutions


# Troubleshooting Dex
Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/troubleshooting



This guide covers common issues you might encounter when working with Dex and how to resolve them.

***

## Installation Issues

### Installing the Dex SDK

The Dex SDK is distributed via Scale's private CodeArtifact repository and requires AWS authentication.

**Prerequisites:**

* AWS CLI installed and configured
* Access to Scale's production AWS account

**Installation steps:**

1. **Authenticate with AWS SSO:**

```bash theme={null}
aws sso login
```

2. **Configure pip for CodeArtifact:**

```bash theme={null}
aws codeartifact login \
    --tool pip \
    --domain scale \
    --repository scale-pypi \
    --profile production-developer \
    --region us-west-2
```

3. **Install the SDK:**
   The exact command can be found on GitHub, in the introduction notebooks.

4. **Verify installation:**

```python theme={null}
import dex_sdk
print(f"Dex SDK version: {dex_sdk.__version__}")
```

### Common Installation Errors

**Token Expired (401 Unauthorized)**

Re-run the CodeArtifact login command to refresh your authentication token.

**AWS CLI Not Configured**

Ensure AWS CLI is installed (`aws --version`) and configure AWS SSO for the production-developer profile.

**Package Not Found**

Verify your AWS credentials are valid: `aws sts get-caller-identity`

**Permission Denied**

Use a virtual environment instead of sudo: `python -m venv venv && source venv/bin/activate`

***

## Runtime Issues

### Connection Problems

**Symptoms:** Cannot connect to Dex service

**Test connection:**

```python theme={null}
import requests
response = requests.get("https://dex.sgp.scale.com/health")
print(f"Status: {response.status_code}")
```

**Solutions:**

* Check network connectivity
* Verify credentials (see Authentication Errors below)
* Confirm Dex service is available

### Authentication Errors

**Symptoms:** Invalid credentials, permission denied

**Verify credentials:**

```python theme={null}
import os
sgp_account_id = os.getenv("SGP_ACCOUNT_ID")
sgp_api_key = os.getenv("SGP_API_KEY")

if not sgp_account_id or not sgp_api_key:
    print("❌ Missing credentials")
```

**Solutions:**

* Set environment variables: `SGP_ACCOUNT_ID` and `SGP_API_KEY`
* Verify your account has SGP access
* Check API key hasn't expired or been revoked

### File Upload Errors

**Symptoms:** Upload fails, file too large, unsupported format

**Supported formats:**

* **Images**: PNG, JPEG, TIFF, HEIC, and more
* **PDFs**: PDF
* **Spreadsheets**: CSV, XLSX, XLS
* **Documents**: PPTX, DOCX, TXT, RTF

**Solutions:**

* Verify file path is correct
* Check file size is under 100MB
* Ensure file format is supported
* Verify file isn't corrupted

### Parsing Errors

**Symptoms:** OCR fails, document can't be processed

**Common causes:**

* Low quality scans (use at least 300 DPI)
* Password-protected documents
* Extremely complex layouts
* Very large documents timing out

**Solutions:**

1. **Check document quality** and resolution
2. **Remove password protection** before uploading
3. **Try a different OCR engine:**

```python theme={null}
from dex_sdk.types import ParseEngine, IrisParseJobParams, IrisParseEngineOptions

parse_result = await dex_file.parse(
    IrisParseJobParams(
        engine=ParseEngine.IRIS,
        options=IrisParseEngineOptions(layout="rt_detr_bce"),
    )
)
```

4. **Split large documents** into smaller chunks
5. **See complex layouts guide:** [Industry Document Types and Layout Challenges](https://llms.reducto.ai/industry-document-types-and-layout-challenges)

### Extraction Errors

**Symptoms:** Extraction fails, schema errors, timeout

**Common causes:**

* Invalid Pydantic schema
* Model timeout or rate limiting
* Insufficient context for extraction
* Schema-data mismatch

**Solutions:**

1. **Validate your schema:**

```python theme={null}
from pydantic import BaseModel, Field

class MySchema(BaseModel):
    field_name: str = Field(description="Clear description")

schema = MySchema.model_json_schema()
```

2. **Use clear prompts** with detailed instructions

3. **Enable debugging:**

```python theme={null}
extract_result = await parse_result.extract(
    extraction_schema=schema,
    user_prompt="Your detailed prompt",
    model="openai/gpt-5.4",
    generate_citations=True,
    generate_confidence=True,
)
```

4. **Use vector stores for large documents:**

```python theme={null}
vector_store = await project.create_vector_store(
    name="My Store",
    engine=VectorStoreEngines.SGP_KNOWLEDGE_BASE,
    embedding_model="openai/text-embedding-3-large",
)
await vector_store.add_parse_results([parse_result.id])

extract_result = await vector_store.extract(
    extraction_schema=schema,
    user_prompt=user_prompt,
    model="openai/gpt-5.4",
)
```

***

## Performance Issues

### Slow Parsing

**Causes:** Large documents, complex layouts, high OCR load

**Solutions:**

* Process documents asynchronously in batches
* Cache parse results for frequently accessed documents

### Slow Extraction

**Causes:** Large context, complex schema, model performance

**Solutions:**

* Use vector stores to reduce context size
* Simplify extraction schema
* Choose faster models for time-sensitive applications
* Use batch processing

### Slow Vector Store Search

**Causes:** Large document collections, complex queries

**Solutions:**

* Use appropriate `top_k` values
* Add filters to narrow search scope
* Use `vector_store.search_in_file()` for file-specific searches
* Create separate vector stores for different categories

***

## Error Handling

### Exception Types

Always wrap Dex operations in try-except blocks:

```python theme={null}
from dex_sdk.exceptions import DexException

try:
    project = await dex_client.create_project(...)
    dex_file = await project.upload_file(...)
    parse_result = await dex_file.parse(...)
    extract_result = await parse_result.extract(...)
except DexException as e:
    print(f"Dex error: {e}")
except Exception as e:
    print(f"Unexpected error: {e}")
```

### Common Exceptions

| Exception             | Cause                   | Solution                                  |
| --------------------- | ----------------------- | ----------------------------------------- |
| `AuthenticationError` | Invalid credentials     | Verify `SGP_ACCOUNT_ID` and `SGP_API_KEY` |
| `FileUploadError`     | File format/size issues | Check format and size limits              |
| `ParsingError`        | OCR failure             | Try different OCR engine                  |
| `ExtractionError`     | Schema or model error   | Validate schema                           |
| `ConnectionError`     | Network issues          | Check connectivity                        |
| `RateLimitError`      | Too many requests       | Implement backoff/retry                   |
| `PermissionError`     | Insufficient access     | Check permissions                         |

***

## Debugging

### Enable Logging

```python theme={null}
import logging

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("dex_sdk")
logger.setLevel(logging.DEBUG)
```

### Inspect Parse Results

```python theme={null}
parse_data = parse_result.data.model_dump()
print(f"Pages: {len(parse_data.get('pages', []))}")
print(f"Chunks: {len(parse_data.get('chunks', []))}")
```

### Validate Extraction Results

```python theme={null}
extraction_data = extract_result.data.model_dump()

if 'citations' in extraction_data:
    print(f"Citations found: {len(extraction_data['citations'])}")

if 'confidence' in extraction_data:
    print(f"Confidence: {extraction_data['confidence']}")
```

### Test Components Individually

```python theme={null}
try:
    project = await dex_client.create_project(...)
    print("✅ Project created")

    dex_file = await project.upload_file(...)
    print("✅ File uploaded")

    parse_result = await dex_file.parse(...)
    print("✅ Parsing completed")

    extract_result = await parse_result.extract(...)
    print("✅ Extraction completed")
except Exception as e:
    print(f"❌ Failed at step: {e}")
```

***

## Getting Help

### Support Channels

* **Slack**: `#dex-help` for Dex team support
* **Documentation**: [Dex Documentation](/docs/capabilities/document-understanding/introduction-to-dex)

### When Reporting Issues

Include:

1. **Error message** - Full error text and stack trace
2. **Code snippet** - Minimal reproducible example
3. **Document type** - File format and characteristics
4. **SDK version** - Output of `pip show dex_sdk`
5. **Environment** - Python version, OS
6. **Expected vs actual behavior**

### Additional Resources

* [Getting Started Guide](getting-started-with-dex) - Quick start tutorial
* [File Management](file-management) - Upload and pagination
* [Parse](parse) - Parse engines and async jobs
* [Vector Stores](vector-stores) - Semantic search and RAG
* [Extract](extract) - Data extraction and batch processing
* [Best Practices](best-practices) - Common patterns and optimization
* [API Reference](dex-sdk-api-reference) - Complete SDK documentation
* [Introduction to Dex](introduction-to-dex) - Core concepts


# Getting Started with IRIS
Source: https://docs.gp.scale.com/docs/capabilities/ocr/getting-started-with-iris



This guide will help you get started with IRIS, Scale's proprietary OCR capability integrated with Dex. You'll learn how to parse documents with IRIS through the Dex SDK, work with parse results, and integrate IRIS into your document processing workflows.

***

## Overview

IRIS is Scale's proprietary OCR capability that provides complete pipeline control for custom document processing needs. When using IRIS through Dex, you get:

* **Use any OCR model**: Experiment with different OCR engines to optimize for your specific documents. DEX can use any model hosted in SGP.
* **Complete Pipeline Control**: Configure layout detection, OCR processing, and assembly independently
* **Unified Document Management**: Upload and manage files through Dex's file system
* **Async Processing**: Non-blocking parse jobs that process documents in the background
* **Project Organization**: Group files and parse results within Dex projects
* **Extensibility**: Add custom OCR models or layout detectors for specialized needs

***

## Prerequisites

Before using IRIS, ensure you have:

* Dex SDK installed
* SGP Account ID and API Key
* Access to a Dex instance

***

## Basic Usage

### Initialize Dex Client

```python theme={null}
import os
from dex_sdk.client import DexClient
from dex_sdk.types import ProjectCredentials, SGPCredentials

# Initialize Dex client
dex_client = DexClient(
    base_url="your-dex-url",
    account_id = os.getenv("SGP_ACCOUNT_ID"),
    api_key = os.getenv("SGP_API_KEY"),
)

# Create a project
project = await dex_client.create_project(name="my-ocr-project")
```

### Parse a Document with IRIS

**What should I use for IrisParseEngineOptions?**
A common query when using Iris is “How can I get the best OCR results without spending time worrying about all the parameters in `IrisParseEngineOptions`?”. This section gives quick-start suggestions that you can use depending on your requirements that have been shown to perform consistently well across use-cases.

### Option 1: I only need text, I don’t need bounding boxes.

**Suggestion:**
text\_ocr="gpt-5.4" + layout="whole\_page"

This bypasses the layout detection step, and instead passes the full page to the OCR model. This has been shown to consistently perform consistently well across benchmarks.

**Suggestion (open-source required):**
`text_ocr="<Qwen/Qwen3.6-27B>" + layout="whole_page"`

This is the same as the previous suggestion, but with the open-source Qwen/Qwen3.6-27B model instead, which has been shown to almost match gpt-5.4 in English and often exceed gpt-5.4 in Arabic. Using the Qwen model with Iris requires a deployed model endpoint available through SGP.

```python theme={null}
from dex_sdk.types import IrisParseEngineOptions, IrisParseJobParams, ParseEngine

# Upload document
dex_file = await project.upload_file("document.pdf")

# Start IRIS parse job
parse_job = await dex_file.start_parse_job(
    IrisParseJobParams(
        engine=ParseEngine.IRIS,
        options=IrisParseEngineOptions(
            layout= "whole_page",
            text_ocr= "openai/gpt-5.4", #or text_ocr= "<path to Qwen/Qwen3.6-27B endpoint>" for the open-source suggestion.
        ),
    ),
)

# Get parse results
parse_result = await parse_job.get_result()

# Access parsed content
chunks = parse_result.data.content.chunks
for chunk in chunks:
    print(chunk.content)
```

### Option 2: I need bounding boxes.

The default config values were designed for this case, so you only need to specify which OCR model to use.

The suggestion and open-source suggestion is the same for this case, with the only difference is we do not specify `layout` so that it uses the default layout model option `rt_detr_bce`.

```python theme={null}
from dex_sdk.types import IrisParseEngineOptions, IrisParseJobParams, ParseEngine

# Upload document
dex_file = await project.upload_file("document.pdf")

# Start IRIS parse job
parse_job = await dex_file.start_parse_job(
    IrisParseJobParams(
        engine=ParseEngine.IRIS,
        options=IrisParseEngineOptions(
            text_ocr= "openai/gpt-5.4", #or text_ocr= "<path to Qwen/Qwen3.6-27B endpoint>" for the open-source suggestion.
        ),
    ),
)
# Get parse results
parse_result = await parse_job.get_result()

```

### Custom evaluation

In order to get the highest OCR accuracy, you can run a custom evaluation workflow on your own data in SGP-Workflows. This includes evaluating the performance of the OCR models in Iris as well as comparing performance to Reducto.

You can evaluate OCR results in 2 ways:

1. Evaluating the OCR performance against the ground truth using DEX & eval cards in SGP Compass.
2. Evaluate the impact of the OCR results on your downstream tasks by plugging in your own downstream tasks and evaluation workflow into our SGP-workflow template in Compass.

***

## Supported Document Types

IRIS supports various document formats:

* **PDF documents** (.pdf)
* **Images** (.png, .jpg, .jpeg, .tiff)
* **Scanned documents** with printed or handwritten text
* **Multi-page documents**

***

## Parsing PDFs

### Basic PDF Parsing

```python theme={null}
from dex_sdk.types import IrisParseEngineOptions, IrisParseJobParams

# Upload PDF file
dex_file = await project.upload_file("path/to/document.pdf")

# Start IRIS parse job
parse_job = await dex_file.start_parse_job(
    IrisParseJobParams(
        options=IrisParseEngineOptions(),
    )
)

# Wait for and retrieve parse results
parse_result = await parse_job.get_result()

# Access parsed content
chunks = parse_result.data.content.chunks
for chunk in chunks:
    print(chunk.content)
```

### Complete PDF Example

```python theme={null}
import os
from dex_sdk.client import DexClient
from dex_sdk.file import DexFile
from dex_sdk.types import ProjectCredentials, SGPCredentials, IrisParseEngineOptions, IrisParseJobParams
from dex_core.models.files import FileEntity


async def parse_pdf_example():
    # Initialize client
    dex_client = DexClient(
        base_url="your-dex-url",
        account_id = os.getenv("SGP_ACCOUNT_ID"),
        api_key = os.getenv("SGP_API_KEY"),
    )

    # Create project
    project = await dex_client.create_project(name="ocr-project")

    # Upload file
    dex_file = await project.upload_file("document.pdf")

    # Verify file upload
    assert isinstance(dex_file, DexFile)
    assert isinstance(dex_file.data, FileEntity)

    # Verify file is in project
    files = await project.list_files()
    assert len(files) > 0, "Project should contain at least one file"

    # Get download URL if needed
    download_url = await dex_file.get_download_url()
    assert download_url.startswith("http")

    # Start IRIS parse job
    parse_job = await dex_file.start_parse_job(
        IrisParseJobParams(
            options=IrisParseEngineOptions(),
        )
    )

    # Get parse results
    parse_result = await parse_job.get_result()

    # Access parsed content
    content_length = len(parse_result.data.content.chunks[0].content)
    print(f"Parsed content length: {content_length}")
```

***

## Parsing Images

IRIS supports various image formats including PNG, JPG, and TIFF.

### Basic Image Parsing

```python theme={null}
# Upload image file
dex_file = await project.upload_file("path/to/image.png")

# Start IRIS parse job
parse_job = await dex_file.start_parse_job(
    IrisParseJobParams(
        options=IrisParseEngineOptions(),
    )
)

# Get parse results
parse_result = await parse_job.get_result()

# Access chunks
chunks = parse_result.data.content.chunks
print(f"Number of chunks: {len(chunks)}")
```

### Complete Image Example

```python theme={null}
async def parse_image_example():
    dex_client = DexClient(
        base_url="your-dex-url",
        account_id = os.getenv("SGP_ACCOUNT_ID"),
        api_key = os.getenv("SGP_API_KEY"),
    )

    project = await dex_client.create_project(name="ocr-project")

    # Upload PNG file
    dex_file = await project.upload_file("image.png")

    assert isinstance(dex_file, DexFile)
    assert isinstance(dex_file.data, FileEntity)

    # Verify file in project
    assert len(await project.list_files()) > 0

    # Get download URL
    download_url = await dex_file.get_download_url()
    assert download_url.startswith("http")

    # Start parse job
    parse_job = await dex_file.start_parse_job(
        IrisParseJobParams(
            options=IrisParseEngineOptions(),
        )
    )

    # Get results
    parse_result = await parse_job.get_result()

    # Access chunks
    print(f"Number of chunks: {len(parse_result.data.content.chunks)}")
```

***

## Uploading Files from Memory

You can also upload files directly from memory streams:

```python theme={null}
import io

# Create file from bytes
file_bytes = b"your file content here"
file_stream = io.BytesIO(file_bytes)

# Upload from stream
dex_file = await project.upload_file(file_stream, filename="document.pdf")

# Parse as usual
parse_job = await dex_file.start_parse_job(
    IrisParseJobParams(
        options=IrisParseEngineOptions(),
    )
)
```

***

## Configuration Options

### IrisParseEngineOptions

Customize IRIS behavior using parse engine options:

```python theme={null}
from dex_sdk.types import IrisParseEngineOptions

# Configure IRIS options
options = IrisParseEngineOptions(
    # Configuration parameters available based on your needs
    # (specific options depend on your Dex version)
)

# Use custom options
parse_job = await dex_file.start_parse_job(
    IrisParseJobParams(options=options)
)
```

***

## Understanding Parse Results

### Result Structure

IRIS returns parsed content organized into chunks, where each chunk represents a section of the document:

```python theme={null}
parse_result = await parse_job.get_result()

# Access all chunks
chunks = parse_result.data.content.chunks

# A ParseResult contains a list of ParseChunks. Each chunk has the
# aggregated text plus the underlying ParseBlocks (one per detected
# region) with bounding boxes, page numbers, and region types.

for i, chunk in enumerate(parse_result.data.content.chunks):
    print(f"Chunk {i}: {chunk.content[:100]}...")
    for block in chunk.blocks:
        print(f"  [{block.type}] page {block.page_number} "
            f"bbox={block.bbox} conf={block.confidence:.2f}")
        print(f"    {block.content[:80]}...")
```

### Working with Different Content Types

IRIS detects and processes different content regions:

Each ParseBlock.type is one of the layout labels IRIS detects. The current label set:

* **Text-family:** text, title, caption, list-item, section-header, page-header, page-footer, formula, footnote
* **Tables:** table
* **Images /** figures: picture

```python theme={null}
  TEXT_LABELS = {
      "text", "title", "caption", "list-item", "section-header",
      "page-header", "page-footer", "formula", "footnote",
  }

  text_blocks = [
      block
      for chunk in parse_result.data.content.chunks
      for block in chunk.blocks
      if block.type in TEXT_LABELS
  ]

  table_blocks = [
      block
      for chunk in parse_result.data.content.chunks
      for block in chunk.blocks
      if block.type == "table"
  ]

  image_blocks = [
      block
      for chunk in parse_result.data.content.chunks
      for block in chunk.blocks
      if block.type == "picture"
  ]
```

***

## File Management

### List Files in Project

```python theme={null}
# List all files
files = await project.list_files()

for file in files:
    print(f"File: {file.data.name}")
    print(f"  ID: {file.data.id}")
    print(f"  Size: {file.data.size_bytes} bytes")
```

### Get File Download URL

```python theme={null}
# Get pre-signed S3 URL for downloading
download_url = await dex_file.get_download_url()

# Use URL with requests or other HTTP client
import requests
response = requests.get(download_url)
file_content = response.content
```

### Check File Details

```python theme={null}
# Access file metadata
print(f"File name: {dex_file.data.name}")
print(f"File ID: {dex_file.data.id}")
print(f"File size: {dex_file.data.size_bytes} bytes")

# Check if file is a FileEntity
from dex_core.models.files import FileEntity
assert isinstance(dex_file.data, FileEntity)
```

***

## Error Handling

### Handle Upload Errors

```python theme={null}
try:
    dex_file = await project.upload_file("path/to/file.pdf")
except FileNotFoundError:
    print("File not found")
except Exception as e:
    print(f"Upload failed: {e}")
```

### Handle Parse Errors

```python theme={null}
try:
    parse_job = await dex_file.start_parse_job(
        IrisParseJobParams(
            options=IrisParseEngineOptions(),
        )
    )
    parse_result = await parse_job.get_result()
except Exception as e:
    print(f"Parse failed: {e}")
```

***

## Processing Multiple Documents

### Batch Processing

```python theme={null}
import asyncio

async def process_documents(file_paths):
    # Upload all files
    dex_files = []
    for path in file_paths:
        dex_file = await project.upload_file(path)
        dex_files.append(dex_file)

    # Start all parse jobs
    parse_jobs = []
    for dex_file in dex_files:
        job = await dex_file.start_parse_job(
            IrisParseJobParams(options=IrisParseEngineOptions())
        )
        parse_jobs.append(job)

    # Wait for all to complete
    results = []
    for job in parse_jobs:
        result = await job.get_result()
        results.append(result)

    return results

# Usage
file_paths = ["doc1.pdf", "doc2.pdf", "doc3.pdf"]
results = await process_documents(file_paths)
```

***

## Multi-Language Support

IRIS supports OCR in multiple languages, including:

* **Latin-based languages**: English, Spanish, French, German, Italian, Portuguese, etc.
* **Arabic**: With specialized model support
* **Asian languages**: Chinese, Japanese, Korean
* **Other languages**: Russian, Hebrew, Hindi, Thai, and more

The appropriate language models are selected automatically based on the document content.

***

## Best Practices

### Use Environment Variables for Credentials

```python theme={null}
import os

# Load credentials from environment
account_id = os.getenv("SGP_ACCOUNT_ID")
api_key = os.getenv("SGP_API_KEY")

if not account_id or not api_key:
    raise ValueError("SGP credentials not found in environment")

credentials = SGPCredentials(
    account_id=account_id,
    api_key=api_key
)
```

### Organize Files by Project

```python theme={null}
# Create client
dex_client = DexClient(
    base_url="your-dex-url",
    api_key=sgp_credentials.api_key,
    account_id=sgp_credentials.account_id,
)

# Create separate projects for different document types
invoices_project = await dex_client.create_project(
    name="invoices"
)

contracts_project = await dex_client.create_project(
    name="contracts"
)
```

### Wait for Parse Completion

```python theme={null}
# Parse jobs are async - ensure completion before accessing results
parse_job = await dex_file.start_parse_job(
    IrisParseJobParams(options=IrisParseEngineOptions())
)

# This waits for completion
parse_result = await parse_job.get_result()

# Now safe to access results
chunks = parse_result.data.content.chunks
```

***

## Common Use Cases

### Extract All Text from Document

```python theme={null}
def extract_all_text(parse_result):
    """Extract all text content from parse result."""
    all_text = []
    for chunk in parse_result.data.content.chunks:
        all_text.append(chunk.content)
    return "\n\n".join(all_text)

# Usage
parse_result = await parse_job.get_result()
full_text = extract_all_text(parse_result)
print(full_text)
```

### Process Forms and Structured Documents

```python theme={null}
# IRIS automatically detects and extracts structured content
parse_result = await parse_job.get_result()

# Access structured chunks
for chunk in parse_result.data.content.chunks:
    # Process based on content type
    if hasattr(chunk, 'region_type'):
        if chunk.region_type == 'table':
            # Handle table data
            process_table(chunk.content)
        elif chunk.region_type == 'text':
            # Handle text data
            process_text(chunk.content)
```

***

## Performance Considerations

### Processing Time

Parse job duration depends on:

* Document length (number of pages)
* Image resolution and quality
* Content complexity (tables, mixed layouts)
* Selected OCR models

### Optimization Tips

* **Batch processing**: Process multiple documents concurrently when possible
* **Pre-processing**: Ensure documents are properly oriented and of good quality
* **Project organization**: Group related documents in the same project for better management

***

## Next Steps

Now that you understand how to use IRIS:

1. Review the [Introduction to IRIS](/docs/capabilities/ocr/introduction-to-iris) to learn about the underlying OCR pipeline
2. Explore the [Dex documentation](/docs/capabilities/document-understanding/introduction-to-dex) for additional capabilities
3. Integrate IRIS into your document processing workflows
4. Use parsed results with Dex's extraction and vector store features

For questions or support, please contact the Scale AI team.


# Introduction to IRIS
Source: https://docs.gp.scale.com/docs/capabilities/ocr/introduction-to-iris



**IRIS is Scale's OCR capability that transforms document images and PDFs into structured text.** It provides a comprehensive "layout-then-OCR" pipeline that combines **layout detection, specialized OCR models, and intelligent assembly** to extract meaningful information from complex documents.

***

## When to Use IRIS?

IRIS is designed for teams that need:

* **Custom OCR pipeline control:** Fine-tune layout detection, OCR models, and assembly for specialized document types
* **Maximum accuracy optimization:** Experiment with 15+ OCR models to find the best fit for your specific documents
* **Minimum latency requirements:** Optimize processing speed for your specific use case through custom configuration
* **Specialized model development:** Build and integrate custom OCR models or layout detectors for unique document challenges
* **Multi-language experimentation:** Test different OCR engines for non-Germanic languages to find optimal accuracy

### For Standard Production Use

If you need stable batch document processing without custom requirements, **use Reducto within Dex**:

* **Production stability** with proven reliability
* **Good for batch processing** with somewhat high but acceptable latency
* **Auto-scaling** with your workload
* **Best accuracy** for English and Germanic languages
* **Lower operational complexity**

See the **[When to choose Iris?](/docs/capabilities/ocr/when-to-choose-iris)** guide for a detailed comparison.

***

## IRIS Capabilities

IRIS provides advanced capabilities for teams with specialized OCR needs:

* **Layout-Aware Processing:** Automatically detects and classifies different regions (text, tables, images, etc.) before applying specialized OCR
* **15+ OCR Models:** Choose from Tesseract, EasyOCR, PaddleOCR, Surya, GPT-4o, Gemini, and specialized models for Arabic and other languages
* **Table-Specific Processing:** Apply specialized OCR models optimized for tabular data extraction
* **Extensible Architecture:** Easily add custom OCR models or layout detectors through simple adapter interfaces
* **Multi-Language Flexibility:** Test and optimize for documents in multiple languages including Arabic with specialized support
* **Complete Pipeline Control:** Inspect and configure each step—layout detection, content extraction, and assembly

***

## How IRIS Works

IRIS follows a three-stage pipeline:

### 1. Layout Detection

Detects and classifies regions in document images with bounding boxes and content types:

* **Text regions** for paragraphs and body content
* **Table regions** for structured tabular data
* **Image regions** for figures and diagrams
* **Other content types** as needed

Supported layout detection models:

* **rt\_detr\_bce:** RT-DETR model (best performance)
* **surya\_bce:** Surya layout detection
* **yolo\_bce:** YOLO-based detection

### 2. OCR Processing

Applies specialized OCR models to each detected region based on content type:

* **Text OCR:** Optimized for continuous text (paragraphs, headings, etc.)
* **Table OCR:** Specialized models for accurate table extraction
* **Image Handling:** Preserves image crops for inclusion in output

Available OCR engines include:

* **Open-source models:** EasyOCR, Tesseract, PaddleOCR, Surya
* **Vision-language models:** GPT-4o, Gemini (via LiteLLM), Qwen2-VL, Qwen2.5-VL
* **Specialized models:** Arabic Nougat (small/base/large), QAARI, MBZUAI-AIN
* **Cloud services:** Azure Computer Vision
* **Advanced models:** SmolDocling

### 3. Assembly

Combines extracted content back into useful formats:

* **MarkdownAssembler:** Converts to structured markdown (default)
* **JSONAssembler:** Outputs detailed JSON for debugging and inspection

***

## Core Features

### Modular Architecture

Each pipeline component (layout detector, OCR model, assembler) is independently configurable and extensible. Add custom models by implementing simple adapter interfaces.

### Inspection and Debugging

Save intermediate results at each pipeline stage to understand and optimize the processing:

* Layout detection bounding boxes and classifications
* Individual OCR outputs per region
* Final assembled output

### Configuration Management

Use command-line arguments or YAML configuration files for reproducible processing:

```yaml theme={null}
layout: rt_detr_bce
text_ocr: easyocr
table_ocr: gemini
output_dir: results
save_intermediate: true
```

### Performance Optimization

First-time model usage downloads weights; subsequent runs use cached models for faster processing.

***

## Common Use Cases

* **Document Digitization:** Convert scanned documents and PDFs to searchable text
* **Form Processing:** Extract structured data from forms with mixed content types
* **Table Extraction:** Accurately capture tabular data from complex documents
* **Multi-Language Documents:** Process documents in various languages including Arabic
* **Research and Analysis:** Extract text and data from academic papers, reports, and technical documents
* **Archive Processing:** Batch process large document collections

***

## Getting Started

IRIS is integrated with Dex for seamless document processing:

### Python Package

Use programmatically through the Dex SDK:

```python theme={null}
from dex_sdk.client import DexClient
from dex_core.models.parse_job import IrisParseEngineOptions, IrisParseJobParams

# Initialize client and project
dex_client = DexClient(base_url="your-dex-url")
project = await dex_client.create_project(name="ocr-project")

# Upload and parse document
dex_file = await project.upload_file("document.pdf")
parse_job = await dex_file.start_parse_job(
    IrisParseJobParams(options=IrisParseEngineOptions())
)
result = await parse_job.get_result()
```

***

## Language Support

IRIS supports multi-language OCR with models optimized for different language families. Specialized support includes:

* **Arabic:** Arabic Nougat models, QAARI, MBZUAI-AIN, Gemini
* **Multi-language:** EasyOCR, Tesseract, PaddleOCR with broad language coverage
* **Universal:** Vision-language models (GPT-4o, Gemini, Qwen) supporting many languages

***

## Key Advantages

### Extensibility

Designed for easy extension. Add support for new OCR libraries, layout detectors, or assembly formats through simple adapter classes.

### Transparency

Inspect every pipeline stage to understand model behavior, debug issues, and optimize results for your specific documents.

### Flexibility

Mix and match components—use different OCR models for text vs. tables, choose layout detectors based on document type, and customize assembly output.

### API Integration

Vision-language models (Gemini, GPT-4o) use LiteLLM for unified API access, making it easy to leverage advanced AI capabilities.

***

## Next Steps

To start using IRIS:

1. Review the [Getting Started with IRIS](/docs/capabilities/ocr/getting-started-with-iris) guide for step-by-step instructions
2. Learn about IRIS's multi-stage OCR pipeline and configuration options
3. Integrate IRIS into your Dex-based document processing workflows

IRIS provides the building blocks for robust document OCR while giving you complete control over the processing pipeline.


# DEX model engine choices
Source: https://docs.gp.scale.com/docs/capabilities/ocr/when-to-choose-iris

A guide to the DEX model engine options

DEX is the SDK that runs document parsing, extraction, retrieval and research flows. There are currently two options for the underlying parsing engine.

1. *IRIS* - Currently on V2, IRIS is Scale's proprietary OCR & document extraction model that has the best performance across both arabic & english texts. This is highly customisable with different models available for layout and parsing steps with high parallelism across pages or sections available in later versions.
2. *REDUCTO* - Currently being deprecated - Reducto is a third party provider that provides end to end parsing of a wide range of documents without the performance & customisation available with IRIS V2

## Quick Decision Guide

### Use Iris (Default engine)

**Iris(v2) is the recommended default DEX engine for all production use cases:**

* You can **maximize accuracy** through testing any OCR model uploaded to SGP
* You can **minimize latency** through specialized optimization
* You need **greater transparency** of the processes within Temporal
* You can use **specialized models** that are either open source or have been built or specifically for unique document types
* You can \*\*customise and control pipelines \*\* for layout detection, OCR, and assembly

### Use Reducto (Soon to be Deprecated)

**Reducto is the legacy OCR DEX engine for most production use cases:**

* You can accept a *third party service* for OCR and parsing integrated into your application
* You need **proven stability** and do not need future functionality or updates from the DEX team
* You're doing **batch processing** where somewhat high latency is acceptable
* You want **automatic scaling** with your workload
* You're processing **English or Germanic language documents**
* You need a **low-complexity, managed solution**

## Understanding the Relationship Between Iris and Dex

**Iris and Dex are not mutually exclusive**—they're complementary. Dex is a document understanding platform that provides primitives for file management, parsing, vector stores, and data extraction. Iris is one of several OCR engines available within Dex.

Think of it this way: *Dex is the platform, Iris is one of the engines*. When you use Dex, you choose which OCR engine to use for parsing:

* **Iris**: Better accuracy & latency for all languages including Arabic & English
* **Custom engines**: Integrate your own OCR solution
* **Reducto**: (Deprecated)

## What is Iris?

Iris is Scale's OCR capability that provides a flexible, modular pipeline for extracting text from documents. It's designed for teams with custom OCR needs who want complete control over the processing pipeline.

**Iris offers:**

* **Customisable OCR models**: Tesseract, EasyOCR, PaddleOCR, Surya, GPT-5.4, Gemini, and any other specialized models uploaded to SGP
* **Complete pipeline control**: Configure layout detection, OCR processing, and assembly separately
* **Inspection capabilities**: Save and review intermediate results at each processing stage
* **Extensibility**: Add custom OCR models through SGP or layout detectors through simple adapters
* **Optimization flexibility**: Fine-tune for maximum accuracy or minimum latency based on your needs
* **Custom pipeline experimentation**: Test different combinations of layout detectors and OCR models
* **Specialized model development**: Build custom OCR for unique document types or languages
* **Performance optimization**: Tune for specific accuracy or latency requirements
* **Non-Germanic language optimization**: Experiment with models to find best accuracy for other languages such as Arabic.

## What is Dex?

Dex is Scale's document understanding platform—a production-ready service that transforms unstructured documents into actionable, structured data. It provides:

* **File Management**: Secure upload, storage, and retrieval with access control
* **Document Parsing**: Convert documents (PDF, DOCX, images) into structured JSON using multiple OCR engines
* **Vector Stores**: Index and search parsed documents with semantic embeddings
* **Data Extraction**: Extract information using custom schemas, prompts, and RAG-enhanced context
* **Project Management**: Organize and isolate data with proper authorization
* **Automatic scaling** with your workload
* **Multiple OCR engine options**: Iris (Recommended default, V2) & Reducto (soon to be deprecated)

## Feature Comparison

| Feature              | Iris (✅ Recommended)                                                                                                    | Reducto (⚠️ Soon to be Deprecated)                                 |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ |
| **Engine**           | Scale proprietary OCR (V2)                                                                                              | Third-party end-to-end parsing                                     |
| **Best For**         | Custom OCR control, specialized requirements                                                                            | Low-complexity managed solution, legacy production                 |
| **Performance**      | Best accuracy and latency across Arabic and English; highly customizable                                                | Wide document support without Iris V2 performance or customization |
| **OCR Models**       | Customisable OCR models (e.g. Tesseract, EasyOCR, PaddleOCR, Surya, GPT-5.4, Gemini, and more) throguh SGP model engine | Managed Reducto engine                                             |
| **Pipeline Control** | Configure layout detection and OCR as needed                                                                            | Managed end-to-end                                                 |
| **Languages**        | All languages, including Arabic                                                                                         | English or Germanic documents                                      |
| **Latency**          | Minimizable through high parallelism (V2)                                                                               | Somewhat high (acceptable for batch)                               |
| **Transparency**     | Greater visibility into processes within Temporal                                                                       | Third-party service integrated into your application               |
| **Future updates**   | Active development from the DEX team                                                                                    | No future functionality or updates from the DEX team               |
| **Typical use case** | Maximize accuracy, minimize latency, specialized or custom models                                                       | Batch processing where higher latency is acceptable                |

## Recommended Workflow

### For Standard Production Applications

**Use Iris as the DEX default model engine:**

1. **SOTA performance**: Accuracy optimization, latency minimization, or specialized models
2. **Customisability**: Test any OCR model hosted in SGP and pipeline configurations
3. **Pipeline control**: Configure layout detection and OCR as separate steps
4. **Language coverage**: Best accuracy and latency for Arabic, English, and other non-Germanic languages
5. **Transparency**: Inspect and debug processing stages within Temporal
6. **Extensibility**: Add custom OCR models or layout detectors through adapters
7. **Future-ready**: Benefit from ongoing development and Iris V2 parallelism improvements from the DEX team

**Use Reducto within Dex** for reliable, production-ready document processing:

1. **Managed infrastructure**: Rely on managed infrastructure for operational simplicity
2. **Language fit**: Process English or Germanic-language documents where Iris customization is not required
3. **Low complexity**: Use a third-party, end-to-end managed parsing service with minimal pipeline setup

## Getting Started

### Dex Documentation

* [Introduction to Dex](/docs/capabilities/document-understanding/introduction-to-dex)
* [Getting Started with Dex](/docs/capabilities/document-understanding/getting-started-with-dex)

### Iris Documentation

* [Introduction to Iris](/docs/capabilities/ocr/introduction-to-iris)
* [Getting Started with Iris](/docs/capabilities/ocr/getting-started-with-iris)

## Summary

* **Dex is the platform**: DEX runs parsing, extraction, retrieval, and research flows—you choose the underlying engine when you parse documents.
* **Iris (V2) is the recommended default**: Scale’s proprietary engine with the best Arabic and English performance, customisable OCR models, and full control over layout, OCR, and assembly.
* **Choose Iris when you need customization**: Optimize accuracy or latency, inspect pipelines in Temporal, use specialized or custom models, and support non-Germanic languages.
* **Reducto is legacy and being deprecated**: A third-party, managed end-to-end option with no future DEX updates—use only for existing low-complexity or English/Germanic batch workloads.
* **Reducto still fits narrow cases**: Proven stability, auto-scaling, and acceptable batch latency when you do not need Iris V2 performance or pipeline control.

Need help deciding? Contact the Dex team at `#dex-help` on Slack.


# Custom Images
Source: https://docs.gp.scale.com/docs/capabilities/training/custom-images

Push Docker images to the SGP registry and use them in training jobs.

Each SGP workspace has a Docker-compatible registry. You push images to it and reference them in job configs; Train takes care of getting the right image onto the right cloud backend.

***

## Push an Image

<Steps>
  <Step title="Authenticate">
    Log in with your SGP credentials:

    ```bash theme={null}
    docker login docker-registry.your-sgp-deployment-url \
      --username $SGP_ACCOUNT_ID \
      --password $SGP_API_KEY
    ```
  </Step>

  <Step title="Build and tag">
    ```bash theme={null}
    docker build -t my-training-job:v1 .
    docker tag my-training-job:v1 docker-registry.your-sgp-deployment-url/my-training-job:v1
    ```
  </Step>

  <Step title="Push">
    ```bash theme={null}
    docker push docker-registry.your-sgp-deployment-url/my-training-job:v1
    ```
  </Step>

  <Step title="Reference in your job config">
    Use the registry URL directly in your job config. Train resolves it automatically at submission:

    ```json theme={null}
    "image_uri": "docker-registry.your-sgp-deployment-url/my-training-job:v1"
    ```

    Train ensures the exact image you pushed is what runs, even if the tag is updated later.
  </Step>
</Steps>

## Multi-Architecture Builds

Cloud training instances run `linux/amd64`. If you're building on Apple Silicon, cross-compile explicitly:

```bash theme={null}
docker buildx build \
  --platform linux/amd64 \
  -t docker-registry.your-sgp-deployment-url/my-training-job:v1 \
  --push .
```

<Warning>
  Pushing a `linux/arm64` image to an `amd64` compute instance will cause the job to fail at startup with an exec format error.
</Warning>

***

## Example Dockerfile

```dockerfile theme={null}
FROM pytorch/pytorch:2.2.0-cuda12.1-cudnn8-runtime

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY train.py .

CMD ["python3", "train.py"]
```

Your training script accesses data through `STORAGE_MOUNT_*_PATH` env vars, with no cloud-specific paths needed:

```python focus={3-4} theme={null}
import os
import torch

data_dir = os.environ["STORAGE_MOUNT_0_PATH"]
output_dir = os.environ["STORAGE_MOUNT_1_PATH"]

# ... training loop ...
torch.save(model.state_dict(), f"{output_dir}/model.pt")
```

***

## Next Steps

* **[Storage Mounts](/docs/capabilities/training/storage-mounts)**: Configure data and output directories
* **[Getting Started](/docs/capabilities/training/getting-started-with-train)**: Full job submission examples per backend


# Getting Started with Train
Source: https://docs.gp.scale.com/docs/capabilities/training/getting-started-with-train

Submit your first GPU training job on Vertex AI, SageMaker, or Azure ML.

This guide walks you through submitting a training job using the Train API.

<h2>Prerequisites</h2>

Before you begin, ensure you have:

* A valid Scale account with SGP access
* Your API key and account ID:

```bash theme={null}
export SGP_API_KEY="your_api_key"
export SGP_ACCOUNT_ID="your_account_id"
```

* A Docker image pushed to the workspace registry (see [Custom Images](/docs/capabilities/training/custom-images))

<h2>Submit a Job</h2>

Jobs are submitted to `POST /v1/{backend}/jobs`. The `{backend}` is one of `vertex-ai`, `sagemaker`, or `azure-ml`. The `job_config` structure mirrors each cloud's native job API.

<Tabs>
  <Tab title="Vertex AI">
    ```bash theme={null}
    curl -X POST "https://train.your-sgp-deployment-url/v1/vertex-ai/jobs" \
      -H "x-api-key: $SGP_API_KEY" \
      -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
      -H "Content-Type: application/json" \
      -d '{
        "name": "mnist-training",
        "job_config": {
          "worker_pool_specs": [{
            "machine_spec": {
              "machine_type": "g2-standard-8",
              "accelerator_type": "NVIDIA_L4",
              "accelerator_count": 1
            },
            "replica_count": 1,
            "container_spec": {
              "image_uri": "docker-registry.your-sgp-deployment-url/pytorch-gpu:v1",
              "command": ["python3", "train.py"],
              "args": ["--epochs", "3"]
            }
          }],
          "scheduling": {"timeout": "86400s"}
        },
        "storage_mounts": [
          {"source_uri": "gs://my-bucket/mnist-data", "mount_path": "/mnt/data", "read_only": true},
          {"source_uri": "gs://my-bucket/output/run-1", "mount_path": "/mnt/output", "read_only": false}
        ]
      }'
    ```

    ```json Response highlight={4} theme={null}
    {
      "id": "87cae349-cd5f-4202-b674-f1652880a8f5",
      "name": "mnist-training",
      "job_type": "vertex_ai",
      "status": "PENDING",
      "cloud_job_identifier": {
        "job_name": "projects/my-project/locations/us-east1/customJobs/1234567890"
      },
      "storage_mounts": [...],
      "account_id": "693af856dde70fd6614ab64c",
      "created_at": "2026-03-11T08:12:25.542507Z",
      "updated_at": "2026-03-11T08:12:25.542507Z"
    }
    ```

    `worker_pool_specs` is an array; add more pools for distributed training. `command` and `args` are separate arrays.
  </Tab>

  <Tab title="SageMaker">
    ```bash theme={null}
    curl -X POST "https://train.your-sgp-deployment-url/v1/sagemaker/jobs" \
      -H "x-api-key: $SGP_API_KEY" \
      -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
      -H "Content-Type: application/json" \
      -d '{
        "name": "mnist-training",
        "job_config": {
          "image": "docker-registry.your-sgp-deployment-url/pytorch-gpu:v1",
          "command": ["python3", "/app/train.py"],
          "resource_config": {
            "instance_count": 1,
            "instance_type": "ml.g4dn.xlarge",
            "volume_size_in_gb": 50
          },
          "hyperparameters": {"learning_rate": "0.001"}
        },
        "storage_mounts": [
          {"source_uri": "s3://my-bucket/mnist-data", "mount_path": "/mnt/data", "read_only": true},
          {"source_uri": "s3://my-bucket/output/run-1", "mount_path": "/mnt/output", "read_only": false}
        ]
      }'
    ```

    ```json Response highlight={4} theme={null}
    {
      "id": "3f2a1b9c-88de-4a12-b901-c2e5f7d04321",
      "name": "mnist-training",
      "job_type": "sagemaker",
      "status": "PENDING",
      "cloud_job_identifier": {
        "training_job_name": "mnist-training-2026-03-11-08-12-25"
      },
      "storage_mounts": [...],
      "account_id": "693af856dde70fd6614ab64c",
      "created_at": "2026-03-11T08:12:25.542507Z",
      "updated_at": "2026-03-11T08:12:25.542507Z"
    }
    ```

    `instance_type` determines GPU: `ml.g4dn.xlarge` = T4, `ml.p4de.24xlarge` = 8×A100 80GB.
  </Tab>

  <Tab title="Azure ML">
    ```bash theme={null}
    curl -X POST "https://train.your-sgp-deployment-url/v1/azure-ml/jobs" \
      -H "x-api-key: $SGP_API_KEY" \
      -H "x-selected-account-id: $SGP_ACCOUNT_ID" \
      -H "Content-Type: application/json" \
      -d '{
        "name": "mnist-training",
        "job_config": {
          "command": "python train.py --epochs 3",
          "environment": {
            "image": "docker-registry.your-sgp-deployment-url/pytorch-gpu:v1"
          },
          "compute": "gpu-cluster",
          "resources": {
            "instance_type": "Standard_NC24ads_A100_v4",
            "instance_count": 1
          },
          "environment_variables": {"LEARNING_RATE": "0.001"}
        },
        "storage_mounts": [
          {"source_uri": "azureml://datastores/traindata/paths/mnist", "mount_path": "/mnt/data", "read_only": true},
          {"source_uri": "azureml://datastores/trainoutput/paths/run-1", "mount_path": "/mnt/output", "read_only": false}
        ]
      }'
    ```

    ```json Response highlight={4} theme={null}
    {
      "id": "a1b2c3d4-ef56-7890-abcd-ef1234567890",
      "name": "mnist-training",
      "job_type": "azure_ml",
      "status": "PENDING",
      "cloud_job_identifier": {
        "job_name": "heroic_grass_22zggyxsxp"
      },
      "storage_mounts": [...],
      "account_id": "693af856dde70fd6614ab64c",
      "created_at": "2026-03-11T08:12:25.542507Z",
      "updated_at": "2026-03-11T08:12:25.542507Z"
    }
    ```

    `command` is a single shell string (not an array). `compute` can be a named cluster or `"serverless"`.
  </Tab>
</Tabs>

<h2>Job Response</h2>

All backends return the same job object shape:

<ResponseField name="id" type="string">
  Unique job identifier. Use this to poll status or cancel the job.
</ResponseField>

<ResponseField name="status" type="string">
  Current job state. Transitions through `PENDING → IN_PROGRESS → COMPLETED / FAILED / CANCELED / EXPIRED`.
</ResponseField>

<ResponseField name="cloud_job_identifier" type="object">
  The cloud provider's native job identifier. Shape varies by backend: `job_name` for Vertex AI and Azure ML, `training_job_name` for SageMaker.
</ResponseField>

<ResponseField name="result" type="object">
  Populated on failure. Contains `failure_reason` with the error message from the cloud provider.
</ResponseField>

<h2>Check Status</h2>

```bash theme={null}
curl "https://train.your-sgp-deployment-url/v1/jobs/$JOB_ID" \
  -H "x-api-key: $SGP_API_KEY" \
  -H "x-selected-account-id: $SGP_ACCOUNT_ID"
```

<h2>Other Endpoints</h2>

**List jobs:**

```bash theme={null}
curl "https://train.your-sgp-deployment-url/v1/jobs" \
  -H "x-api-key: $SGP_API_KEY" \
  -H "x-selected-account-id: $SGP_ACCOUNT_ID"
```

**Cancel a job:**

```bash theme={null}
curl -X POST "https://train.your-sgp-deployment-url/v1/jobs/$JOB_ID/cancel" \
  -H "x-api-key: $SGP_API_KEY" \
  -H "x-selected-account-id: $SGP_ACCOUNT_ID"
```

Returns `409` if the job is already in a terminal state.

<h2>Accessing Mounts in Your Container</h2>

Inside the training container, mounted storage is available via environment variables regardless of cloud:

```python theme={null}
import os

data_path = os.environ["STORAGE_MOUNT_0_PATH"]
output_path = os.environ["STORAGE_MOUNT_1_PATH"]
```

<h2>Next Steps</h2>

* **[Custom Images](/docs/capabilities/training/custom-images)**: Build and push your own training containers
* **[Storage Mounts](/docs/capabilities/training/storage-mounts)**: Mount multiple datasets and configure output paths


# Introduction to Train
Source: https://docs.gp.scale.com/docs/capabilities/training/introduction-to-train

Run GPU training jobs across GCP, AWS, and Azure through a unified API.

**Train is SGP's multi-cloud training job service.** It provides a unified REST API for submitting GPU training jobs to Vertex AI (GCP), SageMaker (AWS), and Azure ML, handling cloud-specific job submission, image resolution, storage mounting, and status tracking transparently.

***

## Why Use Train?

Each cloud ML platform has its own job submission API, container registry, storage format, and authentication model. Teams that want cloud flexibility end up maintaining separate training pipelines per cloud, or locking into one provider early.

Train solves this by sitting in front of all three clouds with a single API surface. You submit a job with an image and a command; Train takes care of pushing the image to the right registry, mounting your storage, and translating the job config into the cloud's native format. Your training code doesn't know or care which cloud it's running on.

* **Single API for all backends:** `vertex-ai`, `sagemaker`, and `azure-ml` are selectable per job
* **Unified image registry:** Push Docker images once to the registry proxy; Train resolves them to the correct cloud registry at submission time
* **Cross-cloud storage mounts:** Mount GCS, S3, or Azure Blob Storage with `STORAGE_MOUNT_*_PATH` env vars, using the same interface everywhere
* **Consistent job lifecycle:** `PENDING → IN_PROGRESS → COMPLETED / FAILED / CANCELED / EXPIRED` across all backends

***

## How It Works

### Train Service

The Train service has two components: a gateway that accepts job submissions and returns a job ID, and a background poller that tracks running jobs and updates their status.

### Supported Backends

| Backend   | Cloud service         | `{backend}` value |
| --------- | --------------------- | ----------------- |
| Vertex AI | GCP Custom Jobs       | `vertex-ai`       |
| SageMaker | AWS Training Jobs     | `sagemaker`       |
| Azure ML  | Azure ML Command Jobs | `azure-ml`        |

### Container Registry

Each workspace has a Docker-compatible registry. You push images to it; Train ensures the exact image you pushed is what runs on the cloud backend, even if the tag is updated later.

***

## Storage Mounts

Training containers on all clouds receive `STORAGE_MOUNT_0_PATH`, `STORAGE_MOUNT_1_PATH`, etc. as environment variables pointing to mounted directories. Your training script reads from `os.environ["STORAGE_MOUNT_0_PATH"]`, with no cloud-specific paths or SDKs needed.

Source URIs use cloud-native formats:

| Cloud | URI format                             |
| ----- | -------------------------------------- |
| GCP   | `gs://bucket/path`                     |
| AWS   | `s3://bucket/path`                     |
| Azure | `azureml://datastores/name/paths/path` |

***

## Common Use Cases

* **Improve agent accuracy on your domain:** Fine-tune a foundation model on your proprietary data so your agents produce more reliable, on-brand responses and make fewer errors on the tasks that matter to your business.
* **Optimize agent performance with feedback:** Use evaluations and human feedback collected in SGP to run reinforcement learning pipelines that directly improve how your agents behave over time.
* **Reduce inference cost and latency:** Distill a large general-purpose model into a smaller one trained specifically for your use case, so agents respond faster and cost less to run at scale.

***

## Authentication

Train uses the same authentication as other SGP services:

```
x-api-key: <your_api_key>
x-selected-account-id: <your_account_id>
```

The registry proxy accepts Docker Basic auth (`account_id:api_key`) during `docker login`, then issues a short-lived token for push and pull operations.

***

## Getting Started

To begin using Train, you'll need a Scale account with SGP access and credentials for at least one cloud backend.

<CardGroup>
  <Card title="Getting Started" icon="rocket" href="/docs/capabilities/training/getting-started-with-train">
    Submit your first training job
  </Card>

  <Card title="Custom Images" icon="docker" href="/docs/capabilities/training/custom-images">
    Push and use your own Docker images
  </Card>

  <Card title="Storage Mounts" icon="hard-drive" href="/docs/capabilities/training/storage-mounts">
    Mount training data and output directories across clouds
  </Card>
</CardGroup>


# Storage Mounts
Source: https://docs.gp.scale.com/docs/capabilities/training/storage-mounts

Mount cloud storage into training containers using a unified interface across GCP, AWS, and Azure.

Train mounts cloud storage into your container before the job starts and sets `STORAGE_MOUNT_0_PATH`, `STORAGE_MOUNT_1_PATH`, etc. as environment variables. Your training script reads from these paths, with no cloud-specific SDKs or storage code needed.

***

## Mount Configuration

Each entry in `storage_mounts` maps a cloud URI to a container path:

```json theme={null}
"storage_mounts": [
  {
    "source_uri": "gs://my-bucket/training-data",
    "mount_path": "/mnt/data",
    "read_only": true
  },
  {
    "source_uri": "gs://my-bucket/output/run-1",
    "mount_path": "/mnt/output",
    "read_only": false
  }
]
```

Inside the container:

```
STORAGE_MOUNT_0_PATH=/mnt/data
STORAGE_MOUNT_1_PATH=/mnt/output
```

Mounts are indexed in order, starting at zero.

***

## URI Formats

| Cloud | Format                                 | Example                                        |
| ----- | -------------------------------------- | ---------------------------------------------- |
| GCP   | `gs://bucket/path`                     | `gs://my-bucket/datasets/imagenet`             |
| AWS   | `s3://bucket/path`                     | `s3://my-bucket/datasets/imagenet`             |
| Azure | `azureml://datastores/name/paths/path` | `azureml://datastores/training/paths/imagenet` |

<Note>
  Azure storage mounts reference registered Azure ML datastores, not raw Blob Storage URLs. Datastores are configured in your Azure ML workspace.
</Note>

***

## Accessing Mounts in Your Container

```python focus={5-6} theme={null}
import os
import torch
from datasets import load_dataset

data_path = os.environ["STORAGE_MOUNT_0_PATH"]
output_path = os.environ["STORAGE_MOUNT_1_PATH"]

dataset = load_dataset(data_path)

# ... training loop ...
torch.save(model.state_dict(), f"{output_path}/checkpoint-epoch-{epoch}.pt")
```

This works identically on GCP, AWS, and Azure.

***

## Read vs. Write

Set `read_only` based on intent. Use `true` for input data and pretrained weights; `false` for outputs, checkpoints, and logs. Marking input mounts read-only prevents accidental writes and may improve performance on some backends.

***

## SageMaker: Input Data Config

SageMaker also supports `input_data_config` for S3 input channels. Unlike FUSE mounts, channels are downloaded to the instance before training starts, which can be faster for large datasets with random access patterns:

```json theme={null}
"input_data_config": [{
  "channel_name": "train",
  "data_source": {
    "s3_data_source": {
      "s3_data_type": "S3Prefix",
      "s3_uri": "s3://my-bucket/datasets/train/",
      "s3_data_distribution_type": "FullyReplicated"
    }
  },
  "input_mode": "File"
}]
```

SageMaker makes this available at `/opt/ml/input/data/train/`. This coexists with `storage_mounts`.

***

## Next Steps

* **[Getting Started](/docs/capabilities/training/getting-started-with-train)**: Full job submission examples per backend
* **[Custom Images](/docs/capabilities/training/custom-images)**: Build and push training containers


# Guide to document understanding in Workflows
Source: https://docs.gp.scale.com/docs/capabilities/workflows/dex-parsing-in-workflows

Tutorial for utilizing unstructured data for your workflows

This guide walks you through parsing documents and extracting structured data in Workflows to help you utilize any unstructured data for your workflows (powered by our `Dex SDK`).
You'll learn how to connect to cloud file storage, filter and parse documents, extract structured data, store parsed data to Knowledge Base, and use the results in your workflows.

### Step 1: Connect to cloud file storage

Start by navigating to `Workflows` in SGP. Click on the `Cloud Storage Browser` card to connect to your cloud file storage.

<img alt="Cloud Storage Browser Card" />

Currently, Workflows supports the following cloud storage providers:

* Amazon S3
* Azure Blob Storage
* Google Cloud Storage

For this example, we'll connect to an Amazon S3 bucket containing resume documents that we want to parse and analyze. Select your cloud storage provider and choose your registered connection from the dropdown. Browse or specify the path to the files you want to process.

<Note>
  To add an S3 bucket for use with the Cloud Storage Browser card, you must first create a bucket using Terraform and add the bucket to the existing policy within the Workflows role. Please reach out to the Workflows team for example PRs that illustrate how to take those steps.
</Note>

### Step 2: Filter files (optional)

Before parsing, you can use a `Filter` card to select only the files you want to process. This helps you narrow down your dataset and avoid parsing unnecessary documents.

<img alt="Filter Files Card" />

Purely as an example, here we only filter for resumes of a certain size.

This step is optional but recommended when working with large file sets to optimize processing time and costs.

### Step 3: Parse & Extract from documents with the Dex Card

This feature utilizes our Document Understanding capability, our `Dex SDK` on the backend.
Add a `Dex` card to extract structured data from your documents. This card utilizes the Dex SDK in the backend and automatically creates a Dex project that you can reference later in the Dex UI.

<img alt="Dex Project configs" />

#### Dex Project Configuration

* `Project Name`: Give your Dex project a descriptive name (e.g., "Resume Parsing Project")
* `Project ID (Optional)`:
  * Enter an existing project ID to reuse a project
  * Leave empty to create a new project each time the workflow runs

#### Parsing

This is the step that converts complex, unstructured files into LLM-ready JSON or markdown format.

<img alt="Dex Parse configs" />

Choose the OCR engine based on your document language and layout:

* `Engine`: Select from available parsing engines (e.g., `Reducto`). **Reducto**: Best for English and Latin-script documents with tables, figures, and complex layouts

For Reducto, configure the following settings to select how documents should be split into chunks:

* `Variable (recommended)`: Automatically determines optimal chunk boundaries
* `Chunk Size (Optional)`: Set to `Auto` to let the engine determine the best size

You can also enable `contextual retrieval`, which uses an LLM to prepend document-level context to each chunk, improving retrieval accuracy.

Control the visibility of the output data:

`Include parsed data in output`: Toggle on to output parsed text content for each chunk

* When enabled: Outputs parsed text content with full details
* When disabled: Outputs one row per file with metadata (project ID, parsed result ID) only

Finally, you can configure parse retries and per-file parse timeouts as per your workflow's needs.

#### Structured Extraction

Optionally extract specific fields from parsed documents using LLM-based extraction. This feature pulls targeted data from unstructured documents into structured metadata columns.

<img alt="Dex Extraction" />

`Run extraction after parse`: Toggle on to enable structured extraction. This uses the Dex SDK on each parse result (per file). Note: Large documents may hit LLM context limits

When enabled, configure the following:

**Schema Definition**

Define the fields you want to extract. Add all the fields you need:

* `Column name`: Name for the extracted field (e.g., "latestPosition")
* `Data type`: Select the data type (String, Number, Boolean, etc.)
* `Description`: Describe what should be extracted (e.g., "What's the latest position they worked at")
* Click `Add field` to define additional extraction fields

Extraction Options

* `Generate citations`: Toggle on to return bounding boxes for extracted values. Enabling this increases the size of each row payload
* `Generate confidence`: Toggle on to return confidence scores for extracted values. Enabling this increases the size of each row payload

Extraction Model Configuration

* `Extraction model`: Specify the LLM to use for extraction (e.g., "openai/gpt-4o", "vertex\_ai/gemini-2.5-pro"). This uses LiteLLM-style model IDs
* `Extraction retries`: Number of retries for structured extraction (0-10). Default: 3 retries

Provide custom instructions to guide the extraction process. For example:

```
From the provided text, extract the following: **Patient Info** - patient name in the document - what is the patient symptom?
```

<Note>
  Extracted field keys in the output will use the `extract_` prefix to distinguish them from standard parse output columns.
</Note>

#### Knowledge Base

Optionally push parsed results into a vector store for semantic search and RAG:

* `Push to Vector Store`: Toggle on to enable Knowledge Base integration
* `Vector Store Name`: Provide a name for your Knowledge Base (e.g., "Resume Vector Store")
* `Embedding Model`: Specify the embedding model (e.g., `openai/text-embedding-3-large`)

When enabled, parsed documents will be automatically indexed in the SGP Knowledge Base, accessible via the Knowledge Base UI for semantic search and retrieval.

### Step 4: View parsed & extracted output

After the Dex card processes your documents, the output is a dataframe with the following columns:

**Standard Parse Columns**

* `parseResultId`: Unique identifier for the parsing result
* `projectId`: The Dex project ID where results are stored
* `engine`: The parsing engine used (e.g., "Reducto")
* `pagesProcessed`: Number of pages processed in the document
* `chunkCount`: Number of chunks created from the document
* `status`: Parsing status (e.g., "completed", "failed")
* `parsedContent`: The extracted text content from the document
* `vectorStoreId`: ID of the Knowledge Base vector store (if enabled)

**Structured Extraction Columns** (if enabled)

When structured extraction is enabled, additional columns are added with the `extract_` prefix:

* `extract_<fieldName>`: The extracted value for each defined field (e.g., `extract_latestPosition`)
* `extract_<fieldName>_citations`: Bounding box coordinates for the extracted value (if citations enabled)
* `extract_<fieldName>_confidence`: Confidence score for the extraction (if confidence enabled)

You can use this dataframe in subsequent workflow cards to analyze, transform, or process the parsed data.

### Step 5: Access Dex project and SGP Knowledge Base

The Dex Parse card automatically creates a Dex project that you can access later for review and management.

If you enabled the "Push to Vector Store" option, navigate to the `Knowledge Base` section in SGP to access your vector store:

* Search parsed documents semantically
* Test retrieval quality
* Use for RAG applications
* Manage embeddings and vector store settings

### Step 6: Use parsed / extracted data in your workflow

Now you can use the parsed content in subsequent workflow cards. For this example, we'll call an agent to categorize resume content into suitable job roles.

#### Call Agent for Analysis

Add a `Call Agent` card to process the parsed content:

* `Select Agent`: Choose your agent from the dropdown
* `Output Column Name`: Specify where agent responses will be stored (e.g., `job_categories`)
* `Prompt Template`: Write your prompt using `{{parsedContent}}` to reference the parsed text

<img alt="Dex Call Agent " />

#### Additional Processing

You can add more cards to:

* Filter or transform the categorized results
* Join with job posting data
* Run evaluations on categorization quality
* Generate reports or summaries

### Step 7: Save your results

Finally, save the workflow output for future reference and analysis. You can save the results with the `Export as Evaluation` card as:

* **SGP Dataset**: Store the categorized resumes as a dataset for future use in training, evaluation, or analysis.
* **SGP Evaluation**: Create an evaluation to track the quality of resume categorization. The next 2 options allow you to observe hillclimbing trends.
* **SGP Dashboard**: Create a dashboard to visualize evaluation results, optionally from an existing template.
* **SGP Evaluation Group**: Add the evaluation to an existing group for side-by-side comparison on the Evaluations tab.

<img alt="Dex Final Export " />


# Guide to automating evals on Workflows
Source: https://docs.gp.scale.com/docs/capabilities/workflows/evals-in-workflows

Tutorial workflow for automated evals for your agent

This guide walks you through building an automated evaluation workflow in Workflows. You'll learn how to query data, call your agent, join ground truth datasets, run LLM-as-Judge evaluations, and schedule everything to run automatically.

### Step 1: Connect to the data source

Start by navigating to `Workflows` in SGP. Choose from the available data connection options: Traces, Data Sources, or CSV. We add more options based on user demand.

<img alt="Workflows Starter Page Pn" />

For this example, we'll connect to a Snowflake data source. The data source has a list of supermarket items with product name and description that we must categorize using our agent.

<img alt="Workflows Snowflake Query Pn" />

Select `Data Sources`, then choose your registered Snowflake connection from the dropdown. Write a SQL query to fetch the data you want to evaluate. You can set a `Row Limit` up to 1,000,000.

<Note>Only read-only SQL queries are allowed. INSERT, UPDATE, DELETE, DROP, and other modifying statements are not permitted.</Note>

### Step 2: Call your agent for completions

Use a `Call Agent` card to generate outputs from your agent for each row in your dataset. The `Agentex` agent you select has to be deployed on your SGP instance to be available in the dropdown.

<img alt="Workflows Call Agent Pn" />

In this example, we ensure our agent categorizes the supermarket items appropriately using our prompt. The configurations are:

* `Select Agent`: Choose the agent you want to evaluate from the dropdown
* `Output Column Name`: Specify the column name where agent responses will be stored (e.g., `agent_response`)
* `Include Traces`: Toggle on if you want to fetch execution traces for debugging
* `Prompt Template`: Write your prompt using `{{variableName}}` syntax to reference columns from your data. Click the variable buttons to insert column references automatically.

Under `Settings`, you can configure:

* `Timeout Duration`: Maximum time in seconds to wait for agent responses (default: 300)
* `Number of retries`: How many times to retry failed calls
* `Response Caching`: Enable to cache responses and avoid redundant API calls

Under `Advanced` settings, you can configure the `completion criteria` that determines when to stop polling for agent messages and the `batch size` that determines how many rows to process concurrently. This card generates the agent task IDs immediately as output.

<img alt="Get agent messages" />

Once you have set up the `Call agent` card, you need to use the `Get agent messages` card right after to get the agent's responses into the dataframe. The `Get agent messages` card takes the `agent task IDs` created in the previous card as input, along with configuration options such as agent `message limits` and `matching criteria`, to receive exactly what you need from the agent response (1st message, last message, tool calls, etc.). The 2-part implementation provides flexibility where you can get the agent response in the 2nd card without having to wait for all the tasks in the 1st card to finish.

### Step 3: Join data from another workflow (ex: Ground Truth)

Use the `Join Workflow` card to bring in data from another Workflows workflow—such as a ground truth dataset for comparison. Here we will compare our agent's categorization against the ground truth categorization for each product.

<img alt="Joins Pn" />

1. `Select Workflow`: Choose the workflow containing your ground truth or reference data
2. `Join Type`: Select how you want to combine the datasets:
   * `Left Join`: Keep all rows from the current workflow
   * `Right Join`: Keep all rows from the target workflow
   * `Inner Join`: Keep only matching rows
   * `Outer Join`: Keep all rows from both workflows
3. `Join Keys`: Select the matching columns from each workflow (e.g., `trace_id`)
4. `Execute target workflow before joining`: Enable this to get fresh data from the target workflow; otherwise, the most recent execution result will be used

### Step 4: Use the LLM as Judge card to do evals

Add an `LLM as Judge` card to evaluate your agent's outputs against the ground truth.

<img alt="Workflows LLM Judge Card" />

1. `Select Judge`: Choose to create a new judge or use an existing one
2. `Judge Type`: Select from available options like
   * `Legacy LLM as Judge (recommended)`: Traditional LLM judge with custom configuration that covers most use cases
   * `Base Evals Agent as Judge`: General-purpose evaluation agent with flexible capabilities
3. `Model`: Choose the LLM model to use for judging (e.g., `openai/gpt-4o`)
4. `Temperature`: Set to 0 for deterministic outputs
5. `LLM Judge Name`: Give your judge a name for future usage
6. `Rubric`: Define evaluation criteria. Use `{{columnName}}` to reference data columns

After configuring the judge, scroll down to the `Evaluation Details` section to export your results as an evaluation that can be accessed later anytime, which is helpful if you are running the evaluations on a schedule.

<img alt="Workflows LLM Judge to Evals" />

* `Evaluation Name`: A descriptive name for your evaluation
* `Description`: Explain what this evaluation measures
* `Tags`: Add tags to filter and organize evaluations by groups later
* `Save judge for future reuse`: Check to save this judge configuration and use it from the existing judge dropdown in other workflows
* `Create new evaluation on re-run`: Check to create a new evaluation each time the workflow runs

Every time you run this LLM as judge card, you'll see a success message with a direct link to view your evaluation results. Depending on what configurations you selected, a new evaluation will be created or the old one will be replaced. They can be accessed and referenced in the `Evaluations` and `Dashboards` tabs anytime respectively.

### Step 5: Open the evals in Evaluations tab or Dashboard tab

Navigate to the `Evaluations` tab in SGP to view your evaluation results. You can search, filter by date range, and filter by tags to find specific evaluations.

<img alt="Workflows links to Evaluation" />

Click on an evaluation to see:

* `Overview`: Visual summary of evaluation metrics and response distributions
* `Data`: Detailed row-by-row results with all columns
* `Metadata`: Information about when and how the evaluation was created

From here, you can also open the evaluation in the `Dashboards` tab for deeper analysis and custom visualizations.

### Step 6: Automate the evals to run on a schedule

To automate your evaluation workflow, click the calendar icon beside the title to set a schedule.

<img alt="Workflows Set Schedule" />

Configure the schedule:

1. `Set a Schedule`: Toggle on to enable scheduling
2. `Days`: Choose `Daily` to run every day, or select specific days of the week
3. `Time`: Set the frequency (e.g., `Hourly`) or a specific time
4. `End Date`: Schedules can run for up to 30 days into the future
5. `Enable Notifications`: Toggle on to receive Slack alerts for failed runs

Click `Save` to activate your scheduled workflow. You can monitor all scheduled runs from the `Execution logs` in the side menu, and view `Workflow history` to track changes across versions.

### Step 7: Create SGP Dashboards to show trends over time

You may export a workflow's output as an evaluation anytime utilizing the `Export as Evaluation` card. You can run that export on a schedule. That workflow can automatically create a new `Evaluation` in the SGP `Evaluations` tab. You can click on the `View Evaluation` button at the bottom of the card to visit that tab anytime.

<img alt="Export Evaluation" />

As you run the evaluation workflow on a schedule, you might want to track the trend line of your metrics in a dashboard. To do that, you must add your `Evaluation` to an `Evaluation Group`. To create an `Evaluation Group` you need 1+ Evaluations. Go to the `Evaluations` tab to create an `Evaluation Group`.

<img alt="Evaluation Group" />

Next, go to the SGP `Dashboards` tab and create a dashboard using the evaluation group you just created. For your use case, you will want to create a `Timeseries Widget` with the metrics across your evaluations inside the same group. [Learn more about Evaluation Group Dashboards →](/docs/v5/evaluation-dashboards/eval-group-dashboards#what-are-evaluation-group-dashboards)

<img alt="Evaluation Group Dashboard" />

After you have set the evaluation group dashboard and the widgets you want, you can update your workflow `Export as Evaluation` card to write every new evaluation run to the `Evaluation Group` you created for this dashboard. Doing so will automatically update the associated dashboard every time you run this workflow. You can visit the updated evaluation group dashboard by clicking on the `Visit Evaluation Group` button at the bottom of the card and then clicking on the `Open in Dashboards` button at the top right in the new window. Congrats! Now, you have an automated evaluation pipeline that updates metrics and visualizations over time.

<img alt="Evaluation Group Update" />

In the future, if you want to create a new dashboard in the same style for another evaluation workflow, when you come to this `Export as Evaluation` step, enable the `Create Dashboard` toggle, and pick your old dashboard as the template such that all the necessary widgets get carried over. You can also create a new empty dashboard with this card directly without any templates.


# How to use Workflows
Source: https://docs.gp.scale.com/docs/capabilities/workflows/introduction-to-workflows

Overview of Durable Agentic Workflow Orchestrator on SGP

Workflows is a no/low code durable workflow orchestration platform. It allows technical and non-technical teams to build, automate, and evaluate AI-driven data pipelines by chaining together data sources, transformations, custom Python scripts, and agent responses into a unified production workflow.

<img alt="Workflows Overview Pn" />

### Getting started

You can access it on SGP (Scale GenAI Platform) through the **Workflows** tab. If you don't see the tab on your SGP, please contact us. To start a workflow you must either import data or connect to a data source. Data sources available vary based on the connections set in the SGP instance you are running Workflows on. You can set up new `Data Sources` under the `Assets` tab.

<img alt="Workflows Starter Page Pn" />

We have provided starter templates that you can `Duplicate` or take inspiration from that are based on the most common workflows engineers run on Workflows. Just press the copy button on the top right to duplicate a workflow.

### Workflows cards

The basic building block in Workflows is called a **card**. Cards can produce data, transform data, or perform actions on data. When you start a new Workflows session, you can add cards to build your workflow.

Each card has its own interface appropriate for its function. Think of it like a form that you have to fill out to tell the card what you want it to do.

<img alt="Basicspython Pn" />

For example, the Python card appropriately has a code editor for you to write the code to transform the data you have imported in a previous step. The card includes all the necessary libraries to transform data.

<img alt="Plottingpython Pn" />

Additionally, the Python card also supports `print` and `matplotlib.pyplot` statements for quick EDA or debugging. If you include those statements, the output will show up in the `Logs` tab and `Visualization` tab respectively, beside the usual `Data` tab.

Cards stack upon each other, allowing you to perform rich transformations or analyses over the previous card’s data. As an example, we can take a CSV that has different LLM completions against a prompt and have an LLM judge it against ground truth for evals.

<img alt="Basicsllm Pn" />

Workflows sessions are long-lived, so you can run cards that take extended periods of time (for example, model evaluations using reasoning LLMs on thousands of rows) without losing your work. You can save cards while iterating, then run them later when you're ready. Each session has a `unique URL identifier` that gives you persistent access to your session.

<img alt="Linkshare Pn" />

Finally, once you are done with a workflow, you can export the final output dataframe as an `Evaluation` that you can always access and monitor later. This is helpful in the cases you are running the evaluation on your agent traces periodically. You can also save it as an SGP `Dataset` that can be downloaded and shared as a CSV.

<img alt="Export Evals Pn" />

In the event that you need data from another workflow in the current one, you can use the `Join Workflow` card to bring the output of another workflow (dataframe) and join it with the output here. Attached image is from an example workflow where we had to bring the mock PII from another workflow to perform an evaluation on whether our agent was correctly implementing PII anonymization.

<img alt="Joins Pn" />

### Key features & associated cards

<img alt="Cards More Pn" />

Workflows offers a variety of cards for pulling, transforming, and performing actions on data. Additional cards will be added based on enterprise use case requirements. Some of the ways cards can be used are:

* **Importing data**: Read data from sources like agent `Traces` tab, a `CSV` upload, or queryable sources like `Postgres`, `Snowflake`, `Mongo`, or `MS SQL` directly.
* **Document understanding**: Connect to cloud storage (Amazon S3, Azure Blob, Google Cloud Storage), parse unstructured documents using `Dex` card, automatically create Dex projects, extract structured data, and push parsed data to Knowledge Base. See the [Guide to Document Parsing with Dex in Workflows](/docs/capabilities/workflows/dex-parsing-in-workflows) for details.
* **Transforming data**: Transform your data using custom `Python` code or simple cards like `select columns`, `filter` or `aggregate`. You can also use `Join Data` to join the output from another workflow into this workflow's output (dataframe).
* **Call LLM or Agent**: Generate output and traces against input from each row of your dataset via cards like `Call LLM` or `Call Agent`. Workflows allows you to call your `Agentex` agents on SGP.
* **Analyze data**: Evaluate your outputs using `LLM as Judge` card and access the evaluation anytime in the SGP `Evaluations` tab.
* **Visualize data**: Generate basic visualizations from your transformed dataset using the `Python` or `Chart` card. For advanced capabilities, export the data as an `Evaluation` and `Dashboard`, and open in the `Dashboards` tab.
* **Export the data**: Export the data as `CSV` or as a `Dataset` on SGP. This dataset can be referred to in the Evaluations tab anytime.
* **Automate & monitor**: `Schedule` the workflow to run hourly or daily. Monitor runs and get notified via Slack. Slack notification feature needs to be enabled for each enterprise.

Most cards are available in **SGP > Workflows > New Workflow**. Workflows instances deployed in customer VPCs may have a limited set of available cards based on customer requirements. You can chat with the Workflows team to activate or disable certain cards depending on your customer's needs.

### Schedule & monitor workflows

If you have a sequence of cards that you want to run repeatedly, you can save them to easily load and re-run later. You can `Share` the workflow via URL. You can add other editors for collaboration. By default, everyone who has access to that SGP instance account `can read` all the workflows, and only the creator of the workflow has editor access.

<img alt="Saveworkflow Pn" />

A powerful feature of Workflows is the ability to execute them on a schedule. You can click on the calendar icon beside the title and toggle the **Set a Schedule** option and configure your desired schedule. Scheduled workflows allow you to automate repetitive tasks.

<img alt="Scheduleworkflow Pn" />

From the side menu, you can view `Execution logs` anytime to monitor past runs. You can also check `Workflow history` to understand how the workflow changed across versions, the `diffs`, and the author responsible for each version.

<img alt="Exec Logs Pn" />

### What's coming next in H1 2026

Here are core features planned on our roadmap.

* **\[Launched] More data source support**: Currently, Workflows supports connection to queryable data sources. We just launched data connectors for cloud file storages like Amazon S3, Azure Blob, and Google Cloud Storage.
* **\[Launched] Document parsing & extraction on Workflows**: Utilize our `Dex` and `Iris` capabilities inside Workflows to `parse` unstructured data across Docs or PDFs into structured data and `extract` the information you need for your workflows.
* **UX & UI improvements**: Make quality of life improvements to solve common pain points faced by users like improved error handling, messages for required missing inputs, improved navigation bars, etc.
* **More integrations with Agentex and AgentOps API and SDK**: Currently, Workflows has integrations with our Evals SDK and the Dex SDK as separate cards. We want to extend the functionalities to cover the majority of SGP's SDKs.

<Note>Creating a custom Workflows card to support your specific enterprise use case on their VPC is also straightforward. If your organization has functionality requirements, please contact us.</Note>


# How to run Workflows using a script
Source: https://docs.gp.scale.com/docs/capabilities/workflows/run-workflows-using-a-script

Run a saved workflow in your code using the Compass HTTP API

This tutorial shows how to run a [Workflows](/docs/capabilities/workflows/introduction-to-workflows) workflow programmatically over HTTPS. You need an API key, your account ID, and a workflow that is already defined and saved in the Workflows UI. You do not need to manage the underlying infrastructure.

## Prerequisites

* A workflow defined and saved in the Workflows UI
* A Scale API key
* Your Scale account ID (the account that owns the workflow)

If you are missing credentials, contact your Scale representative.

## Authentication

Send requests over HTTPS to your SGP host—the same origin you use to open the Workflows UI. Every example below uses a base URL of the form `https://<your-sgp-host>/corp-api/compass/api`.

Include these headers on each request:

```http theme={null}
x-scale-api-key: <your-api-key>
x-selected-account-id: <your-account-id>
```

`<your-api-key>` is your Scale API key. `<your-account-id>` is the ID of the account that owns the workflow you want to run.

## Run and wait

The usual pattern is to start a run, then block until it finishes (up to a configurable timeout).

```typescript theme={null}
const BASE_URL = 'https://<your-sgp-host>/corp-api/compass/api';
const HEADERS = {
  'x-scale-api-key': process.env.SCALE_API_KEY!,
  'x-selected-account-id': process.env.SCALE_ACCOUNT_ID!,
  'Content-Type': 'application/json',
};

async function runWorkflow(workflowId: string, variables?: Record<string, string>) {
  const runRes = await fetch(`${BASE_URL}/v1/workflow/run`, {
    method: 'POST',
    headers: HEADERS,
    body: JSON.stringify({ workflowId, variables }),
  });
  if (!runRes.ok) throw new Error(`Failed to start workflow: ${await runRes.text()}`);
  const { sessionId } = await runRes.json();

  const resultRes = await fetch(
    `${BASE_URL}/v1/sessions/${sessionId}/result?timeout=600000`,
    { headers: HEADERS },
  );
  if (!resultRes.ok) throw new Error(`Failed to get result: ${await resultRes.text()}`);
  return resultRes.json();
}

async function main() {
  const result = await runWorkflow('wf-abc123', { region: 'US', date: '2024-01-01' });
  console.log(`${result.data.length} rows returned`);
  console.log(result.data);
}

main().catch(console.error);
```

### curl equivalent

```bash theme={null}
BASE_URL="https://<your-sgp-host>/corp-api/compass/api"

SESSION_ID=$(curl -s -X POST "$BASE_URL/v1/workflow/run" \
  -H "x-scale-api-key: $SCALE_API_KEY" \
  -H "x-selected-account-id: $SCALE_ACCOUNT_ID" \
  -H "Content-Type: application/json" \
  -d '{"workflowId": "wf-abc123", "variables": {"region": "US"}}' \
  | jq -r '.sessionId')

curl -s "$BASE_URL/v1/sessions/$SESSION_ID/result?timeout=600000" \
  -H "x-scale-api-key: $SCALE_API_KEY" \
  -H "x-selected-account-id: $SCALE_ACCOUNT_ID"
```

## API reference

All paths below are relative to `BASE_URL` (`https://<your-sgp-host>/corp-api/compass/api`).

### `POST /v1/workflow/run`

Starts a workflow run and returns immediately with a `sessionId`.

**Request body:**

```json theme={null}
{
  "workflowId": "wf-abc123",
  "variables": {
    "region": "US",
    "threshold": "0.8"
  }
}
```

`variables` is optional. Values override defaults configured in the workflow and are substituted into `{{variable_name}}` placeholders in your cards. For typical use cases, use flat string values.

**Response:**

```json theme={null}
{ "sessionId": "compass-sess-..." }
```

### `GET /v1/sessions/:sessionId`

Returns the current status of a running session. Use this endpoint when you want to poll instead of blocking on the result endpoint.

**Example response:**

```json theme={null}
{
  "sessionState": {
    "status": "idle",
    "cards": [
      {
        "id": "card-0",
        "context": { "cardType": "import_csv" },
        "status": "completed",
        "output": {}
      }
    ]
  }
}
```

**Session status values**

| Status         | Meaning                                |
| -------------- | -------------------------------------- |
| `starting`     | Session is initializing                |
| `pending_work` | Cards are queued to run                |
| `processing`   | A card is currently executing          |
| `idle`         | All cards finished; results are ready  |
| `errored`      | The session hit an unrecoverable error |
| `inactive`     | Session expired or was cancelled       |

**Card status values**

| Status       | Meaning                         |
| ------------ | ------------------------------- |
| `idle`       | Card has not been scheduled yet |
| `pending`    | Queued to run                   |
| `processing` | Currently executing             |
| `completed`  | Finished successfully           |
| `errored`    | Failed; check `errorMessage`    |

### `GET /v1/sessions/:sessionId/result?timeout=<ms>`

Waits until the session reaches `idle` or `errored`, or until `timeout` milliseconds elapse (default `600000`, ten minutes), then returns the final output.

**Query parameters**

* `timeout` — milliseconds to wait (default `600000`)

**Example response:**

```json theme={null}
{
  "data": [
    { "name": "Alice", "score": 0.92 },
    { "name": "Bob", "score": 0.85 }
  ],
  "schema": [
    { "id": "name", "header": "Name", "type": "string" },
    { "id": "score", "header": "Score", "type": "number" }
  ],
  "start": 0,
  "end": 2,
  "result": {
    "summary": {
      "totalNumberOfRows": 2,
      "executionDurationMillis": 3210
    }
  }
}
```

`data` contains the rows. `schema` describes the columns. `start` and `end` describe the row index range returned.

## Polling instead of blocking

If you prefer short requests instead of a long-lived connection, poll `GET /v1/sessions/:sessionId` until the session is `idle`, then call `GET /v1/sessions/:sessionId/result` to fetch the tabular output.

The following example reuses the same `BASE_URL` and `HEADERS` as in [Run and wait](#run-and-wait).

```typescript theme={null}
async function waitForResult(sessionId: string, pollIntervalMs = 5000, timeoutMs = 600000) {
  const deadline = Date.now() + timeoutMs;

  while (Date.now() < deadline) {
    const res = await fetch(`${BASE_URL}/v1/sessions/${sessionId}`, { headers: HEADERS });
    if (!res.ok) throw new Error(`[${res.status}] Failed to poll session: ${await res.text()}`);
    const { sessionState } = await res.json();

    if (sessionState.status === 'errored' || sessionState.status === 'inactive') {
      throw new Error(`Session ended with status: ${sessionState.status}`);
    }

    if (sessionState.status === 'idle') {
      const failedCards = sessionState.cards.filter((c: { status: string }) => c.status === 'errored');
      if (failedCards.length > 0) {
        throw new Error(
          `${failedCards.length} card(s) failed: ${failedCards.map((c: { errorMessage?: string }) => c.errorMessage).join(', ')}`,
        );
      }
      return sessionState;
    }

    await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
  }

  throw new Error('Timed out waiting for session to complete');
}
```

<Note>
  Polling returns session state (status and card metadata), not the full workflow table. After the session reaches `idle`, call `GET /v1/sessions/:sessionId/result` to retrieve the rows.
</Note>

## Workflow variables

Variables let you parameterize a workflow without editing its definition. They replace `{{variable_name}}` placeholders in card configuration (for example, filter values, prompt templates, or model parameters).

```typescript theme={null}
await runWorkflow('wf-abc123', {
  model: 'gpt-4o',
  prompt: 'Summarize the following in one sentence',
  min_score: '0.7',
});
```

Values you pass override defaults saved on the workflow. Any variable you omit uses the workflow’s saved default.

## Error handling

Error bodies are JSON. Many errors look like this:

```json theme={null}
{
  "error": "Workflow not found or you do not have access.",
  "status_code": 404
}
```

Authentication failures may include both `error` and `message`:

```json theme={null}
{
  "error": "Unauthorized",
  "message": "Missing authentication header."
}
```

| HTTP status | Meaning                                                  |
| ----------- | -------------------------------------------------------- |
| 400         | Malformed request (missing required field, bad format)   |
| 401         | Missing or invalid API key or account ID                 |
| 404         | Workflow or session not found, or you do not have access |
| 500         | Internal server error                                    |

This variant parses JSON error bodies from failed responses:

```typescript theme={null}
async function runWorkflowSafe(workflowId: string, variables?: Record<string, string>) {
  const runRes = await fetch(`${BASE_URL}/v1/workflow/run`, {
    method: 'POST',
    headers: HEADERS,
    body: JSON.stringify({ workflowId, variables }),
  });

  if (!runRes.ok) {
    const err = await runRes.json().catch(() => ({ error: runRes.statusText }));
    throw new Error(`[${runRes.status}] ${err.error ?? err.message}`);
  }

  const { sessionId } = await runRes.json();

  const resultRes = await fetch(
    `${BASE_URL}/v1/sessions/${sessionId}/result?timeout=600000`,
    { headers: HEADERS },
  );

  if (!resultRes.ok) {
    const err = await resultRes.json().catch(() => ({ error: resultRes.statusText }));
    throw new Error(`[${resultRes.status}] ${err.error ?? err.message}`);
  }

  return resultRes.json();
}
```

`runWorkflowSafe` assumes `BASE_URL` and `HEADERS` are defined the same way as in [Run and wait](#run-and-wait).


# AWS Architecture Reference
Source: https://docs.gp.scale.com/docs/infrastructure/aws/aws-architecture-reference

Overview of the SGP AWS architecture

## Architecture Reference

<img />

## Cloud Services

| Service                              | Usage                                                                    |
| ------------------------------------ | ------------------------------------------------------------------------ |
| Elastic Kubernetes Service (EKS)     | SGP applications are deployed to this cluster                            |
| Identity and Access Management (IAM) | Role and policy management for SGP applications access to cloud services |
| Secrets Manager                      | Secret and configuration management for SGP applications                 |
| Key Management Service (KMS)         | Encryption of sensitive data at rest                                     |
| Simple Storage Service (S3)          | Object storage for SGP applications (knowledge bases, models, etc.)      |
| Simple Queue Service (SQS)           | Karpenter spot instance interruption queue                               |
| OpenSearch                           | Search provider for SGP applications                                     |
| Elastic Container Registry (ECR)     | Container registry for custom agent images                               |
| Aurora PostgreSQL                    | Relational databases for SGP applications                                |
| DocumentDB                           | Document-oriented database for SGP applications                          |
| ElastiCache for Redis                | In-memory caching for SGP applications                                   |
| CloudWatch                           | Logging for cloud services                                               |

## Networking

SGP is deployed into a VPC spanning three Availability Zones with separate public and private subnets. All application workloads run in private subnets. Outbound internet access is provided by a NAT Gateway.

### Ingress

| Property             | Default                                                     |
| -------------------- | ----------------------------------------------------------- |
| Load balancer type   | Application Load Balancer (ALB)                             |
| Listener port        | 443 (TLS)                                                   |
| Placement            | External (internet-facing)                                  |
| WAFv2                | Optional — associate an existing Web ACL by name            |
| VPC Endpoint Service | Optional — enables AWS PrivateLink for private connectivity |

<Note>
  The load balancer can be configured as internal (private subnet only) for deployments that require no direct internet exposure.
</Note>

### Service Mesh

Istio is deployed as a cluster add-on and manages all east-west traffic between SGP services. An internal Istio ingress gateway handles traffic routing within the cluster.

## EKS Cluster Default Configuration

### Node Groups

| Node Group   | Size                         | Limits   | Machine Type  | Machine Image                     | Taints                             |
| ------------ | ---------------------------- | -------- | ------------- | --------------------------------- | ---------------------------------- |
| System       | 1                            | —        | m6a.2xlarge   | EKS Default AMI                   | CriticalAddonsOnly=true:NoSchedule |
| Default      | Karpenter Managed Node Group | 1000 CPU | c\*, m\*      | Karpenter Bottlerocket AMI Family |                                    |
| High Storage | Karpenter Managed Node Group | 100 CPU  | c\*, m\*, r\* | Karpenter Bottlerocket AMI Family |                                    |
| GPU          | Karpenter Managed Node Group | 385 CPU  | g5.\*         | Karpenter Bottlerocket AMI Family | nvidia.com/gpu:NoSchedule          |
| GPU (Large)  | Karpenter Managed Node Group | 100 CPU  | p4d.\*        | Karpenter Bottlerocket AMI Family | nvidia.com/gpu:NoSchedule          |

### Cluster Add-ons

| Add-on                       | Purpose                                                            |
| ---------------------------- | ------------------------------------------------------------------ |
| Karpenter                    | Workload-driven node autoscaling for Karpenter-managed node groups |
| Cluster Autoscaler           | Scaling for the system node group                                  |
| Istio                        | Service mesh for east-west traffic management                      |
| cert-manager                 | TLS certificate provisioning and renewal                           |
| AWS Load Balancer Controller | Kubernetes-native ALB and NLB provisioning                         |
| external-dns                 | Automatic Route 53 DNS record management                           |
| AWS Node Termination Handler | Graceful pod drain on spot interruption or node termination        |
| NVIDIA GPU Operator          | GPU driver and device plugin lifecycle management                  |
| AWS EBS CSI Driver           | Persistent volume support for EKS workloads                        |

## Aurora PostgreSQL Default Configuration

| Instance                                              | Service                           | Instance Count | Engine       | Instance Class |
| ----------------------------------------------------- | --------------------------------- | -------------- | ------------ | -------------- |
| agents-service-\<workspace\_id>                       | Agents                            | 2              | PostgreSQL16 | db.r6g.xlarge  |
| duc-\<workspace\_id>-duc-api-aurora-db                | Document Understanding Capability | 3              | PostgreSQL15 | db.r6g.large   |
| reducto-\<workspace\_id>-aurora-db                    | Reducto                           | 3              | PostgreSQL15 | db.r6g.large   |
| sae-\<workspace\_id>-sae-postgres                     | Semantic Analysis Engine          | 1              | PostgreSQL17 | db.r6g.large   |
| scale-egp-\<workspace\_id>-agentex-aurora-db          | Agentex                           | 3              | PostgreSQL15 | db.r6g.large   |
| scale-egp-\<workspace\_id>-egp-api-aurora-db          | EGP API                           | 3              | PostgreSQL15 | db.r6g.large   |
| scale-egp-\<workspace\_id>-identity-service-aurora-db | Identity Service                  | 3              | PostgreSQL15 | db.r6g.large   |
| scale-egp-\<workspace\_id>-llm-engine-aurora-db       | LLM Engine                        | 2              | PostgreSQL14 | db.r6g.large   |
| scale-egp-\<workspace\_id>-spicedb-postgres           | SpiceDB                           | 3              | PostgreSQL16 | db.r6g.large   |
| scale-egp-\<workspace\_id>-temporal-aurora-db         | Temporal                          | 3              | PostgreSQL15 | db.r6g.large   |
| scale-egp-\<workspace\_id>-train-aurora-db            | SGP Train                         | 3              | PostgreSQL15 | db.t3.medium   |

## DocumentDB Default Configuration

| Instance                                 | Service | Instance Count | Instance Class |
| ---------------------------------------- | ------- | -------------- | -------------- |
| scale-egp-\<workspace\_id>-compass-docdb | Compass | 1              | db.t3.medium   |
| \<workspace\_id>-agentex-mongo           | Agentex | 1              | db.r5.large    |

## Customer Sensitive Data

Sensitive data may be stored in the following services, all data is encrypted at rest and in transit:

* Aurora PostgreSQL: For storing metadata, persistent data, and service-specific databases (SGP API Backend, Identity Service, LLM Engine, SpiceDB, Agents Service, SGP Train, and Temporal)
* DocumentDB: For storing Agentex agent data/workflows and Workflows document processing state
* S3 Buckets: For storing large data objects and datasets
* OpenSearch: For managing text embeddings and vector search capabilities


# AWS SGP Deployment
Source: https://docs.gp.scale.com/docs/infrastructure/aws/aws-sgp-deployment

End-to-end guide to deploying SGP in an AWS account.

## Overview

This guide will walk you through the process of deploying SGP in an AWS account. SGP AWS infrastructure is defined by terraform modules managed by Scale.

## Prerequisites

* Access to an AWS account with sufficient permissions to create resources
* The following tools installed:
  * [AWS CLI](https://aws.amazon.com/cli/)
  * [Terraform](https://www.terraform.io/)
  * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
* The following from Scale:
  * The SGP AWS Infrastructure Terraform modules
  * A `workspace_id` and `registration_secret` unique to your deployment
  * `install.sh` script to bootstrap the SGP cluster
* A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional)
* A custom domain for your deployment (optional)

## Installation

### Step 1: Prepare Terraform Root Module

In a new directory, create a `.tf` file to invoke the SGP AWS infrastructure Terraform modules. Reference the `egp-hybrid-cloud-customer-deployment` module as the entrypoint for the SGP infrastructure.

```terraform theme={null}
module "egp_hybrid_deployment" {
  source                = "<path to egp-hybrid-cloud-customer-deployment module>"
  scale_account_id      = "<workspace_id>"
  registration_secret   = "<registration_secret>"  # sensitive — use TF_VAR_registration_secret env var

  domain = "<workspace_id>.workspace.egp.scale.com"  # or your custom domain

  # Core services — enable the components you need
  deploy_egp_api    = true   # EGP API backend — required for the SGP platform
  deploy_spellbook  = false  # Spellbook prompt management service (optional)
  deploy_sgp_models = false  # SGP model serving infrastructure (optional)
  deploy_llm_engine = false  # Model Engine service (optional)
  deploy_agentex    = false  # Agentex agentic workflow service (optional)
  create_agentex_tool_cache = false  # Persistent tool cache for Agentex (optional)
  enable_cloud_build = false # Cloud-based container build service for Agentex (optional)
  add_agents_service = false # Agent Service for building and running agents (optional)
  add_document_understanding_capability = false  # Document Understanding (Dex) service (optional)
  enable_reducto    = false  # Reducto OCR and document parsing service (optional)
  enable_train      = false  # SGP model training infrastructure (optional)
  enable_compass    = false  # Workflows workflow orchestration service (optional)
  enable_ukbp       = false  # Universal Knowledge Base Pipeline (optional)
  enable_system_manager = true  # System Manager deployment orchestrator — required
  enable_monitoring = false  # Observability stack (optional)

  create_temporal_postgresql = true  # Recommended; set false to use an in-cluster Cassandra instead

  omit_egp_control_plane = false  # Set true to disable communication with Scale's hosted control plane

  base_repository = "<base container registry URL>"  # Only needed if not pulling from Scale's registry

  openai_api_key    = "<openai_api_key>"     # sensitive — use TF_VAR_openai_api_key env var
  openai_org_id     = "<openai_org_id>"
  anthropic_api_key = "<anthropic_api_key>"  # sensitive — use TF_VAR_anthropic_api_key env var
  launch_api_key    = "<launch_api_key>"     # sensitive — use TF_VAR_launch_api_key env var

  authentication_type = "SAML"  # "SAML" or "OIDC"

  use_cmk = true  # Recommended for production: encrypts data at rest using AWS KMS

  # If deploying into an existing VPC rather than creating a new one, provide the IDs here
  # vpc_id             = "<existing VPC ID>"
  # private_subnet_ids = ["<subnet-id-1>", "<subnet-id-2>"]
  # public_subnet_ids  = ["<subnet-id-1>", "<subnet-id-2>"]

  access_cidrs = []  # List of CIDR blocks to allow access; defaults to open to internet

  # Load balancer configuration
  load_balancer_type = "application"  # "application" or "network"
  # waf_name          = "<WAFv2 Web ACL name>"  # Optional; only applies to application load balancers
  internal_lb        = false  # Set true to prevent internet access to the load balancer
  # additional_certs  = ["<certificate ARN>"]  # Additional TLS certificates for the load balancer

  # permissions_boundary_arn = "<ARN>"  # Optional: IAM permissions boundary for provisioned roles

  # See the module documentation for the full list of available variables
}

# Example Terraform provider and backend configuration
provider "aws" {
  region = "<your-aws-region>"
}

terraform {
  required_version = ">= 1.1.7"

  backend "s3" {
    bucket       = "sgp-<workspace_id>-terraform-state"
    key          = "sgp-<workspace_id>-sgp.tfstate"
    region       = "<your-aws-region>"
  }
}
```

### Step 2: Provision Infrastructure via Terraform

Run the following commands in the same directory as the previous step's .tf file.

```bash theme={null}
terraform init

terraform plan -out=tfplan

# Inspect this plan to review the planned resources to be provisioned before proceeding
terraform show tfplan 

terraform apply tfplan
```

*This step may take significant time due to resource creation dependencies*

<Note>
  If a pre-created certificate and domain entry are not provided, you will likely encounter the following error:

  ```bash theme={null}
  Error: creating ELBv2 Listener (arn:aws:elasticloadbalancing:us-east-1:12345678910:loadbalancer/net/scale-egp-<workspace_id>/12345678910): operation error Elastic Load Balancing v2: CreateListener, https response error StatusCode: 400, RequestID: 12345678910, api error UnsupportedCertificate: The certificate 'arn:aws:acm:us-east-1:12345678910:certificate/12345678910' must have a fully-qualified domain name, a supported signature, and a supported key size.
  ```

  Resolve this by properly configuring DNS and TLS for your domain before re-running `terraform apply`.
</Note>

### Step 3: Bootstrap the Cluster

Note the following values from the previous step's apply output:

```bash theme={null}
export SYSTEM_MANAGER_IRSA_ROLE_ARN=$(terraform output -raw system_manager_irsa_role_arn)
export SPICEDB_DATASTORE_URI=$(terraform output -raw spicedb_connection_string)
export AWS_REGION="<your-aws-region>"
```

#### Configure EKS access

Configure access for yourself to the provisioned EKS cluster:

1. Navigate to "Elastic Kubernetes Service" on the AWS console
2. Go to the required cluster
3. Go to *Access*
4. Under *IAM access entries* click *Create Access Entry*
5. Select your current AWS IAM role from the dropdown, leave the *Type* as *Standard*
6. Add the `AmazonEKSAdminPolicy` and `AmazonEKSClusterAdminPolicy` to the *Cluster* scope
7. Click create

#### Allowlist your IP (if needed)

1. Navigate to "Elastic Kubernetes Service" on the AWS console
2. Go to the required cluster
3. Go to *Networking*
4. On the right, click *Manage* and click *Endpoint Access*
5. Under *Advanced settings* add your IP address

#### Execute install.sh

Run the `install.sh` script to create the following Kubernetes resources:

* AWS Secret Manager CSI Driver
* Spicedb
* [SGP System Manager](/docs/infrastructure/system-manager)

```bash theme={null}
cd <path to install.sh>

# Set the correct kubectl context
aws eks update-kubeconfig --name scale-egp-<workspace_id> --region <aws_region>

# Execute the install.sh script
./install.sh
```

System Manager is the deployment orchestrator for the SGP platform. It will automatically begin deploying the other services required for the SGP platform within the cluster.

### Step 4: Verify the Deployment

Wait for all services to be ready:

```bash theme={null}
kubectl get helmreleases -A  # All should be Ready=True
kubectl get pods -A          # All should be Running
```

### Step 5: Configure Identity Provider

#### SAML Configuration

In your Identity Provider, configure a new SAML application with the following settings:

* Service Entity ID: `https://auth.<domain>`
* Redirect URI: `https://auth.<domain>/dashboard/org/saml/callback`

Modify the configuration `scale-egp-<workspace_id>/identity-service-saml-secrets` (either via System Manager or directly in AWS Secrets Manager):

```json theme={null}
{
  "id": "<workspace_id>",
  "samlConfiguration": {
        "entityId": "<entity id from your identity provider>",
        "x509Cert": "<x509 certificate from your identity provider>",
        "ssoUrl": "<sso URL from your identity provider>",
        "attributeMappings": {
            "email": "<email attribute name from your identity provider>",
            "firstName": "<first name attribute name from your identity provider>",
            "lastName": "<last name attribute name from your identity provider>"
        }
    }
}   
```

#### OIDC Configuration

In your Identity Provider, configure a new OIDC application with the following settings:

* Redirect URI: `https://auth.<domain>/dashboard/org/oidc/callback`

Modify the configuration `scale-egp-<workspace_id>/identity-service-oidc-secrets` (either via System Manager or directly in AWS Secrets Manager) with the following JSON:

```json theme={null}
{
  "id": "<workspace_id>",
  "oidcConfiguration": {
    "clientId": "<client ID from your identity provider>",
    "clientSecret": "<client secret from your identity provider>",
    "issuer": "<issuer from your identity provider>",
    "authorizationUrl": "<authorization URL from your identity provider>",
    "tokenUrl": "<token URL from your identity provider>",
    "userInfoUrl": "<user info URL from your identity provider>"
  }
}
```

After modifying the secret, restart the `sgp-system-manager` deployment to apply the changes.

```bash theme={null}
kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager
```

## Accessing the Platform

If all goes smoothly, you should be able to navigate to the SGP platform at `https://<workspace_id>.workspace.egp.scale.com` (or your custom domain) and authenticate via the configured identity provider.


# Azure Architecture Reference
Source: https://docs.gp.scale.com/docs/infrastructure/azure/azure-architecture-reference

Overview of the SGP Azure architecture

## Architecture Diagram

```mermaid theme={null}
graph TD
    USER(["Client"])
    INTERNET(["Internet"])

    subgraph AZ["Azure Subscription"]
        FD["Azure Front Door<br/>WAF · CDN · TLS termination"]

        subgraph VNET["Virtual Network — 10.0.0.0/16"]
            INGRESS["Istio Ingress<br/>Internal Load Balancer"]
            NAT["NAT Gateway<br/>Outbound egress"]

            subgraph AKS["AKS Cluster — private API server"]
                SYS["System Node Pool<br/>3× Standard_D4s_v3"]
                APP["User Node Pool<br/>3–10× Standard_D16s_v3"]
            end

            BASTION["Azure Bastion"]
            JUMP["Jump Host VM"]
        end

        KV["Key Vault<br/>Secrets + CMK keys"]
        LAW["Log Analytics Workspace"]

        PG[("PostgreSQL<br/>Flexible Server")]
        REDIS[("Redis Cache<br/>Premium")]
        STOR[("Storage Account")]
        SB[("Service Bus<br/>Premium")]
    end

    USER --> FD
    FD -->|"Private Link"| INGRESS
    INGRESS --> APP
    APP -->|"Private Endpoint"| PG
    APP -->|"Private Endpoint"| REDIS
    APP -->|"Private Endpoint"| STOR
    APP -->|"Private Endpoint"| SB
    APP --> KV
    SYS --> KV
    APP --> LAW
    SYS --> LAW
    APP --> NAT --> INTERNET
    BASTION --> JUMP -->|"kubectl"| AKS
```

***

## Resources by Type

### Compute Resources

| Resource         | Count | Purpose                      |
| ---------------- | ----- | ---------------------------- |
| AKS Cluster      | 1     | Kubernetes orchestration     |
| System Node Pool | 1     | System pods (fixed 3 nodes)  |
| User Node Pool   | 1     | Application workloads        |
| GPU Node Pool    | 0-1   | AI/ML workloads (optional)   |
| Cassandra Pool   | 0-1   | Temporal database (optional) |

**Total VMs:** 6-24 (3 system + 3-10 user + 0-5 GPU + 0-6 Cassandra)

***

### Network Resources

| Resource          | Count | Purpose                  |
| ----------------- | ----- | ------------------------ |
| Virtual Network   | 1     | Network boundary         |
| Subnets           | 4-5   | Network segmentation     |
| NSGs              | 4-5   | Traffic control          |
| Route Tables      | 3-4   | Traffic routing          |
| Private DNS Zones | 7     | Internal name resolution |
| Private Endpoints | 7     | Secure PaaS access       |
| Bastion Host      | 0-1   | Secure VM access         |
| Public IPs        | 0-1   | Bastion endpoint         |

***

### Data & Storage Resources

| Resource          | Count | Purpose                     |
| ----------------- | ----- | --------------------------- |
| PostgreSQL Server | 1     | Relational database         |
| Redis Cache       | 1     | Distributed cache           |
| Storage Account   | 1     | Blob/File storage           |
| AI Search Service | 0–1   | Full-text search (optional) |
| OpenAI Service    | 0–1   | LLM models (optional)       |

***

### Security Resources

| Resource              | Count | Purpose                |
| --------------------- | ----- | ---------------------- |
| Key Vault             | 1     | Secrets management     |
| Managed Identity      | 1-2   | Service authentication |
| RBAC Role Assignments | 10+   | Access control         |

***

### Monitoring Resources

| Resource                | Count | Purpose             |
| ----------------------- | ----- | ------------------- |
| Log Analytics Workspace | 1     | Centralized logging |
| Data Collection Rule    | 1     | AKS metrics         |
| Diagnostic Settings     | 7+    | Resource logging    |
| Datadog Connection      | 0-1   | External monitoring |

***

## Network Architecture

### Address Space Planning

```
VNet: 10.0.0.0/16 (65,536 IPs)
├── AKS Subnet: 10.0.1.0/24 (256 IPs)
├── Bastion Subnet: 10.0.2.0/26 (64 IPs)
├── Database Subnet: 10.0.3.0/24 (256 IPs)
└── Private Endpoints: 10.0.4.0/25 (128 IPs)

Pod CIDR: 10.244.0.0/16 (65,536 IPs)
Service CIDR: 10.243.0.0/16 (65,536 IPs)
```

### Traffic Flow

**Egress (Internet):**

```
Pods/VMs → NAT Gateway → Public IP → Internet
(Stateful, return traffic allowed)
```

**Ingress (Internal):**

```
Service IP → Load Balancer → Pod IP (via CNI)
```

**Database Access:**

```
AKS Pods → Private Endpoint → Private Link → PostgreSQL
(DNS: server.postgres.database.azure.com)
```

**External Service Access:**

```
AKS Pods → API Gateway / Load Balancer → OpenAI / AI Search
(Via Private Endpoints)
```

***

## AKS Configuration Deep Dive

### API Server Access

**Type:** Private cluster (recommended)
**Endpoint:** Internal only
**Access Method:** Bastion host or VPN
**DNS:** k8s..azmk8s.io (private)

### Network Policies

**Engine:** Azure Network Policy
**Scope:** Pod-to-pod communication
**Default:** Allow all (unrestricted)
**Configuration:** Define in Kubernetes manifests

### Container Registry

**Integration:** Azure Container Registry (optional)
**Authentication:** Managed identity or pull secrets
**Pulling:** Private endpoint (optional)

### Monitoring & Observability

**Azure Monitor Agent:** Deployed in kube-system
**Metrics:** CPU, Memory, Disk, Network
**Logs:** Container stdout/stderr, Kubernetes events
**Dashboards:** Pre-built in Log Analytics


# Azure SGP Deployment
Source: https://docs.gp.scale.com/docs/infrastructure/azure/azure-sgp-deployment

End-to-end guide to deploying SGP in an Azure subscription.

## Overview

This guide walks you through deploying SGP in an Azure cloud subscription using the SGP Azure Terraform modules. SGP Azure infrastructure is defined by terraform modules managed by Scale.

## Prerequisites

* Access to an Azure subscription with sufficient permissions to create resources (Contributor + User Access Administrator roles, or equivalent)
* The following tools installed:
  * [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) (`az login` completed)
  * [Terraform](https://www.terraform.io/) (`>= 1.1.7, < 2.0.0`)
  * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
* The following from Scale:
  * The SGP Azure Infrastructure Terraform modules (`azure-terraform/infra`)
  * A `workspace_id` and `registration_secret` unique to your deployment
* A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional)
* A custom domain for your deployment (optional)

## Installation

### Step 1: Build Configuration

Configuration is split across three files. This allows you to reuse the same configuration for multiple environments.

```
infra/
├── main.tfvars.json                         # Selects which domain + environment to deploy
└── config/
    └── <domain_code>/
        ├── default.yaml                     # Domain-wide defaults (all environments)
        └── <location>/
            └── <environment_type>.yaml      # Environment-specific overrides
```

#### File 1: `main.tfvars.json`

This file selects which configuration to load. Edit it to point at your customer and environment before running Terraform.

```json theme={null}
{
  "environment_type": "<dev|qa|staging|uat|prod>",
  "domain_code": "<A unique 3-9 alphanumeric character string to identify yourself>",
  "location": "<Azure region>"
}
```

| Field              | Description                                          | Allowed values                                 |
| ------------------ | ---------------------------------------------------- | ---------------------------------------------- |
| `environment_type` | Environment tier                                     | `dev`, `qa`, `staging`, `uat`, `prod`          |
| `domain_code`      | Customer identifier (3–9 alphanumeric chars/hyphens) | e.g. `acme`, `contoso`                         |
| `location`         | Azure region                                         | e.g. `eastus2`, `northeurope`, `southeastasia` |

#### File 2: `config/<domain_code>/default.yaml`

Customer-wide baseline settings that apply to **all** environments unless overridden. This file is the right place for stable policy and posture decisions: tenant identity, tagging standards, and security baselines for Key Vault, PostgreSQL, Redis, Storage, and other services.

```yaml theme={null}
# Identity
business_unit: "<your_business_unit>"
tenant_id: "<your_azure_tenant_id>"

default_tags:
  costcenter: "<cost_center>"
  product: "<domain_code>"
  deploymenttype: "new"
  safe-to-delete: "no"

# CMK key names (Terraform creates these keys when cmk.create=true in the env YAML)
cmk:
  create: true
  encryption_key_name: "encryptionCMK"
  k8s_key_name: "k8sEncryptionCMK"
  service_bus_key_name: "serviceBusEncryptionCMK"

# Key Vault network posture — secure-by-default baseline.
# Override public_network_access_enabled and network_acls_ip_rules in your env YAML
# if you need temporary public access during bootstrap from a laptop or CI runner.
keyvault:
  public_network_access_enabled: false
  enable_rbac_authorization: true
  sku_name: "standard"
  soft_delete_retention_days: 90    # Cannot be changed after vault creation
  purge_protection_enabled: true
  network_acls_default_action: "Deny"
  network_acls_bypass: "AzureServices"
  network_acls_ip_rules: []

# PostgreSQL baseline
psql:
  public_network_access_enabled: false
  sku_name: "GP_Standard_D2s_v3"
  psql_version: "15"
  backup_retention_days: 7
  bootstrap_aad_principals: false
  auth:
    active_directory_auth_enabled: true
    password_auth_enabled: true
  storage_mb: 32768
  storage_tier: "P10"
  auto_grow_enabled: true
  azure_extensions: "uuid-ossp,ltree"

# Redis baseline
redis:
  public_network_access_enabled: false
  sku_name: "Premium"
  family: "P"
  capacity: 1
  active_directory_authentication_enabled: true
  access_policy_name: "Data Owner"
  non_ssl_port_enabled: false
  tls_version: 1.2
  redis_version: 6

# Storage account baseline
storage_account:
  public_network_access_enabled: false
  min_tls_version: "TLS1_2"
  account_tier: "Standard"
  account_replication_type: "LRS"
  allow_nested_items_to_be_public: false

# OpenAI — leave client.mode empty to let Terraform infer from open_ai.create
open_ai:
  client:
    mode: ""
    key: ""
    org_id: ""
    custom_url: ""

# NAT Gateway baseline
nat_gateway:
  sku_name: "Standard"
  public_ip_sku: "Standard"
  public_ip_allocation_method: "Static"
  zones: ["1", "2", "3"]
  idle_timeout_in_minutes: 10

# AKS baseline
aks:
  load_balancer_sku: "standard"
  only_critical_addons_enabled: true
  key_vault_secret_rotation_enabled: true

# Front Door baseline
frontdoor:
  create: true
  ssl_mode: managed
  minimum_tls_version: "TLS12"
  forwarding_protocol: HttpOnly
  private_link_service_name: "sgp-ingress-lb"
  subdomains: ["auth", "api", "@", "admin"]
  extra_subdomains: []
  dns_ttl_seconds: 60
  ip_filtering:
    enabled: false
    allowed_ip_ranges: []

# System Manager bootstrapping baseline
bootstrapping:
  az_cli_version: "2.59.0"
  helm_chart_version: "2.1.0"
  base_repository: "<your_container_registry_url>"
```

#### File 3: `config/<domain_code>/<location>/<environment_type>.yaml`

Environment-specific configuration. This is where you set everything that differs per environment: resource names, subscription, CIDRs, feature flags, node pool sizing, and bootstrapping state.

```yaml theme={null}
# ── Environment identity ───────────────────────────────────────────────────────
location: eastus2
subscription_id: "<your_azure_subscription_id>"
name_suffix: "<workspace_id>"        # e.g. "acme01"
deployment_id: "<workspace_id>"
deployment_url: "<workspace_id>.workspace.egp.scale.com"
temporal_db_mode: postgresCommonHosted  # Recommended; or "cassandraK8s" (not recommended)

# ── Auth / secrets ────────────────────────────────────────────────────────────
authType: default    # Options: default, SAML, OIDC
RBAC: false
registration_secret: ""     # Auto-generated if empty
identity_service_jwt: ""    # Auto-generated if empty

# ── OIDC (required only when authType = OIDC) ─────────────────────────────────
oidc:
  clientId: ""
  clientSecret: ""
  issuer: ""
  authorizationUrl: ""
  tokenUrl: ""
  userInfoUrl: ""

# ── SAML (required only when authType = SAML) ─────────────────────────────────
saml:
  X509Cert: ""
  SSOUrl: ""
  emailAttrName: ""
  firstNameAttrName: ""
  lastNameAttrName: ""

# ── SSL ───────────────────────────────────────────────────────────────────────
sslMode: e2e    # Options: e2e, managed

# SSL cert/key for Istio ingress (PEM file paths or base64). Required when sslMode=e2e.
istio:
  sslCert: "<path_to_pem_certificate>"
  sslKey: "<path_to_pem_private_key>"

# ── Resource group + networking ───────────────────────────────────────────────
resource_group: "rg-eus2-<domain_code>-<workspace_id>-dev-01"

network:
  vnet: "sgpaz<workspace_id>-vnet"
  int_rt_name: "sgpaz<workspace_id>-rt-int"
  int_nsg_name: "sgpaz<workspace_id>-nsg-int"
  private_cluster: true
  nsg_rules:
    postgresql: true
    redis: true

private_dns:
  create: true
  private_cluster: true

subnets:
  psql:
    name: "sgpaz<workspace_id>-snet-psql"
    type: int
    address_prefixes:
      - "<cidr_block>"    # e.g. "10.95.23.160/27"
    service_endpoints:
      - Microsoft.Storage
    service_delegation:
      name: Microsoft.DBforPostgreSQL/flexibleServers
      actions:
        - Microsoft.Network/virtualNetworks/subnets/join/action

  deployment_scripts:
    name: "sgpaz<workspace_id>-snet-deployscripts"
    type: int
    address_prefixes:
      - "<cidr_block>"
    service_delegation:
      name: Microsoft.ContainerInstance/containerGroups
      actions:
        - Microsoft.Network/virtualNetworks/subnets/action

  aks:
    name: "sgpaz<workspace_id>-snet-aks"
    type: int
    address_prefixes:
      - "<cidr_block>"
    private_endpoint_network_policies: Disabled
    service_endpoints:
      - Microsoft.CognitiveServices

  bastion:
    name: AzureBastionSubnet
    type: int
    address_prefixes:
      - "<cidr_block>"

  jump_host:
    name: "sgpaz<workspace_id>-snet-jumphost"
    type: int
    address_prefixes:
      - "<cidr_block>"

# ── Workload Identity + NAT Gateway ───────────────────────────────────────────
workload_identities:
  enabled: true

nat_gateway:
  enabled: true
  name: "sgpaz<workspace_id>nat"
  zones: []    # [] = regional (no zone pinning)

# ── CMK ───────────────────────────────────────────────────────────────────────
# Set create=true for new deployments. If CMK keys already exist, set create=false
# and provide the key URIs to avoid "already exists (import required)" errors.
cmk:
  enabled: true
  create: true
  # encryption_key_uri: "https://<keyvault>.vault.azure.net/keys/encryptionCMK/<version>"
  # k8s_key_uri: "https://<keyvault>.vault.azure.net/keys/k8sEncryptionCMK/<version>"
  # service_bus_key_uri: "https://<keyvault>.vault.azure.net/keys/serviceBusEncryptionCMK/<version>"

# ── Key Vault ─────────────────────────────────────────────────────────────────
# Override public access settings here if bootstrapping from a laptop or CI runner.
keyvault:
  name: "sgpaz<workspace_id>keyvault"    # Must be globally unique; 3-24 characters
  soft_delete_retention_days: 90          # Must match existing vault if already created
  public_network_access_enabled: true     # Set false once stable; keep true during bootstrap
  network_acls_ip_rules:
    - "<your_public_ip>/32"              # Your laptop or CI runner IP

# ── Core services ─────────────────────────────────────────────────────────────
law:
  create: true

psql:
  create: true
  name: "sgpaz<workspace_id>postgres"    # Must be globally unique
  admin_user: postgres
  bootstrap_aad_principals: true         # Runs once after cluster is up; requires kubectl access

redis:
  create: true
  name: "sgpaz<workspace_id>redis"       # Must be globally unique

storage_account:
  create: true
  name: "sgpaz<workspace_id>storage"    # Must be globally unique; no hyphens
  enable_file_private_endpoint: true

service_bus:
  create: true
  name: "sgpaz<workspace_id>servicebus"
  sku: Premium                            # Premium required for private endpoints
  capacity: 1
  premium_messaging_partitions: 1
  private_endpoint_enabled: true
  public_network_access: false

# ── AI services ───────────────────────────────────────────────────────────────
# Modes: use_openai_via_azure | use_openai_via_custom_endpoint | no_openai
# NOTE: Do not commit real API keys. Keep `key` as a local uncommitted change.
open_ai:
  create: false
  name: "sgpaz<workspace_id>openai"
  client:
    mode: "use_openai_via_custom_endpoint"
    key: "<openai_api_key>"
    custom_url: "<your_azure_openai_endpoint>"

ai_search:
  create: false
  name: "sgpaz<workspace_id>aisearch"

# ── AKS ───────────────────────────────────────────────────────────────────────
aks:
  create: true
  name: "sgpaz<workspace_id>aks"
  node_resource_group: "rg-aks-eus2-<domain_code>-<workspace_id>-dev-01"
  dns_prefix: "egp-k8scluster"
  sku: Free                               # Free | Standard (use Standard for production)
  kubernetes_version: "1.30"
  run_command_enabled: true
  enable_encryption_at_host: true
  use_azure_managed_flux: true
  zones: ["1", "2", "3"]
  istio:
    enabled: true
    ingress_mode: External
    revisions:
      - asm-1-20
  node_pools:
    system:
      name: default
      vm_size: Standard_D4s_v3
      count: 3
      min_count: 3
      max_count: 6
    user:
      enabled: true
      name: user
      vm_size: Standard_D16s_v3
      count: 3
      min_count: 3
      max_count: 10
    gpu:
      enabled: false                      # Set true to enable GPU workloads
      name: gpu
      vm_size: Standard_NV72ads_A10_v5
      min_count: 0
      max_count: 5
    cassandra:
      enabled: false                      # Required only if temporal_db_mode is cassandraK8s
  network:
    pod_cidr: "10.244.0.0/16"
    service_cidr: "10.243.0.0/16"
    dns_service_ip: "10.243.0.10"
    outbound_type: userDefinedRouting

# ── Bastion / jump host ───────────────────────────────────────────────────────
bastion:
  create: true
  admin_username: azureuser
  ssh_public_key: "<your_ssh_public_key>"

# ── Front Door overrides ──────────────────────────────────────────────────────
# Most Front Door settings inherit from default.yaml. Override only what differs.
frontdoor:
  extra_subdomains: []    # e.g. ["chat"] if Agentex UI is enabled

# ── Feature flags ─────────────────────────────────────────────────────────────
feature_flags:
  models: false
  agentex:
    create: false    # Set true to provision Agentex service
  compass:
    create: false    # Set true to provision Workflows service
  dex:
    create: false    # Set true to provision Dex (Document Understanding) service
  reducto:
    create: false    # Set true to provision Reducto service

# ── Bootstrapping (System Manager) ────────────────────────────────────────────
bootstrapping:
  enabled: true
  use_managed_flux: true
  system_manager_version: "<system_manager_image_tag>"
  desiredState: |
    {
      "version": "0.1",
      "packs": [
        { "name": "cert-manager" },
        { "name": "egp" },
        { "name": "identity-service" },
        { "name": "spicedb" },
        { "name": "sgp-apps" },
        { "name": "sgp-models" }
      ]
    }

# ── Policy assignments ────────────────────────────────────────────────────────
policy_assignments:
  enabled: false    # Set true to enforce Azure Policy tag rules

# ── Observability ─────────────────────────────────────────────────────────────
observiqidp:
  monitoring:
    enabled: false
  insights:
    enabled: false
```

<Note>
  **Naming constraints for Azure resources:**

  * Key Vault names: 3–24 alphanumeric characters and hyphens, globally unique
  * Storage Account names: 3–24 lowercase letters and numbers only (no hyphens), globally unique
  * PostgreSQL and Redis names: globally unique within Azure
  * All names must remain stable after first apply — many Azure resources cannot be renamed
</Note>

### Step 2: Provision Infrastructure via Terraform

Navigate to the `infra` directory and initialize:

```bash theme={null}
cd infra

terraform init
```

Review and apply the plan using the `main.tfvars.json` file you configured in the previous step:

```bash theme={null}
terraform plan -var-file=main.tfvars.json -out=tfplan

# Inspect the plan to review resources before proceeding
terraform show tfplan

terraform apply tfplan
```

If the Datadog monitoring integration is enabled in your YAML, pass the API key via environment variable rather than committing it to file:

```bash theme={null}
TF_VAR_datadog_api_key="<your_key>" terraform plan -var-file=main.tfvars.json -out=tfplan
```

*This step may take significant time (30–60 minutes) due to resource creation dependencies, particularly the AKS cluster and PostgreSQL Flexible Server.*

### Step 3: Bootstrap the Cluster

The Azure infrastructure **automatically bootstraps** [SGP System Manager](/docs/infrastructure/system-manager) via an Azure Deployment Script. When `bootstrapping.enabled: true` is set in your configuration, Terraform provisions an Azure Container Instance that:

1. Installs Flux CD on the AKS cluster (using Azure Managed Flux if `use_managed_flux: true`)
2. Applies the System Manager `HelmRepository` and `HelmRelease` Flux CRDs
3. Waits for System Manager to reconcile

Monitor bootstrap progress in the Azure Portal under **Deployment Scripts** in your resource group, or check System Manager logs after bootstrap:

```bash theme={null}
# Get AKS credentials (from inside the VNet or via Bastion — cluster is private by default)
az aks get-credentials \
  --resource-group <resource_group> \
  --name sgpaz<workspace_id>aks \
  --overwrite-existing

# Verify System Manager is running
kubectl get pods -n sgp-system-manager

# Watch System Manager deploy remaining services via Flux
kubectl get helmreleases -A
```

#### Accessing the Private AKS Cluster

Because the AKS cluster is private by default, you must access it from within the provisioned VNet. Two options are provided:

**Option A: From the jump host VM (via Azure Bastion)**

The Bastion host and jump host VM are provisioned when `bastion.create: true`. Connect via the Azure Portal (Bastion blade) or using the helper script:

```bash theme={null}
python3 scripts/connect_private_aks.py --run-mode=local --bootstrap-jump-host
```

**Option B: Using `az aks command invoke`**

When `aks.run_command_enabled: true`, you can run kubectl commands without VPN access:

```bash theme={null}
az aks command invoke \
  --resource-group <resource_group> \
  --name sgpaz<workspace_id>aks \
  --command "kubectl get helmreleases -A"
```

### Step 4: Configure DNS

After `terraform apply` completes, retrieve the Front Door endpoint hostname:

```bash theme={null}
terraform output frontdoor_endpoint_host_name
terraform output frontdoor_dns_zone_id
```

Configure a CNAME record in your DNS provider pointing your `deployment_url` to the Front Door endpoint (the `azurefd.net` hostname from the output above).

<Note>
  If using Azure DNS (the DNS zone is managed by Terraform when `frontdoor.create: true`), CNAME records are created automatically. Verify with:

  ```bash theme={null}
  az network dns record-set list \
    --resource-group <resource_group> \
    --zone-name <deployment_url>
  ```
</Note>

### Step 5: Verify the Deployment

Wait for all services to be ready:

```bash theme={null}
kubectl get helmreleases -A  # All should show Ready=True
kubectl get pods -A          # All should be Running or Completed
```

System Manager continuously reconciles the desired state. The `bootstrapping.desiredState` value in your environment YAML is written to a secret in Azure Key Vault during `terraform apply`, and System Manager reads from that secret at runtime. If a HelmRelease shows `Ready=False`, check its events:

```bash theme={null}
kubectl describe helmrelease <name> -n <namespace>
```

### Step 6: Configure Identity Provider

#### SAML Configuration

Set `authType: "SAML"` in your environment YAML, then configure your Identity Provider with:

* **Service Entity ID**: `https://auth.<deployment_url>`
* **Redirect URI**: `https://auth.<deployment_url>/dashboard/org/saml/callback`

Update the `is-saml-secret` secret in Key Vault (or via System Manager GUI):

```json theme={null}
{
  "id": "<workspace_id>",
  "samlConfiguration": {
    "entityId": "<entity_id_from_your_identity_provider>",
    "x509Cert": "<x509_certificate_from_your_identity_provider>",
    "ssoUrl": "<sso_url_from_your_identity_provider>",
    "attributeMappings": {
      "email": "<email_attribute_name>",
      "firstName": "<first_name_attribute_name>",
      "lastName": "<last_name_attribute_name>"
    }
  }
}
```

#### OIDC Configuration

Set `authType: "OIDC"` in your environment YAML, then configure your Identity Provider with:

* **Redirect URI**: `https://auth.<deployment_url>/dashboard/org/oidc/callback`

Update the `is-oidc-secret` secret in Key Vault (or via System Manager GUI):

```json theme={null}
{
  "id": "<workspace_id>",
  "oidcConfiguration": {
    "clientId": "<client_id_from_your_identity_provider>",
    "clientSecret": "<client_secret_from_your_identity_provider>",
    "issuer": "<issuer_from_your_identity_provider>",
    "authorizationUrl": "<authorization_url>",
    "tokenUrl": "<token_url>",
    "userInfoUrl": "<user_info_url>"
  }
}
```

After modifying an identity secret, restart System Manager to apply the changes:

```bash theme={null}
kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager
```

## Accessing the Platform

If all goes smoothly, you should be able to navigate to the SGP platform at `https://<workspace_id>.workspace.egp.scale.com` (or your custom domain) and authenticate via the configured identity provider.


# GCP Architecture Reference
Source: https://docs.gp.scale.com/docs/infrastructure/gcp/gcp-architecture-reference

Overview of the SGP GCP architecture

## Architecture Diagram

```mermaid theme={null}
graph TD
    USER(["Client"])
    ENGINEER(["Engineer"])

    subgraph GCP["GCP Project"]
        GLB["Global Load Balancer<br/>Static IP · TLS 1.2+"]
        IAP["Cloud IAP"]

        subgraph VPC["VPC — 10.0.0.0/16"]
            INGRESS["Istio Ingress"]

            subgraph GKE["GKE Cluster"]
                SYS["System Pool<br/>3× n2-standard-4"]
                CPU["CPU Pool<br/>4–30× n2-standard-16"]
            end

            BASTION["Bastion VM<br/>e2-micro"]
        end

        subgraph PEERED["VPC-Peered — Cloud SQL"]
            PG_MAIN[("Cloud SQL<br/>PostgreSQL 17 (main)")]
            PG_TEMP[("Cloud SQL<br/>PostgreSQL 17 (temporal)")]
        end

        SM["Secret Manager"]

        subgraph GCS_GROUP["Cloud Storage"]
            BUCKET[("GCS — platform")]
            KB[("GCS — knowledge base")]
        end

        AR["Artifact Registry<br/>Docker + Helm"]
        KMS["Cloud KMS<br/>CMEK (optional)"]
    end

    USER --> GLB
    GLB --> INGRESS
    INGRESS --> CPU
    CPU -->|"VPC Peering · SSL enforced"| PG_MAIN
    CPU -->|"VPC Peering · SSL enforced"| PG_TEMP
    CPU -->|"Workload Identity"| SM
    CPU -->|"Workload Identity"| BUCKET
    CPU -->|"Workload Identity"| KB
    AR -->|"Workload Identity"| CPU
    KMS -.->|"encrypts"| PG_MAIN
    KMS -.->|"encrypts"| PG_TEMP
    KMS -.->|"encrypts"| BUCKET
    ENGINEER --> IAP --> BASTION -->|"private API"| GKE
```

***

## Terraform Structure

SGP's GCP infrastructure is provisioned in two separate Terraform phases with different privilege levels:

```mermaid theme={null}
graph LR
    CREDS(["Personal GCP<br/>Credentials"])

    subgraph PS["projectsetup/ — run once with personal credentials"]
        SA["Terraform<br/>Service Account"]
        AR_PS["Artifact Registries<br/>Docker + Helm"]
        SMK["SA Key stored in<br/>Secret Manager"]
    end

    subgraph DEP["deployments/name/ — SA credentials auto-loaded from Secret Manager"]
        GKE_D["GKE Cluster"]
        SQL_D["Cloud SQL × 2"]
        VPC_D["VPC + Subnets"]
        SEC_D["Secrets + IAM<br/>+ Workload Identity"]
    end

    CREDS -->|"terraform apply"| PS
    SMK -->|"credentials<br/>auto-read at plan time"| DEP
```

| Phase                   | Directory             | Credentials                                          | Scope                                                |
| ----------------------- | --------------------- | ---------------------------------------------------- | ---------------------------------------------------- |
| Privileged bootstrap    | `projectsetup/`       | Your personal GCP identity                           | Service account, Artifact Registries, API enablement |
| Deprivileged main infra | `deployments/<name>/` | Terraform service account (read from Secret Manager) | GKE, Cloud SQL, networking, IAM, secrets             |

The main infrastructure Terraform reads the service account key directly from Secret Manager — no key file needs to exist on disk during the infrastructure run.

***

## Resources by Type

### Compute Resources

| Resource                     | Count | Purpose                                    |
| ---------------------------- | ----- | ------------------------------------------ |
| GKE Cluster                  | 1     | Kubernetes orchestration                   |
| System Node Pool (`default`) | 1     | System pods — tainted `CriticalAddonsOnly` |
| CPU Node Pool (`cpu`)        | 1     | Application workloads                      |
| GPU Node Pool (`gpu`)        | 0–1   | AI/ML workloads (optional)                 |
| Cassandra Node Pool          | 0–1   | Temporal database (optional)               |
| Bastion Host                 | 0–1   | Private cluster access via IAP (optional)  |

**Default node pool sizing:**

| Pool               | Machine Type     | Min Nodes | Max Nodes               |
| ------------------ | ---------------- | --------- | ----------------------- |
| System (`default`) | `n2-standard-4`  | 3         | 10                      |
| CPU (`cpu`)        | `n2-standard-16` | 4         | 30                      |
| GPU (`gpu`)        | `a2-highgpu-1g`  | 0         | 0 (disabled by default) |
| Cassandra          | `n2-standard-4`  | 3         | 6                       |

All pools use `COS_CONTAINERD` image type and are preemptible by default (configurable via `node_pool_config.preemptible`).

***

### Network Resources

| Resource            | Count | Purpose                             |
| ------------------- | ----- | ----------------------------------- |
| VPC Network         | 1     | Network boundary                    |
| Subnetwork          | 1     | Kubernetes nodes                    |
| Secondary IP Ranges | 2     | GKE pod and service CIDRs           |
| Cloud DNS Zone      | 1     | Internal + external name resolution |
| Global Static IP    | 1     | Load balancer ingress endpoint      |
| SSL Policy          | 1     | Minimum TLS 1.2 enforcement         |
| Firewall Rules      | 4–5   | Traffic control (offline mode)      |
| VPC Peering         | 1     | Private connectivity to Cloud SQL   |
| Private Route       | 0–1   | Google APIs access in offline mode  |

***

### Data & Storage Resources

| Resource                      | Count | Purpose                       |
| ----------------------------- | ----- | ----------------------------- |
| Cloud SQL (PostgreSQL 17)     | 1     | Main platform database        |
| Cloud SQL (PostgreSQL 17)     | 1     | Temporal workflow database    |
| GCS Bucket (main)             | 1     | Platform object storage       |
| GCS Bucket (knowledge base)   | 1     | KB document storage           |
| GCS Bucket (monitoring)       | 0–1   | Observability data (optional) |
| Vertex AI Vector Search Index | 0–N   | Vector embeddings (optional)  |
| Cloud Firestore               | 0–1   | Agentex state (optional)      |

Both Cloud SQL instances are private-only (no public IP) and connected to the VPC via VPC peering. SSL is enforced for all database connections (`ENCRYPTED_ONLY`).

***

### Security Resources

| Resource                                                 | Count | Purpose                                     |
| -------------------------------------------------------- | ----- | ------------------------------------------- |
| Secret Manager Secrets                                   | 10+   | Platform configuration and credentials      |
| Cloud KMS Key Ring + Key                                 | 0–1   | Customer Managed Encryption Keys (optional) |
| Service Account (`sgp-<workspace_id>-sa`)                | 1     | Main SGP workload identity SA               |
| Service Account (`sgp-<workspace_id>-node-pool-creator`) | 1     | Node pool creation SA                       |
| Service Account (`sgp-tf-lp-<name>`)                     | 1     | Terraform execution SA (projectsetup)       |
| Workload Identity Pool                                   | 1     | GKE pod → GCP SA binding                    |
| IAP Tunnel (Bastion)                                     | 0–1   | Private cluster access                      |

***

### Artifact Resources (projectsetup phase)

| Resource                   | Count | Purpose              |
| -------------------------- | ----- | -------------------- |
| Artifact Registry (Docker) | 1     | SGP container images |
| Artifact Registry (Helm)   | 1     | SGP Helm charts      |

Both registries are named `sgp-<workspace_id>-docker-repository` and `sgp-<workspace_id>-helm-repository`.

***

### Monitoring Resources

| Resource                   | Count | Purpose                                      |
| -------------------------- | ----- | -------------------------------------------- |
| Cloud Logging              | 1     | GKE system components, API server, workloads |
| VPC Flow Logs              | 0–1   | Network traffic sampling (optional)          |
| GKE Vulnerability Scanning | 1     | Basic vulnerability mode enabled by default  |

***

## Network Architecture

### Address Space

```
VPC: sgp-<workspace_id>-network
└── Subnet: sgp-<workspace_id>-network-kubernetes-subnet
    ├── Primary range:   10.0.0.0/16   (65,536 IPs — GKE nodes)
    ├── Secondary range: 10.2.0.0/16   (65,536 IPs — Kubernetes services)
    └── Secondary range: 10.4.0.0/16   (65,536 IPs — Kubernetes pods)

Cloud SQL VPC Peering:
    └── Reserved range:  /16 block      (managed by service networking)

GKE Control Plane (private):
    └── Master CIDR:     10.5.0.0/28   (configurable via private_gke_master_ipv4_cidr_block)
```

### Traffic Flow

**Ingress (External):**

```
Internet → Global Load Balancer (static IP) → Istio Ingress → Pod
(TLS terminated at load balancer; SSL policy enforces TLS 1.2+)
```

**Database Access:**

```
GKE Pod → VPC Peering → Cloud SQL (private IP)
(SSL enforced; no public IP on Cloud SQL instances)
```

**Google APIs (offline mode):**

```
GKE Pod → Private Google Access → Google APIs (199.36.153.4/30)
(Dedicated route; no default internet gateway route created)
```

**Bastion Access:**

```
Engineer → IAP Tunnel → Bastion VM (e2-micro) → GKE API (private endpoint)
```

***

## GKE Configuration

### Cluster Features

| Feature                    | Value                                        |
| -------------------------- | -------------------------------------------- |
| Datapath provider          | `ADVANCED_DATAPATH` (eBPF-based)             |
| IP stack                   | Dual-stack IPv4/IPv6                         |
| Workload Identity          | Enabled (`<project>.svc.id.goog`)            |
| Secret Manager integration | Enabled                                      |
| Vulnerability scanning     | Basic mode                                   |
| Private nodes              | Enabled (when `offline_mode = true`)         |
| Private endpoint           | Configurable (`enable_gke_private_endpoint`) |
| Master authorized networks | Configurable per deployment                  |
| DNS endpoint               | Enabled (allows external cluster DNS access) |

### Workload Identity

GKE pods authenticate to GCP services using Workload Identity rather than node-level service account keys. Kubernetes service accounts are bound to GCP service accounts via the workload identity pool:

```
<project>.svc.id.goog[<namespace>/<k8s-service-account>]
  → GCP Service Account
    → Secret Manager, GCS, Artifact Registry, etc.
```

Key bindings provisioned by Terraform:

| Kubernetes Identity                             | GCP Role                                                        |
| ----------------------------------------------- | --------------------------------------------------------------- |
| `sgp-system-manager` (system-manager namespace) | `secretmanager.viewer`, `secretmanager.secretAccessor`          |
| `sgp-system-manager-pre-install`                | `secretmanager.viewer`, `secretmanager.secretAccessor`          |
| `egp-api-backend` (egp namespace)               | `secretmanager.secretAccessor`, GCS access via SA impersonation |
| `egp-api-backend-db-setup`                      | `secretmanager.secretAccessor`                                  |
| `agents` (agents namespace)                     | `secretmanager.secretAccessor`                                  |

***

## Secret Manager Secrets

Key secrets provisioned by Terraform:

| Secret Name                             | Contents                                                                       |
| --------------------------------------- | ------------------------------------------------------------------------------ |
| `<prefix>-system-manager-config`        | System Manager runtime configuration (URLs, workspace ID, cloud provider info) |
| `<prefix>-system-manager-desired-state` | Initial desired state (pack list) — managed externally after first apply       |
| `<prefix>-saml-config-secret`           | SAML IdP configuration                                                         |
| `<prefix>-oidc-config-secret`           | OIDC IdP configuration                                                         |
| `terraform-service-account-key-secret`  | Terraform SA private key (used by `deployments/` as provider credentials)      |

The `secret_name_prefix` variable (typically `sgp-<workspace_id>`) scopes secrets for deployments in shared GCP projects.

***

## Customer Managed Encryption Keys (CMEK)

When `useCustomerManagedEncryptionKey = true`, Terraform provisions a Cloud KMS key ring and symmetric encryption key:

| Resource   | Name                            |
| ---------- | ------------------------------- |
| Key Ring   | `sgp-<workspace_id>-key-ring`   |
| Crypto Key | `sgp-<workspace_id>-crypto-key` |

The key is applied to:

* Cloud SQL instances (both main and Temporal)
* GCS buckets

CMEK is recommended for production deployments to maintain cryptographic control over data at rest.

***

## Optional Capabilities

Each optional capability provisions dedicated infrastructure:

| Capability                   | Variable                           | Additional Resources                                                        |
| ---------------------------- | ---------------------------------- | --------------------------------------------------------------------------- |
| Agentex                      | `deployAgentex`                    | Firestore, dedicated service account, GCS bucket                            |
| Workflows                    | `deployCompass`                    | Cloud SQL database, GCS bucket, service account                             |
| Dex (Document Understanding) | `enable_dex`                       | Cloud SQL (`db-custom-4-15360`), GCS bucket, service account                |
| Reducto                      | `enable_reducto`                   | Cloud SQL (`db-custom-2-7680`), GCS bucket, Vision API key, service account |
| Model Engine                 | `deployModelEngine`                | Vertex AI, additional node pools                                            |
| Monitoring                   | `enable_monitoring`                | GCS bucket, IAM bindings                                                    |
| Cloud Build                  | `enable_cloud_build`               | GCS bucket, Artifact Registry, service accounts                             |
| Vertex AI Search             | `vertex_ai_indices`                | Vector Search indices and endpoints                                         |
| LiveKit                      | `provision_livekit_infrastructure` | GCS bucket (audio), IAM bindings                                            |


# GCP SGP Deployment
Source: https://docs.gp.scale.com/docs/infrastructure/gcp/gcp-sgp-deployment

End-to-end guide to deploying SGP in a GCP project.

## Overview

This guide walks you through deploying SGP in a GCP project using the SGP GCP Terraform modules. SGP GCP infrastructure is defined by terraform modules managed by Scale.

GCP deployment is split into two Terraform phases with different privilege levels:

* **`projectsetup/`** — Run once with privileged GCP credentials. Creates a dedicated Terraform service account (with least privileges), enables required GCP APIs, and provisions Artifact Registries for SGP images and Helm charts.
* **`deployments/<your-deployment>/`** — Run with the service account created above. Provisions the GKE cluster, Cloud SQL, networking, and all SGP infrastructure.

This split ensures the main infrastructure Terraform never requires broad privileged credentials.

## Prerequisites

* Access to a GCP project with permissions to manage IAM, APIs, and compute resources
* The following tools installed:
  * [gcloud CLI](https://cloud.google.com/sdk/docs/install) (`gcloud auth login` completed)
  * [Terraform](https://www.terraform.io/) (`>= 1.1.7`)
  * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
  * [helm](https://helm.sh/docs/intro/install/) (for cluster bootstrap)
  * [crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane) or [Docker](https://docs.docker.com/engine/install/) (for copying images)
  * \[sgpctl] ([https://github.com/scaleapi/sgpctl](https://github.com/scaleapi/sgpctl)) (cli tool to execute system manager related commands)
* The following from Scale:
  * The SGP GCP Infrastructure modules (`gcp/`)
  * A `workspace_id` (8-digit number) and `registration_secret` unique to your deployment
* A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional)
* A custom domain for your deployment (optional)

## Installation

### Step 1: Bootstrap the Project

Navigate to the `projectsetup/` directory. This phase creates the Terraform service account and Artifact Registries.

Edit the `locals` block at the top of `projectsetup/main.tf`:

```hcl theme={null}
locals {
  project              = "<your-gcp-project-id>"
  region               = "<gcp-region>"         # e.g. "us-east1"
  zone                 = "<gcp-zone>"            # e.g. "us-east1-a"
  service_account_name = "<workspace_id>-sa"    # Suffix for the service account name
  workspaceID          = "<workspace_id>"        # 8-digit number from Scale, e.g. "90000001"
}
```

Then initialize and apply:

```bash theme={null}
cd projectsetup

terraform init
terraform apply
```

This creates:

* A Terraform service account (`sgp-tf-lp-<service_account_name>`) with the IAM roles needed to provision SGP infrastructure
* A service account key stored as `terraform-service-account-key-secret` in Secret Manager
* Docker and Helm Artifact Registries (`sgp-<workspace_id>-docker-repository`, `sgp-<workspace_id>-helm-repository`)
* All required GCP APIs enabled on the project

The Terraform outputs include the `service_account_email` of the newly created service account, which is useful for auditing. The service account key is automatically stored in Secret Manager and read by the main infrastructure Terraform — no manual key management is required.

### Step 2: Copy SGP Images and Helm Charts

*This step is only required if your deployment is configured to block internet access from the cluster (`offline_mode` in step 3).*

Before provisioning the main infrastructure, copy SGP's Docker images and Helm charts from Scale's registry into your GCP Artifact Registry.

**Download the manifest from Scale's control plane**:

```bash theme={null}
sgpctl manifest download \
  --control-plane-id <workspace_id> \
  --registration-token <token-from-scale-engineer> \
  --version <consolidated-version> \
  --channel <release|dev> \
  --output <version>.manifest.yaml
```

Note: sgpctl manifest download requires internet access to [https://control.egp.scale.com](https://control.egp.scale.com), so it must be run from a connected machine.
Users with air-gapped operator workstations should run it from their laptop and SCP the output to wherever they'll run the mirror commands.

**Authenticate to source + destination registries**:

```bash theme={null}
# Source: Scale's ECR (credentials from Scale engineer)
crane auth login 022465994601.dkr.ecr.us-west-2.amazonaws.com \
  --username AWS \
  --password-stdin

# Destination: customer's GCP Artifact Registry (uses gcloud creds)
gcloud auth configure-docker <region>-docker.pkg.dev
```

**Mirror images and charts**:

```bash theme={null}
# Container images
sgpctl mirror-manifest \
  --manifest-file <version>.manifest.yaml \
  --dest-docker-registry <region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-docker-repository

# Helm charts
sgpctl mirror-helm-charts \
  --manifest-file <version>.manifest.yaml \
  --dest-helm-repo oci://<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-helm-repository
```

**Verify the mirror**:

```bash theme={null}
gcloud artifacts docker images list \
  <region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-docker-repository \
  --include-tags --limit=20

gcloud artifacts docker images list \
  <region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-helm-repository \
  --include-tags --limit=20
```

### Step 3: Configure the Deployment

Copy the `deployments/prototype/` directory and rename it for your deployment:

```bash theme={null}
cp -r deployments/prototype deployments/<your-deployment-name>
cd deployments/<your-deployment-name>
```

Edit the `locals` block at the top of `main.tf`. The deployment reads its Terraform service account credentials automatically from Secret Manager — no key file is required on disk.

```hcl theme={null}
locals {
  project  = "<your-gcp-project-id>"
  region   = "<gcp-region>"                                               # e.g. "us-east1"
  zone     = "<gcp-zone>"                                                 # e.g. "us-east1-b"

  # ── Deployment identity ────────────────────────────────────────────────
  workspaceID        = "<workspace_id>"                                   # 8-digit number from Scale
  registrationSecret = "<registration_secret>"                            # From Scale
  deploymentURL      = "<workspace_id>.workspace.egp.scale.com"          # Or your custom domain

  # ── Repositories ──────────────────────────────────────────────────────
  # baseRepository: used by workloads running inside the cluster to pull images
  # publicBaseRepository: used for initial image pulls during bootstrap (before private DNS resolves)
  # Both point to the same registry when using a private Artifact Registry.
  baseRepository       = "<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-docker-repository"
  publicBaseRepository = "<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-docker-repository"

  # ── System Manager ────────────────────────────────────────────────────
  systemManagerImageTag = "<system_manager_image_tag>"  # From Scale; must match manifest.yaml
  offline_mode          = true                          # true when using your own Artifact Registry

  # ── Auth ──────────────────────────────────────────────────────────────
  authType = "SAML"   # "SAML" or "OIDC"

  # ── Bootstrap ─────────────────────────────────────────────────────────
  # deploy_system_manager=false: Terraform creates infra only; run manual-helm-install.sh to bootstrap
  # deploy_system_manager=true:  Terraform also installs System Manager into the cluster
  deploy_system_manager = false

  # ── DNS ───────────────────────────────────────────────────────────────
  createDNSRecords = false   # Set true if Terraform should manage Cloud DNS records

  # ── Encryption ────────────────────────────────────────────────────────
  useCustomerManagedEncryptionKey = false   # Set true for CMEK (recommended for production)

  # ── Networking ────────────────────────────────────────────────────────
  gke_config = {
    private_endpoint = true
    master_authorized_networks = [
      { cidr_block = "10.0.0.0/16", display_name = "cluster vpc primary range" },
      { cidr_block = "10.2.0.0/16", display_name = "cluster vpc services range" },
      { cidr_block = "10.4.0.0/16", display_name = "cluster vpc pod range" },
    ]
  }
}
```

**If using SAML,** place your IdP's x509 certificate (without BEGIN/END lines) as `x509.cer` in the deployment directory. Update the `samlConfigSecret` in the `module "sgp"` block:

```hcl theme={null}
module "sgp" {
  source = "../../modules/sgp/"
  samlConfigSecret = jsonencode({
    "id" = local.workspaceID
    "samlConfiguration" = {
      "entityId" = "https://auth.${local.deploymentURL}"
      "x509Cert" = file("${path.module}/x509.cer")
      "ssoUrl"   = "<your-idp-sso-url>"
      "attributeMappings" = {
        "email"     = "<email-attribute-from-idp>"
        "firstName" = "<first-name-attribute-from-idp>"
        "lastName"  = "<last-name-attribute-from-idp>"
      }
    }
  })
  # ... other variables
}
```

**If using OIDC,** update `oidcConfigSecret` instead:

```hcl theme={null}
oidcConfigSecret = jsonencode({
  "id"               = local.workspaceID
  "clientId"         = "<client-id-from-idp>"
  "clientSecret"     = "<client-secret-from-idp>"
  "issuer"           = "<issuer-url>"
  "authorizationUrl" = "<authorization-url>"
  "tokenUrl"         = "<token-url>"
  "userInfoUrl"      = "<user-info-url>"
})
```

### Step 4: Provision Infrastructure via Terraform

From the deployment directory, initialize and apply:

```bash theme={null}
terraform init -upgrade

terraform plan -out=tfplan

# Inspect the plan before proceeding
terraform show tfplan

terraform apply tfplan
```

*This step can take 30–60 minutes due to GKE cluster provisioning and Cloud SQL setup.*

After apply completes, connect to the cluster:

```bash theme={null}
gcloud container clusters get-credentials sgp-<workspace_id>-kubernetes-cluster \
  --region <gcp-region> \
  --project <your-gcp-project-id>
```

<Note>
  If `gke_config.private_endpoint = true`, the cluster API server is only accessible from within the VPC. Use the provisioned bastion host (via IAP) or a network-connected runner to access it. Set `bastion_enabled = true` and add your user email to `bastion_iap_members` in the `security_compliance` block to enable bastion access.
</Note>

### Step 5: Bootstrap the Cluster

[SGP System Manager](/docs/infrastructure/system-manager) orchestrates SGP service deployment. If you set `deploy_system_manager = false` in the previous step, bootstrap it manually using the provided script.

Edit `manual-helm-install.sh` in your deployment directory with the correct values:

```bash theme={null}
PROJECT_ID="<your-gcp-project-id>"
PROJECT_NUMBER="<your-gcp-project-number>"    # From: gcloud projects describe <project-id> --format='value(projectNumber)'
BASE_REPOSITORY="<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-docker-repository"
SYSTEM_MANAGER_IMAGE_TAG="<system_manager_image_tag>"
OFFLINE_MODE=true

# Obtain from Cloud SQL instance → "Connect to this instance" in the console, or from Terraform state
SPICEDB_IP_ADDRESS="<cloud-sql-private-ip>"
SPICEDB_PASSWORD="<spicedb-db-password>"
TEMPORAL_DB_PASSWORD="<temporal-db-password>"
```

Then run the bootstrap script:

```bash theme={null}
chmod +x manual-helm-install.sh
./manual-helm-install.sh
```

This installs System Manager into the cluster. System Manager will then begin reconciling the `desired_state.json` file and deploying SGP services automatically.

#### Desired State

The `desired_state.json` file in your deployment directory defines which SGP packs System Manager installs. Update it to reference your GCP Artifact Registry:

```json theme={null}
{
  "version": "0.1",
  "packs": [
    { "name": "flux" },
    {
      "name": "sgp-helm-repository",
      "properties": {
        "helm-repo": {
          "url": "oci://<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-helm-repository"
        }
      }
    },
    { "name": "istio" },
    {
      "name": "sgp-base",
      "properties": {
        "helm-base": {
          "value_overrides": {
            "refreshRegcred": { "enabled": false },
            "gcp": {
              "presharedCertificates": ["sgp-<workspace_id>-ssl-certificate"]
            }
          }
        }
      }
    },
    { "name": "spicedb" },
    { "name": "identity-service" },
    { "name": "temporalf" },
    { "name": "sgp-apps" }
  ]
}
```

See [Step 6](#step-6-configure-ssl-certificates) for the SSL certificate options and how to set the correct value in the `gcp` block above.

### Step 6: Configure SSL Certificates

Three options are available, in order of preference:

#### Option 1: Google Certificate Manager (Recommended)

Provisions a wildcard certificate (`*.your-domain.com`) via DNS authorization. Covers all subdomains automatically.

Enable it in `main.tf`:

```hcl theme={null}
module "sgp" {
  source = "../../modules/sgp/"
  enable_certificate_manager = true
  # ...
}
```

After `terraform apply`, note the two outputs:

* `certificate_manager_dns_auth_record` — a CNAME you must add to your DNS zone for domain validation
* `certificate_map_name` — typically `sgp-<workspace_id>-cert-map`

Add the CNAME to your DNS provider, then update `desired_state.json`:

```json theme={null}
{
  "name": "sgp-base",
  "properties": {
    "helm-base": {
      "value_overrides": {
        "gcp": {
          "certificateMapName": "sgp-<workspace_id>-cert-map"
        }
      }
    }
  }
}
```

The certificate typically provisions within 10–60 minutes of the DNS record being in place.

#### Option 2: Google Managed Certificate

Google provisions and auto-renews per-subdomain certificates. Does not require uploading a certificate, but requires DNS to resolve to the load balancer IP before provisioning, and does not support wildcards.

This is the default when neither `certificateMapName` nor `presharedCertificates` is set in the `sgp-base` pack's `gcp` block. Simply omit those keys from `desired_state.json`.

#### Option 3: Preshared Certificate

Use a certificate you manage and upload to Google Cloud. Required when DNS is not publicly resolvable or Certificate Manager is not available.

Upload the certificate:

```bash theme={null}
gcloud compute ssl-certificates create sgp-<workspace_id>-ssl-certificate \
  --certificate=fullchain1.pem \
  --private-key=privkey1.pem \
  --project <your-gcp-project-id>
```

Reference it in `desired_state.json`:

```json theme={null}
{
  "name": "sgp-base",
  "properties": {
    "helm-base": {
      "value_overrides": {
        "gcp": {
          "presharedCertificates": ["sgp-<workspace_id>-ssl-certificate"]
        }
      }
    }
  }
}
```

<Warning>
  Preshared certificates must be renewed manually. Let's Encrypt certificates expire after 90 days.
</Warning>

After changing the SSL configuration in `desired_state.json`, restart System Manager to apply it:

```bash theme={null}
kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager
```

### Step 7: Configure DNS

Get the external IP address of the load balancer:

```bash theme={null}
# From inside the cluster:
kubectl get svc istio-ingress --namespace istio-ingress -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

# Or from gcloud (the global address is created by Terraform):
gcloud compute addresses describe sgp-<workspace_id>-external-ip \
  --global \
  --project <your-gcp-project-id> \
  --format="value(address)"
```

Create DNS A records in your DNS provider pointing to this IP for:

* `<deployment_url>` (apex)
* `api.<deployment_url>`
* `auth.<deployment_url>`
* `app.<deployment_url>`

If `createDNSRecords = true` in your `main.tf` locals, Terraform manages a Cloud DNS zone and creates these records automatically. Retrieve the name servers from the Terraform output:

```bash theme={null}
terraform output name_servers
```

Then delegate the zone by configuring these as NS records at your domain registrar.

### Step 8: Verify the Deployment

Wait for all services to be ready:

```bash theme={null}
kubectl get helmreleases -A  # All should show Ready=True
kubectl get pods -A          # All should be Running or Completed
```

System Manager continuously reconciles the desired state. If a HelmRelease shows `Ready=False`, check its events:

```bash theme={null}
kubectl describe helmrelease <name> -n <namespace>
```

## Accessing the Platform

Once all HelmReleases are ready and DNS resolves correctly, navigate to `https://<workspace_id>.workspace.egp.scale.com` (or your custom domain) and authenticate via your configured identity provider.


# System Manager
Source: https://docs.gp.scale.com/docs/infrastructure/system-manager

Overview of the SGP System Manager deployment orchestrator

## Overview

System Manager is a Kubernetes operator used to deploy and manage the SGP platform. It is responsible for deploying the SGP platform services and agents to an existing Kubernetes cluster.

## Installation

System Manager is installed as a Helm chart into the Kubernetes cluster during the deployment of the SGP platform. See your cloud provider's corresponding deployment guide for more information.

## Configuration

System Manager is configured via `system-manager-config.json`. This file is stored in the cloud provider's secret manager. To modify the configuration, you can either update the secret directly or use the System Manager GUI, then restart the System Manager deployment to apply the changes.

```bash theme={null}
kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager
```

An example configuration file is shown below. The `aws` block is only present for AWS deployments; GCP and Azure deployments use equivalent `gcp` and `azure` blocks instead. The `baseRepository` format also varies by cloud provider (e.g., `oci://<region>-docker.pkg.dev/<project-id>/sgp-<workspace_id>-helm-repository` for GCP, `oci://<account_id>.dkr.ecr.<region>.amazonaws.com/sgp-<workspace_id>-helm-repository` for AWS).

```json theme={null}
{
  "cloudProvider": "<aws|azure|gcp>",
  "baseRepository": "<helm-repository-oci-url>",
  "samlSetupEnabled": true,
  "oidcSetupEnabled": true,
  "deploymentURL": "https://<workspace_id>.workspace.egp.scale.com",
  "workspaceId": "<workspace_id>",
  "authType": "<saml|oidc>",
  "baseDomain": "<base_domain>",
  "scaletrain_tenant_prefix": "<scaletrain_tenant_prefix>",
  "train_tenant_prefix": "<train_tenant_prefix>",
  "deployAgentex": true,
  "deploySae": true,
  "aws": {
    "accountId": "<aws_account_id>",
    "region": "<aws_region>",
    "prefix": "<prefix>",
    "modelEngineS3Bucket": "scale-egp-<workspace_id>-ml",
    "sqsQueuePolicyTemplate": "",
    "sqsQueueTagTemplate": "",
    "clusterName": "<cluster_name>",
    "karpenterIrsaArn": "<karpenter_irsa_arn>",
    "targetGroupArn": "<target_group_arn>",
    "nodeSubnets": "<node_subnets>",
    "nodeSecurityGroup": "<node_security_group>",
    "postgresHostTemporal": "<postgres_host_temporal>",
    "compassBucketName": "<compass_bucket_name>",
    "compassMongoHost": "<compass_mongo_host>",
    "compassRedisHost": "<compass_redis_host>",
    "reductoBucketName": "<reducto_bucket_name>",
    "reductoDatabaseUrl": "<reducto_database_url>",
    "reductoIrsaRoleArn": "<reducto_irsa_role_arn>",
    "reductoAzureVisionEndpoint": "<reducto_azure_vision_endpoint>",
    "reductoAzureVisionKey": "<reducto_azure_vision_key>",
    "codeBuildProjectName": "<codebuild_project_name>",
    "codeBuildS3Bucket": "<codebuild_s3_bucket>",
    "codeBuildRegistryUrl": "<codebuild_ecr_registry_url>",
    "codeBuildServiceRoleArn": "<codebuild_service_role_arn>",
    "cloudDeployEnabled": true,
    "dex": {
      "irsaRoleArn": "<duc_api_backend_irsa_role_arn>",
      "prefix": "duc-<workspace_id>"
    },
    "identities": {
      "sgpModels": {
        "irsaArn": "<sgp_models_irsa_arn>",
        "secretArns": {
          "backend": "<sgp_models_backend_secret_arn>",
          "model-providers": "<sgp_models_provider_secret_arn>"
        }
      }
    },
    "train": {
      "irsaRoleArn": "<train_irsa_role_arn>",
      "sagemakerExecutionRoleArn": "<train_sagemaker_execution_role_arn>",
      "sagemakerSecurityGroupId": "<train_sagemaker_security_group_id>",
      "databaseHost": "<train_database_host>",
      "dataBucket": "<train_s3_data_bucket_name>",
      "checkpointsBucket": "<train_s3_checkpoints_bucket_name>",
      "outputBucket": "<train_s3_output_bucket_name>",
      "stagingBucket": "<train_s3_staging_bucket_name>"
    },
    "registryProxy": {
      "irsaRoleArn": "<registry_proxy_irsa_role_arn>",
      "ecrRepositoryUrl": "<registry_proxy_ecr_repository_url>",
      "jwtSecret": "<registry_proxy_jwt_secret>",
      "workspaceConfig": "<workspace_config_json>"
    }
  },
  "frontDoorSSLCertB64": "<istio_certificate_b64_resolved>",
  "frontDoorSSLKeyB64": "<istio_key_b64_resolved>",
  "initialDesiredState": "<desired_state_json>",
  "datadog": {
    "enabled": true,
    "env": "<datadog_context>",
    "clusterName": "<cluster_name>",
    "secretName": "<datadog_secret_name>",
    "irsaRoleArn": "<datadog_irsa_role_arn>"
  }
}
```

## Architecture

System Manager runs as a deployment in the `sgp-system-manager` namespace. Its GUI is accessible on port 8000.

```bash theme={null}
kubectl port-forward deployment/sgp-system-manager -n sgp-system-manager 8000:8000
```

Then navigate to `http://localhost:8000` in your browser to access the GUI.

```mermaid theme={null}
graph TD
    ADMIN(["Platform Admin"])

    subgraph CLUSTER["Kubernetes Cluster"]
        subgraph SM_NS["sgp-system-manager namespace"]
            GUI["Web GUI<br/>:8000"]
            API["REST API<br/>/api/v3"]
            WATCHER["Secret Watcher<br/>polls every 3s"]
            INSTALLER["Pack Installer"]
        end

        subgraph RESOURCES["Cluster Resources"]
            NS["Namespaces"]
            SEC["Secrets + Pull Secrets"]
            HR["HelmRelease CRDs"]
        end

        subgraph FLUX_NS["flux-system namespace"]
            FLUX["Flux Helm Controller"]
        end

        PODS["SGP Services<br/>(Helm-managed pods)"]
    end

    subgraph CLOUD["Cloud Provider"]
        SM_SECRET["Secret Manager<br/>desired-state · config"]
        HELM_REPO["Helm Repository<br/>OCI Artifact Registry"]
    end

    ADMIN -->|"browser"| GUI
    GUI --> API
    API -->|"read / write"| SM_SECRET
    WATCHER -->|"detect external changes"| SM_SECRET
    API --> INSTALLER
    INSTALLER --> NS
    INSTALLER --> SEC
    INSTALLER --> HR
    FLUX -->|"watches"| HR
    FLUX -->|"pull chart"| HELM_REPO
    FLUX -->|"deploy"| PODS
```

### Packs

System Manager organizes services into "packs". Each pack is a collection of resources that are deployed together. Generally packs are composed of [FluxCD HelmRelease](https://fluxcd.io/flux/components/helm/helmreleases/) as well as other resources necessary to support a particular service.

When a pack is installed, System Manager renders its resource templates and writes the resulting Kubernetes resources — namespaces, secrets, and FluxCD HelmRelease CRDs — to the cluster. FluxCD then picks up the HelmRelease CRDs and handles pulling and deploying the Helm charts.

### FluxCD

System Manager offloads resource reconciliation to FluxCD. FluxCD is a tool that allows you to manage the lifecycle of your Kubernetes resources. It is responsible for ensuring that the desired state of the resources is maintained. For more information on FluxCD, see the [FluxCD documentation](https://fluxcd.io/flux/components/helm/helmreleases/).

### desired-state.json

The collection of packs that System Manager will deploy is defined in the `desired-state.json` file. This file is stored in the cloud provider's secret manager. To modify the desired state, you can either update the secret directly or use the System Manager GUI, then trigger reconciliation via the System Manager GUI. A sample desired state file is shown below:

```json theme={null}
{
  "version": "0.1",
  "packs": [
    { "name": "flux" },
    { "name": "sgp-helm-repository" },
    { "name": "istio" },
    { "name": "spicedb" },
    { "name": "identity-service" },
    { "name": "temporalf" },
    { "name": "sgp-apps" }
  ]
}
```


# Create Extract Job
Source: https://docs.gp.scale.com/reference/dex/extract/create-extract-job

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/extract
Create a new extract job.

This endpoint creates an extraction job that runs **single-pass** LLM extraction
from a parse result or vector store search context. For agentic (decompose / ReAct)
extraction, use ``POST …/research`` instead; requests with ``use_agentic=True`` are rejected.

Args:
    request: Extract job request containing source_id and parameters

Returns:
    JobEntity: The created job entity

Raises:
    HTTPException: If source not found or workflow startup fails



# Get Extraction
Source: https://docs.gp.scale.com/reference/dex/extract/get-extraction

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/extract/{extraction_id}/result
Get an extraction by ID (job ID) within the specified project.

Args:
    project_id: Project ID that the extraction belongs to
    extraction_id: Extraction ID (job ID) to retrieve

Returns:
    ExtractionEntity: The extraction details

Raises:
    HTTPException: If project or extraction not found, or extraction doesn't belong to project



# List Extractions
Source: https://docs.gp.scale.com/reference/dex/extract/list-extractions

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/extract/results
List all extractions for a project with token-based pagination and filtering.

Only SUCCEEDED extraction jobs are returned. Filter by ids, created_at,
completed_at, or source_id.

Args:
    project_id: Project ID that the extractions belong to
    filter_params: Filter parameters (ids, created_at, completed_at, source_id)
    pagination_params: Pagination parameters (page_size, continuation_token, sort_by, sort_order)

Returns:
    PaginatedItems or list of ExtractionEntity instances

Raises:
    HTTPException: If project not found



# Delete File
Source: https://docs.gp.scale.com/reference/dex/files/delete-file

https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/files/{file_id}
Delete a file from both storage and the database.

This also cleans up associated parse result artifacts from storage.
Parse results are cascade-deleted from the database when the file is removed.



# Get By Id
Source: https://docs.gp.scale.com/reference/dex/files/get-by-id

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/files/{file_id}
Fetch a file by id



# Get File Download Url
Source: https://docs.gp.scale.com/reference/dex/files/get-file-download-url

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/files/{file_id}/download-url
Get a secure, time-limited URL to download the content of a previously uploaded file.

This is useful when the client needs to pass a URL to another service or
download the file through a browser.



# List Files
Source: https://docs.gp.scale.com/reference/dex/files/list-files

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/files
List all files for a project with optional pagination and filtering.

Args:
    project_id: Project ID that the files belong to
    filter_params: Optional filter parameters (ids, created_at_start, created_at_end)
    pagination_params: Optional pagination parameters (page_size, continuation_token, sort_by, sort_order)



# Stream File Content
Source: https://docs.gp.scale.com/reference/dex/files/stream-file-content

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/files/{file_id}/stream
Stream the raw byte content of a file directly.

This endpoint provides direct access to the raw byte content of a file,
making it suitable for streaming applications where the client might not
be able to access the blob storage URL directly.



# Upload File
Source: https://docs.gp.scale.com/reference/dex/files/upload-file

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/files
Upload a file and store it in the configured storage backend.

This endpoint provides the simplest method for uploading files up to a
service-defined limit (e.g., 100 MB).



# Batch Delete Files From Index
Source: https://docs.gp.scale.com/reference/dex/indexes/batch-delete-files-from-index

https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/indexes/{index_id}/files
Delete a batch of files from an existing index. Async via Temporal.



# Batch Upsert Files To Index
Source: https://docs.gp.scale.com/reference/dex/indexes/batch-upsert-files-to-index

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/indexes/{index_id}/files
Add or update a batch of files in an existing index. Async via Temporal.



# Create Index
Source: https://docs.gp.scale.com/reference/dex/indexes/create-index

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/indexes
Create an empty index (synchronous).

Allocates the DB record, writes an empty serialized engine to blob
storage with the provided config, and marks the index ``ready``. Files
are added later via ``POST /indexes/{index_id}/files`` (which kicks off
a batch-mutation workflow).



# Delete Index
Source: https://docs.gp.scale.com/reference/dex/indexes/delete-index

https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/indexes/{index_id}
Delete an index and its blob storage.



# Get Index
Source: https://docs.gp.scale.com/reference/dex/indexes/get-index

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/indexes/{index_id}
Get a specific index with nodes hydrated from blob storage.



# List Indexes
Source: https://docs.gp.scale.com/reference/dex/indexes/list-indexes

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/indexes
List all indexes for a project.



# Navigate Index
Source: https://docs.gp.scale.com/reference/dex/indexes/navigate-index

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/indexes/{index_id}/navigate
Navigate the index hierarchy.



# Search Index
Source: https://docs.gp.scale.com/reference/dex/indexes/search-index

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/indexes/{index_id}/search
Search the index for relevant document IDs.



# Cancel Job
Source: https://docs.gp.scale.com/reference/dex/jobs/cancel-job

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/jobs/{job_id}/cancel
Cancel a running or pending job.

Idempotent: cancelling an already-cancelled job returns 200.
For RUNNING jobs, returns 202 since the workflow handles cancellation
asynchronously.



# Get Job
Source: https://docs.gp.scale.com/reference/dex/jobs/get-job

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/jobs/{job_id}
Get a job by ID within the specified project.

Args:
    project_id: Project ID that the job belongs to
    job_id: Job ID to retrieve

Returns:
    JobEntity: The job details

Raises:
    HTTPException: If project or job not found, or job doesn't belong to project



# List Jobs
Source: https://docs.gp.scale.com/reference/dex/jobs/list-jobs

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/jobs
List all jobs for a project with token-based pagination and filtering.

Args:
    project_id: Project ID that the jobs belong to
    pagination_params: Pagination parameters (page_size, continuation_token, sort_by, sort_order)
    filter_params: Filter parameters (ids, created_at, completed_at, status, operation)

Returns:
    PaginatedItems or list of JobEntity instances

Raises:
    HTTPException: If project not found



# Stream Job Updates
Source: https://docs.gp.scale.com/reference/dex/jobs/stream-job-updates

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/jobs/{job_id}/updates
Stream job trace spans as Server-Sent Events.

Emits three event types:
- ``trace_update``: a new or updated ``TraceSpan`` (JSON-serialised span)
- ``job_status``:   emitted when the database job status changes
- ``done``:         terminal event; the stream closes after this

Clients should reconnect on transient network errors.



# Create Batch Parse Job
Source: https://docs.gp.scale.com/reference/dex/parse/create-batch-parse-job

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/parse/batch
Create a batch parse job that processes multiple documents asynchronously.

If file_ids are provided, only those files are parsed.
If file_ids are omitted, all uploaded files in the project are parsed.

Returns a single batch job that can be polled for progress and results.



# Create Chunk Job
Source: https://docs.gp.scale.com/reference/dex/parse/create-chunk-job

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/parse/{parse_result_id}/rechunk
Rechunk a parse result with a different chunking strategy.

This endpoint initiates rechunking of an existing parse result using the specified
chunking strategy. The operation is performed asynchronously via Temporal workflows.

Args:
    project_id: Project ID that the parse result belongs to
    parse_result_id: Parse result ID to rechunk
    request: Rechunk parse result request with chunking strategy and options

Returns:
    Job entity that can be used to track progress

Raises:
    HTTPException: If the parse result is not found or other errors occur



# Create Parse Job
Source: https://docs.gp.scale.com/reference/dex/parse/create-parse-job

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/parse
Create a parse job that will asynchronously process a document.

This endpoint initiates document parsing using the specified engine (default: Reducto).
The operation is performed asynchronously via Temporal workflows.

Args:
    request: Parse job request with source document ID and parameters

Returns:
    Job entity that can be used to track progress

Raises:
    HTTPException: If the source document is not found or other errors occur



# Get Parse Result
Source: https://docs.gp.scale.com/reference/dex/parse/get-parse-result

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/parse/{parse_result_id}/result
Get a parse result by ID within the specified project.

Args:
    project_id: Project ID that the parse result belongs to
    parse_result_id: Parse result ID to retrieve

Returns:
    ParseResultEntity: The parse result details

Raises:
    HTTPException: If project or parse result not found, or result doesn't belong to project



# List Parse Results
Source: https://docs.gp.scale.com/reference/dex/parse/list-parse-results

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/parse/results
List all parse results for a project with token-based pagination and filtering.

Args:
    project_id: Project ID that the parse results belong to
    filter_params: Filter parameters (ids, created_at, status, engine)
    pagination_params: Pagination parameters (page_size, continuation_token, sort_by, sort_order)

Returns:
    PaginatedItems or list of ParseResultEntityWithoutContent (without content chunks for performance)

Raises:
    HTTPException: If project not found



# Upload Parse Result
Source: https://docs.gp.scale.com/reference/dex/parse/upload-parse-result

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/parse/upload-result
Ingest a pre-computed parse result directly without creating a job.

This endpoint allows direct ingestion of parse results from external tools
while storing them in a dedicated parse_results table for better performance
and indexing.

Args:
    request: Parse result creation request with source document ID and result payload
    storage_gateway: Storage gateway for uploading content to object store

Returns:
    ParseResultEntity representing the stored parse result

Raises:
    HTTPException: If the source document is not found or other errors occur



# Create Project
Source: https://docs.gp.scale.com/reference/dex/projects/create-project

https://dex.sgp.scale.com/openapi.json post /v1/projects
Create a new project.



# Get Project
Source: https://docs.gp.scale.com/reference/dex/projects/get-project

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}
Get a project by ID.



# List Projects
Source: https://docs.gp.scale.com/reference/dex/projects/list-projects

https://dex.sgp.scale.com/openapi.json get /v1/projects
List all projects with pagination and filtering support.



# Update Project
Source: https://docs.gp.scale.com/reference/dex/projects/update-project

https://dex.sgp.scale.com/openapi.json put /v1/projects/{project_id}
Update a project.



# Create Research Job
Source: https://docs.gp.scale.com/reference/dex/research/create-research-job

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/research
Kick off a Dex Research job.

Persists a ``research`` job and starts :class:`~dex_api.temporal.research.workflows.research_workflow.ResearchWorkflow`
on the research task queue.

**Scope:** one ``inputs.vector_stores`` id, **or** one or more ``inputs.parse_results`` ids
(no mixing with vector stores). **Empty inputs:** if the project has exactly one vector store,
that store is used automatically (empty-input shortcut).



# List Research Results
Source: https://docs.gp.scale.com/reference/dex/research/list-research-results

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/research/results
List research jobs for this project (same persistence as ``GET …/jobs``,
forced to ``operation=research``).

Returns :class:`~dex_core.models.jobs.JobEntity` rows (operation is always ``research``).



# Add files to vector store
Source: https://docs.gp.scale.com/reference/dex/vector-stores/add-files-to-vector-store

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/vector-stores/{vector_store_id}/files
Add parsed files to a vector store for indexing and search.
    
    This endpoint initiates an asynchronous process to add files to a vector store:
    1. Creates a background job to track the operation
    2. Starts a Temporal workflow to process the files
    3. Files are parsed, chunked, and embedded into the vector store
    4. Job status can be monitored via the jobs API
    
    **Authentication:**
    - Credentials can be provided either through project-level credentials or request-level credentials
    - For SGP Knowledge Base, SGP credentials are required
    
    **Process:**
    - The operation runs asynchronously in the background
    - Use the returned job entity to monitor progress
    - Files must be previously parsed (have parse_result_ids)
    
    **Returns:**
    - Job entity with ID, status, and metadata for tracking the operation progress
    
    **Example Request:**
    ```json
    {
        "parse_result_ids": ["parse_123", "parse_456"],
        "credentials": {
            "sgp": {
                "api_key": "your-api-key",
                "account_id": "your-account-id"
            }
        }
    }
    ```
    
    **Example Response:**
    ```json
    {
        "id": "job_123456789",
        "operation": "VECTOR_STORE",
        "status": "PENDING",
        "parameters": {
            "parse_result_ids": ["parse_123", "parse_456"]
        },
        "source_id": null,
        "project_id": "proj_987654321",
        "created_at": "2024-01-15T10:30:00Z",
        "updated_at": "2024-01-15T10:30:00Z"
    }
    ```



# Create a new vector store
Source: https://docs.gp.scale.com/reference/dex/vector-stores/create-a-new-vector-store

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/vector-stores
Create a new vector store for a project.
    
    This endpoint creates a vector store using the specified engine and configuration.
    Currently supports SGP Knowledge Base engine, but we plan to support more engines in the future.
    
    **Authentication:**
    - For SGP Knowledge Base, SGP credentials are required
    
    **Returns:**
    - Created vector store entity with ID, name, engine, and creation timestamp
    
    **Example Request:**
    ```json
    {
        "engine": "sgp_knowledge_base",
        "name": "My Knowledge Base",
        "embedding_model": "openai/text-embedding-3-large",
    }
    ```
    
    **Example Response:**
    ```json
    {
        "id": "vs_123456789",
        "project_id": "proj_987654321",
        "name": "My Knowledge Base",
        "engine": "sgp_knowledge_base",
        "created_at": "2024-01-15T10:30:00Z"
    }
    ```



# Delete a vector store
Source: https://docs.gp.scale.com/reference/dex/vector-stores/delete-a-vector-store

https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/vector-stores/{vector_store_id}
Permanently delete a vector store and all its associated data.
    
    This endpoint performs a complete deletion of the vector store:
    1. Deletes the vector store from the third-party engine
    2. Removes all associated files and data
    3. Deletes the vector store record from the database
    
    **Warning:** This operation is irreversible and will permanently delete all data.
    
    **Authentication:**
    - For SGP Knowledge Base, SGP credentials are required
    
    **Returns:**
    - Success message confirming deletion
    
    **Example Response:**
    ```json
    {
        "message": "Vector store deleted successfully"
    }
    ```



# Get vector store details
Source: https://docs.gp.scale.com/reference/dex/vector-stores/get-vector-store-details

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/vector-stores/{vector_store_id}
Retrieve detailed information about a specific vector store.
    
    This endpoint returns the complete metadata for a vector store including:
    - Vector store ID and name
    - Associated project ID
    - Engine type and configuration
    - Creation timestamp
    
    **Returns:**
    - Complete vector store entity with all metadata
    
    **Example Response:**
    ```json
    {
        "id": "vs_123456789",
        "project_id": "proj_987654321",
        "name": "My Knowledge Base",
        "engine": "sgp_knowledge_base",
        "created_at": "2024-01-15T10:30:00Z"
    }
    ```



# List files in vector store
Source: https://docs.gp.scale.com/reference/dex/vector-stores/list-files-in-vector-store

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/vector-stores/{vector_store_id}/files
Retrieve a paginated list of files that have been added to a vector store.
    
    This endpoint returns all files that are currently indexed in the specified vector store,
    including their metadata such as file name, size, MIME type, and creation timestamp.
    
    **Token-based Pagination:**
    - New syntax (recommended): Use `page_size`, `continuation_token`, `sort_by`, `sort_order` query parameters
    - Legacy syntax (deprecated): Use `size` and `continuation_token` query parameters
    - If no pagination parameters are provided, returns a plain list
    - Use the `next_token` from the response to fetch subsequent pages
    - When `next_token` is `null`, you have reached the last page
    
    **Returns:**
    - Paginated list of vector store file entities
    - Total count of files in the vector store
    - `next_token` for pagination (null if no more pages)
    - File metadata including ID, name, size, and MIME type
    
    **Example Response:**
    ```json
    {
        "items": [
            {
                "id": "vsf_123456789",
                "vector_store_id": "vs_123456789",
                "parse_result_id": "pres_987654321",
                "file_id": "file_987654321",
                "file_name": "document.pdf",
                "file_size": 1024000,
                "file_mime_type": "application/pdf",
                "created_at": "2024-01-15T10:30:00Z"
            }
        ],
        "total_count": 1,
        "next_token": "eyJmaWxlX2lkIjoiZmlsZV85ODc2NTQzMjEifQ=="
    }
    ```



# List vector stores
Source: https://docs.gp.scale.com/reference/dex/vector-stores/list-vector-stores

https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/vector-stores
Retrieve a paginated list of vector stores for a project.
    
    This endpoint returns all vector stores associated with the specified project,
    with optional filtering by name and engine type.
    
    **Filtering:**
    - `name`: Filter vector stores by exact name match
    - `engine`: Filter by vector store engine type (e.g., sgp_knowledge_base)
    - `ids`: Filter by vector store IDs (list)
    - `created_at_start`: Filter by creation time (start)
    - `created_at_end`: Filter by creation time (end)
    - `project_id`: Filter by project ID (if not provided in path)
    
    **Sorting:**
    - `sort_by`: Field to sort by (id, created_at, engine, project_id)
    - `sort_order`: Sort order (asc or desc)
    
    **Token-based Pagination:**
    - New syntax (recommended): Use `page_size`, `continuation_token`, `sort_by`, `sort_order` query parameters
    - Legacy syntax (deprecated): Use `size` and `continuation_token` query parameters
    - If no pagination parameters are provided, returns a plain list
    - Use the `next_token` from the response to fetch subsequent pages
    - When `next_token` is `null`, you have reached the last page
    
    **Returns:**
    - Paginated list of vector store entities
    - Total count of matching vector stores
    - `next_token` for pagination (null if no more pages)
    
    **Example Response:**
    ```json
    {
        "items": [
            {
                "id": "vs_123456789",
                "project_id": "proj_987654321",
                "name": "My Knowledge Base",
                "engine": "sgp_knowledge_base",
                "created_at": "2024-01-15T10:30:00Z"
            }
        ],
        "total_count": 1,
        "next_token": "eyJpZCI6InZlY3Rvcl9zdG9yZV8xMjM0NTY3ODkwYWJjZGVmIn0="
    }
    ```



# Remove files from vector store
Source: https://docs.gp.scale.com/reference/dex/vector-stores/remove-files-from-vector-store

https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/vector-stores/{vector_store_id}/files
Remove specific files from a vector store.
    
    This endpoint removes files and their associated embeddings from the vector store:
    1. Removes files from the third-party vector store engine
    2. Deletes file associations from the database
    3. All embeddings and searchable content for these files are permanently removed
    
    **Authentication:**
    - Credentials can be provided either through project-level credentials or request-level credentials
    - For SGP Knowledge Base, SGP credentials are required
    
    **Warning:** This operation permanently removes files and their embeddings from the vector store.
    
    **Returns:**
    - Success message confirming file removal
    
    **Example Request:**
    ```json
    {
        "parse_result_ids": ["parse_123", "parse_456"],
        "credentials": {
            "sgp": {
                "api_key": "your-api-key",
                "base_url": "https://api.example.com"
            }
        }
    }
    ```
    
    **Example Response:**
    ```json
    {
        "message": "Files removed from vector store successfully"
    }
    ```



# Search in vector store
Source: https://docs.gp.scale.com/reference/dex/vector-stores/search-in-vector-store

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/vector-stores/{vector_store_id}/search
Perform semantic search within a vector store.
    
    This endpoint performs vector-based semantic search to find the most relevant content
    based on the provided query. The search uses embeddings to find semantically similar content.
    
    **Search Parameters:**
    - `query`: The search query text
    - `top_k`: Number of results to return (ranked by relevance)
    - `filters`: Optional filters to narrow search scope (e.g., by file_id)
    
    **Authentication:**
    - Credentials can be provided either through project-level credentials or request-level credentials
    - For SGP Knowledge Base, SGP credentials are required
    
    **Returns:**
    - Ranked list of relevant content chunks
    - Each chunk includes content, source blocks, relevance score, and parse result ID
    - Results are ordered by relevance score (highest first)
    
    **Example Request:**
    ```json
    {
        "query": "What are the main features of the product?",
        "top_k": 5,
        "filters": {
            "file_id": "file_123"
        },
        "credentials": {
            "sgp": {
                "api_key": "your-api-key",
                "base_url": "https://api.example.com"
            }
        }
    }
    ```
    
    **Example Response:**
    ```json
    {
        "chunks": [
            {
                "content": "The product includes advanced analytics, real-time monitoring, and automated reporting features.",
                "blocks": [
                    {
                        "id": "block_123",
                        "content": "The product includes advanced analytics...",
                        "type": "text"
                    }
                ],
                "score": 0.95,
                "parse_result_id": "parse_123"
            }
        ]
    }
    ```



# Search in vector store's file
Source: https://docs.gp.scale.com/reference/dex/vector-stores/search-in-vector-stores-file

https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/vector-stores/{vector_store_id}/files/{file_id}/search
Perform semantic search within a vector store and filter down to a specific file.
    Provide the file_id to search within the file, don't confuse it with parse_result_id, because the two are connected.
    
    This endpoint performs vector-based semantic search to find the most relevant content
    based on the provided query. The search uses embeddings to find semantically similar content.
    
    **Search Parameters:**
    - `query`: The search query text
    - `top_k`: Number of results to return (ranked by relevance)
    - `filters`: Optional filters to narrow search scope (e.g., by file_id)
    
    **Authentication:**
    - Credentials can be provided either through project-level credentials or request-level credentials
    - For SGP Knowledge Base, SGP credentials are required
    
    **Returns:**
    - Ranked list of relevant content chunks
    - Each chunk includes content, source blocks, relevance score, and parse result ID
    - Results are ordered by relevance score (highest first)
    
    **Example Request:**
    ```json
    {
        "query": "What are the main features of the product?",
        "top_k": 5,
        "filters": {
            "file_id": "file_123"
        },
        "credentials": {
            "sgp": {
                "api_key": "your-api-key",
                "base_url": "https://api.example.com"
            }
        }
    }
    ```
    
    **Example Response:**
    ```json
    {
        "chunks": [
            {
                "content": "The product includes advanced analytics, real-time monitoring, and automated reporting features.",
                "blocks": [
                    {
                        "id": "block_123",
                        "content": "The product includes advanced analytics...",
                        "type": "text"
                    }
                ],
                "score": 0.95,
                "parse_result_id": "parse_123"
            }
        ]
    }
    ```