# Scale > This file is very large and may be truncated. For an index of this content, please refer to https://docs.gp.scale.com/llms.txt instead. # AWS Architecture Reference Source: https://docs.gp.scale.com/docs/infrastructure/aws/aws-architecture-reference Overview of the SGP AWS architecture ## Architecture Reference ![](https://mintcdn.com/scalegp/zYvSiPc36dH92j52/images/infrastructure/sgp-aws-architecture.png?fit=max\&auto=format\&n=zYvSiPc36dH92j52\&q=85\&s=d8127094a421a4a0e6e567c29a09ae68) ## Cloud Services | Service | Usage | | - | - | | Elastic Kubernetes Service (EKS) | SGP applications are deployed to this cluster | | Identity and Access Management (IAM) | Role and policy management for SGP applications access to cloud services | | Secrets Manager | Secret and configuration management for SGP applications | | Key Management Service (KMS) | Encryption of sensitive data at rest | | Simple Storage Service (S3) | Object storage for SGP applications (knowledge bases, models, etc.) | | Simple Queue Service (SQS) | Karpenter spot instance interruption queue | | OpenSearch | Search provider for SGP applications | | Elastic Container Registry (ECR) | Container registry for custom agent images | | Aurora PostgreSQL | Relational databases for SGP applications | | DocumentDB | Document-oriented database for SGP applications | | ElastiCache for Redis | In-memory caching for SGP applications | | CloudWatch | Logging for cloud services | ## Networking SGP is deployed into a VPC spanning three Availability Zones with separate public and private subnets. All application workloads run in private subnets. Outbound internet access is provided by a NAT Gateway. ### Ingress | Property | Default | | - | - | | Load balancer type | Application Load Balancer (ALB) | | Listener port | 443 (TLS) | | Placement | External (internet-facing) | | WAFv2 | Optional — associate an existing Web ACL by name | | VPC Endpoint Service | Optional — enables AWS PrivateLink for private connectivity | > [!NOTE] > > The load balancer can be configured as internal (private subnet only) for deployments that require no direct internet exposure. ### Service Mesh System Manager's `istio` pack deploys Istio, which manages all east-west traffic between SGP services. An internal Istio ingress gateway handles traffic routing within the cluster. ## EKS Cluster Default Configuration ### Node Groups | Node Group | Size | Limits | Machine Type | Machine Image | Taints | | - | - | - | - | - | - | | System | 3 (one node group of 1 per private subnet) | — | m6a.2xlarge | EKS Default AMI | CriticalAddonsOnly=true:NoSchedule | | Default | Karpenter Managed Node Group | 1000 CPU | c\*, m\*, r\* | Karpenter AL2 AMI Family (default) | | | High Storage | Karpenter Managed Node Group | 100 CPU | c\*, m\*, r\* | Karpenter AL2 AMI Family (default) | high-storage:NoSchedule | | GPU | Karpenter Managed Node Group | 385 CPU | g5.\* | Karpenter AL2 AMI Family (default) | nvidia.com/gpu:NoSchedule | | GPU (Large) | Karpenter Managed Node Group | 100 CPU | p4d.\* | Karpenter AL2 AMI Family (default) | nvidia.com/gpu:NoSchedule | Karpenter AMI families are configurable through `karpenter.amiFamily` in the `egp-base` chart. AL2 is the configured default; AL2023 and Bottlerocket require an explicit configuration. ### Cluster Add-ons Terraform installs the EKS add-ons. System Manager installs the others as packs, when they are listed in its desired state. Karpenter provides node autoscaling; Cluster Autoscaler is not installed. | Add-on | Installed by | Purpose | | - | - | - | | Karpenter | System Manager pack `karpenter` | Workload-driven node autoscaling for Karpenter-managed node groups | | Istio | System Manager pack `istio` | Service mesh for east-west traffic management | | cert-manager | System Manager pack `cert-manager` | TLS certificate provisioning and renewal | | NVIDIA GPU Operator | System Manager pack `gpu-operator` | GPU driver and device plugin lifecycle management | | CoreDNS | EKS add-on | Cluster DNS | | kube-proxy | EKS add-on | Service networking on each node | | Amazon VPC CNI | EKS add-on | Pod networking | | AWS EBS CSI Driver | EKS add-on | Persistent volume support for EKS workloads | | metrics-server | EKS add-on | Resource metrics for autoscaling | | Amazon EFS CSI Driver | EKS add-on, off by default | EFS volumes; on with `enable_efs_csi_driver` or Document Understanding | | EKS Pod Identity Agent | EKS add-on, off by default | EKS Pod Identity; on with `enable_pod_identity_agent` | ## Aurora PostgreSQL Default Configuration These are the default Aurora instance counts for an SGP deployment on AWS; your deployment configuration can override them. | Instance | Service | Instance Count | Engine | Instance Class | | - | - | - | - | - | | agents-service-\-db | Agents | 3 | PostgreSQL16 | db.r6g.xlarge | | duc-\-duc-api-aurora-db | Document Understanding Capability | 3 | PostgreSQL15 | db.r6g.large | | reducto-aurora-db | Reducto | 3 | PostgreSQL15 | db.t4g.medium | | sae-\-sae-postgres | Semantic Analysis Engine | 1 | PostgreSQL17 | db.r6g.large | | scale-egp-\-agentex-aurora-db | Agentex | 3 | PostgreSQL15 | db.r6g.large | | scale-egp-\-egp-api-aurora-db | EGP API | 3 | PostgreSQL15 | db.r6g.large | | scale-egp-\-identity-service-aurora-db | Identity Service | 3 | PostgreSQL15 | db.r6g.large | | scale-egp-\-llm-engine-aurora-db | LLM Engine | 3 | PostgreSQL14 | db.r6g.large | | scale-egp-\-spicedb-postgres | SpiceDB | 3 | PostgreSQL16 | db.r6g.large | | scale-egp-\-temporal-aurora-db | Temporal | 3 | PostgreSQL15 | db.r6g.large | | scale-egp-\-train-aurora-db | SGP Train | 1 | PostgreSQL15 | db.t3.medium | ## DocumentDB Default Configuration | Instance | Service | Instance Count | Instance Class | | - | - | - | - | | scale-egp-\-compass-docdb | Compass | 1 | db.t3.medium | | scale-egp-\-agentex-mongo | Agentex | 1 | db.r5.large | ## Customer Sensitive Data Sensitive data may be stored in the following services, all data is encrypted at rest and in transit: * Aurora PostgreSQL: For storing metadata, persistent data, and service-specific databases (SGP API Backend, Identity Service, LLM Engine, SpiceDB, Agents Service, SGP Train, and Temporal) * DocumentDB: For storing Agentex agent data/workflows and Workflows document processing state * S3 Buckets: For storing large data objects and datasets * OpenSearch: For managing text embeddings and vector search capabilities # AWS SGP Deployment Source: https://docs.gp.scale.com/docs/infrastructure/aws/aws-sgp-deployment End-to-end guide to deploying SGP in your AWS account, together with Scale engineering. ## Overview This guide covers deploying SGP in an AWS account. SGP AWS infrastructure is defined by terraform modules managed by Scale. > [!NOTE] > > SGP is deployed jointly with Scale engineering. Scale provides everything under "The following from Scale" in Prerequisites and stays involved for upgrades and ongoing operation. Contact your Scale team before you start. ## Prerequisites * Access to an AWS account with sufficient permissions to create resources * The following tools installed: * [AWS CLI](https://aws.amazon.com/cli/) * [Terraform](https://www.terraform.io/) 1.9.0 or later * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) * [helm](https://helm.sh/docs/intro/install/) and [jq](https://jqlang.org/download/) (used by `install.sh`) * [sgpctl](https://github.com/scaleapi/sgpctl) (CLI that installs System Manager) * The following from Scale: * The SGP AWS Infrastructure Terraform modules * A `workspace_id` and `registration_token` unique to your deployment * The `bootstrap-helm` folder: `install.sh` and the `system-manager-dependencies` chart it installs * The consolidated version of SGP to install * A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional) * A custom domain for your deployment (optional) ## Installation ### Step 1: Prepare Terraform Root Module In a new directory, create a `.tf` file to invoke the SGP AWS infrastructure Terraform modules. Reference the `egp-hybrid-cloud-customer-deployment` module as the entrypoint for the SGP infrastructure. ```terraform theme={null} module "egp_hybrid_deployment" { source = "" scale_account_id = "" registration_token = var.registration_token # secrets are variables, set in Step 2 domain = ".workspace.egp.scale.com" # or your custom domain # Custom domain: import a certificate for and *. so the first apply doesn't wait for # ACM validation. You still create the CNAMEs listed in Step 2. The private key is stored in Terraform state. # ssl_cert_certificate_body_path = "" # ssl_cert_private_key_path = "" # ssl_cert_certificate_chain_path = "" # Omit for a self-signed certificate # Core services — enable the components you need deploy_egp_api = true # EGP API backend — required for the SGP platform deploy_spellbook = false # Spellbook prompt management service (optional) deploy_llm_engine = false # Model Engine service (optional) enable_agentex = false # Agentex agentic workflow service (optional) create_agentex_tool_cache = false # Persistent tool cache for Agentex (optional) enable_cloud_build = false # Cloud-based container build service for Agentex (optional) add_agents_service = false # Agent Service for building and running agents (optional) add_document_understanding_capability = false # Document Understanding (Dex) service (optional) enable_reducto = false # Reducto OCR and document parsing service (optional) enable_train = false # SGP model training infrastructure (optional) enable_compass = false # Workflows workflow orchestration service (optional) deploy_ukbp = false # Universal Knowledge Base Pipeline (optional) enable_system_manager = true # System Manager deployment orchestrator — required enable_monitoring = false # Observability stack (optional) desired_state_json = jsonencode({ # Packs System Manager installs; when unset, it deploys nothing packs = [ { name = "" }, ] }) create_temporal_postgresql = true # Recommended; set false to use an in-cluster Cassandra instead omit_egp_control_plane = false # true is LLM Engine-only mode: no control plane registration, workspace load balancer, listener or certificate base_repository = "" # Only needed if not pulling from Scale's registry openai_api_key = var.openai_api_key openai_org_id = "" anthropic_api_key = var.anthropic_api_key launch_api_Key = var.launch_api_Key # the module's argument name has a capital K authentication_type = "SAML" # "SAML" or "OIDC" use_cmk = true # Recommended for production: encrypts data at rest using AWS KMS # If deploying into an existing VPC rather than creating a new one, provide the IDs here # vpc_id = "" # private_subnet_ids = ["", ""] # public_subnet_ids = ["", ""] access_cidrs = ["0.0.0.0/0"] # CIDR blocks allowed to reach the platform; ["0.0.0.0/0"] is open to the internet roles_to_access_kubeapi = ["arn:aws:iam:::role/"] # IAM roles given cluster admin access to EKS; leave out the role that runs Terraform, which already has it eks_additional_access_cidrs = ["/32"] # IPs allowed to reach the EKS API endpoint, in addition to Scale's # Load balancer configuration load_balancer_type = "application" # "application" or "network" # waf_name = "" # Optional; only applies to application load balancers internal_lb = false # Set true to prevent internet access to the load balancer # additional_certs = [""] # Additional TLS certificates for the load balancer # permissions_boundary_arn = "" # Optional: IAM permissions boundary for provisioned roles # See the module documentation for the full list of available variables } # Secrets stay out of this file. Step 2 sets each one from an environment variable # named TF_VAR_. variable "registration_token" { type = string sensitive = true } variable "openai_api_key" { type = string sensitive = true default = null } variable "anthropic_api_key" { type = string sensitive = true default = null } variable "launch_api_Key" { type = string sensitive = true default = null } # install.sh needs SpiceDB's connection string in Step 3 output "spicedb_connection_string" { value = module.egp_hybrid_deployment.spicedb_connection_string sensitive = true } # DNS records for a custom domain point at these (see Step 2) output "load_balancer_dns" { value = module.egp_hybrid_deployment.load_balancer_dns } output "certificate_arn" { value = module.egp_hybrid_deployment.certificate_arn } # Example Terraform provider and backend configuration provider "aws" { region = "" } terraform { required_version = ">= 1.9.0" backend "s3" { bucket = "sgp--terraform-state" key = "sgp--sgp.tfstate" region = "" } } ``` Each `packs` entry in `desired_state_json` names a [System Manager](/docs/infrastructure/system-manager) pack; get the pack list for your install from your Scale team. System Manager reads `desired_state_json` only on first install, while its desired state is empty; after that, change packs in the System Manager GUI or API. ### Step 2: Provision Infrastructure via Terraform Run the following commands in the same directory as the previous step's .tf file. Terraform calls Scale's control plane during plan and apply, so the machine running it needs HTTPS access to `https://control.egp.scale.com`. ```bash theme={null} # Set the secrets Step 1 reads, without echoing them or saving them in shell history. # Skip the lines for API keys you don't use. printf 'Registration token: ' && read -rs TF_VAR_registration_token && echo && export TF_VAR_registration_token printf 'OpenAI API key: ' && read -rs TF_VAR_openai_api_key && echo && export TF_VAR_openai_api_key printf 'Anthropic API key: ' && read -rs TF_VAR_anthropic_api_key && echo && export TF_VAR_anthropic_api_key printf 'Model Engine API key: ' && read -rs TF_VAR_launch_api_Key && echo && export TF_VAR_launch_api_Key terraform init terraform plan -out=tfplan # Inspect this plan to review the planned resources to be provisioned before proceeding terraform show tfplan terraform apply tfplan ``` *This step may take significant time due to resource creation dependencies* > [!NOTE] > > Unless you import a certificate with the `ssl_cert_*_path` arguments in Step 1, the first apply will likely fail with this error: > > ```bash theme={null} > Error: creating ELBv2 Listener (arn:aws:elasticloadbalancing:us-east-1:12345678910:loadbalancer/net/scale-egp-/12345678910): operation error Elastic Load Balancing v2: CreateListener, https response error StatusCode: 400, RequestID: 12345678910, api error UnsupportedCertificate: The certificate 'arn:aws:acm:us-east-1:12345678910:certificate/12345678910' must have a fully-qualified domain name, a supported signature, and a supported key size. > ``` > > The listener needs an issued ACM certificate, validated through DNS. For a `.workspace.egp.scale.com` domain, Scale creates the DNS records: wait until ACM shows the certificate as issued, then re-run `terraform apply`. For a custom domain, first create these records with your DNS provider: > > * `` and `*.`: CNAME to the load balancer's DNS name, from `terraform output -raw load_balancer_dns` (use an alias record if `` is the apex of its DNS zone) > * The certificate's validation CNAME, whose name and value this command prints: > > ```bash theme={null} > aws acm describe-certificate --region \ > --certificate-arn "$(terraform output -raw certificate_arn)" \ > --query 'Certificate.DomainValidationOptions[].ResourceRecord' > ``` ### Step 3: Bootstrap the Cluster Export the values `install.sh` and `sgpctl` need. Run these in the same directory as the Step 1 `.tf` file, so `terraform output` reads its state: ```bash theme={null} export WORKSPACE_ID="" export AWS_REGION="" export SPICEDB_DATASTORE_URI="$(terraform output -raw spicedb_connection_string)" # The Secrets Manager secret holding SpiceDB's preshared key export SPICEDB_SECRET_ID="spicedb/$WORKSPACE_ID" ``` #### Configure EKS access The IAM identity that created the cluster in Step 2 has cluster admin access, and so do the roles in `roles_to_access_kubeapi` from Step 1. #### Allowlist your IP (if needed) The EKS API endpoint accepts connections from Scale's addresses and from the `eks_additional_access_cidrs` in Step 1, or only from `access_cidrs` when `omit_egp_control_plane = true`. To run the commands below from your machine, include its IP there. #### Install the prerequisites with install.sh Run the `install.sh` script to install what System Manager needs in the cluster: * The Secrets Store CSI driver and its AWS provider * The `sgp-system-manager` and `spicedb` namespaces * The `spicedb-config` secret, holding the connection string you exported and the preshared key from the SpiceDB secret Terraform created in AWS Secrets Manager `install.sh` reads that secret with your AWS credentials. It does not install System Manager. Run it from the `bootstrap-helm` folder, because it installs the `system-manager-dependencies` chart from there. ```bash theme={null} # Set the correct kubectl context aws eks update-kubeconfig --name "scale-egp-$WORKSPACE_ID" --region "$AWS_REGION" # Execute the install.sh script. Once it succeeds, the connection string is in the cluster, # so remove it from your shell cd ./install.sh && unset SPICEDB_DATASTORE_URI ``` If `install.sh` fails, fix the cause and run it again: the exports are still set. To run it again after it has succeeded, or from a new shell, first repeat the exports above from the Step 1 directory. #### Install System Manager with sgpctl Install [SGP System Manager](/docs/infrastructure/system-manager) with `sgpctl`, using the consolidated version from Scale. `sgpctl` installs the chart from the `scale-egp-charts` Helm repository, so add it first. It also fetches a license key and registry credentials from Scale's control plane, so the machine running it needs HTTPS access to `https://control.egp.scale.com`. `` is the AWS account Terraform deployed into. ```bash theme={null} helm repo add scale-egp-charts https://scale-egp-helm-charts-us-west-2.s3.amazonaws.com/charts # Read the registration token without echoing it or saving it in shell history printf 'Registration token: ' && read -rs SGP_REGISTRATION_TOKEN && echo sgpctl install aws \ --workspace-id "$WORKSPACE_ID" \ --control-plane-id "$WORKSPACE_ID" \ --registration-token "$SGP_REGISTRATION_TOKEN" \ --consolidated-version \ --aws-account-id \ --aws-region "$AWS_REGION" ``` System Manager is the deployment orchestrator for the SGP platform. It deploys the packs listed in `desired_state_json` from Step 1. ### Step 4: Verify the Deployment Wait for all services to be ready: ```bash theme={null} kubectl get helmreleases -A # All should be Ready=True kubectl get pods -A # All should be Running ``` ### Step 5: Configure Identity Provider #### SAML Configuration In your Identity Provider, configure a new SAML application with the following settings: * Service Entity ID: `https://auth.` * Redirect URI: `https://auth./dashboard/org/saml/callback` Modify the configuration `scale-egp-/identity-service-saml-secrets` (either via System Manager or directly in AWS Secrets Manager): ```json theme={null} { "id": "", "samlConfiguration": { "entityId": "", "x509Cert": "", "ssoUrl": "", "attributeMappings": { "email": "", "firstName": "", "lastName": "" } } } ``` #### OIDC Configuration In your Identity Provider, configure a new OIDC application with the following settings: * Redirect URI: `https://auth./dashboard/org/oidc/callback` Modify the configuration `scale-egp-/identity-service-oidc-secrets` (either via System Manager or directly in AWS Secrets Manager) with the following JSON: ```json theme={null} { "id": "", "oidcConfiguration": { "clientId": "", "clientSecret": "", "issuer": "", "authorizationUrl": "", "tokenUrl": "", "userInfoUrl": "" } } ``` After modifying the secret, restart the `sgp-system-manager` deployment to apply the changes. ```bash theme={null} kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager ``` ## Accessing the Platform If all goes smoothly, you should be able to navigate to the SGP platform at `https://.workspace.egp.scale.com` (or your custom domain) and authenticate via the configured identity provider. # Azure Architecture Reference Source: https://docs.gp.scale.com/docs/infrastructure/azure/azure-architecture-reference Overview of the SGP Azure architecture ## Architecture Diagram ```mermaid theme={null} graph TD USER(["Client"]) INTERNET(["Internet"]) subgraph AZ["Azure Subscription"] FD["Azure Front Door
WAF · CDN · TLS termination"] subgraph VNET["Virtual Network — 10.0.0.0/16"] INGRESS["Istio Ingress
Internal Load Balancer"] NAT["NAT Gateway
Outbound egress"] subgraph AKS["AKS Cluster — private API server"] SYS["System Node Pool
3–10× Standard_D4s_v3"] APP["User Node Pool
1–8× Standard_D32s_v3"] end BASTION["Azure Bastion"] JUMP["Jump Host VM"] end KV["Key Vault
Secrets + CMK keys"] LAW["Log Analytics Workspace"] PG[("PostgreSQL
Flexible Server")] REDIS[("Redis Cache
Premium")] STOR[("Storage Account")] SB[("Service Bus
Premium")] end USER --> FD FD -->|"Private Link"| INGRESS INGRESS --> APP APP -->|"Delegated subnet / VNet integration"| PG APP -->|"Private Endpoint"| REDIS APP -->|"Private Endpoint"| STOR APP -->|"Private Endpoint"| SB APP --> KV SYS --> KV APP --> LAW SYS --> LAW APP --> NAT --> INTERNET BASTION --> JUMP -->|"kubectl"| AKS ``` Node pool sizes on this page are the Terraform defaults. Values under `aks.node_pools` in your environment YAML override them field by field; the example in the [deployment guide](/docs/infrastructure/azure/azure-sgp-deployment) sets 3–6 system nodes and 3–10 `Standard_D16s_v3` user nodes. *** ## Resources by Type ### Compute Resources | Resource | Count | Purpose | | - | - | - | | AKS Cluster | 1 | Kubernetes orchestration | | System Node Pool | 1 | System pods (autoscales 3–10 nodes by default) | | User Node Pool | 1 | Application workloads | | GPU Node Pool | 0-1 | AI/ML workloads (optional) | | Cassandra Pool | 0-1 | Temporal database (optional) | Node counts depend on each pool's configured minimum and maximum and on enabled optional pools. *** ### Network Resources | Resource | Count | Purpose | | - | - | - | | Virtual Network | 1 | Network boundary | | Subnets | 4-5 | Network segmentation | | NSGs | 4-5 | Traffic control | | Route Tables | 3-4 | Traffic routing | | Private DNS Zones | 7 | Internal name resolution | | Private Endpoints | 7 | Secure PaaS access | | Bastion Host | 0-1 | Secure VM access | | NAT Gateway public IP | Configuration dependent | Outbound egress | | Bastion public IP | 0-1 | Bastion endpoint when enabled | *** ### Data & Storage Resources | Resource | Count | Purpose | | - | - | - | | PostgreSQL Server | 1 | Relational database | | Redis Cache | 1 | Distributed cache | | Storage Account | 1 | Blob/File storage | | AI Search Service | 0–1 | Full-text search (optional) | | OpenAI Service | 0–1 | LLM models (optional) | *** ### Security Resources | Resource | Count | Purpose | | - | - | - | | Key Vault | 1 | Secrets management | | Cluster managed identities | Configuration dependent | AKS and kubelet authentication | | Workload managed identities | One per enabled service | Service authentication, including cert-manager, identity-service, API backend, annotation, System Manager, secret-mounter, SpiceDB, and agents | | RBAC Role Assignments | 10+ | Access control | *** ### Monitoring Resources | Resource | Count | Purpose | | - | - | - | | Log Analytics Workspace | 1 | Centralized logging | | Data Collection Rule | 1 | AKS metrics | | Diagnostic Settings | 7+ | Resource logging | | Datadog Connection | 0-1 | External monitoring | *** ## Network Architecture ### Address Space Planning ``` VNet: 10.0.0.0/16 (65,536 IPs) ├── AKS Subnet: 10.0.1.0/24 (256 IPs) ├── Bastion Subnet: 10.0.2.0/26 (64 IPs) ├── Database Subnet: 10.0.3.0/24 (256 IPs) └── Private Endpoints: 10.0.4.0/25 (128 IPs) Pod CIDR: 10.244.0.0/16 (65,536 IPs) Service CIDR: 10.243.0.0/16 (65,536 IPs) ``` ### Traffic Flow **Egress (Internet):** ``` Pods/VMs → NAT Gateway → Public IP → Internet (Stateful, return traffic allowed) ``` **Ingress (Internal):** ``` Service IP → Load Balancer → Pod IP (via CNI) ``` **Database Access:** ``` AKS Pods → VNet → PostgreSQL delegated subnet (DNS: server.postgres.database.azure.com) Private Endpoint access is an alternative when configured; core PostgreSQL defaults to VNet integration. ``` **External Service Access:** ``` AKS Pods → API Gateway / Load Balancer → OpenAI / AI Search (Via Private Endpoints) ``` *** ## AKS Configuration Deep Dive ### API Server Access * **Type:** Private cluster (recommended) * **Endpoint:** Internal only * **Access Method:** Bastion host or VPN * **DNS:** A private name that starts with `aks.dns_prefix` and ends in `privatelink..azmk8s.io`. Print it with `az aks show --resource-group --name --query privateFqdn -o tsv`. ### Network Policies * **Engine:** Cilium with Azure CNI Overlay (default, configurable in `aks.network`) * **Scope:** Pod-to-pod communication * **Default:** Allow all (unrestricted) * **Configuration:** Define in Kubernetes manifests ### Container Registry * **Integration:** Azure Container Registry (optional) * **Authentication:** Managed identity or pull secrets * **Pulling:** Private endpoint (optional) ### Monitoring & Observability * **Azure Monitor Agent:** Deployed in kube-system * **Metrics:** CPU, Memory, Disk, Network * **Logs:** Container stdout/stderr, Kubernetes events * **Dashboards:** Pre-built in Log Analytics # Azure SGP Deployment Source: https://docs.gp.scale.com/docs/infrastructure/azure/azure-sgp-deployment End-to-end guide to deploying SGP in your Azure subscription, together with Scale engineering. ## Overview This guide covers deploying SGP in an Azure cloud subscription using the SGP Azure Terraform modules. SGP Azure infrastructure is defined by terraform modules managed by Scale. > [!NOTE] > > SGP is deployed jointly with Scale engineering. Scale provides everything under "The following from Scale" in Prerequisites and stays involved for upgrades and ongoing operation. Contact your Scale team before you start. ## Prerequisites * Access to an Azure subscription with sufficient permissions to create resources (Contributor + User Access Administrator roles, or equivalent) * The following tools installed: * [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) (`az login` completed) * [Terraform](https://www.terraform.io/) (`>= 1.9.0, < 2.0.0`) * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) * The following from Scale: * The SGP Azure Infrastructure Terraform modules (`azure-terraform/infra`) * A `workspace_id`, plus the `control_plane_id` and `registration_token` that register your deployment with Scale's control plane * A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional) * A custom domain for your deployment (optional) ## Installation ### Step 1: Build Configuration Configuration is split across three files. This allows you to reuse the same configuration for multiple environments. ``` infra/ ├── main.tfvars.json # Selects which domain + environment to deploy └── config/ └── / ├── default.yaml # Domain-wide defaults (all environments) └── / └── .yaml # Environment-specific overrides ``` #### File 1: `main.tfvars.json` This file selects which configuration to load. Edit it to select your domain code, region and environment before running Terraform. ```json theme={null} { "environment_type": "", "domain_code": "", "location": "" } ``` | Field | Description | Allowed values | | - | - | - | | `environment_type` | Environment tier | `dev`, `qa`, `staging`, `uat`, `prod` | | `domain_code` | Customer identifier (3–9 alphanumeric chars/hyphens) | e.g. `acme`, `contoso` | | `location` | Azure region | e.g. `eastus2`, `northeurope`, `southeastasia` | #### File 2: `config//default.yaml` Customer-wide baseline settings that apply to **all** environments unless overridden. This file is the right place for stable policy and posture decisions: tenant identity, tagging standards, and security baselines for Key Vault, PostgreSQL, Redis, Storage, and other services. ```yaml theme={null} # Identity business_unit: "" tenant_id: "" default_tags: costcenter: "" product: "" deploymenttype: "new" safe-to-delete: "no" # CMK key names (Terraform creates these keys when cmk.create=true in the env YAML) cmk: create: true encryption_key_name: "encryptionCMK" k8s_key_name: "k8sEncryptionCMK" service_bus_key_name: "serviceBusEncryptionCMK" # Key Vault network posture — secure-by-default baseline. # Override public_network_access_enabled and network_acls_ip_rules in your env YAML # if you need temporary public access during bootstrap from a laptop or CI runner. keyvault: public_network_access_enabled: false enable_rbac_authorization: true sku_name: "standard" soft_delete_retention_days: 90 # Cannot be changed after vault creation purge_protection_enabled: true network_acls_default_action: "Deny" network_acls_bypass: "AzureServices" network_acls_ip_rules: [] # PostgreSQL baseline psql: public_network_access_enabled: false sku_name: "GP_Standard_D2s_v3" psql_version: "15" backup_retention_days: 7 bootstrap_aad_principals: false auth: active_directory_auth_enabled: true password_auth_enabled: true storage_mb: 32768 storage_tier: "P10" auto_grow_enabled: true azure_extensions: "uuid-ossp,ltree" # Redis baseline redis: public_network_access_enabled: false sku_name: "Premium" family: "P" capacity: 1 active_directory_authentication_enabled: true access_policy_name: "Data Owner" non_ssl_port_enabled: false tls_version: 1.2 redis_version: 6 # Storage account baseline storage_account: public_network_access_enabled: false min_tls_version: "TLS1_2" account_tier: "Standard" account_replication_type: "LRS" allow_nested_items_to_be_public: false # OpenAI — leave client.mode empty to let Terraform infer from open_ai.create open_ai: client: mode: "" key: "" org_id: "" custom_url: "" # NAT Gateway baseline nat_gateway: sku_name: "Standard" public_ip_sku: "Standard" public_ip_allocation_method: "Static" zones: ["1", "2", "3"] idle_timeout_in_minutes: 10 # AKS baseline aks: load_balancer_sku: "standard" only_critical_addons_enabled: true key_vault_secret_rotation_enabled: true # Front Door baseline frontdoor: create: true ssl_mode: managed minimum_tls_version: "TLS12" forwarding_protocol: HttpOnly private_link_service_name: "sgp-ingress-lb" subdomains: ["auth", "api", "@", "admin"] extra_subdomains: [] dns_ttl_seconds: 60 ip_filtering: enabled: false allowed_ip_ranges: [] # System Manager bootstrapping baseline bootstrapping: az_cli_version: "2.59.0" helm_chart_version: "2.1.0" base_repository: "" ``` #### File 3: `config///.yaml` Environment-specific configuration. This is where you set everything that differs per environment: resource names, subscription, CIDRs, feature flags, node pool sizing, and bootstrapping state. ```yaml theme={null} # ── Environment identity ─────────────────────────────────────────────────────── location: eastus2 subscription_id: "" name_suffix: "" # e.g. "acme01" deployment_id: "" deployment_url: ".workspace.egp.scale.com" temporal_db_mode: postgresCommonHosted # Recommended; or "cassandraK8s" (not recommended) # ── Auth / secrets ──────────────────────────────────────────────────────────── authType: default # Options: default, SAML, OIDC RBAC: false registration_secret: "" # Auto-generated if empty identity_service_jwt: "" # Auto-generated if empty # ── OIDC (required only when authType = OIDC) ───────────────────────────────── oidc: clientId: "" clientSecret: "" issuer: "" authorizationUrl: "" tokenUrl: "" userInfoUrl: "" # ── SAML (required only when authType = SAML) ───────────────────────────────── saml: X509Cert: "" SSOUrl: "" emailAttrName: "" firstNameAttrName: "" lastNameAttrName: "" # ── SSL ─────────────────────────────────────────────────────────────────────── sslMode: e2e # Options: e2e, managed # SSL cert/key for Istio ingress (PEM file paths or base64). Required when sslMode=e2e. istio: sslCert: "" sslKey: "" # ── Resource group + networking ─────────────────────────────────────────────── resource_group: "rg-eus2---dev-01" network: vnet: "sgpaz-vnet" int_rt_name: "sgpaz-rt-int" int_nsg_name: "sgpaz-nsg-int" private_cluster: true nsg_rules: postgresql: true redis: true private_dns: create: true private_cluster: true subnets: psql: name: "sgpaz-snet-psql" type: int address_prefixes: - "" # e.g. "10.95.23.160/27" service_endpoints: - Microsoft.Storage service_delegation: name: Microsoft.DBforPostgreSQL/flexibleServers actions: - Microsoft.Network/virtualNetworks/subnets/join/action deployment_scripts: name: "sgpaz-snet-deployscripts" type: int address_prefixes: - "" service_delegation: name: Microsoft.ContainerInstance/containerGroups actions: - Microsoft.Network/virtualNetworks/subnets/action aks: name: "sgpaz-snet-aks" type: int address_prefixes: - "" private_endpoint_network_policies: Disabled service_endpoints: - Microsoft.CognitiveServices bastion: name: AzureBastionSubnet type: int address_prefixes: - "" jump_host: name: "sgpaz-snet-jumphost" type: int address_prefixes: - "" # ── Workload Identity + NAT Gateway ─────────────────────────────────────────── workload_identities: enabled: true nat_gateway: enabled: true name: "sgpaznat" zones: [] # [] = regional (no zone pinning) # ── CMK ─────────────────────────────────────────────────────────────────────── # Set create=true for new deployments. If CMK keys already exist, set create=false # and provide the key URIs to avoid "already exists (import required)" errors. cmk: enabled: true create: true # encryption_key_uri: "https://.vault.azure.net/keys/encryptionCMK/" # k8s_key_uri: "https://.vault.azure.net/keys/k8sEncryptionCMK/" # service_bus_key_uri: "https://.vault.azure.net/keys/serviceBusEncryptionCMK/" # ── Key Vault ───────────────────────────────────────────────────────────────── # Override public access settings here if bootstrapping from a laptop or CI runner. keyvault: name: "sgpazkeyvault" # Must be globally unique; 3-24 characters soft_delete_retention_days: 90 # Must match existing vault if already created public_network_access_enabled: true # Set false once stable; keep true during bootstrap network_acls_ip_rules: - "/32" # Your laptop or CI runner IP # ── Core services ───────────────────────────────────────────────────────────── law: create: true psql: create: true name: "sgpazpostgres" # Must be globally unique admin_user: postgres bootstrap_aad_principals: true # Runs once after cluster is up; requires kubectl access redis: create: true name: "sgpazredis" # Must be globally unique storage_account: create: true name: "sgpazstorage" # Must be globally unique; no hyphens enable_file_private_endpoint: true service_bus: create: true name: "sgpazservicebus" sku: Premium # Premium required for private endpoints capacity: 1 premium_messaging_partitions: 1 private_endpoint_enabled: true public_network_access: false # ── AI services ─────────────────────────────────────────────────────────────── # Modes: use_openai_via_azure | use_openai_via_custom_endpoint | no_openai # NOTE: Do not commit real API keys. Keep `key` as a local uncommitted change. open_ai: create: false name: "sgpazopenai" client: mode: "use_openai_via_custom_endpoint" key: "" custom_url: "" ai_search: create: false name: "sgpazaisearch" # ── AKS ─────────────────────────────────────────────────────────────────────── aks: create: true name: "sgpazaks" node_resource_group: "rg-aks-eus2---dev-01" dns_prefix: "egp-k8scluster" sku: Free # Free | Standard (use Standard for production) kubernetes_version: "1.36" run_command_enabled: true enable_encryption_at_host: true use_azure_managed_flux: true zones: ["1", "2", "3"] node_pools: system: name: default vm_size: Standard_D4s_v3 count: 3 min_count: 3 max_count: 6 user: enabled: true name: user vm_size: Standard_D16s_v3 count: 3 min_count: 3 max_count: 10 gpu: enabled: false # Set true to enable GPU workloads name: gpu vm_size: Standard_NV72ads_A10_v5 min_count: 0 max_count: 5 cassandra: enabled: false # Required only if temporal_db_mode is cassandraK8s network: pod_cidr: "10.244.0.0/16" service_cidr: "10.243.0.0/16" dns_service_ip: "10.243.0.10" outbound_type: userDefinedRouting # ── Bastion / jump host ─────────────────────────────────────────────────────── bastion: create: true admin_username: azureuser ssh_public_key: "" # ── Front Door overrides ────────────────────────────────────────────────────── # Most Front Door settings inherit from default.yaml. Override only what differs. frontdoor: extra_subdomains: [] # e.g. ["chat"] if Agentex UI is enabled # ── Feature flags ───────────────────────────────────────────────────────────── feature_flags: models: false agentex: create: false # Set true to provision Agentex service compass: create: false # Set true to provision Workflows service dex: create: false # Set true to provision Dex (Document Understanding) service reducto: create: false # Set true to provision Reducto service # ── Bootstrapping (System Manager) ──────────────────────────────────────────── bootstrapping: enabled: true use_managed_flux: true system_manager_version: "" # Required when enabled is true; Terraform has no default pack list. Get the list for your install from your Scale team desiredState: | { "version": "0.1", "packs": [ { "name": "" } ] } # ── Policy assignments ──────────────────────────────────────────────────────── policy_assignments: enabled: false # Set true to enforce Azure Policy tag rules # ── Observability ───────────────────────────────────────────────────────────── observiqidp: monitoring: enabled: false insights: enabled: false ``` > [!NOTE] > > **Naming constraints for Azure resources:** > > * Key Vault names: 3–24 alphanumeric characters and hyphens, globally unique > * Storage Account names: 3–24 lowercase letters and numbers only (no hyphens), globally unique > * PostgreSQL and Redis names: globally unique within Azure > * All names must remain stable after first apply — many Azure resources cannot be renamed #### SAML Configuration To sign in with SAML, set `authType: "SAML"` in your environment YAML, then configure your Identity Provider with: * **Service Entity ID**: `https://auth.` * **Redirect URI**: `https://auth./dashboard/org/saml/callback` Put the values from your Identity Provider in the `saml` block. Terraform sets the entity ID to `https://auth.`. ```yaml theme={null} saml: X509Cert: "" SSOUrl: "" emailAttrName: "" firstNameAttrName: "" lastNameAttrName: "" ``` #### OIDC Configuration To sign in with OIDC, set `authType: "OIDC"` in your environment YAML, then configure your Identity Provider with: * **Redirect URI**: `https://auth./dashboard/org/oidc/callback` Put the values from your Identity Provider in the `oidc` block. `TF_VAR_oidc_client_secret`, when set, takes precedence over `oidc.clientSecret`, which keeps the client secret out of the YAML. ```yaml theme={null} oidc: clientId: "" clientSecret: "" issuer: "" authorizationUrl: "" tokenUrl: "" userInfoUrl: "" ``` To change these settings after the first install, update the YAML, run `terraform apply`, and restart System Manager: ```bash theme={null} kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager ``` ### Step 2: Provision Infrastructure via Terraform Navigate to the `infra` directory, initialize, and set the control plane ID and registration token from Scale: ```bash theme={null} cd infra terraform init # Read the values without echoing them or saving them in shell history printf 'Control plane ID: ' && read -rs TF_VAR_control_plane_id && echo && export TF_VAR_control_plane_id printf 'Registration token: ' && read -rs TF_VAR_registration_token && echo && export TF_VAR_registration_token ``` Review and apply the plan using the `main.tfvars.json` file you configured in the previous step: ```bash theme={null} terraform plan -var-file=main.tfvars.json -out=tfplan # Inspect the plan to review resources before proceeding terraform show tfplan terraform apply tfplan ``` If the Datadog monitoring integration is enabled in your YAML, pass the API key via environment variable rather than committing it to file: ```bash theme={null} printf 'Datadog API key: ' && read -rs TF_VAR_datadog_api_key && echo && export TF_VAR_datadog_api_key terraform plan -var-file=main.tfvars.json -out=tfplan ``` *This step may take significant time (30–60 minutes) due to resource creation dependencies, particularly the AKS cluster and PostgreSQL Flexible Server.* ### Step 3: Bootstrap the Cluster The Azure infrastructure **automatically bootstraps** [SGP System Manager](/docs/infrastructure/system-manager) via an Azure Deployment Script. When `bootstrapping.enabled: true` is set in your configuration, Terraform provisions an Azure Container Instance that: 1. Installs Flux CD on the AKS cluster (using Azure Managed Flux if `use_managed_flux: true`) 2. Applies the System Manager `HelmRepository` and `HelmRelease` Flux CRDs 3. Waits for System Manager to reconcile Monitor bootstrap progress in the Azure Portal under **Deployment Scripts** in your resource group, or check System Manager logs after bootstrap: ```bash theme={null} # Get AKS credentials (from inside the VNet or via Bastion — cluster is private by default) az aks get-credentials \ --resource-group "" \ --name "sgpazaks" \ --overwrite-existing # Verify System Manager is running kubectl get pods -n sgp-system-manager # Watch System Manager deploy remaining services via Flux kubectl get helmreleases -A ``` #### Accessing the Private AKS Cluster Because the AKS cluster is private by default, you must access it from within the provisioned VNet. Two options are provided: **Option A: From the jump host VM (via Azure Bastion)** A Bastion host is provisioned when `bastion.create: true`. Connect via the Azure Portal (Bastion blade). **Option B: Using `az aks command invoke`** When `aks.run_command_enabled: true`, you can run kubectl commands without VPN access: ```bash theme={null} az aks command invoke \ --resource-group "" \ --name "sgpazaks" \ --command "kubectl get helmreleases -A" ``` ### Step 4: Configure DNS After `terraform apply` completes, retrieve the Front Door endpoint hostname: ```bash theme={null} terraform output frontdoor_endpoint_host_name ``` Configure a CNAME record in your DNS provider pointing your `deployment_url` to the Front Door endpoint (the `azurefd.net` hostname from the output above). > [!NOTE] > > If using Azure DNS (the DNS zone is managed by Terraform when `frontdoor.create: true`), CNAME records are created automatically. Verify with: > > ```bash theme={null} > az network dns record-set list \ > --resource-group "" \ > --zone-name "" > ``` ### Step 5: Verify the Deployment Wait for all services to be ready: ```bash theme={null} kubectl get helmreleases -A # All should show Ready=True kubectl get pods -A # All should be Running or Completed ``` Terraform sets the `system-manager-desired-state` Key Vault secret from `bootstrapping.desiredState` only when the secret is first created; later YAML edits have no effect. After the first install, change the desired state in the System Manager GUI, API, or Key Vault, then install the affected packs. Saving the secret alone does not apply changes. By default, restarting System Manager reinstalls every pack in the desired state; see [System Manager configuration](/docs/infrastructure/system-manager#configuration). To change startup behavior on Azure, contact your Scale deployment team. If a HelmRelease shows `Ready=False`, check its events: ```bash theme={null} kubectl describe helmrelease "" -n "" ``` ## Accessing the Platform If all goes smoothly, you should be able to navigate to the SGP platform at `https://.workspace.egp.scale.com` (or your custom domain) and authenticate via the configured identity provider. # GCP Architecture Reference Source: https://docs.gp.scale.com/docs/infrastructure/gcp/gcp-architecture-reference Overview of the SGP GCP architecture ## Architecture Diagram ```mermaid theme={null} graph TD USER(["Client"]) ENGINEER(["Engineer"]) subgraph GCP["GCP Project"] GLB["Global Load Balancer
Static IP · TLS 1.2+"] IAP["Cloud IAP"] subgraph VPC["VPC — 10.0.0.0/16"] INGRESS["Istio Ingress"] subgraph GKE["GKE Cluster"] SYS["System Pool
3× n2-standard-4"] CPU["CPU Pool
4–30× n2-standard-16"] end BASTION["Bastion VM
e2-micro"] end subgraph PEERED["VPC-Peered — Cloud SQL"] PG_MAIN[("Cloud SQL
PostgreSQL 17 (main)")] PG_TEMP[("Cloud SQL
PostgreSQL 17 (temporal)")] end SM["Secret Manager"] subgraph GCS_GROUP["Cloud Storage"] BUCKET[("GCS — platform")] KB[("GCS — knowledge base")] end AR["Artifact Registry
Docker + Helm"] KMS["Cloud KMS
CMEK (optional)"] end USER --> GLB GLB --> INGRESS INGRESS --> CPU CPU -->|"VPC Peering · SSL enforced"| PG_MAIN CPU -->|"VPC Peering · SSL enforced"| PG_TEMP CPU -->|"Workload Identity"| SM CPU -->|"Workload Identity"| BUCKET CPU -->|"Workload Identity"| KB AR -->|"Workload Identity"| CPU KMS -.->|"encrypts"| PG_MAIN KMS -.->|"encrypts"| PG_TEMP KMS -.->|"encrypts"| BUCKET ENGINEER --> IAP --> BASTION -->|"private API"| GKE ``` *** ## Terraform Structure SGP's GCP infrastructure is provisioned in two separate Terraform phases with different privilege levels: ```mermaid theme={null} graph LR CREDS(["Personal GCP
Credentials"]) subgraph PS["projectsetup/ — run once with personal credentials"] SA["Terraform
Service Account"] AR_PS["Artifact Registries
Docker + Helm"] SMK["SA Key stored in
Secret Manager"] end subgraph DEP["deployments/name/ — SA credentials auto-loaded from Secret Manager"] GKE_D["GKE Cluster"] SQL_D["Cloud SQL × 2"] VPC_D["VPC + Subnets"] SEC_D["Secrets + IAM
+ Workload Identity"] end CREDS -->|"terraform apply"| PS SMK -->|"credentials
auto-read at plan time"| DEP ``` | Phase | Directory | Credentials | Scope | | - | - | - | - | | Privileged bootstrap | `projectsetup/` | Your personal GCP identity | Service account, Artifact Registries, API enablement | | Dedicated execution identity | `deployments//` | Terraform service account (read from Secret Manager) | GKE, Cloud SQL, networking, IAM, secrets | The main infrastructure Terraform uses a dedicated service account with broad administration roles, including compute, GKE, Cloud SQL, Secret Manager, project IAM, service account keys, Artifact Registry, and Storage administration. Configure the deployment to read the key secret created by `projectsetup/`, as described in the [deployment guide](/docs/infrastructure/gcp/gcp-sgp-deployment#step-1-bootstrap-the-project); then no key file needs to exist on disk during the infrastructure run. *** ## Resources by Type ### Compute Resources | Resource | Count | Purpose | | - | - | - | | GKE Cluster | 1 | Kubernetes orchestration | | System Node Pool (`default`) | 1 | System pods — tainted `CriticalAddonsOnly` | | CPU Node Pool (`cpu`) | 1 | Application workloads | | GPU Node Pools (`gpu-`) | 0–N | AI/ML workloads, one pool per `gpu_node_pools` entry (none by default) | | Cassandra Node Pool | 0–1 | Temporal database (optional) | | Bastion Host | 0–1 | Private cluster access via IAP (optional) | **Default node pool sizing:** | Pool | Machine Type | Min Nodes | Max Nodes | | - | - | - | - | | System (`default`) | `n2-standard-4` | 3 | 10 | | CPU (`cpu`) | `n2-standard-16` | 4 | 30 | | GPU (`gpu-`) | Set per `gpu_node_pools` entry | 0 | Set per entry (no GPU pools by default) | | Cassandra | `n2-standard-4` | 3 | 6 | General-purpose pools use `COS_CONTAINERD` and are non-preemptible by default (`node_pool_config.preemptible=false`). The option can enable preemptible nodes for pools that honor it, including the `gvisor` and `gvisor_allow_all` sandbox pools. The KubeVirt, Gitea Actions, ClickHouse, OpenSearch, and Temporal system pools are always non-preemptible. *** ### Network Resources | Resource | Count | Purpose | | - | - | - | | VPC Network | 1 | Network boundary | | Subnetwork | 1 | Kubernetes nodes | | Secondary IP Ranges | 2 | GKE pod and service CIDRs | | Cloud DNS Zone | 1 | Internal + external name resolution | | Global Static IP | 1 | Load balancer ingress endpoint | | SSL Policy | 1 | Minimum TLS 1.2 enforcement | | Firewall Rules | 4–5 | Traffic control (offline mode) | | VPC Peering | 1 | Private connectivity to Cloud SQL | | Private Route | 0–1 | Google APIs access in offline mode | *** ### Data & Storage Resources | Resource | Count | Purpose | | - | - | - | | Cloud SQL (PostgreSQL 17) | 1 | Main platform database | | Cloud SQL (PostgreSQL 17) | 1 | Temporal workflow database | | GCS Bucket (main) | 1 | Platform object storage | | GCS Bucket (knowledge base) | 1 | KB document storage | | GCS Bucket (monitoring) | 0–1 | Observability data (optional) | | Vertex AI Vector Search Index | 0–N | Vector embeddings (optional) | | Cloud Firestore | 0–1 | Agentex state (optional) | Both Cloud SQL instances are private-only (no public IP) and connected to the VPC via VPC peering. SSL is enforced for all database connections (`ENCRYPTED_ONLY`). *** ### Security Resources | Resource | Count | Purpose | | - | - | - | | Secret Manager Secrets | 10+ | Platform configuration and credentials | | Cloud KMS Key Ring + Key | 0–1 | Customer Managed Encryption Keys (optional) | | Service Account (`sgp--sa`) | 1 | Main SGP workload identity SA | | Service Account (`sgp--node-pool-creator`) | 1 | Node pool creation SA | | Service Account (`sgp-tf-lp-`) | 1 | Terraform execution SA (projectsetup) | | Workload Identity Pool | 1 | GKE pod → GCP SA binding | | IAP Tunnel (Bastion) | 0–1 | Private cluster access | *** ### Artifact Resources (projectsetup phase) | Resource | Count | Purpose | | - | - | - | | Artifact Registry (Docker) | 1 | SGP container images | | Artifact Registry (Helm) | 1 | SGP Helm charts | Both registries are named `sgp--docker-repository` and `sgp--helm-repository`. *** ### Monitoring Resources | Resource | Count | Purpose | | - | - | - | | Cloud Logging | 1 | GKE system components, API server, workloads | | VPC Flow Logs | 0–1 | Network traffic sampling (optional) | | GKE Vulnerability Scanning | 1 | Basic vulnerability mode enabled by default | *** ## Network Architecture ### Address Space ``` VPC: sgp--network └── Subnet: sgp--network-kubernetes-subnet ├── Primary range: 10.0.0.0/16 (65,536 IPs — GKE nodes) ├── Secondary range: 10.2.0.0/16 (65,536 IPs — Kubernetes services) └── Secondary range: 10.4.0.0/16 (65,536 IPs — Kubernetes pods) Cloud SQL VPC Peering: └── Reserved range: /16 block (managed by service networking) GKE Control Plane (private): └── Master CIDR: 10.5.0.0/28 (configurable via private_gke_master_ipv4_cidr_block) ``` ### Traffic Flow **Ingress (External):** ``` Internet → Global Load Balancer (static IP) → Istio Ingress → Pod (TLS terminated at load balancer; SSL policy enforces TLS 1.2+) ``` **Database Access:** ``` GKE Pod → VPC Peering → Cloud SQL (private IP) (SSL enforced; no public IP on Cloud SQL instances) ``` **Google APIs (offline mode):** ``` GKE Pod → Private Google Access → Google APIs (199.36.153.4/30) (Dedicated route; no default internet gateway route created) ``` **Bastion Access:** ``` Engineer → IAP Tunnel → Bastion VM (e2-micro) → GKE API (private endpoint) ``` *** ## GKE Configuration ### Cluster Features | Feature | Value | | - | - | | Datapath provider | `ADVANCED_DATAPATH` (eBPF-based) | | IP stack | Dual-stack IPv4/IPv6 | | Workload Identity | Enabled (`.svc.id.goog`) | | Secret Manager integration | Enabled | | Vulnerability scanning | Basic mode | | Private nodes | Enabled (when `offline_mode = true`) | | Private endpoint | Configurable (`enable_gke_private_endpoint`) | | Master authorized networks | Configurable per deployment | | DNS endpoint | Enabled (allows external cluster DNS access) | ### Workload Identity GKE pods authenticate to GCP services using Workload Identity rather than node-level service account keys. Kubernetes service accounts are bound to GCP service accounts via the workload identity pool: ``` .svc.id.goog[/] → GCP Service Account → Secret Manager, GCS, Artifact Registry, etc. ``` Key bindings provisioned by Terraform: | Kubernetes Identity | GCP Role | | - | - | | `sgp-system-manager` (system-manager namespace) | `secretmanager.viewer`, `secretmanager.secretAccessor` | | `sgp-system-manager-pre-install` | `secretmanager.viewer`, `secretmanager.secretAccessor` | | `egp-api-backend` (egp namespace) | `secretmanager.secretAccessor`, GCS access via SA impersonation | | `egp-api-backend-db-setup` | `secretmanager.secretAccessor` | | `agents` (agents namespace) | `secretmanager.secretAccessor` | *** ## Secret Manager Secrets Key secrets provisioned by Terraform: | Secret Name | Contents | | - | - | | `-system-manager-config` | System Manager runtime configuration (URLs, workspace ID, cloud provider info) | | `-system-manager-desired-state` | Desired state (pack list). Terraform creates it empty; on first start, System Manager fills it from `desired_state.json` and manages it afterwards | | `-identity-service-saml-secret` | SAML IdP configuration | | `-identity-service-oidc-secret` | OIDC IdP configuration | | `terraform-service-account-key-secret-2` | Terraform SA private key created by `projectsetup/`. Your deployment reads it as provider credentials after you set this name in [Step 1 of the deployment guide](/docs/infrastructure/gcp/gcp-sgp-deployment#step-1-bootstrap-the-project) | The `secret_name_prefix` variable (typically `sgp-`) scopes secrets for deployments in shared GCP projects. *** ## Customer Managed Encryption Keys (CMEK) When `useCustomerManagedEncryptionKey = true`, Terraform provisions a Cloud KMS key ring and symmetric encryption key: | Resource | Name | | - | - | | Key Ring | `sgp--key-ring` | | Crypto Key | `sgp--crypto-key` | The key is applied to: * Cloud SQL instances (both main and Temporal) * GCS buckets CMEK is recommended for production deployments to maintain cryptographic control over data at rest. *** ## Optional Capabilities Each optional capability provisions dedicated infrastructure: | Capability | Variable | Additional Resources | | - | - | - | | Agentex | `deployAgentex` (default `true`) | Firestore, dedicated service account, GCS bucket | | Workflows | `deployCompass` | Cloud SQL database, GCS bucket, service account | | Dex (Document Understanding) | `enable_dex` | Cloud SQL (`db-custom-4-15360`), GCS bucket, service account | | Reducto | `enable_reducto` | Cloud SQL (`db-custom-2-7680`), GCS bucket, Vision API key, service account | | Model Engine | `deployModelEngine` (module default `true`; `deployments/prototype` sets `false`) | Vertex AI, additional node pools | | Monitoring | `enable_monitoring` | GCS bucket, IAM bindings | | Cloud Build | `enable_cloud_build` | GCS bucket, Artifact Registry, service accounts | | Vertex AI Search | `vertex_ai_indices` | Vector Search indices and endpoints | | LiveKit | `provision_livekit_infrastructure` | GCS bucket (audio), IAM bindings | # GCP SGP Deployment Source: https://docs.gp.scale.com/docs/infrastructure/gcp/gcp-sgp-deployment End-to-end guide to deploying SGP in your GCP project, together with Scale engineering. ## Overview This guide covers deploying SGP in a GCP project using the SGP GCP Terraform modules. SGP GCP infrastructure is defined by terraform modules managed by Scale. > [!NOTE] > > SGP is deployed jointly with Scale engineering. Scale provides everything under "The following from Scale" in Prerequisites and stays involved for upgrades and ongoing operation. Contact your Scale team before you start. GCP deployment separates the bootstrap identity from the Terraform execution identity: * **`projectsetup/`** — Run once with privileged GCP credentials. Creates a dedicated Terraform service account with broad project administration roles, enables required GCP APIs, and provisions Artifact Registries for SGP images and Helm charts. * **`deployments//`** — Run with the service account created above. Provisions the GKE cluster, Cloud SQL, networking, and all SGP infrastructure. The execution identity still has broad privileges, including `roles/compute.admin`, `roles/container.admin`, `roles/cloudsql.admin`, `roles/secretmanager.admin`, `roles/resourcemanager.projectIamAdmin`, `roles/iam.serviceAccountKeyAdmin`, `roles/artifactregistry.admin`, and `roles/storage.admin`. Review the complete role list in the provided `projectsetup/main.tf` before granting access. ## Prerequisites * Access to a GCP project with permissions to manage IAM, APIs, and compute resources * The following tools installed: * [gcloud CLI](https://cloud.google.com/sdk/docs/install) (`gcloud auth login` completed) * [Terraform](https://www.terraform.io/) (`>= 1.10.0`) * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) * [helm](https://helm.sh/docs/intro/install/) (for cluster bootstrap) * [crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane) or [Docker](https://docs.docker.com/engine/install/) (for copying images) * [sgpctl](https://github.com/scaleapi/sgpctl) (cli tool to execute system manager related commands) * The following from Scale: * The SGP GCP Infrastructure modules (`gcp/`) * A `workspace_id` (8-digit number) and `registration_secret` unique to your deployment * A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional) * A custom domain for your deployment (optional) ## Installation ### Step 1: Bootstrap the Project Navigate to the `projectsetup/` directory. This phase creates the Terraform service account and Artifact Registries. Edit the `locals` block at the top of `projectsetup/main.tf`: ```hcl theme={null} locals { project = "" region = "" # e.g. "us-east1" zone = "" # e.g. "us-east1-a" service_account_name = "-sa" # Suffix for the service account name workspaceID = "" # 8-digit number from Scale, e.g. "90000001" } ``` Then initialize and apply: ```bash theme={null} cd projectsetup terraform init terraform apply ``` This creates: * A Terraform service account (`sgp-tf-lp-`) with the IAM roles needed to provision SGP infrastructure * A service account key stored as `terraform-service-account-key-secret-2` in Secret Manager * Docker and Helm Artifact Registries (`sgp--docker-repository`, `sgp--helm-repository`) * All required GCP APIs enabled on the project The Terraform outputs include the `service_account_email` of the newly created service account, which is useful for auditing. The project bootstrap stores the key in `terraform-service-account-key-secret-2`. Before initializing your copied deployment, set the `service-account-private-key` data source to read that secret: ```hcl theme={null} data "google_secret_manager_secret" "service-account-private-key" { secret_id = "terraform-service-account-key-secret-2" provider = google.google-service-account-private-key-secret-pull-provider } ``` Terraform then reads the key from Secret Manager, so you don't need a local key file. ### Step 2: Copy SGP Images and Helm Charts *This step is only required if your deployment is configured to block internet access from the cluster (`offline_mode` in step 3).* Before provisioning the main infrastructure, copy SGP's Docker images and Helm charts from Scale's registry into your GCP Artifact Registry. **Download the manifest from Scale's control plane**: ```bash theme={null} sgpctl manifest download \ --control-plane-id "" \ --registration-token "" \ --version "" \ --channel \ --output ".manifest.yaml" ``` Note: sgpctl manifest download requires internet access to [https://control.egp.scale.com](https://control.egp.scale.com), so it must be run from a connected machine. Users with air-gapped operator workstations should run it from their laptop and SCP the output to wherever they'll run the mirror commands. **Authenticate to source + destination registries**: ```bash theme={null} # Source: Scale's ECR (credentials from Scale engineer) crane auth login 022465994601.dkr.ecr.us-west-2.amazonaws.com \ --username AWS \ --password-stdin # Destination: customer's GCP Artifact Registry (uses gcloud creds) gcloud auth configure-docker "-docker.pkg.dev" ``` **Mirror images and charts**: ```bash theme={null} # Container images sgpctl mirror-manifest \ --manifest-file ".manifest.yaml" \ --dest-docker-registry "-docker.pkg.dev//sgp--docker-repository" # Helm charts sgpctl mirror-helm-charts \ --manifest-file ".manifest.yaml" \ --dest-helm-repo "oci://-docker.pkg.dev//sgp--helm-repository" ``` **Verify the mirror**: ```bash theme={null} gcloud artifacts docker images list \ "-docker.pkg.dev//sgp--docker-repository" \ --include-tags --limit=20 gcloud artifacts docker images list \ "-docker.pkg.dev//sgp--helm-repository" \ --include-tags --limit=20 ``` ### Step 3: Configure the Deployment Copy the `deployments/prototype/` directory and rename it for your deployment: ```bash theme={null} cp -r deployments/prototype "deployments/" cd "deployments/" ``` Edit the `locals` block at the top of `main.tf`. After you set the secret name in Step 1, the deployment reads its Terraform service account credentials from Secret Manager; no key file is required on disk. ```hcl theme={null} locals { project = "" region = "" # e.g. "us-east1" zone = "" # e.g. "us-east1-b" # ── Deployment identity ──────────────────────────────────────────────── workspaceID = "" # 8-digit number from Scale registrationSecret = "" # From Scale sgp_account_id = "" # Or the 24-character SGP account ID from Scale deploymentURL = ".workspace.egp.scale.com" # Or your custom domain # ── Repositories ────────────────────────────────────────────────────── # baseRepository: used by workloads running inside the cluster to pull images # publicBaseRepository: used for initial image pulls during bootstrap (before private DNS resolves) # Both point to the same registry when using a private Artifact Registry. baseRepository = "-docker.pkg.dev//sgp--docker-repository" publicBaseRepository = "-docker.pkg.dev//sgp--docker-repository" # ── System Manager ──────────────────────────────────────────────────── systemManagerImageTag = "" # From Scale; must match manifest.yaml offline_mode = true # true when using your own Artifact Registry # ── Auth ────────────────────────────────────────────────────────────── authType = "SAML" # "SAML" or "OIDC" # ── Bootstrap ───────────────────────────────────────────────────────── # deploy_system_manager=false: Terraform creates infra only; run manual-helm-install.sh to bootstrap # deploy_system_manager=true: Terraform also installs System Manager into the cluster deploy_system_manager = false # ── DNS ─────────────────────────────────────────────────────────────── createDNSRecords = false # Set true if Terraform should manage Cloud DNS records # ── Encryption ──────────────────────────────────────────────────────── useCustomerManagedEncryptionKey = false # Set true for CMEK (recommended for production) aes256EncryptionKey = "" # Required when CMEK is off: 32 characters, from `openssl rand -hex 16` # ── Networking ──────────────────────────────────────────────────────── gke_config = { private_endpoint = true master_authorized_networks = [ { cidr_block = "10.0.0.0/16", display_name = "cluster vpc primary range" }, { cidr_block = "10.2.0.0/16", display_name = "cluster vpc services range" }, { cidr_block = "10.4.0.0/16", display_name = "cluster vpc pod range" }, ] } # ── Bastion ─────────────────────────────────────────────────────────── security_compliance = { # ... labels, logging and firewall settings from the prototype bastion_enabled = true bastion_iap_members = ["user:"] # Accounts that reach the private cluster through the bastion } } ``` Don't change `aes256EncryptionKey` on a running deployment: data encrypted with the old key becomes unreadable. **If using SAML,** place your IdP's x509 certificate (without BEGIN/END lines) as `x509.cer` in the deployment directory. Update the `samlConfigSecret` in the `module "sgp"` block: ```hcl theme={null} module "sgp" { source = "../../modules/sgp/" samlConfigSecret = jsonencode({ "id" = local.workspaceID "samlConfiguration" = { "entityId" = "https://auth.${local.deploymentURL}" "x509Cert" = file("${path.module}/x509.cer") "ssoUrl" = "" "attributeMappings" = { "email" = "" "firstName" = "" "lastName" = "" } } }) # ... other variables } ``` **If using OIDC,** update `oidcConfigSecret` instead: ```hcl theme={null} oidcConfigSecret = jsonencode({ "id" = local.workspaceID "clientId" = "" "clientSecret" = "" "issuer" = "" "authorizationUrl" = "" "tokenUrl" = "" "userInfoUrl" = "" }) ``` #### Desired State The `desired_state.json` file in your deployment directory defines which SGP packs System Manager installs. `terraform apply` copies it into the System Manager configuration, and System Manager reads it only on first install, while its desired state is empty. Finish it, including the certificate setting from Step 4, before you run Terraform in Step 5. After the first install, editing the local file has no effect; update the live desired state in the System Manager GUI, API, or Secret Manager instead, then install the affected packs. Update the file to reference your GCP Artifact Registry: ```json theme={null} { "version": "0.1", "packs": [ { "name": "flux" }, { "name": "sgp-helm-repository", "properties": { "helm-repo": { "url": "oci://-docker.pkg.dev//sgp--helm-repository" } } }, { "name": "istio" }, { "name": "sgp-base", "properties": { "helm-base": { "value_overrides": { "refreshRegcred": { "enabled": false }, "gcp": { "presharedCertificates": ["sgp--ssl-certificate"] } } } } }, { "name": "spicedb" }, { "name": "identity-service" }, { "name": "temporalf" }, { "name": "sgp-apps" } ] } ``` The `gcp` block above uses a preshared certificate. [Step 4](#step-4-configure-ssl-certificates) covers that option and the other two. ### Step 4: Configure SSL Certificates The options are listed in order of preference. Choose one before you run Terraform in Step 5: | Option | `main.tf` | `gcp` block of the `sgp-base` pack in `desired_state.json` | Certificate | | - | - | - | - | | 1. Google Certificate Manager (recommended) | `enable_certificate_manager = true` and `enable_gateway_api = true` | Remove it | A wildcard certificate for all subdomains, validated through DNS | | 2. Google-managed certificate | Neither flag | Remove it | Per-subdomain certificates that Google renews, for ``, `api.`, `auth.` and `app.` only. DNS must resolve to the load balancer first, and wildcards aren't supported. They don't cover Agentex's `chat.` and `agentex.` hosts, so use Option 1 when Agentex is on | | 3. Preshared certificate | Neither flag | `presharedCertificates`, naming your certificate | A certificate you upload and renew. Use it when DNS isn't publicly resolvable or Certificate Manager isn't available | For each option, edit the existing `sgp-base` entry and preserve its other properties. When you switch certificate options, replace the existing `presharedCertificates` list instead of adding to it. Before switching to a managed certificate, make sure every `presharedCertificates` list in your configuration is empty. #### Option 1: Google Certificate Manager (Recommended) Set both flags in `main.tf`, and add an output for the DNS authorization record, which the prototype doesn't export. Terraform passes the certificate map to the `sgp-base` pack: ```hcl theme={null} module "sgp" { source = "../../modules/sgp/" enable_certificate_manager = true enable_gateway_api = true # ... } output "certificate_manager_dns_auth_record" { value = module.sgp.certificate_manager_dns_auth_record } ``` After `terraform apply` in Step 5, run `terraform output certificate_manager_dns_auth_record` and add that CNAME to your DNS zone for domain validation. The certificate typically provisions within 10–60 minutes of the DNS record being in place. Terraform passes the Gateway API setting and certificate-map name to the `sgp-base` pack as defaults, and desired-state `value_overrides` take precedence over them. If you set overrides explicitly, merge the following `gcp` values into the existing `sgp-base` entry, preserving its other properties, and install that pack. A certificate map requires the Gateway template: ```json theme={null} { "name": "sgp-base", "properties": { "helm-base": { "value_overrides": { "gcp": { "gatewayApi": { "enabled": true }, "certificateMapName": "sgp--cert-map", "presharedCertificates": [] } } } } } ``` #### Option 2: Google Managed Certificate Google provisions the certificates after DNS resolves to the load balancer IP in Step 7. This is the traditional GCE Ingress default when `gcp.gatewayApi.enabled` is `false` and `presharedCertificates` is empty. To switch from another certificate option, merge these values into the existing `sgp-base` entry and install the pack: ```json theme={null} { "gcp": { "gatewayApi": { "enabled": false }, "certificateMapName": "", "presharedCertificates": [] } } ``` Place this object under `properties.helm-base.value_overrides`, preserving the entry's other properties. These values override the Gateway settings that Terraform provides. Make sure every `presharedCertificates` list in your configuration is empty. Omitting a key does not clear a Terraform-provided default. #### Option 3: Preshared Certificate Upload the certificate: ```bash theme={null} gcloud compute ssl-certificates create "sgp--ssl-certificate" \ --certificate=fullchain1.pem \ --private-key=privkey1.pem \ --project "" ``` For traditional GCE Ingress, merge these values into the existing `sgp-base` entry, keeping its other properties. Before the first install, edit `desired_state.json`; after it, edit the live desired state. These values disable Gateway API and select the preshared certificate: ```json theme={null} { "name": "sgp-base", "properties": { "helm-base": { "value_overrides": { "gcp": { "gatewayApi": { "enabled": false }, "certificateMapName": "", "presharedCertificates": ["sgp--ssl-certificate"] } } } } } ``` > [!WARNING] > > Preshared certificates must be renewed manually. Let's Encrypt certificates expire after 90 days. Edits to `desired_state.json` after the first install have no effect. To change the SSL configuration then: * Edit the `sgp-base` pack's value overrides in the System Manager GUI or API. An API update replaces the pack's whole `helm-base` object, so send all of its value overrides, including `refreshRegcred`. * To move to Option 1, also set its two flags in `main.tf` and run `terraform apply`. Then explicitly install `sgp-base` in the System Manager GUI to apply the change. With the default startup configuration, restarting System Manager also applies the live state by reinstalling all packs: ```bash theme={null} kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager ``` ### Step 5: Provision Infrastructure via Terraform From the deployment directory, initialize and apply: ```bash theme={null} terraform init -upgrade terraform plan -out=tfplan # Inspect the plan before proceeding terraform show tfplan terraform apply tfplan ``` *This step can take 30–60 minutes due to GKE cluster provisioning and Cloud SQL setup.* After apply completes, connect to the cluster: ```bash theme={null} gcloud container clusters get-credentials "sgp--kubernetes-cluster" \ --region "" \ --project "" ``` > [!NOTE] > > If `gke_config.private_endpoint = true`, the cluster API server is only accessible from within the VPC. Use the bastion host (via IAP, as one of the `bastion_iap_members` from Step 3) or a network-connected runner to access it. ### Step 6: Bootstrap the Cluster [SGP System Manager](/docs/infrastructure/system-manager) orchestrates SGP service deployment. If you set `deploy_system_manager = false` in Step 3, bootstrap it manually using the provided script. Edit `manual-helm-install.sh` in your deployment directory with the correct values: ```bash theme={null} PROJECT_ID="" PROJECT_NUMBER="" # From: gcloud projects describe "" --format='value(projectNumber)' BASE_REPOSITORY="-docker.pkg.dev//sgp--docker-repository" SYSTEM_MANAGER_IMAGE_TAG="" OFFLINE_MODE=true # Obtain from Cloud SQL instance → "Connect to this instance" in the console, or from Terraform state SPICEDB_IP_ADDRESS="" SPICEDB_PASSWORD="" TEMPORAL_DB_PASSWORD="" ``` Then run the bootstrap script: ```bash theme={null} chmod +x manual-helm-install.sh ./manual-helm-install.sh ``` This installs System Manager into the cluster. On its first start, System Manager deploys the packs listed in `desired_state.json`. Startup installation is enabled by default, so each startup installs all packs in the live desired state. If startup installation is disabled, install the packs in the System Manager GUI. See [System Manager configuration](/docs/infrastructure/system-manager#configuration) for the chart-version requirement. ### Step 7: Configure DNS Get the external IP address of the load balancer: ```bash theme={null} # From inside the cluster: kubectl get ingress external-ingress --namespace istio-ingress -o jsonpath='{.status.loadBalancer.ingress[0].ip}' # With enable_gateway_api = true, the address is on the Gateway instead: kubectl get gateway external-gateway --namespace istio-ingress -o jsonpath='{.status.addresses[0].value}' # Or from gcloud (the global address is created by Terraform): gcloud compute addresses describe "sgp--external-ip" \ --global \ --project "" \ --format="value(address)" ``` Create DNS A records in your DNS provider pointing to this IP for: * `` (apex) * `api.` * `auth.` * `app.` If `createDNSRecords = true` in your `main.tf` locals, Terraform manages a Cloud DNS zone and creates these records automatically. Retrieve the name servers from the Terraform output: ```bash theme={null} terraform output name_servers ``` Then delegate the zone by configuring these as NS records at your domain registrar. ### Step 8: Verify the Deployment Wait for all services to be ready: ```bash theme={null} kubectl get helmreleases -A # All should show Ready=True kubectl get pods -A # All should be Running or Completed ``` System Manager renders installed packs into HelmReleases, which Flux then reconciles. After you change the desired state, install the affected packs to apply it. If a HelmRelease shows `Ready=False`, check its events: ```bash theme={null} kubectl describe helmrelease "" -n "" ``` ## Accessing the Platform Once all HelmReleases are ready and DNS resolves correctly, navigate to `https://.workspace.egp.scale.com` (or your custom domain) and authenticate via your configured identity provider. # System Manager Source: https://docs.gp.scale.com/docs/infrastructure/system-manager Overview of the SGP System Manager deployment orchestrator ## Overview System Manager is a Kubernetes operator used to deploy and manage the SGP platform. It is responsible for deploying the SGP platform services and agents to an existing Kubernetes cluster. > [!NOTE] > > System Manager configuration changes and SGP upgrades are made together with Scale engineering. Contact your Scale team before changing the configuration below. You can deploy your own agents without Scale; see [Cloud Build and Deploy](/docs/v5/agents/agentex/cloud-build-and-deploy/overview). ## Installation System Manager is installed as a Helm chart into the Kubernetes cluster during the deployment of the SGP platform. See your cloud provider's corresponding deployment guide for more information. ## Configuration System Manager is configured via the `system-manager-config` secret in the cloud provider's secret manager: * AWS: `scale-egp-/system-manager-config` in Secrets Manager * GCP: `system-manager-config` in Secret Manager, starting with the Terraform `secret_name_prefix` when one is set, for example `sgp--system-manager-config` * Azure: `system-manager-config` in the deployment's Key Vault Update the secret directly or use the System Manager GUI. After the configuration is loaded, explicitly install the affected packs to render and apply their resources; saving a secret alone does not deploy changes. Startup installation is enabled by default: a restart reloads startup settings and reinstalls all packs in the desired state. To disable startup installation, set `env.INIT_ON_STARTUP` to `""` in the System Manager Helm chart values (chart version `27.18.2` or later). Any other value, including `"false"` or `"0"`, leaves it enabled. To restart System Manager: ```bash theme={null} kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager ``` The following AWS example selects SAML authentication. For OIDC, set `authType` to `OIDC`, `samlSetupEnabled` to `false`, and `oidcSetupEnabled` to `true`. The `aws` block is only present for AWS deployments; GCP and Azure deployments use equivalent `gcp` and `azure` blocks instead. `baseRepository` is the container registry that SGP images are pulled from, as a prefix without a URL scheme; configure the Helm chart URL separately in the `sgp-helm-repository` pack. On AWS and Azure, Terraform sets it to Scale's registry, `022465994601.dkr.ecr.us-west-2.amazonaws.com`, unless you override it. The GCP guide sets it to your Artifact Registry, `-docker.pkg.dev//sgp--docker-repository`. `deploymentURL` is a hostname without `https://`. ```json theme={null} { "cloudProvider": "aws", "baseRepository": "", "samlSetupEnabled": true, "oidcSetupEnabled": false, "deploymentURL": ".workspace.egp.scale.com", "workspaceId": "", "authType": "SAML", "baseDomain": "", "train_tenant_prefix": "", "deployAgentex": true, "deploySae": true, "aws": { "accountId": "", "region": "", "prefix": "", "modelEngineS3Bucket": "scale-egp--ml", "sqsQueuePolicyTemplate": "", "sqsQueueTagTemplate": "", "clusterName": "", "karpenterIrsaArn": "", "targetGroupArn": "", "nodeSubnets": ["", ""], "nodeSecurityGroup": "", "postgresHostTemporal": "", "compassBucketName": "", "compassMongoHost": "", "compassRedisHost": "", "reductoBucketName": "", "reductoDatabaseUrl": "", "reductoIrsaRoleArn": "", "reductoAzureVisionEndpoint": "", "reductoAzureVisionKey": "", "codeBuildProjectName": "", "codeBuildS3Bucket": "", "codeBuildRegistryUrl": "", "codeBuildServiceRoleArn": "", "cloudDeployEnabled": true, "dex": { "irsaRoleArn": "", "prefix": "duc-" }, "train": { "irsaRoleArn": "", "sagemakerExecutionRoleArn": "", "sagemakerSecurityGroupId": "", "databaseHost": "", "dataBucket": "", "checkpointsBucket": "", "outputBucket": "", "stagingBucket": "" }, "registryProxy": { "irsaRoleArn": "", "ecrRepositoryUrl": "", "jwtSecret": "", "workspaceConfig": "" } }, "frontDoorSSLCertB64": "", "frontDoorSSLKeyB64": "", "initialDesiredState": "", "datadog": { "enabled": true, "env": "", "clusterName": "", "secretName": "", "irsaRoleArn": "" } } ``` ## Architecture System Manager runs as a deployment in the `sgp-system-manager` namespace. Its GUI is accessible on port 8000. ```bash theme={null} kubectl port-forward deployment/sgp-system-manager -n sgp-system-manager 8000:8000 ``` Then navigate to `http://localhost:8000` in your browser to access the GUI. ```mermaid theme={null} graph TD ADMIN(["Platform Admin"]) subgraph CLUSTER["Kubernetes Cluster"] subgraph SM_NS["sgp-system-manager namespace"] GUI["Web GUI
:8000"] API["REST API
/api/v3"] WATCHER["Secret Watcher
change notifications"] INSTALLER["Pack Installer"] end subgraph RESOURCES["Cluster Resources"] NS["Namespaces"] SEC["Secrets + Pull Secrets"] HR["HelmRelease CRDs"] end subgraph FLUX_NS["flux-system namespace"] FLUX["Flux Helm Controller"] end PODS["SGP Services
(Helm-managed pods)"] end subgraph CLOUD["Cloud Provider"] SM_SECRET["Secret Manager
desired-state · config"] HELM_REPO["Helm Repository
OCI Artifact Registry"] end ADMIN -->|"browser"| GUI GUI --> API API -->|"read / write"| SM_SECRET WATCHER -->|"polls every 3s when configured"| SM_SECRET API --> INSTALLER INSTALLER --> NS INSTALLER --> SEC INSTALLER --> HR FLUX -->|"watches"| HR FLUX -->|"pull chart"| HELM_REPO FLUX -->|"deploy"| PODS ``` ### Packs System Manager organizes services into "packs". Each pack is a collection of resources that are deployed together. Generally packs are composed of [FluxCD HelmRelease](https://fluxcd.io/flux/components/helm/helmreleases/) as well as other resources necessary to support a particular service. When a pack is installed, System Manager renders its resource templates and writes the resulting Kubernetes resources — namespaces, secrets, and FluxCD HelmRelease CRDs — to the cluster. FluxCD then picks up the HelmRelease CRDs and handles pulling and deploying the Helm charts. ### FluxCD System Manager offloads resource reconciliation to FluxCD. FluxCD is a tool that allows you to manage the lifecycle of your Kubernetes resources. Flux reconciles installed HelmRelease changes and manages chart installation and upgrades. System Manager configures workload drift detection in `warn` mode: direct workload edits are reported rather than automatically corrected, and Deployment replica counts are ignored. For more information on FluxCD, see the [FluxCD documentation](https://fluxcd.io/flux/components/helm/helmreleases/). ### Desired state The collection of packs that System Manager will deploy is defined in the `system-manager-desired-state` secret, stored with the same prefix as `system-manager-config` in the cloud provider's secret manager. To modify the desired state, you can either update the secret directly or use the System Manager GUI, then explicitly install the affected packs via the System Manager GUI. Saving the desired state alone does not apply changes. When startup installation is enabled, a restart reinstalls all packs in the desired state, as described in [Configuration](#configuration). A sample desired state is shown below: ```json theme={null} { "version": "0.1", "packs": [ { "name": "flux" }, { "name": "sgp-helm-repository" }, { "name": "istio" }, { "name": "sgp-base" }, { "name": "spicedb" }, { "name": "identity-service" }, { "name": "temporalf" }, { "name": "sgp-apps" } ] } ``` # Creating an Agent Source: https://docs.gp.scale.com/docs/v5/agents/agent-service/lifecycle/creating-agent-service-agents How to create and publish an agent in the SGP UI. > [!WARNING] > > #### Recommendation & Future Development > > While Agent Service remains supported, **Agentex is the recommended path** for all new projects. Future development, new features, and enhancements will be focused primarily on Agentex. ## 1. Give your agent a name The first step is to give your agent a name. This will be used to identify the agent in the SGP UI. 1. Navigate to the [agents page](https://egp.dashboard.scale.com/beta/build). 2. Create a new Agent Service Agent using the top right **Create Agent** button. 3. Fill in the name of the agent and an optional description and click **Create Agent**. ![Create Agent Service Agent](https://mintcdn.com/scalegp/5kwZ4xktkH90oqXb/images/v5/agents/agent-service/create/agent-service-create-agent.png?fit=max\&auto=format\&n=5kwZ4xktkH90oqXb\&q=85\&s=20d8ceda50cbef48e14b35281b3320fd) A draft version of the agent will be created and you will be redirected to the agent's CI/CD page. ## 2. Configure your agent Now you should see your agent's CI/CD page with a YAML editor featured prominently. Any updates you make to the YAML will be automatically saved. ![Blank YAML editor](https://mintcdn.com/scalegp/5kwZ4xktkH90oqXb/images/v5/agents/agent-service/create/agent-service-blank-yaml.png?fit=max\&auto=format\&n=5kwZ4xktkH90oqXb\&q=85\&s=10d295017372737398aa3796f0ac378d) This YAML configuration is what is used to build the agent on the fly using Agent Service. You can copy and paste or drag and drop an existing YAML file from your local device. Alternatively, you can get started by selecting a template from the **Use a template** menu in the bottom. This is a library of preconfigured YAML files or snippets that we are constantly expanding. Selecting the template will immediately update the YAML in the editor, which you can modify. For example, you could swap the `llm_model` from `gpt-4o-mini` to another one like `llama-3-1-70b-instruct` for an open source alternative. Note that these models are directly interfacing with SGPs model API and can be called by name or by using the `model_deployment_id`. ![Template Agent](https://mintcdn.com/scalegp/5kwZ4xktkH90oqXb/images/v5/agents/agent-service/create/agent-service-template-agent.png?fit=max\&auto=format\&n=5kwZ4xktkH90oqXb\&q=85\&s=cc2f8ef1c4d15b7c66ac1dbfdfb33173) Often, it is useful to visualize this workflow. To do that, you can swap to the **Graph** view on the top left, which will show a node visualization of your workflow or state machine application. Clicking on any of the nodes shows its configuration. ![Graph View](https://mintcdn.com/scalegp/5kwZ4xktkH90oqXb/images/v5/agents/agent-service/create/agent-service-graph-view.png?fit=max\&auto=format\&n=5kwZ4xktkH90oqXb\&q=85\&s=f6538127444305ab13a20e66d3607c60) ## 3. Test your agent You can easily test the current configuration (as long as it is a valid YAML). To do that click **Test** in the bottom menu. ![Test Agent](https://mintcdn.com/scalegp/5kwZ4xktkH90oqXb/images/v5/agents/agent-service/create/agent-service-testing.png?fit=max\&auto=format\&n=5kwZ4xktkH90oqXb\&q=85\&s=e168854582dfdcd65b5eb5d36887830c) By changing your YAML configuration and continuously testing its output, you can iteratively adjust your agent until it has reached the desired state. The **Threads** panel on the right supports multi-turn by default. If you want to reset context and start a new thread, just click **New Thread**. The YAML configuration saves automatically after a short delay. Watch the **Autosaving** indicator and wait for saving to finish before deploying. ## 4. Publish your agent When you're ready to deploy, click **Deploy Agent** in the top right, then **Deploy**. If the agent is still a draft, SGP publishes it first and then deploys it. After publication, the configuration editor is read-only, and you can still update the name and description. To change the configuration, create a new variant. See [Deploying an Agent](/docs/v5/agents/agent-service/lifecycle/deploying-agent-service-agents) for the deploy options. ![Published Agent](https://mintcdn.com/scalegp/5kwZ4xktkH90oqXb/images/v5/agents/agent-service/create/agent-service-published.png?fit=max\&auto=format\&n=5kwZ4xktkH90oqXb\&q=85\&s=56c6aeae68be1a9dcb72ce073fd62b1c) ## What's next? Using the templates captures only a tiny fraction of what is possible with Agent Service. To write you own custom configurations you need to learn the set of available nodes, connections and techniques. Learn more in our v4 documentation on [Building Agents](/docs/agents/hello-world). # Deploying an Agent Source: https://docs.gp.scale.com/docs/v5/agents/agent-service/lifecycle/deploying-agent-service-agents How to deploy and interact with agents in the SGP UI. > [!WARNING] > > #### Recommendation & Future Development > > While Agent Service remains supported, **Agentex is the recommended path** for all new projects. Future development, new features, and enhancements will be focused primarily on Agentex. ## Introduction Agents that are configured with YAML in SGP and are fully executed on the platform's backend. The application logic defined in your YAML configuration determines how the agent processes inputs and produces outputs. ## Live Execution from YAML Editor As you configure your agent in YAML, you can execute it within SGP's interface to see how it behaves. The execution output is shown in the right-hand panel, on a per-node basis. This allows you to debug and validate your agent's flow in real time: ![Test Agent](https://mintcdn.com/scalegp/5kwZ4xktkH90oqXb/images/v5/agents/agent-service/create/agent-service-testing.png?fit=max\&auto=format\&n=5kwZ4xktkH90oqXb\&q=85\&s=e168854582dfdcd65b5eb5d36887830c) Outputs can be viewed either as raw JSON or in a more readable, formatted display. Toggle between these modes to best understand or debug your agent's responses. ## Providing User Inputs In the Agent Service CI/CD page, you will be prompted to provide inputs that are designated as `user_question` in the YAML config. ```yaml theme={null} workflow: - name: query_llm type: generation config: llm_model: gpt-4o-mini temperature: 0.2 inputs: input_prompt: user_question ``` These inputs will be rendered in the inputs panel. You can provide them as either raw JSON or fill them in via form fields. If you do not specify user inputs or type them, you will have to use JSON inputs. Hitting submit starts an execution of the agent as a run. ![Submit Input](https://mintcdn.com/scalegp/6cOXFxYysZJE_3yL/images/v5/agents/agent-service/deploy/agent-service-submit-input.png?fit=max\&auto=format\&n=6cOXFxYysZJE_3yL\&q=85\&s=f12677a6238acf64c80cd5abf0631c61) ## Deploying an Agent From the Agent Service CI/CD page, click **Deploy Agent**. Then enter a custom slug (a unique identifier) for your url. By default this is set to the agent's id. Click **Deploy**, which creates a simple web-based chat interface you can access. ![Deploy Agent](https://mintcdn.com/scalegp/6cOXFxYysZJE_3yL/images/v5/agents/agent-service/deploy/agent-service-deploy-agent.png?fit=max\&auto=format\&n=6cOXFxYysZJE_3yL\&q=85\&s=6aebc4534053ddd7fa460e0af65d8cde) When an agent is deployed, you will see a success message and a link to the deployed agent in the header. ![Deployed Agent](https://mintcdn.com/scalegp/6cOXFxYysZJE_3yL/images/v5/agents/agent-service/deploy/agent-service-deployed.png?fit=max\&auto=format\&n=6cOXFxYysZJE_3yL\&q=85\&s=6acccf2f83cfd53b622f8228176fec82) ## Interacting with a Deployed Agent Once deployed, you can visit the web app at the slug you specified. You can chat with the agent in a more intuitive, conversational format. The chat interface supports threads, user feedback, and dynamic inputs (if the agent requires more than one input per turn). ![Chat Interface](https://mintcdn.com/scalegp/5kwZ4xktkH90oqXb/images/v5/agents/agent-service/deploy/agent-service-chat.png?fit=max\&auto=format\&n=5kwZ4xktkH90oqXb\&q=85\&s=7994a8878092642ad7819f59cd550bc1) ## Customizing a Deployed Agent Once an agent is deployed, you can customize it by clicking the **Manage Deployment** tab. From there you can: * Change the slug (URL) of the deployed agent * Change the theme of the deployed agent * Change the evaluation settings of the deployed agent ### Changing the slug To change the slug of the deployed agent, simply enter a new slug in the input field and click **Redeploy**. ![Redeploy Agent](https://mintcdn.com/scalegp/6cOXFxYysZJE_3yL/images/v5/agents/agent-service/deploy/agent-service-redeploy.png?fit=max\&auto=format\&n=6cOXFxYysZJE_3yL\&q=85\&s=c7c2a5dab3e3077f4f2acd483daeadc4) ### Changing the theme To change the theme of the deployed agent, simply select a new theme from the dropdown. This will apply the theme to the deployed agent without having to redeploy. You can also create a new theme by clicking **Create New Theme** in the dropdown. This will open the theme editor, where you can customize the theme to your liking. ![Create Theme](https://mintcdn.com/scalegp/5kwZ4xktkH90oqXb/images/v5/agents/agent-service/deploy/agent-service-create-theme.png?fit=max\&auto=format\&n=5kwZ4xktkH90oqXb\&q=85\&s=509bc3046a264ec5d328b73b71b4666b) ### Changing the evaluation settings If online evaluation is available for your account, the **Online Evaluation** switch controls the application-level setting. Evaluations apply only to deployed applications and interactions made through the chat interface. The setting is not scoped to a single agent version. ![Online Evaluation](https://mintcdn.com/scalegp/6cOXFxYysZJE_3yL/images/v5/agents/agent-service/deploy/agent-service-online-evaluation.png?fit=max\&auto=format\&n=6cOXFxYysZJE_3yL\&q=85\&s=2b280cc8209d17a2f574b499ede631fd) # Monitoring an Agent Source: https://docs.gp.scale.com/docs/v5/agents/agent-service/lifecycle/monitoring-agent-service-agents How to monitor and evaluate agents in the SGP UI. > [!WARNING] > > #### Recommendation & Future Development > > While Agent Service remains supported, **Agentex is the recommended path** for all new projects. Future development, new features, and enhancements will be focused primarily on Agentex. ## Monitoring Executions Every time an agent is run (during testing or otherwise), it is recorded and can be viewed on the traces page. This provides a comprehensive view of all threads and their inputs/outputs, helping you track performance and diagnose issues. You can navigate to the traces page by clicking the **View Traces** button in the agent header. ![View Traces](https://mintcdn.com/scalegp/6cOXFxYysZJE_3yL/images/v5/agents/agent-service/monitor/agent-service-view-trace.png?fit=max\&auto=format\&n=6cOXFxYysZJE_3yL\&q=85\&s=637a00ee69e67d5f09bb7fbfee64dd61) From the traces page you can: * View the inputs and outputs of each thread * View the performance metrics of each thread * View the metadata of each thread * Filter and search for specific threads ![Traces Page](https://mintcdn.com/scalegp/6cOXFxYysZJE_3yL/images/v5/agents/agent-service/monitor/traces-page.png?fit=max\&auto=format\&n=6cOXFxYysZJE_3yL\&q=85\&s=3babe4ca09a20a6b6ec91283389c3746) Clicking on a trace will open the trace details in a modal. ![Individual Trace](https://mintcdn.com/scalegp/6cOXFxYysZJE_3yL/images/v5/agents/agent-service/monitor/individual-trace.png?fit=max\&auto=format\&n=6cOXFxYysZJE_3yL\&q=85\&s=9564f9399d6ef3f044ccae21f398fe03) # Introduction to Agent Service Source: https://docs.gp.scale.com/docs/v5/agents/agent-service/overview An overview of SGP's Agent Building Framework > [!WARNING] > > #### Recommendation & Future Development > > While Agent Service remains supported, **Agentex is the recommended path** for all new projects. Future development, new features, and enhancements will be focused primarily on Agentex. Welcome to the introductory guide for **Agent Service**. This document is designed for engineers who are new to Agent Service and want to learn how to build GenAI agents using our framework. Agent Service provides the backend for SGP applications, enabling the creation, deployment, and execution of powerful AI solutions based on Large Language Models (LLMs) and other AI models. To get a more in depth overview of Agent Service, check out the [v4 docs](/docs/agents/agent-service) page. ## What is Agent Service? Agent Service is a software framework that provides a set of modular building blocks that make it easy to create, deploy, and run AI applications. These applications can perform a variety of tasks, including: * **Document Analysis** * **Chatbot Responses** * **Custom AI Solutions** While Agent Service is primarily designed to support LLM-based applications, it can also accommodate other types of models (for example, multi-modal LLMs) on an ad-hoc basis. ## Key Features * **Modular Design:** Build applications using pre-configured components that can be combined like Lego blocks. * **Flexible Architecture:** Support both simple, linear workflows and complex, interactive applications with ease. * **Seamless Integration with SGP:** Easily integrate with existing SGP tools and APIs for functionalities such as information retrieval and text generation. ## Core Components Agent Service applications are constructed using several key components: ### Node * **Definition:** A node represents a single operation such as generation, retrieval, or data transformation. * **Examples:** * **Generation Nodes:** Handle LLM-based text generation and chat responses. * **Reranker & Retriever Nodes:** Enhance search capabilities and ensure relevance when working with large datasets. * **Citation & Code Execution Nodes:** Provide operations like reference generation or executing code for data transformation. ### Workflow * **Definition:** A workflow is a sequence of nodes arranged in a specific order. It aggregates data from user inputs or previous outputs. * **Features:** * Supports parallel processing for maximum execution speed. * Offers flexibility in how nodes are combined and executed. ### State Machine * **Definition:** A state machine is a structured flow that transitions between different states (or workflows) based on outcomes. * **Capabilities:** * Supports branching and conditional paths. * Incorporates additional user inputs for highly interactive applications. ![schematic overview of agent service](https://mintcdn.com/scalegp/_1tSRXzqUSANQJ5q/images/agent_service_overview.png?fit=max\&auto=format\&n=_1tSRXzqUSANQJ5q\&q=85\&s=684c5973685053b4bccaba16a214f4ca) # Agentex CI/CD Source: https://docs.gp.scale.com/docs/v5/agents/agentex/agentex-monitoring Understand how to set up CI/CD for Agentex Agents ## Navigating to the Agentex CI/CD page 1. To view all Agentex agents in SGP open the **Agents** page in your SGP deployment. 2. Click on an Agentex agent to navigate to the Agentex CI/CD page. ![Agentex CI/CD page](https://mintcdn.com/scalegp/6cOXFxYysZJE_3yL/images/v5/agents/agentex/agentex-ci-cd-page.png?fit=max\&auto=format\&n=6cOXFxYysZJE_3yL\&q=85\&s=ac6d89443f0a5982a995c642cad87b62) ## Agentex CI/CD functionality From the Agentex CI/CD page you can: * View the active version of the Agentex agent * Open the agent chat interface * Navigate to the agent's traces * See the git commits corresponding to each agent deployment ## Agentex versioning The active version of the Agentex agent is displayed at the top of the page. In the Gitea agentex-agents repository, merging a change under `teams/**/agents/**` into `main` builds and deploys the changed agents automatically through the `cloud-build-and-deploy-on-push.yaml` workflow described in [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd). Changes outside those paths do not trigger an agent deployment. To create a new version, merge agent changes into `main` in that Gitea repository, or use manual workflow dispatch. After a successful deployment, the CI/CD page shows the deployed version, source commit, and deployment timestamp. Other git hosts require a separately configured integration. # API and SDK Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/api Programmatic access to Agentex cloud builds and deployments via the REST API and Python/TypeScript SDKs. ## Overview The Build and Deploy API and SDKs give you programmatic control over Agentex cloud builds and deployments. You can create builds, poll for status, stream logs, create deployments, and tear them down through REST endpoints under `/v5/builds` and `/v5/agentex/deployments`, or through the equivalent Python and TypeScript SDK methods. The API and SDKs expose the same underlying resources as the UI, CLI, and CI/CD interfaces, so anything you create or modify programmatically is visible everywhere else. Use them when you need to integrate builds and deployments into custom automation or tooling. For full SDK documentation, see the [Python SDK reference](https://sgp-dev.stldocs.app/api/python) and [TypeScript SDK reference](https://sgp-dev.stldocs.app/api/typescript). ## Authentication Every request to the Build and Deploy API requires two headers: | Header | Description | | - | - | | `x-api-key` | Your SGP API key. Found in the IAM or admin section of the SGP dashboard. | | `x-selected-account-id` | Your SGP account ID. Found in the admin section of the SGP dashboard. | ```bash curl theme={null} curl -s $SGP_CLIENT_BASE_URL/v5/builds \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` ```python Python theme={null} from scale_gp_beta import SGPClient client = SGPClient( api_key="your-api-key", account_id="your-account-id", ) ``` ```typescript TypeScript theme={null} import SGPClient from 'scale-gp'; const client = new SGPClient({ apiKey: process.env.SGP_API_KEY, accountID: process.env.SGP_ACCOUNT_ID, }); ``` > [!NOTE] > > The Python and TypeScript SDKs handle authentication headers automatically once you initialize the client. The examples in the rest of this page assume you have already created a `client` instance as shown above. ## Build lifecycle A build packages your agent source code into a container image. The lifecycle is: **create** the build, **poll** its status until it reaches a terminal state, and optionally **stream logs** or **cancel** it. See the [build status lifecycle](/docs/v5/agents/agentex/cloud-build-and-deploy/overview#build) for the full state diagram. ### Create a build To create a build, send a multipart form request with your build context archive (a `.tar.gz` of your agent source code), the target image name and tag, and the agent: `agent_name` for a new agent's first build, or `agent_id` for every build of an agent that already exists. ```bash curl theme={null} curl -X POST $SGP_CLIENT_BASE_URL/v5/builds \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" \ -F "context_archive=@build-context.tar.gz" \ -F "image_name=my-agent" \ -F "image_tag=v1.2.0" \ -F "agent_name=my-agent" ``` ```python Python theme={null} from pathlib import Path build = client.build.create( context_archive=Path("./build-context.tar.gz"), image_name="my-agent", image_tag="v1.2.0", agent_name="my-agent", ) print(f"Build created: {build.id} (status: {build.build_status})") ``` ```typescript TypeScript (Node.js) theme={null} import { createReadStream } from 'node:fs'; const build = await client.build.create({ context_archive: createReadStream("./build-context.tar.gz"), image_name: "my-agent", image_tag: "v1.2.0", agent_name: "my-agent", }); console.log(`Build created: ${build.id} (status: ${build.build_status})`); ``` The response includes the build ID, the agent's `agent_id`, and an initial status of `queued`. Use the build ID to poll for status updates and retrieve logs. `agent_name` registers a new agent, so it only works for an agent's first build. If an agent with that name already exists, the API returns 400 and asks you to build it with `agent_id`. For every later build, send `agent_id` instead of `agent_name`. Optional parameters: | Parameter | Description | | - | - | | `build_args` | Docker build arguments, as a JSON object string such as `{"PYTHON_VERSION": "3.11"}` | | `platform` | Target platform for the image build (e.g., `linux/amd64`) | | `image_url` | An existing tagged image to register instead of uploading `context_archive`. The build is recorded as `success` right away and can be deployed. Cannot be combined with `build_args` or `platform`. | ### Get build status Poll `GET /v5/builds/{build_id}` to check the current status of a build. ```bash curl theme={null} curl -s $SGP_CLIENT_BASE_URL/v5/builds/$BUILD_ID \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` ```python Python theme={null} build = client.build.retrieve("build_abc123") print(f"Status: {build.build_status}") ``` ```typescript TypeScript theme={null} const build = await client.build.retrieve("build_abc123"); console.log(`Status: ${build.build_status}`); ``` ### Cancel a build You can cancel a build that is in `queued` or `running` state. ```bash theme={null} curl -X POST $SGP_CLIENT_BASE_URL/v5/builds/$BUILD_ID/cancel \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` ## Deploy lifecycle A deployment takes a successfully built container image and rolls it out to a Kubernetes cluster. The lifecycle is: **create** the deployment, **poll** its status until it becomes `healthy` or `failed`, and inspect **logs** for debugging. See the [deployment status lifecycle](/docs/v5/agents/agentex/cloud-build-and-deploy/overview#deployment) for the full state diagram. ### Create a deployment To create a deployment, provide the `manifest_file` (your `manifest.yaml` content as a string), the `environment_config` (one entry from under `environments:` in your `environments.yaml`, as a YAML string, not the whole file), and a `build_id` referencing a successful build. The entry must include `auth`. ```bash curl theme={null} curl -X POST $SGP_CLIENT_BASE_URL/v5/agentex/deployments \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" \ -H "Content-Type: application/json" \ -d '{ "manifest_file": "kind: Agent\nagent:\n name: my-agent\n...", "environment_config": "environment: production\nkubernetes:\n namespace: agents\nauth:\n principal:\n account_id: your-account-id\n...", "build_id": "build_abc123" }' ``` ```python Python theme={null} deployment = client.deploy.create( manifest_file="kind: Agent\nagent:\n name: my-agent\n...", environment_config="environment: production\nkubernetes:\n namespace: agents\nauth:\n principal:\n account_id: your-account-id\n...", build_id="build_abc123", ) print(f"Deployment created: {deployment.id} (status: {deployment.status})") ``` ```typescript TypeScript theme={null} const deployment = await client.deploy.create({ manifest_file: "kind: Agent\nagent:\n name: my-agent\n...", environment_config: "environment: production\nkubernetes:\n namespace: agents\nauth:\n principal:\n account_id: your-account-id\n...", build_id: "build_abc123", }); console.log(`Deployment created: ${deployment.id} (status: ${deployment.status})`); ``` Instead of a `build_id`, you can supply `image_name` and `image_tag` to select an existing successful build in your account. The image must have a recorded build with `build_status=success`; a registry image alone is insufficient. To create a **preview deployment** (ephemeral, with automatic teardown), add the following optional parameters: | Parameter | Description | | - | - | | `preview` | Set to `true` to create a preview deployment | | `preview_label` | A label for the preview (e.g., `pr-482`). Maximum 30 characters. | | `expires_at` | ISO 8601 timestamp for when the preview should be torn down | ### Get deployment status Poll `GET /v5/agentex/deployments/{deployment_id}` to check the current status and view deployment events. ```bash curl theme={null} curl -s $SGP_CLIENT_BASE_URL/v5/agentex/deployments/$DEPLOYMENT_ID \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` ```python Python theme={null} deployment = client.deploy.retrieve("deploy_xyz789") print(f"Status: {deployment.status}") ``` ```typescript TypeScript theme={null} const deployment = await client.deploy.retrieve("deploy_xyz789"); console.log(`Status: ${deployment.status}`); ``` ### Delete a deployment To tear down a deployment and release its cluster resources: ```bash theme={null} curl -X DELETE $SGP_CLIENT_BASE_URL/v5/agentex/deployments/$DEPLOYMENT_ID \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` After a successful DELETE, the deployment record is removed. A subsequent `GET /v5/agentex/deployments/{deployment_id}` returns HTTP 404. Automatic preview expiry instead retains the record with status `deleted` after cleanup. ## Polling pattern Builds and deployments are asynchronous operations. After creating either resource, poll the status endpoint until the resource reaches a terminal state. **Build statuses:** `queued` → `running` → `success` | `failed` | `cancelled` | `timed_out` | `error` | `unknown`. `unknown` requires investigation. A cancel request moves the build to `cancelling` until the provider stops it, then to `cancelled`, or to the status it reached if it finished first. When you delete an agent, its builds move to `deleting`, or to `delete_failed` if the cleanup fails. **Deploy statuses:** `pending` → `in_progress` → `healthy` | `failed` | `cancelled`. A deployment becomes `superseded` when a newer production deployment replaces it. Expired previews retained after automatic cleanup have status `deleted`; manually deleted deployments return HTTP 404. Stop rollout polling on either result or on `superseded`; inspect or poll the replacement deployment separately. ```bash curl theme={null} # Poll a build until it reaches a terminal state while true; do STATUS=$(curl -s $SGP_CLIENT_BASE_URL/v5/builds/$BUILD_ID \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" \ | jq -r '.build_status') echo "Build status: $STATUS" case $STATUS in success) break ;; failed|cancelled|timed_out|error|unknown|deleting|delete_failed) echo "Build cannot be deployed: $STATUS" >&2; exit 1 ;; esac sleep 5 done ``` ```python Python theme={null} import time build = client.build.retrieve("build_abc123") stop_statuses = {"success", "failed", "cancelled", "timed_out", "error", "unknown", "deleting", "delete_failed"} while build.build_status not in stop_statuses: time.sleep(5) build = client.build.retrieve("build_abc123") print(f"Build status: {build.build_status}") if build.build_status != "success": raise RuntimeError(f"Build ended with status: {build.build_status}") ``` ```typescript TypeScript theme={null} const stopStatuses = new Set(["success", "failed", "cancelled", "timed_out", "error", "unknown", "deleting", "delete_failed"]); let build = await client.build.retrieve("build_abc123"); while (!stopStatuses.has(build.build_status)) { await new Promise((r) => setTimeout(r, 5000)); build = await client.build.retrieve("build_abc123"); console.log(`Build status: ${build.build_status}`); } if (build.build_status !== "success") { throw new Error(`Build ended with status: ${build.build_status}`); } ``` > [!NOTE] > > A polling interval of 5 seconds is a reasonable default. Builds typically take 2-10 minutes depending on image complexity. Deployments typically take 1-5 minutes depending on image size and cluster capacity. ## Log streaming Build logs and deployment logs use different retrieval mechanisms. ### Build logs (SSE) Build logs are streamed via Server-Sent Events (SSE) from `GET /v5/builds/{build_id}/logs`. Each event contains a log line from the image build process. ```bash theme={null} curl -N $SGP_CLIENT_BASE_URL/v5/builds/$BUILD_ID/logs \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` The response is a stream of `text/event-stream` data. Each line is a log entry from the build. The stream closes when the build completes. ### Deployment logs (cursor-based pagination) Deployment logs are retrieved with cursor-based pagination from `GET /v5/agentex/deployments/{deployment_id}/logs`. Each response includes a page of log lines, a `next_cursor` for fetching the next page, and a `has_more` flag. ```bash curl theme={null} # Fetch the first page of deployment logs CURSOR="" while true; do RESPONSE=$(curl -s "$SGP_CLIENT_BASE_URL/v5/agentex/deployments/$DEPLOYMENT_ID/logs?limit=100&cursor=$CURSOR" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID") echo "$RESPONSE" | jq -r '.lines[].message' HAS_MORE=$(echo "$RESPONSE" | jq -r '.has_more') if [ "$HAS_MORE" != "true" ]; then break fi CURSOR=$(echo "$RESPONSE" | jq -r '.next_cursor') done ``` ```python Python theme={null} cursor = None while True: params = {"limit": 100} if cursor: params["cursor"] = cursor response = client.deploy.logs( "deploy_xyz789", **params, ) for line in response.lines: print(line.message) if not response.has_more: break cursor = response.next_cursor ``` ```typescript TypeScript theme={null} let cursor: string | undefined; while (true) { const response = await client.deploy.logs("deploy_xyz789", { limit: 100, cursor, }); for (const line of response.lines ?? []) { console.log(line.message); } if (!response.has_more) break; cursor = response.next_cursor; } ``` ## Listing and filtering Use the list endpoints to retrieve builds and deployments for an agent, filtered by common criteria. REST list responses and SDK pages contain an `items` array. `limit` controls page size. Iterating an SDK page fetches subsequent pages automatically; use `page.items` when you need only the first page. ### List builds Retrieve builds filtered by agent name, with control over page size and sort order. The SDK loops below enumerate all matching builds. ```bash curl theme={null} curl -s "$SGP_CLIENT_BASE_URL/v5/builds?agent_name=my-agent&limit=10&sort_order=desc" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` ```python Python theme={null} builds = client.build.list( agent_name="my-agent", limit=10, sort_order="desc", ) for build in builds: print(f"{build.id}: {build.build_status} ({build.image_tag})") ``` ```typescript TypeScript theme={null} const builds = client.build.list({ agent_name: "my-agent", limit: 10, sort_order: "desc", }); for await (const build of builds) { console.log(`${build.id}: ${build.build_status} (${build.image_tag})`); } ``` ### List deployments Retrieve deployments, optionally filtered by preview label to find the latest deployment for a specific branch or PR. The list is sorted oldest first by default, so pass `sort_order=desc` with `limit=1` to get the latest. The Python SDK's `preview_label` filter requires `scale-gp-beta` `0.2.0` or later. ```bash curl theme={null} # Get the latest deployment for a specific PR curl -s "$SGP_CLIENT_BASE_URL/v5/agentex/deployments?preview_label=pr-482&limit=1&sort_order=desc" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` ```python Python theme={null} page = client.deploy.list( preview_label="pr-482", limit=1, sort_order="desc", ) deployments = page.items if deployments: latest = deployments[0] print(f"Latest PR deploy: {latest.id} ({latest.status})") ``` ```typescript TypeScript theme={null} const page = await client.deploy.list({ preview_label: "pr-482", limit: 1, sort_order: "desc", }); const deployments = page.items; if (deployments.length > 0) { const latest = deployments[0]; console.log(`Latest PR deploy: ${latest.id} (${latest.status})`); } ``` ### Undeployed builds To find agents that have completed builds but no healthy deployment, use the undeployed builds endpoint: ```bash theme={null} curl -s $SGP_CLIENT_BASE_URL/v5/builds/undeployed \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` This is useful for identifying agents that may have been built but never deployed, or agents whose deployments have failed and need attention. ## API and SDK reference For full request and response schemas, see the generated API reference and SDK documentation: **API reference:** * [Agentex Cloud Build](/reference/v5/agentex-cloud-build/submit-container-image-build): endpoints for creating, listing, polling, and cancelling builds * [Agentex Cloud Deploy](/reference/v5/agentex-cloud-deploy/deploy-a-built-agent-image): endpoints for creating, listing, inspecting, and deleting deployments * [SGP Cloud Secrets](/reference/v5/sgp-cloud-secrets/list-secrets): deprecated v5 endpoints for managing account-level secrets (see [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets) for what deploys read) **SDK reference:** * [Python SDK](https://sgp-dev.stldocs.app/api/python): `scale_gp_beta` package (`client.build`, `client.deploy`) * [TypeScript SDK](https://sgp-dev.stldocs.app/api/typescript): `scale-gp` package (`client.build`, `client.deploy`) > [!NOTE] > > The Build and Deploy API does not support webhooks or push notifications. You must poll the status endpoints to track build and deployment progress. There is no callback mechanism to notify your system when a build completes or a deployment becomes healthy. ## Next steps * [sgpctl CLI](/docs/v5/agents/agentex/cloud-build-and-deploy/sgpctl): the same operations from the terminal with a simpler interface * [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): automate builds triggered by git push * [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets): manage credentials in the dashboard * [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments): ephemeral deployments for branch and PR testing # CI/CD with Gitea Actions Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd Automate agent builds and deployments with the provided Gitea Actions workflow templates. ## Overview The agentex-agents repository template ships with Gitea Actions workflows that automate the build and deploy lifecycle for your agents. These workflows use the SGP Build and Deploy API to package your agent source code into container images and deploy them to SGP-managed Kubernetes clusters. Each workflow file in the repository is a thin caller into the shared `sgp/agentex-workflows` repository, which holds the workflow logic and helper scripts. Changes under `teams/**/agents/**` trigger build and deploy when pushed to `main`. Pull requests that change agent files trigger a build, and agents with an `eval.yaml` on `main` also get a preview deployment for the eval. You can also trigger builds and deploys manually via workflow dispatch. You do not need to write these workflows from scratch. The template repository includes them pre-configured. Your responsibility is to understand when each workflow runs, how to configure your agent directories, and how to set up the required Actions secret and variables. ## How it works When a pull request or a merge to `main` changes an agent, the CI/CD pipeline detects the change and builds a container image. On the main branch, it first resolves each agent's deployment target, then builds and deploys the agents that have a matching target. The following diagram shows the flow on a merge to main for an agent with one matching deployment target: ```mermaid theme={null} flowchart LR A[Push to main] --> B[Detect changed agents] B --> C[Package build context] C --> D[POST /v5/builds] D --> E[Poll build status] E --> F{Build succeeded?} F -- Yes --> G[POST /v5/agentex/deployments] F -- No --> H[Fail workflow] G --> I[Poll deploy status] I --> J{Deploy healthy?} J -- Yes --> K[Agent serving traffic] J -- No --> L[Fetch logs & fail] ``` The pipeline auto-detects which agents changed by running `git diff` against the previous commit. Only agents with modified files are built. When one push touches several agents, the workflow builds and deploys them one at a time in a single job, waiting for each build (and deployment on main) before starting the next. A failure in one agent does not stop the others, and the run fails if any agent fails. ## The workflows The template repository includes these build and deploy workflow files under `.gitea/workflows/`. Each serves a different purpose in the development lifecycle. | Workflow | Trigger | Builds? | Deploys? | Use case | | - | - | - | - | - | | `cloud-build.yaml` | Pull request opened or updated (agent paths) | Yes | No | Validate builds before merge | | `cloud-build-and-deploy-on-push.yaml` | Push to `main` (agent paths) or manual dispatch | Yes | Yes | Build and deploy on merge to main | | `cloud-deploy.yaml` | Manual dispatch or workflow call | No | Yes | Deploy a specific image version or rollback | | `pr-deploy-preview.yaml` | `/deploy-preview` comment on a pull request | Yes | Preview | Try a pull request's code in a temporary deployment | | `cloud-preview-deploy.yaml` | Manual dispatch | Yes | Preview | Build agents at a given commit and deploy them as previews | | `ci-eval.yaml` | Pull request opened or updated | Yes | Preview | Run an agent's `eval.yaml` against a preview of the pull request | | `comment-eval.yaml` | `/eval` comment on a pull request or manual dispatch | Yes | Preview | Run evals on demand | ### cloud-build.yaml: build on pull requests This workflow triggers when you open or update a pull request that changes files under `teams/**/agents/**`. A feature-branch push without a pull request does not trigger it. It auto-detects which agents were modified, packages the build context for each one, and submits a build request to the SGP Build API via `POST /v5/builds`, one agent at a time. The workflow polls each build's status until it succeeds or fails, with a 600-second timeout. This workflow does not deploy. Its purpose is to validate that your agent builds successfully before you merge. ### cloud-build-and-deploy-on-push.yaml: build and deploy on main This workflow triggers when files under `teams/**/agents/**` change on the main branch, or via manual dispatch. It builds container images for changed agents with a matching deployment target and then deploys them. For each agent, the workflow: 1. Reads the agent's `manifest.yaml` and `environments.yaml` and resolves its deployment target 2. Packages the build context based on the agent's `manifest.yaml` build configuration 3. Submits the build to `POST /v5/builds` and polls until complete 4. Submits a deployment to `POST /v5/agentex/deployments` with the resulting `build_id` 5. Polls the deployment status until it reaches `healthy` or fails The workflow uses a `resolve_inputs.py` helper script to derive the image name from the `agent.name` field in the manifest and to select the `environments.yaml` entry for the current environment, as described in [environments.yaml](#environmentsyaml). Entries for other environments are skipped on merge and deploy through [promotion](#promotion). If more than one entry matches, not counting the one named by `preview_environment`, the run fails for that agent with an error asking for `agent-environment`. ### cloud-deploy.yaml: standalone deploy This workflow is manual-only. It deploys an existing successful build without rebuilding. You can trigger it via manual dispatch or call it from another workflow using `workflow_call`. Inputs include: * `agent-path` (required): path to the agent directory * `agent-environment` (required): key of an `environments.yaml` entry for the current environment * `build-id` OR `image-name` + `image-tag`: specify either a successful build ID or the image name and tag of a successful build in your account The workflow submits `POST /v5/agentex/deployments`, polls until the deployment is healthy, and fetches deploy logs on failure. > [!TIP] > > Use the standalone deploy workflow for rollbacks in the current environment. If a deployment fails, you can redeploy the previous known-good build by providing its `build-id`. A rollback doesn't change what [promotion](#promotion) deploys next: the next promotion still deploys the commit recorded for the entry before it, not the rolled-back build. ### pr-deploy-preview\.yaml: preview a pull request Comment `/deploy-preview teams/my-team/agents/my-agent` on a pull request to build that agent from the pull request's head commit and deploy it as a [preview deployment](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments). The workflow runs only for commenters with write access, for pull requests from the same repository, and for agents that the pull request changes and that already exist on `main`, so a new agent can't be previewed until it merges. The preview is labeled `pr-` followed by the pull request number and expires after 72 hours; add `--ttl-hours` with a value from 1 to 168 to change that. The result is posted as a comment on the pull request. Previews read `manifest.yaml` and `environments.yaml` from `main` and build the image from the pull request's code, so configuration changes must merge before previews use them. `cloud-preview-deploy.yaml` runs the same build and preview deploy by manual dispatch, for a list of agent paths at a given commit. ## Workflow runs After a workflow triggers, you can monitor its progress in the Gitea Actions UI. Each workflow run shows the individual steps, their status, and logs for each stage of the build and deploy process. ![Gitea workflow run detail page showing build and deploy steps](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/ci-cd/workflow-run.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=0b25503aa0a36a0bd9cb9578aed8240d) The workflow run page displays: * Each step in the pipeline with pass/fail indicators * Timing information for build and deploy phases * Console output for debugging failed steps If a build or deployment fails, expand the failed step to see the error output. Common failures include Dockerfile syntax errors, missing dependencies, and secret reference mismatches. ## Manual dispatch You can trigger a build and deploy for a specific agent without pushing code by using the manual dispatch form in Gitea Actions. ![Gitea manual dispatch form with input fields for agent path and configuration](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/ci-cd/manual-dispatch.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=9d4fb9a92255a4b79453827a0e8c6e40) To trigger a manual dispatch: 1. Navigate to the **Actions** tab in your Gitea repository 2. Select the `cloud-build-and-deploy-on-push.yaml` workflow (or `cloud-deploy.yaml` for deploy-only) 3. Click **Run workflow** 4. Fill in the inputs: * **agent-path** (required): the path to your agent directory (for example, `teams/my-team/agents/my-agent`) * **image-name** (optional): overrides the image name derived from `manifest.yaml`. Should match the `agent.name` in your manifest. * **agent-name** (optional): overrides the agent name. Should match `image-name` (the `resolve_inputs.py` helper sets both to the same value by default). * **image-tag** (optional): overrides the auto-generated tag. Use a version number (e.g., `v1.2.0`) or a commit SHA. Defaults to the current commit SHA. * **agent-environment** (optional for build and deploy): must match an environment key in your `environments.yaml` whose entry targets the current environment. Defaults to the entry that matches `CURRENT_ENVIRONMENT`, as on merge. The standalone `cloud-deploy.yaml` workflow requires this input. 5. Click **Run workflow** to start the pipeline > [!NOTE] > > Manual dispatch is useful during initial setup, testing, or when you need to deploy a specific image tag. For routine development, merging changes under `teams/**/agents/**` to main triggers build and deploy automatically. ## Repository structure The CI/CD workflows expect a specific directory layout. Each agent lives in its own directory under `teams//agents//` and contains the configuration files the workflows need. ``` teams/ my-team/ agents/ my-agent/ manifest.yaml # Build and agent configuration environments.yaml # Deployment settings (namespace, resources, etc.) Dockerfile # Container build instructions .dockerignore # Files to exclude from build context src/ main.py # Agent source code requirements.txt # Python dependencies ... another-agent/ manifest.yaml environments.yaml Dockerfile src/ ... ``` The workflows detect changes at the `teams/**/agents/**` path level. A pull request that changes an agent directory triggers build validation; pushing those changes to main triggers build and deploy. ## manifest.yaml The `manifest.yaml` file defines your agent's identity, build context, credentials, and deployment defaults. It is the primary configuration file the CI/CD workflows read when building and deploying your agent. ```yaml theme={null} kind: Agent build: context: # Root directory for the build context, relative to the agent directory root: ../ # Paths to include in the build context (relative to root) include_paths: - golden_agent # Path to the Dockerfile (relative to root) dockerfile: golden_agent/Dockerfile # Path to the .dockerignore file (relative to root) dockerignore: golden_agent/.dockerignore agent: # Agent Communication Protocol type: "async" or "sync" acp_type: async # Unique agent name (used as the image name by default) name: golden-agent # Human-readable description shown in the SGP dashboard description: Generic Claude Agent SDK harness. # Temporal workflow configuration (for async agents) temporal: enabled: true workflows: - name: GoldenAgentWorkflow queue_name: golden_agent_queue # Secrets injected as environment variables at runtime credentials: - env_var_name: ANTHROPIC_API_KEY secret_name: anthropic-api-key secret_key: api-key deployment: image: # Required by the manifest schema; cloud deploys use the build's image instead repository: "" tag: "latest" global: replicaCount: 1 resources: requests: cpu: "500m" memory: "1Gi" limits: cpu: "1000m" memory: "2Gi" ``` ### Key fields **`build.context`** controls what files are sent to the cloud build service. Set `root` to the common ancestor directory, then use `include_paths` to specify which subdirectories to include. The `dockerfile` and `dockerignore` paths are relative to `root`. **`agent`** defines the agent's identity and runtime configuration: * `acp_type` determines whether the agent uses synchronous request-response (`sync`) or asynchronous task-based (`async`) communication. * `name` is used as the default image name and must be unique within your account. * `temporal` configures Temporal workflow settings for async agents. Each workflow entry specifies the workflow class name and task queue. * `credentials` maps secrets stored in SGP to environment variables in the running container. Each entry references a secret by name and key, and injects it as the specified environment variable. See [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets) for how to create and manage these secrets. **`deployment`** sets default resource requests and replica counts. These values can be overridden in `environments.yaml`. > [!NOTE] > > Cloud deploys ignore `deployment.image` and `deployment.imagePullSecrets`. The server deploys the image recorded on the build and sets the pull secrets itself. The manifest schema still requires `deployment.image.repository`, so leave it as an empty string. ## environments.yaml The `environments.yaml` file defines deployment settings for your agent's target environment: the Kubernetes namespace, authentication principal, resource limits, and Helm value overrides. The file has a top-level `environments` map with a named key for each environment. The key name (e.g., `dev.aws` below) is arbitrary. On merge to `main`, the CI/CD pipeline's `resolve_inputs.py` helper deploys the entry whose `environment` field matches the repository's `CURRENT_ENVIRONMENT` variable (`dev` when unset). An entry without an `environment` field counts as the current environment. Entries for other environments are promotion targets; see [Promotion](#promotion). A top-level `preview_environment` key names the entry that preview and eval deployments use, and that entry must be for the current environment. ```yaml theme={null} schema_version: "v1" environments: dev.aws: environment: dev kubernetes: namespace: "sgp-golden-agent" auth: principal: user_id: account_id: helm_overrides: command: - uvicorn - project.acp:acp - --host - 0.0.0.0 - --port - "8000" replicaCount: 1 resources: requests: cpu: 500m memory: 1Gi limits: cpu: 1000m memory: 2Gi temporal-worker: enabled: true replicaCount: 1 resources: requests: cpu: 500m memory: 1Gi limits: cpu: 1000m memory: 2Gi env: - name: SGP_ACCOUNT_ID value: "" - name: SGP_CLIENT_BASE_URL value: "http://egp-api-backend.egp.svc.cluster.local:80/public" ``` ### Key fields **`kubernetes.namespace`** determines where the agent's pods, services, and other resources are created. **`auth.principal`** provides the `user_id` and `account_id` used to authenticate API calls. These must correspond to a valid SGP service account with permissions to create deployments. **`helm_overrides`** customizes the agent's pods. Values specified here take precedence over the defaults in `manifest.yaml`. The following keys are accepted, and a deploy request that sets any other key, such as the image or pull secrets, is rejected: | Key | What it sets | | - | - | | `replicaCount` | Number of pods when autoscaling is off. SGP turns autoscaling on by default, between 1 and 10 pods at 50% CPU, so `replicaCount` takes effect only if you also set `autoscaling.enabled: false` | | `resources` | `requests` and `limits` for `cpu`, `memory` and `ephemeral-storage` | | `autoscaling` | `enabled`, `minReplicas`, `maxReplicas`, `targetCPUUtilizationPercentage` and `targetMemoryUtilizationPercentage` | | `command`, `args` | The container command and its arguments | | `env` | Environment variables, as a list of `name` and `value` entries. Declare secrets as manifest credentials instead | | `healthCheck` | `enabled`, plus `livenessProbe` and `readinessProbe` with `path`, `port`, `initialDelaySeconds`, `periodSeconds`, `timeoutSeconds` and `failureThreshold` | | `datadog` | `enabled` and `environment` | | `podAnnotations` | Annotations for eviction, OpenTelemetry injection, Prometheus scraping and Istio probe rewriting | | `serviceAccount.annotations` | A cloud workload identity: `eks.amazonaws.com/role-arn`, `azure.workload.identity/client-id` or `iam.gke.io/gcp-service-account` | | `podDisruptionBudget.enabled` | Whether to create a pod disruption budget | | `volumes`, `volumeMounts` | `emptyDir` scratch volumes, each with a `sizeLimit`, and their mount paths | | `global.agent.name`, `global.workflow.name`, `global.workflow.taskQueue` | The agent name, and the Temporal workflow name and task queue | | `temporal-worker` | The keys above, except the `global` keys, for the Temporal worker, plus `enabled` | `replicas` and `temporal` are also accepted so that older files keep deploying, but they have no effect. Use `replicaCount` and `temporal-worker`. ### Promotion Entries for environments other than the current one are promotion targets, and they don't deploy on merge. A release admin runs each promotion from the `sgp-release/agentex-promotion` repository. Only release admins can see that repository and start runs there. To promote by hand, a release admin runs `promote.yaml` with these inputs: * `target-environment`: the environment to deploy into. The workflow refuses the current environment. * `agents`: the agents to promote. * `commit-sha`: the full 40-character commit SHA to deploy. The commit must be reachable from `main`. If your platform team turns it on, `scheduled-promote.yaml` also runs in that repository. It reads the schedules in each agent's `environments.yaml` and deploys each due entry with the commit last deployed to the entry listed before it. Every successful promotion records the commit as the git ref `refs/deployed///`. A `cloud-deploy.yaml` rollback doesn't change what is promoted next. ```yaml theme={null} environments: dev: environment: dev auth: principal: user_id: account_id: prod: environment: prod auth: principal: user_id: account_id: schedule: cron: "0 9 * * 1-5" timezone: America/New_York ``` Promotion entries follow these rules: * Entries are promoted in the order they are listed, so the first promotion entry needs an entry before it to promote from. * A promotion entry needs `environment`, `auth.principal.account_id` and a `schedule`. * `schedule.cron` must have `0` in its minute field, because schedules are checked once an hour. `timezone` is optional, defaults to UTC, and must be a whole number of hours from UTC. * Set `schedule.enabled: false` to pause an entry. * Entries for the current environment deploy on merge and can't have a `schedule`. * Each target environment can have only one scheduled entry. * The settings for each target environment live on `sgp-release/agentex-promotion`, and release admins manage them. Don't set them on your agent repository. ## Required secrets and variables The CI/CD workflows authenticate with the SGP API using your Gitea repository's Actions settings. `SGP_API_KEY` is an Actions secret that you add yourself. The others are Actions variables. SGP sets them when it creates the repository, except where the table says otherwise, and it keeps any value you set. | Setting | Kind | Description | | - | - | - | | `SGP_API_KEY` | Secret | API key for authenticating with the SGP Build and Deploy API. Its user needs the `admin`, `manager`, or `editor` role on the account, and an agent's first build needs `admin` or `manager` because it registers the agent. Workflow runs on every branch can read this secret, so anyone who can push a branch can read it. Use a key for a dedicated service account, not a personal admin key. | | `SGP_ACCOUNT_ID` | Variable | Your SGP account identifier, used when the environment entry doesn't set `auth.principal.account_id`. SGP sets it only if your install was set up with your account ID, so add it if it is missing. The build and deploy workflows fail if it is empty. | | `SGP_BASE_URL` | Variable | SGP API base URL. SGP sets it to the in-cluster address `http://egp-api-backend.egp.svc.cluster.local/public`. | | `SGP_UI_BASE_URL` | Variable | SGP dashboard URL, used for the evaluation links that the eval workflows post. | | `AGENTEX_API_URL` | Variable | Agentex API URL, used to look up existing agents. SGP sets it to `http://agentex.agentex.svc.cluster.local`. | | `CURRENT_ENVIRONMENT` | Variable | The environment this repository deploys to on merge. SGP sets it only if your install names its environment. When it is unset, the workflows use `dev`. | To configure them: 1. Navigate to your Gitea repository settings 2. Go to **Actions** > **Secrets** 3. Add `SGP_API_KEY` with your SGP API key value 4. Go to **Actions** > **Variables** and check that `SGP_ACCOUNT_ID` is set to your account ID. Add it if it is missing. Set `SGP_ACCOUNT_ID` as a variable, not a secret; the workflows don't read it from secrets. > [!NOTE] > > The workflows read the API base URL from the `SGP_BASE_URL` variable, except the preview deployments that `/eval` comments create, which always use the in-cluster address. If your cluster uses a different service address, change the variable; the workflow files don't need edits. These repository settings are different from the agent-level secrets defined in `manifest.yaml`. Repository secrets authenticate the CI/CD pipeline itself, while agent secrets (in the `credentials` section of `manifest.yaml`) are injected into the running agent container at runtime. ## Adding a new agent To add a new agent to the CI/CD pipeline: 1. **Create the agent directory** under your team's path: ```bash theme={null} mkdir -p teams/my-team/agents/my-new-agent/src ``` 2. **Add a `manifest.yaml`** with your agent's build context, name, and configuration. Use the example above as a starting point and update the `agent.name`, `build.context` paths, and `credentials` section for your agent. 3. **Add an `environments.yaml`** with your deployment target configuration. Set the `namespace`, `auth.principal`, and any `helm_overrides` for your environment. 4. **Add a `Dockerfile`** and `.dockerignore` at the paths referenced in your `manifest.yaml`. The Dockerfile should produce a container that runs your agent. 5. **Add your agent source code** under `src/` or wherever your Dockerfile expects it. 6. **Create the secrets** that your `manifest.yaml` references. SGP rejects a deployment that references a secret that does not exist. See [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets) for setup instructions. 7. **Open or update a pull request** from your feature branch to trigger `cloud-build.yaml`. Verify that the build completes successfully in the Gitea Actions UI. 8. **Merge to main** to trigger `cloud-build-and-deploy-on-push.yaml`, which builds and deploys the agent when its `environments.yaml` contains a single matching deployment target. 9. **Verify the deployment** in the SGP dashboard. Navigate to your agent's detail page to confirm the deployment status is `healthy`. ## Next steps * [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api): access builds and deployments programmatically * [sgpctl CLI](/docs/v5/agents/agentex/cloud-build-and-deploy/sgpctl): build, deploy, and inspect agents from the terminal * [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets): create and rotate credentials referenced in `manifest.yaml` # Cloud Build and Deploy Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/overview Build container images and deploy Agentex agents to SGP-managed Kubernetes clusters from the UI, CLI, CI/CD, or API. ## Overview Cloud Build and Deploy is the SGP-managed pipeline for packaging your Agentex agent code into a container image and deploying it to a Kubernetes cluster. You push your agent source code, SGP builds a container image in the cloud, and deploys it to a managed cluster where it becomes available for traffic. This removes the need to manage Docker registries, Helm charts, or Kubernetes resources directly. SGP handles image building, artifact storage, Helm releases, health checks, and traffic routing. You interact with the system through the SGP dashboard UI, the `sgpctl` CLI, CI/CD pipelines (Gitea Actions), or the Build and Deploy REST API. Cloud Build and Deploy supports both long-lived production deployments and ephemeral preview deployments for branch and PR testing. Builds and deployments are tracked with status history, event timelines, and container logs in the SGP dashboard. The system is built around two configuration files in your agent repository: `manifest.yaml` (what to build and how to deploy) and `environments.yaml` (where to deploy). These files give you declarative control over the build and deploy lifecycle without writing infrastructure code. ## How it works The following diagram shows the end-to-end flow from pushing code to serving traffic: ```mermaid theme={null} sequenceDiagram participant Dev as Developer participant CI as CI/CD (Gitea Actions) participant Build as SGP Cloud Build participant Deploy as SGP Deploy Service participant K8s as Managed K8s Cluster Dev->>CI: Push agent code CI->>CI: Package build context CI->>Build: POST /v5/builds Build->>Build: Build container image Build-->>CI: Build complete (image reference) CI->>Deploy: POST /v5/agentex/deployments Deploy->>K8s: Helm release with built image K8s->>K8s: Pods start, health checks pass K8s-->>Deploy: Agent healthy Deploy-->>Dev: Traffic routed to agent ``` The pipeline has two phases: 1. **Build phase**: Your CI pipeline (or a manual CLI/API call) sends the agent source code to the SGP Build Service via `POST /v5/builds`. SGP builds a container image using the instructions in your `manifest.yaml`, including the base image, build arguments, and any referenced secrets. The resulting image is stored in a managed container registry. You do not need to configure or authenticate with the registry yourself. 2. **Deploy phase**: A deployment is created via `POST /v5/agentex/deployments`. SGP generates a Helm release targeting the Kubernetes cluster and namespace specified in your `environments.yaml`. The deploy service monitors pod scheduling, image pulls, container startup, and health check responses. When all pods report healthy, SGP routes traffic to the new deployment and marks it as `healthy`. By default, the CI/CD pipeline runs both phases automatically on merge to main. On pull requests, the pipeline builds the changed agents, and agents with an `eval.yaml` on `main` also get a preview deployment for the eval. You can also trigger deployments manually from the UI, CLI, or API. You can redeploy a previous build without rebuilding. > [!NOTE] > > A single build can have multiple deployment iterations (for example, if a deployment fails and you redeploy), but all deployments from a given build target the same environment. ## Access methods You can interact with Cloud Build and Deploy through four interfaces. Each supports the full build and deploy lifecycle. | Interface | Best for | Details | | - | - | - | | **SGP Dashboard UI** | Visual monitoring, manual deployments, debugging | Point-and-click access to builds, deployments, logs, and events. No setup required beyond dashboard access. | | **CI/CD (Gitea Actions)** | Automated builds and deploys, team workflows | Build on every pull request, build and deploy on merge to main. The recommended approach for production workflows. | | **`sgpctl` CLI** | Terminal workflows, scripting, debugging | Command-line tool for creating builds, deploying, inspecting status, and tailing logs. Useful for ad-hoc operations and shell scripts. | | **API and SDK** | Programmatic access, custom integrations | REST API and Python/TypeScript SDKs for full control over builds and deployments. Use when you need to integrate with custom tooling or orchestration systems. | All four interfaces operate on the same underlying resources. A build started from the CLI appears in the dashboard UI, and a deployment created from CI/CD can be inspected via the API. In practice, most teams use CI/CD for automated builds and deploys on main, and supplement with the CLI for debugging and the dashboard for monitoring. ## Typical workflow A typical development cycle looks like this: 1. **Develop locally**: Write and test your agent code on your local machine. Use the Agentex local development tools to verify behavior before pushing. 2. **Open a pull request**: The CI/CD pipeline triggers a build-only workflow to validate your agent compiles and packages correctly. 3. **Merge to main**: On merge, the CI/CD pipeline builds the agent and deploys it automatically using the configuration in `environments.yaml`. 4. **Monitor**: The build and deployment appear on the agent detail page in the SGP dashboard. Build logs stream in real-time, and deployment events show rollout progress. 5. **Verify**: Check the deployment detail page for health status. For preview deployments (created separately from the UI, the API/SDK, or the CI/CD preview and eval workflows), use the dedicated preview endpoint. 6. **Iterate**: If the deployment fails, use the events timeline and logs to diagnose the issue, fix the code, and push again. ## Prerequisites Before using Cloud Build and Deploy, ensure the following are in place: * **An Agentex agent repository**: Your agent code must be in a Gitea repository accessible to SGP. See the [Agentex agent types](/docs/v5/agents/agentex/overview) documentation if you have not yet created an agent. * **A `manifest.yaml` file**: Must exist at the root of your agent repository. Defines how SGP builds and deploys your agent. * **An `environments.yaml` file**: Defines the target environment (cluster, namespace, resource limits) for your deployments. * **Account-level secrets (if needed)**: If your agent requires API keys or other credentials, these must be created in the SGP secrets store before your first deployment. If the **Create Secret** dialog shows **Share with entire tenant**, turn it on for these secrets. See [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets). * **SGP API access**: You need a valid SGP API key whose user has the `admin`, `manager`, or `editor` role on the account to create builds and deployments. An agent's first build also registers the agent, which needs `admin` or `manager`. This is configured in your CI/CD pipeline or CLI. > [!TIP] > > If you do not yet have an agent repository, follow the [Agentex agent builder guide](/docs/v5/agents/agentex/types/agentic-agent) to scaffold a new agent project, then return here to set up the build and deploy pipeline. ## Key concepts The following concepts are central to how Cloud Build and Deploy works. ### Build A build represents a single container image build from your agent's source code. When you trigger a build, SGP packages the build context (your source files, `manifest.yaml`, and any referenced configuration), sends it to the build service, and produces a tagged container image. Each build is associated with a specific git commit and branch, providing full traceability from source to artifact. Builds progress through the following statuses: ```mermaid theme={null} stateDiagram-v2 [*] --> queued queued --> running queued --> cancelled queued --> cancelling running --> success running --> failed running --> cancelled running --> cancelling running --> timed_out running --> error running --> unknown cancelling --> cancelled cancelling --> success cancelling --> failed cancelling --> timed_out cancelling --> error deleting --> delete_failed ``` | Status | Description | | - | - | | `queued` | Build request accepted, waiting for capacity | | `running` | Image build is in progress | | `success` | Image built and stored in the managed registry | | `failed` | Build failed due to an error in your code or Dockerfile | | `cancelling` | A cancel request was accepted and SGP is waiting for the cloud provider to stop the build | | `cancelled` | Build was cancelled by a user or automation | | `timed_out` | Build exceeded the maximum allowed duration | | `error` | An internal error occurred during the build | | `unknown` | The cloud provider reported a status SGP does not recognize; inspect the build before deploying | | `deleting` | SGP is deleting the build and its image, for example after the build's agent was deleted | | `delete_failed` | Deleting the build and its image failed | You can delete a build in any status, including `queued` and `running`, if you have permission and no active deployment still uses it. The build moves to `deleting` and is removed when deletion succeeds. If deletion fails, the build moves to `delete_failed`, and you can delete it again. ### Deployment A deployment is a Helm release of a built container image to a Kubernetes cluster. Each deployment creates or updates the Kubernetes resources needed to run your agent: pods, services, ingress rules, and any associated config maps or secrets. The target namespace and resource configuration come from your `environments.yaml`. Deployments progress through the following statuses: ```mermaid theme={null} stateDiagram-v2 [*] --> pending pending --> in_progress pending --> cancelled in_progress --> healthy in_progress --> failed in_progress --> cancelled pending --> superseded in_progress --> superseded failed --> superseded healthy --> superseded healthy --> deleted ``` | Status | Description | | - | - | | `pending` | Deployment created, not yet started | | `in_progress` | Helm release is being applied to the cluster | | `healthy` | Pods are running and passing health checks | | `failed` | Deployment failed (pod crash, resource limits, misconfiguration) | | `cancelled` | Deployment was cancelled | | `superseded` | A newer production deployment with the same Helm release replaced this one | | `deleted` | The preview expired and was cleaned up automatically | A newer deployment can supersede a `pending`, `in_progress`, `healthy`, or `failed` deployment. A preview that expires before it finishes deploying can also end as `deleted`. When you tear down a deployment manually, SGP removes its record, and retrieving it returns HTTP 404. ### Preview deployment A preview deployment is an ephemeral deployment with an automatic TTL (time-to-live) teardown, designed for branch and PR testing. When you create a preview deployment, SGP provisions a dedicated endpoint where you can test the agent version built from your feature branch without affecting the production deployment. Preview deployments are automatically cleaned up when: * The configured TTL expires (default: 8 hours) * You explicitly tear down the deployment This makes preview deployments well-suited for code review workflows where reviewers need a live agent to test against. See [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments) for configuration details. ### Configuration files Two YAML files in your agent repository control the build and deploy lifecycle: * **`manifest.yaml`**: defines your agent's identity, build context, credentials, and deployment defaults. See the [CI/CD guide](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd#manifestyaml) for a full annotated reference. * **`environments.yaml`**: defines the target environment (cluster, namespace, resource limits, Helm overrides). Read at deploy time, not build time, so you can change settings without rebuilding. See the [CI/CD guide](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd#environmentsyaml) for details. Your `manifest.yaml` can reference account-level [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets) that are injected as environment variables at runtime. ## The SGP dashboard The SGP dashboard provides a visual interface for monitoring builds and deployments. Key pages include: * **Agents page**: lists all registered agents with their current deployment status. Access it from **Agents** in the left navigation. * **Agent detail page**: shows build history, active deployment, and git metadata for a specific agent. * **Build detail page**: shows all deployments created from a build, organized by type (production, preview, in-progress). * **Deployment detail page**: provides the events timeline, container logs, status transitions, and configuration summary for a deployment. See the [UI walkthrough](/docs/v5/agents/agentex/cloud-build-and-deploy/ui-walkthrough) for step-by-step flows through these pages. ## What's next This overview covers the core concepts and the UI surfaces for monitoring builds and deployments. The remaining guides in this section walk through specific workflows in detail: | Guide | Description | | - | - | | [UI walkthrough](/docs/v5/agents/agentex/cloud-build-and-deploy/ui-walkthrough) | Step-by-step flows for building and deploying from the SGP dashboard | | [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd) | Automated build and deploy pipelines triggered by git push | | [sgpctl CLI](/docs/v5/agents/agentex/cloud-build-and-deploy/sgpctl) | Build, deploy, and inspect agents from the terminal | | [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api) | Programmatic access to builds and deployments via REST API and SDKs | | [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets) | Store and reference credentials securely | | [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments) | Ephemeral branch deploys for PR testing | If you are setting up Cloud Build and Deploy for the first time, start with the [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd) guide, which walks through `manifest.yaml` configuration and pipeline setup. For terminal-based workflows, the [sgpctl CLI](/docs/v5/agents/agentex/cloud-build-and-deploy/sgpctl) guide covers installation, authentication, and common commands for building, deploying, and tailing logs. # Preview Deployments Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments Spin up short-lived preview deployments per branch or PR with automatic teardown. ## Overview Preview deployments are ephemeral Helm releases designed for testing agent changes in isolation before promoting to production. Each preview gets its own unique Helm release name, so you can run multiple previews simultaneously (one per branch, PR, or experiment) without interfering with your production deployment or with each other. Previews are always time-boxed. Every preview deployment has an `expires_at` timestamp, defaulting to 8 hours from creation if you do not specify one. When the TTL expires, a background cleanup process automatically tears down the deployment and releases its cluster resources. You can also tear down a preview manually at any time, or promote it to a permanent production deployment when you are satisfied with the results. ## How previews differ from production | | Production | Preview | | - | - | - | | Helm release name | Stable name (e.g. `customer-support-bot`) | Unique per-deploy (e.g. `customer-support-bot-pr-482-a3f9`) | | Lifetime | Permanent until replaced | Ephemeral, eligible for automatic cleanup at `expires_at` | | Default TTL | None | 8 hours | | `preview_label` | Not applicable | Optional grouping label (max 30 characters) | | Simultaneous instances | One active production deployment | Multiple previews can coexist | The unique Helm release name is what makes simultaneous previews possible. Production deployments reuse a stable release name, so deploying a new version replaces the previous one. Preview deployments append a deployment ID suffix to the release name, giving each preview its own set of Kubernetes resources. > [!NOTE] > > Preview deployments are supported through the SGP dashboard UI, the [API/SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api), and the Gitea [CI/CD workflows](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): a `/deploy-preview` comment on a pull request, a manual run of `cloud-preview-deploy.yaml`, and the eval workflows. The `sgpctl` CLI does not currently support creating preview deployments. ## Creating a preview from the UI From any successful build's detail page, click **Create Deployment** (or use the chevron dropdown and select **Create preview deployment**). In the deploy modal, toggle **Deploy as preview** to ON. ![Deploy modal with preview toggle enabled, showing preview label field and TTL picker](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/preview-deployments/deploy-modal-preview.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=5d63f162dfc1f73cc0d297b4b490920c) When the preview toggle is enabled, two additional fields appear: * **Preview label**: an optional free-text label (max 30 characters) for grouping related previews. Use your branch name, PR number, or any identifier that helps you find this preview later. In the screenshot, the label is set to `pr-495`. * **TTL**: how long the preview should live before automatic teardown. The default is 8 hours. Click **Deploy preview** to create the deployment. The preview appears in the Deployments list on the build detail page with a **Preview** badge and an "Expires in X hours" indicator. ## Creating a preview from the API To create a preview deployment programmatically, set `preview` to `true` in your `POST /v5/agentex/deployments` request. You can optionally include a `preview_label` and a custom `expires_at` timestamp. As with any deployment, `environment_config` is one entry from under `environments:` in your `environments.yaml`, not the whole file. The Python SDK accepts `preview`, `preview_label`, and `expires_at` in `scale-gp-beta` `0.2.0` or later. ```bash curl theme={null} curl -X POST $SGP_CLIENT_BASE_URL/v5/agentex/deployments \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" \ -H "Content-Type: application/json" \ -d '{ "manifest_file": "kind: Agent\nagent:\n name: my-agent\n...", "environment_config": "environment: dev\nauth:\n principal:\n account_id: your-account-id\n...", "build_id": "build_abc123", "preview": true, "preview_label": "pr-482" }' ``` ```python Python theme={null} from datetime import datetime, timezone, timedelta from scale_gp_beta import SGPClient client = SGPClient( api_key="your-api-key", account_id="your-account-id", ) deployment = client.deploy.create( manifest_file="kind: Agent\nagent:\n name: my-agent\n...", environment_config="environment: dev\nauth:\n principal:\n account_id: your-account-id\n...", build_id="build_abc123", preview=True, preview_label="pr-482", expires_at=(datetime.now(timezone.utc) + timedelta(hours=8)).isoformat(), ) print(f"Preview created: {deployment.id} (expires: {deployment.expires_at})") ``` ```typescript TypeScript theme={null} import SGPClient from 'scale-gp'; const client = new SGPClient({ apiKey: process.env.SGP_API_KEY, accountID: process.env.SGP_ACCOUNT_ID, }); const expiresAt = new Date(Date.now() + 8 * 60 * 60 * 1000).toISOString(); const deployment = await client.deploy.create({ manifest_file: "kind: Agent\nagent:\n name: my-agent\n...", environment_config: "environment: dev\nauth:\n principal:\n account_id: your-account-id\n...", build_id: "build_abc123", preview: true, preview_label: "pr-482", expires_at: expiresAt, }); console.log(`Preview created: ${deployment.id} (expires: ${deployment.expires_at})`); ``` > [!NOTE] > > Preview deployments always have an `expires_at` value. If you omit it, the API defaults to 8 hours from the time of creation. The timestamp must be timezone-aware (include `Z` or a UTC offset like `+00:00`) and must be in the future. Validation rules for preview-specific fields: | Field | Rule | | - | - | | `preview_label` | Can only be set when `preview` is `true`. Maximum 30 characters. Non-unique; multiple deployments can share the same label. | | `expires_at` | Can only be set when `preview` is `true`. Must be an ISO 8601 timestamp with timezone. Must be in the future. | ## Listing previews by label The `preview_label` field is a non-unique grouping label. Multiple preview deployments can share the same label, which is useful when you redeploy the same branch multiple times. To find the latest preview for a given label, query the deployments list endpoint with the `preview_label` filter, `sort_order=desc`, and `limit=1`. Without `sort_order=desc`, the list is sorted oldest first. Use the first page's `items` to inspect the latest match. Iterating a Python SDK page automatically fetches subsequent pages, even when `limit=1`. ```bash curl theme={null} curl -s "$SGP_CLIENT_BASE_URL/v5/agentex/deployments?preview_label=pr-482&limit=1&sort_order=desc" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` ```python Python theme={null} page = client.deploy.list( preview_label="pr-482", limit=1, sort_order="desc", ) deployments = page.items if deployments: latest = deployments[0] print(f"Latest preview: {latest.id} (status: {latest.status})") ``` ```typescript TypeScript theme={null} const page = await client.deploy.list({ preview_label: "pr-482", limit: 1, sort_order: "desc", }); const deployments = page.items; if (deployments.length > 0) { const latest = deployments[0]; console.log(`Latest preview: ${latest.id} (status: ${latest.status})`); } ``` > [!TIP] > > This pattern is useful in CI pipelines. After deploying a preview, your pipeline can poll this endpoint with the branch name as the label to check deployment health before running integration tests. ## Managing preview deployments Every deployment row in the Deployments list has a three-dot actions menu with **Redeploy**, **View Details** and **Copy Deployment ID**. Preview rows also have **Promote to Production** and **Delete Preview**. ![Three-dot actions menu on a preview deployment showing View details, Copy deployment ID, Redeploy as preview, Promote to production, and Tear down preview](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/preview-deployments/deploy-actions-menu.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=0eb60399a6b76e6e79b89043f6eea470) The available actions are: ### Promote to Production Select **Promote to Production** to make the preview the agent's production deployment. The menu shows this action only for healthy previews. ![Promote to production confirmation dialog](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/preview-deployments/promote-dialog.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=7b6662ee04b7f6d90c817695b433badb) Promotion is an atomic cutover: production traffic switches to the preview immediately. No new deployment is created, and the previous production deployment keeps running until the next production deploy replaces it. ### Delete Preview Select **Delete Preview** to immediately delete the deployment and release its cluster resources. A confirmation dialog appears before the teardown proceeds. ![Tear down confirmation dialog](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/preview-deployments/teardown-dialog.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=8c129f311188d504848caac116c23cbb) Manual teardown removes the Kubernetes resources and the deployment record. A subsequent request for that deployment returns HTTP 404. This action cannot be undone. If you need the same configuration again, use **Redeploy** on a previous deployment or create a new preview from the build detail page. Automatic TTL cleanup instead retains the deployment record with status `deleted`. ### Redeploy Select **Redeploy** to open the deploy modal pre-filled with the deployment's manifest and environment configuration. The modal opens with **Deploy as preview** off and an empty preview label, so to redeploy as a preview, turn the toggle on and enter the label again. ## TTL and expiration Every preview deployment has an `expires_at` timestamp that controls its lifetime. When you create a preview without specifying `expires_at`, the API defaults to 8 hours from the time of creation. In the SGP dashboard, preview deployments display an "Expires in X hours" badge so you can see at a glance how much time remains. As the TTL approaches, the badge counts down. When the `expires_at` time is reached, a background cleanup process automatically tears down the preview deployment and releases its cluster resources. You do not need to take any action. Expired previews are cleaned up automatically. If you need more time with a preview, create a new preview deployment with a longer TTL. You cannot extend the `expires_at` of an existing preview. The new preview can reuse the same `preview_label` so it remains easy to find via the API. > [!NOTE] > > Automatic teardown is handled by a background process that runs periodically. There may be a delay between the `expires_at` timestamp and resource cleanup; do not rely on traffic stopping at that exact timestamp. The retained deployment record has status `deleted` after cleanup completes. ## CI/CD integration Preview deployments work well in CI/CD pipelines for automated branch testing. A typical pattern is: 1. On push to a feature branch, build the agent image 2. Deploy the image as a preview with `preview_label` set to the branch name 3. Poll the deployments endpoint to confirm the preview is healthy 4. Run integration tests against the preview endpoint 5. On merge to main, deploy to production (the preview expires naturally or can be torn down) Here is a simplified workflow snippet showing the deploy and verify steps: ```bash theme={null} set -euo pipefail # environment_config takes one entry from under environments:, not the whole file ENV_CONFIG=$(yq '.environments["dev.aws"]' environments.yaml) # Build the JSON payload safely so YAML newlines and quotes are escaped PAYLOAD=$(jq -n \ --rawfile manifest manifest.yaml \ --arg env "$ENV_CONFIG" \ --arg build_id "$BUILD_ID" \ --arg label "$BRANCH_NAME" \ '{manifest_file: $manifest, environment_config: $env, build_id: $build_id, preview: true, preview_label: $label}') # After a successful build, deploy as preview RESPONSE_BODY=$(mktemp) trap 'rm -f "$RESPONSE_BODY"' EXIT if ! HTTP_STATUS=$(curl -sS -o "$RESPONSE_BODY" -w '%{http_code}' \ -X POST "$SGP_CLIENT_BASE_URL/v5/agentex/deployments" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" \ -H "Content-Type: application/json" \ -d "$PAYLOAD"); then echo "Preview creation request failed" >&2 exit 1 fi case $HTTP_STATUS in 2??) ;; *) echo "Preview creation failed: HTTP $HTTP_STATUS" >&2; cat "$RESPONSE_BODY" >&2; exit 1 ;; esac DEPLOY_ID=$(jq -er '.id' "$RESPONSE_BODY") # Poll until the preview is healthy or cannot continue while true; do HTTP_STATUS=$(curl -sS -o "$RESPONSE_BODY" -w '%{http_code}' \ "$SGP_CLIENT_BASE_URL/v5/agentex/deployments/$DEPLOY_ID" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID") case $HTTP_STATUS in 404) echo "Preview deployment was removed" >&2; exit 1 ;; 2??) ;; *) echo "Preview lookup failed: HTTP $HTTP_STATUS" >&2; cat "$RESPONSE_BODY" >&2; exit 1 ;; esac STATUS=$(jq -er '.status' "$RESPONSE_BODY") echo "Preview deploy status: $STATUS" case $STATUS in healthy) echo "Preview is live"; break ;; failed|cancelled|deleted|superseded) echo "Preview deploy ended: $STATUS" >&2; exit 1 ;; pending|in_progress) ;; *) echo "Unexpected preview status: $STATUS" >&2; exit 1 ;; esac sleep 10 done ``` > [!TIP] > > Set the `preview_label` to your branch name or PR number in CI. This makes it straightforward to look up the latest preview for any branch using the list endpoint with the label filter, and it keeps the SGP dashboard organized when multiple branches have active previews. ## Next steps * [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api): full request and response schemas for preview deployment fields * [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): workflow templates for automated builds * [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets): store and reference credentials used during deployments # Secrets Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/secrets Store credentials as SGP Cloud Secrets and reference them from manifest.yaml to inject environment variables at runtime. ## Overview Agents typically need credentials to function: API keys for LLM providers, database connection strings, authentication tokens for external services. SGP provides a two-layer system for handling these securely. 1. **SGP Cloud Secrets** are account-level key-value pairs stored in your cloud provider's secret manager. You create them once, and any agent in your account can reference them. On accounts where the **Create Secret** dialog shows **Share with entire tenant** (the tenant is your SGP account), agents can reference only shared secrets. 2. **Manifest credentials** are per-agent mappings defined in `manifest.yaml` that bind an SGP Cloud Secret to an environment variable injected into the agent's container at runtime. You create secrets through the SGP dashboard or the Secrets API, then reference them by name in your agent's `manifest.yaml`. At deploy time, SGP syncs the secret value into a Kubernetes secret and mounts it as the specified environment variable in the agent's pod. Your agent code reads the environment variable as it would any other. ## How secrets flow The following diagram traces a secret from creation through to your running agent code. ```mermaid theme={null} flowchart LR A[Create secret via UI or API] --> B[Value stored in cloud secret manager] B --> C[manifest.yaml references secret_name] C --> D[Deploy triggered] D --> E[SGP syncs value to K8s secret] E --> F[Env var injected in pod] F --> G[Agent reads env var] ``` The value you provide when creating a secret is written directly to the cloud provider's secret store (such as Azure Key Vault). It is never stored in the SGP database, and the v5 Secrets API never returns it. At deploy time, the deploy service reads the value from the cloud secret store, creates or updates a Kubernetes secret in the target namespace, and mounts it as the environment variable specified in your `manifest.yaml`. ## Managing secrets in the UI Navigate to **Assets > Secrets** in the SGP dashboard. The secrets table lists all SGP Cloud Secrets for your account. ![Secrets table showing account-level secrets with key, description, created by, and modified columns](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/secrets/secrets-list.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=c0b2e68ed20ec85cff238162e061a9d7) Each row displays: * **Key**: the unique identifier for the secret (for example, `anthropic-api-key` or `redis-url-secret`) * **Description**: an optional note explaining what the secret is used for * **Created by**: the user who created the secret * **Modified**: when the secret was last updated From this table you can edit a secret (to update its value or description) or delete it. ## Creating a secret Click **Create Secret** at the top of the secrets table. Fill in the required fields in the dialog. ![Create Secret dialog with key, value, and description fields](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/secrets/secret-create.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=852f65be0fb8fe569670d8710e78b11a) | Field | Required | Description | | - | - | - | | **Key** | Yes | A unique name for the secret. Lowercase letters, digits, and hyphens only, starting and ending with a letter or digit. At most 90 characters, or 40 on accounts where this dialog shows **Share with entire tenant**. | | **Share with entire tenant** | No | Shown only on some accounts. Off by default, which keeps the secret private to you and means it cannot be used as a deployment credential. Turn it on for any secret an agent's `manifest.yaml` references. Anyone in the account can read a shared secret's value. Scope cannot be changed after the secret is created. | | **Value** | Yes | The credential value. This is encrypted and stored in the cloud secret manager. It is never displayed again after creation. | | **Description** | No | A human-readable note. Recommended so that other team members understand the secret's purpose. | Click **Create Secret** to save. The new secret appears in the table immediately and is available for agents to reference. > [!NOTE] > > The dashboard never shows a secret's value after creation, and the v5 Secrets API never returns it: you can update a value but not retrieve it. If you lose the original value, you must generate a new credential from the source system and update the secret. On accounts that show **Share with entire tenant**, anyone in the account can read a shared secret's value, and the creator can read a private one. ## Deleting a secret Click the delete action on a secret's row. A confirmation dialog requires you to type the secret's key to proceed. ![Delete confirmation dialog requiring the secret name to be typed for confirmation](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/secrets/secret-delete.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=a9070d29223c57e8b905ff797e6b805b) > [!WARNING] > > Deleting a secret is irreversible. Any agent whose `manifest.yaml` references the deleted secret will fail on its next deployment because the secret can no longer be resolved. Verify that no agents depend on a secret before deleting it. ## Secrets API You can manage secrets programmatically through the SGP Cloud Secrets API. The following examples cover the most common operations. > [!WARNING] > > **Deprecation notice:** The v5 Secrets API (`/v5/sgp/secrets`) and the `client.secrets` methods in the Python and TypeScript SDKs are deprecated. Create secrets in the dashboard instead. If the **Create Secret** dialog shows **Share with entire tenant**, deployments read only shared secrets, and secrets created through the v5 API aren't found at deploy time. ### Create a secret ```bash cURL theme={null} curl -X POST "$SGP_CLIENT_BASE_URL/v5/sgp/secrets" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" \ -H "Content-Type: application/json" \ -d '{ "key": "datadog-api-key", "value": "dd-api-xxxxxxxxxxxxxxxxxxxxxxxxxx", "description": "Datadog API key for agent telemetry" }' ``` ```python Python theme={null} from scale_gp_beta import SGPClient client = SGPClient() secret = client.secrets.create( key="datadog-api-key", value="dd-api-xxxxxxxxxxxxxxxxxxxxxxxxxx", description="Datadog API key for agent telemetry", ) print(secret.id, secret.key) ``` ```typescript TypeScript theme={null} import SGPClient from 'scale-gp'; const client = new SGPClient({ apiKey: process.env.SGP_API_KEY, accountID: process.env.SGP_ACCOUNT_ID, }); const secret = await client.secrets.create({ key: "datadog-api-key", value: "dd-api-xxxxxxxxxxxxxxxxxxxxxxxxxx", description: "Datadog API key for agent telemetry", }); console.log(secret.id, secret.key); ``` The response includes the secret's `id`, `key`, `description`, and audit metadata. The `value` field is never returned. ```json theme={null} { "id": "sec_abc123", "key": "datadog-api-key", "description": "Datadog API key for agent telemetry", "cloud_secret_path": "/sgp/secrets/datadog-api-key", "created_by": "usr_def456", "created_at": "2026-05-20T14:30:00Z", "updated_by": "usr_def456", "updated_at": "2026-05-20T14:30:00Z", "object": "sgp_cloud_secret" } ``` ### Update a secret Use `PATCH /v5/sgp/secrets/{secret_id}` to rotate a value or update the description. ```bash cURL theme={null} curl -X PATCH "$SGP_CLIENT_BASE_URL/v5/sgp/secrets/sec_abc123" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" \ -H "Content-Type: application/json" \ -d '{ "value": "dd-api-yyyyyyyyyyyyyyyyyyyyyyyyyy", "description": "Datadog API key for agent telemetry (rotated May 2026)" }' ``` ```python Python theme={null} from scale_gp_beta import SGPClient client = SGPClient() secret = client.secrets.update( "sec_abc123", value="dd-api-yyyyyyyyyyyyyyyyyyyyyyyyyy", description="Datadog API key for agent telemetry (rotated May 2026)", ) print(secret.key, secret.updated_at) ``` ```typescript TypeScript theme={null} import SGPClient from 'scale-gp'; const client = new SGPClient({ apiKey: process.env.SGP_API_KEY, accountID: process.env.SGP_ACCOUNT_ID, }); const secret = await client.secrets.update("sec_abc123", { value: "dd-api-yyyyyyyyyyyyyyyyyyyyyyyyyy", description: "Datadog API key for agent telemetry (rotated May 2026)", }); console.log(secret.key, secret.updated_at); ``` ### List secrets ```bash cURL theme={null} curl "$SGP_CLIENT_BASE_URL/v5/sgp/secrets" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` ```python Python theme={null} from scale_gp_beta import SGPClient client = SGPClient() secrets = client.secrets.list() for secret in secrets: print(secret.key, secret.description) ``` ```typescript TypeScript theme={null} import SGPClient from 'scale-gp'; const client = new SGPClient({ apiKey: process.env.SGP_API_KEY, accountID: process.env.SGP_ACCOUNT_ID, }); const secrets = await client.secrets.list(); for await (const secret of secrets) { console.log(secret.key, secret.description); } ``` For the full API schema including pagination, filtering, and delete operations, see the [SGP Cloud Secrets API reference](/reference/v5/sgp-cloud-secrets/list-secrets). ## Referencing secrets in manifest.yaml Once a secret exists in the SGP secrets store, you reference it from your agent's `manifest.yaml` in the `agent.credentials` array. Each entry maps an SGP Cloud Secret to an environment variable that your agent code can read. ```yaml theme={null} agent: credentials: - env_var_name: ANTHROPIC_API_KEY secret_name: anthropic-api-key secret_key: api-key - env_var_name: OPENAI_API_KEY secret_name: openai-api-key secret_key: api-key - env_var_name: DATABASE_URL secret_name: redis-url-secret secret_key: url ``` | Field | Description | | - | - | | `env_var_name` | The environment variable name injected into the agent's container. Your code reads this with `os.environ["ANTHROPIC_API_KEY"]` or equivalent. | | `secret_name` | The `key` of the SGP Cloud Secret as it appears in the secrets table. Must match exactly. | | `secret_key` | The sub-key within the Kubernetes secret object. This is typically `api-key`, `token`, or `url`, depending on how the secret was provisioned. | At deploy time, SGP resolves each `secret_name` to its corresponding cloud secret, syncs the value into a Kubernetes secret in the agent's namespace, and mounts the value at the specified `secret_key` as the environment variable `env_var_name` in the pod. > [!NOTE] > > The `secret_key` field refers to the key within the Kubernetes secret data map, not a secondary lookup within the cloud secret store. Each SGP Cloud Secret maps to one Kubernetes secret, and `secret_key` selects which data entry within that Kubernetes secret to mount. ## Rotating a secret To rotate a credential: 1. Generate a new credential value from the source system (for example, your LLM provider's dashboard or your database admin panel). 2. Update the SGP Cloud Secret with the new value using the API (`PATCH /v5/sgp/secrets/{secret_id}`) or the **Edit** action in the secrets table UI. 3. Redeploy any agents that reference the rotated secret. The next deployment picks up the new value automatically. Existing running pods continue to use the previous value until they are replaced by a new deployment. Updating a secret does not trigger automatic redeployment. You must explicitly redeploy each affected agent. > [!TIP] > > To minimize downtime during rotation, update the secret value first, then redeploy agents one at a time. If the new credential is invalid, restore a valid secret value before redeploying, or point the manifest at another secret that holds a valid credential. Deployments always read the secret's current value, so redeploying an older build doesn't restore the old credential. ## Security model SGP Cloud Secrets are protected as follows: * **Values**: the v5 Secrets API never returns secret values, and the UI never displays them after creation. On accounts that show **Share with entire tenant**, every user in the account can read a shared secret's value; a private secret is readable only by its creator. The SGP database stores only metadata (key, description, audit fields). The actual value lives exclusively in the cloud provider's secret manager. * **Encrypted at rest**: values are stored using the cloud provider's native encryption (for example, Azure Key Vault with platform-managed keys). * **Injected as Kubernetes secrets**: at deploy time, values are synced into Kubernetes secrets and mounted as environment variables. They are not written to disk or config maps in the cluster. * **Role-based access**: only users with secret management permissions can create, update, or delete secrets. Read access to secret metadata (key and description) is available to users who can view the account's assets. * **Audit trail**: every create, update, and delete operation is recorded with the acting user and timestamp. ## Next steps * [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): full `manifest.yaml` reference including the `credentials` section * [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api): programmatic access to builds, deployments, and secrets * [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments): ephemeral deployments for branch and PR testing # sgpctl CLI Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/sgpctl Build, deploy, and observe Agentex agents from your terminal with sgpctl agentex-cloud. ## Overview `sgpctl` is the command-line interface for managing Agentex cloud builds and deployments. It covers the same operations available through the SGP dashboard and the API/SDK, packaged for terminal workflows. The CLI is organized under `sgpctl agentex-cloud` with two subcommand groups: * **`build`**: submit cloud builds, list builds, stream build logs, and cancel running builds. * **`deploy`**: submit deployments, list deployments, stream deploy logs, and watch rollouts in real time. Every command outputs structured data (JSON or Rich tables) that you can pipe into other tools or inspect directly. ## Setup ### Install sgpctl **Homebrew (macOS):** ```bash theme={null} brew tap scaleapi/sgpctl https://github.com/scaleapi/sgpctl brew install sgpctl ``` **Binary download (Linux/Windows):** Download the latest binary for your platform from the [sgpctl GitHub releases](https://github.com/scaleapi/sgpctl/releases/latest). > [!WARNING] > > The latest release, v0.0.4, is also the version Homebrew installs. Its `build submit` always sends the agent name, so it can only run an agent's first build: the API rejects a build by name for an agent that already exists. To build an existing agent, use the [API](/docs/v5/agents/agentex/cloud-build-and-deploy/api) with `agent_id` or the [CI/CD workflows](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd). ### Configure environment variables All `sgpctl agentex-cloud` commands require three environment variables: ```bash theme={null} export SGP_API_KEY="your-api-key" export SGP_ACCOUNT_ID="your-account-id" export SGP_BASE_URL="https://api.your-sgp-instance.scale.com" ``` > [!NOTE] > > All three environment variables are required. Commands will fail immediately if any of them is missing. You can add these exports to your shell profile (`~/.bashrc`, `~/.zshrc`) or load them from a `.env` file in your CI pipeline. ## Build commands Build commands live under `sgpctl agentex-cloud build`. They let you submit new image builds, query build status, stream logs, and cancel running builds. ### Submit a build Package your agent source code and submit it for a cloud build: ```bash theme={null} sgpctl agentex-cloud build submit \ --manifest ./manifest.yaml \ --image-name my-agent \ --tag v1.2.0 ``` The `submit` command reads your `manifest.yaml`, packages the build context, uploads the resulting tar.gz archive, and creates a build via `POST /v5/builds`. On success, the command prints `Build created:` followed by the full build record as JSON. Use its `id` as the build ID in other commands. An abbreviated example: ```json theme={null} { "id": "8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02", "agent_name": "my-agent", "build_status": "queued", "image_name": "my-agent", "image_tag": "v1.2.0" } ``` **Options:** | Flag | Description | | - | - | | `--manifest`, `-m` | Path to `manifest.yaml` (required) | | `--image-name`, `-i` | Repository name for the built image (required) | | `--tag`, `-t` | Image tag (default: `latest`) | | `--agent-name`, `-n` | Name of a brand-new agent. Use it only for the agent's first build. | | `--build-arg`, `-b` | Build argument as `KEY=VALUE`. Takes a single value, and builds that pass it currently fail with a 400 because the API expects build arguments as a JSON object. | ### Get build details Retrieve the full details of a specific build: ```bash theme={null} sgpctl agentex-cloud build get 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 ``` Returns the build object as JSON, including status, image URL, timestamps, and associated metadata. ### List builds List recent builds, optionally filtered by agent name: ```bash theme={null} sgpctl agentex-cloud build list --agent-name my-agent --limit 10 ``` Example output: | Build ID | Agent Name | Build Status | Image Name | Image Tag | Image URL | Created At | | - | - | - | - | - | - | - | | `8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02` | my-agent | success | my-agent | v1.2.0 | registry.sgp.scale.com/… | 2026-05-20 14:32:10 | | `6d1b4e82-f3a9-4c8b-a2e5-9d7f1b3c8a04` | my-agent | success | my-agent | v1.1.0 | registry.sgp.scale.com/… | 2026-05-18 09:15:43 | | `2c7f9a31-d5e8-4a1f-b6c3-8e2d4f7a9b01` | my-agent | failed | my-agent | v1.0.1 | | 2026-05-17 16:22:05 | **Options:** | Flag | Description | | - | - | | `--agent-name`, `-n` | Filter by agent name | | `--limit`, `-l` | Maximum number of results (default: 20) | | `--sort-order` | Sort direction: `asc` or `desc` | ### Stream build logs Stream real-time build logs for a running or completed build: ```bash theme={null} sgpctl agentex-cloud build logs 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 ``` Logs are delivered via Server-Sent Events (SSE) and print to stdout as they arrive: ``` [2026-05-20 14:32:15] Step 1/8 : FROM python:3.11-slim [2026-05-20 14:32:16] ---> Using cache [2026-05-20 14:32:16] Step 2/8 : WORKDIR /app [2026-05-20 14:32:17] Step 3/8 : COPY requirements.txt . [2026-05-20 14:32:18] Step 4/8 : RUN pip install -r requirements.txt [2026-05-20 14:32:45] Step 5/8 : COPY . . [2026-05-20 14:32:46] Step 6/8 : RUN python -m compileall . [2026-05-20 14:32:48] Step 7/8 : EXPOSE 8080 [2026-05-20 14:32:48] Step 8/8 : CMD ["python", "main.py"] [2026-05-20 14:32:50] Successfully built a1b2c3d4e5f6 [2026-05-20 14:32:52] Successfully tagged registry.sgp.scale.com/my-agent:v1.2.0 ``` The command exits when the build completes or the timeout is reached. | Flag | Description | | - | - | | `--timeout` | Maximum seconds to wait for logs (default: 300) | ### Cancel a build Cancel a build that is queued or running: ```bash theme={null} sgpctl agentex-cloud build cancel 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 ``` The build moves to `cancelling`, then to `cancelled`, or to the status it reached if it finished first. Builds that have already completed cannot be cancelled. ## Deploy commands Deploy commands live under `sgpctl agentex-cloud deploy`. They let you submit deployments, check deployment status, stream logs, and watch rollouts. ### Submit a deployment Deploy an agent using a completed build: ```bash theme={null} sgpctl agentex-cloud deploy submit \ --agent-path ./my-agent \ --env production \ --build-id 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 ``` The `submit` command reads `manifest.yaml` and `environments.yaml` from the directory specified by `--agent-path`, then creates a deployment via `POST /v5/agentex/deployments`. Alternatively, select an existing successful build in your account by image name and tag. The registry image must already have a build record with `build_status=success`: ```bash theme={null} sgpctl agentex-cloud deploy submit \ --agent-path ./my-agent \ --env staging \ --image-name my-agent \ --tag v1.2.0 ``` **Options:** | Flag | Description | | - | - | | `--agent-path`, `-a` | Path to agent directory containing `manifest.yaml` and `environments.yaml` (required) | | `--env`, `-e` | Environment key from `environments.yaml` (required) | | `--build-id`, `-b` | Build ID to deploy | | `--image-name`, `-i` | Image repository name (alternative to `--build-id`) | | `--tag`, `-t` | Image tag (used with `--image-name`) | | `--watch`, `-w` | Enter the live TUI dashboard after submitting | > [!NOTE] > > You must provide either `--build-id` or both `--image-name` and `--tag`. The command will reject the request if neither or both are specified. ### Watch a deployment roll out Pass `--watch` to `deploy submit` to enter a fullscreen Rich dashboard immediately after the deployment is created: ```bash theme={null} sgpctl agentex-cloud deploy submit \ --agent-path ./my-agent \ --env production \ --build-id 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 \ --watch ``` The TUI dashboard shows three sections: a status header at the top (agent name, environment, deployment ID, current status), a Kubernetes events panel on the left, and a container logs panel on the right. Both panels update in real time as the deployment progresses. The dashboard auto-refreshes until you press `Ctrl+C`. It keeps running after the deployment reaches a terminal state such as `healthy` or `failed`. ### Get deployment details Retrieve the full details and events for a specific deployment: ```bash theme={null} sgpctl agentex-cloud deploy get 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05 ``` Returns the deployment object as JSON, including status, events timeline, and configuration. ### List deployments List retained deployment history. Results use the API default order, oldest first, and can include failed or superseded attempts, plus expired previews with status `deleted` after automatic cleanup. Manually deleted records are removed from the history. ```bash theme={null} sgpctl agentex-cloud deploy list --limit 10 ``` Example output: | ID | Agent | Status | Namespace | Created At | | - | - | - | - | - | | `1b8c3a62-f4d9-4e7c-a5b8-3d9f2c6e1a08` | support-bot | failed | prod-agents | 2026-05-19 09:45:33 | | `9e2d5f81-a3c7-4b9e-d1f4-6c8a2e5b7d03` | my-agent | healthy | staging-agents | 2026-05-19 11:20:15 | | `4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05` | my-agent | healthy | prod-agents | 2026-05-20 14:33:00 | **Options:** | Flag | Description | | - | - | | `--limit`, `-n` | Number of results requested per page (default: 20); the command enumerates all pages | ## Streaming logs Both build and deploy commands support log streaming, but they use different transport mechanisms. ### Build logs (SSE) Build logs stream via Server-Sent Events. The `build logs` command opens a persistent connection and prints each log line as it arrives: ```bash theme={null} sgpctl agentex-cloud build logs 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 --timeout 600 ``` The connection closes when the build completes or the timeout expires. This is useful for watching a build in progress from your terminal. ### Deploy logs (cursor-paged polling) Deploy logs use cursor-based polling. The `deploy logs` command fetches new log lines at a configurable interval: ```bash theme={null} sgpctl agentex-cloud deploy logs 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05 ``` By default, the output renders in a Rich Live TUI panel that updates in place. For CI environments or piping to a file, use the `--plain` flag: ```bash theme={null} sgpctl agentex-cloud deploy logs 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05 --plain ``` In plain mode, each log line is written to stdout as a single line with no formatting, making it suitable for CI log capture or `grep`: ``` 2026-05-20T14:33:24Z Starting application... 2026-05-20T14:33:25Z Loading model weights... 2026-05-20T14:33:28Z Model loaded in 3.2s 2026-05-20T14:33:28Z Health check endpoint ready 2026-05-20T14:33:28Z Listening on 0.0.0.0:8080 ``` **Options:** | Flag | Description | | - | - | | `--poll-interval` | Seconds between log fetches (default: 2) | | `--plain` | Plain text output, no TUI (recommended for CI) | ## Common workflows ### Ship from laptop Build an image and deploy it in one sequence: ```bash theme={null} # Submit the build sgpctl agentex-cloud build submit \ --manifest ./manifest.yaml \ --image-name my-agent \ --tag v1.2.0 # Stream logs until the build completes sgpctl agentex-cloud build logs 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 --timeout 600 # Deploy and watch the rollout sgpctl agentex-cloud deploy submit \ --agent-path ./my-agent \ --env production \ --build-id 8f3a2c91-e4b7-4d2a-9f1c-3b5e7a8d6c02 \ --watch ``` After the build completes successfully, `--watch` shows the rollout and continues streaming status and logs until you press `Ctrl+C`. ### Check what is deployed List deployment history and check each deployment's status. The list doesn't show by itself which deployment is serving traffic: when you submit a new deployment, earlier non-preview deployments of the same agent in that environment change to `superseded` before the new one finishes rolling out. For newest-first results, use the [deployment list API](/docs/v5/agents/agentex/cloud-build-and-deploy/api#list-deployments) with `sort_order=desc`: ```bash theme={null} sgpctl agentex-cloud deploy list ``` To get full details on a specific deployment, including the image tag and environment configuration: ```bash theme={null} sgpctl agentex-cloud deploy get 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05 ``` ### Roll back to a previous build Redeploy a known-good build by referencing its build ID: ```bash theme={null} sgpctl agentex-cloud deploy submit \ --agent-path ./my-agent \ --env production \ --build-id 6d1b4e82-f3a9-4c8b-a2e5-9d7f1b3c8a04 \ --watch ``` This creates a new deployment using the image from the previous build. The existing deployment is replaced when the new one becomes healthy. ### Tail logs on a running deployment Stream container logs from a deployment that is already running: ```bash theme={null} sgpctl agentex-cloud deploy logs 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05 ``` Use `--plain` to capture logs in a file: ```bash theme={null} sgpctl agentex-cloud deploy logs 4a9c1b72-e3f8-4d5a-8c2b-1f6e9d3a7b05 --plain > deploy.log ``` ## Next steps * [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api): programmatic access to builds and deployments * [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): automate builds triggered by git push * [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets): store and reference credentials used during builds and deployments # UI Walkthrough Source: https://docs.gp.scale.com/docs/v5/agents/agentex/cloud-build-and-deploy/ui-walkthrough Step-by-step guide to building and deploying agents through the SGP web interface. ## Overview This page walks through the Cloud Build and Deploy workflow in the SGP dashboard. The navigation flow is: ``` Agents grid → Agent detail → Build detail → Deploy modal → Deployment detail ``` **Prerequisites:** * An SGP account with access to the Agents tab * At least one Agentex agent registered to your account * A `manifest.yaml` and `environments.yaml` prepared for your agent (see the [CI/CD guide](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd) for file format details) ## Viewing your agents Open the **Agents** tab from the top navigation bar. This shows all Agentex agents registered to your account. ![Agents grid showing registered Agentex agents with status badges and descriptions](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/overview/agents-grid.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=fdbc8abc6989128250be4340ec47af6a) Click any agent card to open its detail page. ## Agent detail and build history The agent detail page has a metadata header at the top and the build history timeline below. ![Agent detail page showing header metadata and build history timeline](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/overview/agent-detail.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=754ddf130eeea81ffc02b4b0a3403d8f) The action buttons in the header link to the agent's live endpoint (**Open Agent**), its request traces (**View Traces**), and its performance profiler (**View Profiler**). Open Agent is only available when the agent has an active deployment. The **Build History** lists every build for this agent, most recent first. The status badges map to: | Status | Meaning | | - | - | | **Building** | Image build is in progress | | **Deploying** | Build succeeded, deployment is actively rolling out | | **Error** | Build failed during the image build phase | | **Inactive** | Build succeeded but is not currently deployed | | **Success** | Build completed successfully (visible on build detail page) | Click **+ Add Build** to trigger a new cloud build from the UI. > [!NOTE] > > Most teams use CI/CD pipelines for automated builds and reserve the UI button for ad-hoc builds during development. Click any build row to open its build detail page. ## Deploying a build The build detail page shows build metadata, build logs, and all deployments created from this build. ![Build detail page showing a successful build with no deployments and a Create deployment button](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/ui-walkthrough/build-no-deploys.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=1c4c21d09852630dc8498affca101a8d) The **Build Logs** section is collapsible and includes a search bar for filtering output. Use the build duration in the header to distinguish early failures (missing Dockerfile, bad config) from late failures (dependency errors). To deploy, click **Create Deployment** in the Deployments section to open the deploy modal. ![Deploy modal showing preview toggle, manifest file section, and environment variables section](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/ui-walkthrough/deploy-modal-fresh.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=7eeca4fcf13b352c509d5b8cf762bfa9) The modal has two collapsible YAML sections: **Manifest File** (`manifest.yaml`) and **Environment Variables** (one entry from `environments.yaml`). * On your first deployment, paste your `manifest.yaml` content into **Manifest File**, and paste one environment entry from under `environments:` in your `environments.yaml` into **Environment Variables**. A whole `environments.yaml` file is rejected. * On subsequent deployments, both sections pre-populate from the last deployment. You can deploy without expanding them if no changes are needed. * Enable the **Deploy as preview** toggle to create an ephemeral deployment with an automatic TTL instead of a production deployment. See [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments). > [!WARNING] > > SGP rejects a deploy request whose manifest references secrets that don't exist, and the error names the missing secrets. If a secret is deleted after submission, resolving its value can fail during deployment. Ensure secrets are created in the SGP secret store before deploying. See [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets). > > The SGP secret store is off by default. While it is off, SGP rejects any manifest that sets `agent.credentials`, with an error that starts "This deployment has no account secret store". Ask your Scale team to enable it for your install. ## Monitoring a deployment The build detail page lists all deployments from this build. Reload the page to see status updates. ![Build detail page showing multiple deployments with mixed statuses](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/overview/build-detail.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=da847f2fb72513b40996c59cdab55f8a) Production deployments show a gray badge; preview deployments show a purple badge with the PR number and an expiration countdown. Preview deployments are automatically torn down when the TTL expires. Click any deployment row to inspect its event timeline and logs. ![Deployment detail page showing metadata, deployment events timeline, and deploy logs section](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/overview/deploy-detail.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=9652c8d54a7674f9ed629f2f192f6b33) The **Deployment Events** section shows the Kubernetes event sequence for the rollout. A successful deployment progresses through: Scheduled → Pulling → Pulled → Created → Started → ScalingReplicaSet → ReleaseSuccessful. The **Deploy Logs** section shows container runtime output. If the deployment is healthy and the agent produces no stdout on startup, this section may show "No logs available." ### Diagnosing a failing deployment A gap in the event timeline tells you where the problem is: | Last event seen | Likely cause | | - | - | | **Scheduled** | Image could not be pulled. Check the image reference and registry access. | | **Pulling** | Image pull is timing out. Large images or registry rate limits. | | **Started** | Health check is not passing. Check Deploy Logs for runtime errors. | | **ScalingReplicaSet** (no ReleaseSuccessful) | Helm timeout or release hook issue. | > [!TIP] > > If a deployment stays in Deploying longer than expected, check Deployment Events. Missing events after Scheduled usually mean node resource constraints or image pull failures. ## Diagnosing a failed build Failed builds show a **Failed** badge on the build detail page and an **Error** badge in the Build History list (same state, different labels). ![Failed build detail page showing error status and build logs section](https://mintcdn.com/scalegp/vRvys9sF3uVJSFBb/images/v5/cloud-build-and-deploy/ui-walkthrough/build-failed.png?fit=max\&auto=format\&n=vRvys9sF3uVJSFBb\&q=85\&s=c93e974b0d64acef2d3538949766eabd) The **Create Deployment** button is disabled for failed builds. Check the **Build Logs** for error output. A short build duration (under 30 seconds) usually points to a configuration error (missing Dockerfile, invalid manifest) rather than a code error. Common causes: * **Dockerfile not found**: `manifest.yaml` references a path that doesn't exist in the build context * **Dependency install failure**: `pip install`, `npm install`, etc. fails * **Build context too large**: add a `.dockerignore` to exclude unnecessary files * **Invalid manifest**: syntax errors or invalid field values in `manifest.yaml` * **Base image unavailable**: the Dockerfile's base image can't be pulled * **Timeout**: build exceeded the maximum duration > [!TIP] > > If builds fail repeatedly, verify the agent builds locally first with `docker build`. This surfaces issues faster than iterating through the cloud pipeline. ## Next steps * [Preview Deployments](/docs/v5/agents/agentex/cloud-build-and-deploy/preview-deployments): ephemeral branch deployments for PR testing * [Secrets](/docs/v5/agents/agentex/cloud-build-and-deploy/secrets): store and reference credentials * [CI/CD with Gitea Actions](/docs/v5/agents/agentex/cloud-build-and-deploy/ci-cd): automate builds triggered by git push * [API and SDK](/docs/v5/agents/agentex/cloud-build-and-deploy/api): programmatic access to builds and deployments # Introduction to Agentex Source: https://docs.gp.scale.com/docs/v5/agents/agentex/overview Learn about Agentex agent types and how to build them Welcome to the Agentex client documentation. This guide will help you get started with building and interacting with Agentex agents. ## Quick Navigation * 📘 **[Sync Agent Guide →](/docs/v5/agents/agentex/types/sync-agent)** - For simple, direct interactions * 📗 **[Async Agent Guide →](/docs/v5/agents/agentex/types/agentic-agent)** - For complex workflows and production apps **Not sure which to use?** Learn more about [choosing the right agent type](https://agentex.sgp.scale.com/docs/development_guides/tutorials/). # Agentic Agent Source: https://docs.gp.scale.com/docs/v5/agents/agentex/types/agentic-agent How to interact with async Agentex agents using the Agentex SDK > [!WARNING] > > **Deprecation notice:** The `agentic` ACP type is deprecated. Use `async`, which behaves the same. ## Overview Async agents use an **async ACP** architecture that: * Supports both streaming and non-streaming responses * Requires explicit task creation before sending events * Ideal for complex workflows and long-running operations ### Key API Method: `send_event()` For async agents, you always use the **`send_event()`** method to communicate with the agent. This is different from sync agents, which use `send_message()` for direct communication. ```python theme={null} # Async agents use send_event() # 1. Create a task first (REQUIRED for async agents) create_task_response = await client.agents.create_task(agent_id=agent.id, params={}) task = create_task_response.result # 2. Send events to the task await client.agents.send_event( agent_id=agent.id, params={ "task_id": task.id, "content": {"type": "text", "author": "user", "content": "Hello!"} } ) ``` **Key Differences from Sync Agents:** * **Task creation required** - Must create a task before sending events (sync agents don't need this) * **Event-based communication** - Send events to tasks, not direct messages to agent names * **Asynchronous processing** - Events are processed asynchronously, poll or stream for responses The event-based model enables asynchronous processing, state management across multiple turns, and complex workflow orchestration. ### ACP Types: Base vs Temporal Async agents can be implemented using two different backend types: * **Base ACP** - Simple event-driven architecture, suitable for learning and simple use cases * **Temporal** - Robust workflow engine for production, handles race conditions automatically **Important:** From the client's perspective, both base ACP and Temporal agents use the **exact same API** (both use `send_event()`). The choice between base and Temporal is purely a server-side implementation detail that is transparent to client code. You can switch an agent from base to Temporal (or vice versa) without changing any client code. ## Setup ### Prerequisites ```bash theme={null} pip install agentex-sdk ``` ### Environment Variables ```bash theme={null} export AGENTEX_API_KEY="your-api-key" export AGENTEX_BASE_URL="agentex-base-url" export SGP_ACCOUNT_ID="your-account-id" ``` ### Initialize the Client ```python theme={null} import asyncio import os from agentex import AsyncAgentex from agentex.types.text_content import TextContent # Initialize the async Agentex client client = AsyncAgentex( base_url=os.environ.get("AGENTEX_BASE_URL"), default_headers={ "x-api-key": os.environ.get("AGENTEX_API_KEY"), "x-selected-account-id": os.environ.get("SGP_ACCOUNT_ID") } ) ``` ## Basic Usage ### 1. Get Agent Information ```python theme={null} from agentex.types import Agent # List all agents agents = await client.agents.list() for agent in agents: print(f"Agent: {agent.name} - {agent.id} - {agent.acp_type}") # Retrieve a specific agent by name AGENT_NAME = "" # Replace with your agent name agent = await client.agents.retrieve_by_name(agent_name=AGENT_NAME) print(f"Agent ID: {agent.id}") print(f"Agent Type: {agent.acp_type}") print(f"Description: {agent.description}") ``` ### 2. Create a Task Async agents require a task to be created before sending events: ```python theme={null} import uuid # Create a new task create_task_response = await client.agents.create_task( agent_id=agent.id, params={ "name": f"{uuid.uuid4()}-task", "params": {} } ) task = create_task_response.result print(f"Task ID: {task.id}") print(f"Task Status: {task.status}") ``` ### 3. Send an Event ```python theme={null} # Create event content event_content = TextContent( type="text", author="user", content="Hello! Can you explain what quantum computing is?" ) # Send the event to the task event_response = await client.agents.send_event( agent_id=agent.id, params={ "task_id": task.id, "content": event_content } ) event = event_response.result print(f"Event ID: {event.id}") print(f"Event created at: {event.created_at}") ``` ### 4. Receive Messages (Polling) Since async agents process events asynchronously, you need to poll for messages: ```python theme={null} from datetime import datetime has_received_response = False messages = [] timeout = 30 start_time = datetime.now() # Poll for messages while not has_received_response and (datetime.now() - start_time).seconds < timeout: messages = await client.messages.list(task_id=task.id) for message in messages: content = message.content if content.author == "agent": has_received_response = True break await asyncio.sleep(1) # Print received messages print(f"\nReceived {len(messages)} messages:\n") for message in messages: if message.content and message.content.type == "text": print(f"[{message.content.author}]: {message.content.content}") print("-" * 80) ``` ## Streaming Messages ### Helper Function for Streaming Use the built-in helper to subscribe to task messages when testing in a local environment. The helper is synchronous: it takes a synchronous `Agentex` client and a task object with a `name`, such as the task from step 2. ```python theme={null} import os from agentex import Agentex from agentex.lib.utils.dev_tools import subscribe_to_async_task_messages sync_client = Agentex( base_url=os.environ.get("AGENTEX_BASE_URL"), default_headers={ "x-api-key": os.environ.get("AGENTEX_API_KEY"), "x-selected-account-id": os.environ.get("SGP_ACCOUNT_ID") } ) async def send_and_stream_event(task, user_message: str): """Send an event and stream the agent's response.""" # Send the event event_content = TextContent( type="text", author="user", content=user_message ) await client.agents.send_event( agent_id=agent.id, params={ "task_id": task.id, "content": event_content } ) print(f"User: {user_message}\n") # Subscribe to task messages messages = subscribe_to_async_task_messages( client=sync_client, task=task, timeout=30, ) return messages # Usage await send_and_stream_event( task, "Can you tell me about machine learning?" ) ``` ### Custom Streaming Implementation For more control over streaming or for production environments, implement custom streaming logic: ```python theme={null} import json async def custom_send_stream_event(task_id: str, user_message: str): """Send an event and stream the agent's response with custom logic.""" # Send the event event_content = TextContent( type="text", author="user", content=user_message ) await client.agents.send_event( agent_id=agent.id, params={ "task_id": task_id, "content": event_content } ) print(f"User: {user_message}\n") print("Agent: ", end="", flush=True) # Stream events async with client.tasks.with_streaming_response.stream_events( task_id=task_id, timeout=30 ) as stream: try: response_message = "" async for task_message_update in stream.iter_lines(): if task_message_update.startswith("data: "): # Remove 'data: ' prefix task_message_update_json = task_message_update.strip()[6:] task_message_update_data = json.loads(task_message_update_json) if task_message_update_data.get("type") == "delta": delta = task_message_update_data.get("delta", {}) if delta.get("type") == "text": text_delta = delta.get("text_delta", "") response_message += text_delta print(text_delta, end="", flush=True) elif task_message_update_data.get("type") == "done": print("\n") break except Exception as e: print(f"\nError streaming messages: {e}") # Usage await custom_send_stream_event( task.id, "Can you tell me about tennis?" ) ``` ## Multi-Turn Conversations Async agents automatically maintain conversation history: ```python theme={null} # Create a new task for the conversation conversation_task_id = str(uuid.uuid4()) create_task_response = await client.agents.create_task( agent_id=agent.id, id=conversation_task_id, params={} ) conversation_task = create_task_response.result # Define a multi-turn conversation messages = [ "Can you tell me about machine learning?", "What are some practical applications?", "How do I get started learning it?", ] # Send each message and stream responses for message in messages: print(f"Task ID: {conversation_task.id}") await custom_send_stream_event(conversation_task.id, message) print("=" * 80 + "\n") ``` ### View Conversation History Retrieve all messages from a task: ```python theme={null} # Get all messages from the task all_messages = await client.messages.list(task_id=conversation_task.id) print("\n" + "=" * 80) print("COMPLETE CONVERSATION HISTORY") print("=" * 80 + "\n") for idx, message in enumerate(all_messages, 1): if message.content and message.content.type == "text": author = message.content.author.upper() print(f"{idx}. [{author}]") print(f"{message.content.content}\n") print("-" * 80) ``` ## Managing Tasks ### Check Task Status ```python theme={null} # Retrieve task status task_info = await client.tasks.retrieve(task_id=task.id) print(f"Task ID: {task_info.id}") print(f"Status: {task_info.status}") print(f"Status Reason: {task_info.status_reason}") print(f"Created At: {task_info.created_at}") print(f"Updated At: {task_info.updated_at}") ``` ### List All Tasks for an Agent ```python theme={null} # List tasks tasks = await client.tasks.list(agent_id=agent.id) for task in tasks: print(f"Task {task.id}: {task.status}") ``` ## State Management Async agents can maintain state across events: ```python theme={null} # List states for an agent states = await client.states.list(agent_id=agent.id) for state in states: print(f"State ID: {state.id}") print(f"Task ID: {state.task_id}") print(f"State Data: {state.state}") ``` # Sync Agent Source: https://docs.gp.scale.com/docs/v5/agents/agentex/types/sync-agent How to interact with synchronous (sync) Agentex agents using the Agentex SDK ## Overview Sync agents use a **sync ACP** architecture that: * Uses direct message-based communication via RPC * Returns responses with the request (synchronously) * Supports both streaming and non-streaming responses ### Key API Method: `send_message(), send_message_stream()` For sync agents, you always use the **`send_message()`** or **`send_message_stream()`** method to communicate with the agent. This is different from agentic agents, which use `send_event()` for task-based communication. ```python theme={null} # Sync agents use send_message() - no task creation needed! response = client.agents.send_message( agent_name=AGENT_NAME, params={ "content": {"type": "text", "author": "user", "content": "Hello!"} } ) ``` **Key Differences:** * **No task creation required** - You can send messages directly to the agent by name * **No event creation required** - You can send messages directly to the agent by name ## Setup ### Prerequisites ```bash theme={null} pip install agentex-sdk ``` ### Environment Variables ```bash theme={null} export AGENTEX_API_KEY="your-api-key" export AGENTEX_BASE_URL="http://agentex.agentex.azure.workspace.egp.scale.com" export SGP_ACCOUNT_ID="your-account-id" ``` ### Initialize the Client ```python theme={null} import os from agentex import Agentex from agentex.types import TextContent # Initialize the Agentex client client = Agentex( base_url=os.environ.get("AGENTEX_BASE_URL"), api_key="your-api-key", default_headers={ "x-api-key": os.environ.get("AGENTEX_API_KEY"), "x-selected-account-id": os.environ.get("SGP_ACCOUNT_ID"), }, ) AGENT_NAME = "" # Replace with your agent name ``` ## Basic Usage ### Non-Streaming Response Send a message and receive a complete response immediately as part of the response: ```python theme={null} # Send a message without streaming rpc_response = client.agents.send_message( agent_name=AGENT_NAME, params={ "content": { "type": "text", "author": "user", "content": "Hello! Can you explain what machine learning is?", }, "stream": False, }, ) # Extract and print the text content from the response print("User: Hello! Can you explain what machine learning is?\n") print("Agent:") for task_message in rpc_response.result: content = task_message.content if isinstance(content, TextContent): print(content.content) ``` **Output:** ``` User: Hello! Can you explain what machine learning is? Agent: Machine learning is a subset of artificial intelligence that enables systems to learn and improve from experience without being explicitly programmed... ``` ## Streaming Responses For real-time responses, use streaming mode: ```python theme={null} from agentex.types import TextDelta from agentex.types.agent_rpc_result import ( StreamTaskMessageDelta, StreamTaskMessageDone, StreamTaskMessageFull ) print("User: Can you give me a simple example of supervised learning?\n") print("Agent: ", end="", flush=True) # Stream the response for chunk in client.agents.send_message_stream( agent_name=AGENT_NAME, params={ "content": { "type": "text", "author": "user", "content": "Can you give me a simple example of supervised learning?", }, }, ): # The result is a TaskMessageUpdate task_message_update = chunk.result # Print text deltas as they arrive if isinstance(task_message_update, StreamTaskMessageDelta): delta = task_message_update.delta if isinstance(delta, TextDelta): print(delta.text_delta, end="", flush=True) elif isinstance(task_message_update, StreamTaskMessageFull): content = task_message_update.content if isinstance(content, TextContent): print(content.content) print("\n") ``` # Introduction to Agents Source: https://docs.gp.scale.com/docs/v5/agents/overview Gain an understanding of different agent types in SGP ## Agent Types There are two main types of agents in SGP: * [Agentex Agents](/docs/v5/agents/agentex/overview) * [Agent Service Agents](/docs/v5/agents/agent-service/overview) Agentex Agents are built in code and support streaming and non-streaming responses. This is the preferred way to build agents for production use and will have the most support moving forward. Agent Service Agents are built inside of SGP using a YAML configuration. They are more limited in their capabilities and will be deprecated in the future. ## View agents To view all available agents in SGP navigate to the [agents page](https://egp.dashboard.scale.com/beta/build). On this page you can see a grid or table view of all agents for your account. Metadata like the agent type, status, description, and updated time is visible. ![Agents grid view](https://mintcdn.com/scalegp/6cOXFxYysZJE_3yL/images/v5/agents/agents-grid-view.png?fit=max\&auto=format\&n=6cOXFxYysZJE_3yL\&q=85\&s=56803d820e83c0eb4e7b8dc42ceb858c) Clicking on an agent will take you to the agent's detail page. You can also create an Agent Service Agent by clicking the **Create Agent** button. Learn more about [Agent Service Agents Creation](/docs/v5/agents/agent-service/lifecycle/creating-agent-service-agents). # Authentication Source: https://docs.gp.scale.com/docs/v5/authentication Find your SGP API key and account ID in your deployment's identity settings. The next step is to get your API key and account ID, which are needed to make calls to the Scale GP API and to create, modify resources in the right place. Open your organization's SGP deployment using the host provided by your administrator. If you do not have an account, ask your administrator or Scale team to provision one. > [!NOTE] > > The pages available to you depend on your deployment and permissions. If you can't see the pages described below, ask your administrator for access. ## Deployments with Identity & Access Open your profile menu and select **Identity & Access**. This opens the IAM dashboard configured for your deployment. Use its API key and account management pages to obtain your key and the ID of the account you want to use. ## Deployments with Access Management If your profile menu shows **Access Management**, open it and select the account group containing your account. In the group's **Accounts** table, copy the **Account ID** for the account you want to use. The **Account Group ID** shown in the group listing is a different identifier. Use the profile menu's **API Key** entry to obtain your API key. ## Legacy admin dashboard If your profile menu has **API Key** and **Manage Accounts** entries instead, use the legacy admin pages below. On your deployment's SGP host, open `/admin/api-key` to get your API key. The screenshots in this section show this interface. Deployments with Identity & Access redirect these legacy pages, and deployments with Access Management redirect the legacy Accounts page; use the matching section above instead. > [!NOTE] > > If the **API Key** tab says API key management has moved to the IAM dashboard, **Rotate API Key** is disabled. Manage your API keys from **Identity & Access** instead. ![Switch to Accounts and find the account ID.](https://files.readme.io/9a66e81-image.png) ![API Key menu](https://mintcdn.com/scalegp/LGE2JNGqt--hko9O/images/api_key_menu.png?fit=max\&auto=format\&n=LGE2JNGqt--hko9O\&q=85\&s=941446577ec194247ac2afdfd0aa607f) Next, open `/admin/accounts` on the same SGP host and find the account ID for the account that you want to work with. You can also create a new account. Make sure to copy this ID to use it when testing your setup. ![You can also create a new account.](https://mintcdn.com/scalegp/LGE2JNGqt--hko9O/images/account_view.png?fit=max\&auto=format\&n=LGE2JNGqt--hko9O\&q=85\&s=9c7bccfd455da56a9af090ae3f5b22be) If you have an existing account but want to add Users to it, you can click into the account and add users. ![Click into the account to add users.](https://mintcdn.com/scalegp/LGE2JNGqt--hko9O/images/add_users.png?fit=max\&auto=format\&n=LGE2JNGqt--hko9O\&q=85\&s=95894946d54876ed70efa96a18b05610) You can also change the role of a user by clicking on the user and changing the role. ![Change the role of a user.](https://mintcdn.com/scalegp/nt7p86_IWue4gDMA/images/change_role.png?fit=max\&auto=format\&n=nt7p86_IWue4gDMA\&q=85\&s=deba05c0c3e2343165d67396db959ec3) You can also remove a user by clicking on the remove button in the actions column. ![Remove a user.](https://mintcdn.com/scalegp/9TXIMOtQYrZn6pT_/images/remove_user.png?fit=max\&auto=format\&n=9TXIMOtQYrZn6pT_\&q=85\&s=44d39e249da15ed3b36df93211bcdb3e) ## Storing Your Credentials For security and convenience when using the SDK, it's recommended to store your API key and account ID as environment variables. This prevents accidentally exposing your credentials in code repositories. ```bash macOS/Linux theme={null} export SGP_API_KEY="your-api-key-here" export SGP_ACCOUNT_ID="your-account-id-here" ``` ```bash Windows (PowerShell) theme={null} $env:SGP_API_KEY="your-api-key-here" $env:SGP_ACCOUNT_ID="your-account-id-here" ``` ```bash Windows (CMD) theme={null} set SGP_API_KEY=your-api-key-here set SGP_ACCOUNT_ID=your-account-id-here ``` The SDK will automatically use the `SGP_API_KEY` and `SGP_ACCOUNT_ID` environment variables if they are set. Otherwise, you can pass them to the `SGPClient` constructor. ```python Python theme={null} import os from scale_gp_beta import SGPClient client = SGPClient( account_id="your-account-id", api_key="your-api-key", ) ``` The `SGPClient` constructor accepts the following parameters: | Parameter | Description | Default | | - | - | - | | `account_id` | Your SGP account ID, from Identity & Access, the Accounts table inside an Access Management account group, or the legacy Accounts page for your deployment. | `SGP_ACCOUNT_ID` environment variable | | `api_key` | Your SGP API key, from Identity & Access or the legacy API Key page for your deployment. | `SGP_API_KEY` environment variable | | `base_url` | The API URL of your SGP deployment, such as `https://api.`. You can also set it with the `SGP_CLIENT_BASE_URL` environment variable. | `https://api.egp.scale.com` | > [!TIP] > > To make the environment variables persist across sessions, add the export commands to your shell configuration file (e.g., `~/.bashrc`, `~/.zshrc`, or `~/.bash_profile`). # Evaluation Group Dashboards Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/eval-group-dashboards Compare metrics across multiple evaluations with evaluation group dashboards ![Evaluation Group Dashboard](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/eval-group-dashboard-header.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=68a6a7150221cf498db07ee5c6229478) ## What are Evaluation Group Dashboards? Evaluation group dashboards let you aggregate and compare data across multiple evaluations in a single dashboard. Instead of viewing metrics for one evaluation at a time, you can visualize trends, compare performance, and track progress across an entire group of related evaluations. Evaluation group dashboards support all the same widget types and query language as single-evaluation dashboards, with additional features for cross-evaluation analysis. Watch the [evaluation group dashboard demo video](./overview#evaluation-group-dashboard) for a walkthrough. > [!NOTE] > > Dashboards use an XOR constraint -- they belong to either a single evaluation OR an evaluation group, never both. ## Creating an Evaluation Group Dashboard ### Via the UI **1. Open the Create Dashboard Modal** Navigate to the Dashboards page and click "New Dashboard" **2. Select Group** In the create dialog, select the "Groups" tab and choose the evaluation group you want to create a dashboard for. **3. Fill in Details** * **Name**: Give your dashboard a descriptive name * **Description**: Optional description explaining the dashboard's purpose * **Tags**: Optional tags for organization and filtering ![Evaluation Group Dashboard Create Modal](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/eval-group-dashboard-create.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=5ae9abf4a1d50ca30716d315c20b9692) **4. Create** Click "Create" to save your dashboard. You can now add widgets to visualize your group data. ### Via the SDK ```python theme={null} from scale_gp_beta import SGPClient client = SGPClient( api_key="your-api-key", account_id="your-account-id", ) # Create dashboard for an evaluation group dashboard = client.evaluation_dashboards.create( name="Cross-Evaluation Comparison", evaluation_group_id="eval-group-456", description="Compare performance across weekly evaluation runs", tags=["comparison", "weekly"] ) ``` ## Querying Across Evaluations Evaluation group dashboards extend the standard query language with fields for controlling which evaluations to include in computations. ### The `evaluation_ids` Field Add `evaluation_ids` to a query to specify which evaluations in the group to include. If omitted, all evaluations in the group are used. ```json theme={null} { "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" }, "alias": "avg_score" } ], "evaluation_ids": ["eval-abc", "eval-def", "eval-ghi"] } ``` This computes the average score across only the three specified evaluations, rather than all evaluations in the group. ### Per-Aggregation `evaluation_ids` Individual aggregation nodes can also specify their own `evaluation_ids`, which must be a subset of the query-level `evaluation_ids`. This allows you to compare metrics across different evaluation subsets within the same widget. ```json theme={null} { "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data", "evaluation_ids": ["eval-abc"] }, "alias": "eval_a_avg" }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data", "evaluation_ids": ["eval-def"] }, "alias": "eval_b_avg" } ], "evaluation_ids": ["eval-abc", "eval-def"] } ``` > [!WARNING] > > Per-aggregation `evaluation_ids` must be a subset of the query-level `evaluation_ids`. The system validates this constraint and will reject queries where a per-node set includes IDs not present at the query level. ### The `_evaluation_id` Column A special `_evaluation_id` column is automatically available in evaluation group dashboard queries. This column contains the ID of the evaluation that each data row belongs to, allowing you to group or filter by evaluation source. ```json theme={null} { "select": [ {"expression": {"type": "COLUMN", "column": "_evaluation_id"}}, { "expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}, "alias": "avg_score" }, { "expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}, "alias": "item_count" } ], "groupBy": ["_evaluation_id"] } ``` This produces a row per evaluation in the group, showing the average score and item count for each. ## Per-Evaluation Selection in Widgets When creating bar, histogram, metric or grouped table widgets in the UI for an evaluation group dashboard, the widget creator allows you to select which evaluations to include per aggregation. This is particularly useful for: * **Metric widgets**: Compare the same metric across specific evaluations * **Bar and histogram widgets**: Compare multiple metrics across specific evaluations * **Table widgets**: Include different evaluation subsets for different columns The UI evaluation selector maps directly to the `evaluation_ids` field on aggregation nodes in the query. Timeseries, donut, scatter and ungrouped table widgets have one evaluation selector for the whole widget, which maps to the query-level `evaluation_ids` field. ## Auto-Recomputation on Group Changes When evaluations are added to or removed from an evaluation group, all dashboard widgets for that group are automatically recomputed. This ensures your dashboards always reflect the current state of the group. ### How it Works 1. **Membership change detected**: When you add or remove evaluations from a group, the system triggers an asynchronous recomputation workflow 2. **Smart `evaluation_ids` updates**: Widgets whose `evaluation_ids` covered all previous group members are automatically expanded or contracted to reflect the new membership. For example, if a group had evaluations \[A, B] and you add C, widgets covering \[A, B] are updated to \[A, B, C] 3. **Results marked as pending**: Existing widget results are marked with `computation_status: "pending"` while recomputation runs 4. **Recomputation completes**: Each widget is recomputed with the updated evaluation data, and results are updated to `computation_status: "completed"` > [!TIP] > > This allows for widgets to be updated as new evaluations are added to the group. Any widgets computed across all evaluations in the group will be updated to include the new evaluation. > > This is particularly useful for the timeseries widget, where each evaluation is plotted as a member of the x-axis. ## Example: Cross-Evaluation Comparison Dashboard Here's a full example creating an evaluation group dashboard with multiple widget types. ### Step 1: Create the Dashboard ```python theme={null} dashboard = client.evaluation_dashboards.create( name="Weekly Agent Performance", evaluation_group_id="eval-group-weekly", tags=["weekly-review"] ) ``` ### Step 2: Add a Heading ```python theme={null} client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Performance Overview", type="heading" ) ``` ### Step 3: Add Metric Widgets per Evaluation ```python theme={null} # Average score across all evaluations in the group client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Overall Average Score", type="metric", query={ "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } } ] } ) ``` ### Step 4: Add a Timeseries Showing Trends ```python theme={null} client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Score Trend Over Evaluations", type="timeseries", query={ "select": [ {"expression": {"type": "COLUMN", "column": "_evaluation_id"}}, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" }, "alias": "avg_score" }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "accuracy_score", "source": "data" }, "alias": "avg_accuracy" } ], "group_by": ["_evaluation_id"] } ) ``` ### Step 5: Add a Table Grouped by Evaluation ```python theme={null} client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Per-Evaluation Breakdown", type="table", query={ "select": [ {"expression": {"type": "COLUMN", "column": "_evaluation_id"}}, { "expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}, "alias": "items" }, { "expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}, "alias": "avg_score" }, { "expression": { "type": "AGGREGATION", "function": "PERCENTAGE", "column": "*", "params": { "percentage_filters": { "conditions": [{"column": "overall_score", "source": "data", "operator": ">=", "value": 85}] } } }, "alias": "pass_rate" } ], "group_by": ["_evaluation_id"] } ) ``` ## Related Documentation * [Evaluation Dashboards Overview](./overview) - Introduction to dashboards * [Getting Started](./getting-started) - Create your first dashboard * [Timeseries Widget](./widget-types/timeseries) - Ideal for group trend visualization * [Query Language](./query-language) - Complete query syntax reference * [API Reference](/reference/v5/evaluation-dashboards/create-evaluation-dashboard) - Programmatic dashboard management # Latest Snapshots Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/eval-group-latest-snapshots View the results from the most recent evaluation in an evaluation group ## What are Latest Snapshots? A *latest snapshot* is a widget on an evaluation group dashboard that computes against only the most recently created evaluation in a group. > [!NOTE] > > "Most recent" is defined as the evaluation in the group with the latest `created_at` timestamp -- **not** the order in which evaluations were added to the group. ## When to use it * **Snapshot tiles** at the top of a dashboard that need to read the last run's score. * **Comparison views** where you want the latest run highlighted alongside trend visualizations of the full group. * **Always-fresh KPI widgets** that should automatically refocus on a new evaluation as soon as it's added to the group. ## Where it applies Latest snapshots are only available on widgets that issue a query. | Widget type | Latest Only available? | | - | - | | Metric (Query Value) | Yes | | Bar | Yes | | Histogram | Yes | | Donut | Yes | | Table (grouped or flat) | Yes | | Scatter | Yes | | Timeseries | Yes, but rarely useful -- collapses the chart to a single point | | Markdown, Heading | Not applicable | The toggle is **only available on evaluation group dashboards**. ## Enable via the UI Toggle **Latest only** in the widget modal when creating or editing a widget. ![Latest only toggle in the Add Widget modal](https://mintcdn.com/scalegp/yA89mjKr_uD_bm8s/images/v5/evaluation-dashboards/widget-modal-latest-only-toggle.png?fit=max\&auto=format\&n=yA89mjKr_uD_bm8s\&q=85\&s=aed8b272fcf0ec4edc51cb57aece1ae4) ## Enable via the SDK Set `latest_only: true` on the query payload when creating or updating the widget. The flag is supported on both `SeriesQuery` (Bar / Histogram / Donut / Table / Scatter / Timeseries) and `MetricQuery` (Metric / Query Value). `latest_only` can't be combined with per-aggregation `evaluation_ids`. A query with both fails with a 422 error. ```python theme={null} # Metric widget that always shows the average score of the most recent run in the group client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Latest Snapshot: Average Score", type="metric", query={ "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data", }, } ], "latest_only": True, }, ) ``` Omitting `latest_only` or setting it to `false` keeps the existing behavior of computing across the full group. # Getting Started with Dashboards Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/getting-started Step-by-step tutorial for creating your first evaluation dashboard This guide walks you through creating your first evaluation dashboard from scratch, adding widgets, and organizing your layout. You can also watch the [single evaluation dashboard demo video](./overview#single-evaluation-dashboard) for a visual walkthrough. ## Using Sample Data To follow along with this tutorial, you can use our sample evaluation dataset containing 40 agent evaluation items with realistic scores and metadata. Upload the sample data to your account and create an evaluation with it. **Download Sample Data:** > [card] Download sample-evaluation.csv > > Sample dataset with 40 evaluation items across 4 agents (GPT-4, Claude-3, Gemini-Pro, Llama-3) [Open](/assets/sample-data/sample-evaluation.csv) **Data Structure:** The sample data contains evaluation items with this structure: ```json theme={null} { "id": "eval_001", "agent_name": "GPT-4-Turbo-Agent", "agent_version": "1.0", "timestamp": "2026-01-15T10:30:00Z", "evaluator_id": "eval_001", "task_type": "question_answering", "prompt_category": "technical", "response_length": 256, "model_temperature": 0.7, "overall_score": 87, "accuracy_score": 92, "relevance_score": 85, "coherence_score": 89, "helpfulness_score": 84, "fluency_score": 91 } ``` **Key Fields:** * `agent_name`: Model being evaluated (GPT-4-Turbo-Agent, Claude-3-Sonnet-Agent, etc.) * Score columns: Flat numeric fields (`overall_score`, `accuracy_score`, `relevance_score`, `coherence_score`, `helpfulness_score`, `fluency_score`) on a 0–100 scale * `task_type`: Type of task (question_answering, summarization, code_generation, analysis, translation, creative_writing) * `prompt_category`: Category (technical, general, business, language, creative) **How to Use:** 1. Download the CSV file 2. Create a new evaluation via the API or SDK: ```python theme={null} from scale_gp_beta import SGPClient import csv client = SGPClient( api_key="your-api-key", account_id="your-account-id", ) # Aggregations like AVG need numbers; text columns such as agent_version stay strings NUMERIC_COLUMNS = { "response_length", "model_temperature", "overall_score", "accuracy_score", "relevance_score", "coherence_score", "helpfulness_score", "fluency_score", } # Load sample data with open('sample-evaluation.csv', 'r') as f: sample_items = [ {k: float(v) if k in NUMERIC_COLUMNS else v for k, v in row.items()} for row in csv.DictReader(f) ] # Create evaluation with sample data evaluation = client.evaluations.create(evaluation={ "name": "Agent Performance Comparison", "data": sample_items }) print(f"Created evaluation: {evaluation.id}") ``` > [!TIP] > > To connect to your SGP deployment's API, pass its URL to the client as `base_url`. See [Authentication](/docs/v5/authentication). 3. Follow the rest of this guide to create dashboards and widgets using this evaluation > [!NOTE] > > The examples throughout this guide reference fields from this sample dataset. If using your own data, adjust the column names accordingly. ## Prerequisites Before creating a dashboard, you need either: * An existing evaluation with completed results, OR * An evaluation group containing evaluations If you don't have an evaluation yet, see [Next Gen Evaluation Getting Started](../next-gen-evaluation/getting-started) to create one. ## Step 1: Create a New Dashboard ### Via the UI 1. Open your SGP web app 2. Dashboards are off by default. Ask your Scale team to enable them for your account. 3. Click the "Dashboards" tab 4. Click the "New Dashboard" button ![New Dashboard Button](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/new-dashboard-button.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=f3312210fd7522201bec6650de1b85b1) 5. Fill in the dashboard details: * **Name**: Give your dashboard a descriptive name (e.g., "Model Performance Overview") * **Description**: Optional description explaining the dashboard's purpose * **Tags**: Optional tags for organization and filtering * **Evaluation / Evaluation Group**: Select the evaluation or evaluation group you want to create a dashboard for * **Template** *(optional)*: Select an existing single-evaluation dashboard to copy its widget layout 6. Click "Create" to save your dashboard ![New Dashboard Form](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/new-dashboard-form.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=e9325b0cef6dcd1880266c96588e40cc) ### Via the SDK ```python theme={null} from scale_gp_beta import SGPClient client = SGPClient( api_key="your-api-key", account_id="your-account-id", ) # Create dashboard for a single evaluation dashboard = client.evaluation_dashboards.create( name="Demo Dashboard", description="A demo dashboard for the demo evaluation", evaluation_id=evaluation.id, tags=["demo", "documentation"] ) # Create a single-evaluation dashboard from an existing template dashboard_from_template = client.evaluation_dashboards.create( name="Q2 Model Performance", evaluation_id="eval-456", template_dashboard_id="dash-template-abc" # Copies widget layout from this dashboard ) # Or create dashboard for an evaluation group group_dashboard = client.evaluation_dashboards.create( name="Cross-Evaluation Comparison", evaluation_group_id="eval-group-456", tags=["comparison"] ) ``` ## Step 2: Add Your First Widget (Metric) Let's add a metric widget to display the average score across all evaluation items. ### Via the UI 1. From your dashboard page, click "Add Widget" 2. Select "Query Value" as the widget type 3. Configure the widget: * **Query Value Title**: "Average Score" * Under **Data Source**, select "overall_score" as the **Column Name** and "Average" under **Aggregate by** 4. Click "Add" ![Average Score Widget Form](https://mintcdn.com/scalegp/PE5kt-ATjVp6oCI4/images/v5/evaluation-dashboards/average-score-metric-form.png?fit=max\&auto=format\&n=PE5kt-ATjVp6oCI4\&q=85\&s=415c1c6aec62a7e1cd828b35809e2e96) ![Average Score Widget Result](https://mintcdn.com/scalegp/PE5kt-ATjVp6oCI4/images/v5/evaluation-dashboards/average-score-metric-result.png?fit=max\&auto=format\&n=PE5kt-ATjVp6oCI4\&q=85\&s=c6c333ace6fc9bd23208979833f6e27c) > [!TIP] > > Widget results are computed when you create a widget or update its query. Editing a shared widget creates a new widget copy and computes its results; configuration-only edits to an unshared widget keep the existing results. The response includes both the widget configuration and the computed result. ### Via the API ```python theme={null} # Add a metric widget showing average score metric_widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Average Score", type="metric", query={ "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } } ] } ) print(f"Computed result: {metric_widget.result.computed_result}") # Output: {'type': 'metric', 'data': 84.15} ``` ## Step 3: Add a Chart Widget (Bar Chart) Now let's add a bar chart to show score distribution across different models. ### Via the UI 1. Click "Add Widget" again 2. Select "Bar Chart" as the widget type 3. Configure the widget: * **Graph Title**: "Score by Agent" * **Group By**: Select "agent_name" * Under **Data Source** * Click **Add Data Source**, then select "overall_score" as the **Column Name** and "Average" under **Aggregate by** 4. Click "Add" ![Score by Agent Bar Chart Form](https://mintcdn.com/scalegp/PE5kt-ATjVp6oCI4/images/v5/evaluation-dashboards/average-score-bar-chart-form.png?fit=max\&auto=format\&n=PE5kt-ATjVp6oCI4\&q=85\&s=41b473e67a28a2f3226594b69eac83bc) ![Score by Agent Bar Chart Result](https://mintcdn.com/scalegp/PE5kt-ATjVp6oCI4/images/v5/evaluation-dashboards/average-score-bar-chart-result.png?fit=max\&auto=format\&n=PE5kt-ATjVp6oCI4\&q=85\&s=e6c744708e9dd90e1baca4c44ba3fa23) ### Via the SDK ```python theme={null} # Add a bar chart widget showing average score by category bar_widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Score by Agent", type="bar", query={ "select": [ { "expression": { "type": "COLUMN", "column": "agent_name", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } } ], "group_by": ["agent_name"] }, config={ "x_column": "agent_name" } ) ``` ## Step 4: Add Section Headers Use heading widgets to organize your dashboard into logical sections. ### Via the UI 1. Click "Add Widget" 2. Select "Heading" as the widget type 3. Configure the widget: * **Content**: "Graphs" 4. Click "Add" ![Header Widget Form](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/heading/heading-widget-form.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=867c6a081f35170d9a128b2fd8417cb5) ![Header Widget Result](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/heading/heading-widget-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=52e17b4dbd8da2638158460bc7a72867) ### Via the SDK ```python theme={null} # Add a heading widget heading = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Graphs", type="heading" ) ``` ## Step 5: Organize and Configure Layout ### Reorder Widgets Arrange widgets in your preferred order by dragging and dropping in the UI, or update the widget order via the API: ```python theme={null} # Reorder widgets - first widget appears at top client.evaluation_dashboards.update( dashboard_id=dashboard.id, widget_order=[heading.id, metric_widget.id, bar_widget.id] ) ``` > [!NOTE] > > For evaluation group dashboards, see the dedicated [Evaluation Group Dashboards](./eval-group-dashboards) guide for group-specific features like cross-evaluation queries, per-evaluation selection, and auto-recomputation. ## Next Steps * Learn about all available [Widget Types](./widget-types/overview) * Compare across evaluations with [Evaluation Group Dashboards](./eval-group-dashboards) * Capture [Latest Snapshots](./eval-group-latest-snapshots) of your evaluation group * Master the [Query Language](./query-language) for advanced filtering and aggregations * Explore [API Reference](/reference/v5/evaluation-dashboards/create-evaluation-dashboard) for programmatic dashboard creation # Introduction to Evaluation Dashboards Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/overview Create custom dashboards to visualize and analyze evaluation metrics ## What are Evaluation Dashboards? Evaluation Dashboards are customizable visualizations that allow you to analyze and monitor evaluation performance with widgets. Dashboards are a collection of metric cards, charts, and tables that can be used to provide insights into evaluation results. ![Evaluation Dashboards Overview](https://mintcdn.com/scalegp/PE5kt-ATjVp6oCI4/images/v5/evaluation-dashboards/dashboard-overview.png?fit=max\&auto=format\&n=PE5kt-ATjVp6oCI4\&q=85\&s=46c7c1077af9d7a75c5e2cd7899e883f) ## Demo Videos ### Single Evaluation Dashboard [https://drive.google.com/file/d/1VTQsJPaCIZ96I0eVRCp9HraYDo06GulA/preview](https://drive.google.com/file/d/1VTQsJPaCIZ96I0eVRCp9HraYDo06GulA/preview) ### Evaluation Group Dashboard [https://drive.google.com/file/d/1cpa7kiTF__X1vs5iD7Iv-u0fbg0bIBCF/preview](https://drive.google.com/file/d/1cpa7kiTF__X1vs5iD7Iv-u0fbg0bIBCF/preview) ## Key Concepts ### Dashboards A dashboard is a collection of widgets arranged in a custom layout. Each dashboard is scoped to either: * **Single Evaluation**: Visualize metrics for one specific evaluation — see [demo video](#single-evaluation-dashboard) * **Evaluation Group**: Compare metrics across multiple related evaluations — see [Evaluation Group Dashboards](./eval-group-dashboards) and [demo video](#evaluation-group-dashboard) > [!NOTE] > > Dashboards use an XOR constraint - they belong to either a single evaluation OR an evaluation group, never both. ### Widgets Widgets are reusable visualization components that display computed metrics or data. Nine widget types are available: * **Metric**: Display a single computed aggregation (e.g., "Average Score: 0.87") * **Table**: Show aggregations in a table format, with grouping, sorting, and conditional formatting * **Bar Chart**: Visualize data across a categorical dimension * **Histogram**: Visualize data across a numerical dimension * **Donut Chart**: Show proportional distribution of categories * **Scatter Plot**: Explore correlations between two numerical dimensions * **Timeseries**: Track metrics over time across evaluation runs on evaluation group dashboards * **Markdown**: Add static content to the dashboard * **Heading**: Organize dashboard sections Learn more in the [Widget Types reference](./widget-types). ### Queries For data-driven widgets (Metric, Table, Bar, Histogram, Donut, Scatter, Timeseries), you define a query using a SQL-like JSON object. Queries allow you to: * Aggregate data using functions * `AVG`, `SUM`, `COUNT`, `MIN`, `MAX`, `STDDEV`, `VARIANCE`, `PERCENTILE`, `PERCENTAGE`, `COUNT`, `COUNT_DISTINCT` * Filter results based on conditions * Group by dimensions * Sort and limit results See the [Query Language guide](./query-language) for details. ### Widget Results When you create a widget or update its query, the system computes its results by executing the query against your evaluation data. Configuration-only edits to an unshared widget keep the existing computed results. Editing a shared widget creates a copy for your dashboard and computes results for that copy. Results are stored in the database and can be accessed by the widget when it is rendered. For evaluation group dashboards, widget results are also automatically recomputed when evaluations are added to or removed from the group. See [Evaluation Group Dashboards](./eval-group-dashboards) for details. ## Architecture & Data Model Understanding how dashboards are structured in the database helps clarify the relationship between dashboards, widgets, and their computed results. ### Three Core Entities **1. Dashboard** A dashboard is a container that links to an evaluation or evaluation group and maintains an ordered list of widget IDs. ```json theme={null} { "id": "dash-abc123", "name": "Model Performance Dashboard", "evaluation_id": "eval-xyz789", "widget_order": ["widget-1", "widget-2", "widget-3"] } ``` The `widget_order` array defines both which widgets appear on the dashboard and their display order. **2. Widget (Template)** Widgets are reusable computation templates that define: * What data to query (the query definition) * How to display it (the configuration) * The visualization type (metric, table, bar, etc.) ```json theme={null} { "id": "widget-1", "title": "Average Score", "type": "metric", "query": { "select": [{"expression": {"type": "AGGREGATION", "function": "AVG", "column": "score"}}] }, "config": {} } ``` Widgets are stored independently from dashboards and can be reused across multiple dashboards by including the same widget ID in different `widget_order` arrays. This enables the same dashboard template to be used for different evaluations. **3. Widget Result (Computed Output)** Widget results are the actual computed values produced by executing a widget's query against evaluation data. ```json theme={null} { "id": "result-def456", "widget_id": "widget-1", "evaluation_id": "eval-xyz789", "computed_result": {"type": "metric", "data": 0.873}, "computation_status": "completed", "computed_at": "2026-01-15T10:35:02Z" } ``` Results are specific to both a widget and an evaluation, meaning the same widget produces different results when used on different evaluations. ### How They Work Together 1. **Dashboard Creation**: When you create a dashboard, you link it to an evaluation or evaluation group 2. **Adding Widgets**: When you add a widget to a dashboard: * A widget template is created * The widget's ID is appended to the dashboard's `widget_order` * The system automatically computes the widget result for the dashboard's evaluation data 3. **Display**: When rendering a dashboard: * The system fetches the dashboard and its ordered widget list * For each widget ID, it loads the widget definition (template) * It retrieves the corresponding widget result (computed output) * The UI renders widgets in order using the template definition and computed results 4. **Updates**: When you update a widget: * If the widget belongs to only one dashboard: The template is updated in place * If the widget is shared across multiple dashboards: A copy of the widget is created for your dashboard * For an unshared widget, results are recomputed when the query is included in the update; otherwise the existing result is kept, and configuration-only changes affect only how the widget is displayed * For a shared widget, the new copy receives newly computed results ### Reusability Example The same widget template can appear on multiple dashboards: ``` Widget "Average Score" (widget-1) │── Dashboard A (evaluation-1) │ └── Result: 0.873 │── Dashboard B (evaluation-2) │ └── Result: 0.791 │── Dashboard C (evaluation-group-1) └── Result: 0.845 ``` > [!TIP] > > This architecture enables efficient widget reuse while maintaining separate computed results for each evaluation context. ## Managing Widgets ### Editing Widgets You can edit a widget's configuration, query, or title at any time: **Via the UI:** 1. Navigate to your dashboard 2. Click the edit icon (pencil) on the widget you want to modify 3. Update the title, query, or configuration 4. Click "Save Changes" **Via the SDK:** ```python theme={null} updated_widget = client.evaluation_dashboards.widgets.update( dashboard_id=dashboard.id, widget_id=widget.id, title="Updated Widget Title", query={ "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } } ] } ) ``` > [!NOTE] > > **Widget Cloning Behavior**: If a widget is shared across multiple dashboards, editing it will automatically create a copy for your dashboard to prevent affecting other dashboards using the same widget. ### Removing Widgets from a Dashboard Remove widgets you no longer need from your dashboard: **Via the UI:** 1. Navigate to your dashboard 2. Click the delete icon (trash) on the widget 3. Confirm the removal **Via the SDK:** ```python theme={null} client.evaluation_dashboards.widgets.remove( dashboard_id=dashboard.id, widget_id=widget.id ) ``` > [!WARNING] > > Removing a widget from a dashboard only removes it from that dashboard's `widget_order`. The widget isn't deleted: other dashboards that use it keep showing it, and the widget template and its computed results remain stored even when no other dashboard uses the widget. The API has no operation to delete a widget template. ### Reordering Widgets Organize your dashboard by rearranging widget order: **Via the UI:** 1. Navigate to your dashboard 2. Click and drag widgets to reorder them 3. Changes are saved automatically **Via the SDK:** ```python theme={null} # Specify the desired order of widget IDs client.evaluation_dashboards.update( dashboard_id=dashboard.id, widget_order=["widget-3", "widget-1", "widget-2"] # New order ) ``` The first widget ID in the array appears at the top of the dashboard, and subsequent widgets follow in order. ## Duplicating Dashboards Create a dashboard from an existing template to reuse its widgets with another evaluation or evaluation group. > [!WARNING] > > In the UI, you can create single-evaluation dashboards from a template; the API and SDK also support evaluation-group templates and targets. The template must belong to the same account and must not be archived, and every copied widget must be compatible with the target. Queries that use `evaluation_ids` or `latest_only` need an evaluation-group target, and any evaluation IDs they select must belong to that group. ### Use Cases for Duplication * **Template Reuse**: Apply a dashboard template to multiple evaluations * **Comparison**: Create identical dashboards for different evaluation runs * **Iteration**: Start with a proven dashboard structure and customize it ### Creating from a Template **Via the UI:** 1. Navigate to the Dashboards page 2. Go to the dashboard you want to duplicate 3. Click the three dots menu and select "Duplicate Dashboard" 4. Fill in the form — select the target evaluation and optionally choose a template dashboard to copy from 5. Click "Duplicate" **Via the SDK:** ```python theme={null} # Create a new dashboard from an existing template new_dashboard = client.evaluation_dashboards.create( name="Q1 Model Performance", evaluation_id="eval-new-123", template_dashboard_id="dash-template-abc" # Template with compatible single-evaluation widgets ) # Reuse the same compatible widgets with an evaluation-group target new_group_dashboard = client.evaluation_dashboards.create( name="Group Model Performance", evaluation_group_id="eval-group-123", template_dashboard_id="dash-template-abc" ) ``` When you duplicate a dashboard: * The new dashboard copies the `widget_order` from the template * All widgets are reused (same widget IDs) * New widget results are computed for the target evaluation or evaluation group's data * The template dashboard remains unchanged > [!TIP] > > **Best Practice**: Create a "template" dashboard with your standard metrics and visualizations, then duplicate it for each new evaluation run to maintain consistency. ## Next Steps * Ready to create your first dashboard? Watch the [single evaluation dashboard demo](#single-evaluation-dashboard) or follow the [Getting Started guide](./getting-started) for a step-by-step walkthrough. * Want to compare across evaluations? Watch the [evaluation group dashboard demo](#evaluation-group-dashboard) or see the [Evaluation Group Dashboards](./eval-group-dashboards) guide. * For detailed information on widget configuration, see the [Widget Types reference](./widget-types/overview). * To learn the query language for data-driven widgets, read the [Query Language guide](./query-language). * For programmatic dashboard creation, read the [API Reference](/reference/v5/evaluation-dashboards/create-evaluation-dashboard). # Query Language Reference Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/query-language Deep dive into QueryAST for building powerful evaluation queries The QueryAST (Query Abstract Syntax Tree) is a SQL-like query language for aggregating and filtering evaluation data in dashboard widgets. This guide covers the complete query syntax and advanced features. ## Query Basics ### When Are Queries Required? Queries are required for data-driven widgets: * **METRIC**: Requires a MetricQuery (single aggregation) * **TABLE**: Requires a SeriesQuery (multiple rows/columns) * **BAR**: Requires a SeriesQuery (grouped data) * **HISTOGRAM**: Requires a SeriesQuery (distribution data) * **DONUT**: Requires a SeriesQuery (grouped data for slices) * **SCATTER**: Requires a SeriesQuery selecting two raw numeric columns * **TIMESERIES**: Requires a SeriesQuery selecting and grouping by `_evaluation_id`, with one or more aggregations Queries are NOT needed for: * **MARKDOWN**: Static content only * **HEADING**: Text display only ### Data Sources QueryAST can access two data sources for each evaluation item: **`data.*` - User-defined data** * Custom fields you provided when creating the evaluation * Examples: `data.input`, `data.expected_output`, `data.category`, `data.score` **`task_result_cache.*` - Task execution results** * Results from evaluation tasks (chat completions, agent runs, etc.) * Examples: `task_result_cache.llm_score`, `task_result_cache.latency_ms`, `task_result_cache.metadata.confidence` * Only `completed` task results are used. When a query aggregates, filters or groups on a task's result, items whose result is `pending`, `skipped`, `errored` or `prefilled` are left out > [!NOTE] > > Both data sources support nested fields using dot notation. For example, `data.metadata.model_version` or `task_result_cache.output.reasoning`. ### Query Structure Overview A query is a JSON object defining what data to select, how to aggregate it, and what filters to apply. This example includes the optional `filter`, `groupBy`, `orderBy`, and `limit` fields: ```json theme={null} { "select": [ {"expression": {"type": "COLUMN", "column": "category", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data"}, "alias": "avg_score"} ], "filter": {"conditions": [{"column": "score", "source": "data", "operator": ">=", "value": 0}]}, "groupBy": ["category"], "orderBy": [{"column": "avg_score", "direction": "DESC"}], "limit": 100 } ``` In the Python SDK, write `groupBy`, `orderBy` and the filter's `logicalOperators` as `group_by`, `order_by` and `logical_operators`. *** ## Query Types ### MetricQuery Returns a single scalar value. Used exclusively for METRIC widgets. **Structure:** ```json theme={null} { "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data" }, "alias": "average_score" } ] } ``` **Constraints:** * Select clause must have exactly one item * That item must be an `AGGREGATION` (not a raw `COLUMN`) * No `GROUP BY`, `ORDER BY`, or `LIMIT` allowed #### Example: Average Score **Example:** ```json theme={null} { "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data" }, "alias": "avg_score" } ], "filter": { "conditions": [ { "column": "category", "source": "data", "operator": "=", "value": "test" } ] } } ``` **SQL Equivalent:** ```sql theme={null} SELECT AVG(data->>'$.score') AS avg_score FROM evaluation_items WHERE data->>'$.category' = 'test' ``` ### SeriesQuery Returns multiple rows and columns. Used for `TABLE`, `BAR`, `HISTOGRAM`, `DONUT`, `SCATTER`, and `TIMESERIES` widgets. **Structure:** ```json theme={null} { "select": [ {"expression": {"type": "COLUMN", "column": "category", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}, "alias": "count"} ], "groupBy": ["category"], "orderBy": [{"column": "count", "direction": "DESC"}], "limit": 100 } ``` **Constraints:** * Select clause must have at least one item * Can mix `COLUMN` and `AGGREGATION` expressions * When using `GROUP BY`, `COLUMN` items not listed in `groupBy` are left out of the result #### Example: Performance by Category **Example:** ```json theme={null} { "select": [ { "expression": { "type": "COLUMN", "column": "category", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data" }, "alias": "avg_score" } ], "groupBy": ["category"], "orderBy": [ { "column": "avg_score", "direction": "DESC" } ], "limit": 10 } ``` **SQL Equivalent:** ```sql theme={null} SELECT data->>'$.category' AS category, AVG(data->>'$.score') AS avg_score FROM evaluation_items GROUP BY data->>'$.category' ORDER BY avg_score DESC LIMIT 10 ``` *** ## `evaluation_ids` (Evaluation Group Dashboards) When building queries for [evaluation group dashboards](./eval-group-dashboards), you can control which evaluations in the group to include using the `evaluation_ids` field. ### Query-Level `evaluation_ids` Add `evaluation_ids` at the top level of a query to scope it to specific evaluations in the group. If omitted, all evaluations in the group are included. ```json theme={null} { "select": [ { "expression": {"type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data"}, "alias": "avg_score" } ], "evaluation_ids": ["eval-abc", "eval-def"] } ``` ### Per-Aggregation `evaluation_ids` Individual aggregation expressions can specify their own `evaluation_ids` to narrow the scope further. These must be a subset of the query-level `evaluation_ids`. ```json theme={null} { "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data", "evaluation_ids": ["eval-abc"] }, "alias": "eval_a_avg" }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data", "evaluation_ids": ["eval-def"] }, "alias": "eval_b_avg" } ], "evaluation_ids": ["eval-abc", "eval-def"] } ``` > [!NOTE] > > The special `_evaluation_id` column is automatically available in evaluation group dashboard queries. You can use it in `SELECT` and `GROUP BY` to break down data by evaluation source. See the [Evaluation Group Dashboards](./eval-group-dashboards) guide for details. *** ## `SELECT` Clause The `SELECT` clause defines what columns or aggregations to return. ### Column Expression Reference a raw column from the data without aggregation. ```json theme={null} { "expression": { "type": "COLUMN", "column": "category", "source": "data" }, "alias": "category_name" } ``` **Fields:** * `type`: Always "COLUMN" * `column`: Name of the column to select * `source`: "data" or "task_result_cache" (optional, defaults to "data") * `alias`: Optional name for the result column (recommended for aggregation columns) > [!TIP] > > Aliases are especially useful for aggregation columns -- they provide readable column headers in tables, disambiguate when multiple aggregations use the same column, and are required for meaningful labels in chart legends. ### Aggregation Expression Apply an aggregation function to compute a derived value. ```json theme={null} { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data", "params": {} }, "alias": "average_score" } ``` **Fields:** * `type`: Always "AGGREGATION" * `function`: One of the supported aggregation functions (see below) * `column`: Column to aggregate, or `"*"` for `COUNT(*)` * `source`: "data" or "task_result_cache" (optional) * `params`: Function-specific parameters (for `PERCENTILE` and `PERCENTAGE`) * `alias`: Optional name for the result *** ## Aggregation Functions ### `COUNT` Count the number of rows. ```json theme={null} { "type": "AGGREGATION", "function": "COUNT", "column": "*" } ``` **Use Cases:** * Total number of items * Items per category (with `GROUP BY`) * Items matching filter criteria ### `SUM` Sum all values in a column. ```json theme={null} { "type": "AGGREGATION", "function": "SUM", "column": "cost", "source": "task_result_cache" } ``` **Use Cases:** * Total cost across all items * Total tokens used * Cumulative metrics ### `AVG` Calculate the mean value. ```json theme={null} { "type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data" } ``` **Use Cases:** * Average score * Mean latency * Average confidence ### `MIN` / `MAX` Find the minimum or maximum value. ```json theme={null} { "type": "AGGREGATION", "function": "MAX", "column": "latency_ms", "source": "task_result_cache" } ``` **Use Cases:** * Worst-case latency * Best score achieved * Range calculations > [!NOTE] > > The `start_timestamp` and `end_timestamp` fields in `data` are read as datetimes. `SUM`, `AVG`, `STDDEV`, `VARIANCE` and `PERCENTILE` are rejected on them, and `MIN` and `MAX` return ISO 8601 strings. ### `STDDEV` / `VARIANCE` Calculate standard deviation or variance. ```json theme={null} { "type": "AGGREGATION", "function": "STDDEV", "column": "score", "source": "data" } ``` **Use Cases:** * Measure score consistency * Identify high-variance categories * Quality control metrics ### `PERCENTILE` Calculate a specific percentile value. ```json theme={null} { "type": "AGGREGATION", "function": "PERCENTILE", "column": "latency_ms", "source": "task_result_cache", "params": { "percentile": 95 } } ``` **Parameters:** * `percentile`: Number between 0 and 100 (required) **Use Cases:** * P95/P99 latency tracking * Tail performance analysis * SLA monitoring ### `COUNT_DISTINCT` Count unique values in a column. ```json theme={null} { "type": "AGGREGATION", "function": "COUNT_DISTINCT", "column": "user_id", "source": "data" } ``` **Use Cases:** * Number of unique users * Distinct categories present * Cardinality checks ### `PERCENTAGE` Calculate the fraction of rows matching filter criteria. For a scalar metric, specify a concrete column, such as `score`; reserve `"*"` for `COUNT`. The result is a decimal fraction (for example, `0.85`), which the UI can format as a percentage. ```json theme={null} { "type": "AGGREGATION", "function": "PERCENTAGE", "column": "score", "source": "data", "params": { "percentage_filters": { "conditions": [ { "column": "score", "source": "data", "operator": ">=", "value": 0.7 } ] } } } ``` **Parameters:** * `percentage_filters`: Filter object defining which rows to count (required) **Use Cases:** * Pass/fail rates * Percentage above threshold * Compliance metrics *** ## Filter Clause (`WHERE`) Filter clauses narrow down which rows to include in computations. ### Basic Filter Single condition: ```json theme={null} { "filter": { "conditions": [ { "column": "category", "source": "data", "operator": "=", "value": "quality" } ] } } ``` ### Multiple Conditions Combine conditions with logical operators: ```json theme={null} { "filter": { "conditions": [ { "column": "score", "source": "data", "operator": ">=", "value": 0.5 }, { "column": "category", "source": "data", "operator": "=", "value": "production" } ], "logicalOperators": ["AND"] } } ``` > [!WARNING] > > The number of logical operators must be exactly `len(conditions) - 1`. For 3 conditions, you need 2 operators. ### Comparison Operators | Operator | Description | Example Value | | - | - | - | | `=` | Equals | `"test"` or `42` | | `!=` | Not equals | `"draft"` | | `>` | Greater than | `0.5` | | `<` | Less than | `100` | | `>=` | Greater or equal | `0.7` | | `<=` | Less or equal | `1.0` | | `IN` | In list | `["cat1", "cat2"]` | | `NOT IN` | Not in list | `["archived", "deleted"]` | | `LIKE` | Case-insensitive regular-expression search | `"error"` | | `NOT LIKE` | Negated case-insensitive regular-expression search | `"deprecated"` | | `IS NULL` | Value is null | Omit `value` | | `IS NOT NULL` | Value is not null | Omit `value` | `LIKE` and `NOT LIKE` search using regular expressions. Plain text such as `"error"` matches a substring. SQL `%` and `_` wildcards are not translated; `%` is matched literally. Escape regular-expression metacharacters when you want to match them as literal text. ### Complex Filter Example Combine multiple conditions with AND/OR logic: ```json theme={null} { "filter": { "conditions": [ { "column": "score", "source": "data", "operator": ">=", "value": 0.7 }, { "column": "category", "source": "data", "operator": "IN", "value": ["quality", "accuracy"] }, { "column": "status", "source": "data", "operator": "!=", "value": "archived" } ], "logicalOperators": ["AND", "AND"] } } ``` This evaluates as: `(score >= 0.7) AND (category IN ['quality', 'accuracy']) AND (status != 'archived')` > [!NOTE] > > Logical operators are evaluated left-to-right without precedence. The expression `A AND B OR C` evaluates as `((A AND B) OR C)`. *** ## `GROUP BY` Clause Group rows by one or more columns before applying aggregations. ### Single Column Grouping ```json theme={null} { "select": [ { "expression": {"type": "COLUMN", "column": "category", "source": "data"} }, { "expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}, "alias": "count" } ], "groupBy": ["category"] } ``` **SQL Equivalent:** ```sql theme={null} SELECT data->>'$.category' AS category, COUNT(*) AS count FROM evaluation_items GROUP BY data->>'$.category' ``` ### Multi-Column Grouping Group by multiple dimensions: ```json theme={null} { "select": [ { "expression": {"type": "COLUMN", "column": "model", "source": "data"} }, { "expression": {"type": "COLUMN", "column": "category", "source": "data"} }, { "expression": {"type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data"}, "alias": "avg_score" } ], "groupBy": ["model", "category"] } ``` **SQL Equivalent:** ```sql theme={null} SELECT data->>'$.model' AS model, data->>'$.category' AS category, AVG(data->>'$.score') AS avg_score FROM evaluation_items GROUP BY data->>'$.model', data->>'$.category' ``` > [!TIP] > > When using `GROUP BY`, `COLUMN` items in `select` that aren't listed in `groupBy` are left out of the result. > [!NOTE] > > Rows with a null value in any `groupBy` column are dropped before aggregation, so they don't appear as a group. *** ## `ORDER BY` Clause Sort results by one or more columns. ### Basic Sorting ```json theme={null} { "orderBy": [ { "column": "avg_score", "direction": "DESC" } ] } ``` **Directions:** * `ASC`: Ascending (smallest to largest) * `DESC`: Descending (largest to smallest) ### Multi-Column Sorting Sort by multiple columns with different directions: ```json theme={null} { "orderBy": [ { "column": "category", "direction": "ASC" }, { "column": "avg_score", "direction": "DESC" } ] } ``` This sorts by category alphabetically, then by avg_score descending within each category. *** ## `LIMIT` Clause Restrict the number of rows returned. ```json theme={null} { "limit": 20 } ``` **Use Cases:** * Top-N queries (e.g., "Top 10 Items") * Prevent overwhelming table displays * Performance optimization for large datasets **Example: Top 5 Categories** ```json theme={null} { "select": [ {"expression": {"type": "COLUMN", "column": "category", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data"}, "alias": "avg_score"} ], "groupBy": ["category"], "orderBy": [{"column": "avg_score", "direction": "DESC"}], "limit": 5 } ``` *** ## Advanced Examples ### Example 1: Multi-Dimensional Performance Table Break down performance by model and category with multiple statistics. ```json theme={null} { "select": [ { "expression": { "type": "COLUMN", "column": "model", "source": "data" } }, { "expression": { "type": "COLUMN", "column": "category", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "COUNT", "column": "*" }, "alias": "count" }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "score", "source": "data" }, "alias": "avg_score" }, { "expression": { "type": "AGGREGATION", "function": "STDDEV", "column": "score", "source": "data" }, "alias": "stddev_score" }, { "expression": { "type": "AGGREGATION", "function": "MIN", "column": "score", "source": "data" }, "alias": "min_score" }, { "expression": { "type": "AGGREGATION", "function": "MAX", "column": "score", "source": "data" }, "alias": "max_score" } ], "groupBy": ["model", "category"], "orderBy": [ { "column": "model", "direction": "ASC" }, { "column": "avg_score", "direction": "DESC" } ] } ``` ### Example 2: Filtered Percentile Calculation Calculate P95 score for only production items. ```json theme={null} { "select": [ { "expression": { "type": "AGGREGATION", "function": "PERCENTILE", "column": "latency_ms", "source": "task_result_cache", "params": { "percentile": 95 } }, "alias": "p95_latency" } ], "filter": { "conditions": [ { "column": "environment", "source": "data", "operator": "=", "value": "production" }, { "column": "latency_ms", "source": "task_result_cache", "operator": ">", "value": 0 } ], "logicalOperators": ["AND"] } } ``` ### Example 3: Complex Filter with IN and LIKE Find items in specific categories containing certain text. ```json theme={null} { "select": [ { "expression": {"type": "COLUMN", "column": "input", "source": "data"} }, { "expression": {"type": "COLUMN", "column": "category", "source": "data"} }, { "expression": {"type": "COLUMN", "column": "score", "source": "data"} } ], "filter": { "conditions": [ { "column": "category", "source": "data", "operator": "IN", "value": ["quality", "accuracy", "helpfulness"] }, { "column": "input", "source": "data", "operator": "LIKE", "value": "customer support" } ], "logicalOperators": ["AND"] }, "orderBy": [ { "column": "score", "direction": "DESC" } ], "limit": 20 } ``` ### Example 4: Pass/Fail Rate by Model Calculate success rates grouped by model. ```json theme={null} { "select": [ { "expression": {"type": "COLUMN", "column": "model", "source": "data"} }, { "expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}, "alias": "total" }, { "expression": { "type": "AGGREGATION", "function": "PERCENTAGE", "column": "score", "source": "data", "params": { "percentage_filters": { "conditions": [ { "column": "score", "source": "data", "operator": ">=", "value": 0.7 } ] } } }, "alias": "pass_rate" } ], "groupBy": ["model"], "orderBy": [ { "column": "pass_rate", "direction": "DESC" } ] } ``` ### Example 5: Nested Data Access Query nested fields using dot notation. ```json theme={null} { "select": [ { "expression": { "type": "COLUMN", "column": "metadata.model_version", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "output.confidence", "source": "task_result_cache" }, "alias": "avg_confidence" } ], "groupBy": ["metadata.model_version"] } ``` This accesses: * `data.metadata.model_version` - nested field in user data * `task_result_cache.output.confidence` - nested field in task results *** ## Debugging Failed Queries If your widget shows `computation_status: "failed"`, check: 1. **Column names**: Ensure columns exist in your evaluation data 2. **Source specification**: Verify correct source ("data" vs "task_result_cache") 3. **Operator/value types**: Match operator to value type (IN requires list, LIKE requires string) 4. **Required params**: PERCENTILE needs percentile param, PERCENTAGE needs percentage_filters Check the `error_message` field in the widget result for specific error details. *** ## Next Steps * Apply these query patterns in the [Widget Types reference](./widget-types/overview) * See [Getting Started](./getting-started) for end-to-end examples * Explore [API Reference](/reference/v5/evaluation-dashboards/add-widget-to-dashboard) for programmatic query creation # Bar Chart Widget Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/bar Visualize comparisons across categories using bar charts > [!NOTE] > > **Using Sample Data**: The examples on this page use fields from our [sample dataset](/assets/sample-data/sample-evaluation.csv). Download it and create an evaluation with it to follow along with this tutorial. ![Bar Chart Widget Example](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/bar/bar-base-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=ed928382a9eccb12116947aa3c415e22) ## When to Use Use bar chart widgets to: * Compare values across categories or groups * Show rankings or relative performance * Visualize aggregated metrics by dimension * Make patterns and outliers immediately visible ## Configuration ### Required Fields **title** `string` *required* Widget display name shown on the dashboard **type** `string` *required* Must be set to "bar" **query** `object` *required* SeriesQuery with `GROUP BY` for categorical breakdown ### X-Axis Configuration (One Required) You must provide **either** `x_column` **or** `x_column_group`, but not both: **config.x_column** `string` Single column name to use for x-axis categories. Use for simple bar charts grouping by one dimension. **Example:** `"agent_name"` - creates bars for each agent **config.x_column_group** `string[]` Array of column names for multi-dimensional grouping. Use for complex bar charts that group by multiple dimensions. **Example:** `["agent_name", "task_type"]` - creates grouped bars combining agent and task type ### Optional Config Fields **config.stacked** `boolean` (default: true) Controls whether multiple series are stacked or displayed side-by-side. * `true` (default): Stack bars on top of each other (useful for showing totals) * `false`: Display bars side-by-side (better for comparing individual values) **config.show_value_labels** `boolean` (default: false) When `true`, each bar's value is rendered directly on the chart so you don't have to hover to read it. Value labels apply to single-series charts only (charts with one aggregation). They are ignored on multi-series charts, which use a legend instead. ### Query Requirements Bar chart widgets require a **SeriesQuery** with: * `groupBy` for categorical breakdown (required) * Should have at least one aggregation for y-axis values * Can include `filter` for row filtering ## Creating in the UI **1. Open Widget Creator** From your dashboard, click "Add Widget" and select "Bar Chart" **2. Enter Title** Give your chart a descriptive name (e.g., "Agent Name Accuracy Average") **3. Configure Query** ![Bar Chart Query Builder](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/bar/bar-base-form.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=49c77c9ef5908415371769d39aae5759) * **Group By**: Select the categorical dimension - these will be the bars on the y-axis * **Data Source**: Click **Add Data Source**, then select the **Column Name** and the function under **Aggregate by** (Average, Sum, Count) * **Filter**: Add conditions to narrow the data (optional) **4. Show Values on Bars (Optional)** For single-series charts, toggle **Show values on bars** to render each bar's value directly on the chart. This sets `config.show_value_labels`. **5. Create Widget** Click "Add" to generate the bar chart ![Created Bar Chart Widget](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/bar/bar-base-result-hover.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=463281f15b8e7f20a14547a41dbb7b7a) ## Output Format Bar chart widgets use the same series format as tables: ```json theme={null} { "type": "series", "data": [ {"category": "Quality", "score_avg": 0.89}, {"category": "Accuracy", "score_avg": 0.85}, {"category": "Helpfulness", "score_avg": 0.91} ], "column_order": ["category", "score_avg"] } ``` The chart renders each object as a bar. ## Example Use Cases ### Use Case 1: Average Score by Agent Compare performance across different agents. ![Score by Agent Bar Chart](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/bar/bar-score-by-agent.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=0fdcb203c4150bd018c9b7a169ee8270) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Average Score by Agent", type="bar", query={ "select": [ { "expression": { "type": "COLUMN", "column": "agent_name", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } } ], "group_by": ["agent_name"] }, config={ "x_column": "agent_name" } ) ``` ```json JSON theme={null} { "title": "Average Score by Agent", "type": "bar", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}} ], "groupBy": ["agent_name"] }, "config": { "x_column": "agent_name" } } ``` *** ### Use Case 2: Evaluation Count by Task Type Show which task types have the most evaluations. ![Count by Task Type Bar Chart](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/bar/bar-count-by-task-type.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=c3c420492cfd15dc0b97588c21e5a4e1) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Evaluations by Task Type", type="bar", query={ "select": [ {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "group_by": ["task_type"] }, config={"x_column": "task_type"} ) ``` ```json JSON theme={null} { "title": "Evaluations by Task Type", "type": "bar", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "groupBy": ["task_type"] }, "config": { "x_column": "task_type" } } ``` *** ### Use Case 3: High Performance Rate by Agent Visualize percentage of evaluations scoring 85+ across agents. ![High Performance Rate Bar Chart](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/bar/bar-high-perf-rate.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=77fb24a41fcb7ed4a425b5cf352c23bb) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="High Performance Rate by Agent", type="bar", query={ "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, { "expression": { "type": "AGGREGATION", "function": "PERCENTAGE", "column": "overall_score", "params": { "percentage_filters": { "conditions": [{ "column": "overall_score", "source": "data", "operator": ">=", "value": 85 }] } } } } ], "group_by": ["agent_name"] }, config={"x_column": "agent_name"} ) ``` ```json JSON theme={null} { "title": "High Performance Rate by Agent", "type": "bar", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, { "expression": { "type": "AGGREGATION", "function": "PERCENTAGE", "column": "overall_score", "params": { "percentage_filters": { "conditions": [{ "column": "overall_score", "source": "data", "operator": ">=", "value": 85 }] } } } } ], "groupBy": ["agent_name"] }, "config": { "x_column": "agent_name" } } ``` *** ### Use Case 4: Multi-Dimensional Grouping with x_column_group Group by multiple dimensions to create side-by-side bar comparisons. This example shows average scores grouped by both agent and task type. ![Multi-dimensional Grouped Bar Chart](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/bar/bar-multi-group.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=e45135baae13eb2eabdc00747c0b2b1c) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Score by Agent and Task Type", type="bar", query={ "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}}, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "accuracy_score", "source": "data" } } ], "group_by": ["agent_name", "task_type"] }, config={ "x_column_group": ["agent_name", "task_type"], "stacked": False # Display side-by-side for easier comparison } ) ``` ```json JSON theme={null} { "title": "Score by Agent and Task Type", "type": "bar", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "accuracy_score", "source": "data"}} ], "groupBy": ["agent_name", "task_type"] }, "config": { "x_column_group": ["agent_name", "task_type"], "stacked": false } } ``` ## Stacked vs Unstacked Series When you have multiple series (e.g., multiple aggregations or groupings), you can control how they're displayed: ### Stacked Bars (Default) ![Stacked Bar Chart](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/bar/bar-base-result-stacked.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=a9ffc6d6442768edad8356eaf8e9f23f) Bars are stacked horizontally along the numeric axis, showing cumulative totals. Best for: * Showing part-to-whole relationships * Comparing total values across categories * Visualizing composition over categories ```python theme={null} config={"x_column": "agent_name", "stacked": True} # Default ``` ### Unstacked Bars (Side-by-Side) ![Unstacked Bar Chart](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/bar/bar-base-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=ed928382a9eccb12116947aa3c415e22) Bars are displayed side-by-side, making individual values easier to compare. Best for: * Comparing individual series values directly * When exact values matter more than totals * Multi-dimensional analysis (e.g., agent + task type) ```python theme={null} config={"x_column": "agent_name", "stacked": False} ``` ## Display Behavior Bar charts apply the following formatting automatically. These behaviors are built into the rendering, so there are no extra config fields to set. ### Category labels Category labels are shown next to each bar by default, so you can read values without hovering. Long labels are truncated to fit, and hovering a truncated label reveals its full text in a tooltip. ### Per-category colors When a chart has a single series and more than one category (for example, an `AVG` grouped by `agent_name`), each bar is given a distinct color from a rotating palette. This makes individual categories easier to tell apart when there are many bars. Charts with multiple series keep one consistent color per series instead, so the series stay comparable across categories. ### Humanized column names Raw column names are converted to readable Title Case in axis labels, the legend, and tooltips (for example, `experiment_param_value` becomes "Experiment Param Value" and `avgScore` becomes "Avg Score"). Aliases that already contain spaces are shown unchanged, so you can override the displayed text by aliasing a column in your query. ## Related Documentation * [Table Widget](./table) - Tabular alternative with more detail * [Histogram Widget](./histogram) - For numerical distributions * [Query Language](../query-language) - `GROUP BY` and aggregation syntax * [API Reference](/reference/v5/evaluation-dashboards/add-widget-to-dashboard) - Programmatic chart creation # Donut Chart Widget Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/donut Visualize categorical distributions as a donut/pie chart > [!NOTE] > > **Using Sample Data**: The examples on this page use fields from our [sample dataset](/assets/sample-data/sample-evaluation.csv). Download it and create an evaluation with it to follow along with this tutorial. ![Donut Chart Widget Example](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/donut/donut-base-result.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=f746b3f19ca0ee6bc2a0b00e2a9c11a8) ## When to Use Use donut chart widgets to: * Show proportional distribution of categories * Visualize percentage breakdowns at a glance * Compare relative sizes of groups * Highlight dominant categories in your data ## Configuration ### Required Fields **title** `string` *required* Widget display name shown on the dashboard **type** `string` *required* Must be set to "donut" **query** `object` *required* SeriesQuery with `GROUP BY` for categorical breakdown ### Slice Grouping Configuration Set `config.x_column_group`, even when grouping by one column. The API also accepts `config.x_column` in its place, but the donut chart doesn't draw it. **config.x_column_group** `string[]` *required* Column names that identify each slice. Use a one-element array for a single grouping column, or multiple columns for slices representing each unique combination. These columns must also appear in `select` and `groupBy`. **Examples:** `["task_type"]` or `["agent_name", "task_type"]` ### Query Requirements Donut chart widgets require a **SeriesQuery** with: * `groupBy` for categorical breakdown (required) * At least one aggregation for slice values (`COUNT`, `SUM`, `AVG`, etc.) * Can include `filter` to narrow the dataset > [!NOTE] > > The donut chart automatically limits display to the **top 10 slices**. Any remaining categories are grouped into an "Other" slice. ## Creating in the UI **1. Open Widget Creator** From your dashboard, click "Add Widget" and select "Donut Chart" **2. Enter Title** Give your chart a descriptive name (e.g., "Task Type Distribution") **3. Configure Query** ![Donut Chart Query Builder](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/donut/donut-base-form.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=4592c6dc71f8116b7195f622b7203674) * **Column Name** (under **Data Source**): Select the categorical dimension - each unique value becomes a slice * Slices are sized by row count. The UI adds the `COUNT(*)` aggregation for you. * **Filter**: Add conditions to narrow the data (optional) **4. Create Widget** Click "Add" to generate the donut chart ![Created Donut Chart Widget](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/donut/donut-base-result.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=f746b3f19ca0ee6bc2a0b00e2a9c11a8) ## Output Format Donut chart widgets use the same series format as other chart types: ```json theme={null} { "type": "series", "data": [ {"task_type": "question_answering", "grouped_count": 12}, {"task_type": "summarization", "grouped_count": 8}, {"task_type": "code_generation", "grouped_count": 7}, {"task_type": "analysis", "grouped_count": 6} ], "column_order": ["task_type", "grouped_count"] } ``` Each data point becomes a slice, with the aggregation value determining its proportional size. ## Example Use Cases ### Use Case 1: Distribution by Task Type Show how evaluation items are distributed across task types. ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Task Type Distribution", type="donut", query={ "select": [ { "expression": { "type": "COLUMN", "column": "task_type", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "COUNT", "column": "*" } } ], "group_by": ["task_type"] }, config={ "x_column_group": ["task_type"] } ) ``` ```json JSON theme={null} { "title": "Task Type Distribution", "type": "donut", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "groupBy": ["task_type"] }, "config": { "x_column_group": ["task_type"] } } ``` *** ### Use Case 2: Agent Workload Distribution Visualize how many evaluation items each agent has processed. ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Agent Workload", type="donut", query={ "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "group_by": ["agent_name"] }, config={"x_column_group": ["agent_name"]} ) ``` ```json JSON theme={null} { "title": "Agent Workload", "type": "donut", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "groupBy": ["agent_name"] }, "config": { "x_column_group": ["agent_name"] } } ``` *** ### Use Case 3: Prompt Category Breakdown with Filter Show the distribution of prompt categories, filtered to only high-scoring items. ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="High-Scoring Items by Category", type="donut", query={ "select": [ {"expression": {"type": "COLUMN", "column": "prompt_category", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "filter": { "conditions": [ {"column": "overall_score", "source": "data", "operator": ">=", "value": 85} ] }, "group_by": ["prompt_category"] }, config={"x_column_group": ["prompt_category"]} ) ``` ```json JSON theme={null} { "title": "High-Scoring Items by Category", "type": "donut", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "prompt_category", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "filter": { "conditions": [ {"column": "overall_score", "source": "data", "operator": ">=", "value": 85} ] }, "groupBy": ["prompt_category"] }, "config": { "x_column_group": ["prompt_category"] } } ``` ## Related Documentation * [Bar Chart Widget](./bar) - For side-by-side categorical comparisons * [Table Widget](./table) - Detailed numerical breakdowns * [Query Language](../query-language) - `GROUP BY` and aggregation syntax * [API Reference](/reference/v5/evaluation-dashboards/add-widget-to-dashboard) - Programmatic chart creation # Heading Widget Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/heading Organize dashboards with simple section headers ![Heading Widget Example](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/heading/heading-widget-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=52e17b4dbd8da2638158460bc7a72867) ## When to Use Use heading widgets to: * Divide dashboard into logical sections * Label groups of related widgets ## Configuration ### Required Fields **title** `string` *required* The heading text to display **type** `string` *required* Must be set to "heading" **No Query or Config Required**: Heading widgets simply display the title as a styled header. ## Creating in the UI **1. Open Widget Creator** From your dashboard, click "Add Widget" and select "Heading" **2. Enter Heading Text** ![Heading Widget Creator](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/heading/heading-widget-form.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=867c6a081f35170d9a128b2fd8417cb5) Type the heading text in the "Content" field (e.g., "Overview Metrics") **3. Create Widget** Click "Add" to add the heading to your dashboard ![Created Heading Widget](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/heading/heading-widget-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=52e17b4dbd8da2638158460bc7a72867) ## Heading vs Markdown ### Use Heading When: * You only need a simple text header * You want consistent dashboard styling * You're organizing multiple short sections * Simplicity is preferred ### Use Markdown When: * You need formatted text beyond a header * You want to include links, lists, or tables * You're providing detailed context or instructions * You need multiple paragraphs of content ## Related Documentation * [Markdown Widget](./markdown) - For rich text content * [All Widget Types](./overview) - Compare widget types * [Getting Started](../getting-started) - Dashboard organization tips * [API Reference](/reference/v5/evaluation-dashboards/add-widget-to-dashboard) - Programmatic heading creation # Histogram Widget Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/histogram Visualize data distributions across numerical ranges > [!NOTE] > > **Using Sample Data**: The examples on this page use fields from our [sample dataset](/assets/sample-data/sample-evaluation.csv). Download it and create an evaluation with it to follow along with this tutorial. ![Histogram Widget Example](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/histogram/histogram-base-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=f87a9546e9a808ef71cbc3be4af51315) ## When to Use Use histogram widgets to: * Show value distributions (groups data by distinct values) * Visualize frequency counts or aggregated metrics * Identify patterns in numerical data * Display aggregated data in a distribution format ## Configuration ### Required Fields **title** `string` *required* Widget display name shown on the dashboard **type** `string` *required* Must be set to "histogram" **query** `object` *required* SeriesQuery selecting the column ### X-Axis Configuration (One Required) You must provide **either** `x_column` **or** `x_column_group`, but not both: **config.x_column** `string` Single column name to group by for the x-axis. The histogram displays distinct values from this column. **Example:** `"overall_score"` - groups by each score value (75, 80, 85, etc.) **config.x_column_group** `string[]` Array of column names used to form one composite x-axis category per distinct combination of values. **Example:** `["agent_name", "overall_score"]` - creates a category for each agent and score combination. Multiple numeric aggregation columns in the query create separate series. Use `config.stacked` to display those series as stacked or adjacent bars. ### Optional Config Fields **config.stacked** `boolean` (default: true) Controls whether multiple numeric aggregation series are stacked or shown as adjacent bars. * `true` (default): Stack distributions on top of each other * `false`: Show distributions as side-by-side bars for easier comparison **config.show_value_labels** `boolean` (default: false) When `true`, each bar's value is rendered directly on the chart so you don't have to hover to read it. Value labels apply to single-series histograms only (those with one aggregation). They are ignored on multi-series histograms, which use a legend instead. ### Query Requirements Histogram widgets require a **SeriesQuery** with: * Select the column you want to group by (becomes the x-axis) * Select an explicit aggregation (`COUNT`, `SUM`, `AVG`, etc.) for the y-axis in API or SDK queries * If you don't add an aggregation, the UI adds `COUNT(*)`. The API doesn't: a query without an aggregation returns the raw rows * Can include `filter` to narrow the dataset ## Creating in the UI **1. Open Widget Creator** From your dashboard, click "Add Widget" and select "Distribution" **2. Enter Title** Give your histogram a descriptive name (e.g., "Score Distribution") **3. Select Column** Choose the column to group by (this will be the x-axis) **4. Add Aggregations (Optional)** ![Histogram Aggregation Builder](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/histogram/histogram-base-form.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=919da9500b1f9554441549c41416fc27) Add aggregations for the y-axis. If none specified, `COUNT` is used by default. **5. Add Filters (Optional)** ![Histogram Filter Builder](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/histogram/histogram-base-form-filter.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=e5cf7002b675c49c12b176626dce0e05) Optionally filter to specific subsets of data **6. Show Values on Bars (Optional)** For single-series histograms, toggle **Show values on bars** to render each bar's value directly on the chart. This sets `config.show_value_labels`. **7. Create Widget** Click "Add" to generate the histogram ![Created Histogram Widget](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/histogram/histogram-base-result-hover.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=e35d088587fd6febf3c28e43e11708f3) ## Output Format Histogram widgets return grouped data: ```json theme={null} { "type": "series", "data": [ {"overall_score": 75, "grouped_count": 12}, {"overall_score": 80, "grouped_count": 45}, {"overall_score": 85, "grouped_count": 89}, {"overall_score": 90, "grouped_count": 156} ], "column_order": ["overall_score", "grouped_count"] } ``` Each data point represents a distinct value from the grouped column and its corresponding count or aggregated metric. ## Example Use Cases ### Use Case 1: Overall Score Distribution Visualize how overall scores are distributed across evaluation items. The explicit `COUNT` aggregation below counts items for each score value. ![Overall Score Distribution Histogram](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/histogram/histogram-score-distribution.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=411edc88bd12d4d3b01e8b15a278b1d7) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Overall Score Distribution", type="histogram", query={ "select": [ { "expression": { "type": "COLUMN", "column": "overall_score", "source": "data" } }, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "group_by": ["overall_score"] }, config={ "x_column": "overall_score" } ) ``` ```json JSON theme={null} { "title": "Overall Score Distribution", "type": "histogram", "query": { "select": [ { "expression": { "type": "COLUMN", "column": "overall_score", "source": "data" } }, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "groupBy": ["overall_score"] }, "config": { "x_column": "overall_score" } } ``` *** ### Use Case 2: Technical Task Score Distribution Show distribution for evaluations in the technical category only. ![Technical Task Distribution](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/histogram/histogram-technical-distribution.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=7ed009283e0628719a318bf7c564f8b6) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Technical Task Score Distribution", type="histogram", query={ "select": [ {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "filter": { "conditions": [ {"column": "prompt_category", "source": "data", "operator": "=", "value": "technical"} ] }, "group_by": ["overall_score"] }, config={"x_column": "overall_score"} ) ``` ```json JSON theme={null} { "title": "Technical Task Score Distribution", "type": "histogram", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}} ], "filter": { "conditions": [ {"column": "prompt_category", "source": "data", "operator": "=", "value": "technical"} ] }, "groupBy": ["overall_score"] }, "config": { "x_column": "overall_score" } } ``` *** ### Use Case 3: Comparing Distributions Across Categories Compare average fluency and relevance at each overall score using stacked bars. Set `stacked` to `False` in Python or `false` in JSON for adjacent bars. ![Grouped Histogram Comparing Agents](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/histogram/histogram-grouped.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=a3badb6be3afb37721610c080d7eed27) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Score Distribution by Score", type="histogram", query={ "select": [ {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "fluency_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data"}} ], "group_by": ["overall_score"] }, config={ "x_column": "overall_score", "stacked": True } ) ``` ```json JSON theme={null} { "title": "Score Distribution by Score", "type": "histogram", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "fluency_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data"}} ], "groupBy": ["overall_score"] }, "config": { "x_column": "overall_score", "stacked": true } } ``` ## Stacked vs Grouped Distributions When comparing distributions across groups, you can control visualization style: ### Grouped Distributions (Unstacked) ![Grouped Histogram](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/histogram/histogram-grouped-unstacked-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=760209054aa1cc95dc10e72e940d0366) Multiple distributions appear as adjacent bars for each score value. Best for: * Comparing distribution shapes directly * Identifying differences in peaks and spread * Seeing all distributions simultaneously ```python theme={null} config={"x_column": "overall_score", "stacked": False} ``` ### Stacked Distributions ![Stacked Histogram](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/histogram/histogram-grouped-stacked-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=d98146b83fd8ce150114b1de27d5dfce) Distributions stacked vertically to show combined totals. Best for: * Showing total counts across all groups * Part-to-whole relationships * When cumulative view is meaningful ```python theme={null} config={"x_column": "overall_score", "stacked": True} # Default ``` ## Display Behavior Histograms apply the following formatting automatically. These behaviors are built into the rendering, so there are no extra config fields to set. ### Per-category colors When a histogram has a single series and more than one bucket, each bar is given a distinct color from a rotating palette, making individual buckets easier to tell apart. Histograms with multiple series keep one consistent color per series instead, so the series stay comparable across buckets. ### Humanized column names Raw column names are converted to readable Title Case in axis labels, the legend, and tooltips (for example, `overall_score` becomes "Overall Score"). Aliases that already contain spaces are shown unchanged, so you can override the displayed text by aliasing a column in your query. ## Related Documentation * [Bar Chart Widget](./bar) - For categorical distributions * [Metric Widget](./metric) - Show percentiles and statistical measures * [Table Widget](./table) - Detailed numerical breakdowns * [Query Language](../query-language) - Filter and column syntax # Markdown Widget Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/markdown Add rich text documentation, instructions, and notes to dashboards ![Markdown Widget Example](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/markdown/markdown-base-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=5c56cc7a3152f4082f1615d55a5b5e57) ## When to Use Use markdown widgets to: * Document evaluation methodology * Provide context for dashboard viewers * Add instructions or guidelines * Include links to related resources * Create formatted notes and observations * Define thresholds and alert criteria * Credit data sources and team members ## Configuration ### Required Fields **title** `string` *required* Widget title for reference (not displayed in rendered output) **type** `string` *required* Must be set to "markdown" **config.content** `string` *required* Markdown-formatted text to display ### Supported Markdown Syntax Markdown widgets support standard CommonMark syntax: * **Headers**: `# H1`, `## H2`, `### H3` * **Bold**: `**bold text**` * **Italic**: `*italic text*` * **Lists**: Bulleted (`- item`) and numbered (`1. item`) * **Links**: `[text](url)` * **Code**: Inline `` `code` `` and fenced code blocks * **Blockquotes**: `> quote` * **Horizontal rules**: `---` * **Tables**: Standard markdown tables ## Creating in the UI **1. Open Widget Creator** From your dashboard, click "Add Widget" and select "Markdown" **2. Enter Title** Give your widget a reference name (e.g., "Dashboard Overview") **3. Write Content** ![Markdown Editor](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/markdown/markdown-base-form.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=3968526076348283c5852a9d14433439) Write or paste your markdown content in the editor. Most editors provide: **4. Preview** ![Markdown Preview](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/markdown/markdown-base-form-preview.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=c2f1ecd25230cde6e6c35de4958c091a) Check the rendered output to ensure formatting looks correct **5. Create Widget** Click "Add" to add to your dashboard ![Created Markdown Widget](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/markdown/markdown-base-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=5c56cc7a3152f4082f1615d55a5b5e57) ## Related Documentation * [Heading Widget](./heading) - For simple section headers * [All Widget Types](./overview) - Compare widget types * [Getting Started](../getting-started) - Dashboard creation workflow * [API Reference](/reference/v5/evaluation-dashboards/add-widget-to-dashboard) - Programmatic markdown widget creation # Metric Widget Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/metric Display single aggregation values like averages, counts, and percentiles > [!NOTE] > > **Using Sample Data**: The examples on this page use fields from our [sample dataset](/assets/sample-data/sample-evaluation.csv). Download it and create an evaluation with it to follow along with this tutorial. ![Metric Widget Example](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/metric/metric-base-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=ee9eaf3c085b1f481ff6607e639f404b) ## When to Use Use metric widgets to: * Display top-level aggregation values (e.g., "Average Model Score: 0.87") * Show summary statistics (e.g., "Total Items: 1,247") * Highlight key thresholds (e.g., "95th Percentile Latency: 234ms") * Create metric cards for at-a-glance monitoring * Track pass rates and percentages ## Configuration ### Required Fields **title** `string` *required* Widget display name shown on the dashboard **type** `string` *required* Must be set to "metric" **query** `object` *required* MetricQuery with exactly one aggregation function ### Query Requirements Metric widgets require a **MetricQuery** with these constraints: * Must have exactly one item in the `select` clause * That item must be an `AGGREGATION` expression (not a raw column) * Can optionally include `filter` conditions * Cannot use `groupBy`, `orderBy`, or `limit` ## Creating in the UI **1. Open Widget Creator** From your dashboard, click "Add Widget" and select "Query Value" **2. Enter Title** Give your metric a descriptive name (e.g., "Average Score") **3. Configure Query** ![Metric Query Builder](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/metric/metric-base-form.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=d1f4df63b9490038c57d2d2f8ee301f6) * **Column Name** (under **Data Source**): Select the field to aggregate * **Aggregate by**: Choose the function * **Filter** (optional): Add conditions to narrow the data **4. Create Widget** Click "Add" to compute and display the result ![Created Metric Widget](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/metric/metric-base-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=ee9eaf3c085b1f481ff6607e639f404b) ## Output Format Metric widgets return a single scalar value: ```json theme={null} { "type": "metric", "data": 0.873 } ``` The `data` field contains the computed value, which can be: * A decimal number (0.873) * An integer (1247) * A percentage (0.85 representing 85%) * An ISO 8601 string, for `MIN` or `MAX` on the `start_timestamp` or `end_timestamp` fields in `data` ## Example Use Cases ### Use Case 1: Average Overall Score Display the mean overall score across all evaluation items. ![Average Overall Score Metric](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/metric/metric-average-score.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=3665785add22025b82e7ddf0b27a3089) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Average Overall Score", type="metric", query={ "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } } ] } ) ``` ```json JSON theme={null} { "title": "Average Overall Score", "type": "metric", "query": { "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } } ] } } ``` *** ### Use Case 2: Total Evaluation Count Show how many evaluation items are in the dataset. ![Total Evaluations Metric](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/metric/metric-total-evaluations.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=273cb5852f56e52d26f0d15490121566) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Total Evaluations", type="metric", query={ "select": [ { "expression": { "type": "AGGREGATION", "function": "COUNT", "column": "*" } } ] } ) ``` ```json JSON theme={null} { "title": "Total Evaluations", "type": "metric", "query": { "select": [ { "expression": { "type": "AGGREGATION", "function": "COUNT", "column": "*" } } ] } } ``` *** ### Use Case 3: High Performance Rate Calculate percentage of evaluations scoring 85 or above. ![High Performance Rate Metric](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/metric/metric-high-performance-rate.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=b7a507f9572e1c0ebde3eace2db35679) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="High Performance Rate", type="metric", query={ "select": [ { "expression": { "type": "AGGREGATION", "function": "PERCENTAGE", "column": "overall_score", "params": { "percentage_filters": { "conditions": [ { "column": "overall_score", "source": "data", "operator": ">=", "value": 85 } ] } } } } ] } ) ``` ```json JSON theme={null} { "title": "High Performance Rate", "type": "metric", "query": { "select": [ { "expression": { "type": "AGGREGATION", "function": "PERCENTAGE", "column": "overall_score", "params": { "percentage_filters": { "conditions": [ { "column": "overall_score", "source": "data", "operator": ">=", "value": 85 } ] } } } } ] } } ``` *** ### Use Case 4: 95th Percentile Overall Score Show the 95th percentile to understand top-end performance. ![95th Percentile Overall Score Metric](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/metric/metric-p95-overall-score.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=980c7810f239e893a7e5279c022d84c3) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="95th Percentile Overall Score", type="metric", query={ "select": [ { "expression": { "type": "AGGREGATION", "function": "PERCENTILE", "column": "overall_score", "source": "data", "params": { "percentile": 95 } } } ] } ) ``` ```json JSON theme={null} { "title": "95th Percentile Overall Score", "type": "metric", "query": { "select": [ { "expression": { "type": "AGGREGATION", "function": "PERCENTILE", "column": "overall_score", "source": "data", "params": { "percentile": 95 } } } ] } } ``` *** ### Use Case 5: Average Score for Technical Tasks Calculate average score for only technical task types. ![Technical Tasks Avg Score Metric](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/metric/metric-technical-tasks-avg-score.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=0150722f90849aa377bfcfa33107f602) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Technical Tasks Avg Score", type="metric", query={ "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } } ], "filter": { "conditions": [ { "column": "prompt_category", "source": "data", "operator": "=", "value": "technical" } ] } } ) ``` ```json JSON theme={null} { "title": "Technical Tasks Avg Score", "type": "metric", "query": { "select": [ { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } } ], "filter": { "conditions": [ { "column": "prompt_category", "source": "data", "operator": "=", "value": "technical" } ] } } } ``` ## Related Documentation * [Query Language Reference](../query-language) - Full query syntax guide * [All Widget Types](./overview) - Compare different widget types * [Table Widget](./table) - For multi-row displays * [API Reference](/reference/v5/evaluation-dashboards/add-widget-to-dashboard) - Explore programmatic widget creation # Widget Types Overview Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/overview Overview of all available widget types for evaluation dashboards Dashboard widgets are visualization components that display evaluation data in different formats. This guide provides an overview of all available widget types. ## Available Widget Types ### Data-Driven Widgets These widgets require queries and compute results from evaluation data: > [card] [Metric Widget](./metric) > > Display a single computed value like averages, counts, or percentiles > [card] [Table Widget](./table) > > Show multi-dimensional data in rows and columns with grouping, sorting, and conditional formatting > [card] [Bar Chart](./bar) > > Visualize comparisons across categories with bar charts > [card] [Histogram](./histogram) > > Display data distributions across numerical ranges > [card] [Donut Chart](./donut) > > Show proportional distribution of categories as a donut/pie chart > [card] [Scatter Plot](./scatter) > > Explore correlations between two numerical dimensions > [card] [Timeseries](./timeseries) > > Track metrics over time with line charts ### Static Content Widgets These widgets display static content without queries: > [card] [Markdown Widget](./markdown) > > Add rich text documentation, instructions, and notes > [card] [Heading Widget](./heading) > > Organize dashboards with section headers ## Quick Reference | Widget Type | Purpose | Requires Query | Best For | | - | - | - | - | | **Metric** | Single aggregation value | Yes | Top-level metrics, summary stats | | **Table** | Tabular data display | Yes | Detailed breakdowns, grouped data | | **Bar** | Bar chart visualization | Yes | Categorical comparisons | | **Histogram** | Distribution visualization | Yes | Understanding distribution of data | | **Donut** | Proportional distribution | Yes | Category breakdowns, percentage views | | **Scatter** | X-Y relationship visualization | Yes | Correlations, outlier detection | | **Timeseries** | Trends over time | Yes | Tracking metrics across evaluation runs | | **Markdown** | Rich text content | No | Documentation, instructions, notes | | **Heading** | Section headers | No | Dashboard organization | ## Choosing the Right Widget Type ### Display a Single Number Use a **Metric widget** when you need to show one computed value prominently: * Overall average score * Total item count * Pass rate percentage * 95th percentile latency ### Show Detailed Breakdowns Use a **Table widget** when you need to display multiple dimensions and metrics: * Performance by model and category * Top 10 highest/lowest scoring items * Multi-column statistical summaries ### Compare Categories Use a **Bar Chart widget** to visualize how values differ across groups: * Score by category * Model performance comparison * Items per status ### Visualize Distributions Use a **Histogram widget** to understand how values are distributed: * Score distribution * Latency distribution * Confidence level spread ### Show Proportional Breakdowns Use a **Donut Chart widget** to visualize how categories are distributed: * Task type distribution * Agent workload share * Prompt category breakdown ### Explore Correlations Use a **Scatter Plot widget** to examine relationships between two metrics: * Accuracy vs. relevance * Score vs. response length * Latency vs. quality ### Track Trends Over Time Use a **Timeseries widget** to monitor metrics across evaluation runs: * Score progression across evaluations * Multi-metric trends in evaluation groups * Performance tracking over time ### Add Documentation Use **Markdown widgets** for context and instructions: * Dashboard methodology * Alert thresholds * Team contact information * Analysis guidelines ### Organize Sections Use **Heading widgets** to structure your dashboard: * "Overview Metrics" * "Detailed Analysis" * "Performance by Category" ## Widget Configuration Overview All widgets share common fields: **Required Fields:** * `title`: Display name for the widget * `type`: One of: metric, table, bar, histogram, donut, scatter, timeseries, markdown, heading **Query Field (for data-driven widgets):** * `query`: QueryAST object defining the computation **Config Field:** * `config`: Widget-specific display configuration: | Widget type | `config` | | - | - | | `markdown` | Required: `config.content` | | `bar`, `histogram` | Required: exactly one of `config.x_column` or `config.x_column_group` | | `donut` | Required: `config.x_column_group`, even for one column | | `scatter` | Required: `config.x_column` | | Other types | Optional | ## Data Sources Data-driven widgets can query two sources: * **`data.*`**: User-defined evaluation data (your custom fields) * **`task_result_cache.*`**: Task execution results. Only `completed` results are used: items whose result is `pending`, `skipped`, `errored` or `prefilled` are left out of aggregations, filters and groupings on that result. Both support nested fields using dot notation (e.g., `data.metadata.model_version`). ## Reusing Widgets Widgets can be shared across multiple dashboards. When you include the same widget ID in different dashboards' `widget_order`, they all use the same widget definition but produce different results based on their evaluation context. > [!WARNING] > > When reusing widgets, make sure the shape of the data for each evaluation is the same. Otherwise, the widget will not be able to compute the result. ## Next Steps Explore detailed documentation for each widget type: * [Metric Widget](./metric) - Single value displays * [Table Widget](./table) - Tabular data with conditional formatting * [Bar Chart Widget](./bar) - Category comparisons * [Histogram Widget](./histogram) - Distribution visualization * [Donut Chart Widget](./donut) - Proportional distributions * [Scatter Plot Widget](./scatter) - Correlation analysis * [Timeseries Widget](./timeseries) - Trends over time * [Markdown Widget](./markdown) - Rich text content * [Heading Widget](./heading) - Section headers Or continue learning: * [Evaluation Group Dashboards](../eval-group-dashboards) - Compare across evaluations * [Query Language](../query-language) - Build powerful queries * [Getting Started](../getting-started) - Create your first dashboard * [API Reference](/reference/v5/evaluation-dashboards/add-widget-to-dashboard) - Explore programmatic query creation # Scatter Plot Widget Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/scatter Visualize relationships between two numerical dimensions > [!NOTE] > > **Using Sample Data**: The examples on this page use fields from our [sample dataset](/assets/sample-data/sample-evaluation.csv). Download it and create an evaluation with it to follow along with this tutorial. ![Scatter Plot Widget Example](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/scatter/scatter-base-result.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=7d40e53e253e69a5bb12a700ceab1d16) ## When to Use Use scatter plot widgets to: * Explore correlations between two numerical metrics * Identify outliers in multi-dimensional data * Visualize X-Y relationships, with larger dots where several items share a point * Spot clusters or patterns across evaluation items ## Configuration ### Required Fields **title** `string` *required* Widget display name shown on the dashboard **type** `string` *required* Must be set to "scatter" **query** `object` *required* SeriesQuery selecting raw numeric columns for the x and y axes. Select only those two columns to make the axes predictable. ### X-Axis Configuration Set `config.x_column`. The API also accepts `config.x_column_group` in its place, but the scatter plot doesn't draw it. **config.x_column** `string` *required* The `COLUMN` item in `select` to plot on the x-axis, matched by its alias or column name. Must be a numeric column. The first other `COLUMN` item in `select` is plotted on the y-axis. **Example:** `"accuracy_score"` - plots accuracy on the x-axis ### Query Requirements Scatter plot widgets require a **SeriesQuery** with: * At least two numeric `COLUMN` items selected (one for x-axis, one for y-axis) * Only `COLUMN` items. An aggregation collapses the result to one row with no x or y values, and the chart is empty. The UI has no Group By option * Rows without a numeric x or y value are skipped You can include a `filter` to narrow the items. Use raw item rows without `groupBy`, as shown below. > [!NOTE] > > Rows with the same x and y values are drawn as one dot. Dot size grows with the number of rows at that point, scaled relative to the largest duplicate count in the result. Dot size is calculated automatically; you don't need to select a count or size column. ## Creating in the UI **1. Open Widget Creator** From your dashboard, click "Add Widget" and select "Scatter Plot" **2. Enter Title** Give your chart a descriptive name (e.g., "Accuracy vs. Relevance") **3. Configure Axes** ![Scatter Plot Form](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/scatter/scatter-base-form.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=2f6f4bdbb0829e7dc74e4b92c45e7fc2) * **X-Axis**: Select the numeric column for the horizontal axis, with an optional alias * **Y-Axis**: Select the numeric column for the vertical axis, with an optional alias * **Filter**: Add conditions to narrow the data (optional) **4. Create Widget** Click "Add" to generate the scatter plot ![Created Scatter Plot Widget](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/scatter/scatter-base-result.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=7d40e53e253e69a5bb12a700ceab1d16) ## Output Format Scatter plot widgets return one row per evaluation item: ```json theme={null} { "type": "series", "data": [ {"accuracy_score": 92, "relevance_score": 85}, {"accuracy_score": 92, "relevance_score": 85}, {"accuracy_score": 78, "relevance_score": 91} ], "column_order": ["accuracy_score", "relevance_score"] } ``` Each row is a point on the scatter plot. In this example, the two rows at `(92, 85)` are drawn as one larger dot than the single row at `(78, 91)`. ## Example Use Cases ### Use Case 1: Accuracy vs. Relevance Scores Explore the relationship between accuracy and relevance across evaluation items. ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Accuracy vs. Relevance", type="scatter", query={ "select": [ { "expression": { "type": "COLUMN", "column": "accuracy_score", "source": "data" } }, { "expression": { "type": "COLUMN", "column": "relevance_score", "source": "data" } } ] }, config={ "x_column": "accuracy_score" } ) ``` ```json JSON theme={null} { "title": "Accuracy vs. Relevance", "type": "scatter", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "accuracy_score", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "relevance_score", "source": "data"}} ] }, "config": { "x_column": "accuracy_score" } } ``` *** ### Use Case 2: Overall Score vs. Response Length Explore whether longer responses correlate with higher or lower scores. ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Overall Score vs. Response Length", type="scatter", query={ "select": [ { "expression": { "type": "COLUMN", "column": "overall_score", "source": "data" } }, { "expression": { "type": "COLUMN", "column": "response_length", "source": "data" } } ] }, config={ "x_column": "overall_score" } ) ``` ```json JSON theme={null} { "title": "Overall Score vs. Response Length", "type": "scatter", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "response_length", "source": "data"}} ] }, "config": { "x_column": "overall_score" } } ``` ## Related Documentation * [Bar Chart Widget](./bar) - For categorical comparisons * [Histogram Widget](./histogram) - For distribution visualization * [Query Language](../query-language) - Filter and aggregation syntax * [API Reference](/reference/v5/evaluation-dashboards/add-widget-to-dashboard) - Programmatic chart creation # Table Widget Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/table Display multi-dimensional data in rows and columns with grouping and sorting > [!NOTE] > > **Using Sample Data**: The examples on this page use fields from our [sample dataset](/assets/sample-data/sample-evaluation.csv). Download it and create an evaluation with it to follow along with this tutorial. ![Table Widget Example](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/table/table-base-result.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=3e9b4e04543052bd89c7d4f0aa4c2efb) ## When to Use Use table widgets to: * Show detailed breakdowns by multiple dimensions * Display ranked lists (top/bottom performers) * Present multi-column statistical summaries * Create export-friendly data views * Compare metrics across categories * Show individual item details ## Configuration ### Required Fields **title** `string` *required* Widget display name shown on the dashboard **type** `string` *required* Must be set to "table" **query** `object` *required* SeriesQuery returning multiple rows ### Query Requirements Table widgets require a **SeriesQuery** with: * At least one item in the `select` clause * Can mix `COLUMN` and `AGGREGATION` expressions * Supports `filter` for row filtering * Supports `groupBy` for dimensional breakdowns * Supports `limit` for top-N results > [!NOTE] > > When using `groupBy`, `COLUMN` items in `select` that aren't listed in `groupBy` are left out of the result. ## Creating in the UI **1. Open Widget Creator** From your dashboard, click "Add Widget" and select "Table" **2. Turn off Grouping (Optional)** Decide whether to group by columns or not by toggling the "Group Rows" toggle. If you turn off grouping, you will not be able to aggregate data. > [!NOTE] > > When grouping is turned on, we will automatically output the count of the group. **3. Select Columns** If aggregating, specify which columns to group by, if not grouping select the columns you want to display. **4. Enter Title** Give your table a descriptive name (e.g., "Average Scores by Agent") **5. Add Aggregations (Optional)** ![Table Aggregation Builder](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/table/table-base-form-aggregation.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=25e5c3768d58a6624470a0b2a49a611b) Add metrics to display under **Data Source**. Click **Add Data Source**, then set: * **Column Name**: Select the field to aggregate * **Aggregate by**: Choose the function (Average, Count, etc.) **6. Add Filters (Optional)** ![Table Filter Builder](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/table/table-base-form-filter.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=63582210c2d2589c6e281cbf5b400406) Add conditions to narrow the data, these conditions will be applied to the data before it is aggregated. **7. Create Widget** Click "Add" to display the table ![Created Table Widget](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/table/table-base-result-steps.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=6ec4fb35b830a552b904c5295ec9873b) ## Conditional Formatting Apply color-coded formatting rules to table cells to highlight important values at a glance. Conditional formatting works on aggregation columns in grouped tables and on numeric columns in ungrouped tables. ![Table with Conditional Formatting](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/table/table-conditional-formatting.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=473ece437616d7587ef967b968c78049) ### Enabling in the UI When creating or editing a table widget, use the "Formatting" section to add rules. Click "Add Condition", select the column to format, choose a rule type, and pick a color. ![Conditional Formatting Form](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/table/table-conditional-formatting-form.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=cd7d3467b681c585dc6f403ea0ffabfc) ### Rule Types #### Threshold Rules Apply formatting when a cell value meets a comparison condition. | Field | Type | Description | | - | - | - | | `type` | string | Must be `"threshold"` | | `column` | string | Alias of the column to format, or the column name of a `COLUMN` item without an alias. Required unless `column_ref` is set | | `column_ref` | object | The aggregation to format, as `column`, `aggregation`, and optional `params` and `evaluation_ids`. Use it for an aggregation without an alias | | `operator` | string | One of: `=`, `>`, `<`, `>=`, `<=` | | `threshold` | number | Value to compare against | | `background_color` | string | One of: `green`, `yellow`, `red`, `purple`, `blue` | #### Range Rules Apply formatting when a cell value falls within a range (inclusive). | Field | Type | Description | | - | - | - | | `type` | string | Must be `"range"` | | `column` | string | Alias of the column to format, or the column name of a `COLUMN` item without an alias. Required unless `column_ref` is set | | `column_ref` | object | The aggregation to format, as `column`, `aggregation`, and optional `params` and `evaluation_ids`. Use it for an aggregation without an alias | | `min_threshold` | number | Lower bound (inclusive) | | `max_threshold` | number | Upper bound (inclusive) | | `background_color` | string | One of: `green`, `yellow`, `red`, `purple`, `blue` | ### SDK Example ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Agent Scores with Formatting", type="table", query={ "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" }, "alias": "avg_score" } ], "group_by": ["agent_name"] }, config={ "conditional_formatting": [ { "type": "threshold", "column": "avg_score", "operator": ">=", "threshold": 90, "background_color": "green" }, { "type": "range", "column": "avg_score", "min_threshold": 70, "max_threshold": 89, "background_color": "yellow" }, { "type": "threshold", "column": "avg_score", "operator": "<", "threshold": 70, "background_color": "red" } ] } ) ``` ```json JSON theme={null} { "title": "Agent Scores with Formatting", "type": "table", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, { "expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}, "alias": "avg_score" } ], "groupBy": ["agent_name"] }, "config": { "conditional_formatting": [ { "type": "threshold", "column": "avg_score", "operator": ">=", "threshold": 90, "background_color": "green" }, { "type": "range", "column": "avg_score", "min_threshold": 70, "max_threshold": 89, "background_color": "yellow" }, { "type": "threshold", "column": "avg_score", "operator": "<", "threshold": 70, "background_color": "red" } ] } } ``` ### Formatting Behavior * Rules are evaluated in order -- the **first matching rule wins** * Available colors: `green`, `yellow`, `red`, `purple`, `blue` * For percentage columns, formatting comparisons use the display value (0-100), not the raw decimal (0-1) ## Output Format Table widgets return an array of row objects: ```json theme={null} { "type": "series", "data": [ {"category": "Quality", "grouped_count": 342, "score_avg": 0.89}, {"category": "Accuracy", "grouped_count": 298, "score_avg": 0.85}, {"category": "Helpfulness", "grouped_count": 411, "score_avg": 0.91} ], "column_order": ["category", "grouped_count", "score_avg"] } ``` Each object in the `data` array represents one row, with keys corresponding to column names. ## Example Use Cases ### Use Case 1: Agent Performance Breakdown Show statistics grouped by agent with multiple metrics. ![Agent Performance Table](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/table/table-agent-breakdown.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=c27d3734925fb1700fe13aecb1b9b3ec) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Performance by Agent", type="table", query={ "select": [ { "expression": { "type": "COLUMN", "column": "agent_name", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "COUNT", "column": "*" } }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "MIN", "column": "overall_score", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "MAX", "column": "overall_score", "source": "data" } } ], "group_by": ["agent_name"], } ) ``` ```json JSON theme={null} { "title": "Performance by Agent", "type": "table", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "MIN", "column": "overall_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "MAX", "column": "overall_score", "source": "data"}} ], "groupBy": ["agent_name"] } } ``` *** ### Use Case 2: Top 10 Highest-Scoring Evaluations Display the best-performing individual evaluations without aggregation. > [!NOTE] > > In the UI sort using the table header, this will not be reflected in the query. ![Top 10 Evaluations Table](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/table/table-top-10-items.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=6edf26fa2ed1609ed031497726736187) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Top 10 Evaluations", type="table", query={ "select": [ { "expression": { "type": "COLUMN", "column": "agent_name", "source": "data" } }, { "expression": { "type": "COLUMN", "column": "task_type", "source": "data" } }, { "expression": { "type": "COLUMN", "column": "overall_score", "source": "data" } } ], "order_by": [ { "column": "overall_score", "direction": "DESC" } ], "limit": 10 } ) ``` ```json JSON theme={null} { "title": "Top 10 Evaluations", "type": "table", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}} ], "orderBy": [{"column": "overall_score", "direction": "DESC"}], "limit": 10 } } ``` *** ### Use Case 3: Agent Comparison with Multiple Metrics Compare agents across several statistical measures including accuracy and relevance. ![Agent Comparison Table](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/table/table-agent-comparison.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=0bc5687ce49ad7a584177afad49533ec) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Agent Performance Comparison", type="table", query={ "select": [ { "expression": { "type": "COLUMN", "column": "agent_name", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "COUNT", "column": "*" } }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "accuracy_score", "source": "data" } }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data" } } ], "group_by": ["agent_name"] } ) ``` ```json JSON theme={null} { "title": "Agent Performance Comparison", "type": "table", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "COUNT", "column": "*"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "accuracy_score", "source": "data"}}, {"expression": {"type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data"}} ], "groupBy": ["agent_name"] } } ``` *** ### Use Case 4: Low-Scoring Evaluations Review Show evaluations scoring below 80 that need attention. ![Filtered Evaluations Table](https://mintcdn.com/scalegp/3SJ3nurBJCdCWISt/images/v5/evaluation-dashboards/widget-types/table/table-filtered-items.png?fit=max\&auto=format\&n=3SJ3nurBJCdCWISt\&q=85\&s=051d10cdd369cf2ac6fe1a0fbd792721) ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Low-Scoring Evaluations", type="table", query={ "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "accuracy_score", "source": "data"}} ], "filter": { "conditions": [ { "column": "overall_score", "source": "data", "operator": "<", "value": 80 } ] } } ) ``` ```json JSON theme={null} { "title": "Low-Scoring Evaluations", "type": "table", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "agent_name", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "task_type", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "overall_score", "source": "data"}}, {"expression": {"type": "COLUMN", "column": "accuracy_score", "source": "data"}} ], "filter": { "conditions": [ { "column": "overall_score", "source": "data", "operator": "<", "value": 80 } ] } } } ``` ## Related Documentation * [Query Language Reference](../query-language) - `GROUP BY` and `ORDER BY` syntax * [Bar Chart Widget](./bar) - Visual alternative to tables * [Metric Widget](./metric) - For single-value displays * [API Reference](/reference/v5/evaluation-dashboards/add-widget-to-dashboard) - Programmatic table creation # Timeseries Widget Source: https://docs.gp.scale.com/docs/v5/evaluation-dashboards/widget-types/timeseries Track metrics over time with line charts > [!WARNING] > > Timeseries widgets are only supported in evaluation group dashboards. They are not available when creating a dashboard for a single evaluation. ![Timeseries Widget Example](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/timeseries/timeseries-base-result.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=f0faeeb6ae81ecaf29e6683b0023e83e) ## When to Use Use timeseries widgets to: * Track metric trends over time across evaluation runs * Compare performance across evaluations in an evaluation group * Monitor score progression across multiple evaluation iterations * Visualize how metrics change as new evaluations are added to a group ## Configuration ### Required Fields **title** `string` *required* Widget display name shown on the dashboard **type** `string` *required* Must be set to "timeseries" **query** `object` *required* SeriesQuery selecting `_evaluation_id` and at least one aggregation, grouped by `_evaluation_id` ### Optional Config Fields **config.x_axis_label** `string` (default: evaluation_name) How points on the x-axis are labeled: `evaluation_name` or `evaluation_date` (the evaluation's creation date and time). The UI sets it with the **Use Date as X Axis** toggle. ### Query Requirements Timeseries widgets require a **SeriesQuery** with: * A `COLUMN` item for `_evaluation_id` and at least one aggregation in the `select` clause * `groupBy` set to `["_evaluation_id"]` to plot one point per evaluation, as the UI does * The x-axis is determined automatically based on context: * **Evaluation group dashboards**: Plots one point per evaluation, ordered by evaluation creation date, with one line per aggregation * Can include `filter` to narrow the dataset * Supports query-level `evaluation_ids` to choose which evaluations in the group to include (see below) ## Creating in the UI **1. Open Widget Creator** From your dashboard, click "Add Widget" and select "Timeseries" **2. Enter Title** Give your chart a descriptive name (e.g., "Average Score Over Time") **3. Configure Aggregations** ![Timeseries Form](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/timeseries/timeseries-base-form.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=554dd0cfa550c599b3bf62fd9c2e533a) Under **Data Source**, click **Add Data Source** for each metric to track over time: * **Column Name**: Select the numeric field to aggregate * **Aggregate by**: Choose the function (Average, Sum, Count, etc.) * **Alias (optional)**: Name for the metric line (recommended) * For eval group dashboards, the evaluation dropdown at the top of **Data Source** selects which evaluations the widget includes. The selection applies to the whole widget, not to individual aggregations. **4. Add Filters (Optional)** Add conditions to narrow the data before aggregation. **5. Create Widget** Click "Add" to generate the timeseries chart ![Created Timeseries Widget](https://mintcdn.com/scalegp/xaXMSDdu7hTWY1jk/images/v5/evaluation-dashboards/widget-types/timeseries/timeseries-base-result.png?fit=max\&auto=format\&n=xaXMSDdu7hTWY1jk\&q=85\&s=f0faeeb6ae81ecaf29e6683b0023e83e) ## Output Format Timeseries widgets return one row per evaluation, keyed by `_evaluation_id`, when grouped as shown below. To use `_evaluation_name`, include it in the selected columns. The chart labels each point with the evaluation's name or creation date, as set by `config.x_axis_label`, so you don't need to select `evaluation` or `created_at`: ```json theme={null} { "type": "series", "data": [ {"_evaluation_id": "eval-1", "avg_score": 0.82}, {"_evaluation_id": "eval-2", "avg_score": 0.85}, {"_evaluation_id": "eval-3", "avg_score": 0.89} ], "column_order": ["_evaluation_id", "avg_score"] } ``` ## Evaluation Group Context In evaluation group dashboards, timeseries widgets have special behavior: * **One line per metric**: Each metric in the query is plotted as a separate line on the chart * **X-axis**: Points are ordered by evaluation creation date. They are labeled with the evaluation name, or with the creation date when `config.x_axis_label` is `evaluation_date` * **Per-evaluation selection**: You can choose which evaluations the widget includes with `evaluation_ids`, allowing you to compare specific subsets * **Auto-update**: When evaluations are added to or removed from the group, the timeseries automatically updates to reflect the new membership ### Example: Average Score Trend Across Evaluations Track how average scores change across evaluation runs in a group. ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Average Score Trend", type="timeseries", query={ "select": [ {"expression": {"type": "COLUMN", "column": "_evaluation_id"}}, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" }, "alias": "avg_overall_score" } ], "group_by": ["_evaluation_id"], "evaluation_ids": ["eval-1", "eval-2", "eval-3"] } ) ``` ```json JSON theme={null} { "title": "Average Score Trend", "type": "timeseries", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "_evaluation_id"}}, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "overall_score", "source": "data" }, "alias": "avg_overall_score" } ], "groupBy": ["_evaluation_id"], "evaluation_ids": ["eval-1", "eval-2", "eval-3"] } } ``` *** ### Example: Multi-Metric Comparison Over Time Track multiple metrics across evaluations to see how different scores evolve. ```python Python theme={null} widget = client.evaluation_dashboards.widgets.create( dashboard_id=dashboard.id, title="Score Metrics Over Time", type="timeseries", query={ "select": [ {"expression": {"type": "COLUMN", "column": "_evaluation_id"}}, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "accuracy_score", "source": "data" }, "alias": "avg_accuracy" }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data" }, "alias": "avg_relevance" }, { "expression": { "type": "AGGREGATION", "function": "AVG", "column": "coherence_score", "source": "data" }, "alias": "avg_coherence" } ], "group_by": ["_evaluation_id"], "evaluation_ids": ["eval-1", "eval-2", "eval-3", "eval-4"] } ) ``` ```json JSON theme={null} { "title": "Score Metrics Over Time", "type": "timeseries", "query": { "select": [ {"expression": {"type": "COLUMN", "column": "_evaluation_id"}}, { "expression": {"type": "AGGREGATION", "function": "AVG", "column": "accuracy_score", "source": "data"}, "alias": "avg_accuracy" }, { "expression": {"type": "AGGREGATION", "function": "AVG", "column": "relevance_score", "source": "data"}, "alias": "avg_relevance" }, { "expression": {"type": "AGGREGATION", "function": "AVG", "column": "coherence_score", "source": "data"}, "alias": "avg_coherence" } ], "groupBy": ["_evaluation_id"], "evaluation_ids": ["eval-1", "eval-2", "eval-3", "eval-4"] } } ``` ## Related Documentation * [Evaluation Group Dashboards](../eval-group-dashboards) - Full guide on group dashboards * [Bar Chart Widget](./bar) - For categorical comparisons * [Metric Widget](./metric) - For single computed values * [Query Language](../query-language) - Aggregation and filter syntax * [API Reference](/reference/v5/evaluation-dashboards/add-widget-to-dashboard) - Programmatic chart creation # Conditional Rubrics Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/conditional-rubrics Enhance evaluation flexibility and complexity with per-row run conditions ## Overview Rather than a task being applied uniformly across all rows, conditional rubrics allow for rows to be pre-filtered on some `run_condition`. When the condition is false, that task result receives a `"skipped"` status and appears empty in the evaluation results table. Dashboard aggregations on that task-result column exclude its skipped results. The evaluation item itself remains available: `COUNT(*)` and calculations on the original dataset columns can still include the row. The following example filters out rows where the `subject` is not `'math'`. ![conditional-rubric-evaluation-results](https://mintcdn.com/scalegp/LGIHuzR06BuFdDZe/images/nextgen-evaluation/conditional-rubrics/eval-result-with-conditional-rubric.png?fit=max\&auto=format\&n=LGIHuzR06BuFdDZe\&q=85\&s=0d521d3e7ad4c80abd7315a95df9088f) Run conditions are currently supported via the SDK only for auto evaluations of type `guided_decoding`. ### Format The `run_condition` object is a discriminated union on `op` with the following properties supported: * Leaf types: `const`, `var` (item locator references) * Comparison ops: `eq`, `ne`, `lt`, `lte`, `gt`, `gte` * Set ops: `in`, `not_in` * Logical ops: `and`, `or`, `not`, `is_null`, `is_not_null` ### Example #### Simple `run_condition` ``` "run_condition":{ "op": "eq", "left": { "op": "var", "path": "item.subject" }, "right": { "op": "const", "value": "math" } }, ``` #### Nested `run_condition` ``` "run_condition": { "op": "or", "operands": [ { "op": "eq", "left": { "op": "var", "path": "item.Score" }, "right": { "op": "const", "value": 5 } }, { "op": "eq", "left": { "op": "var", "path": "item.Score" }, "right": { "op": "const", "value": 3 } } ] } ``` # Quick Start - AgentOps SDK Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/getting-started Getting started with the ScaleGP AgentOps SDK You can interact with the API through HTTP requests from any language or via our official Python bindings (alpha) ## Authentication Make sure to follow the previous steps described in [Authentication](../authentication) to obtain an API key and account ID. ## Installing the SGP SDK Client The evaluation examples require `scale-gp-beta` `0.1.0` or later. To install a compatible ScaleGP Python client, run: ```bash theme={null} pip install "scale-gp-beta>=0.1.0" ``` ## Configuring the Client The SGP client can be configured with your API key and account ID. Examples on how to use the SGP client can be found in [Recipes](/recipes). The simplest approach is to set environment variables: ```bash theme={null} export SGP_API_KEY="your-api-key-here" export SGP_ACCOUNT_ID="your-account-id-here" ``` Then instantiate the client: ```python theme={null} from scale_gp_beta import SGPClient client = SGPClient() ``` The SDK will automatically use the `SGP_API_KEY` and `SGP_ACCOUNT_ID` environment variables. You can also pass them explicitly: ```python theme={null} from scale_gp_beta import SGPClient client = SGPClient( account_id="your-account-id", api_key="your-api-key", ) ``` The `SGPClient` constructor accepts the following parameters: | Parameter | Description | Default | | - | - | - | | `account_id` | Your SGP account ID. Found in the [Accounts](https://app.sgp.scale.com/admin/accounts) tab of the admin dashboard. | `SGP_ACCOUNT_ID` environment variable | | `api_key` | Your SGP API key. Found in the [API Keys](https://app.sgp.scale.com/admin/api-key) tab of the admin dashboard. | `SGP_API_KEY` environment variable | | `base_url` | The API URL of your SGP deployment, such as `https://api.`. You can also set it with the `SGP_CLIENT_BASE_URL` environment variable. | `https://api.egp.scale.com` | > [!WARNING] > > The Scale GP Python client is under active development. These evaluation examples require version `0.1.0` or later, and some features need a newer version. To upgrade, run `pip install --upgrade scale-gp-beta`. # Auditing Evaluations Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/labeling-evaluations/auditing-evaluations Guide for auditing contributor evaluations ## Overview > [!NOTE] > > Auditing is off by default. Ask your Scale team to enable it for your account. The platform supports auditing in order to enable users to review tasks that are completed by labelers. Currently, the platform supports two levels of auditing. A user can only complete the first level of auditing after the task has been labeled. A user can then only complete the second level of auditing after the first level of auditing has been completed. ## Auditing ![Task Table](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/Task-Table.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=a219258d50b4e49653355fcb9105b338) The task table shows the status of all the tasks. To learn how to assign auditors to tasks, see [Managing Contributor Evaluations](/docs/v5/next-gen-evaluation/labeling-evaluations/managing-contributor-evaluations). Users can assign themselves as auditors with manager permissions. But for each task, the person completing the task, the person doing the first level of audit, and the person doing the second level of audit must all be different people. ### Complete an Audit assigned to you In order to complete an Audit that is assigned to you, click on the **Audit** button in the table of the corresponding task. This will show the task with the auditing functionality. ![Audit Task UI](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/Audit-Task-UI.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=b797c271b1ca168d987228f1795a9abf) * **Mark as "Revision Needed"**: Request the contributor to revise their work * **Fix Task**: Make corrections directly as the auditor * **Approve**: Accept the task as completed correctly If a user has marked the task for review, the audit view will also show that here. This is an example of what a task that has been flagged for review by the labler will look like for the auditor. ![Audit Flag for Review](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/Audit-flag-for-review.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=7700ef8d08cf7ea7609c8816fa0dfc16) There will be a section on the bottom right showing that the original labeler left a comment and flagged the task for review. # Contributor and Auditor Metrics Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/labeling-evaluations/contributor-auditor-metrics Understanding metrics for contributors and auditors > [!NOTE] > > The **Contributor Metrics** section is off by default. Ask your Scale team to enable it for your account. To see an aggregation of the performance of labelers and auditors on the platform, navigate to the **Contributor Metrics** section of annotation tasks. ![Contributor Metrics](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/Contributor-Metrics.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=bdffd402b3c9489d4a441588fecad3f7) This section provides deeper insights into the performance of the contributors that are labeling on this platform. # Labeling Evaluations Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/labeling-evaluations/labeling-evaluations Guide for labeling contributor evaluations ## Overview This guide is designed for human annotators to understand how to label evaluations created through the human evaluations creation flow. For information on creating contributor evaluations, see [Contributor Evaluation](/docs/v5/next-gen-evaluation/ui-evaluations/contributor-evaluation). > [!NOTE] > > **Flag for Review** and auditing are off by default. Ask your Scale team to enable them for your account. ### Start Labeling ![Start Labeling](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/start-labeling.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=c62228e0ce380edd8813bef381a2db36) To start labeling a task, click on the **Label** button next to a task that is assigned to your name. If you don't have any assigned tasks, click **Start Labeling** to claim the next task in the queue. ### Labeling Tasks ![Labeling UI](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/Labeling-UI.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=e302c5af7be92a3dc064a165693ac301) When you open a task, you will see the information related to an evaluation run on the left side of the task screen. On the right side, there will be a series of questions that you should answer as the annotator. * **Save**: There is no save button, and answers are not saved until you submit. If you close a task with unsubmitted answers, the UI asks **Discard changes?**: choose **Keep editing** to return to the task, or **Discard changes** to lose the unsaved answers. * **Skip**: Skip. If you are unsure about how to answer this task, use the Skip button. Skipping unassigns the task from you, moves it to the end of the queue, and opens the next task. * **Submit**: Submit. When you are done with a task, use the Submit button to submit it. Submitting a task allows it to be audited by an auditor. You are allowed to edit a task after you submit if it has not yet been audited. * **Flag for Review**: If you want to submit a task but have concerns about the task you want to flag to an auditor, toggle the flag for review button and leave a comment. The auditor will see your comment and see that you flagged this for review. # Managing Contributor Evaluations Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/labeling-evaluations/managing-contributor-evaluations Guide for managing contributor evaluations ## Overview After setting up a [contributor evaluation](/docs/v5/next-gen-evaluation/ui-evaluations/contributor-evaluation), navigate to the **Annotation Tasks** section in the **Evaluations** tab to manage and monitor your evaluation tasks. ![Annotation Task Overview](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/Annotation-Task-Overview.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=25209d0547f0bf6d8aa9909916c33da6) ## Assigning Tasks Users can assign tasks to others either by going to the **Assigned To** column and selecting a user to assign each individual task to a user. Or they can use the checkbox and bulk assign tasks to a user. ### Individual Task Assignment ![Individual Task Assignment](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/Individual-Assign-Task.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=44049c53d683a7ec0385ddda798dadd2) ### Bulk Assign Tasks ![Bulk Assign Tasks](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/Bulk-Assign-Task.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=14983e08c915d2629929c368b73eb8b1) ## Assigning Auditors to Tasks > [!NOTE] > > Auditing is off by default. Ask your Scale team to enable it for your account. Users can also assign auditors to tasks by selecting the tasks with the checkbox and clicking **Assign Auditors**. The **Audit Stage** column shows the assignee and status of each audit level. The platform allows for two levels of auditing. The first level of auditing can only be completed after the task is completed. The second level of auditing can only be completed after the first level of auditing is completed. For a single task, the user who labels must be different from the first level auditor. The second level auditor must be different from both the first level auditor and the person who did the labeling. ![Select Assign Auditors](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/Select-Assign-Auditors.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=796b53bd1cad4e80555a8a418efe44da) ![Assign Auditors](https://mintcdn.com/scalegp/xptfdMOmHcUOSmtj/images/nextgen-evaluation/contributor-evaluation/Assign-Auditors.png?fit=max\&auto=format\&n=xptfdMOmHcUOSmtj\&q=85\&s=ddce007821ef5b5a9c94144efe296f24) # Introduction to Next Gen Evals Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/overview A simplified and extendable update to SGP's evaluation framework ## Overview Evaluations have been overhauled in the upcoming iteration of SGP to decrease overhead while increasing capabilities for both UI and SDK users. The API is currently available in the latest version of SGP, and access to the new UI is available behind a feature flag, which can be enabled for an account and/or deployment on request. At the most basic level, an evaluation now consists of: 1. **Data**: Unstructured, user defined data, optionally, in the form of a reusable dataset. 2. **Tasks**: A set of instructions to produce results based on said data. For example, a chat completion task based on an `input` field within the user defined data, or a human annotation task to produce the expected output. These tasks can be interdependent and composed to create as simple or complex of an evaluation that your use case requires. ## Creation > [!NOTE] > > Examples in this guide use [`scale-gp-beta`](https://pypi.org/project/scale-gp-beta/)`0.1.0` or later, which runs exclusively on the V5 API. See [SDK installation](./getting-started#installing-the-sgp-sdk-client). ### Simple evaluation creation via SDK The following example illustrates how to create a simple evaluation through the SDK: ```python {9-10, 13-25} theme={null} from scale_gp_beta import SGPClient client = SGPClient(account_id=..., api_key=...) my_evaluation = client.evaluations.create( evaluation={ "name": "My Example Evaluation", "data": [ { "input": "How does NextGen eval on SGP work?" }, { "input": "How do I define tasks?" }, ], "tasks": [ { "task_type": "chat_completion", "alias": "output", "configuration": { "model": "openai/gpt-4o", "messages": [ { "role": "user", "content": "item.input" } ] } } ] } ) ``` In this case, our `data` is made up of two items, each with an `input` field. A single task is specified to generate a chat completion from each data item. ### Evaluation with files creation via SDK Files can be associated with evaluations for use in multimodal tasks or when your evaluation items need to reference uploaded content. First upload your files, then include them in the evaluation: > [!WARNING] > > Evaluations and datasets that contain files must be created via the API or SDK. This functionality is not currently supported in the UI. ```python theme={null} from scale_gp_beta import SGPClient client = SGPClient(account_id=..., api_key=...) # Create evaluation with files my_multimodal_evaluation = client.evaluations.create( evaluation={ "name": "My Multimodal Evaluation", "data": [ { "input_column": "This is the first row of the evaluation" }, { "input_column": "This is the second row of the evaluation" }, ], "files": [ {"file_column1": "sample_file_id_1", "file_column2": "sample_file_id_2"}, {"file_column1": "sample_file_id_3", "file_column2": "sample_file_id_4"}, ], "tasks": [ { "task_type":"contributor_evaluation.question", "alias":"Mispronunciation timestamps_61142007", "configuration":{ "question_id":"61142007-bd18-48a4-b034-a0670a4a23f5", "queue_id":"default", "layout":{ "direction":"column", "children":[ { "data": "item.input" }, { "data": "item.files.file_column1" } ] }, "required": False } } ] } ) ``` In this example, already uploaded files are referenced in the `files` array and they have a one to one mapping to the `data` array. This means that in the resultant evaluation, the first item from data and and the first item from files will collectively form the first row. ## Data Each object within the `data` field can be thought of as a row in a table. We refer to these rows as items, and they can be retrieved like so: ```python theme={null} client.evaluation_items.list( evaluation_id=my_evaluation.id ) ``` **example output (selected fields)** ```python {5,10} theme={null} [ { "id": "0dc5611a-...", "object": "evaluation.item", "data": { "input": "How does NextGen eval on SGP work?" }, }, { "id": "1549a01e-...", "object": "evaluation.item", "data": { "input": "How do I define tasks?" }, } ] ``` Optionally, a dataset can be created for reusability and comparison across different evaluations: ```python {1-7, 12} theme={null} my_dataset = client.datasets.create( name="My Example Dataset", data=[ { "input": "How does NextGen eval on SGP work?" }, { "input": "How do I define tasks?" } ] ) my_evaluation = client.evaluations.create( evaluation={ "name": "My Example Evaluation", "dataset_id": my_dataset.id, "tasks": [ { "task_type": "chat_completion", "alias": "output", "configuration": { "model": "openai/gpt-4o", "messages": [{"role": "user", "content": "item.input"}] } } ] } ) ``` ## Tasks Tasks can be thought of as instructions to produce "results" for each item within the evaluation, and are run asynchronously after evaluation creation. They contain 3 parts: **task_type** `enum` *required* The type of task to run (a list of supported types can be found [here](/reference/v5/evaluations/create-evaluation#body-tasks)) **configuration** `object` *required* Used to specify `task_type` specific parameters, such as the `messages` field for a `chat_completion` task. > [accordion] Referencing item data > > A string following the format `"item."`, here on referred to as an `ItemLocator`, can be used within a configuration to indicate that the value should be pulled from the evaluation item at task execution time. > > For example, in the first code snippet we defined a `chat_completion` task that used an `ItemLocator` as the `content` field for a user message: > > ```python {6} theme={null} > { > "model": "openai/gpt-4o", > "messages": [ > { > "role": "user", > "content": "item.input" > } > ] > } > ``` > > When the task is ultimately executed, a chat completion request will be made for each evaluation item with the `content` field populated based on the item's `input` field. > > Data can also be referenced within a string by wrapping an `ItemLocator` in double curly braces: > > ```python {6} theme={null} > { > "model": "openai/gpt-4o", > "messages": [ > { > "role": "user", > "content": "Answer the following question: {{item.input}}" > } > ] > } > ``` **alias** `string` Aliases are an optional way to specify the field name that will contain the task's result for each evaluation item. The default is a name based on the task type; metric tasks use the metric type and custom functions use the function name. Set an alias when other tasks or dashboards read the result. # Agent Evaluation Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/tasks/agent-evaluation Leverage specialized agents to produce evaluation results. ## Agent-as-a-Judge Use the `auto_evaluation.agent` task type for configuring auto evaluation tasks with specialized agents. > [!NOTE] > > Agent evaluation is off by default. Ask your Scale team to enable it for your account. Until then, creating an evaluation with an `auto_evaluation.agent` task fails with a permission error. **configuration properties** **name** `string` *required* The name of the agent task. **definition** `string` *required* A description of what the agent evaluates. **output_rules** `string[]` *required* Instructions for the agent's evaluation output. **output_type** `enum\<'text' | 'integer' | 'float' | 'boolean'\>` Required when `rubric_id` is not set; if it is omitted, the task fails. Rubric-based tasks use a predefined structured output schema and do not need this field. **designated_to** `object` The agent that runs the task. Omitting `designated_to` selects `APEAgent` with its default configuration. **child attributes** **agent_name** `enum\<'APEAgent' | 'IFAgent'\>` (default: APEAgent) The agent to use. `APEAgent` is the Default Agent and `IFAgent` is the Instruction Following Agent. **config** `object` *required* Set `model` to the model the agent uses to execute the task, in `/` format, for example `openai/gpt-4o`. `model` is the only configuration field that is applied. **rubric_id** `string` The ID of a rubric to score against. When set, the task uses built-in rubric instructions in place of `definition` and `output_rules`, and each result holds `total_score`, `criterion_results` and `version`. **rubric_version** `integer` The version of the rubric given in `rubric_id`. Defaults to the latest version. ### Example Usage The following illustrates a basic example in which an agent task is defined with an Instruction Following Agent to evaluate the adherence of a given response to a prompt. ```python theme={null} client.evaluations.create( evaluation={ "name": "Example Instruction Following Evaluation", "data": [ { "prompt": "You must either describe the process of photosynthesis in exactly 20 words or explain why it’s important in a single sentence under 10 words — choose whichever option is more precise.", "response": "Photosynthesis converts sunlight, water, and carbon dioxide into glucose and oxygen, sustaining plant life and fueling ecosystems.", }, ], "tasks": [ { "task_type": "auto_evaluation.agent", "alias": "instruction-following-agent", "configuration": { "name": "instruction-following-agent", "definition": "Perform comprehensive analysis of instruction following", "output_rules": [ "Return an integer from 1 to 5 scoring adherence; return only the score" ], "output_type": "integer", "designated_to": { "agent_name": "IFAgent", "config": { "model": "openai/o3-mini" } } } } ] } ) ``` # Agentex Evaluation Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/tasks/agentex-evaluation Generate outputs using external Agentex agents. ## Agentex Output Use the `agentex_output` task type to generate outputs from an external Agentex agent for each item in your evaluation. **configuration properties** **agentex_agent_id** `string` *required* The ID of the Agentex agent to use for generating outputs. **input_column** `string` *required* The dataset column to use as input for the agent, specified as an ItemLocator (e.g., `"item.input"`). See [referencing item data](/docs/v5/next-gen-evaluation/overview#referencing-item-data). **include_traces** `boolean` (default: true) Whether to include trace data in the results. When enabled, execution spans from the agent are captured alongside the output. **deployment_id** `string` Pins the evaluation to one Agentex deployment of the agent. When unset, requests go to the agent's default endpoint and follow its current routing rules. Not available in the UI. **timeout_seconds** `integer` (default: 600) Maximum seconds to wait for the agent's first-turn response for each item, from 1 to 1500. **agent_task_params** `object` Extra params merged into the `params` of the Agentex `task/create` call. Use it for agents that require configuration when a task is created, such as a `config_id`. Strings starting with `item.` and `{{item.}}` templates inside it are resolved for each item; other values reach the agent as written. SGP always sets `is_eval: true`. **input_mode** `enum\<'text' | 'data'\>` (default: text) How the resolved `input_column` value is sent to the agent. `text` sends it as a text message, converted to a string. `data` sends it as a data message; the value must be a JSON object or a string that parses to one. Most agents accept only text. **completion_mode** `enum\<'first_message' | 'turn_quiescence'\>` (default: first_message) How the agent's first turn is judged finished. `first_message` grades the first non-empty agent text message after the input. `turn_quiescence` waits until the agent has sent at least one text message and nothing new has arrived for `quiescence_seconds`; use it for tool-using agents. In both modes a finished task ends the wait, and `timeout_seconds` bounds it. **quiescence_seconds** `integer` (default: 30) Seconds without a new message before `turn_quiescence` treats the turn as finished, from 1 to 300. Set it above the agent's longest expected gap between messages, such as a slow tool call. Ignored in `first_message` mode. ### Example Usage The following illustrates a complete example in which an Agentex agent is used to generate responses for each evaluation item. ```python theme={null} from scale_gp_beta import SGPClient import time # Initialize the client client = SGPClient( account_id="your-account-id", api_key="your-api-key" ) # Create an evaluation with Agentex output task evaluation = client.evaluations.create( evaluation={ "name": "My Agent Evaluation", "data": [ {"input": "What is the capital of France?"}, {"input": "Explain quantum computing in simple terms."}, ], "tasks": [ { "task_type": "agentex_output", "alias": "agent_response", "configuration": { "agentex_agent_id": "your-agent-id", "input_column": "item.input", "include_traces": True } } ] } ) print(f"Created evaluation: {evaluation.id}") # Poll until the evaluation reaches a terminal state while True: eval_status = client.evaluations.retrieve(evaluation_id=evaluation.id) print(f"Status: {eval_status.status}") if eval_status.status == "failed": items = client.evaluation_items.list(evaluation_id=evaluation.id) for item in items: if item.task_errors: print(f"Task errors: {item.task_errors}") raise RuntimeError(f"Evaluation {evaluation.id} failed: {eval_status}") if eval_status.status == "completed": items = client.evaluation_items.list(evaluation_id=evaluation.id) for item in items: print(f"\nInput: {item.data['input']}") print(f"Output: {item.data['agent_response']['output']}") if item.data['agent_response'].get('trace'): print(f"Trace spans: {len(item.data['agent_response']['trace'])}") break time.sleep(5) ``` > [!NOTE] > > For Azure workspace deployments, use the `/api` path suffix in the base URL: > > ```python theme={null} > from scale_gp_beta import SGPClient > > client = SGPClient( > account_id="your-account-id", > api_key="your-api-key", > base_url="https://your-workspace.azure.workspace.egp.scale.com/api" > ) > ``` ### Output Structure The task produces the following output for each evaluation item: **Successful response:** When `include_traces=True`, the optional `trace` field contains the agent's execution spans. In this example, no spans were captured. ```json theme={null} { "output": "The agent's final text response", "trace": [] } ``` **Error response:** ```json theme={null} { "output": { "error": "Error message", "error_type": "timeout | null_input | extraction_error | connection_error" } } ``` ### Combining with Other Tasks Evaluation tasks run according to their item-data dependencies. A judge that references `item.agent_response.output` waits for the `agent_response` task; independent tasks can run concurrently. Reference the generated output column in each downstream task to establish the dependency, as in the judge prompt below. ```python theme={null} evaluation = client.evaluations.create( evaluation={ "name": "Agentex with Judge Evaluation", "data": [ { "input": "What is the capital of France?", "expected_output": "Paris" }, ], "tasks": [ # First, generate output from Agentex agent { "task_type": "agentex_output", "alias": "agent_response", "configuration": { "agentex_agent_id": "your-agent-id", "input_column": "item.input", "include_traces": False } }, # This judge depends on item.agent_response.output { "task_type": "auto_evaluation.guided_decoding", "alias": "correctness", "configuration": { "model": "openai/gpt-4o", "prompt": """ Given the user's query: {{item.input}}, The agent's response was: {{item.agent_response.output}} The expected response is: {{item.expected_output}} Did the agent's response correctly answer the question? """, "choices": ["Yes", "No"] } } ] } ) ``` # Auto Evaluation Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/tasks/auto-evaluation Leverage LLM-based tasks to produce evaluation results. ## Guided Decoding Use the `auto_evaluation.guided_decoding` task type for configuring auto evaluation tasks where the set of potential results are well defined. **configuration properties** **model** `string` *required* The identifier of the model (in `/` format) to generate the auto evaluation result. Note this model must support some form of guided decoding (e.g. [OpenAI's response formatting](https://platform.openai.com/docs/guides/structured-outputs)) in order for results to be computed. **system_prompt** `string` An optional system prompt to be sent as the first message of the chat completion request. **prompt** `string` *required* The user prompt containing the evaluation question and any relevant data from the evaluation item (see [referencing item data](/docs/v5/next-gen-evaluation/overview#referencing-item-data)). **run_condition** `object` A condition that determines whether the model will be called for each row. **response_format** `object` A JSON Schema object that defines the response the model must return. Each entry in `properties` is a named response with its own type and constraints. Either `response_format` or `choices` is required. **choices** `string[]` The legacy response format: a non-empty list of answers the model must choose from. When `choices` is present, `response_format` is ignored. **inference_args** `object` Any additional properties to be included in the chat completion request to the model\ (e.g. `{ "temperature": 0 }`). ### Example Usage The following illustrates a basic example in which a guided decoding task is defined to evaluate the correctness of a generated output compared to the ground truth. ```python theme={null} client.evaluations.create( evaluation={ "name": "Example Correctness Evaluation", "data": [ { "input": "What color is the sky?", "expected_output": "Blue", "generated_output": "The sky appears blue during ..." }, ], "tasks": [ { "task_type": "auto_evaluation.guided_decoding", "alias": "correctness", "configuration": { "model": "openai/o3-mini", "prompt": """ Given the user's query: {{item.input}}, The agent's response was: {{item.generated_output}} The expected response is: {{item.expected_output}} Did the agent's response fully represent the expected response? """, "response_format": { "type":"object", "properties": { "question-response": { "type":"string", "enum":["yes","no"] } }, "required":["question-response"] } } } ] } ) ``` When defining a task, you can also customize the response format. For example, you can have the judge LLM provide a reason for the final score, which is often helpful. The following example demonstrates several options. ```python theme={null} tasks = [ { "task_type": "auto_evaluation.guided_decoding", "alias": "multi_response_option_judge", "configuration": { "model": "openai/gpt-4o", "prompt": "Evaluate this response...", "response_format": { "type": "object", "properties": { "is_helpful": { "type": "boolean", "description": "Whether the response is helpful" }, "quality_score": { "type": "integer", "minimum": 1, "maximum": 5, "description": "Quality score from 1 to 5" }, "accuracy_score": { "type": "number", "minimum": 0.0, "maximum": 1.0, "description": "Accuracy as a decimal" }, "category": { "type": "string", "enum": ["excellent", "good", "fair", "poor"], "description": "Quality category" }, "reasoning": { "type": "string", "description": "Explanation of the evaluation" } }, "required": ["is_helpful", "quality_score", "accuracy_score", "category", "reasoning"] } } } ] ``` # Contributor Evaluation Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/tasks/contributor-evaluation Defer evaluation tasks to human contributors. ## Question Use the `contributor_evaluation.question` task type to ask contributors [questions](https://egp.dashboard.scale.com/evaluation/questions/questions) in the annotation UI. **configuration properties** **question_id** `string` *required* The ID of the question to include in the contributor annotation flow. Questions are most easily created through the UI [here](https://egp.dashboard.scale.com/evaluation/questions/questions/new). **layout** `object` *required* Controls the presentation of evaluation item data in the annotation UI. **child attributes** **children** `object[]` *required* A list of elements to be rendered within the layout container. Can be either an individual `Component` object, or a nested `Container`. > [card] > > > [tab] Component > > > > **child attributes** > > > > **data** `ItemLocator` *required* > > > > A pointer to the data in each evaluation item to be displayed within the component. > > > > **label** `string` > > > > An optional display name to be used at the top of the component. Defaults to the field name from the `ItemLocator` used for `data`. > > > [tab] Container > > > > **child attributes** > > > > **children** `object[]` *required* > > > > A list of elements to be rendered within the layout container. Can be either an individual `Component` object, or a nested `Container`. > > > > **direction** `enum\<'row' | 'column'\>` (default: row) > > > > The axis that children are placed in the container. Based on CSS [`flex-direction`](https://developer.mozilla.org/en-US/docs/Web/CSS/flex-direction). **direction** `enum\<'row' | 'column'\>` (default: row) The axis that children are placed in the container. Based on CSS [`flex-direction`](https://developer.mozilla.org/en-US/docs/Web/CSS/flex-direction). **queue_id** `string` (default: default) An optional group identifier. Any `contributor_evaluation` tasks that share the same `queue_id` will be grouped together in the annotation UI. **required** `boolean` (default: false) Whether contributors must answer the question. **rubric_id** `string` The ID of the rubric to use for scoring this question. **prefill_from** `string` A dataset column whose value prefills this question's result. Prefill is off by default. Ask your Scale team to enable it for your account. * The column must exist in the evaluation data. * Every task in the evaluation must be a contributor task. * Rows with a null value in the column stay pending. * With `skip_prefilled_rows` set to `True` on the evaluation, a row isn't queued when every required question on it is prefilled. If no question is required, or a required question has no `prefill_from`, every row is queued. ### Example Usage Assume we have a categorical question with ID `a0b25d45-7c6c-4480-9871-61ae0fd0b819` configured as such: ![](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/correctness-question.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=4697cc6e2832b1266578deed4f5d489f) We can create an annotation queue as part of our evaluation to ask our contributors to answer this question in order to produce the same `correctness` result as the [guided decoding example](/docs/v5/next-gen-evaluation/tasks/auto-evaluation#example-usage): ```python theme={null} client.evaluations.create( evaluation={ "name": "Example Correctness Evaluation", "data": [ { "input": "What color is the sky?", "expected_output": "Blue", "generated_output": "The sky appears blue during ..." }, ], "tasks": [ { "task_type": "contributor_evaluation.question", "alias": "correctness", "configuration": { "question_id": "a0b25d45-7c6c-4480-9871-61ae0fd0b819", "layout": { "direction": "column", "children": [ # Component: { "data": "item.input", "label": "User Query" }, # Nested container: { "direction": "row", "children": [ { "data": "item.expected_output", "label": "Expected Output" }, { "data": "item.generated_output", "label": "Generated Output" } ] } ] }, "queue_id": "default" } } ] } ) ``` # Custom Function Evaluation Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/tasks/custom-function Define your own Python scoring functions and run them as evaluation tasks. ## Overview Use the `custom_function` task type to run your own Python scoring logic on each evaluation item, giving you full control over scoring beyond the built-in metrics. The function source code is extracted, sent to the API, and executed server-side in a sandboxed environment. > [!NOTE] > > The [`scale-gp-beta`](https://pypi.org/project/scale-gp-beta/) SDK provides a `CustomFunction` helper that handles source extraction and serialization for you. ## SDK Helper The `CustomFunction` class wraps a Python callable and provides methods to serialize it for the API and test it with a dry run. ```python theme={null} from scale_gp_beta import SGPClient from scale_gp_beta.lib import CustomFunction client = SGPClient(account_id=..., api_key=...) def string_similarity(expected: str, actual: str) -> float: import difflib return difflib.SequenceMatcher(None, expected, actual).ratio() cf = CustomFunction(func=string_similarity) ``` > [!WARNING] > > Any imports your function needs must be placed **inside the function body**. Module-level imports are not captured when the source is extracted and will not be available at execution time. **CustomFunction constructor parameters** **func** `Callable` *required* The Python function to wrap. Must be a named function defined in a regular `.py` file — lambdas and built-in functions are not supported. **alias** `string` Display name for the task result. Defaults to the function's `__name__`. **arg_mapping** `Dict[str, str]` Maps function parameter names to dataset column names. By default, parameter names are matched directly to column names. Values are automatically prefixed with `item.` unless they already start with `item.` (to support nested paths like `item.nested.field`). ### Allowed Imports Custom functions run in a restricted environment. Only these standard library modules are available: `math`, `json`, `re`, `collections`, `itertools`, `functools`, `random`, `decimal`, `datetime`, `copy`, `textwrap`, `difflib`, `unicodedata` Imported modules expose only their public functions, classes, and constants, not their submodules. From `random`, only `random.Random` is available. Create it inside your function with an explicit, non-`None` seed (for example, `rng = random.Random(42)`); it supports `choice`, `choices`, `randint`, `random`, `randrange`, and `sample`. Module-level helpers such as `random.random()` are unavailable. ### Return Type Functions must return JSON-serializable values: finite numbers, booleans, strings, lists, objects, or `None` (JSON `null`). Non-finite numbers (`NaN` or infinity) and objects that cannot be serialized to JSON produce an error. Structured return values can also be materialized into separate result columns using `configuration.outputs`. ## Using `arg_mapping` By default, function parameter names are matched to dataset column names. Use `arg_mapping` when your function parameters don't match the column names in your data: ```python theme={null} def score(a: float, b: float) -> float: return a - b cf = CustomFunction( func=score, alias="difference_score", arg_mapping={"a": "model_score", "b": "baseline_score"}, ) ``` Values that already start with `item.` are passed through unchanged, which lets you reference nested fields: ```python theme={null} cf = CustomFunction( func=score, arg_mapping={"a": "item.nested.model_score", "b": "baseline_score"}, ) # produces: {"a": "item.nested.model_score", "b": "item.baseline_score"} ``` ## Dry Run Test your function against sample data before creating a full evaluation. The dry run endpoint is synchronous — it executes your function inline and returns per-row results immediately. ```python theme={null} result = cf.dry_run( client=client, sample_data=[ {"expected": "Hello world", "actual": "Hello wrold"}, {"expected": "The quick brown fox jumps over the lazy dog.", "actual": "The quick brown fox jumped over the lazy dog."}, ], ) print(result) ``` ## Discovering Columns If you already have an evaluation and want to see which columns are available for `arg_mapping`: ```python theme={null} from scale_gp_beta.lib import get_evaluation_columns columns = get_evaluation_columns(client, "eval_abc123") for col in columns: print(col.field_name, col.data_type) ``` ## Example Usage The following illustrates a complete example creating an evaluation with a custom function task. ```python theme={null} from scale_gp_beta import SGPClient from scale_gp_beta.lib import CustomFunction client = SGPClient(account_id=..., api_key=...) def string_similarity(expected: str, actual: str) -> float: import difflib return difflib.SequenceMatcher(None, expected, actual).ratio() cf_similarity = CustomFunction(func=string_similarity) def length_ratio(expected: str, actual: str) -> float: if len(expected) == 0: return 0.0 return len(actual) / len(expected) cf_length = CustomFunction(func=length_ratio) evaluation = client.evaluations.create( evaluation={ "name": "Custom Scoring Evaluation", "data": [ {"expected": "Hello world", "actual": "Hello wrold"}, {"expected": "The quick brown fox jumps over the lazy dog.", "actual": "The quick brown fox jumped over the lazy dog."}, ], "tasks": [cf_similarity.serialize(), cf_length.serialize()], } ) ``` You can also create evaluations using the raw task dict directly, without the SDK helper: ```python theme={null} evaluation = client.evaluations.create( evaluation={ "name": "Custom Scoring Evaluation", "data": [ {"expected": "Hello world", "actual": "Hello wrold"}, {"expected": "The quick brown fox jumps over the lazy dog.", "actual": "The quick brown fox jumped over the lazy dog."}, ], "tasks": [ { "task_type": "custom_function", "alias": "string_similarity", "configuration": { "function_source": "def string_similarity(expected: str, actual: str) -> float:\n import difflib\n return difflib.SequenceMatcher(None, expected, actual).ratio()\n", } } ], } ) ``` ### Structured Return Values Use a raw task dictionary when you need separate result columns from a structured return value: ```python theme={null} def score_response(expected: str, actual: str) -> dict: import difflib similarity = difflib.SequenceMatcher(None, expected, actual).ratio() return {"similarity": similarity, "passed": similarity >= 0.9} evaluation = client.evaluations.create( evaluation={ "name": "Structured Scoring Evaluation", "data": [{"expected": "Hello world", "actual": "Hello wrold"}], "tasks": [{ "task_type": "custom_function", "alias": "response_score", "configuration": { "function_source": ( "def score_response(expected: str, actual: str) -> dict:\n" " import difflib\n" " similarity = difflib.SequenceMatcher(None, expected, actual).ratio()\n" " return {\"similarity\": similarity, \"passed\": similarity >= 0.9}\n" ), "outputs": [ {"path": "similarity", "alias": "similarity_score"}, {"path": "passed", "alias": "passed"}, ], }, }], } ) ``` Poll for completion and inspect results: ```python theme={null} import time while True: evaluation = client.evaluations.retrieve(evaluation.id) if evaluation.status in ("completed", "failed"): break time.sleep(5) items = client.evaluation_items.list(evaluation_id=evaluation.id) for item in items: print(item.data) # scores are stored under the task alias key, e.g. data["string_similarity"] if item.task_errors: print(item.task_errors) ``` Once the evaluation completes, you can also view the results — including per-item custom function scores — in the SGP UI. ## Constraints | Constraint | Detail | | :- | :- | | Return type | Any JSON-serializable value (no `NaN` or infinity) | | Imports | Must be inside function body; only allowed stdlib modules | | Function signature | No `*args`, `**kwargs`, async, or lambdas | | Dry run timeout | 10s per row, 30s total | **configuration properties** **function_source** `string` *required* The full source code of the Python function to execute. The `CustomFunction` helper extracts this automatically via `inspect.getsource()`. **arg_mapping** `object` A mapping from function parameter names to `ItemLocator` strings (e.g. `{"param": "item.column_name"}`). When omitted, parameter names are matched directly to column names. **config_args** `object` Literal values for function parameters, such as thresholds or random seeds (e.g. `{"threshold": 0.8}`). A parameter can be set by `arg_mapping` or `config_args`, not both. The serialized JSON can be at most 10,000 characters. **outputs** `object[]` Values from the return value to store as separate result columns, each given as `{"path": ..., "alias": ...}`. `path` is a required dot path into the return value. `alias` names the column and defaults to `path` with dots replaced by underscores. The original return value is always stored under the task alias; without `outputs`, it is stored only there. # Agent as a Judge Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/ui-evaluations/agent-evaluation Leverage specialized agents to produce evaluation results. > [!NOTE] > > Agent judges are off by default. Ask your Scale team to enable them for your account. ## Create Evaluation ### Navigate to the Evaluations tab and click "Create Evaluation". ![create-evaluation](https://mintcdn.com/scalegp/QzzT7_6hBafVZgHm/images/nextgen-evaluation/agent-as-a-judge/create-evaluation.png?fit=max\&auto=format\&n=QzzT7_6hBafVZgHm\&q=85\&s=da1a819832df41bcf72222ae6e666e8b) ### Add Evaluation Details Add in Evaluation name, description (optional), tags (optional), and select a dataset. ![create-evaluation](https://mintcdn.com/scalegp/QzzT7_6hBafVZgHm/images/nextgen-evaluation/agent-as-a-judge/select-eval-dataset.png?fit=max\&auto=format\&n=QzzT7_6hBafVZgHm\&q=85\&s=79979c7939878b1169927278e74f6b69) ### Add an Agent Judge An Agent Judge uses a systematic approach to evaluate your dataset. You can either select an existing agent or create a new one. In the dropdown, you will see all existing judges, both LLM and Agent. ![judge-dropdown](https://mintcdn.com/scalegp/QzzT7_6hBafVZgHm/images/nextgen-evaluation/agent-as-a-judge/judge-dropdown.png?fit=max\&auto=format\&n=QzzT7_6hBafVZgHm\&q=85\&s=9b474c70787da3bcfbd7a391a4878b34) ### Agent Judge Types The judge type picker offers two built-in agent judges, Default Agent and IF Agent, and the Legacy LLM judge.
  • **Default Agent** - Used for general purpose evaluations
  • **IF Agent** - Excels in systematically extracting instructions from a given prompt and assessing a response's adherence to each instruction. The IF Agent can identify implicit and explicit instructions, structure constraints, content requests, and behavioral requirements (tone, audience, vocabulary)
  • **Legacy LLM** - Opens the [LLM Judge](/docs/v5/next-gen-evaluation/ui-evaluations/auto-evaluation#configure-llm-judge) configuration instead of an agent judge.
![select-judge-type](https://mintcdn.com/scalegp/QzzT7_6hBafVZgHm/images/nextgen-evaluation/agent-as-a-judge/select-judge-type.png?fit=max\&auto=format\&n=QzzT7_6hBafVZgHm\&q=85\&s=e7671c065ed49bc14d18f589e33e0dfd) ### Configure Agent Judge Each Agent Judge has its own parameters that can be configured
  • **Model** - The model that the Agent Judge uses to evaluate the dataset.
  • **Output Column Name** - The name of the column the results of this judge will show up on your evaluation results.
  • **Description** - A description of what the agent judge is evaluating.
  • **Output Rules** - A space to provide additional instructions to the agent.
  • **Output Type (optional)** - Text, integer, float, or boolean. If you leave it unset, the output type is Text. Rubric-based tasks use a predefined structured output schema.
![agent-config](https://mintcdn.com/scalegp/QzzT7_6hBafVZgHm/images/nextgen-evaluation/agent-as-a-judge/agent-config.png?fit=max\&auto=format\&n=QzzT7_6hBafVZgHm\&q=85\&s=f753fc2ddea15cb76f38dda5c2e18baf) ### Create Evaluation Select the rows on the dataset you want to run the evaluation, and click **Create Evaluation**. ![create-agent-eval](https://mintcdn.com/scalegp/QzzT7_6hBafVZgHm/images/nextgen-evaluation/agent-as-a-judge/create-agent-eval.png?fit=max\&auto=format\&n=QzzT7_6hBafVZgHm\&q=85\&s=bd47e461623f220591d98d3f108eb7c8) ## View Evaluation Results If you navigate back to the **Evaluations** tab, you should be able to see the results of the evaluation. ### Data The data page will have a column with the results of the Agent Judge ![agent-eval-results](https://mintcdn.com/scalegp/QzzT7_6hBafVZgHm/images/nextgen-evaluation/agent-as-a-judge/agent-eval-results.png?fit=max\&auto=format\&n=QzzT7_6hBafVZgHm\&q=85\&s=36036feff3f2a697cd76683b925e29bc) ### Overview The overview page will have a graph with the visual representation of the evaluation result. ![agent-eval-overview](https://mintcdn.com/scalegp/QzzT7_6hBafVZgHm/images/nextgen-evaluation/agent-as-a-judge/agent-eval-overview.png?fit=max\&auto=format\&n=QzzT7_6hBafVZgHm\&q=85\&s=e28090dc34955651c75389ebd6385d89) # Agentex Agent Output Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/ui-evaluations/agentex-evaluation Generate evaluation outputs using external Agentex agents ## Overview Agentex Agent Output allows you to generate outputs on an evaluation using an Agentex agent. This feature enables you to create an evaluation by selecting a dataset, then configure an Agentex agent to produce a new output column populated by the external agent's responses. This is useful when you want to: * Generate responses from your custom Agentex agents for evaluation * Compare outputs from different agent configurations * Test agent behavior across a variety of inputs in your dataset ## Create Evaluation ### Navigate to the Evaluations tab and click "Create Evaluation". ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/create-evaluation.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=ac97aa300f96b6d6a673417f7e4d70b8) ### Add Evaluation Details Add in Evaluation name, description (optional), tags (optional), and select a dataset. ![evaluation-details](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/evaluation-details.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=4c36a9cf9f0b7ddb332a11d7bfb39159) ### Add an Agentex Agent Output Add an output and select "Agentex Output" from the output type options. This will configure an external Agentex agent to generate outputs for each row in your dataset. > [!NOTE] > > Agentex Output is off by default. Ask your Scale team to enable it for your account. The option appears only when the account has at least one Agentex agent. ![agentex-output-selection](https://mintcdn.com/scalegp/yRlP6_78zayG362m/images/nextgen-evaluation/agentex-output-selection.png?fit=max\&auto=format\&n=yRlP6_78zayG362m\&q=85\&s=960a11018bfc4e52ebc041b2e9a07e27) ### Configure Agentex Output Configure the Agentex agent output with the following parameters:
  • **Output Column Name (optional)** - The name of the header column where the agent's generated outputs will appear in your evaluation results.
  • **Agentex Agent** - The Agentex agent you want to use to generate outputs. This dropdown will display all available Agentex agents in your account. Inactive agents will be greyed out.
  • **Input Column** - The dataset column containing the input data to send to the agent for each row.
  • **Input Delivery** - **Text message** or **JSON data message**. Defaults to JSON for a JSON column and text otherwise. Most agents accept text only.
  • **Agent Task Params** - JSON, or a dataset column, forwarded to the agent when its task is created. Agents that need configuration up front require it, for example an agent that rejects tasks without a `config_id`.
  • **Completion Mode** - **First message** grades the agent's first reply. **Turn quiescence** waits until the agent stops sending messages, which suits tool-using agents.
  • **Quiescence Seconds** - Shown for **Turn quiescence**. Seconds without a new message before the turn counts as finished, from 1 to 300 (default 30).
  • **Timeout Seconds (optional)** - Maximum wait for the agent's response per item, from 1 to 1500 (default 600).
  • **Include Traces** - Toggle to true to include a traces column in the output.
> [!NOTE] > > The input column schema (text or JSON) will be validated against the selected agent's expected schema. If the agent does not have a schema defined, validation will pass without error. ![agentex-output-config](https://mintcdn.com/scalegp/yRlP6_78zayG362m/images/nextgen-evaluation/agentex-output-config.png?fit=max\&auto=format\&n=yRlP6_78zayG362m\&q=85\&s=146bf1a79bb59ff08f8443baae6b60a4) ### Create Evaluation Select the rows on the dataset you want to run the evaluation, and click **Create Evaluation**. ![create-evaluation-final](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/create-evaluation-final.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=4b1458e1bc9b09725015af1be9759540) ## View Evaluation Results If you navigate back to the **Evaluations** tab, you should be able to see the results of the evaluation. ### Data The data page will have a new column containing the outputs generated by your Agentex agent for each row. ![agentex-output-results-data](https://mintcdn.com/scalegp/yRlP6_78zayG362m/images/nextgen-evaluation/agentex-output-results-data.png?fit=max\&auto=format\&n=yRlP6_78zayG362m\&q=85\&s=cc8ae8091f82ec840641e6fc260cd0be) ## Combining with Other Tasks Tasks are scheduled by their data dependencies. An LLM Judge or Contributor Evaluation that references the Agentex output column waits for those outputs; independent tasks can run concurrently. Configure each downstream task to reference the generated output column when building a pipeline. For example, you can: 1. Use an Agentex agent to generate outputs 2. Add an LLM Judge to evaluate the quality of those outputs 3. Add a Contributor Evaluation for human review This allows you to build comprehensive evaluation pipelines that leverage both automated agents and human feedback. # LLM as a Judge Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/ui-evaluations/auto-evaluation Leverage LLM-based tasks to produce evaluation results. ## Create Evaluation ### Navigate to the Evaluations tab and click "Create Evaluation". ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/create-evaluation.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=ac97aa300f96b6d6a673417f7e4d70b8) ### Add Evaluation Details Add in Evaluation name, description (optional), tags (optional), and select a dataset. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/evaluation-details.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=4c36a9cf9f0b7ddb332a11d7bfb39159) ### Add an LLM Judge An LLM Judge prompts an LLM to answer questions to evaluate your dataset. You can either select an existing judge or you can configure a new one. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/llm-judge-selection.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=f46490b49b95e2c8c97426a0631c96dc) ### Configure LLM Judge An LLM Judge prompts an LLM to answer questions to evaluate your dataset.
  • **Output Column Name** - The name of the column the results of this judge will show up on your evaluation results.
  • **Model** - The model that the LLM Judge uses to evaluate the dataset.
  • **System Message (Optional)** - Turn on the toggle to add a system prompt: a set of instructions that defines the role and evaluation criteria for a large language model when it's used to assess, score, or make decisions about AI outputs, human responses, or other content.
  • **User Prompt** - The data to evaluate and instructions on how the model should respond. Use `{{curly braces}}` to insert dataset columns.
  • **Response Options** - Constraints to what your Model can output. Each option has a type: Boolean, Numerical Score, Multi Select, Single Select, or String.
![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/configure-judge.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=f6b323d3dd5c55dbe6a03477f6404d4e) ### Create Evaluation Select the rows on the dataset you want to run the evaluation, and click **Create Evaluation**. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/create-evaluation-final.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=4b1458e1bc9b09725015af1be9759540) ## View Evaluation Results If you navigate back to the **Evaluations** tab, you should be able to see the results of the evaluation. ### Data The data page will have a column with the results of the LLM Judge ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/evaluation-results-data.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=0a98f45202c0edf413d84ba5d80b64a5) ### Overview The overview page will have a graph with the visual representation of the evaluation result. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/evaluation-results-overview.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=dfb64e90448200ce77e680c384ec6202) # Contributor Evaluation Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/ui-evaluations/contributor-evaluation Defer evaluation tasks to human contributors. ## Create Evaluation ### Navigate to the Evaluations tab and click "Create Evaluation". ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/create-evaluation.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=ac97aa300f96b6d6a673417f7e4d70b8) ### Add Evaluation Details Add in Evaluation name, description (optional), tags (optional), and select a dataset. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/evaluation-details.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=4c36a9cf9f0b7ddb332a11d7bfb39159) ### Add Contributor Evaluation Contributor Evaluation lets you set up an evaluation that is labeled by humans. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-evaluation-selection.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=9316cd7565f258d6f2e3dba051975a59) ### Configure Contributor Evaluation Configure the contributor Evaluation. #### Annotation Layout configuration The first step is to configure the data pieces that your contributors will see. You can select which columns you want the data to appear, and you can also configure how the data in the columns will show up. For example, for this configuration: ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-evaluation-configuration.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=1cad81017e5d959628bdbb2d228cdbbb) Annotators will see this layout when they label. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-evaluation-view.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=c80d0ef4189e6db996de2a6f9e0116e2) #### Select Questions Next, select the questions you want your contributor to answer. You can select a pre-existing question, or you can add a new one in this workflow. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-evaluation-select-question.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=fcead6dd6a0d36ff83679e8e45553dbe) ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-evaluation-selected-question.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=0c01d8f6fdefeb66c51c6c91849ca560) The questions that you select will show up in the right hand side of the data the contributors will see. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-evaluation-question-view.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=50dd493d3b006e1082c50341125e20d8) ### Create Evaluation Select the rows on the dataset you want to run the evaluation, and click **Create Evaluation**. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-evaluation-create.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=be4f08083d7ba95eff78965f2c322efc) ### Annotate evaluations When you create an annotation, you can now navigate to that evaluation and see the a tab for **Annotation Tasks**. You or assigned labelers can now start labeling these tasks. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-evaluation-annotation-task.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=5ef2bfa43f22a08a8c805c2fdc052f74) ## View Evaluation Results ### Completed annotations. After all the tasks have been annotated, the tasks will show up as completed on the annotation tasks table. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-evaluations-completed.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=c78b4e7896dc4c60a578fbc229ddf880) ### Data The results will show up as a one column for each question on the data table. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-eval-results-data.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=f81aedf789f48e24360a4c54572734a1) ### Overview The results will also show up as a graph with the visual representation of the evaluation result on the overview page. ![create-evaluation](https://mintcdn.com/scalegp/2sSrpizRElJqluR6/images/nextgen-evaluation/contributor-eval-results-overview.png?fit=max\&auto=format\&n=2sSrpizRElJqluR6\&q=85\&s=524cee05217ced729653a9f2051b81f2) # Voice Evaluations Source: https://docs.gp.scale.com/docs/v5/next-gen-evaluation/voice-evaluations End-to-end guide for creating evaluations with audio files ## Overview Voice evaluations allow you to assess audio content through human annotation tasks. This guide walks you through the complete process of uploading audio files and creating evaluations with timestamp-based questions, such as identifying when mispronunciations occur. > [!NOTE] > > Examples in this guide use [`scale-gp-beta`](https://pypi.org/project/scale-gp-beta/)`0.1.0` or later, which runs exclusively on the V5 API. See [SDK installation](./getting-started#installing-the-sgp-sdk-client). ## Step 1: Upload Audio Files Before creating a voice evaluation, you need to upload your audio files to the platform. There are two methods available: ### Method 1: Direct File Upload Upload files directly from your local filesystem: ```python theme={null} from scale_gp_beta import SGPClient client = SGPClient(account_id=..., api_key=...) # Upload a single audio file with open("audio_sample_1.mp3", "rb") as audio: audio_file_1 = client.files.create(file=audio) with open("audio_sample_2.mp3", "rb") as audio: audio_file_2 = client.files.create(file=audio) # Store the file IDs for later use file_id_1 = audio_file_1.id file_id_2 = audio_file_2.id ``` See the [Upload File API reference](/reference/v5/files/upload-file-bytes-directly) for more details. ### Method 2: Import from Cloud Storage If your files are already stored in cloud storage (S3, GCS, etc.), you can import them in batch: ```python theme={null} from scale_gp_beta import SGPClient client = SGPClient(account_id=..., api_key=...) # Import files from cloud storage import_result = client.files.import_from_cloud( files=[ { "container": "my-s3-bucket", "filename": "audio_sample_1.mp3", "filepath": "evaluations/audio/audio_sample_1.mp3", "file_type": "audio/mpeg" }, { "container": "my-s3-bucket", "filename": "audio_sample_2.mp3", "filepath": "evaluations/audio/audio_sample_2.mp3", "file_type": "audio/mpeg" } ] ) # Check each import before using its file ID for result in import_result.results: if result.status != "SUCCESS": raise RuntimeError(f"Cloud import failed for {result.file.filename}: {result.status}") # Extract file IDs from successful import results file_id_1 = import_result.results[0].file.id file_id_2 = import_result.results[1].file.id ``` See the [Import Files API reference](/reference/v5/files/import-files-from-cloud-storage) for more details. ## Step 2: Create a Question Create a timestamp-based question that annotators will use to mark specific points in the audio: ```python theme={null} # Create a timestamp question for identifying mispronunciations question = client.questions.create( question={ "question_type": "timestamp", "name": "Mispronunciation", "prompt": "When did mispronunciation occur?", "configuration": { "multi": True # Set to False if only one timestamp is expected }, } ) # Store the question ID question_id = question.id ``` > [!NOTE] > > Set `"multi": True` in `configuration` if you want annotators to be able to mark multiple timestamps in a single audio file. For voice evaluations, common question types include: > > * **Mispronunciations**: When did the speaker mispronounce words? > * **Disfluencies**: When did stuttering or filler words occur? > * **Emotion changes**: When did the speaker's tone shift? ## Step 3: Create the Voice Evaluation Now combine the uploaded files and question into a complete evaluation: > [!WARNING] > > Evaluations and datasets that contain files must be created via the API or SDK. This functionality is not currently supported in the UI. ```python theme={null} # Create the voice evaluation voice_evaluation = client.evaluations.create( evaluation={ "name": "Voice Quality Evaluation - Mispronunciation Detection", "description": "Evaluate audio samples for mispronunciations", "data": [ { "audio_description": "Customer service call 1" }, { "audio_description": "Customer service call 2" }, ], "files": [ { "audio_file": file_id_1 }, { "audio_file": file_id_2 }, ], "tasks": [ { "task_type": "contributor_evaluation.question", "alias": "mispronunciation_timestamps", "configuration": { "question_id": question_id, "queue_id": "default", "layout": { "direction": "column", "children": [ { "data": "item.audio_description" }, { "data": "item.files.audio_file" } ] }, "required": False } } ] } ) print(f"Evaluation created with ID: {voice_evaluation.id}") ``` ### Understanding the Structure * **`data`**: Each object represents metadata for one audio sample. The items in `data` have a one-to-one mapping with items in `files`. * **`files`**: Each object contains file IDs that correspond to the uploaded audio files. The first item in `files` pairs with the first item in `data` to form the first evaluation row. * **`tasks`**: Defines what annotators will do. Here, we use a `contributor_evaluation.question` task with a timestamp question. See the [Create Evaluation API reference](/reference/v5/evaluations/create-evaluation) for more details. ## Next Steps After creating your voice evaluation: 1. **Monitor Progress**: Check the evaluation status to see when annotation tasks are completed 2. **Annotate**: Contributors will annotate and answer the questions and tasks you have created 3. **Review Results**: Access the annotated timestamps through the evaluation items 4. **Analyze Data**: Export results to analyze patterns in voice quality For more information on working with evaluations, see the [Next Gen Evaluation Overview](/docs/v5/next-gen-evaluation/overview). ## Example Annotation Flow When contributors access the evaluation queue, they'll see a clean interface displaying the audio player along with any metadata you've included (such as call IDs or descriptions). The audio player allows them to play, pause, and scrub through the audio file. As they listen, they can use the timestamp controls to mark specific moments when events occur (e.g., mispronunciations). If the question sets `"multi": True`, they can add multiple timestamps for each audio file. Once satisfied with their annotations, they submit their responses and move on to the next item in the queue. ![annotator-view-audio-file](https://mintcdn.com/scalegp/FwNxoGgxvj2Jth_l/images/nextgen-evaluation/annotator-view-audio-file.png?fit=max\&auto=format\&n=FwNxoGgxvj2Jth_l\&q=85\&s=aa86cb2afb06fe28b8d5010f7c3b97ab) # SGP Overview Source: https://docs.gp.scale.com/docs/v5/overview/index Overview of SGP ## Overview The Scale GenAI Platform (SGP) is the platform Scale's teams use to build, test and deploy Generative AI applications for Scale's customers, using each customer's own data. The platform supports two distinct workflows related to Generative AI application development. The first one is **Agent Building and Execution**, or the act of having an agent that runs. The second one is **Agent Operations**, or the act of managing the agent, reviewing the data that it outputs, and improving that agent. ![overview-diagram](https://mintcdn.com/scalegp/PXJzYavEL11OqpKz/images/nextgen-evaluation/overview/Sgp-Overview-Diagram.png?fit=max\&auto=format\&n=PXJzYavEL11OqpKz\&q=85\&s=a45e02ffb7da7355ecedf9bf6bcebd60) The ultimate goal of SGP is to enable teams of developers to be able to build and improve multiple agents specializing in multiple goals all on the same platform. Scale's teams use SGP to build and maintain the agents they deliver to customers. ## Agentex - Agent Building and Execution ### View Agents in the Platform The agent builder view shows a list of all the agents that are currently being built or monitored in your account. ![agent-overview](https://mintcdn.com/scalegp/TJSx3Jn9QFSSQ-uS/images/nextgen-evaluation/overview/agent-overview.png?fit=max\&auto=format\&n=TJSx3Jn9QFSSQ-uS\&q=85\&s=8fc9de8881f20d2b31113c1a4a9f912e) Clicking into an agent allows users to interact with an agent, monitor the agent, and view evaluations performed by an agent. Users can also see the deployment history of the agent. ![agent-overview](https://mintcdn.com/scalegp/PXJzYavEL11OqpKz/images/nextgen-evaluation/overview/agent-overview-specific-agent.png?fit=max\&auto=format\&n=PXJzYavEL11OqpKz\&q=85\&s=8268258e2a2b77e4ac88517414b50a5d) ### Building Agents with Agentex SGP's goal for building agents is to provide the most developer friendly framework and environment possible. At the core, building agents is just writing code. Our goal is for all developers to continue developing in the environment they are most familiar with at the fastest speed possible. #### Example Agent This is an example of a medical assistant agent we built, a medical assistant that helps users find doctors at a specific hospital based on their symptoms. The agent diagnoses their symptom and then directs them to a doctor. ![agent-overview](https://mintcdn.com/scalegp/PXJzYavEL11OqpKz/images/nextgen-evaluation/overview/agent-overview-example-agent.png?fit=max\&auto=format\&n=PXJzYavEL11OqpKz\&q=85\&s=2de43ebcbf3fcc057172bc9dda63966f) #### Agentex Building Framework and Philosophy. In today's world, pulling an agent from the internet is easy, most agent frameworks focus on helping users get started on writing an agent. The problem that SGP Agentex aims to solve is to help users quickly get from a locally developed agent into a fully functional production quality agent other people can use. Backend - Agentex provides a simple CLI that enables users to run the agent locally. Frontend - Agentex also provides the ability to run a local UI very quickly to allow users to test. Users can then go to their local machine and run an agent. ![agent-overview](https://mintcdn.com/scalegp/PXJzYavEL11OqpKz/images/nextgen-evaluation/overview/agent-overview-agent-ui.png?fit=max\&auto=format\&n=PXJzYavEL11OqpKz\&q=85\&s=01c1604e19f9792fe58a51db69abae52) This experience enables a fast development cycle: * User can test exactly what's going on. * User can make alterations to the code and change the behavior. * User can test again. Agentex has a built a continuous deployment pipeline that takes changes from the agent and pushes them through a series of steps, and automatically deploys your agent for you. ![agent-overview-ci-cd-pipeline](https://mintcdn.com/scalegp/PXJzYavEL11OqpKz/images/nextgen-evaluation/overview/agent-overview-ci-cd-pipeline.png?fit=max\&auto=format\&n=PXJzYavEL11OqpKz\&q=85\&s=4e7a77f0e6ee3bb45a22a3ea6b9989f3) After making changes, users will automaticaly be able to see on the platform the deployment history of the agent. ![agent-overview](https://mintcdn.com/scalegp/PXJzYavEL11OqpKz/images/nextgen-evaluation/overview/agent-overview-specific-agent-deployment.png?fit=max\&auto=format\&n=PXJzYavEL11OqpKz\&q=85\&s=1a30055c93c84ac41f0d3ad0ce1d3845) ### Interacting with Agents through Agentex A user can interact with an agent by clicking "Open Agent" on the page of the specific Agent. ![agent-overview](https://mintcdn.com/scalegp/PXJzYavEL11OqpKz/images/nextgen-evaluation/overview/agent-overview-specific-agent-click-open-agent.png?fit=max\&auto=format\&n=PXJzYavEL11OqpKz\&q=85\&s=62ba89ac782abcf82e5301b1e2ff6315) This will take users to a **fully deployed, hosted version** of the agent developers and end users can interact with. ![agent-overview-deployed-agent](https://mintcdn.com/scalegp/PXJzYavEL11OqpKz/images/nextgen-evaluation/overview/agent-overview-deployed-agent.png?fit=max\&auto=format\&n=PXJzYavEL11OqpKz\&q=85\&s=aa1865d1af9c667b653c361f8a4bb9d5) Since this is a live deployment, all interactions captured from this deployment will show up in our tracing interface (see below as part of Agent Operations) to assist developers with monitoring their application. ## AgentOps- Agent Monitoring and Improvements ### Tracing Agents with AgentOps All agents built and deployed with Agentex (like in the example above) will have their interactions automatically show up in the tracing library. ![agent-overview-tracing-demo](https://mintcdn.com/scalegp/bfTNCUMX53f6WbCD/images/nextgen-evaluation/overview/agent-overview-tracing-demo.png?fit=max\&auto=format\&n=bfTNCUMX53f6WbCD\&q=85\&s=3e56970677b63db673584e3887925d0f) The AgentOps Tracing Platform also has an SDK that allows users to upload their own traces to get the same experience. Users can also see specific details on the execution of the agent in the tracing view. ![agent-overview-tracing-demo-details](https://mintcdn.com/scalegp/TJSx3Jn9QFSSQ-uS/images/nextgen-evaluation/overview/agent-overview-tracing-demo-details.png?fit=max\&auto=format\&n=TJSx3Jn9QFSSQ-uS\&q=85\&s=5522e1eef4ec902d217395066a513759) Users can see the input, output, and all the steps the agent went through for the execution. Engineers can use this data to understand what tools the agent is calling, what steps the agents are going through for reasoning, and how the agent is performing and make adjustements to the agent. ![agent-overview-tracing-demo-details-output](https://mintcdn.com/scalegp/bfTNCUMX53f6WbCD/images/nextgen-evaluation/overview/agent-overview-tracing-details-output.png?fit=max\&auto=format\&n=bfTNCUMX53f6WbCD\&q=85\&s=da1816932183153695e6ac9844553dcb) To learn more about tracing, refer to the [tracing](/docs/v5/tracing/overview) section. ### Evaluating Agents with AgentOps After identifying areas that can be improved from your agents, the AgentOps platform supports developers in running evaluations on their agents. Users can create evaluation datasets through traces and then run evaluations on the platform. ![agent-overview-evaluations](https://mintcdn.com/scalegp/TJSx3Jn9QFSSQ-uS/images/nextgen-evaluation/overview/agent-overview-evaluations.png?fit=max\&auto=format\&n=TJSx3Jn9QFSSQ-uS\&q=85\&s=2cef82717fae2711ef22cda362dcc93f) The platform supports two types of evaluations: * **LLM-as-a-Judge:** Have an LLM evaluate the dataset * **Contributor Evaluation:** Have humans annotate the dataset To learn more about evaluations, refer to the [evaluations](/docs/v5/next-gen-evaluation/overview) section. ### Summary The combination of all these tools supports the agent development lifecycle. 1. Developing the local agent. 2. Review and deploy the local agent. 3. Test the deployed agent. 4. Monitor the deployed agent through traces. 5. Segment out evaluations from traces. SGP handles deployment, tracing and evaluation, so Scale's teams can spend their time on the business logic of each customer's agents. ![agent-overview-summary](https://mintcdn.com/scalegp/bfTNCUMX53f6WbCD/images/nextgen-evaluation/overview/agent-overview-summary.png?fit=max\&auto=format\&n=bfTNCUMX53f6WbCD\&q=85\&s=b0cfda4339dd1dabea1650486117f8df) # Built-in Tracing Providers Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/agentex/built-in-providers Trace model and tool operations with the Agentex OpenAI Agents, LiteLLM, and LangGraph integrations. Agentex provides three built-in tracing providers that automatically create spans for LLM calls, tool invocations, or both. Each provider integrates with a different framework and has different tracing coverage. Configure the SGP processor first using the [Agentex Tracing Quick Start](./overview#quick-start), then use the integration for your framework below. > [!WARNING] > > Built-in providers give you a baseline — auto-traced LLM calls, tool calls, or both, depending on the provider — but truly useful traces require manual instrumentation by the agent implementer: turn-level spans, custom reasoning steps, structured input/output, and proper parent-child linking. Think of built-in providers as the foundation, not the finished product. See [Span Hierarchy & Best Practices](/docs/v5/tracing/integrations/agentex/span-hierarchy) for how to build on top of them. ## Comparison | Provider | LLM Tracing | Tool Tracing | Framework | | - | :-: | :-: | - | | `TemporalStreamingModelProvider` + `TemporalStreamingHooks` (or `run_turn`) | Auto, with Temporal context | With `trace_id` | OpenAI Agents SDK | | `adk.providers.litellm` | With `trace_id` | No; add manual spans around tools | LiteLLM | | `LangGraphTurn` + `UnifiedEmitter` | No; add spans around model nodes | With `trace_id` | LangGraph | *** ## Providers > [tab] OpenAI Agents SDK > > ### Worker and Workflow Setup > > For [Temporal-style agents](/docs/v5/agents/agentex/types/agentic-agent) using the OpenAI Agents SDK, LLM calls are auto-traced via `TemporalStreamingModelProvider`. `TemporalTracingModelProvider` was removed in agentex-sdk 0.19.0. For non-Agentex usage of the OpenAI Agents SDK with the SGP tracing SDK directly, see the [OpenAI Agents Integration](/docs/v5/tracing/integrations/openai-agents) page. > > Import `TemporalStreamingModelProvider`, `ContextInterceptor`, and the `stream_lifecycle_content` activity from `agentex.lib.core.temporal.plugins.openai_agents`. Register `TemporalStreamingModelProvider` with the worker's `OpenAIAgentsPlugin`, register `ContextInterceptor` in the worker's interceptor list, and add `stream_lifecycle_content` to the worker's activities. Follow the [Temporal OpenAI worker template](https://github.com/scaleapi/scale-agentex-python/blob/main/src/agentex/lib/cli/templates/temporal-openai-agents/project/run_worker.py.j2) for the complete worker configuration. > > The tracing flow is: > > 1. Your workflow sets `_task_id`, `_trace_id`, and `_parent_span_id` on the workflow instance > 2. `ContextInterceptor`, registered on the worker, carries these IDs to each activity the workflow starts > 3. `TemporalStreamingModel`, the model the provider returns, creates a span for each `get_response()` call. It raises a `ValueError` if the task, trace or parent span ID is missing. > > ### Auto-Traced Span Structure > > Each LLM call produces a single span named `streaming_model_get_response`: > > ```python theme={null} > # Span input: > { > "model": "gpt-4o", > "has_system_instructions": True, > "input_type": "list", > "tools_count": 3, > "handoffs_count": 0 > } > > # Span output: > { > "new_items": [ > {"type": "message", "content": [{"type": "text", "text": "..."}]}, > {"type": "function_call", "name": "search", "arguments": "..."} > ], > "final_output": "The agent's text response...", > "usage": { > "input_tokens": 1200, > "output_tokens": 300, > "total_tokens": 1500, > "cached_input_tokens": 0, > "reasoning_tokens": 0 > } > } > ``` > > > [!NOTE] > > > > The span name is always `streaming_model_get_response`. The model name appears in the span's input data. When the model input includes tool calls or tool results, the output also lists them under `tool_calls` and `tool_outputs`. > > ### Run a Turn with Tool Tracing > > Call this method inside your configured workflow class. It creates the parent span, sets the model activity context, and passes the same context to the default tool hooks: > > ```python theme={null} > from agentex.lib import adk > from agentex.lib.core.temporal.plugins.openai_agents import run_turn > > async def run_agent_turn(self, agent, task_id: str, user_message: str): > self._task_id = task_id > self._trace_id = task_id > async with adk.tracing.span( > trace_id=task_id, > task_id=task_id, > name="turn", > input={"message": user_message}, > ) as span: > self._parent_span_id = span.id if span else None > result = await run_turn( > agent, > user_message, > task_id=task_id, > trace_id=task_id, > parent_span_id=self._parent_span_id, > ) > if span: > span.output = {"final_output": result.final_output} > return result > ``` > > `run_turn` creates `TemporalStreamingHooks` by default, passes them `trace_id` and `parent_span_id`, and closes any tool spans left open when the run ends. If you supply custom hooks, pass `trace_id` and `parent_span_id` to their constructor yourself; `run_turn` does not apply those values to custom hooks. > > ### Streaming Hooks > > When you call `Runner.run` directly, create `TemporalStreamingHooks` yourself. It streams tool lifecycle events to the UI in real time. With `trace_id` set, it also opens a span for each tool call: > > ```python theme={null} > from agentex.lib.core.temporal.plugins.openai_agents import TemporalStreamingHooks > > hooks = TemporalStreamingHooks( > task_id=params.task.id, > emit_tool_requests=False, # TemporalStreamingModelProvider already posts the tool request > trace_id=params.task.id, > parent_span_id=turn_span.id if turn_span else None, > ) > try: > result = await Runner.run(agent, input_list, hooks=hooks) > finally: > await hooks.close_open_tool_spans() > ``` > > With `TemporalStreamingModelProvider`, the model posts each tool request (`ToolRequestContent`) itself, so `emit_tool_requests=False` keeps it from appearing twice. The hooks stream: > > * `ToolResponseContent` when a tool finishes (name, result) > * `TextContent` on agent handoffs > > If the run stops in the middle of a tool call, `close_open_tool_spans()` ends that tool's span, which is otherwise never written. > > Each tool span takes the bare tool name (no `tool:` prefix) and is a child of `parent_span_id`. Its input holds the tool arguments under `arguments`, and its output holds the result, cut to 2,000 characters, under `result`. > > ### Tracing Gap: Tool Calls > > > [!WARNING] > > > > Tool executions get their own spans only when you pass `trace_id` to `TemporalStreamingHooks`. Without it, tool calls appear only inside the model span's `output.new_items` array as `function_call` entries. > > > > If you need more detail than the tool span records, create manual spans around your tool logic. See [Manual Spans](/docs/v5/tracing/integrations/agentex/span-hierarchy#manual-spans-for-custom-logic). > [tab] LiteLLM > > ### Available Methods > > The `adk.providers.litellm` module exposes four methods, all of which auto-create tracing spans: > > | Method | Streaming | Auto-send to UI | > | - | :-: | :-: | > | `chat_completion` | No | No | > | `chat_completion_stream` | Yes | No | > | `chat_completion_auto_send` | No | Yes | > | `chat_completion_stream_auto_send` | Yes | Yes | > > The "auto-send" variants automatically create a `TaskMessage` that appears in the Agentex chat UI. Set `LLMConfig.stream=True` for streaming methods. `chat_completion_stream` yields chunks and cannot run inside a Temporal workflow; use `chat_completion_stream_auto_send` there. > > ### Code Example > > ```python theme={null} > from agentex.lib import adk > from agentex.lib.types.llm_messages import LLMConfig > > response = await adk.providers.litellm.chat_completion_stream_auto_send( > task_id=params.task.id, > llm_config=LLMConfig( > model="gpt-4o-mini", > messages=state.messages, > temperature=0.7, > stream=True, > ), > trace_id=params.task.id, > parent_span_id=turn_span.id if turn_span else None, > ) > ``` > > ### Auto-Traced Span Structure > > Each method creates a single span named after the method (e.g., `chat_completion_stream_auto_send`): > > * **Input**: The full `LLMConfig`. model name, messages, temperature, and other settings > * **Output**: The complete `Completion` object or `TaskMessage` > * For streaming methods, chunks are aggregated into the final response before being set as span output > > ### Tracing Gap: Streaming Chunks > > > [!NOTE] > > > > Individual streaming chunks are not traced as separate spans. The LLM span captures the final aggregated response. If you need chunk-level visibility, you'll need custom instrumentation. > [tab] LangGraph > > ### Trace a LangGraph Turn > > `create_langgraph_tracing_handler` was removed in agentex-sdk 0.15.0. To trace a LangGraph turn, wrap the graph's `astream()` result in the `LangGraphTurn` adapter and pass it to `UnifiedEmitter`. The emitter sends the turn to Agentex and, when `trace_id` is set, opens spans under `parent_span_id` as the turn streams. Add manual spans inside model nodes when you need model-call timing and I/O. > > ```python theme={null} > from agentex.lib import adk > from agentex.lib.adk import LangGraphTurn, UnifiedEmitter > > async def run_graph_turn(graph, task_id: str, user_message: str): > async with adk.tracing.span( > trace_id=task_id, > task_id=task_id, > name="turn", > input={"message": user_message}, > ) as span: > stream = graph.astream( > {"messages": [{"role": "user", "content": user_message}]}, > config={"configurable": {"thread_id": task_id}}, > stream_mode=["messages", "updates"], > ) > turn = LangGraphTurn(stream) > emitter = UnifiedEmitter( > task_id=task_id, > trace_id=task_id, > parent_span_id=span.id if span else None, > ) > result = await emitter.auto_send_turn(turn) > if span: > span.output = {"final_output": result.final_text} > return result > ``` > > Call this helper from your async agent's event handler with its compiled graph, task ID, and user message. Sync ACP agents iterate over `emitter.yield_turn(turn)` instead of awaiting `auto_send_turn`. For a complete project, use the [async LangGraph tutorial](https://github.com/scaleapi/scale-agentex-python/tree/main/examples/tutorials/10_async/00_base/100_langgraph). > > ### Span Naming > > | Call Type | Span Name | `__span_type__` | > | - | - | - | > | Tool call | The bare tool name, for example `web_search` | Not set (`STANDALONE` in SGP) | > | Reasoning | `reasoning` | Not set (`STANDALONE` in SGP) | > > ### What Gets Traced > > * **Tool calls**: the span opens on the tool request and closes on the matching tool response. Input is the tool arguments, output is the tool result. Coverage depends on the graph emitting matching tool request/response events. > > * **Reasoning**: one span per streamed reasoning block, with the reasoning text as output. > > > [!NOTE] > > > > LLM calls do not get their own spans. Create manual spans if you need them. > > ### Tracing Gap: Tool I/O Detail > > > [!WARNING] > > > > Tool arguments are recorded as a dict, but the tool result is recorded as the text of the tool message, under an `output` key. If you need structured tool results, supplement with manual spans. *** ## Choosing a Provider > [accordion] I'm using OpenAI Agents SDK on Temporal > > Use `TemporalStreamingModelProvider` for automatic LLM call tracing. The `temporal-openai-agents` CLI template passes it to `OpenAIAgentsPlugin` as `model_provider` and adds `ContextInterceptor` to the worker's interceptors. > > For tool-level tracing, call `run_turn` with `trace_id` and `parent_span_id`, or pass them to `TemporalStreamingHooks`. For other custom logic, create manual spans using `adk.tracing.span()`. See [Manual Spans](/docs/v5/tracing/integrations/agentex/span-hierarchy#manual-spans-for-custom-logic) for examples. > [accordion] I'm using LiteLLM directly > > Use the `adk.providers.litellm` methods. All four methods automatically create tracing spans when you pass `trace_id` and `parent_span_id`. Wrap your own tool executions in manual spans. > > The `auto_send` variants are convenient for chat-style agents since they handle both tracing and UI message creation in one call. > [accordion] I'm using LangGraph > > Wrap `graph.astream()` in `LangGraphTurn` and deliver it with `UnifiedEmitter`, passing `trace_id` and `parent_span_id`. Tool calls and reasoning get their own spans; LLM calls do not, so add manual spans inside model nodes as needed. > > You still need to create turn-level parent spans manually to group operations by conversation turn. # Multi-Agent Tracing Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/agentex/multi-agent-tracing Propagate trace context across orchestrator and sub-agents to build unified, cross-agent trace trees. This page builds on the concepts in [Agentex Tracing Overview](./overview) and [Span Hierarchy & Best Practices](./span-hierarchy). Make sure you're familiar with creating spans and setting `parent_id` before diving into multi-agent propagation. ## The Problem In a [multi-agent system](/docs/v5/agents/agentex/overview), each agent creates its own trace by default (`trace_id = task_id`, and each agent has its own task). The result is fragmented observability: the orchestrator's trace shows only its own spans, and each sub-agent's work lives in a separate, disconnected trace. ```mermaid theme={null} graph TD subgraph Trace1["Trace: orchestrator_task_123"] _s1[ ]:::spacer _s1 ~~~ O1 O1["turn"] O1 --> OD["dispatch sub-agents"] end subgraph Trace2["Trace: research_task_456"] _s2[ ]:::spacer _s2 ~~~ R1 R1["turn"] R1 --> RL1["chat_completion_stream_auto_send"] end subgraph Trace3["Trace: analysis_task_789"] _s3[ ]:::spacer _s3 ~~~ A1 A1["turn"] A1 --> AL1["chat_completion_stream_auto_send"] end subgraph Trace4["Trace: synthesis_task_012"] _s4[ ]:::spacer _s4 ~~~ S1 S1["turn"] S1 --> SL1["chat_completion_stream_auto_send"] end classDef spacer fill:none,stroke:none,color:transparent style Trace1 stroke:#e74c3c style Trace2 stroke:#3498db style Trace3 stroke:#2ecc71 style Trace4 stroke:#9b59b6 ``` With the three child agents used in the example below, you end up with four separate traces instead of one unified view of the entire workflow. *** ## The Solution: Trace ID Propagation Pass the orchestrator's `trace_id` to every sub-agent so all spans land in a single trace tree. You also thread `parent_span_id` through each dispatch so the hierarchy forms correctly. ```mermaid theme={null} graph TD subgraph Unified["Trace: orchestrator_task_123"] _s0[ ]:::spacer _s0 ~~~ O1 O1["turn · orchestrator"] O1 --> Sub1["subtask:research"] Sub1 --> RC["task_create"] Sub1 --> RE["event_send"] Sub1 --> R1["turn · research agent"] R1 --> RL1["chat_completion_stream_auto_send"] O1 --> Sub2["subtask:analysis"] Sub2 --> AC["task_create"] Sub2 --> AE["event_send"] Sub2 --> A1["turn · analysis agent"] A1 --> AL1["chat_completion_stream_auto_send"] O1 --> Sub3["subtask:synthesis"] Sub3 --> SC["task_create"] Sub3 --> SE["event_send"] Sub3 --> S1["turn · synthesis agent"] S1 --> SL1["chat_completion_stream_auto_send"] end classDef spacer fill:none,stroke:none,color:transparent ``` Two things make this work: 1. **Same `trace_id` everywhere**: the orchestrator's `task_id` is used as the `trace_id` for all sub-agent spans 2. **Threaded `parent_span_id`**: each dispatch passes the current span's ID so sub-agent work nests correctly *** ## Context Propagation in Temporal For Temporal-based agents, trace context flows automatically through the interceptor system: ```mermaid theme={null} graph LR subgraph WF["Workflow Side"] WI["Workflow Instance"] CO["OutboundInterceptor"] end subgraph Headers["Temporal Transport"] H["Activity Headers"] end subgraph ACT["Activity Side"] CI["InboundInterceptor"] CV["ContextVars"] TM["TemporalStreamingModel"] end WI --> CO CO --> H H --> CI CI --> CV CV --> TM TM --> S["Auto-traced span"] ``` Set `self._task_id`, `self._trace_id`, and `self._parent_span_id` on the workflow instance. Once all three are set, `ContextInterceptor` passes them to each activity the workflow starts, and the model in that activity uses them as the trace and parent span of its auto-traced span. ### Setting Up Context in Your Workflow Use this method inside your existing workflow class after configuring the worker's `ContextInterceptor` and `TemporalStreamingModelProvider` as described in [Built-in Tracing Providers](./built-in-providers). Store the turn span's ID before running the agent: ```python theme={null} from agentex.lib import adk from agentex.lib.core.temporal.plugins.openai_agents import run_turn async def run_traced_turn(self, agent, task_id: str, user_message: str): self._task_id = task_id self._trace_id = task_id async with adk.tracing.span( trace_id=task_id, task_id=task_id, name="turn" ) as span: self._parent_span_id = span.id if span else None result = await run_turn( agent, user_message, task_id=task_id, trace_id=task_id, parent_span_id=self._parent_span_id, ) if span: span.output = {"final_output": result.final_output} return result ``` > [!NOTE] > > `ContextInterceptor` reads `self._task_id`, `self._trace_id`, and `self._parent_span_id` from the workflow instance. With the interceptor registered in the worker, these attributes propagate to its Temporal activities. Cross-agent ACP requests need explicit application-level propagation, as shown below. *** ## Multi-Agent Code Example ### Orchestrator Agent Use `adk.acp.create_task` to create each child task. The `trace_id` and `parent_span_id` arguments on `create_task` trace only the orchestrator's outbound call. Put the shared context in task `params` so the child can read it from `params.task.params`, then send its user input with `adk.acp.send_event`. This example stores the context under a `trace_context` key in the task params. Use the same keys in both agents: ```python theme={null} from agentex.lib import adk from agentex.types.text_content import TextContent async def dispatch_sub_agents( orchestrator_task_id: str, user_message: str, sub_agents: list[str], ): child_tasks = [] async with adk.tracing.span( trace_id=orchestrator_task_id, task_id=orchestrator_task_id, name="turn", input={"sub_agents": sub_agents}, ) as orchestration_span: for sub_agent_name in sub_agents: async with adk.tracing.span( trace_id=orchestrator_task_id, task_id=orchestrator_task_id, parent_id=orchestration_span.id if orchestration_span else None, name=f"subtask:{sub_agent_name}", input={"agent": sub_agent_name}, ) as dispatch_span: dispatch_id = dispatch_span.id if dispatch_span else None child_task = await adk.acp.create_task( agent_name=sub_agent_name, params={"trace_context": { "trace_id": orchestrator_task_id, "parent_span_id": dispatch_id, }}, trace_id=orchestrator_task_id, parent_span_id=dispatch_id, ) await adk.acp.send_event( task_id=child_task.id, agent_name=sub_agent_name, content=TextContent(author="user", content=user_message), trace_id=orchestrator_task_id, parent_span_id=dispatch_id, ) child_tasks.append(child_task) return child_tasks ``` Call this helper from the orchestrator's event handler with its task ID, the input message, and the deployed child-agent names. It creates fresh tasks by omitting `name`; supplying an existing task name would reuse that task and its history. ### Sub-Agent (Child) Add this handler to the child agent's `acp.py` (an async agent using the base ACP type), alongside the [SGP processor initialization](./overview#quick-start). It reads the context stored with its task, uses the child's own task ID for message routing, and uses the orchestrator's ID for span grouping: ```python theme={null} from agentex.lib import adk from agentex.protocol.acp import SendEventParams from agentex.lib.sdk.fastacp.fastacp import FastACP from agentex.lib.types.fastacp import AsyncACPConfig from agentex.lib.types.llm_messages import LLMConfig acp = FastACP.create(acp_type="async", config=AsyncACPConfig(type="base")) @acp.on_task_event_send async def handle_event_send(params: SendEventParams): task_config = params.task.params or {} trace_context = task_config.get("trace_context") or {} trace_id = trace_context.get("trace_id") or params.task.id parent_id = trace_context.get("parent_span_id") user_message = params.event.content.content async with adk.tracing.span( trace_id=trace_id, task_id=params.task.id, parent_id=parent_id, name="turn", input={"message": user_message}, ) as span: response = await adk.providers.litellm.chat_completion_stream_auto_send( task_id=params.task.id, llm_config=LLMConfig( model="gpt-4o-mini", messages=[{"role": "user", "content": user_message}], stream=True, ), trace_id=trace_id, parent_span_id=span.id if span else None, ) if span: span.output = response.model_dump() if response else {"response": None} ``` The auto-send method already delivers the response to the child task's chat UI. If the orchestrator must collect child results, implement that separately in your agent coordination logic; sending an event does not wait for a child's final response. *** ## Resulting Unified Trace Tree With trace propagation, the research, analysis, and synthesis agents appear under the orchestrator's trace, with the outbound ACP operations and one LiteLLM provider span under each child's turn: ```mermaid theme={null} graph TD T["Trace: orchestrator_task_123"] T --> T1["turn · orchestrator"] T1 --> SR["subtask:research"] SR --> SRC["task_create"] SR --> SRE["event_send"] SR --> SRT1["turn · research agent"] SRT1 --> SRL1["chat_completion_stream_auto_send"] T1 --> SA["subtask:analysis"] SA --> SAC["task_create"] SA --> SAE["event_send"] SA --> SAT1["turn · analysis agent"] SAT1 --> SAL1["chat_completion_stream_auto_send"] T1 --> SS["subtask:synthesis"] SS --> SSC["task_create"] SS --> SSE["event_send"] SS --> SST1["turn · synthesis agent"] SST1 --> SSL1["chat_completion_stream_auto_send"] ``` *** ## Verification **1. Run your orchestrator agent** Start the orchestrator: `agentex agents run --manifest manifest.yaml` **2. Trigger sub-agent dispatch** Send a task or message that causes the orchestrator to dispatch work to sub-agents. **3. Check spans via Agentex API** Spans reach the Agentex API only if your agents register the Agentex processor (`AgentexTracingProcessorConfig()`) in each agent process, as in the [Quick Start](./overview#quick-start). Query for all spans in the trace: ```bash theme={null} curl "https://your-agentex-url/spans?trace_id={orchestrator_task_id}" ``` Verify that sub-agent spans share the orchestrator's `trace_id`. **4. Verify hierarchy** Check that each span's `parent_id` references an existing span's `id`. Sub-agent spans should nest under the orchestrator's dispatch spans. **5. Confirm in the SGP UI** Open the [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view), filter by the orchestrator's `task_id`, and confirm the unified tree shows orchestrator and sub-agent spans together. *** ## Troubleshooting > [accordion] Sub-agent spans appear in a separate trace > > The sub-agent is using its own `task_id` as `trace_id` instead of the orchestrator's. Check that: > > * The orchestrator stores its task ID in `params.trace_context.trace_id` when creating the child task > * The sub-agent reads the context from `params.task.params["trace_context"]` and uses its `trace_id`, falling back to its own task ID only for standalone use > * All `adk.tracing.span()` calls in the sub-agent use the propagated `trace_id` > [accordion] Spans are flat instead of nested > > Parent-child relationships are missing. Check that: > > * `parent_span_id` is stored in the child task's `trace_context` configuration along with `trace_id` > * Sub-agent spans set `parent_id` to the received `parent_span_id` > * Inside the sub-agent, child operations (LLM calls, tools) receive `parent_span_id=span.id` from the enclosing turn span > [accordion] Auto-traced LLM spans are orphaned (no parent) > > The workflow's `self._parent_span_id` was not set before executing the activity. Ensure: > > * `self._parent_span_id = span.id` is set **before** calling `workflow.execute_activity()` > * The turn span is created and its `id` is captured before any activity execution > * For Temporal workflows, the `ContextInterceptor` is registered in the worker's interceptor list # Agentex Tracing Overview Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/agentex/overview Instrument Agentex agents with tracing to capture every turn, LLM call, and tool invocation as structured spans in the SGP Traces UI. ## What is Agentex Tracing? Modern AI agents execute complex, multi-step workflows: reasoning, calling tools, making LLM requests, and coordinating with other agents. Without tracing, these workflows are opaque. When something goes wrong or runs slowly, you're left guessing. [Agentex](/docs/v5/agents/agentex/overview) Tracing gives you deep visibility into your Temporal-style agents by capturing structured spans for every meaningful operation. You get: * **One trace per task by default**: use your Agentex `task_id` as `trace_id`, or propagate an orchestrator trace ID across agents * **A structured span hierarchy**: turns, LLM calls, tool invocations, and custom logic all nest via `parent_id` * **Tool and model I/O capture**: see exactly what went into and came out of each operation * **Profiler-ready data**: consistent span naming enables aggregated performance analysis across all your traces > [!NOTE] > > This guide covers tracing specific to Agentex agents. For general tracing concepts (spans, traces, the Traces UI), see [Introduction to Tracing](/docs/v5/tracing/overview). *** ## Architecture ```mermaid theme={null} graph LR subgraph Agent["Agent SDK process (workflow worker or activity)"] MS["adk.tracing.span()"] AL["TemporalStreamingModelProvider"] AT["LangGraphTurn + UnifiedEmitter"] AP["Agentex tracing processor (if configured)"] SP["SGP tracing processor (if configured)"] end subgraph Agentex["Agentex Backend"] API["Span API"] DB[("PostgreSQL")] API --> DB end subgraph SGP["SGP Platform"] SGPAPI["SGP spans API"] UI["Traces UI"] SGPAPI --> UI end MS --> AP AL --> AP AT --> AP MS --> SP AL --> SP AT --> SP AP --> API SP --> SGPAPI ``` Registered processors run in your agent SDK process and write independently to their backends. Since SDK 0.11.1, the Agentex processor is no longer registered by default. Storing a span in Agentex does not forward it to SGP. Configure the SGP processor in every process that emits spans, including separate Temporal activity workers; add the Agentex processor too if you need both destinations. ### Key Identity Relationships | Concept | ID Field | Typical Value | | - | - | - | | Trace | `trace_id` | = `task_id` (the Agentex task UUID) | | Span | `id` | UUID4, auto-generated | | Parent | `parent_id` | `span.id` of the enclosing span | | Agent | `data.__agent_name__` | Agent name from manifest | | Source | `data.__source__` | `"agentex"` (set by processor) | *** ## Core Concepts ### trace_id = task_id Use the Agentex task ID as `trace_id` for a standalone task so you can find its spans by searching for that ID. For a unified multi-agent trace, pass the orchestrator's trace ID to its children as described in [Multi-Agent Tracing](./multi-agent-tracing); child task IDs still route their messages. ### Span Hierarchy Spans form a tree via `parent_id` relationships. A turn-level span is the parent of the LLM calls and tool invocations that happen during that turn. This hierarchy is what the Traces UI renders as an expandable tree and Gantt chart. ### Span Data Model ```python theme={null} class Span(BaseModel): id: str # UUID4 - unique span identifier name: str # Descriptive operation name trace_id: str # Groups spans into a trace (= task_id) start_time: datetime # UTC timestamp when span began end_time: Optional[datetime] # UTC timestamp when span ended parent_id: Optional[str] # Parent span ID for hierarchy input: Optional[Dict] # Operation input data output: Optional[Dict] # Operation output data data: Optional[Dict] # Metadata (__span_type__, __source__, etc.) ``` *** ## Quick Start Start with a configured Agentex project and install the ADK package, `agentex-sdk`, which provides the `agentex.lib` modules used here. The slim `agentex-client` package alone does not provide the ADK. See the [SDK setup guidance](https://github.com/scaleapi/scale-agentex-python/tree/main/adk). **1. Initialize Tracing in Your Agent Process** Set `SGP_API_KEY`, `SGP_ACCOUNT_ID`, and `SGP_CLIENT_BASE_URL` for your SGP deployment. Add this initialization to your `acp.py`; for Temporal agents, also configure each worker process that creates spans. If any of these variables is missing, the agent fails at startup. ```python theme={null} import os from agentex.lib.core.tracing.tracing_processor_manager import ( add_tracing_processor_config, ) from agentex.lib.types.tracing import ( AgentexTracingProcessorConfig, SGPTracingProcessorConfig, ) # Register both destinations for dual-write tracing. add_tracing_processor_config(AgentexTracingProcessorConfig()) add_tracing_processor_config( SGPTracingProcessorConfig( sgp_api_key=os.environ["SGP_API_KEY"], sgp_account_id=os.environ["SGP_ACCOUNT_ID"], sgp_base_url=os.environ["SGP_CLIENT_BASE_URL"], ) ) ``` Registering the SGP processor also initializes the SGP tracing SDK, so `scale_gp_beta.lib.tracing.create_span()` works in the same process without a separate `tracing.init()` call. | Env Var | Description | | - | - | | `SGP_API_KEY` | Your SGP API key | | `SGP_ACCOUNT_ID` | The account receiving your spans | | `SGP_CLIENT_BASE_URL` | Your deployment's full API base URL. If `sgp_base_url` is not set, the SDK falls back to a built-in default host, not your deployment. | **2. Create a Turn Span and a Traced LLM Call** In the `acp.py` of an async agent that uses the base ACP type, use this event handler after the initialization above. If your project already defines `acp`, keep that instance and add the handler to it. ```python theme={null} from agentex.lib import adk from agentex.protocol.acp import SendEventParams from agentex.lib.sdk.fastacp.fastacp import FastACP from agentex.lib.types.fastacp import AsyncACPConfig from agentex.lib.types.llm_messages import LLMConfig acp = FastACP.create(acp_type="async", config=AsyncACPConfig(type="base")) @acp.on_task_event_send async def handle_event_send(params: SendEventParams): user_message = params.event.content.content async with adk.tracing.span( trace_id=params.task.id, task_id=params.task.id, name="turn", input={"message": user_message}, ) as span: response = await adk.providers.litellm.chat_completion_stream_auto_send( task_id=params.task.id, llm_config=LLMConfig( model="gpt-4o-mini", messages=[{"role": "user", "content": user_message}], stream=True, ), trace_id=params.task.id, parent_span_id=span.id if span else None, ) if span: span.output = response.model_dump() if response else {"response": None} ``` The turn span groups the child LLM span. The auto-send method delivers the response to the Agentex chat UI. **3. View in the Traces UI** Run your agent using its manifest, create a task, and send a user message. Open the SGP Traces page, search for the task ID, and inspect the turn span and its child LLM span. ![Trace detail view showing a turn span with a child LLM span](https://mintcdn.com/scalegp/se_SbjtVoeOEtl0L/images/tracing/agentex-trace-details.png?fit=max\&auto=format\&n=se_SbjtVoeOEtl0L\&q=85\&s=9240957b8d30d189091a7c0004de186e) > [!NOTE] > > For a complete reference on the SGP Tracing SDK, see [Creating Traces and Spans](/docs/v5/tracing/sdk/creating-traces-spans). > [!TIP] > > The [Traces table](/docs/v5/tracing/tracing-ui/traces-table#controlling-the-content-preview) preview column extracts text from specific preferred keys in your span `input` and `output` (e.g., `title`, `query`, `content`). If the preview shows an unhelpful value, add a `title` key with the value you want displayed. *** ## The Two Tracing Systems ### Agentex Native Tracing (`adk.tracing`) This is the primary system for Temporal agents. When the Agentex processor is registered, spans are sent to the Agentex backend API; a configured SGP processor independently receives the same SDK span events. 1. `adk.tracing.span()` creates a span with a UUID4 id 2. When the span ends, the processor writes the complete span, with its output and end time, to the Agentex database 3. If inside a Temporal workflow: routes through Temporal activities instead of direct HTTP A span appears in the Agentex backend only after it ends. ### SGP Tracing (`scale_gp_beta.lib.tracing`) This is the SGP platform's native tracing SDK. You can use it directly or as a dual-write processor alongside Agentex tracing. When the SGP processor is registered, it sends each `adk.tracing` span to SGP and adds these keys to the span's metadata: * `__source__`: `"agentex"` * `__agent_name__`, `__agent_id__`, `__acp_type__`: the agent's name, ID and ACP type The SGP processor also writes each span once, when it ends, so a span shows in the SGP Traces UI only after it finishes. ### Which Should You Use? * **For Temporal agents**: Use `adk.tracing.span()`, which handles Temporal activity routing automatically * **For standalone scripts**: Use `scale_gp_beta.lib.tracing.create_span()` directly * **For dual-write**: Explicitly register both `AgentexTracingProcessorConfig()` and `SGPTracingProcessorConfig(...)`, as shown in the Quick Start > [!WARNING] > > Agentex registers no tracing processor by default. `adk.tracing` spans appear in the SGP Traces UI only if you register the SGP processor, and in the Agentex backend only if you register the Agentex processor. See the Quick Start above. *** ## Next Steps * Want to understand what's auto-traced for you? See the [Built-in Tracing Providers](./built-in-providers) guide for the OpenAI Agents SDK, LiteLLM, and LangGraph integrations. * For naming conventions, input/output patterns, and profiler-friendly trace structure, see [Span Hierarchy & Best Practices](./span-hierarchy). * Building a multi-agent system? See [Multi-Agent Tracing](./multi-agent-tracing) to propagate trace context across orchestrator and sub-agents. * For general tracing concepts and the SGP Tracing SDK, see [Introduction to Tracing](/docs/v5/tracing/overview) and [Creating Traces and Spans](/docs/v5/tracing/sdk/creating-traces-spans). * To understand the Traces UI, see the [Traces Table](/docs/v5/tracing/tracing-ui/traces-table), [Profiler](/docs/v5/tracing/tracing-ui/profiler), and [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view) pages. * New to Agentex? Start with the [Agentex Overview](/docs/v5/agents/agentex/overview) and [Agentex Agent Types](/docs/v5/agents/agentex/types/agentic-agent). # Span Hierarchy & Best Practices Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/agentex/span-hierarchy Naming conventions, input/output patterns, and the ideal trace tree structure for Agentex agents. Well-structured traces make the difference between useful observability and noisy data. This page covers how to name spans consistently, structure your trace tree, and populate span input/output so the [Profiler](/docs/v5/tracing/tracing-ui/profiler) and [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view) give you actionable insights. *** ## Naming Conventions Use consistent, parseable span names. The [SGP Profiler](/docs/v5/tracing/tracing-ui/profiler) aggregates performance metrics by span name. Inconsistent naming splits your data into separate rows and makes bottleneck analysis unreliable. | Span Type | Name Pattern | Example | | - | - | - | | Turn | `turn` | `turn`, with the turn number in `data` | | LLM Call | Named by the built-in provider; `llm:` for manual spans | `streaming_model_get_response`, `chat_completion`, `llm:gpt-4o` | | Tool Call | ``, as the built-in providers name tool spans | `search_docs`, `web_search`, `code_interpreter` | | Reasoning | `reasoning:` | `reasoning:plan`, `reasoning:reflect` | | State Update | `state:` | `state:update`, `state:create` | | Sub-task | `subtask:` | `subtask:market_analysis` | | Research Pass | `research_pass:` | `research_pass:1`, `research_pass:2` | > [!WARNING] > > The SGP Profiler aggregates metrics by span name. A constant name such as `turn` groups all turns together; store the turn number in `data`. Use numbered names such as `research_pass:1` consistently when you want separate profiler rows for each pass. These are conventions for manual spans, not restrictions on valid names. *** ## The Ideal Trace Tree A well-structured multi-turn agent trace looks like this: ```mermaid theme={null} graph TD T["Trace: task_abc123"] T --> T1["turn"] T1 --> T1L1["streaming_model_get_response ✦"] T1 --> T1Tool["search_docs ✦"] T1 --> T1L2["streaming_model_get_response ✦"] T1 --> T1S["state:update"] T --> T2["turn"] T2 --> T2R["reasoning:plan"] T2 --> T2Sub1["subtask:market_analysis"] T2Sub1 --> T2RP1["research_pass:1"] T2RP1 --> T2RP1L["chat_completion ✦"] T2RP1 --> T2RP1T["code_interpreter"] T2Sub1 --> T2RP2["research_pass:2"] T2RP2 --> T2RP2L["chat_completion ✦"] T2 --> T2Sub2["subtask:competitor_review"] T2Sub2 --> T2CRP["research_pass:1"] T2CRP --> T2CRPL["chat_completion ✦"] T2CRP --> T2CRPT["web_search"] T2 --> T2S["state:update"] ``` *✦ = auto-traced by built-in providers* This example tree combines Temporal OpenAI Agents model and tool spans with LiteLLM research calls. The two `turn` spans can have different turn numbers in their metadata; unmarked grouping and tool spans are manually instrumented. In the Temporal OpenAI Agents integration, function tools such as a custom `search_docs` function produce tool spans. Hosted OpenAI tools such as Web Search and Code Interpreter appear as tool messages rather than separate tool spans. Key principles: * **Turn spans are the primary grouping unit**: every operation in a conversation turn nests under its turn span * **Auto-traced spans nest automatically** when you pass `parent_span_id` to built-in providers * **Sub-tasks and research passes** create intermediate grouping for complex multi-step operations * **State updates** capture what changed at the end of a turn *** ## Span Input/Output Best Practices Inspect an individual span's input and output in the [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view) span inspector. The [Profiler](/docs/v5/tracing/tracing-ui/profiler) inspector shows aggregate timing metrics and percentile bars for a span name, with links to matching spans. Open one of those spans to inspect its I/O. > [!TIP] > > The [Traces table](/docs/v5/tracing/tracing-ui/traces-table#controlling-the-content-preview) preview column extracts text from specific preferred keys in your span `input` and `output` (e.g., `title`, `query`, `content`). If the preview shows an unhelpful value, add a `title` key with the value you want displayed. ![Agentex profiler view showing aggregated span metrics](https://mintcdn.com/scalegp/se_SbjtVoeOEtl0L/images/tracing/agentex-profiler.png?fit=max\&auto=format\&n=se_SbjtVoeOEtl0L\&q=85\&s=08a9c631fb2e724cda0f2abf05aea229) > [accordion] Turn spans > > * **Input**: The user message or event that triggered this turn, plus current state > * **Output**: The updated state after processing > > ```python theme={null} > async with adk.tracing.span( > trace_id=task_id, > name="turn", > input={"message": user_message, "state": state.model_dump()}, > data={"turn_number": turn_number}, > ) as span: > # ... process turn ... > if span: > span.output = state.model_dump() > ``` > > Comparing input and output on a turn span lets you quickly see what changed during that turn. > [accordion] LLM spans (auto-traced) > > Names and payloads depend on the [built-in provider](/docs/v5/tracing/integrations/agentex/built-in-providers): > > * **Temporal OpenAI Agents**: `streaming_model_get_response` captures `model`, `has_system_instructions`, `input_type`, `tools_count`, and `handoffs_count`. Its output contains `new_items` (serialized response items), `final_output` (extracted text), and `usage` (token counts). > * **LiteLLM**: Spans use the method name, such as `chat_completion` or `chat_completion_stream_auto_send`. Input is the serialized `LLMConfig`; output is the returned `Completion` or `TaskMessage`, depending on the method. > [accordion] Tool spans > > * **Input**: The tool arguments as a dict > * **Output**: The tool result (string or structured data) > > ```python theme={null} > async with adk.tracing.span( > trace_id=task_id, > name=tool_name, > parent_id=turn_span_id, > input={"query": search_query, "max_results": 10}, > ) as span: > result = await execute_tool(tool_name, search_query) > if span: > span.output = {"results": result, "count": len(result)} > ``` > [accordion] Reasoning spans > > * **Input**: The context or question being reasoned about > * **Output**: The reasoning result, confidence score, and planned next steps > > ```python theme={null} > async with adk.tracing.span( > trace_id=task_id, > name="reasoning:plan", > parent_id=turn_span_id, > input={"context": context, "question": user_question}, > ) as span: > plan = await generate_plan(context, user_question) > if span: > span.output = {"plan": plan, "confidence": 0.95, "steps": 3} > ``` > [accordion] State update spans > > * **Input**: Previous state (or the delta being applied) > * **Output**: The new state > > These are most useful when your state changes are complex and you want to track exactly what was modified. *** ## Manual Spans for Custom Logic When [built-in providers](/docs/v5/tracing/integrations/agentex/built-in-providers) don't cover your use case (agent reasoning, data processing, custom tool execution), create manual spans. ### Using Agentex ADK (Temporal-aware) Use `adk.tracing.span()` inside Temporal workflows and activities. It handles Temporal activity routing automatically. ```python theme={null} from agentex.lib import adk async with adk.tracing.span( trace_id=task_id, name="reasoning:reflect", parent_id=turn_span_id, input={"context": "...", "question": "..."}, data={"__span_type__": "CUSTOM"}, ) as span: result = await my_reasoning_function(...) if span: span.output = {"reasoning": result, "confidence": 0.95} ``` > [!NOTE] > > Always check `if span:` before setting attributes. The span is `None` when `trace_id` is falsy, which prevents errors when tracing is disabled. ### Using SGP SDK Directly (non-Temporal) For standalone scripts or non-Temporal agents, use the SGP tracing SDK directly: ```python theme={null} import scale_gp_beta.lib.tracing as tracing with tracing.create_span( name="research_task_activity_market_analysis", trace_id=task_id, parent_id=parent_span_id, input={"system_prompt": "...", "user_input": "...", "num_passes": 2}, ) as span: result = await run_research(...) span.output = {"output_length": len(result), "success": True} ``` *** ## Span Type Mapping to SGP Set `__span_type__` in span `data` to control how the SGP UI categorizes the span: | `__span_type__` | SGP `operation_type` | Use For | | - | - | - | | `STANDALONE` | STANDALONE | Default, generic operations | | `COMPLETION` | COMPLETION | LLM / model calls | | `CUSTOM` | CUSTOM | Tool calls, custom logic | *** ## Example: Multi-Level Research Activity This complete activity groups multiple research passes under one activity span. Each pass contains a child span created by the LiteLLM provider. Register the activity with your Temporal worker and call it with the shared trace ID and parent span ID: ```python theme={null} from temporalio import activity from agentex.lib import adk from agentex.lib.types.llm_messages import LLMConfig @activity.defn async def research_task_activity( task_id: str, parent_span_id: str, section_title: str, user_input: str, model: str = "gpt-4o-mini", num_passes: int = 2, ) -> list[dict]: results = [] async with adk.tracing.span( name="research_task_activity", trace_id=task_id, task_id=task_id, parent_id=parent_span_id, input={"section_title": section_title, "user_input": user_input}, ) as span: for pass_num in range(num_passes): async with adk.tracing.span( name=f"research_pass:{pass_num + 1}", trace_id=task_id, task_id=task_id, parent_id=span.id if span else None, input={ "pass": pass_num + 1, "prior_summary": results[-1]["summary"] if results else "", }, ) as pass_span: response = await adk.providers.litellm.chat_completion( llm_config=LLMConfig( model=model, messages=[{ "role": "user", "content": ( f"Research {section_title}: {user_input}. " f"Pass {pass_num + 1}. Prior summary: " f"{results[-1]['summary'] if results else ''}" ), }], ), trace_id=task_id, parent_span_id=pass_span.id if pass_span else None, ) message = response.choices[0].message if response.choices else None text = (message.content or "") if message else "" result = {"pass": pass_num + 1, "summary": text[:2000]} results.append(result) if pass_span: pass_span.output = result if span: span.output = { "pass_count": len(results), "summary": results[-1]["summary"] if results else "", } return results ``` Each pass keeps at most 2,000 characters of response text for the next prompt and its manual span output. The provider span still captures its normal response. The resulting hierarchy has one provider span per pass: ```text theme={null} research_task_activity ├── research_pass:1 │ └── chat_completion └── research_pass:2 └── chat_completion ``` *** ## Checklist ### Must-Do * Set `trace_id = task_id` consistently across all spans * Create turn-level spans wrapping each conversation turn * Pass `parent_span_id` to all ADK methods and child operations * Set `span.output` before the span closes (inside the context manager) * Use consistent manual span names (`turn`, `web_search`, `llm:gpt-4o`, `research_pass:1`) and retain the provider's automatic names ### Should-Do * Include relevant context in span `input`, not just raw data but structured metadata * Set `__span_type__` in `data` for proper SGP categorization * Handle errors gracefully by setting error output before the span closes * Use nested spans for multi-step operations (research passes, sub-tasks) ### Avoid * Don't create flat traces where everything sits at the root level. Use `parent_id` hierarchy * Don't put massive payloads in input/output. Summarize or truncate large data * Don't forget to check `if span:` before setting attributes (span is `None` when `trace_id` is falsy) * Don't create spans without `trace_id`. They'll be orphaned and invisible in the UI # OpenAI Agents Integration Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/openai-agents Automatically trace openai-agents workflows with the SGP processor. The SGP Tracing SDK provides an integration for the `openai-agents` library via the `OpenAITracingSGPProcessor`. This processor automatically captures detailed traces from your AI agent runs, translating events from the agent's lifecycle into SGP traces and spans. This provides deep visibility into your agent's execution, including tool calls and model invocations, without requiring manual instrumentation. > [!WARNING] > > This integration requires the [`openai-agents`](https://pypi.org/project/openai-agents/) package. The processor will raise an `ImportError` if the package is not installed. *** ## How it Works The `OpenAITracingSGPProcessor` hooks into the `openai-agents` tracing system. You register it once, and it turns each agent run into an SGP trace: * Each run becomes a trace whose root span has the type `AGENT_WORKFLOW`. The root span ends when the run ends. * Each operation in the run, such as a model call or a tool call, becomes a span under its parent operation. The agent's span types map to SGP span types, for example "function" to `CODE_EXECUTION` and "generation" to `COMPLETION`. * When an operation finishes, its span gets the `input`, `output`, and `metadata` that `openai-agents` reports. If `openai-agents` reports an error, the span records it. * The processor sets a tool call's span metadata when the tool returns, which replaces anything set on that span from inside the tool. To record an error message from a tool, set it on a span the tool creates, as the example below does. *** ## Usage Example To use the processor, you must initialize SGP tracing and then register an instance of `OpenAITracingSGPProcessor` with the `openai-agents` library. ```python theme={null} import os import asyncio import requests from bs4 import BeautifulSoup from googleapiclient.discovery import build from googleapiclient.errors import HttpError from agents import Agent, Runner, function_tool, set_trace_processors from datetime import date import scale_gp_beta.lib.tracing as tracing from scale_gp_beta import SGPClient from scale_gp_beta.lib.tracing.integrations import OpenAITracingSGPProcessor @function_tool def google_search(query: str) -> list[str]: with tracing.create_span("custom_search", input={"query": query}) as search_span: try: api_key, cse_id = os.environ["GOOGLE_API_KEY"], os.environ["GOOGLE_CSE_ID"] with tracing.create_span("build_custom_search"): service = build("customsearch", "v1", developerKey=api_key) with tracing.create_span("execute_custom_search", input={"query": query}) as span: res = service.cse().list(q=query, cx=cse_id, num=5).execute() span.output = {"response": res} urls = [item['link'] for item in res.get('items', [])] return urls except HttpError as e: error_message = f"An error occurred with Google Search: {e}" search_span.set_error(error_message=error_message) return [error_message] except Exception as e: error_message = f"An unexpected error occurred: {e}" search_span.set_error(error_message=error_message) return [error_message] @function_tool def get_article_content(url: str) -> str: with tracing.create_span("fetch_article", input={"url": url}) as fetch_span: try: headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36' } with tracing.create_span("GET_article_content", input={"url": url}, span_type="FILE_RETRIEVAL") as span: response = requests.get(url, headers=headers, timeout=10) response.raise_for_status() span.output = {"text": response.text, "code": response.status_code} with tracing.create_span("parse_article_content", span_type="DATA_MANIPULATION", input={"text": response.text}) as span: soup = BeautifulSoup(response.text, 'html.parser') paragraphs = soup.find_all('p') article_text = "\n".join([p.get_text() for p in paragraphs]) truncated_text = (article_text[:4000] + '...') if len(article_text) > 4000 else article_text span.output = {"truncated_text": truncated_text} return truncated_text except requests.RequestException as e: error_message = f"An error occurred while fetching the URL: {e}" fetch_span.set_error(error_message=error_message) return error_message except Exception as e: error_message = f"An unexpected error occurred during content extraction: {e}" fetch_span.set_error(error_message=error_message) return error_message async def main() -> None: today_str = f"Today's date is {date.today().strftime('%A, %B %d, %Y')}." news_agent = Agent( name="Personalized News Digest Agent", instructions=( f"{today_str}. You are a research assistant. Your goal is to provide a concise summary of recent news based on the user's request. " "First, use the `Google Search` tool to find relevant articles. " "Next, use the `get_article_content` tool on the most promising URLs to read them. " "Finally, synthesize the information from the articles into a coherent summary for the user." ), model="gpt-4-turbo", tools=[google_search, get_article_content], ) user_prompt = input("Please enter your news request: ") result = await Runner.run(news_agent, user_prompt) print(result.final_output) if __name__ == "__main__": SGP_API_KEY, SGP_ACCOUNT_ID = "XXX", "XXX" os.environ["OPENAI_API_KEY"], os.environ["GOOGLE_API_KEY"], os.environ["GOOGLE_CSE_ID"] = "XXX", "XXX", "XXX" client = SGPClient(api_key=SGP_API_KEY, account_id=SGP_ACCOUNT_ID) tracing.init(client) set_trace_processors([OpenAITracingSGPProcessor()]) asyncio.run(main()) ``` # OpenTelemetry Integration Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/opentelemetry Send traces from any framework or SDK with an OpenTelemetry exporter to SGP Traces. > [!NOTE] > > SGP Traces must be enabled on your deployment. The OpenTelemetry GenAI > semantic conventions are still evolving, so the attribute mapping below can > change. Anything that emits OpenTelemetry traces can send them to SGP Traces. Point its OTLP exporter at the endpoint below; no SGP SDK or collector is needed, and usually no code change. This includes frameworks with built-in tracing such as Pydantic AI and the Claude Agent SDK, instrumentation layers such as OpenLLMetry and LangSmith, and your own OpenTelemetry SDK setup. To use the SGP SDK instead, see [Tracing SDK Initialization](/docs/v5/tracing/sdk/initialization). For `openai-agents`, use the native [OpenAI Agents Integration](/docs/v5/tracing/integrations/openai-agents). ## Prerequisites * An SGP API key and the ID of the account that will own the traces. * Your SGP Traces host. The default is `sgp-traces.`, where `` is the domain your SGP API runs under as `api.`. The development deployment, for example, is `https://sgp-traces.dev-sgp.scale.com/v1/traces`. Deployments can override the host; ask your deployment owner if the default does not resolve. * A framework or SDK that exports over OTLP/HTTP protobuf. For hand instrumentation in Python: ```bash theme={null} pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http ``` In Node, use `@opentelemetry/exporter-trace-otlp-proto`. The `-http` package sends JSON regardless of `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` and is rejected with `400`. ## Configure the exporter Set these variables before starting your application. Every OpenTelemetry SDK reads them. Frameworks that manage their own exporter or keep tracing opt-in need one extra step, listed under [Framework notes](#framework-notes). ```bash theme={null} export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="https://sgp-traces./v1/traces" export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL="http/protobuf" export OTEL_EXPORTER_OTLP_TRACES_HEADERS="x-api-key=${SGP_API_KEY},x-selected-account-id=${SGP_ACCOUNT_ID}" export OTEL_EXPORTER_OTLP_TRACES_COMPRESSION="gzip" # optional ``` The trace-specific variables take the full URL, `/v1/traces` included. The generic `OTEL_EXPORTER_OTLP_ENDPOINT` must be the bare `https://sgp-traces.`: the SDK appends `/v1/traces`, and a value that already has it is exported to `/v1/traces/v1/traces`. LangSmith is the exception, see below. Prefer the trace-specific variables; the generic ones also drive metric and log exporters, and SGP Traces accepts traces only. If your framework emits through the global tracer provider but does not register an exporter, add one at startup. Existing instrumentation is unaffected: ```python theme={null} from opentelemetry import trace from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor provider = TracerProvider() provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter())) trace.set_tracer_provider(provider) with trace.get_tracer("my-app").start_as_current_span("hello"): ... provider.shutdown() # flushes pending spans before the process exits ``` ### Framework notes * **Pydantic AI** emits spans only after `Agent.instrument_all()` or `instrument=True`, through the global tracer provider, so register the exporter as above. With Logfire, `logfire.configure(send_to_logfire=False)` builds the exporter from the `OTEL_EXPORTER_OTLP_*` variables and `logfire.instrument_pydantic_ai()` turns the spans on. * **OpenLLMetry (Traceloop)** ignores the `OTEL_EXPORTER_OTLP_*` variables. Set `TRACELOOP_BASE_URL="https://sgp-traces."` (it appends `/v1/traces`) and `TRACELOOP_HEADERS="x-api-key=...,x-selected-account-id=..."`, or pass `api_endpoint` and `headers` to `Traceloop.init()`. * **LangSmith and LangChain** need `pip install "langsmith[otel]"`, `LANGSMITH_TRACING=true` to create runs at all, and `LANGSMITH_TRACING_MODE=otel` to export them over OpenTelemetry (older releases: `LANGSMITH_OTEL_ENABLED=true` plus `LANGSMITH_OTEL_ONLY=true`, otherwise runs also go to LangSmith). LangSmith reads only the generic `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` and uses the endpoint verbatim, so include `/v1/traces`. An already-configured global tracer provider takes precedence. * **Claude Agent SDK and Claude Code** keep telemetry opt-in and trace export in beta. Add `CLAUDE_CODE_ENABLE_TELEMETRY=1`, `CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1`, `OTEL_TRACES_EXPORTER=otlp`, and `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf` to the variables above. Only the built-in `claude_code.llm_request` spans carry `gen_ai.system` and `gen_ai.request.model`, and none carry the semconv input, output, or token keys, so those land as completions with a model and nothing else while interaction and tool spans get no derived fields; the prompt is redacted unless `OTEL_LOG_USER_PROMPTS=1`. For full input, output, and token mapping, instrument the Agent SDK with `otel-instrumentation-claude-agent-sdk` instead. ## Endpoint contract | Setting | Value | | - | - | | Endpoint | `POST https://sgp-traces./v1/traces` | | Protocol | OTLP/HTTP protobuf (`Content-Type: application/x-protobuf`). gRPC and JSON are rejected. | | Compression | `Content-Encoding: gzip` or none. | | Authentication | `x-api-key`: an SGP API key. `x-selected-account-id`: the owning account ID. | | Request limits | 1,000 spans per request. The body limit is set per deployment (128 MiB by default) and enforced before and after gzip decompression. | The default SDK batch of 512 spans fits. ### Errors Service errors follow the OTLP/HTTP specification: a protobuf `Status` body with a message, and the HTTP status decides retry behavior. On mesh-fronted deployments an unauthenticated request may be answered by the mesh instead, as `403` with a plain-text `RBAC: access denied` body or `401` with an empty body. * `400`: the whole request is rejected. Causes: a missing `x-selected-account-id` (including the legacy `x-sgp-account-id` in its place), non-protobuf content type, `Content-Encoding` other than `gzip` or `identity`, invalid gzip, empty body, more than 1,000 spans, an invalid span such as an all-zero trace ID or an end time before its start, or a malformed `sgp.obs_trace_id` or `sgp.obs_span_id`. Fix the request or producer; do not retry. With an OpenTelemetry Collector, set both `send_batch_size: 1000` and `send_batch_max_size: 1000`; under load the defaults emit 8,192-span batches, which are rejected. * `401`: invalid API key. * `403`: the key is not a member of the account, or its role there is not admin, manager, or editor. * `413`: body over the deployment limit. Lower the export batch size. * `429`: rate limited. Cloud deployments apply both a per-account limit and an overall limit, so a request can be throttled by its account limit even when overall capacity remains. On-premises deployments apply only the overall limit by default. Ask your deployment owner for your deployment's limits. Gateway throttling answers with a plain-text body and no `Retry-After`. The service's own load shedding answers with an OTLP `Status` and `Retry-After`. The OTLP specification makes `429`, `502`, `503`, and `504` retryable and exporters from 2026 onward follow it, but older ones differ: the Python OTLP/HTTP exporter up to 1.44.0 retries only `408` and `5xx`, so a `429` drops the batch. Keep exporters current, stay within your deployment's limit, or export through a Collector, which retries `429` with backoff. ## How spans map to SGP Every span keeps its OpenTelemetry identity and structure: trace and span IDs, parent, name, kind, status, timing, attributes, events, links, and resource and scope attributes. An `UNSET` status is stored as success, matching SGP SDK writes. The keys `sgp.obs_trace_id` and `sgp.obs_span_id` are reserved: they move into SGP's observability correlation fields and must be valid W3C IDs. Spans with [GenAI semantic convention](https://opentelemetry.io/docs/specs/semconv/gen-ai/) attributes also get SGP's tracing fields, so they render like SGP SDK spans. This is best effort and never rejects a span: * **Input and output**: `gen_ai.input.messages` and `gen_ai.output.messages`, then the indexed `gen_ai.prompt.N` and `gen_ai.completion.N` attributes, then `gen_ai.prompt` and `gen_ai.completion`, then framework-specific keys, then a tool span's `gen_ai.tool.call.arguments` and `gen_ai.tool.call.result`, and finally the legacy `gen_ai.content.*` events. * **Token usage**: `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens`, or the `prompt_tokens` and `completion_tokens` spellings. * **Operation type**: `gen_ai.operation.name`, then `traceloop.span.kind`, then rerank and vector-store signals, then a tool span (`gen_ai.tool.name`, `gen_ai.tool.call.*`) with no model or tokens (custom step), then `gen_ai.system` or `gen_ai.provider.name` (completion). A span with derived input or output and none of these is also a completion. The operations `fetch_response`, `guardrail_check`, and `agent_handoff` are custom steps, and so is `unknown` on a span with `gen_ai.system` or `gen_ai.provider.name` but no model, tokens, input, or output. * **Model and provider metadata** from the same keys are merged into the span's attributes. Any producer of standard `gen_ai.*` attributes gets these fields, including LangSmith and `otel-instrumentation-claude-agent-sdk`. If you use the `opentelemetry-instrumentation-openai-agents-v2` bridge instead of the native integration, each agent's name arrives on its `invoke_agent` span, and the bridge's default name and ID (`OpenAI Agent`, `agent`) on its other spans are read as unknown. Its `agent_name` and `agent_id` arguments replace every agent's name with one, so pass them only in a single-agent app, and not under `opentelemetry-instrument`, which instruments the bridge first and ignores a second call. SGP also reads Pydantic AI (`pydantic_ai.*`) and OpenLLMetry (`traceloop.*`, `llm.*`) keys, and OpenInference and Vercel AI SDK token counts. Spans with none of these keep their raw attributes and get no derived fields. ## Migrating from the legacy forwarder > [!WARNING] > > **Deprecation notice:** The `opentelemetry.` forwarder is deprecated and being decommissioned. Send traces to SGP Traces instead. The forwarder spoke the same protocol, so migration is a host and header change: | | Legacy forwarder | SGP Traces | | - | - | - | | Endpoint | `https://opentelemetry./v1/traces` | `https://sgp-traces./v1/traces` | | API key header | `x-sgp-api-key` | `x-api-key` | | Account header | `x-sgp-account-id` | `x-selected-account-id` | Do not mix the two. The native endpoint does not read the legacy headers: without `x-api-key` the request is rejected as unauthenticated by the service or, behind a mesh, as `403 RBAC: access denied`; with `x-api-key` but only the legacy account header it is a `400` for the missing `x-selected-account-id`. ## Viewing traces Exported traces appear on the [Traces page](/docs/v5/tracing/tracing-ui/traces-table) for the selected account. Open one to inspect its spans in the [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view). # Temporal Integration Source: https://docs.gp.scale.com/docs/v5/tracing/integrations/temporal Emit SGP traces from Temporal workflows and activities without breaking workflow determinism. > [!NOTE] > > If you're using **agentex agents on Temporal**, you don't need this page — agentex provides its own SGP tracing integration through its tracing processors. This guide is for users writing their own Temporal workflows or activities that emit SGP spans directly. [Temporal](https://temporal.io/) workflows are deterministic: workflow code is replayed from history on every worker, so it cannot perform network IO, generate non-deterministic UUIDs, or read system time directly. This makes integrating any tracing SDK with Temporal a question of **where** the tracing calls live. Activities, by contrast, are regular Python functions running on a worker. They are free to do network IO, so the SGP Tracing SDK works inside an activity exactly like in any other process. This guide covers the two patterns that come up in practice. ## Pattern 1: Trace at the activity boundary (recommended) Use this when "one activity = one span" gives you the granularity you need. It is the simplest and matches how most Temporal applications are observed. * Activities call `tracing.create_span(...)` directly using the SDK's context manager. * The caller of the workflow creates the parent span and passes `trace_id` and `parent_span_id` as workflow input. * The workflow forwards those IDs to each activity. Each activity span is rooted under the caller's parent. ```python theme={null} from datetime import timedelta import scale_gp_beta.lib.tracing as tracing from scale_gp_beta import SGPClient from temporalio import activity, workflow from temporalio.client import Client from temporalio.worker import Worker @activity.defn async def fetch_data_activity(url: str, trace_id: str, parent_span_id: str) -> dict: with tracing.create_span( name="fetch_data", trace_id=trace_id, parent_id=parent_span_id, input={"url": url}, ) as span: # ... actual work ... result = {"status": "ok", "url": url} span.output = result return result @workflow.defn class MyWorkflow: @workflow.run async def run(self, urls: list[str], trace_id: str, parent_span_id: str) -> list[dict]: results = [] for url in urls: result = await workflow.execute_activity( fetch_data_activity, args=(url, trace_id, parent_span_id), start_to_close_timeout=timedelta(seconds=30), ) results.append(result) return results async def run_worker(): tracing.init(SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID")) client = await Client.connect("localhost:7233") worker = Worker( client, task_queue="my-queue", workflows=[MyWorkflow], activities=[fetch_data_activity], ) await worker.run() ``` The caller starts the workflow inside an existing span, passing the IDs through workflow input: ```python theme={null} import scale_gp_beta.lib.tracing as tracing from scale_gp_beta import SGPClient from temporalio.client import Client # Initialize the SDK once at the caller process entry point. The caller and # the worker run in different processes, each with its own queue manager, so # tracing.init() is required in both. tracing.init(SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID")) async def kick_off(): client = await Client.connect("localhost:7233") with tracing.create_trace("my_pipeline") as trace: with tracing.create_span("orchestrate_pipeline") as parent: await client.execute_workflow( MyWorkflow.run, args=[["https://a.example", "https://b.example"], trace.trace_id, parent.span_id], id="my-workflow-1", task_queue="my-queue", ) ``` > [!WARNING] > > Use `workflow.uuid4()` (Temporal's deterministic UUID helper), not `uuid.uuid4()`, when generating IDs inside workflow code. Standard library `uuid.uuid4()` is non-deterministic and will break workflow replay. > [!NOTE] > > The SDK's background queue worker is a daemon thread inside the **activity worker** process, not the workflow. It is fine to use the default `tracing.init()` configuration. The queue flushes on size or cadence triggers and on worker shutdown. ## Pattern 2: Buffer in workflow, flush via activity (advanced) Use this when you want span boundaries that mirror workflow steps rather than activity invocations, or when activity calls are too coarse-grained to capture the hierarchy you want. The shape: * The workflow buffers lightweight span dicts (the SDK's `SpanCreateParam` `TypedDict`) in workflow state. No network IO, no SDK calls. * A dedicated `flush_spans` activity receives the buffered list and writes them via `client.spans.upsert_batch(items=...)`. * The flush activity runs: * when the buffer hits a size threshold, * on a periodic timer, * before the workflow completes or calls `continue_as_new`. ```python theme={null} from datetime import timedelta from scale_gp_beta import AsyncSGPClient from scale_gp_beta.types import SpanCreateParam as SpanRequest from temporalio import activity, workflow from temporalio.common import RetryPolicy # Module-level client so we reuse the connection pool across activity invocations. _client = AsyncSGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID") # 4xx-style errors that should not retry: bad input, auth failure, validation. # 5xx, rate limits, and transport errors are retried by Temporal per the policy below. _NON_RETRYABLE = [ "BadRequestError", "AuthenticationError", "PermissionDeniedError", "NotFoundError", "UnprocessableEntityError", ] @activity.defn async def flush_spans_activity(spans: list[SpanRequest]) -> None: # No tracing.init() needed here: this activity bypasses the SDK queue and # writes directly via upsert_batch. tracing.init() is only required if you # call tracing.create_span(...) inside an activity (Pattern 1). await _client.spans.upsert_batch(items=spans) @workflow.defn class BufferedTracingWorkflow: def __init__(self) -> None: self._buffer: list[SpanRequest] = [] def _enqueue(self, span: SpanRequest) -> None: self._buffer.append(span) async def _maybe_flush(self, force: bool = False) -> None: if not self._buffer: return if not force and len(self._buffer) < 200: return spans_to_send = self._buffer self._buffer = [] await workflow.execute_activity( flush_spans_activity, args=(spans_to_send,), start_to_close_timeout=timedelta(seconds=30), schedule_to_close_timeout=timedelta(minutes=10), retry_policy=RetryPolicy( initial_interval=timedelta(seconds=1), maximum_interval=timedelta(seconds=30), maximum_attempts=5, non_retryable_error_types=_NON_RETRYABLE, ), ) @workflow.run async def run(self, items: list[str]) -> None: trace_id = workflow.uuid4().hex root_span_id = workflow.uuid4().hex root: SpanRequest = { "id": root_span_id, "trace_id": trace_id, "name": "workflow_root", "start_timestamp": workflow.now().isoformat(), } self._enqueue(root) for item in items: child: SpanRequest = { "id": workflow.uuid4().hex, "trace_id": trace_id, "parent_id": root_span_id, "name": f"process_{item}", "start_timestamp": workflow.now().isoformat(), } # Real work goes through an activity: # await workflow.execute_activity(do_work, item, start_to_close_timeout=...) child["end_timestamp"] = workflow.now().isoformat() self._enqueue(child) await self._maybe_flush() # Set root's end_timestamp and re-enqueue so the upsert lands. The # upsert_batch endpoint is keyed on `id`, so a second send with # end_timestamp set just updates the existing row. This handles the # case where root was already flushed mid-loop (buffer hit 200) and # the in-memory mutation alone would be lost. root["end_timestamp"] = workflow.now().isoformat() self._enqueue(root) await self._maybe_flush(force=True) ``` > [!NOTE] > > `SpanCreateParam` is a `TypedDict`, so values are plain `dict` at runtime. That means no `asdict(...)` conversion before the flush activity, and Temporal's default data converter serializes them as JSON cleanly. > [!WARNING] > > A non-retryable failure (any error class listed in `non_retryable_error_types`) drops the buffered batch. The data is gone — there is no second chance for those spans. Choose the non-retryable list deliberately: list only errors where retrying truly cannot succeed (bad input, auth) and let everything else retry. ### Why `upsert_batch` and not `create` * **Idempotency.** Temporal retries activities on transient failures. `upsert_batch` is keyed on the client-supplied span `id`, so a retry that partially succeeded will re-converge to the right state. `client.spans.create` is not safe under retry because a partial success leaves orphan rows. * **Throughput.** One HTTP call carries many spans. The batch endpoint accepts up to 1000 items per request. > [!NOTE] > > Keep batches at 200 to 500 items per flush in practice. That keeps payloads under 1 MB and HTTP timeouts comfortable, and gives you headroom under the 1000-item server cap. ## Span integrity discipline The server rejects the whole batch if any child's `trace_id` does not match its parent's. This is easy to trip if parent IDs are minted in workflow code but child IDs are minted in an activity without the trace context piped through. Always pass `trace_id` (and `parent_span_id` where relevant) explicitly when crossing the workflow/activity boundary. ## Distributed tracing across workflows If one workflow starts another (child workflow, or an external `start_workflow` call), propagate `trace_id` and the originating span's `id` via workflow input or memo, and use them in the child workflow as `parent_id` for its root span. The same propagation rule from the [Distributed Tracing](/docs/v5/tracing/sdk/multi-process) page applies; Temporal's workflow-input mechanism is a clean place to carry it. # Introduction to Tracing Source: https://docs.gp.scale.com/docs/v5/tracing/overview Gain deep visibility into your workflows with tracing. ## Overview Modern applications, especially those leveraging AI agents and complex tool use, are distributed and often opaque. Understanding the flow of data, performance bottlenecks, and error propagation across different services and functions is critical for debugging, optimization, and ensuring reliability. Our Tracing capabilities provide **deep observability** into your workflows by allowing you to generate, collect, and visualize **spans** and **traces**. This system is designed to significantly decrease the overhead of understanding complex operations while increasing your ability to debug and monitor. At its core, Tracing now consists of: 1. **Spans**: Individual units of work within your application. 2. **Traces**: Collections of interconnected spans representing a complete workflow. 3. **Dedicated Traces Page**: A new UI for visualizing, filtering, and drilling down into your tracing data. 4. **Flexible SDK & APIs**: Tools to instrument your Python applications and integrate tracing directly into your code. ## Traces & Spans Concepts To effectively use tracing, it's helpful to understand the fundamental concepts: * **Span:** A span represents a single **unit-of-work** within a trace, such as a function call, a database query, or an external API request. It captures crucial information about that operation, including: * `name`: A descriptive name for the operation. * `input` & `output`: Data flowing into and out of the operation. * `trace_id`: The identifier of the overall workflow this span belongs to. * `parent_id`: The identifier of the span that directly invoked this operation, allowing us to build a nested hierarchy. * `start_time` & `end_time`: Timestamps for the duration of the operation. The API names these fields `start_timestamp` & `end_timestamp`. * `metadata`: Arbitrary key-value pairs for additional context or custom annotations. * `status`: Indicates if the span completed successfully or with an error. * `type`: Optionally give the span a type, to better filtering and visualization * **Trace:** A trace is a **collection of spans** that together describe a complete, end-to-end workflow, such as a user's web request, an AI agent's complex decision-making process, or a batch processing job. * Every trace begins with a single **"root-span"**, which represents the starting point of that entire workflow. * Server-side, we primarily record and visualize spans. The "trace" resource itself is implicitly derived from the relationships and common `trace_id` among spans, with the root span providing the top-level details for the workflow. For a deeper dive into the standardized tracing model, refer to the OpenTelemetry specification on Traces & Spans: [https://opentelemetry.io/docs/specs/otel/overview/#tracing-signal](https://opentelemetry.io/docs/specs/otel/overview/#tracing-signal). ## Traces Page in the UI The new dedicated Traces page in the SGP UI provides powerful capabilities to monitor, filter, and drill down into your trace data: * **High-Level Monitoring:** Get an immediate overview of your application's workflows. * **Filtering:** Use basic filters now, with more in-depth filtering capabilities coming soon. You'll be able to search on: * **Static fields**: Like a specific trace ID or a span type. * **Dynamic fields**: Custom key-value pairs found in your span's metadata. * **Trace Detail View:** Clicking on any trace reveals a detailed breakdown, allowing you to: * View run-time statistics for each span. * Inspect input, output, and metadata associated with each span. * Trigger thread-forking (for SGP native traces) for advanced debugging. ## Getting Started with Tracing > [!NOTE] > > Examples in this guide use the [`scale-gp-beta`](https://pypi.org/project/scale-gp-beta/) package, which runs exclusively on the V5 API. The SGP Tracing SDK provides a convenient and ergonomic way to instrument your Python applications. ### Initialization Before you can create any traces or spans, you should initialize the tracing SDK with your `SGPClient`. It's best practice to do this once at your application's entry point. It is not strictly required to call `tracing.init()`, the SDK will attempt to create a default client using default ENV variables, if not set we log a warning and will not send tracing data. ```python theme={null} import scale_gp_beta.lib.tracing as tracing from scale_gp_beta import SGPClient # Initialize with your API Key and Account ID client = SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID") tracing.init(client=client) ``` The tracing SDK uses the `SGPClient` for all data export requests. This means you can leverage existing client configurations, such as `base_url` or `timeout` settings, by passing a pre-configured client to `tracing.init()`. ### Creating Your First Trace & Spans The most straightforward and recommended way to create traces and spans is by using them as context managers (`with` statements). This ensures that spans are automatically started, ended, and errors are captured, simplifying your instrumentation. ### Define a Workflow as a Trace Use `tracing.create_trace()` as a context manager to define a new end-to-end workflow. This automatically creates the root span for your trace. ```python theme={null} import scale_gp_beta.lib.tracing as tracing def my_application_flow(): with tracing.create_trace(name="my_customer_onboarding_process", metadata={"customer_tier": "premium"}): print("Starting customer onboarding...") # ... logic for onboarding print("Customer onboarding complete.") ``` ### Instrument Units of Work as Spans Inside a `create_trace` block, use `tracing.create_span()` as a context manager. These spans will automatically inherit the `trace_id` from the current trace and will be nested under the currently active span (or the root span if no other span is active). > [!WARNING] > > If you are working in a distributed setting, please see [Distributed Tracing](/docs/v5/tracing/sdk/multi-process) ### Quick Start Examples For more examples, see [Creating Traces and Spans](/docs/v5/tracing/sdk/creating-traces-spans). ## Traces API Endpoints For direct interaction with span data via the API, beyond the SDK, you can use the following endpoints: * [Create a Span](/reference/v5/spans/create-a-span) * [Get a Span Within a Trace](/reference/v5/spans/get-a-span-within-a-trace) * [Update Span](/reference/v5/spans/update-a-span) * [Batch Create Spans](/reference/v5/spans/create-spans-in-a-batch) * [Batch Upsert Spans](/reference/v5/spans/upsert-spans-in-a-batch) * [Search Spans](/reference/v5/spans/search-spans) > [!WARNING] > > **Deprecation notice:** [Get a span](/reference/v5/spans/get-a-span) (`GET /v5/spans/{span_id}`) is deprecated. Use [Get a span within a trace](/reference/v5/spans/get-a-span-within-a-trace) (`GET /v5/traces/{trace_id}/spans/{span_id}`) instead. # Configuration Options Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/configuration-options Configure the SGP Tracing SDK's behavior. The SGP Tracing SDK allows you to customize its behavior through a combination of environment variables and parameters passed during SDK initialization. ## Environment Variables These environment variables provide a quick way to configure common settings, especially for deployment environments. | Environment Variable | Description | | :- | :- | | `DISABLE_SCALE_TRACING` | If set to any value (e.g., `true`, `1`), tracing data will **not** be exported. In this mode, the SDK returns "No-Op" variants of Trace and Span objects, which allow your code to run without errors but perform no actual data collection or sending. | | `SGP_API_KEY` | Your SGP API Key. If set, the SDK can automatically initialize without needing to pass an `SGPClient` explicitly to `tracing.init()`. | | `SGP_ACCOUNT_ID` | Your SGP Account ID. If set, the SDK can automatically initialize without needing to pass an `SGPClient` explicitly to `tracing.init()`. | | `SGP_CLIENT_BASE_URL` | Custom base URL for API requests. If set, this overrides the default environment URL. Mutually exclusive with the `environment` parameter. | ## Programmatic Configuration The `tracing.init()` function, typically called at your application's entry point, accepts various parameters to configure the tracing behavior. Programmatic settings take precedence over environment variables. Set `SGP_CLIENT_BASE_URL` to your deployment's full API URL before running this example. ```python theme={null} import os import scale_gp_beta.lib.tracing as tracing from scale_gp_beta import SGPClient # Example of a comprehensive initialization tracing.init( client=SGPClient( api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID", base_url=os.environ["SGP_CLIENT_BASE_URL"], timeout=30.0 ), disabled=False, # Explicitly enable or disable tracing (overrides ENV var if False) ) ``` ### `client` An initialized `SGPClient` instance. If provided, the SDK will use this client for all tracing data export requests. This allows you to configure HTTP settings (like `base_url`, `timeout`, `max_retries` for the client itself) that will also apply to your tracing data. ### `disabled` * Set to `True` to explicitly disable tracing. When disabled, `create_trace()` and `create_span()` will return "No-Op" objects that perform no data collection or export. * Set to `False` to explicitly enable tracing. * Defaults to `False` if not specified, unless `DISABLE_SCALE_TRACING` environment variable is set. # Creating Traces and Spans Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/creating-traces-spans Instrument your Python code with the Tracing SDK. The SGP Tracing SDK offers flexible ways to instrument your application by creating traces and spans. You can choose between two main approaches: using **context managers** for automatic lifecycle management, or **explicit control** for manual start/end handling. ## Context Managers (Recommended) The most straightforward and robust way to create traces and spans is by using them as Python context managers (`with` statements). This approach automatically handles span start and end times, associates spans with the correct trace context, and captures exceptions, significantly reducing boilerplate and potential errors. ### Creating a Trace with a Root Span Every trace begins with a root span. Use `tracing.create_trace()` as a context manager to define a new top-level workflow. This automatically creates the root span for your trace, setting the context for all child spans created within its block. ```python {1, 5} theme={null} import scale_gp_beta.lib.tracing as tracing def my_workflow(): # This creates a new trace and its root span with tracing.create_trace(name="my_application_workflow", metadata={"env": "production"}): print("Starting my application workflow...") # ... your main workflow logic goes here ... print("Application workflow completed.") ``` ### Creating Spans within a Trace Inside a `tracing.create_trace` block, use `tracing.create_span()` as a context manager. These spans will automatically: * Inherit the `trace_id` from the current trace. * Be associated as a child of the currently active span (if another `create_span` is active) or the root span. ```python {2, 6, 18, 20} theme={null} import time import scale_gp_beta.lib.tracing as tracing def fibonacci(curr: int) -> int: # This span becomes a child of the current active span (e.g., "main_execution") with tracing.create_span("fibonacci_calculation", input={"curr": curr}) as span: time.sleep(0.05) # Simulate some work if curr < 2: span.output = {"res": curr} return curr res = fibonacci(curr - 1) + fibonacci(curr - 2) span.output = {"res": res} return res def main_traced_example(): # This creates the main trace and its root span with tracing.create_trace("my_fibonacci_trace"): # This span will be a child of the "my_fibonacci_trace" root span with tracing.create_span("main_execution", metadata={"version": "1.0"}) as main_span: fib_result = fibonacci(5) main_span.output = {"final_fib_result": fib_result} print(f"Fibonacci(5) = {fib_result}") ``` > [!TIP] > > The [Traces table](/docs/v5/tracing/tracing-ui/traces-table#controlling-the-content-preview) preview column extracts text from specific preferred keys in your span `input` and `output` (e.g., `title`, `query`, `content`). If the preview shows an unhelpful value, add a `title` key with the value you want displayed. ## Explicit Control For advanced scenarios where context managers are not suitable (e.g., integrating with existing systems that manage context, or reporting historical data), you can manually manage the lifecycle of your spans. This approach requires more diligence to ensure `start()` and `end()` are always called, and consistency is maintained across your trace. ### Manually Managing Spans (without implicit context) You can create `Span` objects and explicitly control their `trace_id` and `parent_id` for fine-grained hierarchy management. This is particularly useful when you need to bridge tracing across different processes, services, or when a context manager is not practical. Remember to manually call `span.start()` to begin the span's lifecycle and `span.end()` to complete it. For error handling, you'll need to wrap your logic in a `try...finally` block. ```python {5,15-16,20,23,26,35-36,41} theme={null} import uuid import time import random from typing import Any, Dict import scale_gp_beta.lib.tracing as tracing class MockDatabase: def __init__(self) -> None: self._data = { "SELECT * FROM users WHERE id = 1;": {"id": 1, "name": "Alice"}, "SELECT * FROM users WHERE id = 2;": {"id": 2, "name": "Bob"}, } def execute_query(self, query: str, trace_id: str) -> Dict[str, Any]: # Manually create, start, set output, and end the span db_span = tracing.create_span("db_query", input={"query": query}, trace_id=trace_id) db_span.start() # Manually start the span try: time.sleep(random.uniform(0.1, 0.3)) # Simulate delay result = self._data.get(query, {}) db_span.output = {"result": result} return result finally: db_span.end() # Manually end the span def get_user_from_db_explicit(db: MockDatabase, user_id: int, trace_id: str) -> Dict[str, Any]: with tracing.create_span("get_user_from_db", input={"user_id": user_id}, trace_id=trace_id): query = f"SELECT * FROM users WHERE id = {user_id};" return db.execute_query(query, trace_id) def main_explicit_control_example(): db = MockDatabase() my_trace_id = str(uuid.uuid4()) # Manually create a root span for the trace, providing the trace_id explicitly main_span = tracing.create_span("main_explicit_call", metadata={"env": "local"}, trace_id=my_trace_id) main_span.start() # Manually start the main span try: user = get_user_from_db_explicit(db, 1, my_trace_id) print(f"Retrieved user: {user.get('name')}") finally: main_span.end() # Manually end the main span ``` ### Exporting Historical or Pre-defined Tracing Data You can pre-define `start_time`, `end_time`, `span_id`, `parent_id`, and `trace_id` when creating a span. This is useful for reporting historical data, replaying events, or reconstructing traces from external sources. After setting these attributes, call `span.flush()` to send the data. ```python theme={null} import uuid from datetime import datetime, timezone, timedelta import scale_gp_beta.lib.tracing as tracing parent_span_id = str(uuid.uuid4()) trace_id = str(uuid.uuid4()) child_span_id = str(uuid.uuid4()) now = datetime.now(timezone.utc) # Parent Span: Create and configure a span with specific IDs and timestamps. parent_span = tracing.create_span( "my_parent_span_name", input={"test": "input"}, output={"test": "output"}, metadata={"test": "metadata"}, span_id=parent_span_id, trace_id=trace_id, ) parent_span.start_time = (now - timedelta(minutes=10)).isoformat() parent_span.end_time = now.isoformat() parent_span.flush(blocking=True) # Send this span immediately # Child Span: Link to the parent and trace using explicit IDs and timestamps. child_span = tracing.create_span( "my_child_span_name", input={"test": "another input"}, output={"test": "another output"}, metadata={"test": "another metadata"}, span_id=child_span_id, trace_id=trace_id, parent_id=parent_span_id, # Link to the parent span ) child_span.start_time = (now - timedelta(minutes=6)).isoformat() child_span.end_time = (now - timedelta(minutes=2)).isoformat() child_span.flush() # Send this span immediately (defaults to blocking=True) ``` > [!WARNING] > > `span.flush()` by default blocks the main thread until the request has finished. For non-blocking behavior, use `span.flush(blocking=False)` to enqueue the request for the background worker. This is generally recommended for performance-sensitive applications. # Flushing Tracing Data Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/flushing-data Understand how and when to manually send buffered tracing data to SGP. By default, tracing data (spans) are collected in an in-memory queue and sent to the SGP platform **asynchronously in batches** by a background worker. This batching mechanism is designed for efficiency and minimal performance impact on your application. However, there are specific scenarios where you might need to force an immediate export of tracing data: * **Critical workflow points:** To ensure data is persisted before sensitive operations or transitions in a distributed workflow. * **Debugging:** To see tracing data appear on the Traces page immediately during development. ## How to Flush Tracing Data The SDK provides methods to manually trigger data export: ### Flushing the entire queue Use `tracing.flush_queue()` to force a synchronous export of *all* spans currently buffered in the SDK's internal queue. This method will block the calling thread until all pending spans have been sent. ```python {12} theme={null} import scale_gp_beta.lib.tracing as tracing import time # Simulate creating some spans with tracing.create_trace("short_lived_process_trace"): with tracing.create_span("task_alpha"): time.sleep(0.1) with tracing.create_span("task_beta"): time.sleep(0.1) # Force all pending spans to be sent before the application exits tracing.flush_queue() print("All pending spans have been flushed to SGP.") # In a multi-worker or distributed application, you might use this # before enqueuing a job to ensure parent trace data is sent first. ``` ### Flushing individual spans or traces You can also explicitly flush an individual `Span` object. When you call `span.flush()`, that specific span will be sent. For `Trace` objects, flushing (`trace.flush()`) will send the root span. ```python theme={null} import uuid from datetime import datetime, timezone, timedelta import scale_gp_beta.lib.tracing as tracing now = datetime.now(timezone.utc) # Example: Flushing a single span immediately after creation single_span = tracing.create_span("critical_single_operation", trace_id=str(uuid.uuid4())) single_span.start_time = now.isoformat() single_span.end_time = (now + timedelta(seconds=1)).isoformat() single_span.output = {"status": "completed"} single_span.flush() # This span is sent to the backend immediately # Example: Flushing a trace's root span my_trace = tracing.create_trace(name="important_trace") my_trace.root_span.start_time = now.isoformat() # ... perform operations within the trace ... my_trace.root_span.end_time = (now + timedelta(seconds=5)).isoformat() my_trace.flush() ``` > [!WARNING] > > By default, `span.flush()` and `trace.flush()` are **blocking** operations. This means the calling thread will pause until the data is successfully sent to the SGP platform. > > For non-blocking behavior (recommended in performance-sensitive applications), pass `blocking=False`: > > ```python theme={null} > my_span = tracing.create_span("my_async_span", trace_id="...") > my_span.start() > # ... do work ... > my_span.end() > my_span.flush(blocking=False) # Enqueue for background worker, don't block > ``` ### Automatic Shutdown Flush You typically do not need to manually call `flush_queue()` at the very end of your program. When the SDK's background worker is shut down (e.g., when your Python application exits normally), it will automatically attempt to flush any remaining tracing data in the queue before exiting. # Helper Methods Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/helper-methods Access the currently active span and trace in your application's execution context. The SGP Tracing SDK leverages Python's `contextvars` to automatically manage the active span and trace within your application's execution flow. This allows you to easily retrieve information about the current operation without explicitly passing objects around. You can retrieve the currently active span or trace in the execution context using the following helper methods: ### `tracing.current_span()` Returns the `Span` instance representing the operation currently active in the local execution context. This is typically the innermost span created by `tracing.create_span()`. ### `tracing.current_trace()` Returns the `Trace` instance representing the overall workflow currently active in the local execution context. This will be the trace established by `tracing.create_trace()`. ### Example Usage This example demonstrates how `current_span()` and `current_trace()` can be used to inspect the active context within nested operations: ```python {1,4-5} theme={null} import scale_gp_beta.lib.tracing as tracing def process_data_step(): current_span_instance = tracing.current_span() current_trace_instance = tracing.current_trace() if current_span_instance: print(f"Current Span: {current_span_instance.name} (ID: {current_span_instance.span_id})") if current_trace_instance: print(f"Current Trace: (ID: {current_trace_instance.trace_id})") print("Processing data...") def main_workflow_with_helpers(): with tracing.create_trace("my_main_workflow"): print("Main workflow started.") with tracing.create_span("data_processing"): process_data_step() ``` ### When are these methods useful? * **Debugging:** Quickly inspect the current tracing context during development or when logging. * **Contextual Logging:** Enrich log messages with `span_id` or `trace_id` without needing to pass them explicitly. * **Conditional Logic:** Implement logic that depends on the existence or attributes of the current span/trace. These methods are primarily for introspection and should not be used to create or manage the lifecycle of spans, which should be done via `create_span()` and `create_trace()`. # Initialization Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/initialization Set up and initialize the Scale GP Tracing SDK in your Python applications. The SGP Tracing SDK provides a convenient way to instrument your Python applications with tracing capabilities, allowing you to generate, manage, and send spans to the Scale GP platform. This enables detailed monitoring and debugging of your workflows. > [!NOTE] > > Python SDK tracing is currently only available in the Beta SDK. ## Installation First, install the `scale-gp-beta` package using pip: ```sh theme={null} pip install --pre scale-gp-beta ``` ## Initialisation Before you can create any traces or spans, you must initialize the tracing SDK. It's best practice to do this once at your application's entry point. You have two primary options for initialization: ### Initialize with an `SGPClient` This method gives you full control over the `SGPClient`'s configuration (e.g., `base_url`, `timeout`). ```Python theme={null} import scale_gp_beta.lib.tracing as tracing from scale_gp_beta import SGPClient # Replace with your actual API Key and Account ID client = SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID") tracing.init(client=client) ``` ### Initialize using Environment Variables If you have set the `SGP_API_KEY` and `SGP_ACCOUNT_ID` environment variables, you can omit passing a client. The SDK will automatically attempt to create a default `SGPClient` for you. ```bash theme={null} export SGP_API_KEY="sk-your-api-key" export SGP_ACCOUNT_ID="your-account-id" ``` Then, in your Python code: ```python theme={null} import scale_gp_beta.lib.tracing as tracing # The SDK will automatically find SGP_API_KEY and SGP_ACCOUNT_ID # from your environment variables. tracing.init() ``` ## Disabling Tracing You can control whether tracing data is collected and exported. This is useful for development, testing, or specific environments where tracing is not desired. ### Environment Variable ```bash theme={null} export DISABLE_SCALE_TRACING=true ``` ### Programmatically Pass the `disabled=True` parameter directly to the `init()` function. This will override the environment variable setting. ```python theme={null} import scale_gp_beta.lib.tracing as tracing from scale_gp_beta import SGPClient # Tracing will be disabled, regardless of environment variables tracing.init(client=SGPClient(api_key="...", account_id="..."), disabled=True) ``` # Distributed Tracing Source: https://docs.gp.scale.com/docs/v5/tracing/sdk/multi-process Implement tracing across multiple processes and workers in your Python app. Distributed applications, especially those involving multiple Python processes, threads, or asynchronous workers, introduce unique challenges for tracing. While the SGP Tracing SDK automatically manages span context within a single Python process (using `contextvars`), this automatic propagation **does not extend across process boundaries or when work is explicitly dispatched to new, independent execution contexts.** > [!NOTE] > > The SGP backend no longer expects spans to arrive in-order. This means there is a chance that child spans may become orphans if their respective parent span fails to send. This guide outlines key strategies for effective tracing in multi-process and multi-worker environments. ## Understanding Context Propagation When using context managers (`with tracing.create_span(...)`), the SDK automatically sets the current span and trace in a context-local variable. However, when you: * Spawn a new Python process (e.g., using `multiprocessing`). * Enqueue a task to a background job queue (e.g., Celery, RQ). * Dispatch work to a separate thread pool where contextvars might not propagate by default (though `threading` typically handles this better than `multiprocessing`). The new execution context will **not** automatically inherit the `trace_id` or `parent_id` from the originating process. You must explicitly pass this context. ## Strategies for Distributed Tracing ### One Trace Per Worker (Simplest) For parallel work where strict hierarchical linking of *every* operation across workers isn't necessary, the easiest approach is to create an independent trace for each worker or process. You can then use a `group_id` to logically link these independent traces together, allowing you to see all related activity in the Traces page, even if they don't form a single, continuous trace hierarchy. This is ideal for scenarios where workers process independent units of work concurrently. ```python theme={null} import uuid import time import multiprocessing import scale_gp_beta.lib.tracing as tracing from scale_gp_beta import SGPClient def process_item_in_worker(item_id: int, shared_group_id: str): # Each worker creates its own independent trace, ensure there is a working client with init or by setting ENV Vars try: with tracing.create_trace(name=f"worker_process_for_item_{item_id}", group_id=shared_group_id): with tracing.create_span("item_processing_task"): time.sleep(0.5) print(f"Worker {multiprocessing.current_process().name}: Processed item {item_id}") finally: tracing.flush_queue() # fork and forkserver workers exit without flushing the queue def main_multi_process_example(): batch_group_id = str(uuid.uuid4()) processes = [] for i in range(3): p = multiprocessing.Process(target=process_item_in_worker, args=(i, batch_group_id), name=f"Worker-{i}") processes.append(p) p.start() for p in processes: p.join() if __name__ == "__main__": tracing.init(SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID")) main_multi_process_example() ``` ### Extending a Trace Across Workers If you need to maintain a single, continuous trace hierarchy where operations performed in separate workers are direct children of a span in the main process (or another worker), you must manually propagate the tracing context. This involves: 1. **Retrieving** the `group_id`, `trace_id` and `span_id` of the parent span in the originating process. 2. **Passing** these IDs to the new worker/process (e.g., as function arguments or message queue payload fields). 3. **Using** these explicit IDs when creating new spans in the worker, ensuring they are correctly linked as children. Be careful about setting the `Scope` object directly in your worker process as this scope may well be shared across jobs leading to incorrect tracing and leaking of tracing data. #### Example: Passing Context via Function Arguments ```python theme={null} import time import multiprocessing import scale_gp_beta.lib.tracing as tracing from scale_gp_beta import SGPClient def perform_subtask_in_worker(task_name: str, trace_id: str, parent_span_id: str): # Initialize tracing in the worker process. Can skip if you have SGP_API_KEY and SGP_ACCOUNT_ID set # tracing.init(SGPClient(api_key="...", account_id="...")) try: with tracing.create_span( name=f"worker_subtask_{task_name}", trace_id=trace_id, parent_id=parent_span_id, metadata={"worker_id": multiprocessing.current_process().name} ) as worker_span: print(f"Worker {multiprocessing.current_process().name}: Performing {task_name}") time.sleep(0.3) worker_span.output = {"status": "completed", "result": f"Task {task_name} finished"} finally: tracing.flush_queue() # fork and forkserver workers exit without flushing the queue def main_distributed_trace_example(): with tracing.create_trace("main_orchestrator_trace") as trace: processes = [] for i in range(2): p = multiprocessing.Process( target=perform_subtask_in_worker, args=(f"subtask_{i+1}", trace.trace_id, trace.span_id), name=f"ChildWorker-{i+1}" ) processes.append(p) p.start() for p in processes: p.join() if __name__ == "__main__": tracing.init(SGPClient(api_key="YOUR_API_KEY", account_id="YOUR_ACCOUNT_ID")) main_distributed_trace_example() ``` #### Important Considerations * **Worker Initialization:** A `spawn` or `forkserver` worker starts without the parent's tracing client: call `tracing.init()` at the start of the worker function, or set `SGP_API_KEY` and `SGP_ACCOUNT_ID` in its environment. A `fork` worker inherits the parent's client. * **Flushing Before Exit:** Call `tracing.flush_queue()` in a `finally` block at the end of the worker function, as both examples do. `fork` (the Linux default through Python 3.13) and `forkserver` (from 3.14) workers skip the SDK's exit-time flush, so their queued spans are lost otherwise, including those from a job that raised. * **Error Handling:** In distributed systems, be diligent with error handling and ensure spans are ended correctly, even if an exception occurs. Context managers (`with tracing.create_span(...)`) handle this automatically within their scope. # Profiler Source: https://docs.gp.scale.com/docs/v5/tracing/tracing-ui/profiler Analyze aggregated performance metrics and identify bottlenecks across your traces. ## Profiler The Profiler provides an aggregated overview of where your agent spends its time, helping you pinpoint which spans are consistently causing delays across many executions. Instead of analyzing individual traces one by one, the Profiler shows performance metrics aggregated from all traces matching your filters, making it easy to identify systemic bottlenecks. ### Agent Selection When you first open the Profiler tab, you'll see a list of agents to analyze. Select an agent to view its performance profile. ![agent-selection](https://mintcdn.com/scalegp/ff6HDxVm9Tf71q9I/images/tracing/agent-selection.png?fit=max\&auto=format\&n=ff6HDxVm9Tf71q9I\&q=85\&s=1e5d77c8136648c28802666484947316) ### Performance Metrics Once an agent is selected, the profiler displays key performance metrics at the top: * **Average Duration**: Total duration of the span names shown in the table, divided by the total number of matching spans. * **Total Spans**: The total span count for the selected agent and filters. * **P95 Latency**: The highest per-span-name P95 duration among the span names shown in the table. * **Unique Spans**: The number of distinct span names shown in the table (up to 100). * **View Agent Button**: Takes you to the Agents page to view the agent details. ![profiler-tab](https://mintcdn.com/scalegp/ndqkzfgZxrbrb4Lv/images/tracing/profiler-tab.png?fit=max\&auto=format\&n=ndqkzfgZxrbrb4Lv\&q=85\&s=5e271ac1e46df00b0f3cb5cf8f73f532) Clicking "View Agent" navigates you to the Agents page where you can see additional details about the agent: ![agent-page](https://mintcdn.com/scalegp/ff6HDxVm9Tf71q9I/images/tracing/agent.png?fit=max\&auto=format\&n=ff6HDxVm9Tf71q9I\&q=85\&s=8e33512c3cabc1109e0307876084f26a) ### Aggregated Span Table The span table is the core of the Profiler. Instead of listing individual traces, it lists a page of unique span names pertaining to the selected agent (e.g., `generate_assistant_message`, `followup_search_state`) and shows performance metrics aggregated from all traces. Key columns include: * **Span Name**: The name of the operation. * **Average Duration**: The average time this span takes to complete. * **Total Time (%)**: The percentage of total execution time across all traces consumed by this span. Sorting by this column is the most effective way to find major bottlenecks. * **P50 Latency**: The median execution time, representing typical performance. * **P95 Latency**: The 95th percentile latency, crucial for understanding worst-case performance and identifying outliers. * **Frequency (Avg. per Trace)**: How many times this span is called on average in a single execution. Click any row to view detailed metrics in the Span Inspector. ### Aggregate Flame Graph The flame graph provides a visual representation of aggregated span performance: * Each horizontal bar represents a span, with width proportional to its Total Time (%). * The visualization provides an immediate, intuitive map of your application's biggest latency contributors. * Spans are stacked vertically to show parent-child relationships. * Colors distinguish root-level spans, child spans, and the currently selected span. * Click on any bar in the flame graph to open the Span Inspector for detailed metrics. ### Span Inspector When you click a row in the span table or a bar in the flame graph, the Inspector panel slides out with a deep dive on that specific span. ![span-inspector](https://mintcdn.com/scalegp/ndqkzfgZxrbrb4Lv/images/tracing/span-inspector.png?fit=max\&auto=format\&n=ndqkzfgZxrbrb4Lv\&q=85\&s=c294b15b6851380c49f21181ac8ceb6d) **Key Metrics**: Two numbers for the selected span: * **% Exec. Time**: The percentage of total execution time consumed by this span across all traces. * **Avg Duration**: Mean execution time for this span. **Latency Distribution**: One bar each for the min, p50 (median), p95, p99, and max durations of this span name, drawn relative to the max. The bars show whether latency is consistent or has a long tail of slow outliers; they do not show duration frequencies. **Actions**: Links to the Traces page: * **View Slowest Spans**: Takes you to the Traces page filtered to this span name, with the span scope set to **All Spans** and results sorted by duration, slowest first, over the profiler's date range. It does not apply a percentile cutoff. * **View Recent Spans**: Takes you to the Traces page with **All Spans** selected and the same span name filter over the last 30 days. ## Example Workflow ### Debugging Inconsistent Agent Performance **The Problem**: A developer notices that the "Deep Research V3" agent's duration varies wildly. Some traces are fast, but others (like a 117.7s execution) are painfully slow. They don't know where to start looking. **Step 1: Navigate to Profiler** From the Traces view, they click the Profiler tab and select the "Deep Research V3" agent to analyze performance over the last 7 days. **Step 2: Identify the Bottleneck** They sort the Aggregated Span Table by **Total Time (%)** to find major bottlenecks. They immediately see that `final_report_state` is at the top, consuming 38% of all agent execution time. They also notice its P95 latency is over 2x its P50 median (85.6s vs 35.5s), confirming it's a source of inconsistent performance. **Step 3: Inspect the Span** They click the `final_report_state` row in the span table. The Span Inspector panel slides out, and its latency distribution shows p95, p99, and max well above p50. The gap suggests some calls take substantially longer than typical calls. **Step 4: Find Specific Examples** In the Span Inspector, they click "View Slowest Spans" to see the actual problem cases. **Step 5: Debug** They arrive at the Traces page with **All Spans** selected, filtered to `final_report_state` spans and sorted with the slowest first within the selected date range. They click the top row in this list, which brings them to the detailed waterfall view. Now, instead of guessing, they know to focus their investigation on the `final_report_state` span and its children to solve the systemic issue. ## Common Use Cases ### Identifying Performance Bottlenecks Use the Profiler to identify which spans are consuming the most time across your traces: 1. Select an agent to view its aggregated performance profile. 2. Sort the Aggregated Span Table by **Total Time (%)** to find major bottlenecks. 3. Examine the aggregate flame graph to visualize which operations take the longest—width represents each span's contribution to total execution time. 4. Look for spans with high P95 or P99 latencies relative to their P50 median, indicating inconsistent performance. 5. Click on slow spans to view detailed statistics and latency distribution in the Span Inspector. 6. Use "View Slowest Spans" to investigate specific instances of poor performance. ### Comparing Agent Performance Monitor and compare performance across different agents: 1. Switch between agents in the Agent Selection view. 2. Compare per-span-name latency metrics across agents over the same date range. 3. Identify which agents or workflows need optimization. 4. Use the flame graph to understand differences in execution patterns. ### Monitoring Performance Over Time Track performance trends and identify regressions: 1. Regularly check P95 and P99 latencies for critical spans. 2. Compare current metrics against historical baselines. 3. Investigate spikes in execution time by viewing recent traces containing affected spans. 4. Use span-level metrics to validate the impact of optimizations. # Trace Detail View Source: https://docs.gp.scale.com/docs/v5/tracing/tracing-ui/trace-detail-view Inspect individual traces with comprehensive span-level details and visualizations. ## Trace Detail View Clicking on a trace opens the detail view, which provides a comprehensive breakdown of the entire workflow. ![trace-detail-view](https://mintcdn.com/scalegp/ndqkzfgZxrbrb4Lv/images/tracing/trace-detail-view.png?fit=max\&auto=format\&n=ndqkzfgZxrbrb4Lv\&q=85\&s=1b6ad9bf8ef5d08f9cd01da3c978ec58) ### Navigation Controls At the top of the trace detail view, you'll find several navigation and viewing options: * **Previous/Next Trace**: Navigate between traces in your current filtered results without returning to the main table. * **Download Trace**: Export the complete trace data for offline analysis or sharing. * **Gantt Chart View**: Toggle to a Gantt chart visualization to see span timing and dependencies. * **Search**: Search within the trace for specific spans or content. ### Data Display Options The detail view provides flexible viewing options for trace and span data: * **Pretty/Raw Toggle**: Switch between formatted (Pretty) and raw (Raw) views of the data for easier reading or debugging. * **Format Selection**: Toggle between JSON and YAML formats to view the data in your preferred structure. * **Edit Mode**: Click the **Edit** button to modify span data for thread forking (available when viewing span details). See the [Thread Forking](#thread-forking) section for more details. * **Expand View**: Open a full-screen view of the data for detailed inspection. ### Trace Visualization Graph The trace graph displays a hierarchical visualization of all spans within the trace: * **Hierarchical Layout**: Spans are arranged in a tree structure on the left showing parent-child relationships, with each span displaying its name and duration. * **Timeline View**: Each span is positioned horizontally based on its start time and duration, making it easy to identify sequential vs. parallel operations. * **Interactive Navigation**: Click on any span in the graph to view its detailed information in the side panel. ### Span Details Panel When you select a span from the graph, the details panel on the right reveals comprehensive information: * **Input**: The data that was passed into the operation, helping you understand what the span received. * **Output**: The data produced by the operation, showing the result of the span's execution. * **Initial State**: The starting state or context for the span, particularly useful for agent workflows. * **Metadata**: Custom key-value pairs attached to the span for additional context. * **Span Attributes**: All span properties including trace ID, parent ID, and span type. * **Assessments**: Click the **Assessments** button to add evaluations or annotations to the span. You can create assessments to track quality, correctness, or other criteria for specific spans in your traces. ### Performance Analysis The trace detail view helps identify performance issues: * **Duration Comparison**: Quickly spot spans that take disproportionately long compared to siblings. * **Critical Path**: Identify the sequence of spans that determines the overall trace duration. * **Parallel Execution**: Visualize which operations run concurrently vs. sequentially. ## Thread Forking > [!NOTE] > > Available for traces generated by SGP native applications. For traces created by SGP's native agent system, you can trigger thread forking directly from the trace detail view. This advanced debugging feature allows you to: * Resume execution from a specific point in the trace. * Test different execution paths without rerunning the entire workflow. * Debug complex decision-making logic in AI agents. To fork a trace: 1. Select a span in the trace graph where you want to resume execution. 2. In the span details panel, click the **Edit** button to modify the input, output, or initial state. 3. Make your desired changes to test different scenarios. 4. Click **Save and Execute** to create a new execution thread from that point forward. This creates a new trace with your modified parameters, allowing you to experiment with different inputs or states without affecting the original trace. ![forking](https://mintcdn.com/scalegp/ndqkzfgZxrbrb4Lv/images/tracing/forking.png?fit=max\&auto=format\&n=ndqkzfgZxrbrb4Lv\&q=85\&s=0557eea614156ce0b2954bf927bb4b7a) ![forking-example](https://mintcdn.com/scalegp/ff6HDxVm9Tf71q9I/images/tracing/forking-example.png?fit=max\&auto=format\&n=ff6HDxVm9Tf71q9I\&q=85\&s=67d7fdf8d725b162d9c9b0f1e586f0c5) ## Common Use Cases ### Debugging Individual Trace Failures When investigating a specific failed trace: 1. Open the trace detail view from the Traces table. 2. Use the trace visualization graph to identify which span failed. 3. Inspect the failed span's input, output, and metadata to understand what went wrong. 4. Review parent spans to see what data was passed to the failed operation. 5. Check sibling spans to see if related operations also encountered issues. ### Understanding Execution Flow To understand how a complex workflow executes: 1. Open the trace detail view for a representative trace. 2. Use the hierarchical layout to see the parent-child relationships between spans. 3. Examine the timeline view to identify sequential vs. parallel operations. 4. Review input and output for each span to understand data flow through the system. 5. Use the search function to quickly locate specific operations within the trace. ### Analyzing Slow Traces When investigating performance issues in a specific trace: 1. Open the slow trace from the Traces table (sorted by duration). 2. Use the trace visualization to identify the critical path—the sequence of spans that determines total duration. 3. Look for spans with unusually long durations compared to typical executions. 4. Examine span metadata for context about why operations took longer than expected. 5. Use the Gantt Chart view to see timing and dependencies more clearly. ### Testing Alternative Execution Paths For SGP native applications, use thread forking to experiment with different execution scenarios. See the [Thread Forking](#thread-forking) section above for detailed instructions on how to modify span data and create alternative execution paths. # Traces Source: https://docs.gp.scale.com/docs/v5/tracing/tracing-ui/traces-table Monitor and analyze your application traces through the dedicated Traces page. ## Overview The Traces page provides a comprehensive interface for monitoring, filtering, and analyzing trace data from your instrumented applications. This dedicated UI gives you visibility into your workflows, allowing you to identify performance bottlenecks, debug errors, and understand the execution flow of complex operations. At the top right of the page, you'll find a date range selector that allows you to filter traces by time period. Adjusting this date range updates both the overview metrics graphs and the traces table to show only data from your selected timeframe. You can also toggle auto-refresh to automatically update the view with new traces as they arrive. ![tracing](https://mintcdn.com/scalegp/ndqkzfgZxrbrb4Lv/images/tracing/tracing-page.png?fit=max\&auto=format\&n=ndqkzfgZxrbrb4Lv\&q=85\&s=d5efcae4da6988860221e99a0870221b) ## Overview Metrics At the top of the Traces page, you'll find three interactive graphs that provide high-level insights into your trace data over time. You can switch between different views by clicking the tabs: ### Total Requests The Total Requests view shows the volume of traces over time, giving you a quick overview of activity levels. Hover over any point in the graph to see the exact count of traces for that time period. This helps you identify traffic patterns, usage spikes, or periods of low activity. ![total-requests](https://mintcdn.com/scalegp/ndqkzfgZxrbrb4Lv/images/tracing/total-requests.png?fit=max\&auto=format\&n=ndqkzfgZxrbrb4Lv\&q=85\&s=53a6636a1dfc17e4439d840d6e1d3816) ### Latency The Latency view displays performance metrics across your traces, showing p50 (median), p75, and p95 percentile response times. This multi-line graph helps you understand not just average performance, but also worst-case scenarios and performance consistency. Spikes in p95 latency can indicate performance issues affecting a subset of requests. ![latency](https://mintcdn.com/scalegp/ndqkzfgZxrbrb4Lv/images/tracing/latency.png?fit=max\&auto=format\&n=ndqkzfgZxrbrb4Lv\&q=85\&s=743460536de6b0144307dc8587772326) ### Errors The Errors view tracks the rate of failed traces over time, allowing you to quickly spot error patterns or incidents. Use this view to correlate error spikes with deployments. ![errors](https://mintcdn.com/scalegp/ff6HDxVm9Tf71q9I/images/tracing/errors.png?fit=max\&auto=format\&n=ff6HDxVm9Tf71q9I\&q=85\&s=cf8b57196567d32fdce7bd08b0461643) ## Traces Table The main view displays a table of all traces collected from your applications. Each row represents a single trace (workflow), showing key information at a glance: * **Content**: A preview of the trace's input and output data, giving you a quick glimpse of what the workflow processed. * **Trace Name**: The descriptive name identifying the trace. * **Timestamp**: When the trace began execution. * **Duration**: The total time taken for the entire workflow to complete. * **Type**: The span type for categorization. Click on any trace row to open the detailed trace view and inspect individual spans. ![traces-table](https://mintcdn.com/scalegp/ndqkzfgZxrbrb4Lv/images/tracing/traces-table.png?fit=max\&auto=format\&n=ndqkzfgZxrbrb4Lv\&q=85\&s=ae7e7040e9bd9e007dac519f68e20b44) ### Controlling the Content Preview The **Content** column extracts a text preview from the root span's `input` and `output` fields. The preview system checks specific preferred keys in priority order before falling back to the first available value. **Input preview** priority: `output_text`, `output-text`, `user_message`, `query`, `serp_query`, `content`, `message`, `prompt`, `title`, `value`, `result`, `text`, `raw_output` **Output preview** priority: `summary_text`, `content_text`, `summary`, `content` → then the same general key list above If your input or output doesn't use one of these keys, the preview may display an unhelpful value (e.g., showing a numeric field like `age` instead of a meaningful identifier). In the default **Root Spans** view, add the preferred keys to the root span's `input` and `output`. A child span's keys control that child's preview only when **All Spans** is selected. After [initializing tracing](/docs/v5/tracing/sdk/configuration-options), you can populate both the root preview and a detailed child span like this: ```python theme={null} import scale_gp_beta.lib.tracing as tracing record = { "title": "MRN-20394", "mrn": "MRN-20394", "age": 58, "department": "cardiology", } with tracing.create_trace(name="patient_intake") as trace: trace.root_span.input = record with tracing.create_span("process_record", input=record) as span: # Replace this result with your intake processing output. result = {"status": "processed", "risk_score": 0.72} span.output = result trace.root_span.output = { "summary": "Intake complete — MRN-20394", **result, } ``` > [!TIP] > > You don't need to remove your existing keys. Add a preferred key alongside your other fields. The preview uses it, and the full data stays available in the [Trace Detail View](/docs/v5/tracing/tracing-ui/trace-detail-view). ### Working with Selected Traces You can select traces from the table using the checkboxes on the left of each trace row. Once you've selected traces, additional options become available: * **Add to Dataset**: Create a new dataset from the selected traces for evaluations. * **Download**: Export the selected traces for offline analysis or sharing. ![add-to-dataset](https://mintcdn.com/scalegp/ff6HDxVm9Tf71q9I/images/tracing/add-to-dataset.png?fit=max\&auto=format\&n=ff6HDxVm9Tf71q9I\&q=85\&s=333f5e086630d3d0a53a0b2ac554a7af) ![dataset](https://mintcdn.com/scalegp/ff6HDxVm9Tf71q9I/images/tracing/dataset.png?fit=max\&auto=format\&n=ff6HDxVm9Tf71q9I\&q=85\&s=833373243fb92a0dedb3ae2130818218) ## Searching Traces The search bar above the Traces table lets you find traces by content. The query runs across each span's `input` and `output` fields. If the query is an exact match for a `trace_id`, the corresponding trace is returned directly — useful for jumping to a specific trace from a log line, error report, or shared link. ## Filtering Traces Use the filter controls on the left side of the page to narrow down the traces displayed in the table. You can use the search box at the top of the filters panel to quickly find specific filter options by typing keywords. Available filters include: #### Duration Filter traces by their execution time using either a min-to-max range input or an interactive slider. This helps you quickly identify slow workflows or find traces within a specific performance window. #### Span Scope Filter traces based on the scope of spans you want to examine. You can choose between **Root Spans** (only the top-level spans of each trace) or **All Spans** (every span within the traces). #### Status Filter by completion status to quickly identify successful traces or those that encountered errors during execution. #### Application Filter traces by the application that generated them, useful when monitoring multiple applications or services. #### Span Name Filter by specific span names to find traces containing particular operations or function calls. #### Span Type Filter by span type to view traces of a particular category, such as LLM calls, database queries, or API requests. #### Agent Name Filter traces generated by specific agents, helping you monitor individual agent performance and behavior. #### Agent ID Filter by unique agent identifiers for precise agent-level filtering. #### ACP Type Filter traces by ACP (Agent Control Protocol) type for agent-specific workflows. #### Assessment Filter traces based on assessment criteria or evaluation results. ![filters](https://mintcdn.com/scalegp/ff6HDxVm9Tf71q9I/images/tracing/filters.png?fit=max\&auto=format\&n=ff6HDxVm9Tf71q9I\&q=85\&s=712a09ee96c4a877ae905ab30300253e) ## Common Use Cases ### Debugging Errors When a workflow fails, use the Traces page to: 1. Filter by status to show only error traces. 2. Open the trace detail view to see exactly which span failed. 3. Inspect the error span's input, output, and metadata to understand the root cause. 4. Review parent spans to see what led to the failure. ### Performance Optimization To identify performance bottlenecks: 1. Sort traces by duration to find slow workflows. 2. Open the trace graph to visualize the execution timeline. 3. Identify spans with long durations that could be optimized. 4. Look for sequential operations that could be parallelized. ### Monitoring Workflows For ongoing monitoring of your applications: 1. Use the time range filter to view recent traces. 2. Monitor the success/error ratio in the table. 3. Set up custom metadata filters to track specific features or user segments. 4. Review trace patterns to ensure expected execution flows. # Generate legacy text completion from prompt Source: https://docs.gp.scale.com/reference/v5/inference/generate-legacy-text-completion-from-prompt /openapi/v5.yaml post /v5/completions Generates a legacy text completion from a raw `prompt` (a string or list of strings). Use this endpoint for non-chat, prompt-in/text-out inference using the OpenAI text-completion contract; use /v5/chat/completions when you have a structured `messages` array, /v5/responses for the OpenAI Responses API, and /v5/inference for payloads that follow no OpenAI schema. The model is selected from `model` given as `vendor/name` and routed to the matching per-vendor gateway. When `stream` is set the response is delivered as server-sent events; otherwise a single `text_completion` object is returned. Token usage is recorded for the account, read from the final chunk on streaming responses. # Generate OpenAI chat completion from messages Source: https://docs.gp.scale.com/reference/v5/inference/generate-openai-chat-completion-from-messages /openapi/v5.yaml post /v5/chat/completions Generates a chat completion from an OpenAI-style `messages` array. Use this endpoint for standard messages-based chat inference; use /v5/completions instead when you have a raw text `prompt` rather than messages, /v5/responses for the OpenAI Responses API contract, and /v5/inference when the payload does not follow any OpenAI schema. The request accepts the OpenAI Chat Completions parameters plus a fixed set of vendor generation parameters such as `top_k`, `guided_choice`, and `thinking`; other extra fields are dropped, and fields that set the provider endpoint, credentials, or headers (for example `api_base`, `api_key`, `extra_headers`) are rejected with a 422. The model is selected from `model` given as `vendor/name`; most vendors are served through the litellm proxy gateway, while OpenAI may use a native gateway. When `stream` is set the response is delivered as server-sent events of `chat.completion.chunk`; otherwise a single `chat.completion` object is returned. Token usage is recorded for the account, and for streaming responses it is read from the final chunk. # List available chat completion models Source: https://docs.gp.scale.com/reference/v5/inference/list-available-chat-completion-models /openapi/v5.yaml get /v5/chat/completions/models Lists the models available for use with /v5/chat/completions. Results are served directly from the stored inference-models catalog filtered to the chat-completion model type, not from a live provider probe, so availability reflects the catalog's recorded status. Pass the optional `model_vendor` query parameter to restrict results to a single vendor. Results are paginated, and each entry reports the model name, vendor, type, and availability. If the underlying catalog query fails the endpoint returns an empty list rather than raising an error. # Run inference with free-form payload Source: https://docs.gp.scale.com/reference/v5/inference/run-inference-with-free-form-payload /openapi/v5.yaml post /v5/inference Runs a model using a free-form, vendor-native request payload rather than a fixed OpenAI schema. Use this endpoint when the target model does not fit the OpenAI chat, responses, or text-completion contracts: the `args` field is an arbitrary dict passed straight through to the selected vendor gateway, and the reply is returned inside `response` as arbitrary JSON (object, array, string, number, or boolean). Prefer /v5/chat/completions, /v5/responses, or /v5/completions when your request matches one of those OpenAI-standard shapes, since those return typed OpenAI response objects. The model is chosen from the `model` field formatted as `vendor/name`, which selects the per-vendor inference gateway. When the request enables streaming, the response is sent as server-sent events with each chunk wrapped as a `generic_inference.chunk` object; otherwise a single `generic_inference` object is returned. # Run OpenAI Responses API (OpenAI only) Source: https://docs.gp.scale.com/reference/v5/inference/run-openai-responses-api-openai-only /openapi/v5.yaml post /v5/responses Runs inference using the OpenAI Responses API contract, taking an `input` rather than a chat `messages` array. This endpoint is restricted to OpenAI-vendor models: requests whose `model` (`vendor/name`) resolves to any other vendor are rejected with an unsupported-vendor error, unlike /v5/chat/completions and /v5/inference which support many vendors. Use it when you specifically need Responses API features such as `previous_response_id` chaining, `instructions`, or `store`; use /v5/chat/completions for the standard messages-based chat contract. When `stream` is set the response streams as server-sent events; otherwise a single `response` object is returned. # Archive a question Source: https://docs.gp.scale.com/reference/v5/questions/archive-a-question /openapi/v5.yaml delete /v5/questions/{question_id} Soft-delete a question by setting its ``archived_at`` timestamp. This is not a permanent delete: the row is retained and can be brought back with the restore endpoint. Once archived, the question is excluded from list results by default (unless ``include_archived`` is set) and can no longer be updated until it is restored. The archived question is returned in the response with its ``archived_at`` populated. # Create a question Source: https://docs.gp.scale.com/reference/v5/questions/create-a-question /openapi/v5.yaml post /v5/questions Create a new question owned by the caller's account. The question is one of six types selected by the required ``question_type`` discriminator (categorical, rating, number, free_text, form, or timestamp), and each type carries its own ``configuration`` block; ``name`` and ``prompt`` are required for every type. Questions are the reusable units that are grouped into question sets and referenced by rubrics, so create them before assembling a question set. The created question is scoped to the caller's account and records the creating identity. The question type and its configuration are set at creation time and cannot be changed afterward through the update endpoint. # Get a question Source: https://docs.gp.scale.com/reference/v5/questions/get-a-question /openapi/v5.yaml get /v5/questions/{question_id} Retrieve a single question in the caller's account by its ID. Both active and archived questions are returned; the archived state is conveyed by the ``archived_at`` field on the response. The lookup is scoped to the caller's account, and a question that does not exist within that account results in a not-found error. # List questions Source: https://docs.gp.scale.com/reference/v5/questions/list-questions /openapi/v5.yaml get /v5/questions Return a paginated list of the caller's account's questions. Archived questions are excluded by default; set ``include_archived`` to true to include them alongside active ones. Pass ``ids`` to restrict the result to a specific set of question IDs. Results are paginated through the standard pagination parameters. Use this to discover existing questions before adding them to a question set, or to audit which questions have been archived. # Restore an archived question Source: https://docs.gp.scale.com/reference/v5/questions/restore-an-archived-question /openapi/v5.yaml post /v5/questions/{question_id}/restore Restore a previously archived question, reversing an archive. This clears the question's ``archived_at`` timestamp, returning it to the active state so it reappears in default listings and becomes editable again. It is the inverse of the archive (DELETE) operation and is intended for questions that were archived by mistake or are needed again. # Update a question Source: https://docs.gp.scale.com/reference/v5/questions/update-a-question /openapi/v5.yaml patch /v5/questions/{question_id} Update a question's display name. Only the ``name`` is mutable through this endpoint; a question's type, prompt, and type-specific configuration are fixed at creation and cannot be changed here. Updating an archived question is rejected with a client error, so restore it first if it needs to be edited. The update is scoped to the caller's account. # Add a criterion to a rubric Source: https://docs.gp.scale.com/reference/v5/rubrics/add-a-criterion-to-a-rubric /openapi/v5.yaml post /v5/rubrics/{rubric_id}/criteria Add a single criterion to an existing rubric. Creates a new criterion at version 1 under the given rubric. Criteria are versioned independently of the rubric they belong to. The request is rejected if the target rubric is archived. # Archive a rubric Source: https://docs.gp.scale.com/reference/v5/rubrics/archive-a-rubric /openapi/v5.yaml delete /v5/rubrics/{rubric_id} Archive a rubric (soft delete). This is a soft delete, not a permanent one: it records an archived timestamp on the rubric while retaining the rubric and its version history, so it can later be brought back via the restore variant of the update endpoint. An archived rubric is hidden from the list endpoint but stays retrievable by id. The rubric's criteria are left untouched — archiving does not cascade to or delete them. Archiving a rubric that is already archived is rejected. # Create a rubric Source: https://docs.gp.scale.com/reference/v5/rubrics/create-a-rubric /openapi/v5.yaml post /v5/rubrics Create a rubric, optionally with its initial criteria in the same request. The rubric is created for the caller's account at version 1. If the optional `criteria` field is supplied it must contain at least one entry, and each entry is created as a criterion attached to the new rubric; omit it to create an empty rubric and add criteria later via the criteria endpoint. The response returns the created rubric together with the full detail of any criteria created in this call. # Get a rubric Source: https://docs.gp.scale.com/reference/v5/rubrics/get-a-rubric /openapi/v5.yaml get /v5/rubrics/{rubric_id} Retrieve a single rubric with the full detail of its current criteria. Returns the latest version of the rubric along with all of its current (latest-version) criteria in full — not the slim title/weight summary returned by the list endpoint. Archived rubrics remain retrievable through this endpoint even though they are hidden from the list endpoint. # List criterion versions Source: https://docs.gp.scale.com/reference/v5/rubrics/list-criterion-versions /openapi/v5.yaml get /v5/rubrics/{rubric_id}/criteria/{rubric_criteria_id}/versions List the full version history of a single criterion, paginated. Because criterion updates are append-only, this returns one entry per version, exposing every prior state of the criterion, ordered by version descending (newest first) by default. The request is rejected if the criterion does not belong to the rubric named in the path. # List rubrics Source: https://docs.gp.scale.com/reference/v5/rubrics/list-rubrics /openapi/v5.yaml get /v5/rubrics List the current account's rubrics, paginated. Only the latest version of each rubric is returned, and archived rubrics are excluded (this endpoint has no option to include them). The optional `title` filter matches as a case-insensitive substring, and the optional `tags` filter narrows results to rubrics carrying the given tags. Each returned rubric carries only a slim criteria summary (title and weight per criterion) rather than full criteria; use the get-rubric endpoint to retrieve full criteria detail. # Update a criterion Source: https://docs.gp.scale.com/reference/v5/rubrics/update-a-criterion /openapi/v5.yaml patch /v5/rubrics/{rubric_id}/criteria/{rubric_criteria_id} Apply a partial update to a criterion, creating a new version. Updates are append-only: rather than overwriting the criterion in place, each update inserts a new immutable version with an incremented version number, so every prior state stays available through the versions endpoint. Fields not supplied are carried forward from the current version. The request is rejected if the criterion does not belong to the rubric named in the path. # Update or restore a rubric Source: https://docs.gp.scale.com/reference/v5/rubrics/update-or-restore-a-rubric /openapi/v5.yaml patch /v5/rubrics/{rubric_id} Update a rubric's fields, or restore a previously archived rubric. The request body is a discriminated union. Send `{"restore": true}` to restore an archived rubric (clearing its archived state); send any other field set (such as `title` or `tags`) to update those fields, carrying forward any field not supplied. Both paths are append-only and produce a new version of the rubric rather than mutating the existing row. Updating the fields of an archived rubric is rejected — restore it first. Restoring a rubric that is not archived is a no-op that returns it unchanged without creating a new version. # Configure Vector Store Source: https://docs.gp.scale.com/reference/v5/vector-stores/configure-vector-store /openapi/v5.yaml post /v5/vector-stores/{vector_store_name}/configure Update the indexed metadata fields configuration for a vector store. This replaces the current set of indexed metadata fields. Only indexed fields can be used for filtering during query, list, and count operations; non-indexed fields are still stored and returned, but cannot be filtered on. **Field Types:** Only STRING, NUMBER, and BOOLEAN fields can be indexed (maximum 20 fields). OBJECT and LIST types are stored but cannot be indexed for filtering. **Adding Fields:** New indexed fields can be added at any time. They are indexed for documents upserted after the change; to make existing documents filterable on a new field, re-upsert them. **Removing Fields:** Omitting a field removes it from this configuration, so it can no longer be filtered on. The underlying index is append-only, so removal does not reclaim storage or reduce write overhead; the field stays in the physical index until the store is recreated. Prefer indexing only the fields you filter on. **Note:** The `name` and `embedding_config` are immutable after creation. # Create Vector Store Source: https://docs.gp.scale.com/reference/v5/vector-stores/create-vector-store /openapi/v5.yaml post /v5/vector-stores/create Create a new vector store for storing and querying document embeddings. The vector store name must be unique within your account and follow naming conventions (3-63 characters, alphanumeric with hyphens/underscores). Once created, the embedding configuration and dimensions are immutable and cannot be changed. To use a different model, you must create a new vector store. **Embedding Configuration:** Provide `embedding_config` (for base or custom model deployments), `embedding_model` (shorthand for a base model), or `dimensions` only (raw embeddings). - With `embedding_config` or `embedding_model`: dimensions are auto-derived, and documents can be upserted with text content (auto-embedded) or with pre-computed embeddings. - With `dimensions` only: the store accepts only pre-computed embeddings. Semantic/hybrid queries are not supported (lexical search only). **Indexed Fields:** Optionally specify metadata fields to index at creation time. Only indexed fields can be used for filtering -- indexing is required, not just a performance optimization. Additional indexed fields can be added later using the configure endpoint, but cannot be removed once added. Keep in mind that each indexed field increases write latency and storage overhead, so only index fields you actively filter on. # Delete Vectors Source: https://docs.gp.scale.com/reference/v5/vector-stores/delete-vectors /openapi/v5.yaml post /v5/vector-stores/{vector_store_name}/delete Delete documents from a vector store by document IDs or metadata filter criteria. **Delete by IDs:** Provide an array of document IDs to delete specific documents. Non-existent documents are silently skipped. **Delete by Filter:** Use metadata filters to delete all documents matching the specified criteria (e.g., delete all documents where `status: "archived"`). The filter must specify at least one condition and cannot be empty. To delete all documents, use the drop endpoint instead. **Filter Operators:** Supports MongoDB-style operators including equality (`{"field": "value"}`), comparison (`$gt`, `$gte`, `$lt`, `$lte`, `$eq`, `$ne`), logical (`$and`, `$or`, `$not`), and membership (`$in`, `$nin`). Only indexed metadata fields can be used for filtering. **Best Practice:** Use the count endpoint with the same filter to preview the number of documents that will be deleted before executing the deletion operation. # Drop Vector Store Source: https://docs.gp.scale.com/reference/v5/vector-stores/drop-vector-store /openapi/v5.yaml post /v5/vector-stores/{vector_store_name}/drop Permanently delete a vector store and all its contents. **⚠️ WARNING:** This is a destructive operation that cannot be undone. All documents, embeddings, metadata, and index configurations will be permanently deleted. Data recovery is not possible after deletion. # Get Vector Store Source: https://docs.gp.scale.com/reference/v5/vector-stores/get-vector-store /openapi/v5.yaml get /v5/vector-stores/{vector_store_name} Retrieve detailed configuration and metadata for a specific vector store. Returns the store's embedding model, dimensions, indexed metadata field definitions, creation timestamp, and last update timestamp. Use this to verify store settings before performing operations or to display store information in your application. # List Vector Stores Source: https://docs.gp.scale.com/reference/v5/vector-stores/list-vector-stores /openapi/v5.yaml get /v5/vector-stores List all vector stores in your account with pagination. Returns vector stores sorted by creation date (newest first). Each store includes its configuration, embedding model, dimensions, indexed fields, and timestamps. # Upsert Vectors Source: https://docs.gp.scale.com/reference/v5/vector-stores/upsert-vectors /openapi/v5.yaml post /v5/vector-stores/{vector_store_name}/upsert Insert new documents or update existing documents in a vector store. **Upsert Behavior:** If a document ID already exists, it will be completely replaced with the new content and metadata. The previous document's text, embedding, and all metadata fields are discarded. If the ID does not exist, a new document is created. **Document Content:** Each document supports several modes: - `content` only: text is automatically embedded using the store's configured model. - `embedding` only: pre-computed embedding vector is used directly. Dimension must match the store's configuration. - Both `content` and `embedding`: the pre-computed embedding is stored and text is kept for retrieval/search. - Neither (metadata-only): only metadata is updated on an existing document without re-embedding. If the document does not exist, it will appear as a failure in the batch response. A store created without an embedding model (dimensions-only) only accepts documents with pre-computed `embedding`. **Batch Operations:** This endpoint supports batch operations with partial success handling and mixed document types (some with raw embeddings, some with content) in the same call. **Metadata:** Supports nested metadata with string, number, boolean, object, and array types. Null values are not permitted—omit the field or use an empty string instead. # Create Agent Config Source: https://docs.gp.scale.com/reference/v5/agent-configs/create-agent-config /openapi/v5.yaml post /v5/agent_configs Create a reusable agent configuration (system prompt, harness, model, allowed tools) under the caller's account. The config is a template that a chat session or a non-chat trigger later turns into task params; creating one does not start any task. ``persistent_workspace`` opts tasks made from this config into a durable ``/workspace`` that survives sandbox death (off by default, and fixed once a task starts), and ``repos`` overrides which repositories provisioning clones into that workspace — omit it (null) to use the deployment default, or pass an empty list to clone nothing. A ``repos`` override is rejected with a 422 on the model-agnostic (litellm) harness unless ``persistent_workspace`` is also true, because non-persistent litellm tasks run in a pre-cloned warm-pool sandbox where the override would be ignored. ``allowed_tools`` may name MCP servers (Slack, Linear, GitHub, ...) alongside harness tools, and granting an MCP server name authorizes every tool it exposes. # Delete Agent Config (hard delete) Source: https://docs.gp.scale.com/reference/v5/agent-configs/delete-agent-config-hard-delete /openapi/v5.yaml delete /v5/agent_configs/{agent_config_id} Permanently delete an agent config; this is a hard delete, not an archive, so the row is removed and cannot be restored. Because a config is only a template, deleting it does not stop or alter any task already created from it. A user caller can only delete a config they created unless fine-grained access control grants access. The response echoes the deleted id. # Get Agent Config Source: https://docs.gp.scale.com/reference/v5/agent-configs/get-agent-config /openapi/v5.yaml get /v5/agent_configs/{agent_config_id} Fetch a single stored agent configuration by id, including its ``persistent_workspace`` flag and any ``repos`` override. This returns the saved record as-is and does not compute task params; use ``{agent_config_id}/resolve`` when you need the config projected into the params a task would run with. A user caller can only read a config they created unless fine-grained access control grants access, while a service account can read any config under the account; a missing or out-of-scope id returns a 404. # List Agent Configs Source: https://docs.gp.scale.com/reference/v5/agent-configs/list-agent-configs /openapi/v5.yaml get /v5/agent_configs List agent configurations visible to the caller, with cursor-based pagination. A user caller sees only the configs they created unless fine-grained access control (FGAC) grants them access to others; a service-account caller sees every config under the account. This returns the stored config records as-is — use ``{agent_config_id}/resolve`` instead when you need a config projected into the params a task would run with. Because deleting a config removes it outright rather than archiving it, there are no archived configs to page through here. # List MCP Tool Names Source: https://docs.gp.scale.com/reference/v5/agent-configs/list-mcp-tool-names /openapi/v5.yaml get /v5/agent_configs/mcp_tools Return the fixed set of tool names that route to MCP servers rather than harness-provided tools. These are the subset of ``allowed_tools`` values (Slack, Linear, GitHub, ...) the platform treats as MCP servers; the list is a static enum, not account data, so it is identical for every caller. It is exposed mainly so the generated SDK carries the ``McpTool`` type and the frontend can assert its own MCP-versus-harness tool classifier against the backend to prevent drift. It does not list configs or the tools actually granted to any particular config. # Update Agent Config Source: https://docs.gp.scale.com/reference/v5/agent-configs/update-agent-config /openapi/v5.yaml patch /v5/agent_configs/{agent_config_id} Partially update a stored agent config; only fields present in the request body are changed. ``persistent_workspace`` and ``repos`` are both patchable — an explicit null on ``repos`` clears the override back to the deployment default, while the always-present fields (name, system_prompt, harness, allowed_tools, model, and persistent_workspace) reject an explicit null. Because ``persistent_workspace`` and ``repos`` are read when a task is created and are fixed for a task's life, changing them affects only tasks created afterward, not one already running. The optional ``task_id`` query parameter opts into a live-config side effect: after the row is persisted, the changed pass-through fields (system_prompt, model, harness, and allowed_tools split into harness tools versus MCP servers) are shallow-merged into that running task's params on Agentex in a background task so the worker picks them up on its next turn; ``persistent_workspace`` and ``repos`` are intentionally not forwarded to a running task. That side effect runs after the response is sent, is best-effort, and no-ops if the task does not exist or the caller does not own it. A user caller can only update a config they created unless fine-grained access control grants access. # List container image builds Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/list-container-image-builds /openapi/v5.yaml get /v5/builds List container image build records for the caller's account as a paginated collection. Each entry is one individual build with its current lifecycle status; pass `agent_name`, `source_commit`, or `working_tree_hash` to return only matching builds. Archived builds are excluded, and results are limited to builds the caller is permitted to read. Use this to enumerate individual builds; to instead list agents that have been built but not yet deployed, use `GET /v5/builds/undeployed`. # Submit container image build Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/submit-container-image-build /openapi/v5.yaml post /v5/builds Create a container image build record. This multipart request accepts exactly one image source. Provide `context_archive` to stream a tar.gz build context to the configured cloud build provider, or provide `image_url` to register an existing tagged image as an immediately successful build without invoking the provider. `build_args` and `platform` apply only to archive builds. Exactly one of `agent_name` or `agent_id` must identify the target agent. `agent_name` creates a new build-only agent and must be unused. `agent_id` targets an existing agent. Archive builds normally return in a queued or running state; prebuilt images can immediately be passed to Cloud Deploy using the returned build ID. Archives must be non-empty and no larger than 500 MB. # Update or assign annotation tasks in bulk Source: https://docs.gp.scale.com/reference/v5/annotation-tasks/update-or-assign-annotation-tasks-in-bulk /openapi/v5.yaml patch /v5/annotation-tasks/batch Update many annotation tasks in one request, operating in one of two mutually exclusive modes. In ids mode, provide a list of task IDs together with an `assigned_to` and/or a `status` (limited to `pending_redo` or `completed`) to apply those field changes to each listed task; only tasks that currently exist are affected and IDs that do not resolve are silently skipped. In audit_assignment mode, provide an evaluation ID, queue ID, and evaluation item IDs to assign Level 1 and/or Level 2 auditors for those items, creating the underlying audit tasks when they do not yet exist; a Level 2 auditor cannot be assigned unless a Level 1 auditor is already present. Exactly one of `ids` or `audit_assignment` must be supplied, and `assigned_to`/`status` are only valid in ids mode. Access and state checks run before any write, so the request is rejected as a whole with nothing persisted if any task is inaccessible, if a status change is attempted without a modification role, or if an already-completed task would be reassigned; audit assignment also requires a modification role. Reassigning a task to a new user emits a task-start lineage event, and the response returns the affected annotation tasks. # Create Credential Source: https://docs.gp.scale.com/reference/v5/credentials/create-credential /openapi/v5.yaml post /v5/credentials Create a new credential for storing sensitive data like API keys, tokens, or other secrets. # Delete Credential Source: https://docs.gp.scale.com/reference/v5/credentials/delete-credential /openapi/v5.yaml delete /v5/credentials/{credential_id} Permanently delete a credential and all its associated data. # Get Credential by ID Source: https://docs.gp.scale.com/reference/v5/credentials/get-credential-by-id /openapi/v5.yaml get /v5/credentials/{credential_id} Retrieve a specific credential by its unique identifier. # Get Credential by Name Source: https://docs.gp.scale.com/reference/v5/credentials/get-credential-by-name /openapi/v5.yaml get /v5/credentials/name/{credential_name} Retrieve a specific credential by its name. # Get Credential Secret by ID Source: https://docs.gp.scale.com/reference/v5/credentials/get-credential-secret-by-id /openapi/v5.yaml post /v5/credentials/{credential_id}/secret Retrieve the plaintext payload of a credential by its ID. # Get Credential Secret by Name Source: https://docs.gp.scale.com/reference/v5/credentials/get-credential-secret-by-name /openapi/v5.yaml post /v5/credentials/name/{credential_name}/secret Retrieve the plaintext payload of a credential by its name. # List Credentials Source: https://docs.gp.scale.com/reference/v5/credentials/list-credentials /openapi/v5.yaml get /v5/credentials Retrieve a paginated list of all credentials for the current account with optional name filtering. # Update Credential Source: https://docs.gp.scale.com/reference/v5/credentials/update-credential /openapi/v5.yaml patch /v5/credentials/{credential_id} Update an existing credential's properties including name, description, type, payload, and metadata. # Add items to an existing dataset Source: https://docs.gp.scale.com/reference/v5/dataset-items/add-items-to-an-existing-dataset /openapi/v5.yaml post /v5/dataset-items/batch Add one or more items in a single batch to a dataset that already exists. Use this to append items to an existing dataset; to create a dataset together with its initial items in one call, use POST /v5/datasets instead (this endpoint does not create datasets). Items are only ever added in a batch — there is no single-item create endpoint. The target dataset is identified by the required dataset_id in the request body, and each entry in data (at least one is required) becomes one item, optionally paired with the files entry at the same list index — when files is supplied it must be the same length as data. Each referenced file must exist and be a supported media format, and item keys may not collide with reserved dataset- or evaluation-item field names. The target dataset must not be archived. The batch is applied as a new dataset version snapshot rather than mutating existing versions, and the created items are returned. # Delete a dataset item Source: https://docs.gp.scale.com/reference/v5/dataset-items/delete-a-dataset-item /openapi/v5.yaml delete /v5/dataset-items/{dataset_item_id} Soft-delete a dataset item by marking it as removed in the current dataset version. The database row is not removed; instead the item's end version is set to the current dataset version so it no longer appears at the latest version while remaining retrievable at earlier versions, and a new dataset version snapshot is created to record the removal. An item that is already archived or already deleted cannot be deleted again. The response echoes the item id with deleted set to true. # Get a dataset item Source: https://docs.gp.scale.com/reference/v5/dataset-items/get-a-dataset-item /openapi/v5.yaml get /v5/dataset-items/{dataset_item_id} Retrieve a single dataset item by its identifier. The lookup is scoped to the caller's account. By default the current (latest) version of the item is returned; passing version returns the item as it existed at that dataset version, which allows reading an item that was later updated or soft-deleted. # List dataset items Source: https://docs.gp.scale.com/reference/v5/dataset-items/list-dataset-items /openapi/v5.yaml get /v5/dataset-items List dataset items belonging to the caller's account, paginated. dataset_id is optional: when omitted, items across all of the account's datasets are returned; when provided, results are restricted to that dataset. A specific version may be requested only together with a dataset_id — passing version without dataset_id is rejected. By default only current items at the latest version are listed (soft-deleted items are excluded); archived items are excluded unless include_archived is true. # Update a dataset item Source: https://docs.gp.scale.com/reference/v5/dataset-items/update-a-dataset-item /openapi/v5.yaml patch /v5/dataset-items/{dataset_item_id} Update the data payload and optional file associations of a dataset item. The request replaces the item's data in full and, if provided, its file associations; there is no partial-field merge. Data keys may not collide with reserved dataset- or evaluation-item field names, and any referenced files must exist and be a supported media format. Items that are archived or soft-deleted cannot be updated. The update is applied as a new dataset version rather than overwriting the previous record, so prior versions remain retrievable. # Archive a dataset Source: https://docs.gp.scale.com/reference/v5/datasets/archive-a-dataset /openapi/v5.yaml delete /v5/datasets/{dataset_id} Soft-delete a dataset by archiving it; the row is retained and can be restored later. This sets the dataset's `archived_at` timestamp rather than permanently removing it, and cascades the same archival to the dataset's items and versions. Archiving is not idempotent: if the dataset is already archived the call fails. The dataset can be brought back with the PATCH restore request. The response reports the dataset id with `deleted` set to true. # Create a dataset Source: https://docs.gp.scale.com/reference/v5/datasets/create-a-dataset /openapi/v5.yaml post /v5/datasets Create a dataset and populate it with its initial items in a single call. Use this when the dataset does not exist yet: it creates the dataset and seeds it with the items you provide. To add more items to a dataset that already exists, use POST /v5/dataset-items/batch instead. A name is required, and the request must include at least one item in `data`; the optional `files` field associates files with the created items. The provided items are inserted as dataset items as a side effect of creation, and any `tags` are attached to the dataset. The returned dataset is at version 1. # Get a dataset Source: https://docs.gp.scale.com/reference/v5/datasets/get-a-dataset /openapi/v5.yaml get /v5/datasets/{dataset_id} Retrieve a single dataset by its id. By default an archived dataset is not returned; set `include_archived` to true to fetch a dataset regardless of whether it has been archived. # List datasets Source: https://docs.gp.scale.com/reference/v5/datasets/list-datasets /openapi/v5.yaml get /v5/datasets Return a paginated list of the account's datasets. Archived datasets are excluded unless `include_archived` is set to true. Results can be narrowed by `tags` and by the standard filterable parameters; a `name` filter matches as a case-insensitive substring rather than an exact match. # Update or restore a dataset Source: https://docs.gp.scale.com/reference/v5/datasets/update-or-restore-a-dataset /openapi/v5.yaml patch /v5/datasets/{dataset_id} Update a dataset's fields, or restore a previously archived dataset. The request body is a union discriminated by its contents: a body of `{"restore": true}` triggers a restore, and any other body is treated as a partial update of the dataset's `name`, `description`, and `tags`. Restore clears `archived_at` on the dataset and cascades the un-archival to its items and versions; restoring a dataset that is not archived returns it unchanged. A partial update on an archived dataset fails, since archived datasets cannot be modified. # Create Evaluation Chart Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/create-evaluation-chart /openapi/v5.yaml post /v5/evaluation-charts Deprecated: evaluation group dashboards are the preferred way to construct charts. Create a new evaluation chart for an evaluation or evaluation group. # Delete Evaluation Chart Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/delete-evaluation-chart /openapi/v5.yaml delete /v5/evaluation-charts/{chart_id} Deprecated: evaluation group dashboards are the preferred way to construct charts. Delete an evaluation chart and its columns. # Get Evaluation Chart Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/get-evaluation-chart /openapi/v5.yaml get /v5/evaluation-charts/{chart_id} Deprecated: evaluation group dashboards are the preferred way to construct charts. Get a single evaluation chart by ID. # List Evaluation Charts Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/list-evaluation-charts /openapi/v5.yaml get /v5/evaluation-charts Deprecated: evaluation group dashboards are the preferred way to construct charts. List charts filtered by evaluation_id or evaluation_group_id. # Patch Evaluation Chart Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/patch-evaluation-chart /openapi/v5.yaml patch /v5/evaluation-charts/{chart_id} Deprecated: evaluation group dashboards are the preferred way to construct charts. Partially update chart metadata (title, type). Columns cannot be modified via PATCH. # Replace Evaluation Chart Source: https://docs.gp.scale.com/reference/v5/evaluation-charts/replace-evaluation-chart /openapi/v5.yaml put /v5/evaluation-charts/{chart_id} Deprecated: evaluation group dashboards are the preferred way to construct charts. Replace chart metadata and/or columns. Columns are replaced if provided. # Add Widget to Dashboard Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/add-widget-to-dashboard /openapi/v5.yaml post /v5/evaluation-dashboards/{dashboard_id}/widgets Create a new widget, append it to the dashboard, and compute its result in one call. The widget is persisted as its own entity, its ID is appended to the dashboard's `widget_order`, and — unless it is a `markdown` or `heading` widget — its result is computed synchronously against the dashboard's evaluation (or evaluation-group) data before the response returns. Validation is type-specific: `markdown` requires `config.content`, `bar`/`histogram`/`donut`/`scatter` require exactly one of `config.x_column` or `config.x_column_group`, `table` configs with conditional formatting are schema-validated, and all other chart types require a `query`. The dashboard must exist and not be archived. A computation failure does not fail the request: the widget is still created and returned with a result whose `computation_status` is `failed` and an `error_message` set, while `markdown` and `heading` widgets return a null result. # Create Evaluation Dashboard Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/create-evaluation-dashboard /openapi/v5.yaml post /v5/evaluation-dashboards Create a dashboard bound to exactly one evaluation or evaluation group. The request must set exactly one of `evaluation_id` or `evaluation_group_id` (enforced as XOR) — supplying both or neither is rejected, and the referenced evaluation or group must already exist in the caller's account or the call fails with a not-found error. If `template_dashboard_id` is provided, the new dashboard copies that template's `widget_order` and synchronously computes a result for each of those widgets against the new dashboard's evaluation data before returning. Any `widget_order` supplied is validated to contain only existing, non-duplicate widget IDs. The dashboard is created and returned immediately; individual widgets are added afterward through the widget sub-endpoints. # Delete Evaluation Dashboard Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/delete-evaluation-dashboard /openapi/v5.yaml delete /v5/evaluation-dashboards/{dashboard_id} Soft-delete a dashboard by setting its archived timestamp. The dashboard row is retained and marked archived rather than physically removed, so it stops appearing in default listings but can still be fetched with `include_archived=true`; the archived dashboard is returned by this call. Associated widgets and widget results are not deleted or detached. Returns a not-found error if the dashboard does not exist in the caller's account. # Get Evaluation Dashboard Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/get-evaluation-dashboard /openapi/v5.yaml get /v5/evaluation-dashboards/{dashboard_id} Fetch a single dashboard by ID within the caller's account. By default only the dashboard's own fields are returned. Pass `views=widgets` and/or `views=widget_results` to eagerly load the dashboard's widgets and their last-computed results as nested relationships; without those views the `widgets` and `widget_results` fields are omitted from the response entirely. Set `include_archived=true` to retrieve a soft-deleted dashboard. Returns a not-found error if no matching dashboard exists in the account. # List Evaluation Dashboards Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/list-evaluation-dashboards /openapi/v5.yaml get /v5/evaluation-dashboards List dashboards in the caller's account, paginated, with optional filters. Filter to a single evaluation with `evaluation_id` or to a group with `evaluation_group_id`; passing both is rejected, since a dashboard is bound to one or the other. `tags` matches case-insensitively (values are lowercased before lookup), `created_by_ids` filters by creator identity, and `search` matches the dashboard name and tags. Archived dashboards are excluded unless `include_archived` is true. The returned items carry only the dashboards' own fields — widgets and widget results are never embedded here; use the get-by-id endpoint with the `widgets`/`widget_results` views to load those. # Patch Evaluation Dashboard Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/patch-evaluation-dashboard /openapi/v5.yaml patch /v5/evaluation-dashboards/{dashboard_id} Partially update a dashboard's `name`, `description`, `tags`, or `widget_order`. Only these mutable fields can change — the dashboard's bound evaluation or group is fixed at creation and cannot be reassigned here, and unset fields are left untouched (partial-update semantics). Reorder existing widgets by sending a new `widget_order`, whose entries are validated to be existing, non-duplicate widget IDs before the update is applied; creating and attaching new widgets is done through the widget sub-endpoints, not by editing `widget_order`. Archived dashboards cannot be updated and return a not-found error. # Remove Widget from Dashboard Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/remove-widget-from-dashboard /openapi/v5.yaml delete /v5/evaluation-dashboards/{dashboard_id}/widgets/{widget_id} Detach a widget from this dashboard without deleting the widget itself. This removes the widget's ID from the dashboard's `widget_order` only; the underlying widget entity and any computed widget results are left intact, so a widget shared with other dashboards continues to work there. The widget must currently be in this dashboard's `widget_order`, otherwise a not-found error is returned. Responds with 204 No Content on success. # Update Dashboard Widget Source: https://docs.gp.scale.com/reference/v5/evaluation-dashboards/update-dashboard-widget /openapi/v5.yaml patch /v5/evaluation-dashboards/{dashboard_id}/widgets/{widget_id} Update a widget's fields within this dashboard, using copy-on-write for shared widgets. If the widget belongs only to this dashboard it is updated in place; if it is referenced by more than one dashboard, a new widget is created with the updates applied and swapped into this dashboard's `widget_order`, leaving the other dashboards' copy untouched. The widget must already be in this dashboard's `widget_order`, otherwise the call is rejected. The result is recomputed synchronously when the `query` changes or when the widget was cloned; otherwise the existing cached result is returned. For `table` widgets, conditional-formatting column references are re-resolved against the (possibly updated) query on each save. The dashboard must exist and not be archived. # Archive Evaluation Group Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/archive-evaluation-group /openapi/v5.yaml delete /v5/evaluation-groups/{group_id} Archive (soft-delete) an evaluation group and its dependent records. This is a soft delete: the group is marked deleted rather than removed, so it no longer appears in default list/get results but can still be fetched with the include-deleted flags. The same delete timestamp cascades as a soft delete to the group's members, row identifiers, and its dashboard charts and their chart columns; the member evaluations themselves are not affected. The archived group is returned in the response. # Create Evaluation Group Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/create-evaluation-group /openapi/v5.yaml post /v5/evaluation-groups Create a new evaluation group bundling one or more existing evaluations. The request must list at least one evaluation ID, and duplicate IDs are rejected. Every listed evaluation is validated for existence and account access before the group is created; an unknown or inaccessible evaluation ID fails the whole request with a 400 rather than creating a partial group. The named evaluations are added as group members in the same transaction, and any supplied row_identifiers (an evaluation_id-to-column-name mapping used for cross-dataset joins) are persisted alongside them, ignoring entries for evaluation IDs not in the group. The returned group includes its members enriched with each evaluation's name, tags, and creation time. Unlike the PUT and PATCH update endpoints, creation does not trigger any dashboard-widget recomputation. # Get Evaluation Group Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/get-evaluation-group /openapi/v5.yaml get /v5/evaluation-groups/{group_id} Fetch a single evaluation group by its ID. By default only non-archived groups are returned; set include_deleted to also resolve a soft-deleted group, whose response includes its soft-deleted member rows. `members` is always returned, so a client can confirm membership without passing a view, and `views` adds further expansions on top (e.g. views=row_identifiers). The full member set is returned with no cap, and each member is enriched with its evaluation's name, tags, and creation time where that evaluation is readable from this account; those fields are null otherwise. Use the schema endpoint instead when you need the per-evaluation column schemas of the group's members. # Get Evaluation Group Schema Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/get-evaluation-group-schema /openapi/v5.yaml get /v5/evaluation-groups/{group_id}/schema Return a separate column schema for each active member evaluation of the group. Rather than a single merged schema, the response holds one schema entry per active member evaluation (each with its field list, total item count, and sampling info), which lets a caller filter columns down to a chosen subset of the group's evaluations. Schemas are computed from the member evaluations' items; include_archived controls whether archived items are counted in that analysis. A group with no active members returns an empty schema list. This differs from the plain get endpoint, which returns group metadata and members but not their column schemas. # List Evaluation Groups Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/list-evaluation-groups /openapi/v5.yaml get /v5/evaluation-groups List the calling account's evaluation groups with optional filters. Results are paginated and scoped to the authenticated account. The optional filters combine with AND semantics: name does a case-insensitive partial (substring) match on the group name, tags returns only groups whose tag list contains all of the supplied tags, and evaluation_id returns only groups that have the given evaluation as an active member. By default archived (soft-deleted) groups are excluded; set include_deleted to include them. The views parameter optionally expands each group with its members and/or row_identifiers, and returned members are enriched with each evaluation's name, tags, and creation time. # Patch Evaluation Group Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/patch-evaluation-group /openapi/v5.yaml patch /v5/evaluation-groups/{group_id} Partially update an evaluation group's attributes. Only the fields present in the request are changed; unset fields are left untouched. Unlike the PUT endpoint, metadata is merged into the existing metadata key-by-key rather than replaced wholesale, and group membership (evaluation_ids) cannot be changed here -- use PUT to replace members. At least one field must be supplied or the request fails with a 400. Supplying row_identifiers replaces the group's entire row identifier set (an evaluation_id-to-column-name mapping for cross-dataset joins); passing an empty mapping clears it. # Update Evaluation Group Source: https://docs.gp.scale.com/reference/v5/evaluation-groups/update-evaluation-group /openapi/v5.yaml put /v5/evaluation-groups/{group_id} Replace an evaluation group's attributes and, optionally, its membership. This is a full replacement of the group's attributes: every attribute field (name, description, tags, metadata) is written from the request, so omitting an optional field clears it rather than leaving it unchanged. Use PATCH instead to update only selected fields and to merge (rather than overwrite) metadata. Membership is replaced only when evaluation_ids is supplied: the resulting members become exactly that set (added/removed by diff against the current members), it must contain at least one ID with no duplicates, and every ID is validated for existence and account access or the request fails with a 400. When membership actually changes, a best-effort Temporal workflow is started to recompute the group's dashboard widgets; this is fire-and-forget and its failure does not fail the request. Removing members hard-deletes the corresponding chart columns and any charts left with no columns. # Export evaluation items to a file Source: https://docs.gp.scale.com/reference/v5/evaluation-items/export-evaluation-items-to-a-file /openapi/v5.yaml post /v5/evaluation-items/export Export all evaluation items for a single evaluation as a downloadable file. The evaluation is specified by `evaluation_id` in the request body. `export_format` selects CSV, JSON, or JSONL; for CSV the per-item `data` and `files` fields are flattened into individual columns and metric-like result columns are expanded. `export_method` controls delivery: `direct` returns the file contents inline in the response, while `signed_url` uploads the file to object storage and returns a pre-signed download URL. Requesting `signed_url` in an environment where object storage is not configured fails with a 501, so use `direct` there instead. Set `include_archived=true` to include archived items in the export. This endpoint reads items only and does not modify the evaluation. # Get a single evaluation item Source: https://docs.gp.scale.com/reference/v5/evaluation-items/get-a-single-evaluation-item /openapi/v5.yaml get /v5/evaluation-items/{evaluation_item_id} Retrieve a single evaluation item by its ID within the caller's account. By default only non-archived items are returned; pass `include_archived=true` to also retrieve an item that has been archived. The response merges the item's cached task results into its `data` field and exposes a `task_errors` map keyed by task alias, so a task that failed on this item surfaces as an entry there rather than as a request error. Use this to inspect one item's input data and per-task results; to page through many items, use the list endpoint instead. The request fails if no item with the given ID exists in the caller's account. # List and filter evaluation items Source: https://docs.gp.scale.com/reference/v5/evaluation-items/list-and-filter-evaluation-items /openapi/v5.yaml get /v5/evaluation-items Return a paginated list of evaluation items belonging to the caller's account. Pass `evaluation_id` to restrict the results to a single evaluation's items. The `completion_status` filter selects items by whether any of their tasks errored: `failed` returns only items that have task errors, `passed` returns only items with no task errors, and `all` (or omitting the parameter) returns every item. Archived items are excluded unless `include_archived=true`. Each returned item has its cached task results merged into `data` and its errors exposed in `task_errors`, identically to the single-item endpoint. # Add Test Criteria to Evaluation Source: https://docs.gp.scale.com/reference/v5/evaluations/add-test-criteria-to-evaluation /openapi/v5.yaml post /v5/evaluations/{evaluation_id}/tasks Add a new test criteria to an existing evaluation. Narrowed to contributor question tasks (`contributor_evaluation.question`); other task types must be configured when the evaluation is first created and are rejected here. The request is also rejected if the evaluation is archived, if a test criteria with the same alias already exists, or if any contributor annotation task for the evaluation has already been claimed or completed. Because only contributor question tasks are accepted, the added criteria is applied synchronously and contributors answer it against the evaluation's existing items — no async job or Temporal workflow is started. # Archive Evaluation Source: https://docs.gp.scale.com/reference/v5/evaluations/archive-evaluation /openapi/v5.yaml delete /v5/evaluations/{evaluation_id} Archive (soft-delete) an evaluation. Sets the evaluation's archived timestamp rather than permanently deleting it, and cascades the archive to the evaluation's items and dashboards while removing it from any evaluation groups. The evaluation can later be brought back with a restore request to the update endpoint. # Create Evaluation Source: https://docs.gp.scale.com/reference/v5/evaluations/create-evaluation /openapi/v5.yaml post /v5/evaluations Create an evaluation together with its items, optionally running test criteria against them. Accepts three request shapes: standalone (inline `data`), from an existing dataset (`dataset_id` with optional per-item references), or with a new reusable dataset created inline from `data`. When the evaluation includes tasks that require execution (for example an LLM judge or custom function), an async job and a Temporal workflow are started and the evaluation is returned immediately with status `running`; task results and `error_count` populate asynchronously. When it includes only contributor tasks, taxonomy-only input, or no tasks, no workflow runs and it is returned with status `completed`. Optional `tasks`, `metadata`, `tags`, and `taxonomy_params` are persisted alongside the evaluation and its items. # Dry-run a custom-function evaluation task Source: https://docs.gp.scale.com/reference/v5/evaluations/dry-run-a-custom-function-evaluation-task /openapi/v5.yaml post /v5/evaluations/tasks/dry-run Dry-run a custom-function evaluation task against sample rows to validate it before creating an evaluation. Runs the task's Python scoring function over each row in ``sample_data`` (1-10 rows) and returns a per-row result — ``success`` with the returned ``result`` or an ``error`` — plus the resolved function name and parameters, so a broken function or misconfigured task is caught before it is used in a real evaluation. Nothing is persisted and no evaluation is created; results are returned inline. Only the ``custom_function`` task type is supported; any other ``task_type`` returns 400. The ``configuration`` must be a valid custom-function config: a Python ``function_source`` (up to 10,000 characters), an optional ``arg_mapping`` of function parameters to ``item.`` locators (auto-derived from the function signature when omitted), and optional literal ``config_args``. An invalid configuration or a function that fails to load returns 400; exceeding the dry-run's total time budget returns 408 with whatever rows completed. # Filter Evaluations Source: https://docs.gp.scale.com/reference/v5/evaluations/filter-evaluations /openapi/v5.yaml post /v5/evaluations/filter Filter evaluations by metadata, status, and tags. Accepts up to 10 filters combined with AND logic, each comparing a key against a value with an operator (`==`, `!=`, `>=`, `<=`, `IN`, `NOT_IN`). Filter on metadata keys returned by the metadata-keys endpoint, plus the built-in `status` and `tag` keys. Archived evaluations are excluded unless `include_archived` is set, and the `tasks` view includes task configurations in each result. Use this for metadata or status filtering; for simple name or tag lookups the list endpoint is sufficient. # Get Available Evaluation Statuses Source: https://docs.gp.scale.com/reference/v5/evaluations/get-available-evaluation-statuses /openapi/v5.yaml get /v5/evaluations/statuses List the possible evaluation status values. Returns every value of the evaluation status enumeration, suitable for populating a status filter in clients. # Get Available Metadata Keys Source: https://docs.gp.scale.com/reference/v5/evaluations/get-available-metadata-keys /openapi/v5.yaml get /v5/evaluations/metadata-keys List the metadata keys available for filtering evaluations. Returns the distinct metadata keys present on the account's evaluations, plus the built-in `status` and `tag` keys, sorted alphabetically. Use the returned keys to build requests to the evaluations filter endpoint. # Get Evaluation Source: https://docs.gp.scale.com/reference/v5/evaluations/get-evaluation /openapi/v5.yaml get /v5/evaluations/{evaluation_id} Retrieve a single evaluation by ID. Returns the evaluation with its datasets, async-job progress, metadata, and task-error count. Archived evaluations are excluded unless `include_archived` is set. Pass the `tasks` view to include the evaluation's task configurations in the response. # Get Evaluation Data Schema Source: https://docs.gp.scale.com/reference/v5/evaluations/get-evaluation-data-schema /openapi/v5.yaml get /v5/evaluations/{evaluation_id}/schema Describe the data schema of an evaluation's items. Inspects the item `data` and task-result fields and returns each discovered field with its flattened key path, JSON type, source, and the number of items containing it, ordered alphabetically by field name. For large evaluations the schema may be inferred from a sample of items, in which case `is_sampled` is set and `sample_size` reports how many were analyzed. Set `include_archived` to include archived items in the analysis. # Get Evaluation Taxonomy Source: https://docs.gp.scale.com/reference/v5/evaluations/get-evaluation-taxonomy /openapi/v5.yaml get /v5/evaluations/{evaluation_id}/taxonomy Get the taxonomy JSON for an evaluation's contributor question tasks. Returns the raw taxonomy document stored for the evaluation. Responds with a not-found error if the evaluation has no taxonomy. # List Evaluations Source: https://docs.gp.scale.com/reference/v5/evaluations/list-evaluations /openapi/v5.yaml get /v5/evaluations List evaluations for the account, with pagination. Supports filtering by case-insensitive name substring and by tags; archived evaluations are excluded unless `include_archived` is set. Pass the `tasks` view to include each evaluation's task configurations in the response. Use this for simple name or tag lookups; to filter on metadata key-value pairs or status, use the filter endpoint instead. # Update or Restore Evaluation Source: https://docs.gp.scale.com/reference/v5/evaluations/update-or-restore-evaluation /openapi/v5.yaml patch /v5/evaluations/{evaluation_id} Update an evaluation's mutable fields, or restore it from the archive. The action is selected by the request body: a restore request un-archives the evaluation and cascades the restore to its items and dashboards, while any other body applies a partial update to fields such as name, description, tags, and metadata (metadata is applied as an RFC 7396 merge patch). Updating an already-archived evaluation is rejected — restore it first. The evaluation row is locked for the duration of the write to avoid concurrent-update races. # Update Test Criteria Configuration Source: https://docs.gp.scale.com/reference/v5/evaluations/update-test-criteria-configuration /openapi/v5.yaml patch /v5/evaluations/{evaluation_id}/tasks/{alias} Replace the full configuration of a single test criteria, identified by its alias. The alias must match an existing test criteria on the evaluation, and the replacement configuration is validated against the evaluation's current items before being applied. The request is rejected if the evaluation is archived, if no test criteria matches the alias, or if any contributor annotation task for the evaluation has already been claimed or completed — at that point labelers are in-flight and mutating the task definition would corrupt their work. # Delete a file Source: https://docs.gp.scale.com/reference/v5/files/delete-a-file /openapi/v5.yaml delete /v5/files/{file_id} Delete a file by id, removing its database record. This is a hard delete: the file's row is removed from the database, not soft-archived. The underlying stored object is deleted only when no other file record still references the same stored content (identical checksum and MIME type); when another record shares the blob, the blob is retained. If the id refers to an incomplete multipart upload placeholder, its staged parts are aborted instead. Returns a confirmation carrying the deleted file's id. # Download file content Source: https://docs.gp.scale.com/reference/v5/files/download-file-content /openapi/v5.yaml get /v5/files/{file_id}/content Download the raw bytes of a file by id. Streams the stored object's content back directly (not a redirect or signed URL), with the response Content-Type set to the file's stored MIME type and Content-Disposition set to attachment. Use the metadata endpoint for size, filename, and other attributes. # Get file metadata Source: https://docs.gp.scale.com/reference/v5/files/get-file-metadata /openapi/v5.yaml get /v5/files/{file_id} Retrieve a single file's metadata by id. Returns the file record (id, filename, MIME type, size, tags, and related fields) but not the file's bytes; use the content endpoint to download the bytes. # Import files from cloud storage Source: https://docs.gp.scale.com/reference/v5/files/import-files-from-cloud-storage /openapi/v5.yaml post /v5/files/cloud_imports Register files that already exist in cloud blob storage as file records, in one batch. Unlike upload, this transfers no bytes: each entry references an existing object by its container/bucket, filepath, filename, and MIME type, and only a metadata record pointing at that object is created. Files are imported independently and the response reports a per-file status. The response is 200 when every import succeeds and 207 when results are mixed, with each result marked SUCCESS or a failure reason (file does not exist, invalid permissions, or unknown error). Failed entries carry only the submitted filename and MIME type rather than a full file record. Only containers (or key prefixes within them) that the deployment's operator has allowed for the caller's account can be imported; anything else fails with invalid permissions. # List files Source: https://docs.gp.scale.com/reference/v5/files/list-files /openapi/v5.yaml get /v5/files List the account's files with pagination. Optionally filter by ``filename`` (case-insensitive partial match). Results are scoped to the files the caller is authorized to read. Files marked hidden (internal or service-owned artifacts) are excluded from this listing but remain retrievable by id. Returns file metadata only, not content. # Update a file Source: https://docs.gp.scale.com/reference/v5/files/update-a-file /openapi/v5.yaml patch /v5/files/{file_id} Update a file's mutable metadata by id. Only the file's ``tags`` can be modified; filename, content, size, and other attributes are immutable through this endpoint. The supplied tags replace the file's existing tags. Returns the updated file metadata. # Upload file bytes directly Source: https://docs.gp.scale.com/reference/v5/files/upload-file-bytes-directly /openapi/v5.yaml post /v5/files Upload a file's bytes directly and create a file record. Send the file as multipart/form-data in the ``file`` field; the request body carries the raw bytes, unlike cloud_imports which only references blobs that already exist in cloud storage. The upload is rejected if it exceeds 25 MB; larger or direct-to-storage uploads use a separate upload flow. The server detects the content type from the bytes and rejects content that fails the structural validation for that type. Content is deduplicated by checksum and MIME type, so uploading bytes identical to an existing file reuses the already-stored object instead of storing a second copy. Returns the created file's metadata. # Create a custom model Source: https://docs.gp.scale.com/reference/v5/models/create-a-custom-model /openapi/v5.yaml post /v5/models Create a custom model record in your account and begin deploying it through a supported serving vendor. A model here is a record for a model you deploy and serve through Scale's own inference vendors: only the `launch` and `llmengine` vendors are accepted and any other vendor is rejected. This is distinct from `GET /v5/chat/completions/models`, which lists the models already available to call for chat completions rather than creating or managing these records. The call is asynchronous — the record is created in a deploying status, a deployment job is recorded, and a Temporal workflow is started to perform the deployment, so the model is not ready for inference when this returns. A model name must be unique per vendor within your account; if a model with the same name and vendor already exists the request fails unless `on_conflict` is set to `update`, in which case the existing model is updated instead. # Delete a custom model Source: https://docs.gp.scale.com/reference/v5/models/delete-a-custom-model /openapi/v5.yaml delete /v5/models/{model_id} Permanently delete a custom model record and tear down its deployment at the serving vendor. This is a hard delete: the model row is removed from your account entirely and cannot be restored afterward. Before the record is removed, if the model has an associated vendor deployment that deployment is torn down at its serving vendor. A model that is currently deploying cannot be deleted and the request fails until deployment finishes. This operates on the model records managed by this API, distinct from the `GET /v5/chat/completions/models` catalog of models callable for chat completions. # Get a custom model Source: https://docs.gp.scale.com/reference/v5/models/get-a-custom-model /openapi/v5.yaml get /v5/models/{model_id} Retrieve a single custom model record by its ID. Returns the model record — including its vendor, configuration, and current deployment status — for a model managed through this API and owned by the caller's account. This is distinct from `GET /v5/chat/completions/models`, which lists the models available to call for chat completions rather than returning a single managed record. # List custom models Source: https://docs.gp.scale.com/reference/v5/models/list-custom-models /openapi/v5.yaml get /v5/models List the custom model records registered in your account. Returns a paginated list of the model records managed through this API — models your account deploys through the `launch` or `llmengine` serving vendors — optionally filtered by name and by model vendor, and scoped to the caller's account. This is different from `GET /v5/chat/completions/models`, which lists the models available to invoke for chat completions; this endpoint returns the managed records along with their deployment status, not the catalog of callable completion models. # Update a custom model Source: https://docs.gp.scale.com/reference/v5/models/update-a-custom-model /openapi/v5.yaml patch /v5/models/{model_id} Update a custom model record; vendor-configuration changes are applied asynchronously by redeploying the model. This supports three kinds of update: changing model metadata only, renaming the model, and changing the vendor configuration. A vendor-configuration change is asynchronous — it puts the model back into a deploying status, records an update job, and starts a Temporal workflow to redeploy, so the new configuration is not live when this returns; metadata-only and rename changes take effect immediately. The vendor configuration supplied must match the model's own vendor (`launch` or `llmengine`), and only those two vendors are supported. A model that is currently deploying cannot be modified and the request fails until deployment finishes. When renaming with `on_conflict` set to `swap`, the name is exchanged with an existing model of the same name and vendor instead of failing on the uniqueness constraint. # Create span assessment Source: https://docs.gp.scale.com/reference/v5/span-assessment/create-span-assessment /openapi/v5.yaml post /v5/span-assessments Attach a new assessment to a span within a trace. A span assessment records feedback on a single span's output. Its assessment_type selects one of comment, rating (an integer 1-5), approval (approved or rejected), rubric (key-value rule pairs), metadata (arbitrary JSON), or overwrite (a corrected span output), and exactly the content field matching that type must be supplied; a free-text comment may additionally accompany any type. trace_id is required, while span_id is optional and, when omitted, the assessment is attached to the trace's root span. The call returns 404 if the given span_id and trace_id do not identify an existing span, or if no root span is found for the trace. Creating an overwrite for a span that already holds a live overwrite returns 409, whether that overwrite is yours or another user's. Update or delete the existing one instead. Use the list endpoint to read a span's or trace's existing assessments. # Get span assessment by ID Source: https://docs.gp.scale.com/reference/v5/span-assessment/get-span-assessment-by-id /openapi/v5.yaml get /v5/span-assessments/{span_assessment_id} Retrieve a single span assessment by its identifier. Returns the assessment's type and content fields, the span and trace it is attached to, and the identity that created it. Returns 404 if no assessment with that id exists for the caller's account. Use the list endpoint instead when you have a span or trace id rather than an assessment id. ``x-project-id`` answers 404 for an assessment on a trace outside that project, when ``PROJECT_SCOPED_SPAN_LISTING`` is on for the account. # List span or trace assessments Source: https://docs.gp.scale.com/reference/v5/span-assessment/list-span-or-trace-assessments /openapi/v5.yaml get /v5/span-assessments Return the assessments attached to a given span or trace. Results are scoped to the caller's account. At least one of span_id or trace_id must be supplied as a query parameter, and a request providing neither returns 400. Filtering by trace_id returns assessments across every span of that trace, whereas span_id returns only that span's assessments, and supplying both restricts the result to that span within that trace. An optional assessment_type narrows the results to a single type. Use the get-by-id endpoint when you already have a specific assessment id. ``x-project-id`` narrows the result to assessments on traces whose root span carries that project, when ``PROJECT_SCOPED_SPAN_LISTING`` is on for the account. # Permanently delete span assessment Source: https://docs.gp.scale.com/reference/v5/span-assessment/permanently-delete-span-assessment /openapi/v5.yaml delete /v5/span-assessments/{span_assessment_id} Permanently delete a span assessment by its identifier. Only the assessment's original creator, or an account admin or manager, may delete it; a request from any other identity returns 403. This is a hard delete: the assessment row is removed rather than archived, so it cannot be restored afterward. The response echoes the deleted assessment's id. Returns 404 if no assessment with that id exists for the caller's account. # Update span assessment Source: https://docs.gp.scale.com/reference/v5/span-assessment/update-span-assessment /openapi/v5.yaml patch /v5/span-assessments/{span_assessment_id} Update the content of an existing span assessment. Only the assessment's original creator, or an account admin or manager, may update it; a request from any other identity returns 403. Supplied fields overwrite the stored values and the merged result is re-validated against the assessment's type, so the content must stay consistent with assessment_type (the matching content field present and no conflicting fields set) or the call returns 422. The span and trace an assessment is attached to cannot be changed through this endpoint. Returns 404 if no assessment with that id exists for the caller's account. # Create a span Source: https://docs.gp.scale.com/reference/v5/spans/create-a-span /openapi/v5.yaml post /v5/spans Create a single span and return the persisted span. Use this for one-off span ingestion; to write many spans in one request use POST /v5/spans/batch. When ``id`` is omitted the server generates a UUID. Depending on per-account server configuration the span is persisted to the legacy trace store, to the tracing service, or both. ``end_timestamp`` must not precede ``start_timestamp``, which is rejected with a 422. When the tracing service is the primary store, its 400, 413 and 422 rejections keep that status and detail, as does a 403 when the request carried its own API key, and every other failure returns a 503 with a Retry-After header. A span whose parent belongs to another trace is rejected with 409 before either store is written. The parent is looked up in Postgres, so that check ends once the account's spans stop being written there. Which writes the tracing service rejects depends in part on its storage engine: on Postgres deployments a NUL byte or invalid UTF-8 inside ``trace_id``, ``id``, ``parent_id`` or ``group_id`` rejects the span with a 400 naming the field, because replacing the byte would change the identity the response echoes; a NUL byte or invalid UTF-8 in any other field is replaced with U+FFFD and the span is persisted. ClickHouse deployments store the bytes verbatim. Credential redaction: values in the free-form ``input``, ``output``, ``metadata``, and ``expected`` objects, and in ``name``, that are credential-shaped (bearer/JWT, API keys, connection-string passwords) or under a credential-named key are replaced with ``[REDACTED:credential]`` before the span is persisted, so the stored and returned span reflects the redacted value. # Create spans in a batch Source: https://docs.gp.scale.com/reference/v5/spans/create-spans-in-a-batch /openapi/v5.yaml post /v5/spans/batch Create multiple spans (up to 1000) in a single request and return the created spans. Prefer this over repeated POST /v5/spans calls when ingesting many spans at once; use PUT /v5/spans/batch instead when a span with the same ``id`` may already exist, since this endpoint inserts new spans rather than overwriting. A batch larger than 1000 spans is rejected with a validation error. Each item follows the same id-generation and per-account dual-write rules as the single-span create, including that ``end_timestamp`` must not precede ``start_timestamp``, which rejects the request with a 422. When the tracing service is the primary store, its 400, 413 and 422 rejections keep that status and detail, as does a 403 when the request carried its own API key, and every other failure returns a 503 with a Retry-After header. A batch that forms a cycle, or a span whose parent belongs to another trace, is rejected with 409 before either store is written. A parent outside the batch is looked up in Postgres, so that part of the check ends once the account's spans stop being written there. Postgres rejects an ``id`` that already exists, while the tracing service treats the write as an upsert. Which writes the tracing service rejects depends in part on its storage engine: on Postgres deployments a NUL byte or invalid UTF-8 inside ``trace_id``, ``id``, ``parent_id`` or ``group_id`` fails the whole batch with a 400 naming the field, because replacing the byte would change the identity the response echoes; a NUL byte or invalid UTF-8 in any other field is replaced with U+FFFD and the span is persisted. ClickHouse deployments store the bytes verbatim. Credential redaction: values in the free-form ``input``, ``output``, ``metadata``, and ``expected`` objects, and in ``name``, that are credential-shaped (bearer/JWT, API keys, connection-string passwords) or under a credential-named key are replaced with ``[REDACTED:credential]`` before the span is persisted, so the stored and returned span reflects the redacted value. # Create trace deletion request Source: https://docs.gp.scale.com/reference/v5/spans/create-trace-deletion-request /openapi/v5.yaml post /v5/spans/deletions # Export spans Source: https://docs.gp.scale.com/reference/v5/spans/export-spans /openapi/v5.yaml post /v5/spans/export Export spans matching the given filters, either synchronously or as an async job. For accounts on the tracing service the export runs as a background job: the endpoint returns 202 with an ``export_id`` to poll via GET /v5/spans/export/{export_id}, and only ``signed_url`` output is supported (a ``direct`` request is rejected with 422, and a request with no authorized traces returns 403). For accounts still on the legacy trace store the export is synchronous and returns 200 with either the raw content (``direct``) or a pre-signed URL (``signed_url``); a ``signed_url`` request in an environment without object storage returns 501, and requests filtering on 5000 or more trace ids are rejected with 422. A reversed time window (``from_ts`` after ``to_ts``) is rejected with 422 on either path. Set ``with_children`` to expand matched root spans to their full trace and ``include_assessments`` to attach assessments inline. ``x-project-id`` narrows the result to traces whose root span carries that project, when ``PROJECT_SCOPED_SPAN_LISTING`` is on for the account, on either store. # Get a span Source: https://docs.gp.scale.com/reference/v5/spans/get-a-span /openapi/v5.yaml get /v5/spans/{span_id} Retrieve a single span by its id. The span is read from the legacy trace store or, for accounts migrated to the tracing service, from that service — with automatic fallback to the legacy trace store on error unless the account is in strict mode, where a tracing-service failure surfaces as a 503. Access is authorized against the span's parent trace, so a span in a trace the caller cannot read is rejected; an unknown id returns 404. ``input_tokens`` and ``output_tokens`` carry the token usage the span's producer reported at ingest, and are absent both for a span that reported none and for every span served from the legacy trace store, which keeps no token counts. # Get a span within a trace Source: https://docs.gp.scale.com/reference/v5/spans/get-a-span-within-a-trace /openapi/v5.yaml get /v5/traces/{trace_id}/spans/{span_id} Retrieve the full span payload using its complete trace-scoped identity. Use this endpoint when both identifiers are known; GET /v5/spans/{span_id} remains available for callers that have only a span id. Reads use the account's configured source, the legacy trace store or the tracing service, with fallback to the legacy trace store outside strict mode. Access is authorized against the trace, and an unknown span or a span belonging to another trace returns 404. ``input_tokens`` and ``output_tokens`` carry the token usage the span's producer reported at ingest, and are absent both for a span that reported none and for every span served from the legacy trace store, which keeps no token counts. # Get span export status Source: https://docs.gp.scale.com/reference/v5/spans/get-span-export-status /openapi/v5.yaml get /v5/spans/export/{export_id} Retrieve the status, and when complete the result location, of an async span export. Poll this with the ``export_id`` returned by POST /v5/spans/export. Only exports that ran as async jobs on the tracing service have a pollable status; synchronous exports from the legacy trace store return their result directly and have no export id to poll here. The lookup is proxied to the tracing service and scoped to the caller's account; a tracing-service failure surfaces as a 503. # Get span metrics grouped by name Source: https://docs.gp.scale.com/reference/v5/spans/get-span-metrics-grouped-by-name /openapi/v5.yaml post /v5/spans/metrics/by-span Compute duration metrics aggregated by span name across the filtered set. Returns one row per distinct span name plus overall totals (unique traces and total spans considered). Each row includes occurrence count, min/avg/max and p50/p90/p95/p99 durations, summed duration, share of total time, and average occurrences per trace — use this to profile which span names dominate latency. Results can be sorted (``sort_by``, default ``total_ms``) and paginated with ``limit``/``offset``, and are narrowed to traces the caller is authorized to read. A reversed time window (``from_ts`` after ``to_ts``) is rejected with 422. When the tracing service serves the read, percentiles are exact on Postgres deployments and approximate estimates on ClickHouse deployments; durations are derived from microsecond timestamps on Postgres, so a value can differ by one millisecond at a boundary. On Postgres deployments the exact percentiles sort the matching rows in memory: a by-span request, or a metrics request whose ``bucket_minutes`` spans the whole window, over a window larger than the deployment's work memory and temp-file bound is refused rather than degraded, with 400 ``QUERY_TEMP_FILE_LIMIT_EXCEEDED`` and the tracing service's detail, with or without ``search_texts`` and on either read source, never as a fallback to the legacy Postgres store. Narrow the time window or filters; ClickHouse deployments never return that code. ``x-project-id`` narrows the result to traces whose root span carries that project, when ``PROJECT_SCOPED_SPAN_LISTING`` is on for the account, on either store. # Get time-bucketed span metrics Source: https://docs.gp.scale.com/reference/v5/spans/get-time-bucketed-span-metrics /openapi/v5.yaml post /v5/spans/metrics Compute time-bucketed span metrics over the requested window and filters. ``metrics`` optionally selects count, errors, or latency (p50/p75/p95); omitting it preserves the all-metrics response. ``bucket_minutes`` sets the requested bucket size, but the effective size is enlarged automatically so the response never exceeds 100 buckets (the value actually used is returned in ``bucket_minutes``). Both ``from_ts`` and ``to_ts`` are required. A reversed time window (``from_ts`` after ``to_ts``) is rejected with 422. The underlying scan is capped at 50,000 matching spans, so metrics over a very large window may be based on a truncated set. Results are narrowed to traces the caller is authorized to read. When the tracing service serves the read, percentiles are exact on Postgres deployments and approximate estimates on ClickHouse deployments; durations are derived from microsecond timestamps on Postgres, so a value can differ by one millisecond at a boundary. On Postgres deployments the exact percentiles sort the matching rows in memory: a by-span request, or a metrics request whose ``bucket_minutes`` spans the whole window, over a window larger than the deployment's work memory and temp-file bound is refused rather than degraded, with 400 ``QUERY_TEMP_FILE_LIMIT_EXCEEDED`` and the tracing service's detail, with or without ``search_texts`` and on either read source, never as a fallback to the legacy Postgres store. Narrow the time window or filters; ClickHouse deployments never return that code. ``x-project-id`` narrows the result to traces whose root span carries that project, when ``PROJECT_SCOPED_SPAN_LISTING`` is on for the account, on either store. # Get trace deletion request status Source: https://docs.gp.scale.com/reference/v5/spans/get-trace-deletion-request-status /openapi/v5.yaml get /v5/spans/deletions/{deletion_request_id} # List span summaries for a trace Source: https://docs.gp.scale.com/reference/v5/spans/list-span-summaries-for-a-trace /openapi/v5.yaml get /v5/traces/{trace_id}/spans List payload-free structural span summaries for rendering a trace tree. Use this endpoint to build and navigate the hierarchy without downloading each span's input, output, or metadata; fetch a selected span's full payload from GET /v5/traces/{trace_id}/spans/{span_id}. Pages contain at most 100 summaries in ascending start-time order. Both ClickHouse- and Postgres-backed reads return service-issued opaque continuation cursors. The cursors are mutually exclusive, and supplying both returns 422. Reads are authorized against the requested trace. A trace holding more spans than the tracing service's Postgres engine can page is refused with 400 ``QUERY_TEMP_FILE_LIMIT_EXCEEDED`` in strict mode and served from the legacy trace store otherwise, since no window or filter narrows a single trace; ClickHouse deployments never return that code. # List trace deletion requests Source: https://docs.gp.scale.com/reference/v5/spans/list-trace-deletion-requests /openapi/v5.yaml get /v5/spans/deletions # Search spans Source: https://docs.gp.scale.com/reference/v5/spans/search-spans /openapi/v5.yaml post /v5/spans/search Search and list spans matching a set of filters, returning a keyset-paginated page. Filters in the request body include trace and span ids, names, statuses, types, free-text search, metadata, duration bounds, and more, scoped to an optional time window. Results are keyset-paginated on indexed columns rather than offset-paginated, and ``total`` is not computed (it is always 0); use the pagination cursors to page through results. Reads route to the legacy trace store or the tracing service per account (with fallback to the legacy trace store outside strict mode), and results are narrowed to traces the caller is authorized to read — a filter that resolves to no authorized traces yields an empty page rather than an error. A reversed time window (``from_ts`` after ``to_ts``) is rejected with 422, as is a request whose combined ``trace_ids``, ``span_ids``, ``excluded_span_ids``, ``excluded_trace_ids``, and ``parent_ids`` count exceeds 10000. ``sort_by`` accepts ``start_timestamp``, ``duration_ms``, ``input_tokens`` and ``output_tokens``; the token counts are read from usage reported at ingest, and a span whose producer reported none sorts as zero, so it lands last descending and first ascending. The two token sorts are rejected for an account still served by the legacy trace store, which keeps no token count to order by. Any other unsupported sort falls back to timestamp order there. Spans sharing a sort key are ordered by ``trace_id`` and ``id`` bytewise on both tracing-service engines. Text search tokenization, indexed-prefix, word-length, locale and metadata-bytes behavior differs between Postgres and ClickHouse deployments of the tracing service as the ``search_texts`` field describes. ``x-project-id`` narrows the result to traces whose root span carries that project, when ``PROJECT_SCOPED_SPAN_LISTING`` is on for the account, on either store. # Update a span Source: https://docs.gp.scale.com/reference/v5/spans/update-a-span /openapi/v5.yaml patch /v5/spans/{span_id} Partially update a span's mutable fields and return the updated span. Only the provided fields among name, end timestamp, output, metadata, and status are changed. This endpoint is available only for accounts still served solely by the legacy trace store: once an account begins dual-writing to the tracing service — which is upsert-only and has no partial-update operation — PATCH returns 501 and PUT /v5/spans/batch must be used instead. Updates are authorized against the span's parent trace, and an unknown id returns 404. Credential redaction: values in the free-form ``input``, ``output``, ``metadata``, and ``expected`` objects, and in ``name``, that are credential-shaped (bearer/JWT, API keys, connection-string passwords) or under a credential-named key are replaced with ``[REDACTED:credential]`` before the span is persisted, so the stored and returned span reflects the redacted value. # Upsert spans in a batch Source: https://docs.gp.scale.com/reference/v5/spans/upsert-spans-in-a-batch /openapi/v5.yaml put /v5/spans/batch Insert or replace multiple spans (up to 1000) in a single request. Use this for idempotent ingestion when spans may already exist, unlike POST /v5/spans/batch, which only inserts. Items without an ``id`` are assigned a generated UUID. The legacy trace store treats ``id`` as global and collapses repeated ``id``s to the last occurrence. The tracing service keys spans by ``trace_id`` and ``id``; it retains cross-trace ID collisions, and for a repeated pair keeps a completed span over an in-progress one, otherwise the later occurrence wins. In dual-write phases each store applies its own rule, and the primary store determines the returned list. A batch larger than 1000 spans is rejected with a validation error, as is any item whose ``end_timestamp`` precedes its ``start_timestamp``, which returns a 422. When the tracing service is the primary store, its 400, 413 and 422 rejections keep that status and detail, as does a 403 when the request carried its own API key, and every other failure returns a 503 with a Retry-After header. A batch that forms a cycle, or a span whose parent belongs to another trace, is rejected with 409 before either store is written. A parent outside the batch is looked up in Postgres, so that part of the check ends once the account's spans stop being written there. Which writes the tracing service rejects depends in part on its storage engine: on Postgres deployments a NUL byte or invalid UTF-8 inside ``trace_id``, ``id``, ``parent_id`` or ``group_id`` fails the whole batch with a 400 naming the field, because replacing the byte would change the identity the response echoes; a NUL byte or invalid UTF-8 in any other field is replaced with U+FFFD and the span is persisted. ClickHouse deployments store the bytes verbatim. Credential redaction: values in the free-form ``input``, ``output``, ``metadata``, and ``expected`` objects, and in ``name``, that are credential-shaped (bearer/JWT, API keys, connection-string passwords) or under a credential-named key are replaced with ``[REDACTED:credential]`` before the span is persisted, so the stored and returned span reflects the redacted value. # Count Vectors Source: https://docs.gp.scale.com/reference/v5/vector-stores/count-vectors /openapi/v5.yaml post /v5/vector-stores/{vector_store_name}/count Count documents in a vector store, optionally filtered by metadata. **Use Cases:** - Monitor vector store size and growth over time - Preview the number of documents matching a filter before deletion - Validate data ingestion by comparing expected versus actual document counts - Analyze document distribution across metadata categories **Filtering:** Apply the same metadata filter syntax as delete and list operations. Only indexed fields can be used for filtering. An empty filter counts all documents in the store. # Get Vector Source: https://docs.gp.scale.com/reference/v5/vector-stores/get-vector /openapi/v5.yaml get /v5/vector-stores/{vector_store_name}/vectors/{vector_id} Retrieve a single document by its unique ID. Returns the document's full content, metadata, and optionally its embedding vector. Use this endpoint for direct lookups when the exact document ID is known. For content similarity search, use the query endpoint. # List Vectors Source: https://docs.gp.scale.com/reference/v5/vector-stores/list-vectors /openapi/v5.yaml get /v5/vector-stores/{vector_store_name}/vectors List documents in a vector store with cursor-based pagination. **Use Cases:** Browse documents, export content, audit stored data, or retrieve documents by metadata without semantic search. **Ordering:** Documents are returned in storage order (insertion order), not ranked by similarity. For similarity-based retrieval, use the query endpoint. **Filtering:** Apply metadata filters to narrow results to specific subsets (e.g., all documents where `category: "research"`). Only indexed fields can be used for filtering. **Pagination:** Uses cursor-based pagination for efficient traversal of large datasets. Pass the `next_cursor` from each response as `starting_after` to retrieve the next page, or `prev_cursor` as `ending_before` to retrieve the previous page. A null cursor indicates no further pages exist. **Embedding Vectors:** Setting `include_vectors=true` includes the full embedding vector arrays in the response. This significantly increases payload size and reduces the maximum page size from 1000 to 100 documents. Enable only when raw vectors are required for external processing. # Query Vectors Source: https://docs.gp.scale.com/reference/v5/vector-stores/query-vectors /openapi/v5.yaml post /v5/vector-stores/{vector_store_name}/query Query documents using similarity search with optional reranking. Primary endpoint for semantic search, question-answering, and RAG (Retrieval-Augmented Generation) applications. Returns documents ranked by relevance to the query text with similarity scores. **Query Types:** - `semantic` (default): Approximate nearest-neighbor search using HNSW over cosine similarity of document embeddings. Optimal for question-answering, conceptual search, and finding semantically related content without requiring exact keyword matches. - `lexical`: Keyword-based text search (BM25 algorithm). Optimal for exact phrase matching, proper nouns, and scenarios where keyword presence is more important than semantic similarity. - `hybrid`: Combines semantic and lexical approaches with weighted scoring. Provides maximum recall by identifying documents matching either semantically or lexically. **Metadata Filtering:** Narrow the search scope by applying metadata filters (e.g., search only documents where `category: "technical"`). Only indexed fields can be used for filtering. Filters are applied before similarity search for optimal efficiency. **Reranking (Advanced):** Optionally enhance result quality using a cross-encoder reranking model. The reranker rescores the initial results using a more sophisticated model that evaluates the complete query-document pair (not solely embeddings). This adds 100-500ms latency but significantly improves precision for high-stakes applications. **Reranking Strategy:** Set `top_k` higher than the desired final count (e.g., 50) to retrieve more candidates from the initial search. Then configure `rerank_top_n` to the desired final count (e.g., 10) to return only the most relevant documents after reranking. This two-stage approach maximizes both recall and precision. **Performance Metrics:** The response includes detailed timing breakdowns (embedding generation time, index query time, reranking time) to facilitate search pipeline optimization and latency analysis. **Similarity Scores:** Each result includes a `score` field indicating relevance. Higher scores indicate greater relevance. Score ranges and semantics vary by query type (semantic scores use cosine similarity, lexical scores use BM25, hybrid scores combine both approaches). # Capabilities Changelog Source: https://docs.gp.scale.com/docs/capabilities/changelog Updates to the Capabilities documentation for workflows, document understanding, and OCR. This page tracks updates and additions to Scale's Capabilities documentation. *** ## Latest Updates ### How to run workflows using a Script **Added:** May 2026 New tutorial for starting saved Workflows runs from your own code using the Compass HTTP API—authentication, run-and-wait, session polling, workflow variables, and structured error handling. #### New documentation page **[How to run workflows using a Script](/docs/capabilities/workflows/run-workflows-using-a-script)** * Call `POST /v1/workflow/run` and retrieve results with `GET /v1/sessions/:sessionId/result` * Optional polling with `GET /v1/sessions/:sessionId` when you do not want a long-lived request * Pass string variables that override declared workflow defaults and populate `${variable_name}` placeholders in cards * Reference for session and card status values and common HTTP error responses *** ### Dex Documentation Refresh **Updated:** April 2026 Refreshed the Dex documentation to match the current async-only SDK and the latest document-understanding doc structure. #### What's Updated * Removed stale `DexSyncClient` guidance from Dex docs * Corrected SDK method names and examples to match the current Python SDK * Updated runtime guidance to Python 3.10+ and current 0.4.x SDK usage * Clarified wrapper-object access patterns and placed `get_download_url()` under `DexFile` in the SDK reference #### Updated Pages * [Getting Started with Dex](/docs/capabilities/document-understanding/getting-started-with-dex) * [Dex SDK API Reference](/docs/capabilities/document-understanding/dex-sdk-api-reference) * [Best Practices](/docs/capabilities/document-understanding/best-practices) * [Introduction to Dex](/docs/capabilities/document-understanding/introduction-to-dex) *** ### Document Understanding Workflows **Added:** April 2026 Workflows now supports document parsing & extraction using Dex directly within workflows. This integration enables you to process unstructured documents, extract structured data, and use the results in downstream workflow operations. #### What's New **New Documentation Page:** **[Guide to Document Understanding with Dex in Workflows](/docs/capabilities/workflows/dex-parsing-in-workflows)** * Step-by-step tutorial for building document understanding in workflows * Connect to cloud file storage (Amazon S3, Azure Blob, Google Cloud Storage) * Filter and select documents for processing * Parse documents with configurable Dex engines (Reducto) * Automatically create and manage Dex projects * Extract structured data from parsed documents * Push parsed data to Knowledge Base for semantic search and RAG * Use output data in subsequent workflow cards #### Key Features **Dex Card:** * **Project Management**: Automatically create Dex projects or reuse existing ones * **Multiple Parse Engines**: Support for Reducto and other parsing engines * **Flexible Configuration**: Configurable chunking methods and chunk sizes * **Structured Extraction**: Pulls targeted data from unstructured documents with citation & confidence scores * **Knowledge Base Integration**: Push parsed results to vector stores for semantic search * **Output Control**: Choose between detailed parsed content or metadata-only outputs **Data Connector for Cloud File Storages:** * Connect to Amazon S3, Azure Blob Storage, and Google Cloud Storage * Browse and select files from registered cloud storage connections **Workflow Integration:** * Access parsed content in downstream cards using `{{parsedContent}}` variable * Call agents to analyze, categorize, or transform parsed documents * Export parsed data to SGP Datasets or Evaluations or SGP Knowledge Base * Monitor parsing status and track results #### Some Use Cases * **Resume Parsing**: Extract structured data from resumes and categorize candidates * **Document Analysis**: Parse contracts, invoices, or reports and extract key information * **Content Extraction**: Convert unstructured documents into structured datasets for agent training or analysis * **Semantic Search**: Build Knowledge Base indexes from document collections *** ### Automated Evaluation Workflows **Added:** January 2026 Workflows is an AI-powered low-code agentic workflow orchestrator that enables enterprise operators to import data, transform it, call LLMs or Agents, analyze or evaluate the output, and automate the entire workflow—all in one interface. #### What's New **Two updated documentation pages:** 1. **Updated: [Introduction to Workflows](/docs/capabilities/workflows/introduction-to-workflows)** * Key features and available card types * Scheduling and monitoring workflows * Roadmap for H1 2026 2. **New: [Guide to Automating Evals on Workflows](/docs/capabilities/workflows/evals-in-workflows)** * Step-by-step tutorial for building automated evaluation workflows * Connecting to data sources (Snowflake, Postgres, MongoDB, MS SQL) * Calling Agentex agents for completions * Joining ground truth datasets from other workflows * Configuring LLM as Judge evaluations * Exporting results to Evaluations and Dashboards * Scheduling automated evaluation runs #### Updates to Key Features **Data Source Connections:** Query data directly from sources like `Snowflake`, `Postgres`, `MongoDB`, or `MS SQL` **Workflow Building Cards:** * `Call Agent`: Generate outputs from your Agentex agents with configurable timeout, retries, and response caching * `Join Workflow`: Combine datasets from multiple workflows with Left, Right, Inner, or Outer joins * `LLM as Judge`: Evaluate agent outputs with customizable rubrics and judge configurations **SGP Integrations:** * Access sources configured in SGP `Data Sources` tab in your workflows * Export datasets directly to SGP `Datasets` tab * Export evaluation results directly to SGP `Evaluations` tab **Notifications for automations:** Receive Slack notifications for failed or passed runs #### How to see the features Workflows is available as the `Workflows` tab in SGP. Navigate to **SGP > Workflows > New Workflow**. Contact the Scale team if you don't see it on your SGP instance. > [!NOTE] > > Workflows instances deployed in customer VPCs may have a limited set of available cards. Contact the Workflows team to activate or disable cards based on your requirements. *** ### Dex SDK: Version 0.4.0 **Released:** February 2025 Major improvements to the Dex SDK introducing unified entity operations, pagination, filtering, sorting, and enhanced async job support with SGP tracing integration. #### 🔄 Unified Entity Operations **New Feature**: Consistent `get` and `list` operations are now available for all entities within a project. **What's New:** * **Unified API**: All entities (files, jobs, parse_results, extractions, vector_stores) now support standardized `get` and `list` operations * **Project-scoped operations**: Access all entity types through your project instance * **Consistent patterns**: Same approach works across all entity types **Example:** ```python theme={null} # List operations available for all entities files = await project.list_files() jobs = await project.list_jobs() parse_results = await project.list_parse_results() extractions = await project.list_extractions() vector_stores = await project.list_vector_stores() ``` #### 📄 Pagination, Filtering, and Sorting **New Feature**: Token-based pagination with filtering and sorting for improved performance and discoverability. **What's New:** * **Token-based pagination**: Set page size and use continuation tokens to navigate through results * **Entity-specific filters**: Available for all entities with dedicated filter types * **Flexible sorting**: Sort results by any field in ascending or descending order * **Performance improvements**: Process large datasets more efficiently **Pagination:** ```python theme={null} from dex_sdk.types import PaginationParams pagination_params = PaginationParams( page_size=10, sort_by="created_at", sort_order="desc", continuation_token=None ) files = await project.list_files(pagination_params=pagination_params) ``` **Filtering:** ```python theme={null} from dex_sdk.types import FileListFilter, JobListFilter, ParseResultListFilter, ExtractionListFilter, VectorStoreListFilter from datetime import datetime, timedelta # Filter for entities created in the last day file_filter = FileListFilter(created_at_start=datetime.now() - timedelta(days=1)) job_filter = JobListFilter(created_at_start=datetime.now() - timedelta(days=1)) parse_result_filter = ParseResultListFilter(created_at_start=datetime.now() - timedelta(days=1)) extraction_filter = ExtractionListFilter(created_at_start=datetime.now() - timedelta(days=1)) vector_store_filter = VectorStoreListFilter(created_at_start=datetime.now() - timedelta(days=1)) # Apply filters files = await project.list_files(pagination_params=pagination_params, filter=file_filter) jobs = await project.list_jobs(pagination_params=pagination_params, filter=job_filter) parse_results = await project.list_parse_results(pagination_params=pagination_params, filter=parse_result_filter) extractions = await project.list_extractions(pagination_params=pagination_params, filter=extraction_filter) vector_stores = await project.list_vector_stores(pagination_params=pagination_params, filter=vector_store_filter) ``` #### 🔧 Enhanced Type Support **Improved**: Better type safety with conditional return types based on pagination parameters. **What's New:** * **Type overrides**: SDK provides different types for paginated vs non-paginated responses * **Backward compatibility**: Existing code continues to work without changes * **Better IDE support**: Enhanced autocomplete and type checking #### ⚡ Async Job Improvements **New Feature**: Enhanced async job support with SGP tracing integration for better monitoring and debugging. **What's New:** * **Job monitoring**: Easily track job status and progress * **Throttling support**: Control job throughput to match your needs * **SGP trace integration**: Jobs are now connected to SGP traces for end-to-end observability * **Trace retrieval**: Fetch complete trace data for any job **Example:** ```python theme={null} from dex_sdk.types import ReductoParseJobParams, ReductoParseEngineOptions, ReductoChunkingOptions, ReductoChunkingMethod, ParseEngine, JobStatus import asyncio # Start a parse job parse_job = await project.start_parse_job( dex_file=dex_file, parameters=ReductoParseJobParams( engine=ParseEngine.REDUCTO, options=ReductoParseEngineOptions( chunking=ReductoChunkingOptions( chunk_mode=ReductoChunkingMethod.VARIABLE, ) ), ), ) # Monitor job progress while parse_job.data.status != JobStatus.SUCCEEDED and parse_job.data.status != JobStatus.FAILED: await asyncio.sleep(1) await parse_job.refresh() # Get the result parse_result = await parse_job.get_result() # Retrieve SGP traces for the job from scale_gp_beta import SGPClient sgp_client = SGPClient(...) spans = [span for span in sgp_client.spans.search( sort_by="created_at", sort_order="desc", extra_metadata={"job_id": "job_parse_eb62b57e8c4940028b82c559903ed003"}, parents_only=True, )] trace_id = spans[0].trace_id spans = [span for span in sgp_client.spans.search(trace_ids=[trace_id])] ``` **Benefits:** * **Better observability**: Track your jobs through SGP's tracing infrastructure * **Easier debugging**: Access detailed execution traces for failed jobs * **Performance monitoring**: Analyze job performance and identify bottlenecks * **Request correlation**: Connect job execution to API requests and traces #### How to Update **1. Install/Update SDK:** Use your configured CodeArtifact credentials to upgrade the Dex SDK to version 0.4.0 or higher. **2. Explore New Features:** * Add pagination to large list operations for better performance * Use filters to find specific entities by creation time or other criteria * Integrate SGP tracing to monitor your async jobs * Leverage enhanced type support for better IDE experience **3. Backward Compatibility:** Version 0.4.0 is fully backward compatible. Your existing code will continue to work without modifications. *** ### Dex Document Understanding: Version 0.3.2 **Released:** November 2025 Significant updates to the Dex document understanding service and SDK, including authentication changes, data retention policies, and comprehensive documentation improvements. #### 🔐 Authentication Changes **Breaking Change**: The authentication method for Dex has been updated to improve security and simplify the API. **What Changed:** * **SGP credentials are now passed directly to `DexClient`** instead of being stored in project configurations * Every API request is now authenticated using your SGP API key and account ID * The `ProjectCredentials` and `SGPCredentials` types are deprecated and will be removed in a future version **Migration Required:** Old way (deprecated): ```python theme={null} from dex_sdk import DexClient from dex_sdk.types import ProjectCredentials, SGPCredentials dex_client = DexClient(base_url="https://dex.sgp.scale.com") project = await dex_client.create_project( name="My Project", credentials=ProjectCredentials( sgp=SGPCredentials( account_id=os.getenv("SGP_ACCOUNT_ID"), api_key=os.getenv("SGP_API_KEY"), ), ), ) ``` New way (version 0.3.2+): ```python theme={null} import os from dex_sdk import DexClient # Pass credentials when creating the client dex_client = DexClient( base_url="https://dex.sgp.scale.com", api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"), ) # Create project without credentials parameter project = await dex_client.create_project( name="My Project", ) ``` **Benefits:** * **Enhanced Security**: SGP credentials are no longer stored in the Dex database * **Simpler API**: Credentials are set once at client initialization * **Consistent Authentication**: Every request is authenticated the same way #### 🗄️ Data Retention Policies **New Feature**: Dex now supports configurable data retention policies for automatic lifecycle management of files and processing artifacts. **What's New:** * **Automatic cleanup**: Set retention periods for files and result artifacts to automatically delete data after a specified time * **Flexible configuration**: Configure different retention periods for files vs. processing artifacts * **Project-level control**: Retention policies are configured per project and can be updated at any time **Usage:** Configure retention when creating or updating a project: ```python theme={null} from datetime import timedelta from dex_sdk.types import ProjectConfiguration, RetentionPolicy project = await dex_client.create_project( name="My Project", configuration=ProjectConfiguration( retention=RetentionPolicy( files=timedelta(days=30), # Files expire after 30 days result_artifacts=timedelta(days=7), # Parse/extract results expire after 7 days ) ) ) ``` **Use Cases:** * **Compliance**: Meet regulatory requirements (GDPR, HIPAA) by enforcing data retention limits * **Cost Management**: Reduce storage costs by automatically cleaning up old files and artifacts * **Security**: Minimize data exposure by limiting how long sensitive documents are stored See the [Getting Started with Dex](/docs/capabilities/document-understanding/getting-started-with-dex#configuring-data-retention) guide for detailed examples. #### 📚 Documentation Improvements **Major Update**: Comprehensive expansion of the Dex SDK documentation with detailed type information and practical examples. **API Reference Enhancements:** * **Common Types Section**: Added comprehensive documentation for all core data models organized by category: * **Project Types**: `ProjectEntity`, `ProjectStatus`, `ProjectConfiguration`, `RetentionPolicy` * **File Types**: `FileEntity`, `FileStatus`, `FileDownloadURL` * **Job Types**: `JobEntity`, `JobOperationType`, `JobStatus` * **Parse Types**: `ParseResultEntity`, `ParseResultMetadata`, `ParseChunk`, `ParseBlock`, `BoundingBox`, `ParseEngine`, `ReductoChunkingMethod` * **Extraction Types**: `ExtractionEntity`, `ExtractionResult`, `ExtractedField`, `ExtractionCitation`, `ExtractionParameters`, `UsageInfo` * **Vector Store Types**: `VectorStoreEntity`, `VectorStoreEngines`, `VectorStoreChunk`, `SearchConfig` * **Parse Job Parameters**: New section documenting parsing configuration options **Enhanced Examples:** * **Working with Parse Results**: How to access parse metadata, iterate through chunks and blocks, access bounding boxes and confidence scores * **Working with Extraction Results**: How to access extracted field values, citations, confidence scores, and token usage information **Updated Documentation:** * **Introduction**: Updated File Management section to mention data retention policies * **Getting Started Guide**: Added "Configuring Data Retention" section with practical examples * **API Reference**: Enhanced type documentation and examples #### 🔧 Type System Updates **ParseResult Naming Changes:** The ParseResult classes have been harmonized for consistency: * **Request objects** now end with `*Request` (e.g., `CustomParseResultRequest`) * **Entity objects** now end with `*Entity` (e.g., `ParseResultEntity`) Migration: ```python theme={null} # Old import (deprecated) from dex_sdk.types import CustomParseResult # New import from dex_sdk.types import CustomParseResultRequest ``` **Improved Type Safety:** * Fixed typing inconsistencies for entities returned by the DEX API * Better type hints for all SDK methods * Enhanced autocomplete support in IDEs #### How to Update **1. Install/Update SDK:** Use your configured CodeArtifact credentials to upgrade the Dex SDK to version 0.3.2 or higher. **2. Update Your Code:** 1. Add `api_key` and `account_id` parameters to `DexClient()` initialization 2. Remove `credentials` parameter from `create_project()` calls 3. Remove imports of `ProjectCredentials` and `SGPCredentials` **3. Test Your Integration:** Run your scripts to ensure they work with the new authentication method. **MCP Integration Status:** Authentication via Model Context Protocol (MCP) is still in progress. If you encounter issues, please reach out to the Dex team at `#dex-help`. *** ### New Capability: IRIS OCR **Added:** October 2024 IRIS is Scale's OCR capability that transforms document images and PDFs into structured text through an intelligent multi-stage pipeline. #### What's New **Two new documentation pages:** 1. **[Introduction to IRIS](/docs/capabilities/ocr/introduction-to-iris)** * Overview of IRIS OCR capability * Three-stage pipeline architecture (layout detection, OCR processing, assembly) * 15+ supported OCR models including open-source and vision-language models * Multi-language support with specialized Arabic models * Common use cases and key advantages 2. **[Getting Started with IRIS](/docs/capabilities/ocr/getting-started-with-iris)** * Comprehensive guide to using IRIS through Dex SDK * Prerequisites and setup instructions * Parsing PDFs and images with complete examples * Configuration options for parse engine * Understanding parse results and chunk structure * File management and error handling * Batch processing examples * Multi-language support details * Best practices for production use * Performance considerations and optimization tips #### Key Features * **Layout-Aware Processing**: Automatically detects text, tables, and images before OCR * **Multiple OCR Engines**: Choose from Tesseract, EasyOCR, PaddleOCR, Surya, GPT-4o, Gemini, and more * **Table-Specific Processing**: Specialized models optimized for accurate table extraction * **Multi-Language Support**: Process documents in 35+ languages including Arabic * **Dex Integration**: Seamless integration with Dex's document understanding platform * **Async Processing**: Non-blocking parse jobs with project-based organization #### How to Access IRIS is available through the Dex SDK as a parse engine option: ```python theme={null} from dex_core.models.parse_job import IrisParseEngineOptions, IrisParseJobParams parse_job = await dex_file.start_parse_job( IrisParseJobParams(options=IrisParseEngineOptions()) ) ``` *** ## Configuration Updates ### Navigation Structure Improvements **Updated:** October 2024 * Added explicit V5 (beta) version tags to all Capabilities navigation groups * Ensures proper scoping of Capabilities documentation to V5 * Improved navigation organization for better user experience **Affected Sections:** * Getting Started * Document Understanding * OCR * Workflows *** ## Support and Feedback ### Workflows-Specific Questions For questions or issues related to SGP Workflows: * **Slack**: `#compass` * **Documentation**: [Introduction to Workflows](/docs/capabilities/workflows/introduction-to-workflows) * **Tutorial**: [Guide to Automating Evals on Workflows](/docs/capabilities/workflows/evals-in-workflows) ### Dex-Specific Questions For questions or issues related to Dex: * **Slack**: `#dex-help` * **Documentation**: [Getting Started with Dex](/docs/capabilities/document-understanding/getting-started-with-dex) * **API Reference**: [Dex SDK API Reference](/docs/capabilities/document-understanding/dex-sdk-api-reference) ### General Capabilities Documentation Have suggestions for improving our Capabilities documentation? Please contact the Scale AI team or submit feedback through your account dashboard. # API Reference Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/api-reference REST API reference for Document Understanding (Dex) — projects, files, parse, extract, and vector stores. The Document Understanding API provides REST endpoints for document processing. Use the API for non-Python integrations or when you need direct control over HTTP requests. Browse the full REST API in the **[Dex API](/reference/dex/projects/list-projects)** tab. Key endpoint groups: | Group | Description | | - | - | | **Projects** | Create, list, and manage projects | | **Files** | Upload files, get download URLs, stream content | | **Parse** | Create parse jobs, list results, rechunk | | **Extract** | Create extract jobs, list extractions | | **Vector Stores** | Create stores, add files, search | | **Jobs** | Get job status and monitor progress | *** ## Authentication All requests require: * **`x-api-key`**: Your SGP API key * **`x-selected-account-id`**: Your SGP account ID ```bash theme={null} curl -X GET "https://dex.sgp.scale.com/v1/projects" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` *** ## Base URL | Environment | Base URL | | - | - | | Production | `https://dex.sgp.scale.com` | *** ## Next Steps * **[Getting Started](getting-started-with-dex)**: Step-by-step tutorial with the Python SDK * **[Dex SDK API Reference](dex-sdk-api-reference)**: Python SDK methods and types * **[Best Practices](best-practices)**: Optimization and common patterns # Best Practices Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/best-practices Quick start, optimization tips, data retention, and common patterns for Dex document understanding. ## 30-Second Quick Start With citations and confidence both disabled, `extract` returns your schema model. See [Extraction Response Shapes](extract#extraction-response-shapes) before enabling either flag. ```python theme={null} import os import asyncio from pydantic import BaseModel, Field from dex_sdk import DexClient from dex_sdk.types import (ParseEngine, IrisParseJobParams, IrisParseEngineOptions) class Schema(BaseModel): field: str = Field(description="Description") async def main(): client = DexClient(base_url="https://dex.sgp.scale.com", api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID")) project = await client.create_project(name="My Project") file = await project.upload_file("doc.pdf") parsed = await file.parse( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions(text_ocr="openai/gpt-5.4", layout="whole_page" ), ) ) result = await parsed.extract( extraction_schema=Schema, user_prompt="Extract data", model="openai/gpt-5.4", generate_citations=False, generate_confidence=False, ) print(result.model_dump()) asyncio.run(main()) ``` *** ## Async Client Pattern The current Dex SDK is async-only. Prefer `async with DexClient(...)` when you are making multiple calls so the underlying HTTP connection can be reused: ```python theme={null} from dex_sdk import DexClient async with DexClient( base_url="https://dex.sgp.scale.com", api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"), ) as client: project = await client.create_project(name="My Project") ``` *** ## Data Retention Policies Configure automatic data lifecycle management for compliance and cost optimization. ### Setting Retention Policies ```python theme={null} from datetime import timedelta from dex_core.models.projects import ProjectUpdateRequest from dex_sdk.types import ProjectConfiguration, RetentionPolicy # Create project with retention policy project = await dex_client.create_project( name="Compliant Project", configuration=ProjectConfiguration( retention=RetentionPolicy( files=timedelta(days=30), # Files expire after 30 days result_artifacts=timedelta(days=7), # Parse/extract results expire after 7 days ) ) ) ``` ### Updating Retention Policies ```python theme={null} await dex_client.update_project( project_id=project.id, project_data=ProjectUpdateRequest( configuration=ProjectConfiguration( retention=RetentionPolicy( files=timedelta(days=90), # Extend to 90 days result_artifacts=timedelta(days=30), # Extend to 30 days ) ) ) ) ``` ### Retention Policy Use Cases **Compliance (GDPR, HIPAA):** ```python theme={null} retention=RetentionPolicy( files=timedelta(days=2555), # 7 years result_artifacts=timedelta(days=365), # 1 year ) ``` **Cost Optimization:** ```python theme={null} retention=RetentionPolicy( files=timedelta(days=30), # Raw files for 1 month result_artifacts=timedelta(days=7), # Results for 1 week ) ``` **Security (Minimize Exposure):** ```python theme={null} retention=RetentionPolicy( files=timedelta(days=1), # Delete files after 1 day result_artifacts=timedelta(hours=24), # Delete results after 24 hours ) ``` *** ## Optimizing Extraction Accuracy 1. **Write Clear Prompts** ```python theme={null} # Good: Specific and detailed user_prompt = """Extract the following fields: - Invoice number: Find in top-right corner or header - Total amount: The final amount including tax - Date: Invoice date, not payment date""" # Bad: Vague user_prompt = "Extract invoice info" ``` 2. **Design Good Schemas** ```python theme={null} # Good: Descriptive fields with clear types class InvoiceData(BaseModel): invoice_number: str = Field(description="The invoice number (format: INV-XXXXX)") total_amount: float = Field(description="Total amount in USD, including tax") invoice_date: str = Field(description="Invoice date in YYYY-MM-DD format") # Bad: No descriptions class InvoiceData(BaseModel): number: str amount: float date: str ``` 3. **Enable Citations and Confidence** * Always set `generate_citations=True` for debugging * Use `generate_confidence=True` to filter low-confidence results 4. **Use Vector Stores for Large Documents** * Documents > 50 pages benefit from RAG-enhanced extraction * Vector stores improve accuracy for cross-document queries *** ## Getting SGP Traces for Extraction Use `start_extract_job` instead of `extract` to get async jobs that are linked to SGP traces. Search for traces by job ID to debug extraction latency, token usage, or failures. ```python theme={null} import os from scale_gp_beta import SGPClient from dex_sdk.types import ExtractionParameters # Start extract job (returns immediately) extract_job = await parse_result.start_extract_job( ExtractionParameters( model="openai/gpt-5.4", extraction_schema=MySchema.model_json_schema(), user_prompt="Extract the data", ) ) # Wait for completion result = await extract_job.get_result() # Retrieve SGP traces for the extract job sgp_client = SGPClient( api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"), ) spans = list(sgp_client.spans.search( sort_by="created_at", sort_order="desc", extra_metadata={"job_id": extract_job.data.id}, parents_only=True, )) if spans: trace_id = spans[0].trace_id all_spans = list(sgp_client.spans.search(trace_ids=[trace_id])) for span in all_spans: print(f"Span: {span.name}, Duration: {span.duration_ms}ms") ``` *** ## Common Workflows **Single Document Processing:** ``` Upload → Parse → Extract → Review Results ``` **Multi-Document Analysis:** ``` Upload All → Parse All → Create Vector Store → Add to Store → Extract → Aggregate ``` **Iterative Refinement:** ``` Parse → Review → Adjust Chunking/Engine → Re-parse → Extract ``` **Production Pipeline:** ``` Upload → Parse → Store in Vector DB → Extract on Demand → Cache Results ``` *** ## Performance Optimization ### Reduce Latency 1. **Tune downstream chunking**: Rechunking controls embedding, retrieval, and extraction context size; it does not shorten the preceding OCR parse. For parse latency, choose the layout mode, OCR models, and supported concurrency settings for your workload. 2. **Limit direct extraction pages**: In SDK `0.7.1` or later, pass 1-based `pages` ranges to [direct file extraction](extract#extract-from-file-direct). Parsing has no page-range option; upload a file containing only the required pages when you need a smaller parse. 3. **Batch operations**: Process multiple files in parallel 4. **Cache parse results**: Reuse parsed documents for multiple extractions ### Reduce Costs 1. **Set retention policies**: Auto-delete old data 2. **Choose right model**: Use smaller models when possible 3. **Optimize prompts**: Shorter prompts = lower token costs 4. **Filter before extraction**: Use vector search to find relevant chunks first *** ## Common Errors & Quick Fixes | Error | Cause | Quick Fix | | - | - | - | | `DexAPIError` | The API returned an error status, for example `401` or `403` for missing or invalid credentials | Read `e.response.status_code` and `e.response.text`; check `SGP_API_KEY` and `SGP_ACCOUNT_ID` env vars | | `ValueError` | A parse, extract, or research job failed, was cancelled, or only partially succeeded; or `upload_file` got a path that doesn't exist | Read the error message; try a different engine or check document quality | | `TimeoutError` | Polling reached `max_attempts` before the job finished | Raise `max_attempts`, or start the job and poll it yourself | | `httpx.TransportError` | Network issues | Check internet connection, verify base URL | *** ## Access Response Data Project, file, parse, and vector-store methods return **wrapper objects**. Use `.data` for full entity fields, and use convenience properties like `.id` when available: ```python theme={null} # Correct project = await client.create_project(name="Test") project_id = project.data.id project_name = project.data.name # Also valid project_id = project.id ``` **Common Response Attributes:** ```python theme={null} project.data.id # Project ID project.data.name # Project name project.data.created_at # Creation timestamp dex_file.data.id # File ID dex_file.data.filename # Filename dex_file.data.size_bytes # File size parse_result.data.id # Parse result ID parse_result.data.engine # Engine used parse_result.data.parse_metadata.pages_processed # Pages count ``` When citations or confidence are enabled, parse-result and vector-store `extract` return extracted fields at `extract_result.data.data` and token usage, when present, at `extract_result.data.usage_info`. With both disabled, they return your schema model; use `.model_dump()`, as in the quick start above. Extract jobs and direct file extraction wrap metadata responses differently; see [Extraction Response Shapes](extract#extraction-response-shapes). *** ## Type Import Quick Lookup | Type | Import From | Used For | | - | - | - | | `DexClient` | `dex_sdk` | Client initialization | | `ParseEngine` | `dex_sdk.types` | OCR engine selection | | `IrisParseJobParams` | `dex_sdk.types` | Iris configuration | | `ReductoParseJobParams` | `dex_sdk.types` | Reducto configuration | | `IrisParseEngineOptions` | `dex_sdk.types` | Iris parser options | | `ReductoParseEngineOptions` | `dex_sdk.types` | Parser options | | `ExtractionParameters` | `dex_sdk.types` | Extraction config | | `VectorStoreEngines` | `dex_sdk.types` | Vector store engines | | `VectorStoreSearchResult` | `dex_sdk.types` | Search results | | `ProjectConfiguration` | `dex_sdk.types` | Project config | | `RetentionPolicy` | `dex_sdk.types` | Data retention | | `PaginationParams` | `dex_sdk.types` | Pagination config | | `FileListFilter` | `dex_sdk.types` | File filtering | | `JobListFilter` | `dex_sdk.types` | Job filtering | | `ParseResultListFilter` | `dex_sdk.types` | Parse result filtering | | `ExtractionListFilter` | `dex_sdk.types` | Extraction filtering | | `VectorStoreListFilter` | `dex_sdk.types` | Vector store filtering | | `JobStatus` | `dex_sdk.types` | Job status enum | | `ReductoChunkingMethod` | `dex_sdk.types` | Chunking method enum | | `ReductoChunkingOptions` | `dex_sdk.types` | Chunking config | **Response Types** (accessed via `.data`, no import needed): * `ProjectEntity`, `FileEntity`, `ParseResultEntity`, `VectorStoreEntity` For extraction response types, see [Extraction Response Shapes](extract#extraction-response-shapes). *** ## Next Steps * **[Getting Started](getting-started-with-dex)**: Step-by-step tutorial * **[File Management](file-management)**: Upload and manage files * **[API Reference](dex-sdk-api-reference)**: Complete SDK documentation * **[Troubleshooting](troubleshooting)**: Detailed error solutions # Chunking Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/chunking Choose chunking strategies for parsed documents, Dex rechunking or Reducto parse-time chunking. Dex provides flexible chunking for parsed documents. You can apply chunking in two ways: **Dex rechunking** (parse once, then rechunk with different strategies) or **engine-specific chunking** during the initial parse (Reducto only). *** ## IRIS v2 Chunk Structure IRIS v2 always returns a `chunks` array in the parse result. `chunk_mode: disabled` is a Reducto-only concept (soon to be deprecated) that does not exist in IRIS v2. Chunk count and size are driven by **how IRIS segments each page**, not by a separate chunking enum. | Control | Values | Effect on chunks | | - | - | - | | `layout` | `"rt_detr_bce"` (default) | One chunk per detected layout region (text, table, image, etc.) | | `layout` | `"pp_doclayout_v3"` | One chunk per detected layout region | | `layout` | `"whole_page"` | One region per page → **one chunk per page** | | `e2e_ocr` + `e2e_response_parser` | e.g. DeepSeek OCR2 | Full-page e2e OCR; parser defines region/box structure | | Layout filters | `confidence_threshold`, `containment_threshold`, `strict_containment_filter`, table/image thresholds | Fewer or more regions → fewer or more chunks | | `img_method` | `"description"`, `"base64"`, `"skip"` | Whether image regions become chunks (`skip` omits image regions) | These options live on `IrisParseEngineOptions` inside `IrisParseJobParams`. They are **not** the same as Dex’s four rechunk strategies below—they control IRIS’s native output only. ### Parse with IRIS v2 ```python theme={null} from dex_sdk.types import ( ParseEngine, IrisParseJobParams, IrisParseEngineOptions, ) parse_result = await dex_file.parse( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( layout="rt_detr_bce", # default: region-level chunks # layout="whole_page", # alternative: one chunk per page text_ocr="openai/gpt-5.4", ), ) ) ``` ## Dex Chunking Strategies Dex offers four chunking strategies that work on any parse result—including IRIS v2 via rechunking. Use these when you need consistent, configurable chunk boundaries across documents or for embeddings/RAG | Strategy | Description | Best For | Embedding Suitability | | - | - | - | - | | `token_size` | Splits by token count using a tokenizer (e.g., tiktoken) | LLM APIs with token limits, cost optimization | Excellent | | `recursive` | Recursively splits using separators (paragraphs → sentences → words) | Articles, documentation, RAG systems | Excellent | | `by_page` | Splits by page boundaries, grouping complete pages | Legal documents, forms, reports | May be large | | `by_section` | Starts a new chunk at each block whose type is `Section Header`, `Header`, or `Title` | Technical manuals, wikis, academic papers | Good | *** `by_section` uses parsed block types rather than scanning for Markdown `#` prefixes. Markdown headings must first be represented as heading blocks, or you can split the text separately with a recursive strategy. `by_section` starts a new section at blocks labeled `Section Header`, `Header`, or `Title` (case-sensitive). The `section_headers` option has no effect. IRIS emits `Title` and `Section-header`, so only `Title` starts a new section when rechunking IRIS output. ## Parse Once, Post-chunking as needed With IRIS 2, you parse without Dex rechunking (omit chunking_options), then apply Dex strategies on the parse result. This is the recommended pattern: one parse, many chunking experiments. ```python theme={null} from dex_sdk.types import IrisParseJobParams, IrisParseEngineOptions, ParseEngine from dex_core.models.chunking import ( TokenSizeChunkingOptions, RecursiveChunkingOptions, PageChunkingOptions, SectionChunkingOptions, ) # Step 1: Parse with IRIS v2 (native region- or page-level chunks) parse_result = await dex_file.parse( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( layout="pp_doclayout_v3", # detect regions, including section headings ), ) ) # Step 2: Rechunk with Dex strategies # Token-based (for LLM APIs, embedding models) rechunked = await parse_result.rechunk( TokenSizeChunkingOptions( chunk_size=512, chunk_overlap=50, encoding_name="cl100k_base", ) ) # Recursive (preserves paragraphs and sentences) rechunked = await parse_result.rechunk( RecursiveChunkingOptions( chunk_size=1000, chunk_overlap=200, ) ) # Page-based (legal docs, forms) rechunked = await parse_result.rechunk( PageChunkingOptions(pages_per_chunk=1) ) # Section-based (structured documents with headers) # IRIS Title blocks start sections; Section-header blocks stay in the current section. rechunked = await parse_result.rechunk( SectionChunkingOptions( include_header_in_chunk=True, ) ) ``` *** ## Async Rechunking For long-running documents, start the rechunk job and poll for completion: ```python theme={null} # Start rechunk job (returns immediately) job = await parse_result.start_rechunk_job( TokenSizeChunkingOptions(chunk_size=512, chunk_overlap=50) ) # Wait for completion and get rechunked result rechunked = await job.get_rechunked_result() ``` *** ## Chunking Decision Tree **IRIS v2 native (`IrisParseEngineOptions`):** * **Many small, layout-aware chunks** → `layout="rt_detr_bce"` (default) * **One chunk per page** → `layout="whole_page"` * **Full-page e2e model** → `e2e_ocr` + `e2e_response_parser` * **Fewer regions (fewer chunks)** → raise confidence thresholds or tighten containment filters **Dex rechunk (after IRIS v2 parse):** **Use `token_size` when:** * Working with LLM APIs that have token limits * Embedding models with specific token limits * Cost optimization **Use `recursive` when:** * General document chunking for RAG * Preserving paragraphs and sentences * Articles, blog posts, documentation **Use `by_page` when:** * Legal documents, forms, reports * Page references matter * Page structure should be preserved **Use `by_section` when:** * Documents have clear section headers * Technical manuals, wikis, academic papers * Semantic coherence within topics *** ## Reducto Chunking Parse-Time - (Legacy) > [!WARNING] > > **Deprecation notice:** Reducto is being deprecated. Use Iris for new projects. When using the **Reducto** parse engine, you can chunk during the initial parse instead of rechunking. Reducto's methods are layout-aware and use document structure. | Method | Chunk Size | Best For | Embedding | Location Tracking | | - | - | - | - | - | | **VARIABLE** | Auto (optimal) | General use, embeddings | Excellent | Good | | **BLOCK** | Small (\~100-500 chars) | Precise locations, UI overlays | Too small | Excellent | | **SECTION** | Medium (\~1000-3000 chars) | Structured documents | Good | Good | | **PAGE** | Large (full page) | Page-oriented docs | May be large | Excellent | | **PAGE_SECTIONS** | Medium-Large | Hybrid needs | Good | Good | | **DISABLED** | Very large (entire doc) | Special cases | Too large | Excellent | **Recommendation:** Use `VARIABLE` for most cases, especially with embeddings. ```python theme={null} # (Legacy) Reducto parse-time chunking from dex_sdk.types import ( ParseEngine, ReductoParseJobParams, ReductoChunkingMethod, ReductoChunkingOptions, ReductoParseEngineOptions, ) # Reducto variable chunking (parse-time) parse_result = await dex_file.parse( ReductoParseJobParams( engine=ParseEngine.REDUCTO, options=ReductoParseEngineOptions( chunking=ReductoChunkingOptions( chunk_mode=ReductoChunkingMethod.VARIABLE, chunk_size=None, ) ), ) ) # Reducto block chunking for precise location tracking parse_result = await dex_file.parse( ReductoParseJobParams( engine=ParseEngine.REDUCTO, options=ReductoParseEngineOptions( chunking=ReductoChunkingOptions(chunk_mode=ReductoChunkingMethod.BLOCK) ), ) ) ``` ### Pattern: Retry with Different Chunking ```python theme={null} # (Legacy) Reducto retry pattern # Try 1: Variable chunking result1 = await file.parse( ReductoParseJobParams( engine=ParseEngine.REDUCTO, options=ReductoParseEngineOptions( chunking=ReductoChunkingOptions( chunk_mode=ReductoChunkingMethod.VARIABLE, ) ), ) ) # If not satisfactory, try block chunking result2 = await file.parse( ReductoParseJobParams( engine=ParseEngine.REDUCTO, options=ReductoParseEngineOptions( chunking=ReductoChunkingOptions( chunk_mode=ReductoChunkingMethod.BLOCK, ) ), ) ) ``` *** ### Reducto-only decision tree (Legacy) **Use Reducto `VARIABLE` when:** * Still on Reducto and want layout-aware chunking at parse time * General document processing with parser-optimal chunk sizes **Use Reducto `BLOCK` when:** * Need precise bounding box information * Building UI overlays on documents * Not using for embeddings **Use Reducto `DISABLED` when:** * You plan to rechunk with Dex strategies (same pattern as IRIS v2) ``` What's your primary goal? (Reducto only) ├─ Embeddings/Semantic Search → VARIABLE ├─ Precise bounding boxes/UI → BLOCK ├─ Page-by-page processing → PAGE └─ Structured document navigation → SECTION ``` *** ## Next Steps * **[Vector Stores](vector-stores)**: Add chunks to vector stores for semantic search * **[Extract](extract)**: Extract structured data from parse results * **[Parse](parse)**: Parse engine options and configuration (IRIS v2) # Dex SDK API Reference Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/dex-sdk-api-reference Python SDK methods and examples for Dex projects, files, parsing, extraction, and vector stores. This reference documents the Python SDK methods for Scale's Dex document understanding capability. ## DexClient The main client for interacting with the Dex service. ### Project Management * `create_project(name, configuration)` - Create a new project with optional configuration * `list_projects()` - List all accessible projects * `get_project(project_id)` - Retrieve a specific project * `update_project(project_id, project_data)` - Update project name, configuration, or status **Example:** ```python theme={null} import os from datetime import timedelta from dex_sdk import DexClient from dex_sdk.types import ProjectConfiguration, RetentionPolicy # Initialize client with SGP credentials dex_client = DexClient( base_url="https://dex.sgp.scale.com", api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"), ) # Create project (credentials are passed via client initialization) project = await dex_client.create_project( name="My Project", ) # Create project with data retention policy project = await dex_client.create_project( name="My Compliant Project", configuration=ProjectConfiguration( retention=RetentionPolicy( files=timedelta(days=30), result_artifacts=timedelta(days=7), ) ) ) # Update project configuration from dex_core.models.projects import ProjectUpdateRequest await dex_client.update_project( project_id=project.id, project_data=ProjectUpdateRequest( configuration=ProjectConfiguration( retention=RetentionPolicy( files=timedelta(days=90), result_artifacts=timedelta(days=30), ) ) ), ) ``` *** ## Project Represents a Dex project with isolated data and credentials. ### File Operations * `upload_file(file_path)` - Upload a document to the project * `list_files(pagination_params, filter)` - List all uploaded files with optional pagination and filtering * `get_file_by_id(file_id)` - Get file metadata **Example:** ```python theme={null} # Upload a file dex_file = await project.upload_file("path/to/document.pdf") # List files with pagination from dex_sdk.types import PaginationParams pagination_params = PaginationParams( page_size=10, sort_by="created_at", sort_order="desc", continuation_token=None ) files = await project.list_files(pagination_params=pagination_params) # List files with filtering from dex_sdk.types import FileListFilter from datetime import datetime, timedelta file_filter = FileListFilter(created_at_start=datetime.now() - timedelta(days=1)) files = await project.list_files(pagination_params=pagination_params, filter=file_filter) # Get a temporary download URL for an uploaded file download_url = await dex_file.get_download_url() ``` ### Job Operations * `list_jobs(pagination_params, filter)` - List all jobs in the project with optional pagination and filtering * `get_job_by_id(id)` - Get job details and status **Example:** ```python theme={null} # List jobs with pagination and filtering from dex_sdk.types import PaginationParams, JobListFilter from datetime import datetime, timedelta pagination_params = PaginationParams( page_size=10, sort_by="created_at", sort_order="desc", ) job_filter = JobListFilter(created_at_start=datetime.now() - timedelta(days=1)) jobs = await project.list_jobs(pagination_params=pagination_params, filter=job_filter) ``` ### Parse Result Operations * `list_parse_results(pagination_params, filter)` - List all parse results with optional pagination and filtering * `get_parse_result_by_id(id)` - Get parse result details **Example:** ```python theme={null} # List parse results with filtering from dex_sdk.types import PaginationParams, ParseResultListFilter parse_result_filter = ParseResultListFilter(created_at_start=datetime.now() - timedelta(days=7)) parse_results = await project.list_parse_results( pagination_params=PaginationParams(sort_by="created_at"), filter=parse_result_filter, ) ``` ### Extraction Operations * `list_extractions(pagination_params, filter)` - List all extractions with optional pagination and filtering * `get_extraction_by_id(id)` - Get extraction details **Example:** ```python theme={null} # List extractions with filtering from dex_sdk.types import ExtractionListFilter, PaginationParams extraction_filter = ExtractionListFilter(created_at_start=datetime.now() - timedelta(days=7)) extractions = await project.list_extractions( pagination_params=PaginationParams(sort_by="created_at"), filter=extraction_filter, ) ``` ### Vector Store Operations * `create_vector_store(name, engine, embedding_model)` - Create a vector store with the SGP Knowledge Base engine (the SDK default) or the SGP Vector Store engine (recommended). For Models API embeddings, use `embedding_type="models_api"` and `model_deployment_id` instead of `embedding_model`; see [Vector Store Engines](vector-stores#vector-store-engines) for SDK and deployment requirements. * `list_vector_stores(pagination_params, filter)` - List all vector stores with optional pagination and filtering * `get_vector_store_by_id(id)` - Get vector store details * `delete()` - Delete a vector store from a `DexVectorStore` instance **Example:** ```python theme={null} from dex_sdk.types import VectorStoreEngines vector_store = await project.create_vector_store( name="My Vector Store", engine=VectorStoreEngines.SGP_KNOWLEDGE_BASE, embedding_model="openai/text-embedding-3-large", ) # List vector stores with filtering from dex_sdk.types import PaginationParams, VectorStoreListFilter vector_store_filter = VectorStoreListFilter(created_at_start=datetime.now() - timedelta(days=7)) vector_stores = await project.list_vector_stores( pagination_params=PaginationParams(sort_by="created_at"), filter=vector_store_filter, ) same_vector_store = await project.get_vector_store_by_id(vector_store.id) await same_vector_store.delete() ``` *** ## DexFile Represents an uploaded file in Dex. ### Parsing * `parse(params)` - Parse document to structured format (automatically polls for completion) * `start_parse_job(params)` - Start a parse job and return immediately (recommended for async workflows) * `get_download_url()` - Get a temporary download URL for the uploaded file **Example:** ```python theme={null} from dex_sdk.types import ( ParseEngine, IrisParseJobParams, IrisParseEngineOptions, ) iris_params = IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( # default layout="rt_detr_bce" → region-level chunks (closest to layout-aware parsing) # layout="whole_page", # optional: one chunk per page # text_ocr="openai/gpt-5.4", # optional ), ) # Standard parsing (auto-polls for completion) parse_result = await dex_file.parse(iris_params) # max_attempts=100 is optional # Async job workflow (recommended for better control) parse_job = await dex_file.start_parse_job(iris_params) parse_result = await parse_job.get_result() # See Job Monitoring section for how to track job progress ``` ### Working with Parse Results After parsing, you can access the structured content including chunks and blocks. **Example:** ```python theme={null} # Parse the file parse_result = await dex_file.parse(parse_params) # Access metadata metadata = parse_result.data.parse_metadata print(f"Source: {metadata.filename} ({metadata.pages_processed} pages, engine: {parse_result.data.engine})") # Access content chunks for i, chunk in enumerate(parse_result.data.content.chunks): print(f"\nChunk {i}: {chunk.content[:100]}... ({len(chunk.blocks)} blocks)") for block in chunk.blocks: print(f" [{block.type}] Page {block.page_number}, " f"confidence: {block.confidence:.2f}, " f"pos: ({block.bbox.left:.2f}, {block.bbox.top:.2f})") ``` *** ## ParseResult Represents the result of a document parsing operation. ### Extraction * `extract(extraction_schema, model, user_prompt=None, *, system_prompt=None, generate_citations=True, generate_confidence=True, model_kwargs=None)` - Extract structured data with user prompt, schema, model, and options **Parameters:** * `extraction_schema` (BaseModel): Pydantic model class for extraction (pass the class directly, not `model_json_schema()`) * `model` (str): LLM model to use (e.g., "openai/gpt-5.4") * `user_prompt` (str | None): Natural language instructions for extraction * `system_prompt` (str | None): High-level instructions for the extraction model * `generate_citations` (bool): Include source citations in results (default: `True`) * `generate_confidence` (bool): Include confidence scores in results (default: `True`) * `model_kwargs` (dict | None): Additional kwargs for the LLM model Parameters after `user_prompt` are keyword-only. **Returns:** A `DexExtractResult`. Read the extracted fields with `.data.data` and token usage with `.data.usage_info`. When `generate_citations` and `generate_confidence` are both `False`, `extract` returns an instance of `extraction_schema` instead. **Example:** ```python theme={null} from pydantic import BaseModel, Field class InvoiceData(BaseModel): invoice_number: str = Field(description="The invoice number") total_amount: float = Field(description="Total amount in dollars") date: str = Field(description="Invoice date") extract_result = await parse_result.extract( extraction_schema=InvoiceData, user_prompt="Extract invoice details from this document.", model="openai/gpt-5.4", generate_citations=True, generate_confidence=True, ) ``` ### Working with Extraction Results When citations or confidence are enabled, parse-result and vector-store `extract` return extracted fields at `extract_result.data.data`. With both disabled, they return your schema model. See [Extraction Response Shapes](extract#extraction-response-shapes) for extract jobs and direct file extraction. **Example:** ```python theme={null} # Extract data extract_result = await parse_result.extract( extraction_schema=InvoiceData, user_prompt="Extract invoice details from this document.", model="openai/gpt-5.4", generate_citations=True, generate_confidence=True, ) # Access the extraction result result = extract_result.data # Access structured data for field_name, field in result.data.items(): confidence = f"{field.confidence:.2f}" if field.confidence is not None else "unavailable" print(f"{field_name}: {field.value} (confidence: {confidence})") if field.citations: for cite in field.citations: loc = f", pos: ({cite.bbox.left:.2f}, {cite.bbox.top:.2f})" if cite.bbox else "" print(f" → Page {cite.page}: {cite.content[:50]}...{loc}") # Access usage information if result.usage_info: usage = result.usage_info print(f"\nTokens: {usage.total_tokens} total ({usage.prompt_tokens} prompt + {usage.completion_tokens} completion)") ``` *** ## VectorStore Represents a vector store for semantic search and RAG-enhanced extraction. ### Indexing * `add_parse_results(parse_result_ids)` - Add parsed documents to vector store by parse result IDs **Example:** ```python theme={null} # Add parsed documents to vector store await vector_store.add_parse_results([parse_result.id]) ``` ### Search * `search(query, top_k=5, filters=None, *, query_type=None, rerank_config=None)` - Search across all documents in the vector store. `query_type` (`"semantic"`, `"lexical"`, or `"hybrid"`) and `rerank_config` work on SGP Vector Store only * `search_in_file(*, file_id=None, parse_id=None, query, top_k=5, filters=None, query_type=None, rerank_config=None)` - Search within a specific file or parse result with optional filters. All arguments are keyword-only **Example:** ```python theme={null} # Search across all documents results = await vector_store.search( query="What is the total revenue?", top_k=5, ) # Search within a specific file file_results = await vector_store.search_in_file( file_id=dex_file.id, query="What is the total revenue?", top_k=5, filters=None, ) ``` ### Extraction * `extract(extraction_schema, model, user_prompt=None, *, system_prompt=None, generate_citations=True, generate_confidence=True, model_kwargs=None, top_k=5, filters=None, rerank_config=None)` - Extract structured data from entire vector store with RAG context **Example:** ```python theme={null} # Extract from vector store with RAG context extract_result = await vector_store.extract( extraction_schema=FinancialData, user_prompt="Extract financial summary from all documents.", model="openai/gpt-5.4", generate_citations=True, generate_confidence=True, ) ``` *** ## Job Monitoring and SGP Tracing **New in v0.4.0**: Enhanced async job support with SGP tracing integration for better observability and debugging. ### Async Job Workflow Use `start_parse_job()` for better control over async operations and access to SGP traces. **Example:** ```python theme={null} from dex_sdk.types import ( IrisParseJobParams, IrisParseEngineOptions, ParseEngine, JobStatus, ) import asyncio iris_params = IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( # layout="rt_detr_bce" is default (region-level chunks) # layout="whole_page", # one chunk per page ), ) # Start a parse job parse_job = await project.start_parse_job( dex_file=dex_file, parameters=iris_params ) # Monitor job progress while parse_job.data.status not in [ JobStatus.SUCCEEDED, JobStatus.PARTIALLY_SUCCEEDED, JobStatus.FAILED, JobStatus.CANCELLED, ]: await asyncio.sleep(1) await parse_job.refresh() # Get the result parse_result = await parse_job.get_result() ``` ### Retrieving SGP Traces Jobs are now connected to SGP traces for end-to-end observability. You can retrieve complete trace data for any job. **Example:** ```python theme={null} from scale_gp_beta import SGPClient # Initialize SGP client sgp_client = SGPClient( api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"), ) # Search for spans by job ID spans = [span for span in sgp_client.spans.search( sort_by="created_at", sort_order="desc", extra_metadata={"job_id": parse_job.data.id}, parents_only=True, )] # Get full trace if spans: trace_id = spans[0].trace_id all_spans = [span for span in sgp_client.spans.search(trace_ids=[trace_id])] # Analyze trace data for span in all_spans: print(f"Span: {span.name}, Duration: {span.duration_ms}ms") ``` **Benefits:** * **Better observability**: Track jobs through SGP's tracing infrastructure * **Easier debugging**: Access detailed execution traces for failed jobs * **Performance monitoring**: Analyze job performance and identify bottlenecks * **Request correlation**: Connect job execution to API requests and traces *** ## Parse Job Parameters When parsing documents, you can specify different engines and options to customize the parsing behavior. ### Iris Parse Parameters **IrisParseJobParams** - Parameters for the Iris OCR engine (Default engine). **Fields:** * `engine` (ParseEngine): Set to `ParseEngine.IRIS` * `options` (IrisParseEngineOptions): Parsing options * `vector_store_metadata` (dict | None): Metadata to populate into the vector store for filtering when searching. Keys must match the `vector_store_metadata_schema` of the target vector store. **IrisParseEngineOptions:** | Field | Type | Default (if omitted) | Description | | - | - | - | - | | `model_gateway` | `"sgp"` | `"sgp"` | Uses your SGP account and API key. Available in SDK `0.8.2` and later. | | `layout` | `"rt_detr_bce"` \| `"pp_doclayout_v3"` \| `"whole_page"` \| `null` | `"rt_detr_bce"` | Layout detector. `rt_detr_bce` and `pp_doclayout_v3` → one chunk per detected region; `whole_page` → one chunk per page | | `text_ocr` | `str` \| `null` | `"openai/gpt-5.4"` | SGP model ID for text regions | | `table_ocr` | `str` \| `null` | `"openai/gpt-5.4"` | SGP model ID for table regions | | `text_prompt` | `str` \| `null` | built-in default | User prompt for text extraction (VLMs) | | `text_system_prompt` | `str` \| `null` | built-in default | System prompt for text extraction (VLMs) | | `table_prompt` | `str` \| `null` | built-in default | User prompt for table extraction (VLMs) | | `model_parameters` | `dict` \| `null` | — | Extra LLM inference args (e.g. `max_tokens`, `temperature`, `reasoning_effort` \[`"low"` \| `"medium"` \| `"high"` \| `null`]); merged into SGP requests | | `confidence_threshold` | `float` \| `null` | `0.4` | Min confidence for **text** layout boxes | | `table_confidence_threshold` | `float` \| `null` | `0.5` | Min confidence for **table** layout boxes | | `image_confidence_threshold` | `float` \| `null` | `0.5` | Min confidence for **image** layout boxes | | `containment_threshold` | `float` \| `null` | `0.8` | Drop nested boxes when the smaller box is this fraction inside a larger one | | `strict_containment_filter` | `bool` \| `null` | `true` | If `true`, drop any contained box; if `false`, only drop same-label overlaps | | `img_method` | `"description"` \| `"base64"` \| `"skip"` \| `null` | `"description"` | How image regions are emitted: LLM description, inline base64, or omit | | `img_description_prompt` | `str` \| `null` | built-in default | Prompt when `img_method="description"` | | `image_description_model` | `str` \| `null` | `"openai/gpt-5.4"` | SGP model for image descriptions | **Example:** ```python theme={null} from dex_sdk.types import ( ParseEngine, IrisParseJobParams, IrisParseEngineOptions, ) parse_result = await dex_file.parse( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( layout="rt_detr_bce", # or "whole_page" text_ocr="openai/gpt-5.4", confidence_threshold=0.5, ), ) ) ``` ### Reducto Parse Parameters (Legacy) **ReductoParseJobParams** - Parameters for the Reducto OCR engine. **Best for:** English and Latin-script documents (Spanish, French, German, Italian, Portuguese, etc.) with tables, figures, and complex layouts. **Fields:** * `engine` (ParseEngine): Set to `ParseEngine.REDUCTO` * `options` (ReductoParseEngineOptions): Parsing options * `advanced_options` (dict): Advanced options for fine-tuning * `experimental_options` (dict): Experimental features * `priority` (bool): Whether to prioritize this job (default: False) * `vector_store_metadata` (dict | None): Metadata to populate into the vector store for filtering when searching. Keys must match the `vector_store_metadata_schema` of the target vector store. **ReductoParseEngineOptions:** * `chunking` (ReductoChunkingOptions | None): Chunking configuration **ReductoChunkingOptions:** * `chunk_mode` (ReductoChunkingMethod): Chunking method (default: `VARIABLE`) * `DISABLED`: No chunking * `BLOCK`: Block-level chunks * `PAGE`: Page-level chunks * `PAGE_SECTIONS`: Page sections * `SECTION`: Section-level chunks * `VARIABLE`: Variable-size chunks based on content * `chunk_size` (int | None): Custom chunk size **Example:** ```python theme={null} # (Legacy) Reducto parse parameters from dex_sdk.types import ( ParseEngine, ReductoParseJobParams, ReductoChunkingMethod, ReductoChunkingOptions, ReductoParseEngineOptions, ) parse_params = ReductoParseJobParams( engine=ParseEngine.REDUCTO, options=ReductoParseEngineOptions( chunking=ReductoChunkingOptions( chunk_mode=ReductoChunkingMethod.VARIABLE, chunk_size=None, ) ), priority=False, ) parse_result = await dex_file.parse(parse_params) ``` *** ## Common Types This section documents the core data models and types used throughout the Dex SDK. ### Type Categories **Importable Types** - Types you can import from `dex_sdk.types` to configure your requests: * Configuration types (ProjectConfiguration, RetentionPolicy) * Pagination types (PaginationParams, FileListFilter, JobListFilter, ParseResultListFilter, ExtractionListFilter, VectorStoreListFilter) - **New in v0.4.0** * Parse parameter types (IrisParseJobParams, ReductoParseJobParams, etc.) * Enum types (ParseEngine, ReductoChunkingMethod, VectorStoreEngines, JobStatus) - **JobStatus new in v0.4.0** **Response Types** - Types returned by the SDK, accessible via the `.data` attribute on wrapper objects: * When you call SDK methods, you get wrapper objects (DexProject, DexFile, DexParseResult, etc.) * Access the underlying data via `.data`: `project.data.id`, `file.data.filename` * These entities are automatically validated but don't need to be imported *** ### Configuration Types #### ProjectConfiguration Configuration options for a Dex project. **Import:** `from dex_sdk.types import ProjectConfiguration` **Fields:** * `retention` (RetentionPolicy | None): Data retention policy for the project **Example:** ```python theme={null} from datetime import timedelta from dex_sdk.types import ProjectConfiguration, RetentionPolicy config = ProjectConfiguration( retention=RetentionPolicy( files=timedelta(days=30), result_artifacts=timedelta(days=7), ) ) ``` #### RetentionPolicy Defines data retention periods for automatic cleanup of files and processing artifacts. **Import:** `from dex_sdk.types import RetentionPolicy` **Fields:** * `files` (timedelta | None): Retention period for uploaded files. Files older than this period are automatically deleted. If `None`, files are retained indefinitely. * `result_artifacts` (timedelta | None): Retention period for parse results, extraction results, and job artifacts. If `None`, artifacts are retained indefinitely. **Example:** ```python theme={null} from datetime import timedelta from dex_sdk.types import RetentionPolicy # 30-day file retention, 7-day artifact retention policy = RetentionPolicy( files=timedelta(days=30), result_artifacts=timedelta(days=7), ) # Keep files indefinitely, but clean up artifacts after 14 days policy = RetentionPolicy( files=None, result_artifacts=timedelta(days=14), ) ``` **Use Cases:** * **Compliance**: Meet regulatory requirements (GDPR, HIPAA, etc.) * **Cost Management**: Automatically clean up old data to reduce storage costs * **Security**: Limit exposure of sensitive documents by enforcing retention limits > **Note**: The retention period is calculated from the creation time of the file or artifact. Retention policies can be updated at any time using `update_project()`. #### PaginationParams **New in v0.4.0**: Parameters for paginated list operations. **Import:** `from dex_sdk.types import PaginationParams` **Fields:** * `page_size` (int | None): Number of items to return per page (default: 20, min: 1, max: 100) * `sort_by` (str | None): Field name to sort by (e.g., `"created_at"`) * `sort_order` (str | None): Sort order, either `"asc"` or `"desc"` (default: `"asc"`) * `continuation_token` (str | None): Token for fetching next/previous page **Example:** ```python theme={null} from dex_sdk.types import PaginationParams pagination_params = PaginationParams( page_size=10, sort_by="created_at", sort_order="desc", continuation_token=None # Set to next_token from previous response for pagination ) files = await project.list_files(pagination_params=pagination_params) ``` #### List Filter Types **New in v0.4.0**: Filter types for list operations on different entity types. **Import:** `from dex_sdk.types import FileListFilter, JobListFilter, ParseResultListFilter, ExtractionListFilter, VectorStoreListFilter` **Common Fields:** * `created_at_start` (datetime | None): Filter for entities created after this time * `created_at_end` (datetime | None): Filter for entities created before this time **Example:** ```python theme={null} from dex_sdk.types import FileListFilter, JobListFilter, PaginationParams from datetime import datetime, timedelta pagination_params = PaginationParams(sort_by="created_at") # Filter files created in the last 24 hours file_filter = FileListFilter( created_at_start=datetime.now() - timedelta(days=1) ) files = await project.list_files(pagination_params=pagination_params, filter=file_filter) # Filter jobs created in the last week job_filter = JobListFilter( created_at_start=datetime.now() - timedelta(days=7) ) jobs = await project.list_jobs(pagination_params=pagination_params, filter=job_filter) ``` **Available Filter Types:** * `FileListFilter` - Filter uploaded files * `JobListFilter` - Filter jobs * `ParseResultListFilter` - Filter parse results * `ExtractionListFilter` - Filter extractions * `VectorStoreListFilter` - Filter vector stores #### ExtractionParameters Parameters for extraction operations. **Import:** `from dex_sdk.types import ExtractionParameters` **Fields:** * `model` (str): LLM model to use (e.g., `"openai/gpt-5.4"`) * `model_kwargs` (dict | None): Additional kwargs for the LLM model * `extraction_schema` (dict): JSON schema defining the desired output structure * `system_prompt` (str | None): High-level instructions for the extraction model * `user_prompt` (str | None): Specific hints about the current document * `generate_citations` (bool): Whether to return bounding boxes for extracted values (default: True) * `generate_confidence` (bool): Whether to return confidence scores (default: True) *** ### Parse Configuration Types #### ParseEngine Enum of available OCR engines. **Import:** `from dex_sdk.types import ParseEngine` **Values:** * `IRIS` = "iris" * `REDUCTO` = "reducto" * `CUSTOM` = "custom" > [!WARNING] > > **Deprecation notice:** Reducto is being deprecated. Use Iris for new projects. #### ReductoParseJobParams (Deprecated) Parameters for the Reducto OCR engine. **Import:** `from dex_sdk.types import ReductoParseJobParams` See the [Parse Job Parameters](#parse-job-parameters) section for detailed usage. #### IrisParseJobParams Parameters for the Iris OCR engine. **Import:** `from dex_sdk.types import IrisParseJobParams` See the [Parse Job Parameters](#parse-job-parameters) section for detailed usage. #### ReductoChunkingMethod (Deprecated) Enum of chunking methods for Reducto parser. **Import:** `from dex_sdk.types import ReductoChunkingMethod` **Values:** * `DISABLED` = "disabled" * `BLOCK` = "block" * `PAGE` = "page" * `PAGE_SECTIONS` = "page_sections" * `SECTION` = "section" * `VARIABLE` = "variable" #### ReductoChunkingOptions (Deprecated) Chunking configuration for Reducto parser. **Import:** `from dex_sdk.types import ReductoChunkingOptions` **Fields:** * `chunk_mode` (ReductoChunkingMethod): Chunking method * `chunk_size` (int | None): Custom chunk size #### ReductoParseEngineOptions (Deprecated) Options for Reducto parser. **Import:** `from dex_sdk.types import ReductoParseEngineOptions` **Fields:** * `chunking` (ReductoChunkingOptions | None): Chunking configuration #### IrisParseEngineOptions Options for Iris parser. **Import:** `from dex_sdk.types import IrisParseEngineOptions` **Fields:** * `layout` (str | None): Layout detection model * `text_ocr` (str | None): Text OCR model * `table_ocr` (str | None): Table OCR model * `text_prompt` (str | None): Custom prompt for text extraction * `table_prompt` (str | None): Custom prompt for table extraction * `left_to_right` (bool | None): Sort regions left-to-right * `confidence_threshold` (float | None): Minimum confidence threshold * `containment_threshold` (float | None): Containment threshold for filtering *** ### Vector Store Types #### VectorStoreEngines Enum of available vector store engines. Prefer `SGP_VECTOR_STORE` with SDK `0.4.3` or later and an SGP deployment that supports SGP Vector Store. Use the legacy `SGP_KNOWLEDGE_BASE` engine on deployments that don't support SGP Vector Store. See [Vector Store Engines](vector-stores#vector-store-engines) for embedding options. **Import:** `from dex_sdk.types import VectorStoreEngines` **Values:** * `SGP_VECTOR_STORE` = "sgp_vector_store" — SGP Vector Store; available in SDK `0.4.3` or later * `SGP_KNOWLEDGE_BASE` = "sgp_knowledge_base" — legacy Knowledge Base #### VectorStoreSearchResult Result from vector store search operations containing matching chunks. **Import:** `from dex_sdk.types import VectorStoreSearchResult` **Fields:** * `chunks` (list\[VectorStoreChunk]): List of matching chunks with relevance scores **Example:** ```python theme={null} # Perform search search_results = await vector_store.search( query="What is the total revenue?", top_k=5, ) # Access chunks for chunk in search_results.chunks: print(f"Score: {chunk.score:.2f}") print(f"Content: {chunk.content[:100]}...") print(f"File ID: {chunk.file_id}") ``` #### VectorStoreChunk Represents a single chunk returned from vector store search operations. **Fields:** * `content` (str): Text content of the chunk * `score` (float): Relevance score for the search query * `file_id` (str | None): ID of the file this chunk belongs to * `parse_result_id` (str | None): ID of the parse result this chunk belongs to * `metadata` (dict\[str, Any] | None): Additional metadata from the chunk, which may include information like chunk indices, embeddings metadata, or other custom fields added during indexing * `blocks` (list): List of block objects with layout information **Example:** ```python theme={null} # Access chunk details for chunk in search_results.chunks: print(f"Score: {chunk.score:.2f}") print(f"Content: {chunk.content[:100]}...") print(f"File: {chunk.file_id}, Parse Result: {chunk.parse_result_id}") # Access additional metadata if chunk.metadata: print(f"Chunk metadata: {chunk.metadata}") # Access block layout information for block in chunk.blocks: print(f" Page {block.page_number}, type: {block.type}") ``` *** ### Response Entity Types Project, file, parse, and vector-store methods return wrapper objects with a `.data` attribute. Extraction responses depend on the method and metadata flags; see [Extraction Response Shapes](extract#extraction-response-shapes). You typically don't need to import these entity types directly. #### Working with Response Data Use `.data` to read project, file, and parse entity fields: ```python theme={null} # Create a project project = await dex_client.create_project(name="My Project") print(f"Project: {project.data.name} ({project.data.id}) created at {project.data.created_at}") # Upload a file dex_file = await project.upload_file("document.pdf") print(f"File: {dex_file.data.filename} ({dex_file.data.size_bytes} bytes) → {dex_file.data.id}") # Parse a file parse_result = await dex_file.parse(parse_params) metadata = parse_result.data.parse_metadata print(f"Parsed: {metadata.pages_processed} pages with {parse_result.data.engine} → {parse_result.data.id}") ``` #### Common Response Entity Fields **ProjectEntity** (accessed via `project.data`): * `id` (str): Project ID with `proj_` prefix * `name` (str): Project readable name * `status` (str): Project status (`"active"` or `"archived"`) * `configuration` (ProjectConfiguration | None): Project configuration * `created_at` (datetime): When the project was created * `archived_at` (datetime | None): When the project was archived **FileEntity** (accessed via `dex_file.data`): * `id` (str): File ID with `file_` prefix * `project_id` (str): Project ID that the file belongs to * `filename` (str): Original filename * `size_bytes` (int): File size in bytes * `mime_type` (str): MIME type of the file * `status` (str): Current file status * `created_at` (datetime): When the file was uploaded **ParseResultEntity** (accessed via `parse_result.data`): * `id` (str): Parse result ID with `pres_` prefix * `project_id` (str): Project ID * `source_document_id` (str): Source document ID that was parsed * `engine` (str): Engine used for parsing * `parse_metadata` (object): Metadata including `filename`, `pages_processed` * `content` (object): Parsed content with `chunks` * `created_at` (datetime): When the parse result was created **DexExtractResult**: * Parse-result and vector-store `.extract()` with metadata enabled: `data` is an `ExtractionResult`; fields are at `data.data` and token usage at `data.usage_info` when available. * Extract-job `.get_result()` with metadata enabled, or direct file `.extract()` with confidence enabled: `data` is an `ExtractionEntity`; `result` aliases `data.result`. Fields are at `result.data` and token usage at `result.usage_info` when available. With both metadata flags disabled, parse-result and vector-store convenience `.extract()` calls return the schema model. Direct file `.extract()` returns the schema model when confidence is disabled. Raw extract-job results have a separate dictionary shape; see [Extraction Response Shapes](extract#extraction-response-shapes). **ExtractionEntity** (accessed via `.data` on a `DexExtractResult` from extract-job `.get_result()` or direct file `.extract()`; `DexExtractResult.result` is shorthand for `.data.result`): * `id` (str): Extraction result ID * `source_id` (str): Source ID that was extracted from * `result` (object): The extraction result with `data` and `usage_info` * `parameters` (ExtractionParameters): Parameters used for extraction * `created_at` (datetime): When the extraction was completed * `processing_time_ms` (int | None): Processing time in milliseconds **VectorStoreEntity** (accessed via `vector_store.data`): * `id` (str): Vector store ID with `vector_store_` prefix * `project_id` (str): Project ID * `name` (str): Name of the vector store * `engine` (str): Engine used for vector store * `created_at` (datetime): When the vector store was created *** ### Deprecated Types > [!WARNING] > > **Deprecation notice:** `ProjectCredentials` and `SGPCredentials` are deprecated as of version 0.3.2. Pass credentials to the `DexClient` constructor instead. See the version 0.3.2 entry in the [Changelog](/docs/capabilities/changelog) for migration instructions. *** ## Error Handling The SDK raises `DexAPIError` (a subclass of `httpx.HTTPStatusError`) when the API returns an error status, `ValueError` when a job fails or is cancelled, and `TimeoutError` when polling reaches `max_attempts`. For detailed troubleshooting guidance, see the **[Troubleshooting Guide](troubleshooting)**. ```python theme={null} from dex_sdk import DexAPIError try: parse_result = await dex_file.parse(...) except DexAPIError as e: print(f"Error: {e}") ``` *** ## Async/Await Pattern The Dex SDK is fully async. Use `await` with all SDK methods: ```python theme={null} import asyncio import os async def main(): # Initialize client with credentials dex_client = DexClient( base_url="https://dex.sgp.scale.com", api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"), ) project = await dex_client.create_project(name="My Project") dex_file = await project.upload_file("document.pdf") parse_result = await dex_file.parse(...) extract_result = await parse_result.extract(...) # Run in Jupyter/IPython await main() # Run in regular Python script asyncio.run(main()) ``` *** ## See Also * **[Getting Started Guide](getting-started-with-dex)**: Quick start tutorial for beginners * **[File Management](file-management)**: Upload, pagination, and file types * **[Parse](parse)**: Parse engines and async job monitoring * **[Chunking](chunking)**: Chunking strategies * **[Vector Stores](vector-stores)**: Semantic search and RAG-enhanced extraction * **[Extract](extract)**: Data extraction and batch processing * **[Best Practices](best-practices)**: Quick start, retention policies, and optimization * **[Introduction to Dex](introduction-to-dex)**: Core concepts and architecture overview * **[Troubleshooting Guide](troubleshooting)**: Common issues and solutions # Extract Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/extract Extract structured data from parse results or vector stores using LLM-powered extraction. Extract structured data from documents using Pydantic schemas and LLM models. Extract from single parse results or use [vector stores](vector-stores#rag-enhanced-extraction) for RAG-enhanced extraction on large documents. *** ## Extraction Response Shapes The response shape depends on the method and metadata flags. Dex SDK `0.5.1`–`0.8.x` uses these shapes: | Method | Metadata flags | Read extracted fields | Read usage, when present | | - | - | - | - | | Parse-result or vector-store `.extract()` | Citations or confidence enabled | `extract_result.data.data` | `extract_result.data.usage_info` | | Parse-result or vector-store `.extract()` | Both flags disabled | Schema model; use `.model_dump()` | Not returned on the schema model | | Extract-job `.get_result()` | Citations or confidence enabled | `extract_result.result.data` | `extract_result.result.usage_info` | | Direct file `.extract()` | Confidence enabled | `extract_result.result.data` | `extract_result.result.usage_info` | | Direct file `.extract()` | Confidence disabled | Schema model; use `.model_dump()` | Not returned on the schema model | | Job `.wait_for_success(JobOperationType.EXTRACT)` | Either setting | Raw dictionary: `raw_result["data"]` | `raw_result["usage_info"]` | Structured field maps contain cells with `value`, `citations`, and `confidence`. Extract-job `.get_result()` wraps an extraction entity, so `.result` is an alias for `.data.result`. Parse-result and vector-store convenience `.extract()` wrap the extraction payload directly; use `.data.data` for those metadata responses. Raw job results preserve the server's dictionary, as shown in [Batch Extraction](#batch-extraction). With metadata disabled, its `data` contains plain schema values. The schema-model convenience response does not expose usage metadata. *** ## Extract from Parse Result ```python theme={null} extract_result = await parse_result.extract( extraction_schema=MySchema, user_prompt="Extract the data", model="openai/gpt-5.4", generate_citations=True, generate_confidence=True, ) ``` *** ## Extract from Vector Store For large documents or multi-file analysis, use [RAG-enhanced extraction](vector-stores#rag-enhanced-extraction) from a vector store. This improves accuracy by retrieving relevant chunks before extraction. ```python theme={null} result = await vector_store.extract( extraction_schema=MySchema, user_prompt="Extract from all documents", model="openai/gpt-5.4", ) ``` *** ## Extract from File (Direct) Extract directly from an uploaded file, skipping the parse step. Direct file extraction is available in SDK `0.5.0` or later; selecting page ranges requires SDK `0.7.1` or later. **When to use:** structured extraction from images or small documents (≈1–5 pages) where visual structure is vital for accurate extraction — passports, ID cards, forms, invoices, and similar. For text-dense or large documents, prefer parse-then-extract. ```python theme={null} from dex_sdk.types import PageRange result = await file.extract( extraction_schema=MySchema, user_prompt="Extract the fields", model="openai/gpt-5.4", generate_confidence=True, pages=[PageRange(start=1, end=3)], # Optional: inclusive, 1-based ranges ) for field_name, field in result.result.data.items(): print(f"{field_name}: {field.value}") ``` Supports PDF, image (PNG, JPG, WebP), and office (DOCX, DOC, PPTX, PPT) files. Omit `pages` to use the whole document. Citations are not supported in this mode. Parsing has no page-range parameter. *** ## Extract with Citations ```python theme={null} result = await parse_result.extract( extraction_schema=Schema, user_prompt="Extract data", model="openai/gpt-5.4", generate_citations=True, # Get source locations ) for field_name, field in result.data.data.items(): if field.citations: print(f"{field_name} found on page {field.citations[0].page}") ``` *** ## Filter Low Confidence Results ```python theme={null} result = await parse_result.extract( extraction_schema=Schema, user_prompt="Extract data", model="openai/gpt-5.4", generate_confidence=True, ) high_confidence = { k: v for k, v in result.data.data.items() if v.confidence and v.confidence > 0.8 } ``` *** ## Batch Extraction Start extract jobs for multiple documents, then monitor and collect results. This pattern gives you control over job status and lets you handle failures per document. `wait_for_success(JobOperationType.EXTRACT)` returns the raw result dictionary, with extracted `data` and, when present, `usage_info`. ```python theme={null} import asyncio from dex_core.models.jobs import JobOperationType # Start extract jobs for all parse results (returns immediately) extract_parameters = ExtractionParameters( model="openai/gpt-5.4", extraction_schema=InvoiceData.model_json_schema(), user_prompt="Extract invoice details.", generate_citations=True, generate_confidence=True, ) extract_jobs = [ await parse_result.start_extract_job(extract_parameters) for parse_result in parse_results ] # Monitor jobs until all complete while any( job.data.status not in [ JobStatus.SUCCEEDED, JobStatus.PARTIALLY_SUCCEEDED, JobStatus.FAILED, JobStatus.CANCELLED, ] for job in extract_jobs ): await asyncio.sleep(1) await asyncio.gather(*[job.refresh() for job in extract_jobs]) # Collect results from successful jobs extractions = [] for i, job in enumerate(extract_jobs): if job.data.status == JobStatus.SUCCEEDED: extraction = await job.wait_for_success(JobOperationType.EXTRACT) extractions.append(extraction) print(f"Document {i+1}: {extraction['data']}") else: print(f"Document {i+1} failed") ``` To wait for all jobs and collect raw results in one call, reuse `extract_parameters` and the imports above with `asyncio.gather`: ```python theme={null} # Start all jobs in parallel extract_jobs = await asyncio.gather(*[ parse_result.start_extract_job(extract_parameters) for parse_result in parse_results ]) # Wait for all to complete and collect results extractions = await asyncio.gather(*[ job.wait_for_success(JobOperationType.EXTRACT) for job in extract_jobs ]) for i, extraction in enumerate(extractions): print(f"Document {i+1}: {extraction['data']}") ``` *** ## Appendix: Essential Imports ```python theme={null} from pydantic import BaseModel, Field from dex_core.models.jobs import JobOperationType from dex_sdk.types import ExtractionParameters, JobStatus ``` *** ## Next Steps * **[Vector Stores](vector-stores)**: RAG-enhanced extraction for large documents * **[Best Practices](best-practices)**: Optimize extraction accuracy * **[API Reference](dex-sdk-api-reference)**: Complete SDK documentation # File Management Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/file-management Upload, list, and manage files in Dex projects with pagination and filtering. Manage files, projects, and collections in Dex. The same pagination and filtering patterns apply to jobs, parse results, extractions, and vector stores. *** ## Create Project ```python theme={null} import os from dex_sdk import DexClient dex_client = DexClient( base_url="https://dex.sgp.scale.com", api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"), ) project = await dex_client.create_project(name="My Project") ``` ### Create Project with Retention ```python theme={null} project = await dex_client.create_project( name="My Project", configuration=ProjectConfiguration( retention=RetentionPolicy( files=timedelta(days=30), result_artifacts=timedelta(days=7), ) ) ) ``` *** ## Upload File ```python theme={null} dex_file = await project.upload_file("document.pdf") ``` ### Process Multiple Files ```python theme={null} import asyncio files_to_upload = ["doc1.pdf", "doc2.pdf", "doc3.pdf"] dex_files = await asyncio.gather(*[project.upload_file(f) for f in files_to_upload]) print(f"Uploaded {len(dex_files)} files") ``` *** ## Pagination and Filtering **New in v0.4.0**: Efficiently manage large collections with pagination and filtering. ### Listing Files with Pagination ```python theme={null} pagination_params = PaginationParams( page_size=50, sort_by="created_at", sort_order="desc", continuation_token=None ) result = await project.list_files(pagination_params=pagination_params) # Access items for file in result.items: print(f"File: {file.data.filename} ({file.data.created_at})") # Paginate through remaining results while result.next_token: pagination_params.continuation_token = result.next_token result = await project.list_files(pagination_params=pagination_params) # Process result.items ``` ### Filtering by Creation Time ```python theme={null} from datetime import datetime, timedelta # Filter files created in the last 24 hours file_filter = FileListFilter( created_at_start=datetime.now() - timedelta(days=1) ) recent_files = await project.list_files( pagination_params=PaginationParams(sort_by="created_at"), filter=file_filter, ) # Combine pagination and filtering for files pagination_params = PaginationParams(page_size=20, sort_by="created_at", sort_order="desc") file_filter = FileListFilter(created_at_start=datetime.now() - timedelta(days=30)) recent_files = await project.list_files( pagination_params=pagination_params, filter=file_filter ) ``` ### Available List Operations All entity types support unified `list_*` operations with pagination and filtering: * `list_files(pagination_params, filter)` - List uploaded files * `list_jobs(pagination_params, filter)` - List all jobs * `list_parse_results(pagination_params, filter)` - List parse results * `list_extractions(pagination_params, filter)` - List extractions * `list_vector_stores(pagination_params, filter)` - List vector stores *** ## Supported File Types Dex supports a wide variety of document formats: ### Images PNG, JPEG/JPG, GIF, BMP, TIFF, PCX, PPM, APNG, PSD, CUR, DCX, FTEX, PIXAR, HEIC ### Documents PDF, DOCX, DOC, DOTX, WPD, TXT, RTF, PPTX, PPT ### Spreadsheets CSV, XLSX, XLSM, XLS, XLTX, XLTM, QPW > [!NOTE] > > For best results with spreadsheets, use XLSX format. CSV files are processed as-is without layout analysis. *** ## Access Response Data SDK methods return **wrapper objects**; access data via `.data`: ```python theme={null} # Correct project_id = project.data.id project_name = project.data.name dex_file.data.id dex_file.data.filename dex_file.data.size_bytes ``` *** ## Appendix: Essential Imports ```python theme={null} from dex_sdk.types import ( PaginationParams, FileListFilter, ProjectConfiguration, RetentionPolicy, ) from datetime import timedelta ``` *** ## Next Steps * **[Parse](parse)**: Parse documents with IRIS or Reducto * **[Vector Stores](vector-stores)**: Create vector stores for semantic search * **[API Reference](dex-sdk-api-reference)**: Complete SDK documentation # Getting Started with Dex Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/getting-started-with-dex This guide will help you get started with Scale's Dex service - a document understanding capability that extracts accurate, structured information from unstructured documents. ## Overview Dex is Scale's document understanding capability that provides composable primitives for: * **File Management** - Secure file upload, storage, and retrieval * **Document Parsing** - Convert any document (PDFs, DOCX, images, etc.) into structured JSON format with multiple OCR engines * **Vector Stores** - Embed, index and search parsed document corpora * **Data Extraction** - Extract specific information using custom schemas, prompts, and RAG-enhanced context * **Project Management** - Organize and isolate data with proper credential management and authorization ## Prerequisites Before using Dex, ensure you have: * ✅ A valid Scale account with SGP (Scale General Platform) access * ✅ Your SGP account ID and API key set as environment variables: ```bash theme={null} export SGP_ACCOUNT_ID="your_account_id" export SGP_API_KEY="your_api_key" ``` * ✅ Python 3.12+ installed * ✅ Dex SDK installed (see Installation section) ## Installation ### Install the Dex SDK Your Scale team provides the `dex-sdk` package (version 0.8.2 or higher), together with the `dex-core` and `data-sources-core` packages it depends on. Once you have them, install all three with pip in your Python 3.12+ environment: ```bash theme={null} pip install dex_sdk-0.8.2-py3-none-any.whl dex_core-0.8.2-py3-none-any.whl data_sources_core-0.2.4-py3-none-any.whl ``` If your Scale team gives you a package index URL instead, install with `pip install dex-sdk==0.8.2 --index-url `. This will install all required dependencies including Pydantic. > **Note**: The examples in this guide target SDK 0.8.2, which uses SGP API key and account ID authentication. Direct file extraction and extract jobs return a different response shape from the extraction examples below; see [Extraction Response Shapes](extract#extraction-response-shapes). Version 0.4.0 introduced pagination, filtering, and enhanced async job support. See the [Changelog](/docs/capabilities/changelog) for details. ## Quick Start ### 1. Initialize Dex Client ```python theme={null} import os from pydantic import BaseModel, Field from dex_sdk import DexClient from dex_sdk.types import ParseEngine, IrisParseJobParams, IrisParseEngineOptions # Initialize the Dex client with SGP credentials dex_client = DexClient( base_url="https://dex.sgp.scale.com", api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"), ) ``` ### 2. Create a Project Projects isolate your data and credentials for tracing, billing, and SGP model calls. Every operation is tied to a project. ```python theme={null} # Create a project project = await dex_client.create_project(name="My Dex Project") print(f"Created project: {project.data.id}") ``` > **Tip**: Keep one project per use case or group of related files for clean traceability. ### 3. Upload a Document Upload your document to the project. Dex supports PDFs, images, spreadsheets, and more. ```python theme={null} # Upload a file to the project dex_file = await project.upload_file("path/to/your/document.pdf") print(f"Uploaded: {dex_file.data.filename} → {dex_file.data.id}") ``` **Supported file types:** PDF, PNG, JPEG, DOCX, XLSX, CSV, and many more. See [File Management](file-management#supported-file-types) for the complete list. ### 4. Parse the Document Parse converts your document into a structured format with text, tables, and layout information. ```python theme={null} # Parse the document with Iris V2 parse_result = await dex_file.parse( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( layout="rt_detr_bce", text_ocr="openai/gpt-5.4", table_ocr="openai/gpt-5.4" ), ) ) print(f"Parsed: {parse_result.data.parse_metadata.pages_processed} pages") ``` > **Note**: Parsing is asynchronous. The SDK automatically polls for completion. ### 5. Extract Structured Data Define a schema and extract specific information from your document. ```python theme={null} # Define your extraction schema using Pydantic class InvoiceData(BaseModel): """Schema for invoice extraction""" invoice_number: str = Field(description="The invoice number") total_amount: float = Field(description="Total amount in dollars") date: str = Field(description="Invoice date in YYYY-MM-DD format") vendor_name: str = Field(description="Name of the vendor") # Extract data with a clear prompt extract_result = await parse_result.extract( extraction_schema=InvoiceData, user_prompt="Extract invoice details from this document.", model="openai/gpt-5.4", generate_citations=True, generate_confidence=True, ) # Access the extracted data result = extract_result.data for field_name, field in result.data.items(): confidence = f"{field.confidence:.2f}" if field.confidence is not None else "unavailable" print(f"{field_name}: {field.value} (confidence: {confidence})") ``` *** ## Complete Example Here's a complete working example you can copy and run: ```python theme={null} import os import asyncio from pydantic import BaseModel, Field from dex_sdk import DexClient from dex_sdk.types import ParseEngine, IrisParseJobParams, IrisParseEngineOptions class InvoiceData(BaseModel): invoice_number: str = Field(description="The invoice number") total_amount: float = Field(description="Total amount in dollars") async def main(): # 1. Initialize client dex_client = DexClient( base_url="https://dex.sgp.scale.com", api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"), ) # 2. Create project project = await dex_client.create_project(name="Invoice Processing") # 3. Upload document dex_file = await project.upload_file("invoice.pdf") # 4. Parse document parse_result = await dex_file.parse( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( layout="rt_detr_bce", text_ocr="openai/gpt-5.4", table_ocr="openai/gpt-5.4" ), ) ) # 5. Extract data extract_result = await parse_result.extract( extraction_schema=InvoiceData, user_prompt="Extract invoice number and total amount.", model="openai/gpt-5.4", generate_citations=True, ) print(extract_result.data.data) # Run the example asyncio.run(main()) ``` *** ## Next Steps Now that you've completed the basics, explore these topics: ### Learn More * **[File Management](file-management)**: Upload, pagination, and supported file types * **[Parse](parse)**: Parse engines and async job monitoring * **[Chunking](chunking)**: Chunking strategies for documents * **[Vector Stores](vector-stores)**: Semantic search and RAG-enhanced extraction * **[Extract](extract)**: Batch extraction and extraction patterns * **[Best Practices](best-practices)**: Quick start, retention policies, and optimization ### Deep Dive into the API * **[API Reference](dex-sdk-api-reference)**: Complete SDK documentation with all methods and types * **[Troubleshooting Guide](troubleshooting)**: Common issues and solutions * **[Changelog](/docs/capabilities/changelog)**: Latest updates and breaking changes ### Additional Resources * **REST API**: For non-Python integrations, see the [API Reference](api-reference) * **Support**: Questions? Ask your Scale team * **Examples**: More examples in the [Introduction guide](introduction-to-dex#common-use-cases) *** ## Common Questions **Q: How do I process multiple documents?** A: Upload multiple files to the same project, parse each one, then optionally use vector stores for cross-document search. See [Extract](extract#batch-extraction). **Q: Does the SDK provide a synchronous client?** A: The current SDK is async-only. Use `DexClient` with `await`, or `async with DexClient(...)` to reuse a single HTTP connection across multiple calls. **Q: How do I configure data retention policies?** A: Set retention policies when creating a project. See [Best Practices](best-practices#data-retention-policies). **Q: What OCR engines are available?** A: Iris, the recommended engine, and Reducto, which is being deprecated. See **[When to choose Iris?](/docs/capabilities/ocr/when-to-choose-iris)**. **Q: How do I list files with pagination?** (New in v0.4.0) A: Use `PaginationParams` with `list_files()` to control page size and sorting. See [API Reference](dex-sdk-api-reference#file-operations). **Q: How do I monitor async jobs?** (New in v0.4.0) A: Use `start_parse_job()` for better control and access to SGP traces. See [API Reference](dex-sdk-api-reference#job-monitoring-and-sgp-tracing). # Indexing Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/indexing Build a hierarchical index over your project's files for fast BM25 search and tree navigation. An index organizes your project's files into a searchable tree. Leaf nodes (level 0) represent individual files, and higher-level nodes (level 1+) hold generated summaries that group related files—like folders. You can run fast lexical (BM25) search over the files, or navigate the hierarchy level by level. > [!NOTE] > > Indexing is a newly released Dex capability. The API may change in future SDK versions. The index lifecycle is: **create → add/remove files (async) → search/navigate (sync) → delete**. *** ## Index vs. Vector Store Indexes and [vector stores](vector-stores) are both retrieval primitives, but they serve different access patterns: | | Index | Vector Store | | - | - | - | | **Search** | Lexical BM25 over file summaries | Semantic, lexical, or hybrid over chunks | | **Structure** | Hierarchical tree you can navigate | Flat collection of chunks | | **Best for** | Finding and browsing relevant files in a corpus; agentic navigation | Retrieving passages for RAG-enhanced extraction | Both can be combined as inputs to a [research](research) job. *** ## Creating an Index Indexing currently supports only the file-system index engine (`FileSystemIndexCreateRequest`). This engine builds the tree from the folder structure of your Dex project: folders are inferred from each file's name, treating everything before the last `/` as the folder path. For example, a file named `finance/2016/report.pdf` becomes a leaf under a `finance/2016` folder, with a parent `finance` folder above it. Folder nodes are summarized using the configured `model` and prompts. ```python theme={null} from dex_sdk import FileSystemIndexCreateRequest index = await project.create_index( FileSystemIndexCreateRequest( name="financial-reports", model="openai/gpt-5.2", # used to generate folder/file summaries file_summarization_prompt="", folder_summarization_prompt="", ) ) print("Status:", index.status) # PENDING, READY, UPDATING, or FAILED print("Nodes:", len(index.nodes)) ``` *** ## Adding Files `add_files` starts an async job and blocks until it completes. Adding files is best-effort per file, so the result reports which files succeeded and which failed. ```python theme={null} result = await index.add_files( file_ids=["file_1", "file_2", "file_3"] ) print("Succeeded:", result.succeeded_file_ids) for failed in result.failed_files: print(f"Failed {failed.file_id}: {failed.error}") # Retry failed files result = await index.add_files( file_ids=[failed.file_id for failed in result.failed_files] ) ``` To start the job without blocking, use `start_add_files_job`, which returns immediately so you can poll the job yourself: ```python theme={null} job = await index.start_add_files_job(file_ids=["file_1", "file_2"]) print("Job started:", job.id) ``` *** ## Searching `search` runs BM25 over the leaf nodes (files) and returns the most relevant matches. ```python theme={null} response = await index.search(query="quarterly revenue", top_k=5) for node in response.nodes: print(f"[level {node.level}] {node.id}: {node.text[:80]}") ``` Restrict the search to a subtree by passing `current_node_id`: ```python theme={null} response = await index.search( query="quarterly revenue", top_k=5, current_node_id="", # search only within this node's subtree ) ``` *** ## Navigating the Hierarchy `navigate` ranks the children of a node so you can walk the tree from broad summaries down to individual files. Pass `current_node_id=None` to start from the root. ```python theme={null} response = await index.navigate(query="finance", current_node_id=None, top_k=5) for node in response.nodes: print(f"[level {node.level}] {node.id}: {node.text[:80]}") ``` *** ## Removing Files and Deleting Removing files is all-or-nothing: on success, every requested file is removed from the index. ```python theme={null} # Remove files (blocks until the job finishes) await index.remove_files(file_ids=["file_1", "file_2"]) # Re-fetch the latest index state from the API await index.refresh() # Permanently delete the index and its storage await index.delete() ``` *** ## Listing and Retrieving Indexes ```python theme={null} # List all indexes in the project indexes = await project.list_indexes() # Fetch a single index by ID (with hydrated nodes) index = await project.get_index_by_id("index_123") ``` *** ## Appendix: Essential Imports ```python theme={null} from dex_sdk import DexClient, FileSystemIndexCreateRequest ``` *** ## Next Steps * **[Research](research)**: Run agentic research jobs that use indexes as input * **[Vector Stores](vector-stores)**: Semantic and hybrid search for RAG-enhanced extraction * **[Extract](extract)**: Extract structured data from parse results or vector stores # Introduction to Dex Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/introduction-to-dex Understand how Dex parses, chunks, and extracts structured data from documents. **Dex is Scale's document understanding service that transforms unstructured documents into actionable, structured data.** It is a comprehensive platform that combines **advanced OCR, natural language processing, and machine learning** to extract meaningful information from PDFs, images, spreadsheets, and more. *** ## Why Use Dex? Around 80-90% of enterprise data lives within unstructured formats such as PDFs and DOCX files. Dex solves the most common challenges of programmatic document processing: * **Format Diversity:** Process any document type with a single API—business reports, financial documents, legal contracts, healthcare records, and more. * **Unstructured Data:** Convert complex layouts into structured JSON with semantic understanding, including text, tables, charts, and infographics. * **Quality Variations:** Handle scanned, handwritten, and low-quality documents with high accuracy across multiple languages. * **Scalability:** Process thousands of documents efficiently with built-in scalable infrastructure. * **Flexibility:** Choose from multiple OCR engines and customize extraction with your own tools and workflows. *** ## Core Primitives Dex is designed as a capability rather than a standalone product, centered around composable primitives that can be used, extended, and combined: ### File Management Upload, retrieve, and securely store confidential documents. Supports persistent storage with metadata tracking, secure access patterns, and configurable data retention policies for automatic lifecycle management. ### Parse Convert documents into machine-readable formats using multiple OCR engines. Dex extracts: * **Plain text** in multiple languages (English, Spanish, Arabic, German, and more) * **Tables** including small and large tabular data (up to 500+ rows) * **Checkboxes** for form processing * **Images and figures** with bounding box information * **Charts** for data visualization analysis ### Vector Stores Vectorize and index parsed documents for semantic search and retrieval. Vector stores enable: * **Semantic search** over document chunks with embedding-based similarity * **Context management** for multi-file processing and large documents * **Lexical and hybrid search** alongside semantic search (SGP Vector Store engine) * **Document summarization** for quick overview generation ### Extract Extract structured data from parse results or document collections using: * **Custom schemas** defined with Pydantic models * **Natural language prompts** to guide extraction * **Citations** that link extracted data to source locations * **Confidence scores** for quality assessment * **RAG-enhanced extraction** using vector store context * **Agentic extraction** through [Research](/docs/capabilities/document-understanding/research) jobs; the extract endpoint itself runs single-pass extraction and rejects `use_agentic=true` *** ## Ways to Interact with Dex Dex provides multiple interfaces to support different use cases: * **REST API:** OpenAPI-documented endpoints for direct integration * **Python SDK:** High-level async wrapper for rapid development * **MCP Server:** Model Context Protocol integration for agent-based workflows (coming soon...) *** ## Common Use Cases * **Financial Services:** Automate invoice processing, tax document analysis, and financial report extraction. * **Healthcare:** Extract patient information from medical records, insurance claims, and healthcare forms. * **Legal:** Analyze contracts, process discovery documents, and extract key clauses and obligations. * **Business Operations:** Process HR documents, supply chain orders, customer service tickets, and business reports. *** ## Understanding Industry Document Challenges Different industries face unique document processing challenges based on their document types and layouts. For a comprehensive overview of typical document formats and layout challenges across finance, healthcare, insurance, and legal sectors, see [Industry Document Types and Layout Challenges](https://llms.reducto.ai/industry-document-types-and-layout-challenges). This guide covers: * **Finance:** SEC filings, research reports, and financial statements with multi-column layouts, complex footnotes, and embedded visualizations * **Healthcare:** Medical records and clinical documentation with handwritten elements, scanned materials, and variable form structures * **Insurance:** Claims forms (CMS-1500, UB-04) combining typed prompts with handwritten responses on poor-quality scans * **Legal:** Contracts and court filings requiring hierarchical structure preservation through complex sections and redlined annotations Understanding these document-specific challenges can help you optimize your Dex configuration for better extraction accuracy and results. *** ## Language Support Dex supports multi-language document processing with good support for germanic languages. For non-germanic, there are 35 languages including but not limited to: Afrikaans: 🇿🇦 - Albanian: 🇦🇱 - Arabic: 🇸🇦 - Armenian: 🇦🇲 - Belarusian: 🇧🇾 - Bengali: 🇧🇩 - Bulgarian: 🇧🇬 - Catalan: 🇪🇸 - Chinese: 🇨🇳 - Croatian: 🇭🇷 - Czech: 🇨🇿 - Danish: 🇩🇰 - Dutch: 🇳🇱 - English: 🇬🇧 - Estonian: 🇪🇪 - Filipino: 🇵🇭 - Finnish: 🇫🇮 - French: 🇫🇷 - German: 🇩🇪 - Greek: 🇬🇷 - Gujarati: 🇮🇳 - Hebrew: 🇮🇱 - Hindi: 🇮🇳 - Hungarian: 🇭🇺 - Icelandic: 🇮🇸 - Indonesian: 🇮🇩 - Italian: 🇮🇹 - Japanese: 🇯🇵 - Kannada: 🇮🇳 - Khmer: 🇰🇭 - Korean: 🇰🇷 - Lao: 🇱🇦 - Latvian: 🇱🇻 - Lithuanian: 🇱🇹 - Macedonian: 🇲🇰 - Malay: 🇲🇾 - Malayalam: 🇮🇳 - Marathi: 🇮🇳 - Nepali: 🇳🇵 - Norwegian: 🇳🇴 - Persian: 🇮🇷 - Polish: 🇵🇱 - Portuguese: 🇵🇹 - Punjabi: 🇮🇳 - Romanian: 🇷🇴 - Russian: 🇷🇺 - Serbian: 🇷🇸 - Slovak: 🇸🇰 - Slovenian: 🇸🇮 - Spanish: 🇪🇸 - Swedish: 🇸🇪 - Tagalog: 🇵🇭 - Tamil: 🇮🇳 - Telugu: 🇮🇳 - Thai: 🇹🇭 - Turkish: 🇹🇷 - Ukrainian: 🇺🇦 - Vietnamese: 🇻🇳 - Yiddish: 🇮🇱 *** ## Key Features ### Citations and Traceability Every extracted field can be associated with its source location (page number, bounding box, text snippet), enabling auditability and human review. ### Confidence Scoring Assigns confidence scores to extracted fields based on model outputs, helping you filter and prioritize results for downstream review. ### Flexible OCR Engine Support Choose from multiple OCR engines: * **Iris**: See **[When to choose Iris?](/docs/capabilities/ocr/when-to-choose-iris)**. * **Reducto**: see documentation [here](https://docs.reducto.ai/overview) > [!WARNING] > > **Deprecation notice:** Reducto is being deprecated. Use Iris for new projects. ### Data Lifecycle Management Configurable retention policies automatically manage the lifecycle of files and processing artifacts, helping you meet compliance requirements and optimize storage costs. *** ## Infrastructure and Deployment Scale engineering deploys Dex as part of your SGP installation. This section describes what it runs. ### Core Components Dex is deployed as a Kubernetes service with the following components: **Application Pods:** * **API Pod**: Handles REST API requests for document processing * **Temporal Worker Pod**: Processes document understanding workflows * The API and worker pods run from Scale-provided container images with different entry points. **Infrastructure Services:** * **Temporal Server**: Workflow orchestration. Your SGP release sets the Temporal version. * **Postgres Database**: Application data and Temporal state. AWS uses Aurora PostgreSQL 15, on-premises deployments use PostgreSQL 15, and Azure uses Azure Database for PostgreSQL Flexible Server. * **Object Storage**: S3, MinIO, or Azure Blob Storage for document artifacts **Deployment:** * Deployed via Helm charts on Kubernetes * Supported on AWS, Azure, and GCP * Requires namespace and service configuration ### GPU and Hardware Requirements GPU requirements depend on which OCR engine and extraction models you use. **OCR Engine Options:** **Cloud Provider OCR (No provisioned GPU):** * AWS Textract * Azure Vision Read * GCP Vision API **Iris:** * For custom OCR needs, with any OCR model available through SGP; available models depend on your deployment. See **[When to choose Iris?](/docs/capabilities/ocr/when-to-choose-iris)**. **Reducto Local OCR (Soon to be deprecated):** * **Small (CPU)**: No GPU. Germanic languages. \~3s/page latency. * **Medium**: A10G GPU. Germanic languages. * **Large**: 8xH100 GPUs. 40+ languages (no Arabic RTL, degraded CJK). **Extraction, Key-Value, Tables:** * Azure OpenAI or Claude via Bedrock (no GPU, recommended) * Hosted H100 (slower auto-scaling) ### Recommended Configuration **For minimal GPU requirements:** * OCR: Reducto Small (CPU) or Azure Vision Read * Extraction: Azure OpenAI or Claude via AWS Bedrock * Infrastructure: Kubernetes cluster with Postgres and object storage *** ## Getting Started To begin using Dex, you'll need a Scale account with SGP access. **Quick Links:** * **[Getting Started Guide](/docs/capabilities/document-understanding/getting-started-with-dex)**: Step-by-step tutorial for your first extraction * **[File Management](/docs/capabilities/document-understanding/file-management)**: Upload, pagination, and file types * **[Parse](/docs/capabilities/document-understanding/parse)**: Parse engines and async jobs * **[Chunking](/docs/capabilities/document-understanding/chunking)**: Chunking strategies * **[Vector Stores](/docs/capabilities/document-understanding/vector-stores)**: Semantic search and RAG * **[Extract](/docs/capabilities/document-understanding/extract)**: Data extraction and batch processing * **[Best Practices](/docs/capabilities/document-understanding/best-practices)**: Quick start and optimization * **[API Reference](/docs/capabilities/document-understanding/dex-sdk-api-reference)**: Complete SDK documentation Dex makes document understanding accessible to developers while delivering the power and accuracy required for production applications. # Parse Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/parse Parse documents with Iris, monitor async jobs, and handle multi-language documents. Convert documents to structured format for extraction. Dex supports multiple parse engines and async job monitoring. *** ## Parse Document (Default) ```python theme={null} parse_result = await dex_file.parse( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( layout="rt_detr_bce", text_ocr="openai/gpt-5.4", table_ocr="openai/gpt-5.4", confidence_threshold=0.5, img_method="description", ), ) ) ``` *** ### Which OCR Engine? See **[When to choose Iris?](/docs/capabilities/ocr/when-to-choose-iris)** for custom OCR needs. *** ## Async Job Monitoring **New in v0.4.0**: Use `start_parse_job` for better control over async operations and access to SGP traces. ### Monitor Parse Job ```python theme={null} import asyncio from dex_sdk.types import ( ParseEngine, IrisParseJobParams, IrisParseEngineOptions, JobStatus, ) # Start a parse job (returns immediately) parse_job = await project.start_parse_job( dex_file=dex_file, parameters=IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( layout="rt_detr_bce" ) ) ) # Monitor job progress while parse_job.data.status not in [ JobStatus.SUCCEEDED, JobStatus.PARTIALLY_SUCCEEDED, JobStatus.FAILED, JobStatus.CANCELLED, ]: await asyncio.sleep(1) await parse_job.refresh() print(f"Job status: {parse_job.data.status}") # Get result if parse_job.data.status == JobStatus.SUCCEEDED: parse_result = await parse_job.get_result() print("Parse completed successfully") else: print(f"Parse failed: {parse_job.data.error}") ``` ### Retrieving SGP Traces for Debugging ```python theme={null} from scale_gp_beta import SGPClient sgp_client = SGPClient( api_key=os.getenv("SGP_API_KEY"), account_id=os.getenv("SGP_ACCOUNT_ID"), ) # Search for job traces spans = list(sgp_client.spans.search( sort_by="created_at", sort_order="desc", extra_metadata={"job_id": parse_job.data.id}, parents_only=True, )) if spans: trace_id = spans[0].trace_id all_spans = list(sgp_client.spans.search(trace_ids=[trace_id])) for span in all_spans: print(f"Span: {span.name}, Duration: {span.duration_ms}ms") ``` *** ## Process Multiple Files ```python theme={null} import asyncio from dex_sdk.types import ( ParseEngine, IrisParseJobParams, IrisParseEngineOptions, ) # Parse all files in parallel parse_tasks = [ dex_file.parse( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( layout="rt_detr_bce", # native region-level chunks (IRIS v2 default) ), ) ) for dex_file in dex_files ] parse_results = await asyncio.gather(*parse_tasks) print(f"Parsed {len(parse_results)} documents") ``` *** ## Multi-Language Support Dex supports 35+ languages with automatic language detection. ### Supported Languages Germanic languages have excellent support. Additional 35+ languages include: * European: Spanish, French, German, Italian, Portuguese, Dutch, Swedish, Norwegian, Danish, Polish, Russian, Ukrainian * Asian: Chinese, Japanese, Korean, Thai, Vietnamese, Khmer, Lao * Middle Eastern: Arabic, Hebrew, Persian, Turkish * Indian: Hindi, Bengali, Tamil, Telugu, Malayalam, Kannada, Gujarati, Marathi, Punjabi * And many more... See the [Introduction guide](introduction-to-dex#language-support) for the complete list. *** ## Next Steps * **[Chunking](chunking)**: Choose chunking strategies for your documents * **[Extract](extract)**: Extract structured data from parse results * **[Vector Stores](vector-stores)**: Use vector stores for large documents # Research Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/research Run agentic research jobs over parse results, vector stores, and indexes to answer complex questions with citations. Research runs an agentic job that answers complex questions over your documents. It decomposes the task, searches and cites your sources, aggregates findings, and validates the output. Inputs are scoped to [parse results](parse), [vector stores](vector-stores), and/or [indexes](indexing), and results come back as structured fields—or a markdown report—each backed by citations and usage metrics. > [!NOTE] > > Research is a newly released Dex capability. The API may change in future SDK versions. *** ## Prerequisites A research job needs at least one source to search. Before running a job, prepare one or more of: * **[Parse results](parse)** — parsed documents in your project * **A [vector store](vector-stores)** — for semantic/hybrid chunk search (at most one per job) * **An [index](indexing)** — for locating relevant files in a corpus or files which are not ingested in the vector store (at most one per job) *** ## Running a Research Job Build a `ResearchJobRequest` describing the inputs and the task, start the job, then poll for the result. ```python theme={null} from dex_sdk import ResearchInputs, ResearchJobRequest, ResearchTask req = ResearchJobRequest( inputs=ResearchInputs( vector_stores=[vector_store.id], indexes=[index.id], ), task=ResearchTask( description="Between 'UK - DP' and 'DACH - DP', which had higher cash income in FY 2016?", output_schema=MyOutputSchema.model_json_schema(), ), model="openai/gpt-5.2", ) # Start the job job = await project.start_research_job(req) print("Research job:", job.id) # Poll to completion and get the typed result res = await job.get_result(max_attempts=1000) ``` *** ## Reading Results The result's `data` maps each output field to a value, confidence, and citations. Citations are a discriminated union—use `citation.kind` to tell block citations (from PDFs) from spreadsheet citations. ```python theme={null} for name, field in res.data.items(): print(f"{name}: {field.value} (confidence={field.confidence})") for citation in field.citations: if citation.kind == "block": print(f" [block] file={citation.file_id} page={citation.page} bbox={citation.bbox}") else: # "spreadsheet" print(f" [spreadsheet] file={citation.file_id} sheet={citation.sheet} cell={citation.location}") # Other details print("Total tokens used:", res.usage_info.total_tokens) print("Research trace:", res.trace) ``` *** ## Structured vs. Report Output Choose the output shape on the `ResearchTask`: * **Structured** — set `output_schema` to a JSON Schema. Results populate `res.data` with one entry per schema field. * **Report** — set `report_schema` to a markdown template. Access the rendered report with the `res.report` shortcut. ```python theme={null} task = ResearchTask( description="Summarize FY2016 segment performance.", report_schema="## Summary\n{summary}\n\n## Key figures\n{figures}", ) # ...after the job completes: print(res.report) # markdown string, or None for structured runs ``` *** ## Advanced Configuration For finer control, configure tools and per-stage behavior. **Per-tool config** — enable/disable tools and pass tool-specific options (e.g. `top_k`, filters, rerank config): ```python theme={null} from dex_sdk.types import ResearchToolConfigEntry req = ResearchJobRequest( inputs=ResearchInputs(vector_stores=[vector_store.id], indexes=[index.id]), task=ResearchTask( description="...", output_schema=MyOutputSchema.model_json_schema(), ), model="openai/gpt-5.2", tools={ # Vector store search tools: similarity_search, lexical_search, hybrid_search. # Index search tools: search_index, navigate_index. "similarity_search": ResearchToolConfigEntry( enabled=True, config={"top_k": 10}, ), "lexical_search": ResearchToolConfigEntry(enabled=False, config={}), }, ) ``` **Per-stage overrides** — override the prompt, model, temperature, or reasoning effort for any stage of the agentic pipeline (decomposition, subtasks, aggregation, output validation): ```python theme={null} from dex_sdk.types import ( ResearchAdvanced, ResearchAdvancedStep, ResearchReasoningBlock, ) req.advanced = ResearchAdvanced( aggregation=ResearchAdvancedStep( prompt="", model="openai/gpt-5.4", temperature=0.0, reasoning=ResearchReasoningBlock(effort="high"), ), ) ``` Additional flags on the request: * `enable_subtable_citations` — cite specific table cells/ranges rather than whole tables. *** ## Listing Research Jobs ```python theme={null} jobs = await project.list_research_jobs() for job in jobs: print(job.id) ``` *** ## Appendix: Essential Imports ```python theme={null} from dex_sdk import DexClient, ResearchInputs, ResearchJobRequest, ResearchTask ``` The advanced configuration types (`ResearchToolConfigEntry`, `ResearchAdvanced`, `ResearchAdvancedStep`, `ResearchReasoningBlock`, and the result types `ResearchResult`, `BlockCitation`, `SpreadsheetCitation`) are re-exported from `dex_sdk.types`. *** ## Next Steps * **[Indexing](indexing)**: Build a hierarchical index to use as a research input * **[Vector Stores](vector-stores)**: Semantic and hybrid search for RAG-enhanced extraction * **[Extract](extract)**: Extract structured data from a single parse result or vector store # Troubleshooting Dex Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/troubleshooting Resolve Dex SDK installation, configuration, parsing, and extraction issues. This guide covers common issues you might encounter when working with Dex and how to resolve them. *** ## Installation Issues ### Installing the Dex SDK Your Scale team provides the `dex-sdk` package and the `dex-core` and `data-sources-core` packages it depends on. Install them as described in [Getting Started with Dex](getting-started-with-dex#install-the-dex-sdk). **Verify installation:** ```python theme={null} import dex_sdk from importlib.metadata import version print(f"Dex SDK version: {version('dex-sdk')}") ``` ### Common Installation Errors **Package Not Found** pip can't install `dex-sdk`, `dex-core`, or `data-sources-core` from PyPI. Install all three from the files or package index URL your Scale team provided. **Permission Denied** Use a virtual environment instead of sudo: `python -m venv venv && source venv/bin/activate` *** ## Runtime Issues ### Connection Problems **Symptoms:** Cannot connect to Dex service **Test connection:** ```python theme={null} import requests response = requests.get("https://dex.sgp.scale.com/health") print(f"Status: {response.status_code}") ``` **Solutions:** * Check network connectivity * Verify credentials (see Authentication Errors below) * Confirm Dex service is available ### Authentication Errors **Symptoms:** Invalid credentials, permission denied **Verify credentials:** ```python theme={null} import os sgp_account_id = os.getenv("SGP_ACCOUNT_ID") sgp_api_key = os.getenv("SGP_API_KEY") if not sgp_account_id or not sgp_api_key: print("❌ Missing credentials") ``` **Solutions:** * Set environment variables: `SGP_ACCOUNT_ID` and `SGP_API_KEY` * Verify your account has SGP access * Check API key hasn't expired or been revoked ### File Upload Errors **Symptoms:** Upload fails, file too large, unsupported format **Supported formats:** * **Images**: PNG, JPEG, TIFF, HEIC, and more * **PDFs**: PDF * **Spreadsheets**: CSV, XLSX, XLS * **Documents**: PPTX, DOCX, TXT, RTF **Solutions:** * Verify file path is correct * Check file size is under 100MB * Ensure file format is supported * Verify file isn't corrupted ### Parsing Errors **Symptoms:** OCR fails, document can't be processed **Common causes:** * Low quality scans (use at least 300 DPI) * Password-protected documents * Extremely complex layouts * Very large documents timing out **Solutions:** 1. **Check document quality** and resolution 2. **Remove password protection** before uploading 3. **Try a different OCR engine:** ```python theme={null} from dex_sdk.types import ParseEngine, IrisParseJobParams, IrisParseEngineOptions parse_result = await dex_file.parse( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions(layout="rt_detr_bce"), ) ) ``` 4. **Split large documents** into smaller chunks 5. **See complex layouts guide:** [Industry Document Types and Layout Challenges](https://llms.reducto.ai/industry-document-types-and-layout-challenges) ### Extraction Errors **Symptoms:** Extraction fails, schema errors, timeout **Common causes:** * Invalid Pydantic schema * Model timeout or rate limiting * Insufficient context for extraction * Schema-data mismatch **Solutions:** 1. **Validate your schema:** ```python theme={null} from pydantic import BaseModel, Field class MySchema(BaseModel): field_name: str = Field(description="Clear description") ``` Pass the Pydantic class to `parse_result.extract` or `vector_store.extract`. For the separate `start_extract_job` path, place `MySchema.model_json_schema()` in `ExtractionParameters.extraction_schema`. 2. **Use clear prompts** with detailed instructions 3. **Enable debugging:** ```python theme={null} extract_result = await parse_result.extract( extraction_schema=MySchema, user_prompt="Your detailed prompt", model="openai/gpt-5.4", generate_citations=True, generate_confidence=True, ) ``` 4. **Use vector stores for large documents:** ```python theme={null} vector_store = await project.create_vector_store( name="My Store", engine=VectorStoreEngines.SGP_VECTOR_STORE, embedding_model="openai/text-embedding-3-large", ) await vector_store.add_parse_results([parse_result.id]) extract_result = await vector_store.extract( extraction_schema=MySchema, user_prompt=user_prompt, model="openai/gpt-5.4", ) ``` *** ## Performance Issues ### Slow Parsing **Causes:** Large documents, complex layouts, high OCR load **Solutions:** * Process documents asynchronously in batches * Cache parse results for frequently accessed documents ### Slow Extraction **Causes:** Large context, complex schema, model performance **Solutions:** * Use vector stores to reduce context size * Simplify extraction schema * Choose faster models for time-sensitive applications * Use batch processing ### Slow Vector Store Search **Causes:** Large document collections, complex queries **Solutions:** * Use appropriate `top_k` values * Add filters to narrow search scope * Use `vector_store.search_in_file()` for file-specific searches * Create separate vector stores for different categories *** ## Error Handling ### Exception Types Always wrap Dex operations in try-except blocks: ```python theme={null} from dex_sdk import DexAPIError try: project = await dex_client.create_project(...) dex_file = await project.upload_file(...) parse_result = await dex_file.parse(...) extract_result = await parse_result.extract(...) except DexAPIError as e: print(f"Dex error: {e}") except Exception as e: print(f"Unexpected error: {e}") ``` ### Common Exceptions | Exception | Cause | Solution | | - | - | - | | `DexAPIError` | The API returned an error status, for example `401` or `403` for invalid credentials or insufficient access | Read `e.response.status_code` and `e.response.text`; verify `SGP_ACCOUNT_ID` and `SGP_API_KEY` | | `ValueError` | A parse, extract, or research job failed, was cancelled, or only partially succeeded; or `upload_file` got a path that doesn't exist | Read the error message; try a different OCR engine or validate the schema | | `TimeoutError` | Polling reached `max_attempts` before the job finished | Raise `max_attempts`, or start the job and poll it yourself | | `httpx.TransportError` | Network issues | Check connectivity | *** ## Debugging ### Enable Logging ```python theme={null} import logging logging.basicConfig(level=logging.DEBUG) logger = logging.getLogger("dex_sdk") logger.setLevel(logging.DEBUG) ``` ### Inspect Parse Results ```python theme={null} print(f"Pages: {parse_result.data.parse_metadata.pages_processed}") print(f"Chunks: {len(parse_result.data.content.chunks)}") ``` ### Validate Extraction Results Enable citations and confidence to inspect each extracted field. This example reuses `parse_result` and the `MySchema` Pydantic class from [Extraction Errors](#extraction-errors). When citations or confidence are enabled, extracted fields are at `extract_result.data.data`. With both disabled, `extract` returns your schema model. See [Extraction Response Shapes](extract#extraction-response-shapes) for extract jobs and direct file extraction. ```python theme={null} extract_result = await parse_result.extract( extraction_schema=MySchema, user_prompt="Extract the fields described by the schema.", model="openai/gpt-5.4", generate_citations=True, generate_confidence=True, ) for name, field in extract_result.data.data.items(): print(f"{name}: {field.value}") if field.citations: print(f" Citations: {len(field.citations)}") if field.confidence is not None: print(f" Confidence: {field.confidence}") ``` ### Test Components Individually ```python theme={null} try: project = await dex_client.create_project(...) print("✅ Project created") dex_file = await project.upload_file(...) print("✅ File uploaded") parse_result = await dex_file.parse(...) print("✅ Parsing completed") extract_result = await parse_result.extract(...) print("✅ Extraction completed") except Exception as e: print(f"❌ Failed at step: {e}") ``` *** ## Getting Help ### Support Channels * **Your Scale team**: Dex support * **Documentation**: [Dex Documentation](/docs/capabilities/document-understanding/introduction-to-dex) ### When Reporting Issues Include: 1. **Error message** - Full error text and stack trace 2. **Code snippet** - Minimal reproducible example 3. **Document type** - File format and characteristics 4. **SDK version** - Output of `pip show dex_sdk` 5. **Environment** - Python version, OS 6. **Expected vs actual behavior** ### Additional Resources * [Getting Started Guide](getting-started-with-dex) - Quick start tutorial * [File Management](file-management) - Upload and pagination * [Parse](parse) - Parse engines and async jobs * [Vector Stores](vector-stores) - Semantic search and RAG * [Extract](extract) - Data extraction and batch processing * [Best Practices](best-practices) - Common patterns and optimization * [API Reference](dex-sdk-api-reference) - Complete SDK documentation * [Introduction to Dex](introduction-to-dex) - Core concepts # Vector Stores Source: https://docs.gp.scale.com/docs/capabilities/document-understanding/vector-stores Create vector stores for semantic search and RAG-enhanced extraction from large or multi-file documents. Vector stores enable semantic search and RAG-enhanced extraction for large documents or multi-file processing. *** ## When to Use Vector Stores? ``` Is your document > 50 pages? ├─ Yes → Use vector store └─ No → Do you have multiple documents to query together? ├─ Yes → Use vector store └─ No → Extract directly from parse result ``` *** ## Vector Store Engines Dex supports two vector store engines. Prefer `SGP_VECTOR_STORE` for new implementations. | Engine | Best For | Embedding Options | Search Features | | - | - | - | - | | `SGP_VECTOR_STORE` | **Recommended.** Document RAG, advanced search, reranking | Base models or Models API deployment (SDK `0.5.0` or later) | Semantic, lexical, hybrid search; optional reranking | | `SGP_KNOWLEDGE_BASE` | Legacy only | Base models or Models API deployment | Semantic search | > [!WARNING] > > SGP Vector Store requires Dex SDK `0.4.3` or later and an SGP deployment that supports SGP Vector Store. Upgrade an older SDK before selecting `VectorStoreEngines.SGP_VECTOR_STORE`. If your deployment has only the legacy Knowledge Base backend, use `SGP_KNOWLEDGE_BASE`. ### SGP Vector Store — Recommended The SGP Vector Store engine is the preferred vector store implementation. Use it for: * Document RAG workflows within Dex * Lexical (keyword) or hybrid search in addition to semantic search * Reranking search results for higher relevance **Embedding configuration:** Base embedding models (`embedding_type="base"`, e.g., `openai/text-embedding-3-large`, `sentence-transformers/all-mpnet-base-v2`) or, with SDK `0.5.0` or later, a custom model deployed via the Models API (`embedding_type="models_api"` with `model_deployment_id`). **Search features:** * **Query types:** `semantic`, `lexical`, or `hybrid` * **Reranking:** Optional `rerank_config` (a `RerankConfig` with `model`, `top_n`, and `instruction`) for improved result ordering ### SGP Knowledge Base — Legacy The SGP Knowledge Base engine is SGP's legacy vector store implementation. Its use is discouraged for new projects. Consider it only when: * You need integration with SGP data connectors for ingesting from external sources * Your SGP platform does not yet support SGP Vector Store **Embedding configuration:** * **Base models** (`embedding_type="base"`): Use pre-configured embedding models such as `openai/text-embedding-3-large` * **Models API** (`embedding_type="models_api"`): Use a custom model deployment via `model_deployment_id` **Search:** Semantic search only. Reranking and query type options are not supported. > [!NOTE] > > `create_vector_store` returns `DexSGPKnowledgeBase` for `SGP_KNOWLEDGE_BASE` and `DexSGPVectorStore` for `SGP_VECTOR_STORE`. Use `DexSGPVectorStore.get_by_id` only for SGP Vector Store IDs—it will raise if given a Knowledge Base ID. *** ## Creating a Vector Store ```python theme={null} from dex_sdk.types import VectorStoreEngines # Create an SGP Vector Store (recommended) vs_store = await project.create_vector_store( name="Financial Documents Store", engine=VectorStoreEngines.SGP_VECTOR_STORE, embedding_model="openai/text-embedding-3-large", ) # Or create a Knowledge Base vector store (legacy; for data connector integration) kb_store = await project.create_vector_store( name="Financial Documents Store", engine=VectorStoreEngines.SGP_KNOWLEDGE_BASE, embedding_model="openai/text-embedding-3-large", ) ``` To use a Models API embedding deployment with SGP Vector Store (SDK `0.5.0` or later): ```python theme={null} vs_store = await project.create_vector_store( name="Financial Documents Store with Custom Embeddings", engine=VectorStoreEngines.SGP_VECTOR_STORE, embedding_type="models_api", model_deployment_id="your-embedding-deployment-id", ) ``` *** ## Adding Documents to Vector Store ```python theme={null} # Add parsed files to the vector store await vector_store.add_parse_results([parse_result.id]) print("Parse results added to vector store") ``` *** ## Semantic Search ```python theme={null} # Perform semantic search across all documents search_results = await vector_store.search( query="What is the total taxable income?", top_k=5, ) # Search within a specific file file_search_results = await vector_store.search_in_file( file_id=dex_file.id, query="What is the total taxable income?", top_k=5, filters=None, # Optional filters like {"file_id": "file_123"} ) # Access search results for chunk in search_results.chunks: print(f"Score: {chunk.score:.2f}") print(f"Content: {chunk.content[:100]}...") print(f"File: {chunk.file_id}, Page: {chunk.blocks[0].page_number if chunk.blocks else 'N/A'}") # Access additional metadata from the chunk if chunk.metadata: print(f"Metadata: {chunk.metadata}") ``` SGP Vector Store only: use `query_type` and reranking for improved results: ```python theme={null} from dex_core.models.vector_store.vector_store import RerankConfig # Hybrid search with reranking (SGP Vector Store only) search_results = await vs_store.search( query="What is the total taxable income?", top_k=10, query_type="hybrid", rerank_config=RerankConfig(model="cohere.rerank-english-v3.0", top_n=5), ) ``` *** ## Custom Metadata Schema You can define custom metadata fields on your vector store and attach metadata to each parse result. This metadata is indexed and filterable when searching, so you can narrow results by document attributes such as department, document type, or priority. **1. Define the schema when creating the vector store** Specify `vector_store_metadata_schema` as a dict mapping field names to types: `"string"`, `"int"`, `"double"`, or `"boolean"`. ```python theme={null} vector_store = await project.create_vector_store( name="Documents by Department", engine=VectorStoreEngines.SGP_VECTOR_STORE, embedding_model="openai/text-embedding-3-large", vector_store_metadata_schema={ "department": "string", "priority": "int", "is_public": "boolean", }, ) ``` **2. Attach metadata when parsing** Pass `vector_store_metadata` in your parse job parameters. The metadata is stored with the parse result and indexed when you add it to the vector store. ```python theme={null} from dex_sdk.types import ParseEngine, IrisParseJobParams, IrisParseEngineOptions parse_result = await dex_file.parse( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( layout="rt_detr_bce", text_ocr="openai/gpt-5.4", table_ocr="openai/gpt-5.4", confidence_threshold=0.5, img_method="description", ), vector_store_metadata={ "department": "legal", "priority": 1, "is_public": False, }, ) ) await vector_store.add_parse_results([parse_result.id]) ``` For custom parse results, pass `vector_store_metadata` when creating the parse result via `DexParseResult.from_custom_results`. **3. Filter on custom metadata when searching** Use the `filters` parameter in search. Filter format depends on the engine: | Operation | SGP Vector Store | SGP Knowledge Base | Example | | - | - | - | - | | Equal | `$eq` | `eq` | `{"department": {"$eq": "legal"}}` | | Not equal | `$ne` | `ne` | `{"department": {"$ne": "archived"}}` | | Greater than | `$gt` | `gt` | `{"priority": {"$gt": 1}}` | | Greater than or equal | `$gte` | `gte` | `{"priority": {"$gte": 1}}` | | Less than | `$lt` | `lt` | `{"priority": {"$lt": 5}}` | | Less than or equal | `$lte` | `lte` | `{"priority": {"$lte": 5}}` | | In (list of values) | `$in` | `in` | `{"department": {"$in": ["legal", "finance"]}}` | | Not in (list of values) | `$nin` | `nin` | `{"department": {"$nin": ["archived"]}}` | | And (combine conditions) | `$and` | `and` | `{"$and": [{"department": {"$eq": "legal"}}, {"priority": {"$gte": 1}}]}` | | Or (any condition matches) | `$or` | `or` | `{"$or": [{"department": {"$eq": "legal"}}, {"department": {"$eq": "finance"}}]}` | | Not (negate condition) | `$not` | `not` | `{"$not": {"department": {"$eq": "archived"}}}` | ```python theme={null} # Filter by department (SGP Vector Store) search_results = await vector_store.search( query="contract terms", top_k=10, filters={"department": {"$eq": "legal"}}, ) # Filter by multiple values (SGP Vector Store) search_results = await vector_store.search( query="quarterly report", top_k=10, filters={"department": {"$in": ["legal", "finance"]}}, ) # Filter by numeric value (SGP Vector Store) search_results = await vector_store.search( query="policy", top_k=10, filters={"priority": {"$gte": 1}}, ) # Same filter on SGP Knowledge Base — operators have no $ prefix search_results = await knowledge_base.search( query="contract terms", top_k=10, filters={"department": {"eq": "legal"}}, ) ``` > [!NOTE] > > The schema must be defined when creating the vector store. Fields in `vector_store_metadata` that are not in `vector_store_metadata_schema` are not indexed and cannot be used for filtering. *** ## RAG-Enhanced Extraction Extract data using vector store context for improved accuracy on large documents: This example enables citations and confidence, so extracted fields are at `extract_result.data.data`. With both disabled, `extract` returns your schema model. See [Extraction Response Shapes](extract#extraction-response-shapes) for other response shapes. ```python theme={null} # Extract from vector store with RAG context extract_result = await vector_store.extract( extraction_schema=FinancialData, user_prompt="""Extract financial data from all documents in this store. Focus on quarterly income statements.""", model="openai/gpt-5.4", generate_citations=True, generate_confidence=True, ) # Access extracted data with citations result = extract_result.data for field_name, field in result.data.items(): confidence = f"{field.confidence:.2f}" if field.confidence is not None else "unavailable" print(f"{field_name}: {field.value} (confidence: {confidence})") if field.citations: for cite in field.citations: print(f" → From {cite.file_id}, page {cite.page}") ``` ### Pattern: RAG for Large Documents ```python theme={null} # 1. Create vector store vs = await project.create_vector_store( name="Store", engine=VectorStoreEngines.SGP_VECTOR_STORE, embedding_model="openai/text-embedding-3-large", ) # 2. Add documents await vs.add_parse_results([parse_result.id]) # 3. Extract with context result = await vs.extract( extraction_schema=Schema, user_prompt="Extract from all docs", model="openai/gpt-5.4", ) ``` *** ## Next Steps * **[Extract](extract)**: Extract structured data from parse results or vector stores * **[Chunking](chunking)**: Optimize chunking for vector store embeddings * **[API Reference](dex-sdk-api-reference)**: Complete SDK documentation # Getting Started with IRIS Source: https://docs.gp.scale.com/docs/capabilities/ocr/getting-started-with-iris Configure IRIS OCR through the Dex SDK, parse documents, and inspect structured results. This guide will help you get started with IRIS, Scale's proprietary OCR capability integrated with Dex. You'll learn how to parse documents with IRIS through the Dex SDK, work with parse results, and integrate IRIS into your document processing workflows. *** ## Overview IRIS is Scale's proprietary OCR capability that provides complete pipeline control for custom document processing needs. When using IRIS through Dex, you get: * **Use any OCR model**: Experiment with different OCR engines to optimize for your specific documents. DEX can use any model hosted in SGP. * **Complete Pipeline Control**: Configure layout detection and OCR processing independently * **Unified Document Management**: Upload and manage files through Dex's file system * **Async Processing**: Non-blocking parse jobs that process documents in the background * **Project Organization**: Group files and parse results within Dex projects * **Extensibility**: Add custom OCR models through SGP for specialized needs *** ## Prerequisites Before using IRIS, ensure you have: * Dex SDK installed * SGP Account ID and API Key * Access to a Dex instance *** ## Basic Usage ### Initialize Dex Client ```python theme={null} import os from dex_sdk.client import DexClient # Initialize Dex client dex_client = DexClient( base_url="your-dex-url", account_id = os.getenv("SGP_ACCOUNT_ID"), api_key = os.getenv("SGP_API_KEY"), ) # Create a project project = await dex_client.create_project(name="my-ocr-project") ``` ### Parse a Document with IRIS **What should I use for IrisParseEngineOptions?** A common query when using Iris is “How can I get the best OCR results without spending time worrying about all the parameters in `IrisParseEngineOptions`?”. This section gives quick-start suggestions that you can use depending on your requirements that have been shown to perform consistently well across use-cases. ### Option 1: I only need text, I don’t need bounding boxes. **Suggestion:** text_ocr="openai/gpt-5.4" + layout="whole_page" This bypasses the layout detection step, and instead passes the full page to the OCR model. This has been shown to consistently perform consistently well across benchmarks. **Suggestion (open-source required):** `text_ocr="" + layout="whole_page"` This is the same as the previous suggestion, but with the open-source Qwen/Qwen3.6-27B model instead, which has been shown to almost match gpt-5.4 in English and often exceed gpt-5.4 in Arabic. Using the Qwen model with Iris requires a deployed model endpoint available through SGP. ```python theme={null} from dex_sdk.types import IrisParseEngineOptions, IrisParseJobParams, ParseEngine # Upload document dex_file = await project.upload_file("document.pdf") # Start IRIS parse job parse_job = await dex_file.start_parse_job( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( layout= "whole_page", text_ocr= "openai/gpt-5.4", #or text_ocr= "" for the open-source suggestion. ), ), ) # Get parse results parse_result = await parse_job.get_result() # Access parsed content chunks = parse_result.data.content.chunks for chunk in chunks: print(chunk.content) ``` ### Option 2: I need bounding boxes. The default config values were designed for this case, so you only need to specify which OCR model to use. The suggestion and open-source suggestion is the same for this case, with the only difference is we do not specify `layout` so that it uses the default layout model option `rt_detr_bce`. ```python theme={null} from dex_sdk.types import IrisParseEngineOptions, IrisParseJobParams, ParseEngine # Upload document dex_file = await project.upload_file("document.pdf") # Start IRIS parse job parse_job = await dex_file.start_parse_job( IrisParseJobParams( engine=ParseEngine.IRIS, options=IrisParseEngineOptions( text_ocr= "openai/gpt-5.4", #or text_ocr= "" for the open-source suggestion. ), ), ) # Get parse results parse_result = await parse_job.get_result() ``` ### Custom evaluation In order to get the highest OCR accuracy, you can run a custom evaluation workflow on your own data in SGP-Workflows. This includes evaluating the performance of the OCR models in Iris as well as comparing performance to Reducto. You can evaluate OCR results in 2 ways: 1. Evaluating the OCR performance against the ground truth using DEX & eval cards in Workflows. 2. Evaluate the impact of the OCR results on your downstream tasks by plugging in your own downstream tasks and evaluation workflow into our SGP-workflow template in Workflows. *** ## Supported Document Types IRIS supports various document formats: * **PDF documents** (.pdf) * **Images** (.png, .jpg, .jpeg, .tiff) * **Scanned documents** with printed or handwritten text * **Multi-page documents** *** ## Parsing PDFs ### Basic PDF Parsing ```python theme={null} from dex_sdk.types import IrisParseEngineOptions, IrisParseJobParams # Upload PDF file dex_file = await project.upload_file("path/to/document.pdf") # Start IRIS parse job parse_job = await dex_file.start_parse_job( IrisParseJobParams( options=IrisParseEngineOptions(), ) ) # Wait for and retrieve parse results parse_result = await parse_job.get_result() # Access parsed content chunks = parse_result.data.content.chunks for chunk in chunks: print(chunk.content) ``` ### Complete PDF Example ```python theme={null} import os from dex_sdk.client import DexClient from dex_sdk.file import DexFile from dex_sdk.types import IrisParseEngineOptions, IrisParseJobParams, PaginationParams from dex_core.models.files import FileEntity async def parse_pdf_example(): # Initialize client dex_client = DexClient( base_url="your-dex-url", account_id = os.getenv("SGP_ACCOUNT_ID"), api_key = os.getenv("SGP_API_KEY"), ) # Create project project = await dex_client.create_project(name="ocr-project") # Upload file dex_file = await project.upload_file("document.pdf") # Verify file upload assert isinstance(dex_file, DexFile) assert isinstance(dex_file.data, FileEntity) # Verify file is in project files = await project.list_files(pagination_params=PaginationParams(sort_by="created_at")) assert len(files.items) > 0, "Project should contain at least one file" # Get download URL if needed download_url = await dex_file.get_download_url() assert download_url.startswith("http") # Start IRIS parse job parse_job = await dex_file.start_parse_job( IrisParseJobParams( options=IrisParseEngineOptions(), ) ) # Get parse results parse_result = await parse_job.get_result() # Access parsed content content_length = len(parse_result.data.content.chunks[0].content) print(f"Parsed content length: {content_length}") ``` *** ## Parsing Images IRIS supports various image formats including PNG, JPG, and TIFF. ### Basic Image Parsing ```python theme={null} # Upload image file dex_file = await project.upload_file("path/to/image.png") # Start IRIS parse job parse_job = await dex_file.start_parse_job( IrisParseJobParams( options=IrisParseEngineOptions(), ) ) # Get parse results parse_result = await parse_job.get_result() # Access chunks chunks = parse_result.data.content.chunks print(f"Number of chunks: {len(chunks)}") ``` ### Complete Image Example ```python theme={null} async def parse_image_example(): dex_client = DexClient( base_url="your-dex-url", account_id = os.getenv("SGP_ACCOUNT_ID"), api_key = os.getenv("SGP_API_KEY"), ) project = await dex_client.create_project(name="ocr-project") # Upload PNG file dex_file = await project.upload_file("image.png") assert isinstance(dex_file, DexFile) assert isinstance(dex_file.data, FileEntity) # Verify file in project files = await project.list_files(pagination_params=PaginationParams(sort_by="created_at")) assert len(files.items) > 0 # Get download URL download_url = await dex_file.get_download_url() assert download_url.startswith("http") # Start parse job parse_job = await dex_file.start_parse_job( IrisParseJobParams( options=IrisParseEngineOptions(), ) ) # Get results parse_result = await parse_job.get_result() # Access chunks print(f"Number of chunks: {len(parse_result.data.content.chunks)}") ``` *** ## Uploading Files from Memory You can also upload files directly from memory streams: ```python theme={null} import io # Create file from bytes file_bytes = b"your file content here" file_stream = io.BytesIO(file_bytes) # Upload from stream dex_file = await project.upload_file(file_stream, filename="document.pdf") # Parse as usual parse_job = await dex_file.start_parse_job( IrisParseJobParams( options=IrisParseEngineOptions(), ) ) ``` *** ## Configuration Options ### IrisParseEngineOptions Customize IRIS behavior using parse engine options: ```python theme={null} from dex_sdk.types import IrisParseEngineOptions # Configure IRIS options options = IrisParseEngineOptions( # Configuration parameters available based on your needs # (specific options depend on your Dex version) ) # Use custom options parse_job = await dex_file.start_parse_job( IrisParseJobParams(options=options) ) ``` *** ## Understanding Parse Results ### Result Structure IRIS returns parsed content organized into chunks, where each chunk represents a section of the document: ```python theme={null} parse_result = await parse_job.get_result() # Access all chunks chunks = parse_result.data.content.chunks # A ParseResult contains a list of ParseChunks. Each chunk has the # aggregated text plus the underlying ParseBlocks (one per detected # region) with bounding boxes, page numbers, and region types. for i, chunk in enumerate(parse_result.data.content.chunks): print(f"Chunk {i}: {chunk.content[:100]}...") for block in chunk.blocks: print(f" [{block.type}] page {block.page_number} " f"bbox={block.bbox} conf={block.confidence:.2f}") print(f" {block.content[:80]}...") ``` ### Working with Different Content Types IRIS detects and processes different content regions: Each `ParseBlock.type` is a layout label. IRIS returns capitalized labels such as `Text`, `Section-header`, and `Table`. The example uses `.lower()` to compare them with the lowercase labels below: * **Text-family:** text, title, caption, list-item, section-header, page-header, page-footer, formula, footnote * **Tables:** table * **Images / figures:** picture ```python theme={null} TEXT_LABELS = { "text", "title", "caption", "list-item", "section-header", "page-header", "page-footer", "formula", "footnote", } text_blocks = [ block for chunk in parse_result.data.content.chunks for block in chunk.blocks if block.type.lower() in TEXT_LABELS ] table_blocks = [ block for chunk in parse_result.data.content.chunks for block in chunk.blocks if block.type.lower() == "table" ] image_blocks = [ block for chunk in parse_result.data.content.chunks for block in chunk.blocks if block.type.lower() == "picture" ] ``` *** ## File Management ### List Files in Project ```python theme={null} # List the first page of files from dex_sdk.types import PaginationParams files = await project.list_files(pagination_params=PaginationParams(sort_by="created_at")) for file in files.items: print(f"File: {file.data.filename}") print(f" ID: {file.data.id}") print(f" Size: {file.data.size_bytes} bytes") ``` ### Get File Download URL ```python theme={null} # Get pre-signed S3 URL for downloading download_url = await dex_file.get_download_url() # Use URL with requests or other HTTP client import requests response = requests.get(download_url) file_content = response.content ``` ### Check File Details ```python theme={null} # Access file metadata print(f"File name: {dex_file.data.filename}") print(f"File ID: {dex_file.data.id}") print(f"File size: {dex_file.data.size_bytes} bytes") # Check if file is a FileEntity from dex_core.models.files import FileEntity assert isinstance(dex_file.data, FileEntity) ``` *** ## Error Handling ### Handle Upload Errors ```python theme={null} try: dex_file = await project.upload_file("path/to/file.pdf") except ValueError as e: print(e) # "File not found: path/to/file.pdf" except Exception as e: print(f"Upload failed: {e}") ``` ### Handle Parse Errors ```python theme={null} try: parse_job = await dex_file.start_parse_job( IrisParseJobParams( options=IrisParseEngineOptions(), ) ) parse_result = await parse_job.get_result() except Exception as e: print(f"Parse failed: {e}") ``` *** ## Processing Multiple Documents ### Batch Processing ```python theme={null} import asyncio async def process_documents(file_paths): # Upload all files dex_files = [] for path in file_paths: dex_file = await project.upload_file(path) dex_files.append(dex_file) # Start all parse jobs parse_jobs = [] for dex_file in dex_files: job = await dex_file.start_parse_job( IrisParseJobParams(options=IrisParseEngineOptions()) ) parse_jobs.append(job) # Wait for all to complete results = [] for job in parse_jobs: result = await job.get_result() results.append(result) return results # Usage file_paths = ["doc1.pdf", "doc2.pdf", "doc3.pdf"] results = await process_documents(file_paths) ``` *** ## Multi-Language Support Language coverage depends on the OCR models you configure. Confirm that your models support each required language, then evaluate accuracy on representative text, tables, and scanned pages. Choose models that support your document languages through `text_ocr` and `table_ocr` (both default to `openai/gpt-5.4`), or configure one task-conditioned model. Iris routes regions to those configured models by content type; it does not choose a different model automatically based on detected language. *** ## Best Practices ### Use Environment Variables for Credentials ```python theme={null} import os # Load credentials from environment account_id = os.getenv("SGP_ACCOUNT_ID") api_key = os.getenv("SGP_API_KEY") if not account_id or not api_key: raise ValueError("SGP credentials not found in environment") ``` ### Organize Files by Project ```python theme={null} # Create client dex_client = DexClient( base_url="your-dex-url", api_key=api_key, account_id=account_id, ) # Create separate projects for different document types invoices_project = await dex_client.create_project( name="invoices" ) contracts_project = await dex_client.create_project( name="contracts" ) ``` ### Wait for Parse Completion ```python theme={null} # Parse jobs are async - ensure completion before accessing results parse_job = await dex_file.start_parse_job( IrisParseJobParams(options=IrisParseEngineOptions()) ) # This waits for completion parse_result = await parse_job.get_result() # Now safe to access results chunks = parse_result.data.content.chunks ``` *** ## Common Use Cases ### Extract All Text from Document ```python theme={null} def extract_all_text(parse_result): """Extract all text content from parse result.""" all_text = [] for chunk in parse_result.data.content.chunks: all_text.append(chunk.content) return "\n\n".join(all_text) # Usage parse_result = await parse_job.get_result() full_text = extract_all_text(parse_result) print(full_text) ``` ### Process Forms and Structured Documents Replace `process_table` and `process_text` with your own processing functions. ```python theme={null} # IRIS automatically detects and extracts structured content parse_result = await parse_job.get_result() TEXT_LABELS = { "text", "title", "caption", "list-item", "section-header", "page-header", "page-footer", "formula", "footnote", } # Access content types on blocks within each chunk for chunk in parse_result.data.content.chunks: for block in chunk.blocks: if block.type.lower() == "table": process_table(block.content) elif block.type.lower() in TEXT_LABELS: process_text(block.content) ``` *** ## Performance Considerations ### Processing Time Parse job duration depends on: * Document length (number of pages) * Image resolution and quality * Content complexity (tables, mixed layouts) * Selected OCR models ### Optimization Tips * **Batch processing**: Process multiple documents concurrently when possible * **Pre-processing**: Ensure documents are properly oriented and of good quality * **Project organization**: Group related documents in the same project for better management *** ## Next Steps Now that you understand how to use IRIS: 1. Review the [Introduction to IRIS](/docs/capabilities/ocr/introduction-to-iris) to learn about the underlying OCR pipeline 2. Explore the [Dex documentation](/docs/capabilities/document-understanding/introduction-to-dex) for additional capabilities 3. Integrate IRIS into your document processing workflows 4. Use parsed results with Dex's extraction and vector store features For questions or support, please contact the Scale AI team. # Introduction to IRIS Source: https://docs.gp.scale.com/docs/capabilities/ocr/introduction-to-iris Explore IRIS layout detection, OCR models, and document assembly for custom parsing pipelines. **IRIS is Scale's OCR capability that transforms document images and PDFs into structured text.** It provides a comprehensive "layout-then-OCR" pipeline that combines **layout detection, specialized OCR models, and intelligent assembly** to extract meaningful information from complex documents. *** ## When to Use IRIS? IRIS is designed for teams that need: * **Custom OCR pipeline control:** Fine-tune layout detection, OCR models, and assembly for specialized document types * **Maximum accuracy optimization:** Experiment with the OCR models available through SGP to find the best fit for your specific documents * **Minimum latency requirements:** Optimize processing speed for your specific use case through custom configuration * **Specialized model development:** Use OCR models you deploy in SGP for unique document challenges * **Multi-language experimentation:** Evaluate available OCR models on the languages in your documents ### For Standard Production Use > [!WARNING] > > **Deprecation notice:** Reducto is being deprecated. Use Iris for new projects. Existing projects that need stable batch document processing without custom requirements can stay on **Reducto within Dex** until it is retired: * **Production stability** with proven reliability * **Good for batch processing** with somewhat high but acceptable latency * **Auto-scaling** with your workload * **Lower operational complexity** Before production use, validate parsing accuracy on representative documents from your workload. See the **[When to choose Iris?](/docs/capabilities/ocr/when-to-choose-iris)** guide for a detailed comparison. *** ## IRIS Capabilities IRIS provides advanced capabilities for teams with specialized OCR needs: * **Layout-Aware Processing:** Automatically detects and classifies different regions (text, tables, images, etc.) before applying specialized OCR * **SGP-hosted OCR models:** Choose the SGP models that read text regions (`text_ocr`) and tables (`table_ocr`), named as `vendor/name` (default `openai/gpt-5.4`) * **Table-Specific Processing:** Apply specialized OCR models optimized for tabular data extraction * **Extensible Architecture:** Add custom OCR models by deploying them in SGP; layout detection uses one of the built-in models * **Multi-Language Evaluation:** Test deployed models on your required languages and document types * **Complete Pipeline Control:** Configure layout detection and content extraction through `IrisParseEngineOptions` *** ## How IRIS Works The default Iris mode follows the three stages below. Two alternatives change this pipeline: * `native_text_mode=True` uses the embedded text layer of native PDFs after layout detection and calls no OCR models. Regions without embedded text, including scanned pages, produce blank content. * `e2e_ocr` sends a full page to one end-to-end model, bypassing separate layout detection. Set its required `e2e_response_parser`; do not also set `layout`, `text_ocr`, or `table_ocr`. Native-text and end-to-end OCR modes are mutually exclusive. ### 1. Layout Detection Detects and classifies regions in document images with bounding boxes and content types: * **Text regions** for paragraphs and body content * **Table regions** for structured tabular data * **Image regions** for figures and diagrams * **Other content types** as needed Supported layout detection models: * **rt_detr_bce:** RT-DETR model (best performance) * **pp_doclayout_v3:** PP-DocLayoutV3 model * **whole_page:** No detection; each page is one region ### 2. OCR Processing Applies specialized OCR models to each detected region based on content type: * **Text OCR:** Optimized for continuous text (paragraphs, headings, etc.) * **Table OCR:** Specialized models for accurate table extraction * **Image Handling:** Describes image regions with `image_description_model`, embeds them as base64, or skips them, depending on `img_method` Text and table regions go to the SGP models you name in `text_ocr` and `table_ocr`, written as `vendor/name` (for example `openai/gpt-5.4`, the default). Any vision model available through SGP can be used, including models you deploy in SGP. Use `e2e_ocr` instead for full-page OCR. A task-conditioned model configuration can supply model-specific prompts and output formats. See [Getting Started with IRIS](/docs/capabilities/ocr/getting-started-with-iris) for a parsing walkthrough. ### 3. Assembly Combines extracted content into a Dex parse result: a list of chunks, each with its text and the blocks (regions) it came from, including each block's type, bounding box, page number, and confidence. *** ## Core Features ### Modular Architecture The layout detector and the OCR models are configured independently. Add custom OCR models by deploying them in SGP. ### Inspection and Debugging Each parse result keeps the layout and OCR output for every region, so you can inspect the processing: * Layout detection bounding boxes and classifications (`block.bbox`, `block.type`) * Individual OCR outputs per region (`block.content`, plus `block.error` when a region could not be read) * Final assembled output (`chunk.content`) ### Configuration Management Set options on `IrisParseEngineOptions` in each parse request: ```python theme={null} IrisParseEngineOptions( layout="rt_detr_bce", text_ocr="openai/gpt-5.4", table_ocr="openai/gpt-5.4", ) ``` ### Performance Optimization Choose layout and OCR configurations by measuring latency and accuracy on representative documents. *** ## Common Use Cases * **Document Digitization:** Convert scanned documents and PDFs to searchable text * **Form Processing:** Extract structured data from forms with mixed content types * **Table Extraction:** Accurately capture tabular data from complex documents * **Multi-Language Documents:** Evaluate and process documents in languages supported by your configured models * **Research and Analysis:** Extract text and data from academic papers, reports, and technical documents * **Archive Processing:** Batch process large document collections *** ## Getting Started IRIS is integrated with Dex for seamless document processing: ### Python Package Use programmatically through the Dex SDK: ```python theme={null} from dex_sdk.client import DexClient from dex_core.models.parse_job import IrisParseEngineOptions, IrisParseJobParams # Initialize client and project dex_client = DexClient(base_url="your-dex-url") project = await dex_client.create_project(name="ocr-project") # Upload and parse document dex_file = await project.upload_file("document.pdf") parse_job = await dex_file.start_parse_job( IrisParseJobParams(options=IrisParseEngineOptions()) ) result = await parse_job.get_result() ``` *** ## Language Support Language support comes from the OCR models you configure, and the models available depend on your deployment. Confirm that a model suited to each required language is available, then evaluate accuracy on representative text, tables, and scanned pages. *** ## Key Advantages ### Extensibility Use gateway model paths and configurable prompts to adapt OCR to your document types. Custom models must be deployed in SGP before IRIS can use them. ### Transparency Inspect every pipeline stage to understand model behavior, debug issues, and optimize results for your specific documents. ### Flexibility Mix and match components—use different OCR models for text vs. tables, choose layout detectors based on document type, and customize assembly output. ### API Integration The default `model_gateway="sgp"` (SDK `0.8.2` and later) uses your SGP account and API key to access the models configured for your deployment. *** ## Next Steps To start using IRIS: 1. Review the [Getting Started with IRIS](/docs/capabilities/ocr/getting-started-with-iris) guide for step-by-step instructions 2. Learn about IRIS's multi-stage OCR pipeline and configuration options 3. Integrate IRIS into your Dex-based document processing workflows IRIS provides the building blocks for robust document OCR while giving you complete control over the processing pipeline. # DEX model engine choices Source: https://docs.gp.scale.com/docs/capabilities/ocr/when-to-choose-iris A guide to the DEX model engine options DEX is the SDK that runs document parsing, extraction, retrieval and research flows. There are currently two options for the underlying parsing engine. 1. *IRIS* - Scale's configurable OCR engine (V2), with a selectable layout detector and configurable OCR models. 2. *REDUCTO* - Currently being deprecated - Reducto is a third party provider that provides end to end parsing of a wide range of documents without the performance & customisation available with IRIS V2 > [!WARNING] > > **Deprecation notice:** Reducto is being deprecated. Use Iris for new projects. On a dedicated SGP deployment, Iris V2 is off by default. Ask your Scale team to enable it. ## Quick Decision Guide ### Use Iris (Default engine) **Iris(v2) is the recommended default DEX engine for all production use cases:** * You can **evaluate accuracy** by testing OCR models available through SGP * You can **minimize latency** through specialized optimization * You need **greater transparency** of the processes within Temporal * You can use **specialized models** for unique document types * You can **customise and control pipelines** for layout detection, OCR, and assembly ### Use Reducto (Soon to be Deprecated) **Reducto is the legacy OCR DEX engine for most production use cases:** * You can accept a *third party service* for OCR and parsing integrated into your application * You need **proven stability** and do not need future functionality or updates from the DEX team * You're doing **batch processing** where somewhat high latency is acceptable * You want **automatic scaling** with your workload * You're processing **English or Germanic language documents** * You need a **low-complexity, managed solution** ## Understanding the Relationship Between Iris and Dex **Iris and Dex are not mutually exclusive**—they're complementary. Dex is a document understanding platform that provides primitives for file management, parsing, vector stores, and data extraction. Iris is one of several OCR engines available within Dex. Think of it this way: *Dex is the platform, Iris is one of the engines*. When you use Dex, you choose which OCR engine to use for parsing: * **Iris**: Configurable OCR; evaluate accuracy, latency, and language coverage for your deployed models * **Custom engines**: Integrate your own OCR solution * **Reducto**: (Deprecated) ## What is Iris? Iris is Scale's OCR capability that provides a flexible, modular pipeline for extracting text from documents. It's designed for teams with custom OCR needs who want complete control over the processing pipeline. **Iris offers:** * **Customisable OCR models**: Any model available through SGP, named as `vendor/name` (for example `openai/gpt-5.4`), including specialized models uploaded to SGP * **Complete pipeline control**: Configure layout detection and OCR processing separately * **Inspection capabilities**: Review each region's bounding box, type, confidence, and OCR output in the parse result * **Extensibility**: Add custom OCR models through SGP; layout detection uses `rt_detr_bce`, `pp_doclayout_v3`, or `whole_page` * **Optimization flexibility**: Fine-tune for maximum accuracy or minimum latency based on your needs * **Custom pipeline experimentation**: Test different combinations of layout detectors and OCR models * **Specialized model development**: Build custom OCR for unique document types or languages * **Performance optimization**: Tune for specific accuracy or latency requirements * **Non-Germanic language optimization**: Experiment with models to find best accuracy for other languages such as Arabic. ## What is Dex? Dex is Scale's document understanding platform—a production-ready service that transforms unstructured documents into actionable, structured data. It provides: * **File Management**: Secure upload, storage, and retrieval with access control * **Document Parsing**: Convert documents (PDF, DOCX, images) into structured JSON using multiple OCR engines * **Vector Stores**: Index and search parsed documents with semantic embeddings * **Data Extraction**: Extract information using custom schemas, prompts, and RAG-enhanced context * **Project Management**: Organize and isolate data with proper authorization * **Automatic scaling** with your workload * **Multiple OCR engine options**: Iris (Recommended default, V2) & Reducto (soon to be deprecated) ## Feature Comparison | Feature | Iris (✅ Recommended) | Reducto (⚠️ Soon to be Deprecated) | | - | - | - | | **Engine** | Scale proprietary OCR (V2) | Third-party end-to-end parsing | | **Best For** | Custom OCR control, specialized requirements | Low-complexity managed solution, legacy production | | **Performance** | Tunable through model and pipeline choice | Managed pipeline with limited tuning | | **OCR Models** | Any model available through SGP, such as `openai/gpt-5.4` | Managed Reducto engine | | **Pipeline Control** | Configure layout detection and OCR as needed | Managed end-to-end | | **Languages** | Depends on your deployed OCR models | Verify coverage for your document languages | | **Latency** | Minimizable through high parallelism (V2) | Somewhat high (acceptable for batch) | | **Transparency** | Greater visibility into processes within Temporal | Third-party service integrated into your application | | **Future updates** | Active development from the DEX team | No future functionality or updates from the DEX team | | **Typical use case** | Maximize accuracy, minimize latency, specialized or custom models | Batch processing where higher latency is acceptable | ## Recommended Workflow ### For Standard Production Applications **Use Iris as the DEX default model engine:** 1. **SOTA performance**: Accuracy optimization, latency minimization, or specialized models 2. **Customisability**: Test OCR models available through SGP and pipeline configurations 3. **Pipeline control**: Configure layout detection and OCR as separate steps 4. **Language coverage**: Test deployed models on Arabic, English, and any other languages in your documents 5. **Transparency**: Inspect and debug processing stages within Temporal 6. **Extensibility**: Add custom OCR models through SGP 7. **Future-ready**: Benefit from ongoing development and Iris V2 parallelism improvements from the DEX team **Use Reducto within Dex** for reliable, production-ready document processing: 1. **Managed infrastructure**: Rely on managed infrastructure for operational simplicity 2. **Language fit**: Process English or Germanic-language documents where Iris customization is not required 3. **Low complexity**: Use a third-party, end-to-end managed parsing service with minimal pipeline setup ## Getting Started ### Dex Documentation * [Introduction to Dex](/docs/capabilities/document-understanding/introduction-to-dex) * [Getting Started with Dex](/docs/capabilities/document-understanding/getting-started-with-dex) ### Iris Documentation * [Introduction to Iris](/docs/capabilities/ocr/introduction-to-iris) * [Getting Started with Iris](/docs/capabilities/ocr/getting-started-with-iris) ## Summary * **Dex is the platform**: DEX runs parsing, extraction, retrieval, and research flows—you choose the underlying engine when you parse documents. * **Iris (V2) is the recommended default**: Configure layout, gateway OCR models, and assembly, then validate the pipeline on your document types and languages. * **Choose Iris when you need customization**: Optimize accuracy or latency, inspect pipelines in Temporal, use specialized or custom models, and support non-Germanic languages. * **Reducto is legacy and being deprecated**: A third-party, managed end-to-end option with no future DEX updates—use only for existing low-complexity or English/Germanic batch workloads. * **Reducto still fits narrow cases**: Proven stability, auto-scaling, and acceptable batch latency when you do not need Iris V2 performance or pipeline control. Need help deciding? Ask your Scale team. # SGP Capabilities Overview Source: https://docs.gp.scale.com/docs/capabilities/overview SGP provides a comprehensive suite of capabilities for building, deploying, and managing AI applications at scale. Our platform combines powerful tools for development, evaluation, and production deployment. ### 📄 Document Understanding Advanced document processing and information extraction capabilities. [Learn more about Document Understanding →](/docs/capabilities/document-understanding/introduction-to-dex) **Key Features:** * **Universal Format Support**: PDFs, images, spreadsheets, presentations * **Intelligent Parsing**: Convert documents to structured JSON * **Custom Extraction**: Define schemas for specific data needs * **High Accuracy OCR**: Advanced text recognition and extraction * **Project Management**: Secure, isolated document processing ### OCR Iris is Scale's OCR engine for Dex parse jobs. It detects layout regions on each page and sends each region to an SGP model to read it. [Learn more about OCR](/docs/capabilities/ocr/introduction-to-iris) **Key Features:** * **Layout detection**: Split pages into regions with `rt_detr_bce` or `pp_doclayout_v3`, or treat each page as one region with `whole_page` * **Model choice**: Pick the SGP models that read text, tables, and images, such as `openai/gpt-5.4` * **Structured output**: Get chunks and blocks with block types, bounding boxes, and confidence scores * **Part of Dex**: Run Iris from Dex parse jobs with `ParseEngine.IRIS` ### ⚙️ Workflows No-code/low-code durable workflow orchestration platform that enables enterprise operators to import data, transform it, call LLMs or Agents, analyze or evaluate the output, and automate the entire workflow. [Learn more about Workflows →](/docs/capabilities/workflows/introduction-to-workflows) **Key Features:** * **Importing data**: Read data from various sources like agent traces, CSVs, blob storages, or queryable data sources. * **Transforming data**: Transform your data using custom `Python` code or simple functions. Extract structured data from PDFs & documents. * **Call LLMs or Agents**: Generate output and traces against input from each row of your dataset using LLMs or `Agentex` agents. * **Evaluate data**: Evaluate your outputs using `LLM as Judge` and access the evaluation anytime on SGP. * **Visualize data**: Generate visualizations like charts & build custom dashboards. * **Export the data**: Export the data as a CSV or SGP dataset or SGP Evaluation. * **Automate & monitor**: Schedule the workflow to run periodically. Execute a workflow via an API call. Monitor logs & get alerts. ### Training Train custom models on your own data without managing cloud infrastructure. Submit training jobs through a single API and Train handles the rest, whether your workload runs on GCP, AWS, or Azure. [Learn more about Training](/docs/capabilities/training/introduction-to-train) **Key Features:** * **One API on every cloud**: Run jobs on Vertex AI, SageMaker, or Azure ML, whichever your deployment's cloud provides, through the same job API * **Bring your own image**: Push a Docker image once and reference it in any job * **Portable data access**: Mount GCS, S3, or Azure Blob Storage into your container. On Vertex AI and Azure ML, Train exposes each mount through the same `STORAGE_MOUNT__PATH` environment variables * **Simple job lifecycle**: Track progress from submission to completion with a consistent status model across all backends ### Code Authoring A git server and CI runners that run inside your SGP deployment, powered by Gitea. [Learn more about Code Authoring](/docs/capabilities/code-authoring/introduction-to-code-authoring) **Key Features:** * **Git over HTTPS**: Clone, push, and review pull requests with your SGP API key as the git password * **In-cluster CI/CD**: Run GitHub Actions-compatible workflows from `.gitea/workflows/` on runners inside your deployment * **SGP credentials only**: Sign in to the web UI with your SGP login and call the REST API with your SGP API key # Custom Images Source: https://docs.gp.scale.com/docs/capabilities/training/custom-images Push Docker images to the SGP registry and use them in training jobs. Each SGP workspace has a Docker-compatible registry. You push images to it and reference them in job configs; Train takes care of getting the right image onto the right cloud backend. *** ## Push an Image **1. Authenticate** Log in with your SGP credentials: ```bash theme={null} docker login docker-registry.your-sgp-deployment-url \ --username $SGP_ACCOUNT_ID \ --password $SGP_API_KEY ``` **2. Build and tag** ```bash theme={null} docker build -t my-training-job:v1 . docker tag my-training-job:v1 docker-registry.your-sgp-deployment-url/my-training-job:v1 ``` **3. Push** ```bash theme={null} docker push docker-registry.your-sgp-deployment-url/my-training-job:v1 ``` **4. Reference in your job config** Use the registry URL directly in your job config. Train resolves it automatically at submission: ```json theme={null} "image_uri": "docker-registry.your-sgp-deployment-url/my-training-job:v1" ``` Train ensures the exact image you pushed is what runs, even if the tag is updated later. ## Multi-Architecture Builds Cloud training instances run `linux/amd64`. If you're building on Apple Silicon, cross-compile explicitly: ```bash theme={null} docker buildx build \ --platform linux/amd64 \ -t docker-registry.your-sgp-deployment-url/my-training-job:v1 \ --push . ``` > [!WARNING] > > Pushing a `linux/arm64` image to an `amd64` compute instance will cause the job to fail at startup with an exec format error. *** ## Example Dockerfile ```dockerfile theme={null} FROM pytorch/pytorch:2.2.0-cuda12.1-cudnn8-runtime WORKDIR /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY train.py . CMD ["python3", "train.py"] ``` On Vertex AI and Azure ML, your training script accesses data through `STORAGE_MOUNT_*_PATH` env vars, with no cloud-specific paths needed (SageMaker does not set them; see [Storage Mounts](/docs/capabilities/training/storage-mounts)): ```python focus={3-4} theme={null} import os import torch data_dir = os.environ["STORAGE_MOUNT_0_PATH"] output_dir = os.environ["STORAGE_MOUNT_1_PATH"] # ... training loop ... torch.save(model.state_dict(), f"{output_dir}/model.pt") ``` *** ## Next Steps * **[Storage Mounts](/docs/capabilities/training/storage-mounts)**: Configure data and output directories * **[Getting Started](/docs/capabilities/training/getting-started-with-train)**: Full job submission examples per backend # Getting Started with Train Source: https://docs.gp.scale.com/docs/capabilities/training/getting-started-with-train Submit your first GPU training job on Vertex AI, SageMaker, or Azure ML. This guide walks you through submitting a training job using the Train API. ## Prerequisites Before you begin, ensure you have: * A valid Scale account with SGP access * Your API key and account ID: ```bash theme={null} export SGP_API_KEY="your_api_key" export SGP_ACCOUNT_ID="your_account_id" ``` * A Docker image pushed to the workspace registry (see [Custom Images](/docs/capabilities/training/custom-images)) ## Submit a Job Jobs are submitted to `POST /v1/{backend}/jobs`. The `{backend}` is one of `vertex-ai`, `sagemaker`, or `azure-ml`, and each install enables only the backends configured for its cloud. The `job_config` structure mirrors each cloud's native job API. Every `storage_mounts` location must be storage your install allows; see [Storage Mounts](/docs/capabilities/training/storage-mounts). > [tab] Vertex AI > > ```bash theme={null} > curl -X POST "https://train.your-sgp-deployment-url/v1/vertex-ai/jobs" \ > -H "x-api-key: $SGP_API_KEY" \ > -H "x-selected-account-id: $SGP_ACCOUNT_ID" \ > -H "Content-Type: application/json" \ > -d '{ > "name": "mnist-training", > "job_config": { > "worker_pool_specs": [{ > "machine_spec": { > "machine_type": "g2-standard-8", > "accelerator_type": "NVIDIA_L4", > "accelerator_count": 1 > }, > "replica_count": 1, > "container_spec": { > "image_uri": "docker-registry.your-sgp-deployment-url/pytorch-gpu:v1", > "command": ["python3", "train.py"], > "args": ["--epochs", "3"] > } > }], > "scheduling": {"timeout_seconds": 86400} > }, > "storage_mounts": [ > {"source_uri": "gs:///mnist-data", "mount_path": "/mnt/data", "read_only": true}, > {"source_uri": "gs:///run-1", "mount_path": "/mnt/output", "read_only": false} > ] > }' > ``` > > ```json Response highlight={4} theme={null} > { > "id": "87cae349-cd5f-4202-b674-f1652880a8f5", > "name": "mnist-training", > "job_type": "vertex_ai", > "status": "PENDING", > "cloud_job_identifier": { > "job_name": "projects/my-project/locations/us-east1/customJobs/1234567890" > }, > "storage_mounts": [...], > "account_id": "693af856dde70fd6614ab64c", > "created_at": "2026-03-11T08:12:25.542507Z", > "updated_at": "2026-03-11T08:12:25.542507Z" > } > ``` > > `worker_pool_specs` is an array; add more pools for distributed training. Train resolves only the first pool's `image_uri` from the workspace registry; other pools' `image_uri` values go to Vertex AI unchanged. `command` and `args` are separate arrays. > [tab] SageMaker > > ```bash theme={null} > curl -X POST "https://train.your-sgp-deployment-url/v1/sagemaker/jobs" \ > -H "x-api-key: $SGP_API_KEY" \ > -H "x-selected-account-id: $SGP_ACCOUNT_ID" \ > -H "Content-Type: application/json" \ > -d '{ > "name": "mnist-training", > "job_config": { > "image": "docker-registry.your-sgp-deployment-url/pytorch-gpu:v1", > "command": ["python3", "/app/train.py"], > "resource_config": { > "instance_count": 1, > "instance_type": "ml.g4dn.xlarge", > "volume_size_in_gb": 50 > }, > "output_data_config": {"s3_output_path": "s3:///output"}, > "checkpoint_config": {"s3_uri": "s3://"}, > "hyper_parameters": {"learning_rate": "0.001"} > }, > "storage_mounts": [ > {"source_uri": "s3:///mnist-data", "mount_path": "/mnt/data", "read_only": true}, > {"source_uri": "s3:///run-1", "mount_path": "/mnt/output", "read_only": false} > ] > }' > ``` > > ```json Response highlight={4} theme={null} > { > "id": "3f2a1b9c-88de-4a12-b901-c2e5f7d04321", > "name": "mnist-training", > "job_type": "sagemaker", > "status": "PENDING", > "cloud_job_identifier": { > "training_job_name": "mnist-training-2026-03-11-08-12-25" > }, > "storage_mounts": [...], > "account_id": "693af856dde70fd6614ab64c", > "created_at": "2026-03-11T08:12:25.542507Z", > "updated_at": "2026-03-11T08:12:25.542507Z" > } > ``` > > `instance_type` determines GPU: `ml.g4dn.xlarge` = T4, `ml.p4de.24xlarge` = 8×A100 80GB. `output_data_config` and `checkpoint_config` are required; the request fails with `422` without them. A writable `s3://` storage mount replaces `output_data_config.s3_output_path`, so this job's S3 output path is `s3:///run-1`. > [tab] Azure ML > > ```bash theme={null} > curl -X POST "https://train.your-sgp-deployment-url/v1/azure-ml/jobs" \ > -H "x-api-key: $SGP_API_KEY" \ > -H "x-selected-account-id: $SGP_ACCOUNT_ID" \ > -H "Content-Type: application/json" \ > -d '{ > "name": "mnist-training", > "job_config": { > "command": "python train.py --epochs 3", > "environment": { > "image": "docker-registry.your-sgp-deployment-url/pytorch-gpu:v1" > }, > "compute": "gpu-cluster", > "resources": { > "instance_type": "Standard_NC24ads_A100_v4", > "instance_count": 1 > }, > "environment_variables": {"LEARNING_RATE": "0.001"} > }, > "storage_mounts": [ > {"source_uri": "azureml://datastores/traindata/paths/mnist", "mount_path": "/mnt/data", "read_only": true}, > {"source_uri": "azureml://datastores/trainoutput/paths/run-1", "mount_path": "/mnt/output", "read_only": false} > ] > }' > ``` > > ```json Response highlight={4} theme={null} > { > "id": "a1b2c3d4-ef56-7890-abcd-ef1234567890", > "name": "mnist-training", > "job_type": "azure_ml", > "status": "PENDING", > "cloud_job_identifier": { > "job_name": "heroic_grass_22zggyxsxp" > }, > "storage_mounts": [...], > "account_id": "693af856dde70fd6614ab64c", > "created_at": "2026-03-11T08:12:25.542507Z", > "updated_at": "2026-03-11T08:12:25.542507Z" > } > ``` > > `command` is a single shell string (not an array). `compute` can be a named cluster or `"serverless"`. ## Job Response All backends return the same job object shape: **id** `string` *required* Unique job identifier. Use this to poll status or cancel the job. **status** `string` *required* Current job state. Transitions through `PENDING → IN_PROGRESS → COMPLETED / FAILED / CANCELED / EXPIRED`. A cancel request moves an active job to `CANCELING` until the backend reports a final state. **cloud_job_identifier** `object` The cloud provider's native job identifier. Shape varies by backend: `job_name` for Vertex AI and Azure ML, `training_job_name` for SageMaker. **result** `object` Populated on failure. Contains `failure_reason` with the error message from the cloud provider. ## Check Status ```bash theme={null} curl "https://train.your-sgp-deployment-url/v1/jobs/$JOB_ID" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` ## Other Endpoints **List jobs:** ```bash theme={null} curl "https://train.your-sgp-deployment-url/v1/jobs" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` **Cancel a job:** ```bash theme={null} curl -X POST "https://train.your-sgp-deployment-url/v1/jobs/$JOB_ID/cancel" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` Returns `202` with the job in `CANCELING`. Cancelling a job that is already `CANCELING` or `CANCELED` returns it unchanged. Returns `409` if the job already ended as `COMPLETED`, `FAILED`, or `EXPIRED`. **List a job's lifecycle events:** ```bash theme={null} curl "https://train.your-sgp-deployment-url/v1/jobs/$JOB_ID/events" \ -H "x-api-key: $SGP_API_KEY" \ -H "x-selected-account-id: $SGP_ACCOUNT_ID" ``` ## Accessing Mounts in Your Container On Vertex AI and Azure ML, mounted storage is available via environment variables: ```python theme={null} import os data_path = os.environ["STORAGE_MOUNT_0_PATH"] output_path = os.environ["STORAGE_MOUNT_1_PATH"] ``` SageMaker does not set these variables; see [Storage Mounts](/docs/capabilities/training/storage-mounts) for where its mounts appear. ## Next Steps * **[Custom Images](/docs/capabilities/training/custom-images)**: Build and push your own training containers * **[Storage Mounts](/docs/capabilities/training/storage-mounts)**: Mount multiple datasets and configure output paths # Introduction to Train Source: https://docs.gp.scale.com/docs/capabilities/training/introduction-to-train Run GPU training jobs across GCP, AWS, and Azure through a unified API. **Train is SGP's multi-cloud training job service.** It provides a unified REST API for submitting GPU training jobs to Vertex AI (GCP), SageMaker (AWS), and Azure ML, handling cloud-specific job submission, image resolution, storage mounting, and status tracking transparently. *** ## Why Use Train? Each cloud ML platform has its own job submission API, container registry, storage format, and authentication model. Teams that want cloud flexibility end up maintaining separate training pipelines per cloud, or locking into one provider early. Train solves this with a single API surface in front of the cloud your deployment runs on. You submit a job with an image and a command; Train takes care of pushing the image to the right registry, mounting your storage, and translating the job config into the cloud's native format. * **Single API for all backends:** `vertex-ai`, `sagemaker`, and `azure-ml` share one job API; each install enables the backends configured for its cloud * **Unified image registry:** Push Docker images once to the registry proxy; Train resolves them to the correct cloud registry at submission time * **Cross-cloud storage mounts:** Mount GCS, S3, or Azure Blob Storage with the same `storage_mounts` field on every backend * **Consistent job lifecycle:** `PENDING → IN_PROGRESS → COMPLETED / FAILED / CANCELED / EXPIRED` across all backends, with `CANCELING` while a cancel request is in progress *** ## How It Works ### Train Service The Train service has two components: a gateway that accepts job submissions and returns a job ID, and a background poller that tracks running jobs and updates their status. ### Supported Backends | Backend | Cloud service | `{backend}` value | | - | - | - | | Vertex AI | GCP Custom Jobs | `vertex-ai` | | SageMaker | AWS Training Jobs | `sagemaker` | | Azure ML | Azure ML Command Jobs | `azure-ml` | ### Container Registry Each workspace has a Docker-compatible registry. You push images to it; Train ensures the exact image you pushed is what runs on the cloud backend, even if the tag is updated later. *** ## Storage Mounts Training containers on Vertex AI and Azure ML receive `STORAGE_MOUNT_0_PATH`, `STORAGE_MOUNT_1_PATH`, etc. as environment variables pointing to mounted directories. Your training script reads from `os.environ["STORAGE_MOUNT_0_PATH"]`, with no cloud-specific paths or SDKs needed. SageMaker delivers read-only mounts as input channels under `/opt/ml/input/data/` instead; see [Storage Mounts](/docs/capabilities/training/storage-mounts). Source URIs use cloud-native formats: | Cloud | URI format | | - | - | | GCP | `gs://bucket/path` | | AWS | `s3://bucket/path` | | Azure | `azureml://datastores/name/paths/path` | *** ## Common Use Cases * **Improve agent accuracy on your domain:** Fine-tune a foundation model on your proprietary data so your agents produce more reliable, on-brand responses and make fewer errors on the tasks that matter to your business. * **Optimize agent performance with feedback:** Use evaluations and human feedback collected in SGP to run reinforcement learning pipelines that directly improve how your agents behave over time. * **Reduce inference cost and latency:** Distill a large general-purpose model into a smaller one trained specifically for your use case, so agents respond faster and cost less to run at scale. *** ## Authentication Train uses the same authentication as other SGP services: ``` x-api-key: x-selected-account-id: ``` The registry proxy accepts Docker Basic auth (`account_id:api_key`) during `docker login`, then issues a short-lived token for push and pull operations. *** ## Getting Started To begin using Train, you'll need a Scale account with SGP access and credentials for at least one cloud backend. > [card] [Getting Started](/docs/capabilities/training/getting-started-with-train) > > Submit your first training job > [card] [Custom Images](/docs/capabilities/training/custom-images) > > Push and use your own Docker images > [card] [Storage Mounts](/docs/capabilities/training/storage-mounts) > > Mount training data and output directories across clouds # Guide to document understanding in Workflows Source: https://docs.gp.scale.com/docs/capabilities/workflows/dex-parsing-in-workflows Tutorial for utilizing unstructured data for your workflows This guide walks you through parsing documents and extracting structured data in Workflows to help you utilize any unstructured data for your workflows (powered by our `Dex SDK`). You'll learn how to connect to cloud file storage, filter and parse documents, extract structured data, store parsed data to Knowledge Base, and use the results in your workflows. ### Step 1: Connect to cloud file storage Start by navigating to `Workflows` in SGP. Click on the `Cloud Storage Browser` card to connect to your cloud file storage. ![Cloud Storage Browser Card](https://mintcdn.com/scalegp/gkIBcTQjtllZVAnH/images/compass/cloud-storage-browser.png?fit=max\&auto=format\&n=gkIBcTQjtllZVAnH\&q=85\&s=c2cfb0d3d8553f39fd8659304e5ada87) Currently, Workflows supports the following cloud storage providers: * Amazon S3 * Azure Blob Storage * Google Cloud Storage For this example, we'll connect to an Amazon S3 bucket containing resume documents that we want to parse and analyze. Select your cloud storage provider and choose your registered connection from the dropdown. Browse or specify the path to the files you want to process. > [!NOTE] > > The Cloud Storage Browser card can read an S3 bucket only after Workflows is given access to it. Ask your Scale team to enable access to your bucket. ### Step 2: Filter files (optional) Before parsing, you can use a `Filter` card to select only the files you want to process. This helps you narrow down your dataset and avoid parsing unnecessary documents. ![Filter Files Card](https://mintcdn.com/scalegp/gkIBcTQjtllZVAnH/images/compass/filter-files.png?fit=max\&auto=format\&n=gkIBcTQjtllZVAnH\&q=85\&s=5dd9c201734bc3638c5b0297a56e1345) Purely as an example, here we only filter for resumes of a certain size. This step is optional but recommended when working with large file sets to optimize processing time and costs. ### Step 3: Parse & Extract from documents with the Dex Card This feature utilizes our Document Understanding capability, our `Dex SDK` on the backend. Add a `Dex` card to extract structured data from your documents. This card utilizes the Dex SDK in the backend and creates or reuses a Dex project, depending on the `Dex Project` option below, that you can reference later in the Dex UI. ![Dex Project configs](https://mintcdn.com/scalegp/0fQRQwgv9afUT3rO/images/compass/dex-project.png?fit=max\&auto=format\&n=0fQRQwgv9afUT3rO\&q=85\&s=6c781435c29e975a9a95d76328751b5a) #### Dex Project Configuration Under `Dex Project`, choose how the card picks its project: * `Create New`: Creates a new project each time the workflow runs. Set `Project Name` to a descriptive name (e.g., "Resume Parsing Project") * `Upsert (by Name)`: Reuses the project whose name matches `Project Name` across runs * `Use Existing (by ID)`: Uses the project in `Project ID` #### Parsing This is the step that converts complex, unstructured files into LLM-ready JSON or markdown format. ![Dex Parse configs](https://mintcdn.com/scalegp/0fQRQwgv9afUT3rO/images/compass/dex-parse.png?fit=max\&auto=format\&n=0fQRQwgv9afUT3rO\&q=85\&s=9cf77e21769f35143ed6863aebd8d79e) > [!WARNING] > > **Deprecation notice:** Reducto is being deprecated. Use Iris for new projects. Choose the OCR engine based on your document language and layout: * `Engine`: Select `Reducto` or `Iris`. **Reducto**: Best for English and Latin-script documents with tables, figures, and complex layouts. **Iris**: Best for non-English, non-Latin scripts (Arabic, CJK, Thai, Hindi, etc.) For Iris, you can set the `Layout Model`, `Text OCR Model`, and `Table OCR Model`, along with prompts and confidence thresholds. For Reducto, configure the following settings to select how documents should be split into chunks: * `Variable (recommended)`: Automatically determines optimal chunk boundaries * `Chunk Size (Optional)`: Set to `Auto` to let the engine determine the best size You can also enable `contextual retrieval`, which uses an LLM to prepend document-level context to each chunk, improving retrieval accuracy. Control the visibility of the output data: `Include parsed data in output`: Toggle on to include `parsedContent`, which joins the text of all chunks. Output remains one row per input file for each selected parse configuration. * When enabled: Includes the joined parsed text * When disabled: Includes file metadata and result identifiers without parsed text Finally, you can configure parse retries and per-file parse timeouts as per your workflow's needs. #### Structured Extraction Optionally extract specific fields from parsed documents using LLM-based extraction. This feature pulls targeted data from unstructured documents into structured metadata columns. ![Dex Extraction](https://mintcdn.com/scalegp/0fQRQwgv9afUT3rO/images/compass/structuredextraction.png?fit=max\&auto=format\&n=0fQRQwgv9afUT3rO\&q=85\&s=0a29dd96bd6c7883b5795333664114de) `Run extraction after parse`: Toggle on to enable structured extraction. This uses the Dex SDK on each parse result (per file). Note: Large documents may hit LLM context limits When enabled, configure the following: **Schema Definition** Define the fields you want to extract. Add all the fields you need: * `Column name`: Name for the extracted field (e.g., "latestPosition") * `Data type`: Select the data type (String, Number, Boolean, etc.) * `Description`: Describe what should be extracted (e.g., "What's the latest position they worked at") * Click `Add field` to define additional extraction fields Extraction Options * `Generate citations`: Toggle on to return bounding boxes for extracted values. Enabling this increases the size of each row payload * `Generate confidence`: Toggle on to return confidence scores for extracted values. Enabling this increases the size of each row payload Extraction Model Configuration * `Extraction model`: Specify the LLM to use for extraction (e.g., "openai/gpt-4o", "vertex_ai/gemini-2.5-pro"). This uses LiteLLM-style model IDs * `Extraction retries`: Number of retries for structured extraction (0-10). Default: 3 retries Provide custom instructions to guide the extraction process. For example: ``` From the provided text, extract the following: **Patient Info** - patient name in the document - what is the patient symptom? ``` > [!NOTE] > > Extracted field keys in the output use `extract` followed by the column name with its first letter capitalized (for example, `latestPosition` becomes `extractLatestPosition`) to distinguish them from standard parse output columns. #### Knowledge Base Optionally push parsed results into a vector store for semantic search and RAG: * `Push to Knowledge Base`: Choose `None`, `Create New`, `Use Existing (by ID)`, or `Upsert (by Column)` * `Vector Store Name` (`Create New`): Provide a name for your Knowledge Base (e.g., "Resume Vector Store"). The Dex project ID is appended to the final name * `Vector Store ID` (`Use Existing (by ID)`): The ID of an existing vector store. Requires `Use Existing (by ID)` for the project * `KB Name Column` (`Upsert (by Column)`): A column holding vector store names. The card creates one vector store per unique name and pushes later rows with the same name to it * `Embedding Model` (`Create New` and `Upsert (by Column)`): Specify the embedding model (default `openai/text-embedding-3-large`) When `Push to Knowledge Base` is not `None`, parsed documents will be automatically indexed in the SGP Knowledge Base, accessible via the Knowledge Base UI for semantic search and retrieval. ### Step 4: View parsed & extracted output After the Dex card processes your documents, the output has one row per input file and selected parse configuration, with the following columns: **Standard Parse Columns** * `parseResultId`: Unique identifier for the parsing result * `projectId`: The Dex project ID where results are stored * `engine`: The parsing engine used (`reducto` or `iris`) * `pagesProcessed`: Number of pages processed in the document * `chunkCount`: Number of chunks created from the document * `status`: `queued`, `downloading`, `parsing`, or `enriching` while processing; terminal values are `success` or `error` (on `error`, `errorMessage` holds the message) * `parsedContent`: Joined text from all parsed chunks when included in output * `vectorStoreId`: ID of the Knowledge Base vector store (if enabled) **Structured Extraction Columns** (if enabled) When structured extraction is enabled, additional columns are added with the `extract` prefix. Their values are stringified: * `extract`: The extracted value for each defined field (e.g., `extractLatestPosition`) * `extractCitations`: JSON-string map of citations keyed by extraction output names, such as `extractLatestPosition` (if citations enabled) * `extractConfidence`: JSON-string map of confidence values keyed by the same names (if confidence enabled) * `extractError`: The error message when extraction failed for the file. Parsing can still have `status=success`; check this column separately. Parse the metadata maps before accessing a field. In a downstream Python card, pass each output record to the helper. The `row` below shows the shape of one Dex output record: ```python theme={null} import json def parse_extraction_metadata(row: dict) -> tuple[dict, dict]: citations = json.loads(row.get("extractCitations") or "{}") confidence = json.loads(row.get("extractConfidence") or "{}") return citations, confidence row = { "extractLatestPosition": "Software Engineer", "extractCitations": '{"extractLatestPosition": null}', "extractConfidence": '{"extractLatestPosition": 0.93}', } citations, confidence = parse_extraction_metadata(row) print(citations.get("extractLatestPosition")) print(confidence.get("extractLatestPosition")) ``` You can use this dataframe in subsequent workflow cards to analyze, transform, or process the parsed data. ### Step 5: Access Dex project and SGP Knowledge Base The Dex card creates a Dex project (or reuses one, depending on the `Dex Project` option) that you can access later for review and management. If you set `Push to Knowledge Base` to an option other than `None`, navigate to the `Knowledge Base` section in SGP to access your vector store: * Search parsed documents semantically * Test retrieval quality * Use for RAG applications * Manage embeddings and vector store settings ### Step 6: Use parsed / extracted data in your workflow Now you can use the parsed content in subsequent workflow cards. For this example, we'll call an agent to categorize resume content into suitable job roles. #### Call Agent for Analysis Add a `Call Agent` card to process the parsed content: * `Select Agent`: Choose your agent from the dropdown * `Output Column Prefix`: Specify the prefix for the card's output columns (e.g., `job_categories`). The card writes the agent task ID and status columns, not the agent's reply; see [Evaluations in Workflows](/docs/capabilities/workflows/evals-in-workflows) for the full list * `Prompt Template`: Write your prompt using `{{parsedContent}}` to reference the parsed text ![Dex Call Agent ](https://mintcdn.com/scalegp/gkIBcTQjtllZVAnH/images/compass/dex-call-agent.png?fit=max\&auto=format\&n=gkIBcTQjtllZVAnH\&q=85\&s=7db09e865f022b0d99b7e7243a4c4cdd) #### Additional Processing You can add more cards to: * Filter or transform the categorized results * Join with job posting data * Run evaluations on categorization quality * Generate reports or summaries ### Step 7: Save your results Finally, save the workflow output for future reference and analysis. You can save the results with the `Export as Evaluation` card as: * **SGP Dataset**: Store the categorized resumes as a dataset for future use in training, evaluation, or analysis. * **SGP Evaluation**: Create an evaluation to track the quality of resume categorization. The next 2 options allow you to observe hillclimbing trends. * **SGP Dashboard**: Create a dashboard to visualize evaluation results, optionally from an existing template. * **SGP Evaluation Group**: Add the evaluation to an existing group for side-by-side comparison on the Evaluations tab. ![Dex Final Export ](https://mintcdn.com/scalegp/gkIBcTQjtllZVAnH/images/compass/dex-final-export.png?fit=max\&auto=format\&n=gkIBcTQjtllZVAnH\&q=85\&s=3b7a17470c90ad0fe23d095367e36124) # Guide to automating evals on Workflows Source: https://docs.gp.scale.com/docs/capabilities/workflows/evals-in-workflows Tutorial workflow for automated evals for your agent This guide walks you through building an automated evaluation workflow in Workflows. You'll learn how to query data, call your agent, join ground truth datasets, run LLM-as-Judge evaluations, and schedule everything to run automatically. ### Step 1: Connect to the data source Start by navigating to `Workflows` in SGP. Choose from the available data connection options: Traces, Data Sources, or CSV. We add more options based on user demand. ![Workflows Starter Page Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/compass-starter-page.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=fd2d11aaaed7f4c1fa09f023ec5b988c) For this example, we'll connect to a Snowflake data source. The data source has a list of supermarket items with product name and description that we must categorize using our agent. ![Workflows Snowflake Query Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/compass-snowflake-query.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=4ebae440c429da4dba570e222724ee0b) Select `Data Sources`, then choose your registered Snowflake connection from the dropdown. Write a SQL query to fetch the data you want to evaluate. You can set a `Row Limit` up to 1,000,000. > [!NOTE] > > Only read-only SQL queries are allowed. INSERT, UPDATE, DELETE, DROP, and other modifying statements are not permitted. ### Step 2: Call your agent for completions Use a `Call Agent` card to generate outputs from your agent for each row in your dataset. The `Agentex` agent you select has to be deployed on your SGP instance to be available in the dropdown. ![Workflows Call Agent Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/compass-call-agent.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=b6ad669adaa9fc350c619d1b40954c41) In this example, we ensure our agent categorizes the supermarket items appropriately using our prompt. The configurations are: * `Select Agent`: Choose the agent you want to evaluate from the dropdown * `Output Column Prefix`: Specify the prefix for the card's output columns (e.g., `agent_response`). The card doesn't write the agent's reply. It writes: * `agent_response_task_id`: the agent task ID * `agent_response_status`: why the card stopped waiting (`criteria_met`, `task_terminal`, `timeout`, or `poll_error`) * `agent_response_task_status`: the agent task's own status, such as `COMPLETED` or `FAILED` * `Include Raw Messages`: Toggle on to write all agent messages (unfiltered) to a separate `agent_response_messages` column * `Prompt Template`: Write your prompt using `{{variableName}}` syntax to reference columns from your data. Click the variable buttons to insert column references automatically. Under `Settings`, you can configure: * `Timeout Duration`: Maximum time in seconds to wait for agent responses (default: 300) * `Number of retries`: How many times to retry failed calls * `Response Caching`: Enable to cache responses and avoid redundant API calls Under `Advanced` settings, you can configure the `completion criteria` that determines when to stop polling for agent messages and the `batch size` that determines how many rows to process concurrently. This card generates the agent task IDs immediately as output. ![Get agent messages](https://mintcdn.com/scalegp/kpG6pvbOwFDG6SYY/images/compass/agentmessages.png?fit=max\&auto=format\&n=kpG6pvbOwFDG6SYY\&q=85\&s=f7b6ebad52ac18eb76d889e3f284f443) Once you have set up the `Call agent` card, you need to use the `Get agent messages` card right after to get the agent's responses into the dataframe. The `Get agent messages` card takes the `agent task IDs` created in the previous card as input, along with configuration options such as agent `message limits` and `matching criteria`, to receive exactly what you need from the agent response (1st message, last message, tool calls, etc.). The 2-part implementation provides flexibility where you can get the agent response in the 2nd card without having to wait for all the tasks in the 1st card to finish. ### Step 3: Join data from another workflow (ex: Ground Truth) Use the `Join Workflow` card to bring in data from another Workflows workflow—such as a ground truth dataset for comparison. Here we will compare our agent's categorization against the ground truth categorization for each product. ![Joins Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/compass-joins.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=ded7bec0af816c4d8d2a93c86036f45d) 1. `Select Workflow`: Choose the workflow containing your ground truth or reference data 2. `Join Type`: Select how you want to combine the datasets: * `Left Join`: Keep all rows from the current workflow * `Right Join`: Keep all rows from the target workflow * `Inner Join`: Keep only matching rows * `Outer Join`: Keep all rows from both workflows 3. `Join Keys`: Select the matching columns from each workflow (e.g., `trace_id`) 4. `Execute target workflow before joining`: Enable this to get fresh data from the target workflow; otherwise, the most recent execution result will be used ### Step 4: Use the LLM as Judge card to do evals Add an `LLM as Judge` card to evaluate your agent's outputs against the ground truth. ![Workflows LLM Judge Card](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/llmjudgecard.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=5fa6030f05d6cff321986fe350420a98) 1. `Select Judge`: Choose to create a new judge or use an existing one 2. `Judge Type`: Select from available options like * `Legacy LLM as Judge (recommended)`: Traditional LLM judge with custom configuration that covers most use cases * `Base Evals Agent as Judge`: General-purpose evaluation agent with flexible capabilities 3. `Model`: Choose the LLM model to use for judging (e.g., `openai/gpt-4o`) 4. `Temperature`: Set to 0 for deterministic outputs 5. `LLM Judge Name`: Give your judge a name for future usage 6. `Rubric`: Define evaluation criteria. Use `{{columnName}}` to reference data columns After configuring the judge, scroll down to the `Evaluation Details` section to export your results as an evaluation that can be accessed later anytime, which is helpful if you are running the evaluations on a schedule. ![Workflows LLM Judge to Evals](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/oneclickevals.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=1065c97a4e09cdb7a4eacd046c15f46c) * `Evaluation Name`: A descriptive name for your evaluation * `Description`: Explain what this evaluation measures * `Tags`: Add tags to filter and organize evaluations by groups later * `Save judge for future reuse`: Check to save this judge configuration and use it from the existing judge dropdown in other workflows * `Create new evaluation on re-run`: Check to create a new evaluation each time the workflow runs Every time you run this LLM as judge card, you'll see a success message with a direct link to view your evaluation results. Depending on what configurations you selected, a new evaluation will be created or the old one will be replaced. They can be accessed and referenced in the `Evaluations` and `Dashboards` tabs anytime respectively. ### Step 5: Open the evals in Evaluations tab or Dashboard tab Navigate to the `Evaluations` tab in SGP to view your evaluation results. You can search, filter by date range, and filter by tags to find specific evaluations. ![Workflows links to Evaluation](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/compass-evals-link.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=ed77608337992701c4e78602a2e0a8bb) Click on an evaluation to see: * `Overview`: Visual summary of evaluation metrics and response distributions * `Data`: Detailed row-by-row results with all columns * `Metadata`: Information about when and how the evaluation was created From here, you can also open the evaluation in the `Dashboards` tab for deeper analysis and custom visualizations. ### Step 6: Automate the evals to run on a schedule To automate your evaluation workflow, click the calendar icon beside the title to set a schedule. ![Workflows Set Schedule](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/llmjudge-schedule.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=b54ea94d88ba90f83dcd2e5e14e3ecd6) Configure the schedule: 1. `Set a Schedule`: Toggle on to enable scheduling 2. `Days`: Choose `Daily` to run every day, or select specific days of the week 3. `Time`: Set the frequency (e.g., `Hourly`) or a specific time 4. `End Date`: Schedules can run for up to 180 days into the future. New schedules end after 30 days by default, and an end date more than 30 days out triggers a cost warning. Every schedule needs an end date; the SGP schedule form doesn't offer indefinite schedules. Click `Save` to activate your scheduled workflow. You can monitor all scheduled runs from the `Execution logs` in the side menu, and view `Workflow history` to track changes across versions. To get alerts, open `Workflow Alerts` with the bell button in the workflow header. Alerts go to Slack, Microsoft Teams, or Email when the workflow succeeds, fails, or its schedule is about to expire. A channel is available only if it is configured on your deployment. ### Step 7: Create SGP Dashboards to show trends over time You may export a workflow's output as an evaluation anytime utilizing the `Export as Evaluation` card. You can run that export on a schedule. That workflow can automatically create a new `Evaluation` in the SGP `Evaluations` tab. You can click on the `View Evaluation` button at the bottom of the card to visit that tab anytime. ![Export Evaluation](https://mintcdn.com/scalegp/VCzihNterdsJT0MH/images/compass/exporteval.png?fit=max\&auto=format\&n=VCzihNterdsJT0MH\&q=85\&s=8914f1af5db07d22fa22ad911cf547be) As you run the evaluation workflow on a schedule, you might want to track the trend line of your metrics in a dashboard. To do that, you must add your `Evaluation` to an `Evaluation Group`. To create an `Evaluation Group` you need 1+ Evaluations. Go to the `Evaluations` tab to create an `Evaluation Group`. ![Evaluation Group](https://mintcdn.com/scalegp/VCzihNterdsJT0MH/images/compass/evalgroup.png?fit=max\&auto=format\&n=VCzihNterdsJT0MH\&q=85\&s=2031d34be06dbca995fe50aa99963826) Next, go to the SGP `Dashboards` tab and create a dashboard using the evaluation group you just created. For your use case, you will want to create a `Timeseries Widget` with the metrics across your evaluations inside the same group. [Learn more about Evaluation Group Dashboards →](/docs/v5/evaluation-dashboards/eval-group-dashboards#what-are-evaluation-group-dashboards) ![Evaluation Group Dashboard](https://mintcdn.com/scalegp/VCzihNterdsJT0MH/images/compass/evalgroupdashboard.png?fit=max\&auto=format\&n=VCzihNterdsJT0MH\&q=85\&s=ed3d3443c2c25233b352dcc87829e20c) After you have set the evaluation group dashboard and the widgets you want, you can update your workflow `Export as Evaluation` card to write every new evaluation run to the `Evaluation Group` you created for this dashboard. Doing so will automatically update the associated dashboard every time you run this workflow. You can visit the updated evaluation group dashboard by clicking on the `Visit Evaluation Group` button at the bottom of the card and then clicking on the `Open in Dashboards` button at the top right in the new window. Congrats! Now, you have an automated evaluation pipeline that updates metrics and visualizations over time. ![Evaluation Group Update](https://mintcdn.com/scalegp/VCzihNterdsJT0MH/images/compass/evalgroupupdate.png?fit=max\&auto=format\&n=VCzihNterdsJT0MH\&q=85\&s=edb8b72d3bc62bd3c7d1226c9fe7420e) In the future, if you want to create a new dashboard in the same style for another evaluation workflow, when you come to this `Export as Evaluation` step, enable the `Create Dashboard` toggle, and pick your old dashboard as the template such that all the necessary widgets get carried over. You can also create a new empty dashboard with this card directly without any templates. # How to use Workflows Source: https://docs.gp.scale.com/docs/capabilities/workflows/introduction-to-workflows Overview of Durable Agentic Workflow Orchestrator on SGP Workflows is a no/low code durable workflow orchestration platform. It allows technical and non-technical teams to build, automate, and evaluate AI-driven data pipelines by chaining together data sources, transformations, custom Python scripts, and agent responses into a unified production workflow. ![Workflows Overview Pn](https://mintcdn.com/scalegp/ioUVe8ByVLFhFpgF/images/compass/compass-workflow-overview.png?fit=max\&auto=format\&n=ioUVe8ByVLFhFpgF\&q=85\&s=5ec786758cf47664f726f69cc435d669) ### Getting started You can access it on SGP (Scale GenAI Platform) through the **Workflows** tab. If you don't see the tab on your SGP, please contact us. To start a workflow you must either import data or connect to a data source. Data sources available vary based on the connections set in the SGP instance you are running Workflows on. You can set up new `Data Sources` under the `Assets` tab. ![Workflows Starter Page Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/compass-starter-page.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=fd2d11aaaed7f4c1fa09f023ec5b988c) To start from an existing workflow, open it and use the `Clone workflow` button in the workflow header. ### Workflows cards The basic building block in Workflows is called a **card**. Cards can produce data, transform data, or perform actions on data. When you start a new Workflows session, you can add cards to build your workflow. Each card has its own interface appropriate for its function. Think of it like a form that you have to fill out to tell the card what you want it to do. ![Basicspython Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/basicspython.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=47415f65442a0467fe7b25524edcee21) For example, the Python card appropriately has a code editor for you to write the code to transform the data you have imported in a previous step. The card includes all the necessary libraries to transform data. ![Plottingpython Pn](https://mintcdn.com/scalegp/tj6Jl-ypYZg4E3Hm/images/compass/plotpython.png?fit=max\&auto=format\&n=tj6Jl-ypYZg4E3Hm\&q=85\&s=fd7e6b06e8afacf3509d70104e4370d5) Additionally, the Python card also supports `print` and `matplotlib.pyplot` statements for quick EDA or debugging. If you include those statements, the output will show up in the `Logs` tab and `Visualization` tab respectively, beside the usual `Data` tab. Cards stack upon each other, allowing you to perform rich transformations or analyses over the previous card’s data. As an example, we can take a CSV that has different LLM completions against a prompt and have an LLM judge it against ground truth for evals. ![Basicsllm Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/basicsllm.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=53ddaed19f5278548a7aa84790629b91) Workflows sessions are long-lived, so you can run cards that take extended periods of time (for example, model evaluations using reasoning LLMs on thousands of rows) without losing your work. You can save cards while iterating, then run them later when you're ready. Each session has a `unique URL identifier` that gives you persistent access to your session. ![Linkshare Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/linkshare.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=4d5595892c101f7faabdbe6d1d2a0fd7) Finally, once you are done with a workflow, you can export the final output dataframe as an `Evaluation` that you can always access and monitor later. This is helpful in the cases you are running the evaluation on your agent traces periodically. You can also save it as an SGP `Dataset` that can be downloaded and shared as a CSV. ![Export Evals Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/compass-export-evals.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=549a06e0bc3ae87d8fbdadf04f2c88a0) In the event that you need data from another workflow in the current one, you can use the `Join Workflow` card to bring the output of another workflow (dataframe) and join it with the output here. Attached image is from an example workflow where we had to bring the mock PII from another workflow to perform an evaluation on whether our agent was correctly implementing PII anonymization. ![Joins Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/compass-joins.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=ded7bec0af816c4d8d2a93c86036f45d) ### Key features & associated cards ![Cards More Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/compass-more-cards.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=5ac1a4c14c51abec236d484f9d52210a) Workflows offers a variety of cards for pulling, transforming, and performing actions on data. Additional cards will be added based on enterprise use case requirements. Some of the ways cards can be used are: * **Importing data**: Read data from sources like agent `Traces` tab, a `CSV` upload, or queryable sources like `Postgres`, `Snowflake`, `Mongo`, or `MS SQL` directly. * **Document understanding**: Connect to cloud storage (Amazon S3, Azure Blob, Google Cloud Storage), parse unstructured documents using `Dex` card, automatically create Dex projects, extract structured data, and push parsed data to Knowledge Base. See the [Guide to Document Parsing with Dex in Workflows](/docs/capabilities/workflows/dex-parsing-in-workflows) for details. * **Transforming data**: Transform your data using custom `Python` code or simple cards like `select columns`, `filter` or `aggregate`. You can also use `Join Workflow` to join the output from another workflow into this workflow's output (dataframe). * **Call LLM or Agent**: Generate output and traces against input from each row of your dataset via cards like `Call LLM` or `Call Agent`. Workflows allows you to call your `Agentex` agents on SGP. * **Analyze data**: Evaluate your outputs using `LLM as Judge` card and access the evaluation anytime in the SGP `Evaluations` tab. * **Visualize data**: Generate basic visualizations from your transformed dataset using the `Python` or `Chart` card. For advanced capabilities, export the data as an `Evaluation` and `Dashboard`, and open in the `Dashboards` tab. * **Export the data**: Export the data as `CSV` or as a `Dataset` on SGP. This dataset can be referred to in the Evaluations tab anytime. * **Automate & monitor**: `Schedule` the workflow to run hourly or daily. Monitor runs and get notified through `Workflow Alerts` via Slack, Microsoft Teams, or Email. Each channel must be configured on the deployment. Most cards are available in **SGP > Workflows > New Workflow**. Workflows instances deployed in customer VPCs may have a limited set of available cards based on customer requirements. You can chat with the Workflows team to activate or disable certain cards depending on your customer's needs. ### Schedule & monitor workflows If you have a sequence of cards that you want to run repeatedly, you can save them to easily load and re-run later. You can `Share` the workflow via URL. You can add other editors for collaboration. By default, everyone who has access to that SGP instance account `can read` all the workflows, and only the creator of the workflow has editor access. ![Saveworkflow Pn](https://mintcdn.com/scalegp/tj6Jl-ypYZg4E3Hm/images/compass/saveworkflow.png?fit=max\&auto=format\&n=tj6Jl-ypYZg4E3Hm\&q=85\&s=95bdda23ddd558362ac52c4675f0de8d) A powerful feature of Workflows is the ability to execute them on a schedule. You can click on the calendar icon beside the title and toggle the **Set a Schedule** option and configure your desired schedule. Scheduled workflows allow you to automate repetitive tasks. ![Scheduleworkflow Pn](https://mintcdn.com/scalegp/tj6Jl-ypYZg4E3Hm/images/compass/scheduleworkflow.png?fit=max\&auto=format\&n=tj6Jl-ypYZg4E3Hm\&q=85\&s=61db5e3830dbb7a521ba06777150181d) From the side menu, you can view `Execution logs` anytime to monitor past runs. You can also check `Workflow history` to understand how the workflow changed across versions, the `diffs`, and the author responsible for each version. ![Exec Logs Pn](https://mintcdn.com/scalegp/tAM0OCM-XdRmgv0q/images/compass/compass-exec-logs.png?fit=max\&auto=format\&n=tAM0OCM-XdRmgv0q\&q=85\&s=2326dd380c73f9ae246ba031f5b134cb) ### What's coming next in H1 2026 Here are core features planned on our roadmap. * **\[Launched] More data source support**: Currently, Workflows supports connection to queryable data sources. We just launched data connectors for cloud file storages like Amazon S3, Azure Blob, and Google Cloud Storage. * **\[Launched] Document parsing & extraction on Workflows**: Utilize our `Dex` and `Iris` capabilities inside Workflows to `parse` unstructured data across Docs or PDFs into structured data and `extract` the information you need for your workflows. * **UX & UI improvements**: Make quality of life improvements to solve common pain points faced by users like improved error handling, messages for required missing inputs, improved navigation bars, etc. * **More integrations with Agentex and AgentOps API and SDK**: Currently, Workflows has integrations with our Evals SDK and the Dex SDK as separate cards. We want to extend the functionalities to cover the majority of SGP's SDKs. > [!NOTE] > > Creating a custom Workflows card to support your specific enterprise use case on their VPC is also straightforward. If your organization has functionality requirements, please contact us. # How to run Workflows using a script Source: https://docs.gp.scale.com/docs/capabilities/workflows/run-workflows-using-a-script Run a saved workflow in your code using the Workflows HTTP API This tutorial shows how to run a [Workflows](/docs/capabilities/workflows/introduction-to-workflows) workflow programmatically over HTTPS. You need an API key, your account ID, and a workflow that is already defined and saved in the Workflows UI. You do not need to manage the underlying infrastructure. ## Prerequisites * A workflow defined and saved in the Workflows UI * A Scale API key * Your Scale account ID (the account that owns the workflow) If you are missing credentials, contact your Scale representative. ## Authentication Send requests over HTTPS to your SGP host—the same origin you use to open the Workflows UI. Every example below uses a base URL of the form `https:///corp-api/compass/api`. Include these headers on each request: ```http theme={null} x-scale-api-key: x-selected-account-id: ``` `` is your Scale API key. `` is the ID of the account that owns the workflow you want to run. ## Run and wait The usual pattern is to start a run, then block until it finishes (up to a configurable timeout). By default, the SGP route to Workflows closes requests after 120 seconds, so use this pattern for runs that finish within two minutes. For longer runs, see [Polling instead of blocking](#polling-instead-of-blocking). ```typescript theme={null} const BASE_URL = 'https:///corp-api/compass/api'; const HEADERS = { 'x-scale-api-key': process.env.SCALE_API_KEY!, 'x-selected-account-id': process.env.SCALE_ACCOUNT_ID!, 'Content-Type': 'application/json', }; async function runWorkflow(workflowId: string, variables?: Record) { const runRes = await fetch(`${BASE_URL}/v1/workflow/run`, { method: 'POST', headers: HEADERS, body: JSON.stringify({ workflowId, variables }), }); if (!runRes.ok) throw new Error(`Failed to start workflow: ${await runRes.text()}`); const { sessionId } = await runRes.json(); const resultRes = await fetch( `${BASE_URL}/v1/sessions/${sessionId}/result?timeout=110000`, { headers: HEADERS }, ); if (!resultRes.ok) throw new Error(`Failed to get result: ${await resultRes.text()}`); return resultRes.json(); } async function main() { const result = await runWorkflow('cmp_abc123', { region: 'US', date: '2024-01-01' }); console.log(`${result.data.length} rows returned`); console.log(result.data); } main().catch(console.error); ``` ### curl equivalent ```bash theme={null} BASE_URL="https:///corp-api/compass/api" SESSION_ID=$(curl -s -X POST "$BASE_URL/v1/workflow/run" \ -H "x-scale-api-key: $SCALE_API_KEY" \ -H "x-selected-account-id: $SCALE_ACCOUNT_ID" \ -H "Content-Type: application/json" \ -d '{"workflowId": "cmp_abc123", "variables": {"region": "US"}}' \ | jq -r '.sessionId') curl -s "$BASE_URL/v1/sessions/$SESSION_ID/result?timeout=110000" \ -H "x-scale-api-key: $SCALE_API_KEY" \ -H "x-selected-account-id: $SCALE_ACCOUNT_ID" ``` ## API reference All paths below are relative to `BASE_URL` (`https:///corp-api/compass/api`). ### `POST /v1/workflow/run` Starts a workflow run and returns immediately with a `sessionId`. **Request body:** ```json theme={null} { "workflowId": "cmp_abc123", "variables": { "region": "US", "threshold": "0.8" } } ``` `variables` is optional. Declare each key in the saved workflow first. Incoming values override those declared defaults and populate `${variable_name}` placeholders in card configuration; unknown keys are ignored. For typical use cases, use flat string values. Double-brace templates such as `{{column_name}}` refer to dataframe columns in card prompts. **Response:** ```json theme={null} { "sessionId": "compass-sess-..." } ``` ### `GET /v1/sessions/:sessionId` Returns the current status of a running session. Use this endpoint when you want to poll instead of blocking on the result endpoint. **Example response:** ```json theme={null} { "sessionId": "compass-sess-...", "status": "completed", "cards": [ { "index": 0, "cardType": "import_csv", "status": "completed" } ] } ``` Each card also carries `startTime` and `endTime` (epoch milliseconds) once they are recorded, and `errorMessage` when its status is `errored`. When `status` is `failed`, the response includes `errorReason`: `cancelled`, `timed_out`, `terminated`, `card_error`, or `workflow_error`. **Session status values** | Status | Meaning | | - | - | | `running` | The run is in progress | | `completed` | The run finished successfully; results are ready | | `failed` | The run failed, was cancelled, was terminated, or timed out | Stop polling on `completed` or `failed`. **Card status values** | Status | Meaning | | - | - | | `idle` | Card has not been scheduled yet | | `pending` | Queued to run | | `processing` | Currently executing | | `completed` | Finished successfully | | `errored` | Failed; check `errorMessage` | | `skipped` | Did not run because a conditional action's condition was false | ### `GET /v1/sessions/:sessionId/result?timeout=` Waits until the run finishes, then returns the final output. If the run doesn't finish within `timeout` milliseconds (default `600000`, ten minutes), the request fails with `408`. By default, the SGP route to Workflows closes requests after 120 seconds, so a request that waits longer fails with `504` before `timeout` elapses. For runs over two minutes, poll `GET /v1/sessions/:sessionId` instead. **Query parameters** * `timeout` — milliseconds to wait (default `600000`) **Example response:** ```json theme={null} { "data": [ { "name": "Alice", "score": 0.92 }, { "name": "Bob", "score": 0.85 } ], "schema": [ { "id": "name", "header": "Name", "type": "text" }, { "id": "score", "header": "Score", "type": "number" } ], "metadata": { "totalRows": 2, "executionDurationMillis": 3210 } } ``` `data` contains the rows. `schema` describes the columns. `metadata` carries the total row count and the run's duration when they are known. ## Polling instead of blocking If you prefer short requests instead of a long-lived connection, poll `GET /v1/sessions/:sessionId` until its top-level status is `completed` or `failed`. After `completed`, call `GET /v1/sessions/:sessionId/result` to fetch the tabular output. After `failed`, inspect the session's `errorReason` and the errored cards' `errorMessage` fields. The following example reuses the same `BASE_URL` and `HEADERS` as in [Run and wait](#run-and-wait). ```typescript theme={null} async function waitForResult(sessionId: string, pollIntervalMs = 5000, timeoutMs = 600000) { const deadline = Date.now() + timeoutMs; while (Date.now() < deadline) { const res = await fetch(`${BASE_URL}/v1/sessions/${sessionId}`, { headers: HEADERS }); if (!res.ok) throw new Error(`[${res.status}] Failed to poll session: ${await res.text()}`); const session = await res.json(); if (session.status === 'failed') { const errors = session.cards .filter((card: { status: string }) => card.status === 'errored') .map((card: { errorMessage?: string }) => card.errorMessage); throw new Error(`Workflow failed: ${errors.join(', ') || session.errorReason || 'unknown reason'}`); } if (session.status === 'completed') { return session; } await new Promise((resolve) => setTimeout(resolve, pollIntervalMs)); } throw new Error('Timed out waiting for session to complete'); } ``` > [!NOTE] > > Polling returns session state (status and card metadata), not the full workflow table. After the session reaches `completed`, call `GET /v1/sessions/:sessionId/result` to retrieve the rows. ## Workflow variables Variables let you parameterize a workflow without editing its definition. Declare variables and their defaults in the saved workflow, then reference them as `${variable_name}` in card configuration (for example, filter values, prompt templates, or model parameters). Use `{{column_name}}` separately for dataframe columns in card prompts. ```typescript theme={null} await runWorkflow('cmp_abc123', { model: 'gpt-4o', prompt: 'Summarize the following in one sentence', min_score: '0.7', }); ``` Values you pass override defaults saved on the workflow. Any variable you omit uses the workflow’s saved default. Variables that are not saved on the workflow are dropped. ## Error handling Error bodies are JSON. Many errors look like this: ```json theme={null} { "error": "Workflow not found or you do not have access.", "status_code": 404 } ``` Authentication failures may include both `error` and `message`: ```json theme={null} { "error": "Unauthorized", "message": "x-scale-api-key header is required." } ``` | HTTP status | Meaning | | - | - | | 400 | Malformed request (missing required field, bad format), or missing `x-selected-account-id` header | | 401 | Missing or invalid API key, or the session belongs to another user | | 403 | The API key has no access to the account in `x-selected-account-id` | | 404 | Workflow or session not found, or you do not have access | | 408 | The run didn't finish within the result endpoint's `timeout` | | 500 | Internal server error | | 504 | The request outlived the 120-second route limit; poll the session instead | This variant parses JSON error bodies from failed responses: ```typescript theme={null} async function runWorkflowSafe(workflowId: string, variables?: Record) { const runRes = await fetch(`${BASE_URL}/v1/workflow/run`, { method: 'POST', headers: HEADERS, body: JSON.stringify({ workflowId, variables }), }); if (!runRes.ok) { const err = await runRes.json().catch(() => ({ error: runRes.statusText })); throw new Error(`[${runRes.status}] ${err.error ?? err.message}`); } const { sessionId } = await runRes.json(); const resultRes = await fetch( `${BASE_URL}/v1/sessions/${sessionId}/result?timeout=110000`, { headers: HEADERS }, ); if (!resultRes.ok) { const err = await resultRes.json().catch(() => ({ error: resultRes.statusText })); throw new Error(`[${resultRes.status}] ${err.error ?? err.message}`); } return resultRes.json(); } ``` `runWorkflowSafe` assumes `BASE_URL` and `HEADERS` are defined the same way as in [Run and wait](#run-and-wait). # Create Api Key Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/create-api-key https://agentex.sgp.scale.com/openapi.json post /agent_api_keys # Delete API key by ID Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/delete-api-key-by-id https://agentex.sgp.scale.com/openapi.json delete /agent_api_keys/{id} Delete API key by ID. # Delete API key by name Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/delete-api-key-by-name https://agentex.sgp.scale.com/openapi.json delete /agent_api_keys/name/{api_key_name} Delete API key by name. # List API keys for an agent ID Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/list-api-keys-for-an-agent-id https://agentex.sgp.scale.com/openapi.json get /agent_api_keys List API keys for an agent ID. # Return named API key for the agent ID Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/return-named-api-key-for-the-agent-id https://agentex.sgp.scale.com/openapi.json get /agent_api_keys/name/{name} Return named API key for the agent ID. # Return the API key by ID Source: https://docs.gp.scale.com/reference/agentex/agent-apikeys/return-the-api-key-by-id https://agentex.sgp.scale.com/openapi.json get /agent_api_keys/{id} Return API key by ID. # Get Agent Task Tracker Source: https://docs.gp.scale.com/reference/agentex/agent-task-tracker/get-agent-task-tracker https://agentex.sgp.scale.com/openapi.json get /tracker/{tracker_id} Get agent task tracker by tracker ID # List Agent Task Trackers Source: https://docs.gp.scale.com/reference/agentex/agent-task-tracker/list-agent-task-trackers https://agentex.sgp.scale.com/openapi.json get /tracker List all agent task trackers, optionally filtered by query parameters. # Update Agent Task Tracker Source: https://docs.gp.scale.com/reference/agentex/agent-task-tracker/update-agent-task-tracker https://agentex.sgp.scale.com/openapi.json put /tracker/{tracker_id} Update agent task tracker by tracker ID # Delete Agent by ID Source: https://docs.gp.scale.com/reference/agentex/agents/delete-agent-by-id https://agentex.sgp.scale.com/openapi.json delete /agents/{agent_id} Delete an agent by its unique ID. # Delete Agent by Name Source: https://docs.gp.scale.com/reference/agentex/agents/delete-agent-by-name https://agentex.sgp.scale.com/openapi.json delete /agents/name/{agent_name} Delete an agent by its unique name. # Forward request to agent by ID Source: https://docs.gp.scale.com/reference/agentex/agents/forward-request-to-agent-by-id https://agentex.sgp.scale.com/openapi.json get /agents/forward/name/{agent_name}/{path} Forward a request to an agent by its unique ID. # Forward request to agent by ID Source: https://docs.gp.scale.com/reference/agentex/agents/forward-request-to-agent-by-id-1 https://agentex.sgp.scale.com/openapi.json post /agents/forward/name/{agent_name}/{path} Forward a request to an agent by its unique ID. # Get Agent by ID Source: https://docs.gp.scale.com/reference/agentex/agents/get-agent-by-id https://agentex.sgp.scale.com/openapi.json get /agents/{agent_id} Get an agent by its unique ID. # Get Agent by Name Source: https://docs.gp.scale.com/reference/agentex/agents/get-agent-by-name https://agentex.sgp.scale.com/openapi.json get /agents/name/{agent_name} Get an agent by its unique name. # Handle Agent RPC by ID Source: https://docs.gp.scale.com/reference/agentex/agents/handle-agent-rpc-by-id https://agentex.sgp.scale.com/openapi.json post /agents/{agent_id}/rpc Handle JSON-RPC requests for an agent by its unique ID. # Handle Agent RPC by Name Source: https://docs.gp.scale.com/reference/agentex/agents/handle-agent-rpc-by-name https://agentex.sgp.scale.com/openapi.json post /agents/name/{agent_name}/rpc Handle JSON-RPC requests for an agent by its unique name. # List Agents Source: https://docs.gp.scale.com/reference/agentex/agents/list-agents https://agentex.sgp.scale.com/openapi.json get /agents List all registered agents, optionally filtered by query parameters. # Register Agent Source: https://docs.gp.scale.com/reference/agentex/agents/register-agent https://agentex.sgp.scale.com/openapi.json post /agents/register Register a new agent or update an existing one. # Delete Thread Source: https://docs.gp.scale.com/reference/agentex/checkpoints/delete-thread https://agentex.sgp.scale.com/openapi.json post /checkpoints/delete-thread # Get Checkpoint Tuple Source: https://docs.gp.scale.com/reference/agentex/checkpoints/get-checkpoint-tuple https://agentex.sgp.scale.com/openapi.json post /checkpoints/get-tuple # List Checkpoints Source: https://docs.gp.scale.com/reference/agentex/checkpoints/list-checkpoints https://agentex.sgp.scale.com/openapi.json post /checkpoints/list # Put Checkpoint Source: https://docs.gp.scale.com/reference/agentex/checkpoints/put-checkpoint https://agentex.sgp.scale.com/openapi.json post /checkpoints/put # Put Writes Source: https://docs.gp.scale.com/reference/agentex/checkpoints/put-writes https://agentex.sgp.scale.com/openapi.json post /checkpoints/put-writes # Get Deployment by ID Source: https://docs.gp.scale.com/reference/agentex/deployment-history/get-deployment-by-id https://agentex.sgp.scale.com/openapi.json get /deployment-history/{deployment_id} Get a deployment record by its unique ID. # List Deployments for an agent Source: https://docs.gp.scale.com/reference/agentex/deployment-history/list-deployments-for-an-agent https://agentex.sgp.scale.com/openapi.json get /deployment-history List deployment history for an agent. # Get Event Source: https://docs.gp.scale.com/reference/agentex/events/get-event https://agentex.sgp.scale.com/openapi.json get /events/{event_id} # List Events Source: https://docs.gp.scale.com/reference/agentex/events/list-events https://agentex.sgp.scale.com/openapi.json get /events List events for a specific task and agent. Optionally filter for events after a specific sequence ID. Results are ordered by sequence_id. # Batch Create Messages Source: https://docs.gp.scale.com/reference/agentex/messages/batch-create-messages https://agentex.sgp.scale.com/openapi.json post /messages/batch # Batch Update Messages Source: https://docs.gp.scale.com/reference/agentex/messages/batch-update-messages https://agentex.sgp.scale.com/openapi.json put /messages/batch # Create Message Source: https://docs.gp.scale.com/reference/agentex/messages/create-message https://agentex.sgp.scale.com/openapi.json post /messages # Get Message Source: https://docs.gp.scale.com/reference/agentex/messages/get-message https://agentex.sgp.scale.com/openapi.json get /messages/{message_id} # List Messages Source: https://docs.gp.scale.com/reference/agentex/messages/list-messages https://agentex.sgp.scale.com/openapi.json get /messages List messages for a task with offset-based pagination. For cursor-based pagination with infinite scroll support, use /messages/paginated. # List Messages Paginated Source: https://docs.gp.scale.com/reference/agentex/messages/list-messages-paginated https://agentex.sgp.scale.com/openapi.json get /messages/paginated List messages for a task with cursor-based pagination. This endpoint is designed for infinite scroll UIs where new messages may arrive while paginating through older ones. Args: task_id: The task ID to filter messages by limit: Maximum number of messages to return (default: 50) cursor: Opaque cursor string for pagination. Pass the `next_cursor` from a previous response to get the next page. direction: Pagination direction - "older" to get older messages (default), "newer" to get newer messages. Returns: PaginatedMessagesResponse with: - data: List of messages (newest first when direction="older") - next_cursor: Cursor for fetching the next page (null if no more pages) - has_more: Whether there are more messages to fetch Example: First request: GET /messages/paginated?task_id=xxx&limit=50 Next page: GET /messages/paginated?task_id=xxx&limit=50&cursor= # Update Message Source: https://docs.gp.scale.com/reference/agentex/messages/update-message https://agentex.sgp.scale.com/openapi.json put /messages/{message_id} # Create Schedule Source: https://docs.gp.scale.com/reference/agentex/schedules/create-schedule https://agentex.sgp.scale.com/openapi.json post /agents/{agent_id}/schedules Create a new schedule for recurring workflow execution for an agent. # Delete Schedule Source: https://docs.gp.scale.com/reference/agentex/schedules/delete-schedule https://agentex.sgp.scale.com/openapi.json delete /agents/{agent_id}/schedules/{schedule_name} Delete a schedule permanently. # Get Schedule Source: https://docs.gp.scale.com/reference/agentex/schedules/get-schedule https://agentex.sgp.scale.com/openapi.json get /agents/{agent_id}/schedules/{schedule_name} Get details of a schedule by its name. # List Agent Schedules Source: https://docs.gp.scale.com/reference/agentex/schedules/list-agent-schedules https://agentex.sgp.scale.com/openapi.json get /agents/{agent_id}/schedules List all schedules for an agent. # Pause Schedule Source: https://docs.gp.scale.com/reference/agentex/schedules/pause-schedule https://agentex.sgp.scale.com/openapi.json post /agents/{agent_id}/schedules/{schedule_name}/pause Pause a schedule to stop it from executing. # Trigger Schedule Source: https://docs.gp.scale.com/reference/agentex/schedules/trigger-schedule https://agentex.sgp.scale.com/openapi.json post /agents/{agent_id}/schedules/{schedule_name}/trigger Trigger a schedule to run immediately, regardless of its regular schedule. # Unpause Schedule Source: https://docs.gp.scale.com/reference/agentex/schedules/unpause-schedule https://agentex.sgp.scale.com/openapi.json post /agents/{agent_id}/schedules/{schedule_name}/unpause Unpause/resume a schedule to allow it to execute again. # Create Span Source: https://docs.gp.scale.com/reference/agentex/spans/create-span https://agentex.sgp.scale.com/openapi.json post /spans Create a new span with the provided parameters # Get Span Source: https://docs.gp.scale.com/reference/agentex/spans/get-span https://agentex.sgp.scale.com/openapi.json get /spans/{span_id} Get a span by ID # List Spans Source: https://docs.gp.scale.com/reference/agentex/spans/list-spans https://agentex.sgp.scale.com/openapi.json get /spans List all spans for a given trace ID # Partial Update Span Source: https://docs.gp.scale.com/reference/agentex/spans/partial-update-span https://agentex.sgp.scale.com/openapi.json patch /spans/{span_id} Update a span with the provided output data and mark it as complete # Create Task State Source: https://docs.gp.scale.com/reference/agentex/states/create-task-state https://agentex.sgp.scale.com/openapi.json post /states # Delete Task State Source: https://docs.gp.scale.com/reference/agentex/states/delete-task-state https://agentex.sgp.scale.com/openapi.json delete /states/{state_id} # Get State by State ID Source: https://docs.gp.scale.com/reference/agentex/states/get-state-by-state-id https://agentex.sgp.scale.com/openapi.json get /states/{state_id} Get a state by its unique state ID. # List States Source: https://docs.gp.scale.com/reference/agentex/states/list-states https://agentex.sgp.scale.com/openapi.json get /states List all states, optionally filtered by query parameters. # Update Task State Source: https://docs.gp.scale.com/reference/agentex/states/update-task-state https://agentex.sgp.scale.com/openapi.json put /states/{state_id} # Delete Task by ID Source: https://docs.gp.scale.com/reference/agentex/tasks/delete-task-by-id https://agentex.sgp.scale.com/openapi.json delete /tasks/{task_id} Delete a task by its unique ID. # Delete Task by Name Source: https://docs.gp.scale.com/reference/agentex/tasks/delete-task-by-name https://agentex.sgp.scale.com/openapi.json delete /tasks/name/{task_name} Delete a task by its unique name. # Get Task by ID Source: https://docs.gp.scale.com/reference/agentex/tasks/get-task-by-id https://agentex.sgp.scale.com/openapi.json get /tasks/{task_id} Get a task by its unique ID. # Get Task by Name Source: https://docs.gp.scale.com/reference/agentex/tasks/get-task-by-name https://agentex.sgp.scale.com/openapi.json get /tasks/name/{task_name} Get a task by its unique name. # List Tasks Source: https://docs.gp.scale.com/reference/agentex/tasks/list-tasks https://agentex.sgp.scale.com/openapi.json get /tasks List all tasks. # Stream Task Events by ID Source: https://docs.gp.scale.com/reference/agentex/tasks/stream-task-events-by-id https://agentex.sgp.scale.com/openapi.json get /tasks/{task_id}/stream Stream events for a task by its unique ID. # Stream Task Events by Name Source: https://docs.gp.scale.com/reference/agentex/tasks/stream-task-events-by-name https://agentex.sgp.scale.com/openapi.json get /tasks/name/{task_name}/stream Stream events for a task by its unique name. # Update Task by ID Source: https://docs.gp.scale.com/reference/agentex/tasks/update-task-by-id https://agentex.sgp.scale.com/openapi.json put /tasks/{task_id} Update mutable fields for a task by its unique ID. # Update Task by Name Source: https://docs.gp.scale.com/reference/agentex/tasks/update-task-by-name https://agentex.sgp.scale.com/openapi.json put /tasks/name/{task_name} Update mutable fields for a task by its unique Name. # Delete Agent Source: https://docs.gp.scale.com/reference/v5/agentex-agent/delete-agent /openapi/v5.yaml delete /v5/agentex/agents/{agent_id} Delete an agent and reclaim its deployments and builds. The agent record is deleted synchronously; its deployments (previews included) and builds are then torn down asynchronously by a durable Temporal workflow. Returns the deleted agent. Single-deployment removal already exists at `DELETE /v5/agentex/deployments/{deployment_id}`; this is the agent-level teardown. # Cancel a build Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/cancel-a-build /openapi/v5.yaml post /v5/builds/{build_id}/cancel Request cancellation of a build and return 202 Accepted. Cancellation runs asynchronously: the endpoint verifies the build exists and then hands it off to a background workflow that marks the build as cancelling, asks the cloud build provider to stop it, and polls until the build reaches a terminal state before recording the final status. Only builds that are still queued or running are actually cancelled; if the build has already finished, or cancellation is already in progress, the request is still accepted but has no effect. The returned record reflects the build's state at the time of the call, so it will not yet show the pending cancellation. The request is rejected if no such build exists for the caller's account. # Get a build Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/get-a-build /openapi/v5.yaml get /v5/builds/{build_id} Retrieve a single build by its ID, including its current lifecycle status. Because builds run asynchronously after submission, the status returned here reflects the latest known state and may still be queued or running; call this endpoint again to observe progression to a terminal state such as success, failed, cancelled, or timed out. Responds with a not-found error if no such build exists for the caller's account. # List agents built but not deployed Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/list-agents-built-but-not-deployed /openapi/v5.yaml get /v5/builds/undeployed List agents that exist only as cloud builds and have no healthy deployment yet. Unlike listing builds, this aggregates by agent: it returns one entry per distinct agent name whose builds have never reached a healthy deploy, each carrying that agent's most recent build and its total build count. The result is a plain list (not paginated) and is limited to builds the caller is permitted to read. Use this to surface agents that were built but still need to be deployed; use `GET /v5/builds` when you need the individual build records instead. # Stream build logs Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-build/stream-build-logs /openapi/v5.yaml get /v5/builds/{build_id}/logs Stream a build's logs as Server-Sent Events. The response has content-type `text/event-stream`; each event carries one log line as an `AgentexCloudBuildLogLine` object, delivered as lines become available from the cloud build provider. The stream ends when the provider stops producing output. Responds with a not-found error if no such build exists for the caller's account. # Delete and tear down a deployment Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-deploy/delete-and-tear-down-a-deployment /openapi/v5.yaml delete /v5/agentex/deployments/{deployment_id} Delete an agent deployment and tear down its Kubernetes resources. Deletes the deployment's Kubernetes resources first, then marks the record as `DELETED`; the underlying Helm release is subsequently uninstalled asynchronously by FluxCD once the resource is removed. If the Kubernetes teardown fails, the record is left unchanged and the call errors, so the delete can be safely retried. Rejects the call with a client error if the deployment is already in a terminal state (`DELETED`, `CANCELLED`, or `SUPERSEDED`) — a `SUPERSEDED` record shares its Helm release with the deployment that replaced it, so deleting it would tear down the live release. Returns 404 if no deployment with this ID exists for the caller's account. This removes a running deployment, not the image build behind it, which is managed separately through the agentex builds API. Returns 409 while the agent's key migration is active. # Deploy a built agent image Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-deploy/deploy-a-built-agent-image /openapi/v5.yaml post /v5/agentex/deployments Deploy a successfully built agent image to Kubernetes as a Helm release. Takes a completed build (referenced by `build_id`, or by `image_name` + `image_tag`) together with the agent manifest and environment config, then starts an asynchronous Temporal workflow that provisions the agent as a Helm release. The call returns immediately with the deployment record in `PENDING` status — it does not wait for the release to become healthy; poll `GET /v5/agentex/deployments/{deployment_id}` for status and Kubernetes events and `GET /v5/agentex/deployments/{deployment_id}/logs` for progress. This is the deploy counterpart to `POST /v5/agentex/builds`: a build produces the container image, a deployment runs that image. The referenced build must have finished successfully, and the manifest's agent name must match the build's agent. Set `preview=True` for an ephemeral deployment: it gets a globally unique Helm release name (so concurrent redeploys never collide), an optional `preview_label` for grouping, and an expiry (`expires_at`, defaulting to 8 hours from now); `preview_label` and `expires_at` are rejected on non-preview deploys. A non-preview (production) deploy instead supersedes any prior active deployment that shares its Helm release name. Fails with a client error if the build is missing or not in a successful state, if the manifest or environment YAML is invalid or their agent names disagree, or if a secret referenced by the manifest does not exist. Returns 409 if the target agent has an active key migration or an applicable incomplete managed-key handoff. # Get agent deployment logs Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-deploy/get-agent-deployment-logs /openapi/v5.yaml get /v5/agentex/deployments/{deployment_id}/logs Get structured deployment log lines, with cursor-based pagination. Returns the deployment's log lines in time order together with a `next_cursor` and a `has_more` flag. Poll to stream logs incrementally: make the first call without a cursor, then pass the previous response's `next_cursor` as `cursor` on each subsequent call, stopping once the deployment reaches a terminal status. Unlike `GET /v5/agentex/deployments/{deployment_id}`, which returns the deployment's status and Kubernetes events, this returns the raw log output from the deploy process. Returns 404 if no deployment with this ID exists for the caller's account. # Get an agent deployment Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-deploy/get-an-agent-deployment /openapi/v5.yaml get /v5/agentex/deployments/{deployment_id} Get a single agent deployment by ID, including its current status and Kubernetes events. Returns the deployment record with its latest status and the associated Kubernetes events (`deploy_events`) observed for the release, which are useful for diagnosing why a deployment is still pending or unhealthy. Poll this after `POST /v5/agentex/deployments` to track the asynchronous deploy to completion. For the incremental log output rather than status and events, use `GET /v5/agentex/deployments/{deployment_id}/logs`. Returns 404 if no deployment with this ID exists for the caller's account. # List agent deployments Source: https://docs.gp.scale.com/reference/v5/agentex-cloud-deploy/list-agent-deployments /openapi/v5.yaml get /v5/agentex/deployments List the account's agent deployments, with pagination and optional filters. Returns the deployments the caller is authorized to read. Optionally filter by `build_id`, by `agent_name` (matched through each deployment's associated build), or by `preview_label`. A `preview_label` is non-unique — many deployments can share one (for example every deploy for a branch) — so combine it with `sort_order=desc&limit=1` to fetch the latest deployment for that label; the default order is oldest first. This lists deployments (the running or attempted agent instances); to list the image builds they run, use the agentex builds API. # Create Secret Source: https://docs.gp.scale.com/reference/v5/sgp-cloud-secrets/create-secret /openapi/v5.yaml post /v5/sgp/secrets Create an account-level secret. The secret value is stored in the cloud provider's secret store. SGP only stores metadata (key name, description, audit info). The value is never returned by any API. Returns 409 if a secret with the same key already exists. # Delete Secret Source: https://docs.gp.scale.com/reference/v5/sgp-cloud-secrets/delete-secret /openapi/v5.yaml delete /v5/sgp/secrets/{secret_id} Delete a secret from both the cloud provider and SGP metadata. # Get Secret Source: https://docs.gp.scale.com/reference/v5/sgp-cloud-secrets/get-secret /openapi/v5.yaml get /v5/sgp/secrets/{secret_id} Get a single secret's metadata by ID. The value is never returned. # List Secrets Source: https://docs.gp.scale.com/reference/v5/sgp-cloud-secrets/list-secrets /openapi/v5.yaml get /v5/sgp/secrets List secret metadata for the account. Values are never returned. # Update Secret Source: https://docs.gp.scale.com/reference/v5/sgp-cloud-secrets/update-secret /openapi/v5.yaml patch /v5/sgp/secrets/{secret_id} Update an existing secret's description and/or value. If value is provided, the cloud provider secret is updated. The secret value is never returned by any API. # Introduction to Code Authoring Source: https://docs.gp.scale.com/docs/capabilities/code-authoring/introduction-to-code-authoring Write, review, and ship code from a git server that runs inside your own SGP deployment. **Code Authoring is SGP's built-in version control and CI/CD surface.** Every SGP deployment can run its own git server (powered by [Gitea](https://about.gitea.com/)) inside the same cluster as the rest of the platform, with CI runners colocated alongside it. You get a normal git experience: clone over HTTPS, push branches, open pull requests, review diffs, and watch CI run. What is different is that there is nothing to set up. No external git host, no runner to provision, no network path to negotiate between a CI runner and your cluster, and no second set of credentials: your existing SGP API key and SGP login are the only credentials involved. > [!NOTE] > > Your git server lives at `https://gitea.` and never leaves your SGP deployment. Source code, build history, and CI logs stay inside your environment's trust boundary. *** ## Why it exists Standing up version control used to be the slowest part of a new deployment. It meant creating a repository on a vendor-managed host, wiring up mirroring, copying CI workflow files by hand, provisioning a runner, and then opening a network path from that runner into the target cluster. That last step was usually the blocker, because corporate security policy rarely allows an external runner to reach an internal cluster. Code Authoring removes the problem by construction. The git server and its CI runners are colocated with the workloads they build, so the "runner reaches the cluster" question never comes up. *** ## What you get out of the box When Code Authoring is enabled on your deployment, SGP provisions: | Component | What it is | | - | - | | **Git server** | A private git server at `https://gitea.`, with a web UI, a REST API, and HTTPS git access. | | **CI/CD** | GitHub Actions-compatible workflows defined in `.gitea/workflows/`. Anything you can express as an Action, you can run here. | | **CI runners** | In-cluster runners that execute those workflows. Nothing to install or connect. | You sign in with your existing SGP credentials: there are no separate git accounts, passwords, or SSH keys to manage. *** ## Getting started **1. Get your credentials** You need two things: * **Your SGP API key**, from the SGP UI under **Settings → API Keys** * **Network access** to `https://gitea.`, the same way you reach the SGP UI **2. Clone a repository** Your SGP API key is used as the git password. The username field is ignored, but it is conventional to use your SGP user ID or email. ```bash theme={null} git clone https://:@gitea.//.git ``` To avoid pasting the key on every operation, store it with a standard git credential helper, the same way you would a personal access token: ```bash theme={null} git config --global credential.helper osxkeychain # macOS git config --global credential.helper store # Linux (plaintext file) ``` **3. Browse in the web UI** Open `https://gitea.` in a browser. If you already have an SGP session you are signed in automatically; otherwise choose **Sign in with Scale SSO** and authenticate with your normal identity provider. From there you can review pull requests, read diffs, inspect CI runs, and manage repository settings. *** ## How it works Every request (git, REST, or browser) passes through the same SGP authentication layer before it reaches the git server: ```mermaid theme={null} flowchart TD A[git CLI / browser / SDK] -->|HTTPS: SGP API key or SGP session| B[SGP auth layer] B -->|validates credential, resolves account role,
provisions your git account on first use| C[Git server, in-cluster] C --> D[In-cluster CI runners] D -->|SGP APIs| E[Your workloads] ``` The auth layer validates your credential against SGP's identity service, creates your git account the first time it sees you, and then forwards the request. You never create a git account, and there is no separate password to rotate. Workflows are ordinary GitHub Actions-compatible YAML in `.gitea/workflows/`, triggered by the events you would expect: pushes, pull requests, comments, schedules, and manual dispatch. Because the runners sit inside your deployment, a workflow step can call the SGP APIs directly over the in-cluster address. Store your SGP credential as a repository Actions secret and any job can build, deploy, evaluate, or query through the platform: ```yaml theme={null} - name: Call the SGP API env: SGP_API_KEY: ${{ secrets.SGP_API_KEY }} SGP_ACCOUNT_ID: ${{ vars.SGP_ACCOUNT_ID }} SGP_BASE_URL: ${{ vars.SGP_BASE_URL }} run: | curl -sS -H "x-api-key: $SGP_API_KEY" "$SGP_BASE_URL/v5/..." ``` Secrets and variables are managed per repository under **Settings → Actions → Secrets** and **Settings → Actions → Variables** in the web UI. > [!WARNING] > > Treat an SGP API key stored as an Actions secret as a deployment credential. Anyone who can push a workflow to the repository can use it, so keep repository write access aligned with who you would trust with that key. *** ## Access control Repositories are scoped to an SGP account. The git server's own organization, team, and repository permissions decide what users can do. New users receive the default repository access configured on the git server; your administrator can change it. Service-account API keys can reach only repositories bound to an SGP account, with access set by the service account's role on that account: | SGP account role | Repository access | | - | - | | `admin`, `manager` | Read and write: browse, clone, fetch, push, open and merge pull requests | | `member` | Read only: browse, clone, and fetch | | Any other role, or no role | No access | A repository is bound to an SGP account when its `SGP_ACCOUNT_ID` Actions variable holds that account's ID. The role is checked on every request, so removing a service account's role revokes its key's git access immediately. *** ## Working over the API Anything you can do in the web UI you can do programmatically. The git server exposes a REST API at `https://gitea./api/v1/...`, authenticated with your SGP API key: ```bash theme={null} curl -H "x-api-key: $SGP_API_KEY" \ https://gitea./api/v1/repos///branches ``` Create repositories, open pull requests, list commits, read and write files in batch, and manage branch protection, all with your standard SGP credential. Interactive API documentation for your deployment is served at `https://gitea./api/swagger`, and the raw OpenAPI specification at `https://gitea./swagger.v1.json`. Standard git tooling works too: the `tea` CLI and the Gitea Go SDK authenticate with your SGP API key in place of a native access token. *** ## Current limits A few things to know before you plan around them: * **HTTPS only.** SSH git access is not offered. Every operation goes over HTTPS with your SGP API key, which covers CLI, CI, and programmatic use. * **Squash merges only.** The default branch is `main` and squash is the only merge style, so history stays linear across every repository. * **Brief maintenance windows.** The git server runs as a single instance, so platform upgrades involve roughly thirty seconds of unavailability. Running workloads are unaffected; only git operations pause. # Storage Mounts Source: https://docs.gp.scale.com/docs/capabilities/training/storage-mounts Mount cloud storage into training containers using a unified interface across GCP, AWS, and Azure. Train mounts cloud storage into your container before the job starts. On Vertex AI and Azure ML it also sets `STORAGE_MOUNT_0_PATH`, `STORAGE_MOUNT_1_PATH`, etc. as environment variables. Your training script reads from these paths, with no cloud-specific SDKs or storage code needed. *** ## Mount Configuration Each entry in `storage_mounts` maps a cloud URI to a container path: ```json theme={null} "storage_mounts": [ { "source_uri": "gs:///training-data", "mount_path": "/mnt/data", "read_only": true }, { "source_uri": "gs:///run-1", "mount_path": "/mnt/output", "read_only": false } ] ``` Inside the container on Vertex AI, each variable holds the Cloud Storage FUSE path of its `source_uri`: ``` STORAGE_MOUNT_0_PATH=/gcs//training-data STORAGE_MOUNT_1_PATH=/gcs//run-1 ``` Mounts are indexed in order, starting at zero. Each backend handles a mount differently: | Backend | `STORAGE_MOUNT__PATH` | `mount_path` | | - | - | - | | Vertex AI | `/gcs//`, the Cloud Storage FUSE path of `source_uri` | Train symlinks it to the FUSE path when the job sets `command`. It runs the command under `bash -c`, so the image needs bash | | Azure ML | The path where Azure ML mounted the input or output | Train symlinks it to that path when the job sets `command` | | SageMaker | Not set | Not used. Each read-only `s3://` mount becomes an input channel named `mount__`, where `` is the last segment of `mount_path`, downloaded to `/opt/ml/input/data/mount__/`. The first writable `s3://` mount becomes the job's S3 output path and replaces `output_data_config.s3_output_path` | *** ## URI Formats | Cloud | Format | Example | | - | - | - | | GCP | `gs://bucket/path` | `gs:///datasets/imagenet` | | AWS | `s3://bucket/path` | `s3:///datasets/imagenet` | | Azure | `azureml://datastores/name/paths/path` | `azureml://datastores/traindata/paths/imagenet` | > [!NOTE] > > On Azure ML, `source_uri` can be an `azureml://` datastore URI, a `wasbs://` or `abfss://` URI, or an `https://` URI on `*.blob.core.windows.net` or `*.dfs.core.windows.net`. Train skips mounts with any other URI and submits the job without them. Datastores are configured in your Azure ML workspace. > [!NOTE] > > Train rejects a job with `403` if its `storage_mounts` reference storage your account isn't allowed to use. By default, jobs can use only your deployment's Train storage: > > * AWS and GCP: the Train data, checkpoints, output, and staging buckets > * Azure: the `traindata`, `traincheckpoints`, and `trainoutput` datastores, which point to the `data`, `checkpoints`, and `output` containers of the training storage account > > Ask your Scale team for your Train bucket names, or to allow other storage for your account. *** ## Accessing Mounts in Your Container ```python focus={5-6} theme={null} import os import torch from datasets import load_dataset data_path = os.environ["STORAGE_MOUNT_0_PATH"] output_path = os.environ["STORAGE_MOUNT_1_PATH"] dataset = load_dataset(data_path) # ... training loop ... torch.save(model.state_dict(), f"{output_path}/checkpoint-epoch-{epoch}.pt") ``` This works on Vertex AI and Azure ML. SageMaker does not set `STORAGE_MOUNT_*` variables; read inputs from `/opt/ml/input/data/` instead. *** ## Read vs. Write Set `read_only` based on intent. Use `true` for input data and pretrained weights; `false` for outputs, checkpoints, and logs. Marking input mounts read-only prevents accidental writes and may improve performance on some backends. *** ## SageMaker: Input Data Config SageMaker also supports `input_data_config` for S3 input channels. Channels are downloaded to the instance before training starts, which can be faster for large datasets with random access patterns: ```json theme={null} "input_data_config": [{ "channel_name": "train", "data_source": { "s3_data_source": { "s3_data_type": "S3Prefix", "s3_uri": "s3:///datasets/train/", "s3_data_distribution_type": "FullyReplicated" } }, "input_mode": "File" }] ``` SageMaker makes this available at `/opt/ml/input/data/train/`. This coexists with `storage_mounts`. *** ## Next Steps * **[Getting Started](/docs/capabilities/training/getting-started-with-train)**: Full job submission examples per backend * **[Custom Images](/docs/capabilities/training/custom-images)**: Build and push training containers # AWS Architecture Reference Source: https://docs.gp.scale.com/docs/infrastructure/aws/aws-architecture-reference Overview of the SGP AWS architecture ## Architecture Reference ![](https://mintcdn.com/scalegp/zYvSiPc36dH92j52/images/infrastructure/sgp-aws-architecture.png?fit=max\&auto=format\&n=zYvSiPc36dH92j52\&q=85\&s=d8127094a421a4a0e6e567c29a09ae68) ## Cloud Services | Service | Usage | | - | - | | Elastic Kubernetes Service (EKS) | SGP applications are deployed to this cluster | | Identity and Access Management (IAM) | Role and policy management for SGP applications access to cloud services | | Secrets Manager | Secret and configuration management for SGP applications | | Key Management Service (KMS) | Encryption of sensitive data at rest | | Simple Storage Service (S3) | Object storage for SGP applications (knowledge bases, models, etc.) | | Simple Queue Service (SQS) | Karpenter spot instance interruption queue | | OpenSearch | Search provider for SGP applications | | Elastic Container Registry (ECR) | Container registry for custom agent images | | Aurora PostgreSQL | Relational databases for SGP applications | | DocumentDB | Document-oriented database for SGP applications | | ElastiCache for Redis | In-memory caching for SGP applications | | CloudWatch | Logging for cloud services | ## Networking SGP is deployed into a VPC spanning three Availability Zones with separate public and private subnets. All application workloads run in private subnets. Outbound internet access is provided by a NAT Gateway. ### Ingress | Property | Default | | - | - | | Load balancer type | Application Load Balancer (ALB) | | Listener port | 443 (TLS) | | Placement | External (internet-facing) | | WAFv2 | Optional — associate an existing Web ACL by name | | VPC Endpoint Service | Optional — enables AWS PrivateLink for private connectivity | > [!NOTE] > > The load balancer can be configured as internal (private subnet only) for deployments that require no direct internet exposure. ### Service Mesh System Manager's `istio` pack deploys Istio, which manages all east-west traffic between SGP services. An internal Istio ingress gateway handles traffic routing within the cluster. ## EKS Cluster Default Configuration ### Node Groups | Node Group | Size | Limits | Machine Type | Machine Image | Taints | | - | - | - | - | - | - | | System | 3 (one node group of 1 per private subnet) | — | m6a.2xlarge | EKS Default AMI | CriticalAddonsOnly=true:NoSchedule | | Default | Karpenter Managed Node Group | 1000 CPU | c\*, m\*, r\* | Karpenter AL2 AMI Family (default) | | | High Storage | Karpenter Managed Node Group | 100 CPU | c\*, m\*, r\* | Karpenter AL2 AMI Family (default) | high-storage:NoSchedule | | GPU | Karpenter Managed Node Group | 385 CPU | g5.\* | Karpenter AL2 AMI Family (default) | nvidia.com/gpu:NoSchedule | | GPU (Large) | Karpenter Managed Node Group | 100 CPU | p4d.\* | Karpenter AL2 AMI Family (default) | nvidia.com/gpu:NoSchedule | Karpenter AMI families are configurable through `karpenter.amiFamily` in the `egp-base` chart. AL2 is the configured default; AL2023 and Bottlerocket require an explicit configuration. ### Cluster Add-ons Terraform installs the EKS add-ons. System Manager installs the others as packs, when they are listed in its desired state. Karpenter provides node autoscaling; Cluster Autoscaler is not installed. | Add-on | Installed by | Purpose | | - | - | - | | Karpenter | System Manager pack `karpenter` | Workload-driven node autoscaling for Karpenter-managed node groups | | Istio | System Manager pack `istio` | Service mesh for east-west traffic management | | cert-manager | System Manager pack `cert-manager` | TLS certificate provisioning and renewal | | NVIDIA GPU Operator | System Manager pack `gpu-operator` | GPU driver and device plugin lifecycle management | | CoreDNS | EKS add-on | Cluster DNS | | kube-proxy | EKS add-on | Service networking on each node | | Amazon VPC CNI | EKS add-on | Pod networking | | AWS EBS CSI Driver | EKS add-on | Persistent volume support for EKS workloads | | metrics-server | EKS add-on | Resource metrics for autoscaling | | Amazon EFS CSI Driver | EKS add-on, off by default | EFS volumes; on with `enable_efs_csi_driver` or Document Understanding | | EKS Pod Identity Agent | EKS add-on, off by default | EKS Pod Identity; on with `enable_pod_identity_agent` | ## Aurora PostgreSQL Default Configuration These are the default Aurora instance counts for an SGP deployment on AWS; your deployment configuration can override them. | Instance | Service | Instance Count | Engine | Instance Class | | - | - | - | - | - | | agents-service-\-db | Agents | 3 | PostgreSQL16 | db.r6g.xlarge | | duc-\-duc-api-aurora-db | Document Understanding Capability | 3 | PostgreSQL15 | db.r6g.large | | reducto-aurora-db | Reducto | 3 | PostgreSQL15 | db.t4g.medium | | sae-\-sae-postgres | Semantic Analysis Engine | 1 | PostgreSQL17 | db.r6g.large | | scale-egp-\-agentex-aurora-db | Agentex | 3 | PostgreSQL15 | db.r6g.large | | scale-egp-\-egp-api-aurora-db | EGP API | 3 | PostgreSQL15 | db.r6g.large | | scale-egp-\-identity-service-aurora-db | Identity Service | 3 | PostgreSQL15 | db.r6g.large | | scale-egp-\-llm-engine-aurora-db | LLM Engine | 3 | PostgreSQL14 | db.r6g.large | | scale-egp-\-spicedb-postgres | SpiceDB | 3 | PostgreSQL16 | db.r6g.large | | scale-egp-\-temporal-aurora-db | Temporal | 3 | PostgreSQL15 | db.r6g.large | | scale-egp-\-train-aurora-db | SGP Train | 1 | PostgreSQL15 | db.t3.medium | ## DocumentDB Default Configuration | Instance | Service | Instance Count | Instance Class | | - | - | - | - | | scale-egp-\-compass-docdb | Compass | 1 | db.t3.medium | | scale-egp-\-agentex-mongo | Agentex | 1 | db.r5.large | ## Customer Sensitive Data Sensitive data may be stored in the following services, all data is encrypted at rest and in transit: * Aurora PostgreSQL: For storing metadata, persistent data, and service-specific databases (SGP API Backend, Identity Service, LLM Engine, SpiceDB, Agents Service, SGP Train, and Temporal) * DocumentDB: For storing Agentex agent data/workflows and Workflows document processing state * S3 Buckets: For storing large data objects and datasets * OpenSearch: For managing text embeddings and vector search capabilities # AWS SGP Deployment Source: https://docs.gp.scale.com/docs/infrastructure/aws/aws-sgp-deployment End-to-end guide to deploying SGP in your AWS account, together with Scale engineering. ## Overview This guide covers deploying SGP in an AWS account. SGP AWS infrastructure is defined by terraform modules managed by Scale. > [!NOTE] > > SGP is deployed jointly with Scale engineering. Scale provides everything under "The following from Scale" in Prerequisites and stays involved for upgrades and ongoing operation. Contact your Scale team before you start. ## Prerequisites * Access to an AWS account with sufficient permissions to create resources * The following tools installed: * [AWS CLI](https://aws.amazon.com/cli/) * [Terraform](https://www.terraform.io/) 1.9.0 or later * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) * [helm](https://helm.sh/docs/intro/install/) and [jq](https://jqlang.org/download/) (used by `install.sh`) * [sgpctl](https://github.com/scaleapi/sgpctl) (CLI that installs System Manager) * The following from Scale: * The SGP AWS Infrastructure Terraform modules * A `workspace_id` and `registration_token` unique to your deployment * The `bootstrap-helm` folder: `install.sh` and the `system-manager-dependencies` chart it installs * The consolidated version of SGP to install * A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional) * A custom domain for your deployment (optional) ## Installation ### Step 1: Prepare Terraform Root Module In a new directory, create a `.tf` file to invoke the SGP AWS infrastructure Terraform modules. Reference the `egp-hybrid-cloud-customer-deployment` module as the entrypoint for the SGP infrastructure. ```terraform theme={null} module "egp_hybrid_deployment" { source = "" scale_account_id = "" registration_token = var.registration_token # secrets are variables, set in Step 2 domain = ".workspace.egp.scale.com" # or your custom domain # Custom domain: import a certificate for and *. so the first apply doesn't wait for # ACM validation. You still create the CNAMEs listed in Step 2. The private key is stored in Terraform state. # ssl_cert_certificate_body_path = "" # ssl_cert_private_key_path = "" # ssl_cert_certificate_chain_path = "" # Omit for a self-signed certificate # Core services — enable the components you need deploy_egp_api = true # EGP API backend — required for the SGP platform deploy_spellbook = false # Spellbook prompt management service (optional) deploy_llm_engine = false # Model Engine service (optional) enable_agentex = false # Agentex agentic workflow service (optional) create_agentex_tool_cache = false # Persistent tool cache for Agentex (optional) enable_cloud_build = false # Cloud-based container build service for Agentex (optional) add_agents_service = false # Agent Service for building and running agents (optional) add_document_understanding_capability = false # Document Understanding (Dex) service (optional) enable_reducto = false # Reducto OCR and document parsing service (optional) enable_train = false # SGP model training infrastructure (optional) enable_compass = false # Workflows workflow orchestration service (optional) deploy_ukbp = false # Universal Knowledge Base Pipeline (optional) enable_system_manager = true # System Manager deployment orchestrator — required enable_monitoring = false # Observability stack (optional) desired_state_json = jsonencode({ # Packs System Manager installs; when unset, it deploys nothing packs = [ { name = "" }, ] }) create_temporal_postgresql = true # Recommended; set false to use an in-cluster Cassandra instead omit_egp_control_plane = false # true is LLM Engine-only mode: no control plane registration, workspace load balancer, listener or certificate base_repository = "" # Only needed if not pulling from Scale's registry openai_api_key = var.openai_api_key openai_org_id = "" anthropic_api_key = var.anthropic_api_key launch_api_Key = var.launch_api_Key # the module's argument name has a capital K authentication_type = "SAML" # "SAML" or "OIDC" use_cmk = true # Recommended for production: encrypts data at rest using AWS KMS # If deploying into an existing VPC rather than creating a new one, provide the IDs here # vpc_id = "" # private_subnet_ids = ["", ""] # public_subnet_ids = ["", ""] access_cidrs = ["0.0.0.0/0"] # CIDR blocks allowed to reach the platform; ["0.0.0.0/0"] is open to the internet roles_to_access_kubeapi = ["arn:aws:iam:::role/"] # IAM roles given cluster admin access to EKS; leave out the role that runs Terraform, which already has it eks_additional_access_cidrs = ["/32"] # IPs allowed to reach the EKS API endpoint, in addition to Scale's # Load balancer configuration load_balancer_type = "application" # "application" or "network" # waf_name = "" # Optional; only applies to application load balancers internal_lb = false # Set true to prevent internet access to the load balancer # additional_certs = [""] # Additional TLS certificates for the load balancer # permissions_boundary_arn = "" # Optional: IAM permissions boundary for provisioned roles # See the module documentation for the full list of available variables } # Secrets stay out of this file. Step 2 sets each one from an environment variable # named TF_VAR_. variable "registration_token" { type = string sensitive = true } variable "openai_api_key" { type = string sensitive = true default = null } variable "anthropic_api_key" { type = string sensitive = true default = null } variable "launch_api_Key" { type = string sensitive = true default = null } # install.sh needs SpiceDB's connection string in Step 3 output "spicedb_connection_string" { value = module.egp_hybrid_deployment.spicedb_connection_string sensitive = true } # DNS records for a custom domain point at these (see Step 2) output "load_balancer_dns" { value = module.egp_hybrid_deployment.load_balancer_dns } output "certificate_arn" { value = module.egp_hybrid_deployment.certificate_arn } # Example Terraform provider and backend configuration provider "aws" { region = "" } terraform { required_version = ">= 1.9.0" backend "s3" { bucket = "sgp--terraform-state" key = "sgp--sgp.tfstate" region = "" } } ``` Each `packs` entry in `desired_state_json` names a [System Manager](/docs/infrastructure/system-manager) pack; get the pack list for your install from your Scale team. System Manager reads `desired_state_json` only on first install, while its desired state is empty; after that, change packs in the System Manager GUI or API. ### Step 2: Provision Infrastructure via Terraform Run the following commands in the same directory as the previous step's .tf file. Terraform calls Scale's control plane during plan and apply, so the machine running it needs HTTPS access to `https://control.egp.scale.com`. ```bash theme={null} # Set the secrets Step 1 reads, without echoing them or saving them in shell history. # Skip the lines for API keys you don't use. printf 'Registration token: ' && read -rs TF_VAR_registration_token && echo && export TF_VAR_registration_token printf 'OpenAI API key: ' && read -rs TF_VAR_openai_api_key && echo && export TF_VAR_openai_api_key printf 'Anthropic API key: ' && read -rs TF_VAR_anthropic_api_key && echo && export TF_VAR_anthropic_api_key printf 'Model Engine API key: ' && read -rs TF_VAR_launch_api_Key && echo && export TF_VAR_launch_api_Key terraform init terraform plan -out=tfplan # Inspect this plan to review the planned resources to be provisioned before proceeding terraform show tfplan terraform apply tfplan ``` *This step may take significant time due to resource creation dependencies* > [!NOTE] > > Unless you import a certificate with the `ssl_cert_*_path` arguments in Step 1, the first apply will likely fail with this error: > > ```bash theme={null} > Error: creating ELBv2 Listener (arn:aws:elasticloadbalancing:us-east-1:12345678910:loadbalancer/net/scale-egp-/12345678910): operation error Elastic Load Balancing v2: CreateListener, https response error StatusCode: 400, RequestID: 12345678910, api error UnsupportedCertificate: The certificate 'arn:aws:acm:us-east-1:12345678910:certificate/12345678910' must have a fully-qualified domain name, a supported signature, and a supported key size. > ``` > > The listener needs an issued ACM certificate, validated through DNS. For a `.workspace.egp.scale.com` domain, Scale creates the DNS records: wait until ACM shows the certificate as issued, then re-run `terraform apply`. For a custom domain, first create these records with your DNS provider: > > * `` and `*.`: CNAME to the load balancer's DNS name, from `terraform output -raw load_balancer_dns` (use an alias record if `` is the apex of its DNS zone) > * The certificate's validation CNAME, whose name and value this command prints: > > ```bash theme={null} > aws acm describe-certificate --region \ > --certificate-arn "$(terraform output -raw certificate_arn)" \ > --query 'Certificate.DomainValidationOptions[].ResourceRecord' > ``` ### Step 3: Bootstrap the Cluster Export the values `install.sh` and `sgpctl` need. Run these in the same directory as the Step 1 `.tf` file, so `terraform output` reads its state: ```bash theme={null} export WORKSPACE_ID="" export AWS_REGION="" export SPICEDB_DATASTORE_URI="$(terraform output -raw spicedb_connection_string)" # The Secrets Manager secret holding SpiceDB's preshared key export SPICEDB_SECRET_ID="spicedb/$WORKSPACE_ID" ``` #### Configure EKS access The IAM identity that created the cluster in Step 2 has cluster admin access, and so do the roles in `roles_to_access_kubeapi` from Step 1. #### Allowlist your IP (if needed) The EKS API endpoint accepts connections from Scale's addresses and from the `eks_additional_access_cidrs` in Step 1, or only from `access_cidrs` when `omit_egp_control_plane = true`. To run the commands below from your machine, include its IP there. #### Install the prerequisites with install.sh Run the `install.sh` script to install what System Manager needs in the cluster: * The Secrets Store CSI driver and its AWS provider * The `sgp-system-manager` and `spicedb` namespaces * The `spicedb-config` secret, holding the connection string you exported and the preshared key from the SpiceDB secret Terraform created in AWS Secrets Manager `install.sh` reads that secret with your AWS credentials. It does not install System Manager. Run it from the `bootstrap-helm` folder, because it installs the `system-manager-dependencies` chart from there. ```bash theme={null} # Set the correct kubectl context aws eks update-kubeconfig --name "scale-egp-$WORKSPACE_ID" --region "$AWS_REGION" # Execute the install.sh script. Once it succeeds, the connection string is in the cluster, # so remove it from your shell cd ./install.sh && unset SPICEDB_DATASTORE_URI ``` If `install.sh` fails, fix the cause and run it again: the exports are still set. To run it again after it has succeeded, or from a new shell, first repeat the exports above from the Step 1 directory. #### Install System Manager with sgpctl Install [SGP System Manager](/docs/infrastructure/system-manager) with `sgpctl`, using the consolidated version from Scale. `sgpctl` installs the chart from the `scale-egp-charts` Helm repository, so add it first. It also fetches a license key and registry credentials from Scale's control plane, so the machine running it needs HTTPS access to `https://control.egp.scale.com`. `` is the AWS account Terraform deployed into. ```bash theme={null} helm repo add scale-egp-charts https://scale-egp-helm-charts-us-west-2.s3.amazonaws.com/charts # Read the registration token without echoing it or saving it in shell history printf 'Registration token: ' && read -rs SGP_REGISTRATION_TOKEN && echo sgpctl install aws \ --workspace-id "$WORKSPACE_ID" \ --control-plane-id "$WORKSPACE_ID" \ --registration-token "$SGP_REGISTRATION_TOKEN" \ --consolidated-version \ --aws-account-id \ --aws-region "$AWS_REGION" ``` System Manager is the deployment orchestrator for the SGP platform. It deploys the packs listed in `desired_state_json` from Step 1. ### Step 4: Verify the Deployment Wait for all services to be ready: ```bash theme={null} kubectl get helmreleases -A # All should be Ready=True kubectl get pods -A # All should be Running ``` ### Step 5: Configure Identity Provider #### SAML Configuration In your Identity Provider, configure a new SAML application with the following settings: * Service Entity ID: `https://auth.` * Redirect URI: `https://auth./dashboard/org/saml/callback` Modify the configuration `scale-egp-/identity-service-saml-secrets` (either via System Manager or directly in AWS Secrets Manager): ```json theme={null} { "id": "", "samlConfiguration": { "entityId": "", "x509Cert": "", "ssoUrl": "", "attributeMappings": { "email": "", "firstName": "", "lastName": "" } } } ``` #### OIDC Configuration In your Identity Provider, configure a new OIDC application with the following settings: * Redirect URI: `https://auth./dashboard/org/oidc/callback` Modify the configuration `scale-egp-/identity-service-oidc-secrets` (either via System Manager or directly in AWS Secrets Manager) with the following JSON: ```json theme={null} { "id": "", "oidcConfiguration": { "clientId": "", "clientSecret": "", "issuer": "", "authorizationUrl": "", "tokenUrl": "", "userInfoUrl": "" } } ``` After modifying the secret, restart the `sgp-system-manager` deployment to apply the changes. ```bash theme={null} kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager ``` ## Accessing the Platform If all goes smoothly, you should be able to navigate to the SGP platform at `https://.workspace.egp.scale.com` (or your custom domain) and authenticate via the configured identity provider. # Azure Architecture Reference Source: https://docs.gp.scale.com/docs/infrastructure/azure/azure-architecture-reference Overview of the SGP Azure architecture ## Architecture Diagram ```mermaid theme={null} graph TD USER(["Client"]) INTERNET(["Internet"]) subgraph AZ["Azure Subscription"] FD["Azure Front Door
WAF · CDN · TLS termination"] subgraph VNET["Virtual Network — 10.0.0.0/16"] INGRESS["Istio Ingress
Internal Load Balancer"] NAT["NAT Gateway
Outbound egress"] subgraph AKS["AKS Cluster — private API server"] SYS["System Node Pool
3–10× Standard_D4s_v3"] APP["User Node Pool
1–8× Standard_D32s_v3"] end BASTION["Azure Bastion"] JUMP["Jump Host VM"] end KV["Key Vault
Secrets + CMK keys"] LAW["Log Analytics Workspace"] PG[("PostgreSQL
Flexible Server")] REDIS[("Redis Cache
Premium")] STOR[("Storage Account")] SB[("Service Bus
Premium")] end USER --> FD FD -->|"Private Link"| INGRESS INGRESS --> APP APP -->|"Delegated subnet / VNet integration"| PG APP -->|"Private Endpoint"| REDIS APP -->|"Private Endpoint"| STOR APP -->|"Private Endpoint"| SB APP --> KV SYS --> KV APP --> LAW SYS --> LAW APP --> NAT --> INTERNET BASTION --> JUMP -->|"kubectl"| AKS ``` Node pool sizes on this page are the Terraform defaults. Values under `aks.node_pools` in your environment YAML override them field by field; the example in the [deployment guide](/docs/infrastructure/azure/azure-sgp-deployment) sets 3–6 system nodes and 3–10 `Standard_D16s_v3` user nodes. *** ## Resources by Type ### Compute Resources | Resource | Count | Purpose | | - | - | - | | AKS Cluster | 1 | Kubernetes orchestration | | System Node Pool | 1 | System pods (autoscales 3–10 nodes by default) | | User Node Pool | 1 | Application workloads | | GPU Node Pool | 0-1 | AI/ML workloads (optional) | | Cassandra Pool | 0-1 | Temporal database (optional) | Node counts depend on each pool's configured minimum and maximum and on enabled optional pools. *** ### Network Resources | Resource | Count | Purpose | | - | - | - | | Virtual Network | 1 | Network boundary | | Subnets | 4-5 | Network segmentation | | NSGs | 4-5 | Traffic control | | Route Tables | 3-4 | Traffic routing | | Private DNS Zones | 7 | Internal name resolution | | Private Endpoints | 7 | Secure PaaS access | | Bastion Host | 0-1 | Secure VM access | | NAT Gateway public IP | Configuration dependent | Outbound egress | | Bastion public IP | 0-1 | Bastion endpoint when enabled | *** ### Data & Storage Resources | Resource | Count | Purpose | | - | - | - | | PostgreSQL Server | 1 | Relational database | | Redis Cache | 1 | Distributed cache | | Storage Account | 1 | Blob/File storage | | AI Search Service | 0–1 | Full-text search (optional) | | OpenAI Service | 0–1 | LLM models (optional) | *** ### Security Resources | Resource | Count | Purpose | | - | - | - | | Key Vault | 1 | Secrets management | | Cluster managed identities | Configuration dependent | AKS and kubelet authentication | | Workload managed identities | One per enabled service | Service authentication, including cert-manager, identity-service, API backend, annotation, System Manager, secret-mounter, SpiceDB, and agents | | RBAC Role Assignments | 10+ | Access control | *** ### Monitoring Resources | Resource | Count | Purpose | | - | - | - | | Log Analytics Workspace | 1 | Centralized logging | | Data Collection Rule | 1 | AKS metrics | | Diagnostic Settings | 7+ | Resource logging | | Datadog Connection | 0-1 | External monitoring | *** ## Network Architecture ### Address Space Planning ``` VNet: 10.0.0.0/16 (65,536 IPs) ├── AKS Subnet: 10.0.1.0/24 (256 IPs) ├── Bastion Subnet: 10.0.2.0/26 (64 IPs) ├── Database Subnet: 10.0.3.0/24 (256 IPs) └── Private Endpoints: 10.0.4.0/25 (128 IPs) Pod CIDR: 10.244.0.0/16 (65,536 IPs) Service CIDR: 10.243.0.0/16 (65,536 IPs) ``` ### Traffic Flow **Egress (Internet):** ``` Pods/VMs → NAT Gateway → Public IP → Internet (Stateful, return traffic allowed) ``` **Ingress (Internal):** ``` Service IP → Load Balancer → Pod IP (via CNI) ``` **Database Access:** ``` AKS Pods → VNet → PostgreSQL delegated subnet (DNS: server.postgres.database.azure.com) Private Endpoint access is an alternative when configured; core PostgreSQL defaults to VNet integration. ``` **External Service Access:** ``` AKS Pods → API Gateway / Load Balancer → OpenAI / AI Search (Via Private Endpoints) ``` *** ## AKS Configuration Deep Dive ### API Server Access * **Type:** Private cluster (recommended) * **Endpoint:** Internal only * **Access Method:** Bastion host or VPN * **DNS:** A private name that starts with `aks.dns_prefix` and ends in `privatelink..azmk8s.io`. Print it with `az aks show --resource-group --name --query privateFqdn -o tsv`. ### Network Policies * **Engine:** Cilium with Azure CNI Overlay (default, configurable in `aks.network`) * **Scope:** Pod-to-pod communication * **Default:** Allow all (unrestricted) * **Configuration:** Define in Kubernetes manifests ### Container Registry * **Integration:** Azure Container Registry (optional) * **Authentication:** Managed identity or pull secrets * **Pulling:** Private endpoint (optional) ### Monitoring & Observability * **Azure Monitor Agent:** Deployed in kube-system * **Metrics:** CPU, Memory, Disk, Network * **Logs:** Container stdout/stderr, Kubernetes events * **Dashboards:** Pre-built in Log Analytics # Azure SGP Deployment Source: https://docs.gp.scale.com/docs/infrastructure/azure/azure-sgp-deployment End-to-end guide to deploying SGP in your Azure subscription, together with Scale engineering. ## Overview This guide covers deploying SGP in an Azure cloud subscription using the SGP Azure Terraform modules. SGP Azure infrastructure is defined by terraform modules managed by Scale. > [!NOTE] > > SGP is deployed jointly with Scale engineering. Scale provides everything under "The following from Scale" in Prerequisites and stays involved for upgrades and ongoing operation. Contact your Scale team before you start. ## Prerequisites * Access to an Azure subscription with sufficient permissions to create resources (Contributor + User Access Administrator roles, or equivalent) * The following tools installed: * [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) (`az login` completed) * [Terraform](https://www.terraform.io/) (`>= 1.9.0, < 2.0.0`) * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) * The following from Scale: * The SGP Azure Infrastructure Terraform modules (`azure-terraform/infra`) * A `workspace_id`, plus the `control_plane_id` and `registration_token` that register your deployment with Scale's control plane * A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional) * A custom domain for your deployment (optional) ## Installation ### Step 1: Build Configuration Configuration is split across three files. This allows you to reuse the same configuration for multiple environments. ``` infra/ ├── main.tfvars.json # Selects which domain + environment to deploy └── config/ └── / ├── default.yaml # Domain-wide defaults (all environments) └── / └── .yaml # Environment-specific overrides ``` #### File 1: `main.tfvars.json` This file selects which configuration to load. Edit it to select your domain code, region and environment before running Terraform. ```json theme={null} { "environment_type": "", "domain_code": "
", "location": "" } ``` | Field | Description | Allowed values | | - | - | - | | `environment_type` | Environment tier | `dev`, `qa`, `staging`, `uat`, `prod` | | `domain_code` | Customer identifier (3–9 alphanumeric chars/hyphens) | e.g. `acme`, `contoso` | | `location` | Azure region | e.g. `eastus2`, `northeurope`, `southeastasia` | #### File 2: `config//default.yaml` Customer-wide baseline settings that apply to **all** environments unless overridden. This file is the right place for stable policy and posture decisions: tenant identity, tagging standards, and security baselines for Key Vault, PostgreSQL, Redis, Storage, and other services. ```yaml theme={null} # Identity business_unit: "" tenant_id: "" default_tags: costcenter: "" product: "" deploymenttype: "new" safe-to-delete: "no" # CMK key names (Terraform creates these keys when cmk.create=true in the env YAML) cmk: create: true encryption_key_name: "encryptionCMK" k8s_key_name: "k8sEncryptionCMK" service_bus_key_name: "serviceBusEncryptionCMK" # Key Vault network posture — secure-by-default baseline. # Override public_network_access_enabled and network_acls_ip_rules in your env YAML # if you need temporary public access during bootstrap from a laptop or CI runner. keyvault: public_network_access_enabled: false enable_rbac_authorization: true sku_name: "standard" soft_delete_retention_days: 90 # Cannot be changed after vault creation purge_protection_enabled: true network_acls_default_action: "Deny" network_acls_bypass: "AzureServices" network_acls_ip_rules: [] # PostgreSQL baseline psql: public_network_access_enabled: false sku_name: "GP_Standard_D2s_v3" psql_version: "15" backup_retention_days: 7 bootstrap_aad_principals: false auth: active_directory_auth_enabled: true password_auth_enabled: true storage_mb: 32768 storage_tier: "P10" auto_grow_enabled: true azure_extensions: "uuid-ossp,ltree" # Redis baseline redis: public_network_access_enabled: false sku_name: "Premium" family: "P" capacity: 1 active_directory_authentication_enabled: true access_policy_name: "Data Owner" non_ssl_port_enabled: false tls_version: 1.2 redis_version: 6 # Storage account baseline storage_account: public_network_access_enabled: false min_tls_version: "TLS1_2" account_tier: "Standard" account_replication_type: "LRS" allow_nested_items_to_be_public: false # OpenAI — leave client.mode empty to let Terraform infer from open_ai.create open_ai: client: mode: "" key: "" org_id: "" custom_url: "" # NAT Gateway baseline nat_gateway: sku_name: "Standard" public_ip_sku: "Standard" public_ip_allocation_method: "Static" zones: ["1", "2", "3"] idle_timeout_in_minutes: 10 # AKS baseline aks: load_balancer_sku: "standard" only_critical_addons_enabled: true key_vault_secret_rotation_enabled: true # Front Door baseline frontdoor: create: true ssl_mode: managed minimum_tls_version: "TLS12" forwarding_protocol: HttpOnly private_link_service_name: "sgp-ingress-lb" subdomains: ["auth", "api", "@", "admin"] extra_subdomains: [] dns_ttl_seconds: 60 ip_filtering: enabled: false allowed_ip_ranges: [] # System Manager bootstrapping baseline bootstrapping: az_cli_version: "2.59.0" helm_chart_version: "2.1.0" base_repository: "" ``` #### File 3: `config///.yaml` Environment-specific configuration. This is where you set everything that differs per environment: resource names, subscription, CIDRs, feature flags, node pool sizing, and bootstrapping state. ```yaml theme={null} # ── Environment identity ─────────────────────────────────────────────────────── location: eastus2 subscription_id: "" name_suffix: "" # e.g. "acme01" deployment_id: "" deployment_url: ".workspace.egp.scale.com" temporal_db_mode: postgresCommonHosted # Recommended; or "cassandraK8s" (not recommended) # ── Auth / secrets ──────────────────────────────────────────────────────────── authType: default # Options: default, SAML, OIDC RBAC: false registration_secret: "" # Auto-generated if empty identity_service_jwt: "" # Auto-generated if empty # ── OIDC (required only when authType = OIDC) ───────────────────────────────── oidc: clientId: "" clientSecret: "" issuer: "" authorizationUrl: "" tokenUrl: "" userInfoUrl: "" # ── SAML (required only when authType = SAML) ───────────────────────────────── saml: X509Cert: "" SSOUrl: "" emailAttrName: "" firstNameAttrName: "" lastNameAttrName: "" # ── SSL ─────────────────────────────────────────────────────────────────────── sslMode: e2e # Options: e2e, managed # SSL cert/key for Istio ingress (PEM file paths or base64). Required when sslMode=e2e. istio: sslCert: "" sslKey: "" # ── Resource group + networking ─────────────────────────────────────────────── resource_group: "rg-eus2---dev-01" network: vnet: "sgpaz-vnet" int_rt_name: "sgpaz-rt-int" int_nsg_name: "sgpaz-nsg-int" private_cluster: true nsg_rules: postgresql: true redis: true private_dns: create: true private_cluster: true subnets: psql: name: "sgpaz-snet-psql" type: int address_prefixes: - "" # e.g. "10.95.23.160/27" service_endpoints: - Microsoft.Storage service_delegation: name: Microsoft.DBforPostgreSQL/flexibleServers actions: - Microsoft.Network/virtualNetworks/subnets/join/action deployment_scripts: name: "sgpaz-snet-deployscripts" type: int address_prefixes: - "" service_delegation: name: Microsoft.ContainerInstance/containerGroups actions: - Microsoft.Network/virtualNetworks/subnets/action aks: name: "sgpaz-snet-aks" type: int address_prefixes: - "" private_endpoint_network_policies: Disabled service_endpoints: - Microsoft.CognitiveServices bastion: name: AzureBastionSubnet type: int address_prefixes: - "" jump_host: name: "sgpaz-snet-jumphost" type: int address_prefixes: - "" # ── Workload Identity + NAT Gateway ─────────────────────────────────────────── workload_identities: enabled: true nat_gateway: enabled: true name: "sgpaznat" zones: [] # [] = regional (no zone pinning) # ── CMK ─────────────────────────────────────────────────────────────────────── # Set create=true for new deployments. If CMK keys already exist, set create=false # and provide the key URIs to avoid "already exists (import required)" errors. cmk: enabled: true create: true # encryption_key_uri: "https://.vault.azure.net/keys/encryptionCMK/" # k8s_key_uri: "https://.vault.azure.net/keys/k8sEncryptionCMK/" # service_bus_key_uri: "https://.vault.azure.net/keys/serviceBusEncryptionCMK/" # ── Key Vault ───────────────────────────────────────────────────────────────── # Override public access settings here if bootstrapping from a laptop or CI runner. keyvault: name: "sgpazkeyvault" # Must be globally unique; 3-24 characters soft_delete_retention_days: 90 # Must match existing vault if already created public_network_access_enabled: true # Set false once stable; keep true during bootstrap network_acls_ip_rules: - "/32" # Your laptop or CI runner IP # ── Core services ───────────────────────────────────────────────────────────── law: create: true psql: create: true name: "sgpazpostgres" # Must be globally unique admin_user: postgres bootstrap_aad_principals: true # Runs once after cluster is up; requires kubectl access redis: create: true name: "sgpazredis" # Must be globally unique storage_account: create: true name: "sgpazstorage" # Must be globally unique; no hyphens enable_file_private_endpoint: true service_bus: create: true name: "sgpazservicebus" sku: Premium # Premium required for private endpoints capacity: 1 premium_messaging_partitions: 1 private_endpoint_enabled: true public_network_access: false # ── AI services ─────────────────────────────────────────────────────────────── # Modes: use_openai_via_azure | use_openai_via_custom_endpoint | no_openai # NOTE: Do not commit real API keys. Keep `key` as a local uncommitted change. open_ai: create: false name: "sgpazopenai" client: mode: "use_openai_via_custom_endpoint" key: "" custom_url: "" ai_search: create: false name: "sgpazaisearch" # ── AKS ─────────────────────────────────────────────────────────────────────── aks: create: true name: "sgpazaks" node_resource_group: "rg-aks-eus2---dev-01" dns_prefix: "egp-k8scluster" sku: Free # Free | Standard (use Standard for production) kubernetes_version: "1.36" run_command_enabled: true enable_encryption_at_host: true use_azure_managed_flux: true zones: ["1", "2", "3"] node_pools: system: name: default vm_size: Standard_D4s_v3 count: 3 min_count: 3 max_count: 6 user: enabled: true name: user vm_size: Standard_D16s_v3 count: 3 min_count: 3 max_count: 10 gpu: enabled: false # Set true to enable GPU workloads name: gpu vm_size: Standard_NV72ads_A10_v5 min_count: 0 max_count: 5 cassandra: enabled: false # Required only if temporal_db_mode is cassandraK8s network: pod_cidr: "10.244.0.0/16" service_cidr: "10.243.0.0/16" dns_service_ip: "10.243.0.10" outbound_type: userDefinedRouting # ── Bastion / jump host ─────────────────────────────────────────────────────── bastion: create: true admin_username: azureuser ssh_public_key: "" # ── Front Door overrides ────────────────────────────────────────────────────── # Most Front Door settings inherit from default.yaml. Override only what differs. frontdoor: extra_subdomains: [] # e.g. ["chat"] if Agentex UI is enabled # ── Feature flags ───────────────────────────────────────────────────────────── feature_flags: models: false agentex: create: false # Set true to provision Agentex service compass: create: false # Set true to provision Workflows service dex: create: false # Set true to provision Dex (Document Understanding) service reducto: create: false # Set true to provision Reducto service # ── Bootstrapping (System Manager) ──────────────────────────────────────────── bootstrapping: enabled: true use_managed_flux: true system_manager_version: "" # Required when enabled is true; Terraform has no default pack list. Get the list for your install from your Scale team desiredState: | { "version": "0.1", "packs": [ { "name": "" } ] } # ── Policy assignments ──────────────────────────────────────────────────────── policy_assignments: enabled: false # Set true to enforce Azure Policy tag rules # ── Observability ───────────────────────────────────────────────────────────── observiqidp: monitoring: enabled: false insights: enabled: false ``` > [!NOTE] > > **Naming constraints for Azure resources:** > > * Key Vault names: 3–24 alphanumeric characters and hyphens, globally unique > * Storage Account names: 3–24 lowercase letters and numbers only (no hyphens), globally unique > * PostgreSQL and Redis names: globally unique within Azure > * All names must remain stable after first apply — many Azure resources cannot be renamed #### SAML Configuration To sign in with SAML, set `authType: "SAML"` in your environment YAML, then configure your Identity Provider with: * **Service Entity ID**: `https://auth.` * **Redirect URI**: `https://auth./dashboard/org/saml/callback` Put the values from your Identity Provider in the `saml` block. Terraform sets the entity ID to `https://auth.`. ```yaml theme={null} saml: X509Cert: "" SSOUrl: "" emailAttrName: "" firstNameAttrName: "" lastNameAttrName: "" ``` #### OIDC Configuration To sign in with OIDC, set `authType: "OIDC"` in your environment YAML, then configure your Identity Provider with: * **Redirect URI**: `https://auth./dashboard/org/oidc/callback` Put the values from your Identity Provider in the `oidc` block. `TF_VAR_oidc_client_secret`, when set, takes precedence over `oidc.clientSecret`, which keeps the client secret out of the YAML. ```yaml theme={null} oidc: clientId: "" clientSecret: "" issuer: "" authorizationUrl: "" tokenUrl: "" userInfoUrl: "" ``` To change these settings after the first install, update the YAML, run `terraform apply`, and restart System Manager: ```bash theme={null} kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager ``` ### Step 2: Provision Infrastructure via Terraform Navigate to the `infra` directory, initialize, and set the control plane ID and registration token from Scale: ```bash theme={null} cd infra terraform init # Read the values without echoing them or saving them in shell history printf 'Control plane ID: ' && read -rs TF_VAR_control_plane_id && echo && export TF_VAR_control_plane_id printf 'Registration token: ' && read -rs TF_VAR_registration_token && echo && export TF_VAR_registration_token ``` Review and apply the plan using the `main.tfvars.json` file you configured in the previous step: ```bash theme={null} terraform plan -var-file=main.tfvars.json -out=tfplan # Inspect the plan to review resources before proceeding terraform show tfplan terraform apply tfplan ``` If the Datadog monitoring integration is enabled in your YAML, pass the API key via environment variable rather than committing it to file: ```bash theme={null} printf 'Datadog API key: ' && read -rs TF_VAR_datadog_api_key && echo && export TF_VAR_datadog_api_key terraform plan -var-file=main.tfvars.json -out=tfplan ``` *This step may take significant time (30–60 minutes) due to resource creation dependencies, particularly the AKS cluster and PostgreSQL Flexible Server.* ### Step 3: Bootstrap the Cluster The Azure infrastructure **automatically bootstraps** [SGP System Manager](/docs/infrastructure/system-manager) via an Azure Deployment Script. When `bootstrapping.enabled: true` is set in your configuration, Terraform provisions an Azure Container Instance that: 1. Installs Flux CD on the AKS cluster (using Azure Managed Flux if `use_managed_flux: true`) 2. Applies the System Manager `HelmRepository` and `HelmRelease` Flux CRDs 3. Waits for System Manager to reconcile Monitor bootstrap progress in the Azure Portal under **Deployment Scripts** in your resource group, or check System Manager logs after bootstrap: ```bash theme={null} # Get AKS credentials (from inside the VNet or via Bastion — cluster is private by default) az aks get-credentials \ --resource-group "" \ --name "sgpazaks" \ --overwrite-existing # Verify System Manager is running kubectl get pods -n sgp-system-manager # Watch System Manager deploy remaining services via Flux kubectl get helmreleases -A ``` #### Accessing the Private AKS Cluster Because the AKS cluster is private by default, you must access it from within the provisioned VNet. Two options are provided: **Option A: From the jump host VM (via Azure Bastion)** A Bastion host is provisioned when `bastion.create: true`. Connect via the Azure Portal (Bastion blade). **Option B: Using `az aks command invoke`** When `aks.run_command_enabled: true`, you can run kubectl commands without VPN access: ```bash theme={null} az aks command invoke \ --resource-group "" \ --name "sgpazaks" \ --command "kubectl get helmreleases -A" ``` ### Step 4: Configure DNS After `terraform apply` completes, retrieve the Front Door endpoint hostname: ```bash theme={null} terraform output frontdoor_endpoint_host_name ``` Configure a CNAME record in your DNS provider pointing your `deployment_url` to the Front Door endpoint (the `azurefd.net` hostname from the output above). > [!NOTE] > > If using Azure DNS (the DNS zone is managed by Terraform when `frontdoor.create: true`), CNAME records are created automatically. Verify with: > > ```bash theme={null} > az network dns record-set list \ > --resource-group "" \ > --zone-name "" > ``` ### Step 5: Verify the Deployment Wait for all services to be ready: ```bash theme={null} kubectl get helmreleases -A # All should show Ready=True kubectl get pods -A # All should be Running or Completed ``` Terraform sets the `system-manager-desired-state` Key Vault secret from `bootstrapping.desiredState` only when the secret is first created; later YAML edits have no effect. After the first install, change the desired state in the System Manager GUI, API, or Key Vault, then install the affected packs. Saving the secret alone does not apply changes. By default, restarting System Manager reinstalls every pack in the desired state; see [System Manager configuration](/docs/infrastructure/system-manager#configuration). To change startup behavior on Azure, contact your Scale deployment team. If a HelmRelease shows `Ready=False`, check its events: ```bash theme={null} kubectl describe helmrelease "" -n "" ``` ## Accessing the Platform If all goes smoothly, you should be able to navigate to the SGP platform at `https://.workspace.egp.scale.com` (or your custom domain) and authenticate via the configured identity provider. # GCP Architecture Reference Source: https://docs.gp.scale.com/docs/infrastructure/gcp/gcp-architecture-reference Overview of the SGP GCP architecture ## Architecture Diagram ```mermaid theme={null} graph TD USER(["Client"]) ENGINEER(["Engineer"]) subgraph GCP["GCP Project"] GLB["Global Load Balancer
Static IP · TLS 1.2+"] IAP["Cloud IAP"] subgraph VPC["VPC — 10.0.0.0/16"] INGRESS["Istio Ingress"] subgraph GKE["GKE Cluster"] SYS["System Pool
3× n2-standard-4"] CPU["CPU Pool
4–30× n2-standard-16"] end BASTION["Bastion VM
e2-micro"] end subgraph PEERED["VPC-Peered — Cloud SQL"] PG_MAIN[("Cloud SQL
PostgreSQL 17 (main)")] PG_TEMP[("Cloud SQL
PostgreSQL 17 (temporal)")] end SM["Secret Manager"] subgraph GCS_GROUP["Cloud Storage"] BUCKET[("GCS — platform")] KB[("GCS — knowledge base")] end AR["Artifact Registry
Docker + Helm"] KMS["Cloud KMS
CMEK (optional)"] end USER --> GLB GLB --> INGRESS INGRESS --> CPU CPU -->|"VPC Peering · SSL enforced"| PG_MAIN CPU -->|"VPC Peering · SSL enforced"| PG_TEMP CPU -->|"Workload Identity"| SM CPU -->|"Workload Identity"| BUCKET CPU -->|"Workload Identity"| KB AR -->|"Workload Identity"| CPU KMS -.->|"encrypts"| PG_MAIN KMS -.->|"encrypts"| PG_TEMP KMS -.->|"encrypts"| BUCKET ENGINEER --> IAP --> BASTION -->|"private API"| GKE ``` *** ## Terraform Structure SGP's GCP infrastructure is provisioned in two separate Terraform phases with different privilege levels: ```mermaid theme={null} graph LR CREDS(["Personal GCP
Credentials"]) subgraph PS["projectsetup/ — run once with personal credentials"] SA["Terraform
Service Account"] AR_PS["Artifact Registries
Docker + Helm"] SMK["SA Key stored in
Secret Manager"] end subgraph DEP["deployments/name/ — SA credentials auto-loaded from Secret Manager"] GKE_D["GKE Cluster"] SQL_D["Cloud SQL × 2"] VPC_D["VPC + Subnets"] SEC_D["Secrets + IAM
+ Workload Identity"] end CREDS -->|"terraform apply"| PS SMK -->|"credentials
auto-read at plan time"| DEP ``` | Phase | Directory | Credentials | Scope | | - | - | - | - | | Privileged bootstrap | `projectsetup/` | Your personal GCP identity | Service account, Artifact Registries, API enablement | | Dedicated execution identity | `deployments//` | Terraform service account (read from Secret Manager) | GKE, Cloud SQL, networking, IAM, secrets | The main infrastructure Terraform uses a dedicated service account with broad administration roles, including compute, GKE, Cloud SQL, Secret Manager, project IAM, service account keys, Artifact Registry, and Storage administration. Configure the deployment to read the key secret created by `projectsetup/`, as described in the [deployment guide](/docs/infrastructure/gcp/gcp-sgp-deployment#step-1-bootstrap-the-project); then no key file needs to exist on disk during the infrastructure run. *** ## Resources by Type ### Compute Resources | Resource | Count | Purpose | | - | - | - | | GKE Cluster | 1 | Kubernetes orchestration | | System Node Pool (`default`) | 1 | System pods — tainted `CriticalAddonsOnly` | | CPU Node Pool (`cpu`) | 1 | Application workloads | | GPU Node Pools (`gpu-`) | 0–N | AI/ML workloads, one pool per `gpu_node_pools` entry (none by default) | | Cassandra Node Pool | 0–1 | Temporal database (optional) | | Bastion Host | 0–1 | Private cluster access via IAP (optional) | **Default node pool sizing:** | Pool | Machine Type | Min Nodes | Max Nodes | | - | - | - | - | | System (`default`) | `n2-standard-4` | 3 | 10 | | CPU (`cpu`) | `n2-standard-16` | 4 | 30 | | GPU (`gpu-`) | Set per `gpu_node_pools` entry | 0 | Set per entry (no GPU pools by default) | | Cassandra | `n2-standard-4` | 3 | 6 | General-purpose pools use `COS_CONTAINERD` and are non-preemptible by default (`node_pool_config.preemptible=false`). The option can enable preemptible nodes for pools that honor it, including the `gvisor` and `gvisor_allow_all` sandbox pools. The KubeVirt, Gitea Actions, ClickHouse, OpenSearch, and Temporal system pools are always non-preemptible. *** ### Network Resources | Resource | Count | Purpose | | - | - | - | | VPC Network | 1 | Network boundary | | Subnetwork | 1 | Kubernetes nodes | | Secondary IP Ranges | 2 | GKE pod and service CIDRs | | Cloud DNS Zone | 1 | Internal + external name resolution | | Global Static IP | 1 | Load balancer ingress endpoint | | SSL Policy | 1 | Minimum TLS 1.2 enforcement | | Firewall Rules | 4–5 | Traffic control (offline mode) | | VPC Peering | 1 | Private connectivity to Cloud SQL | | Private Route | 0–1 | Google APIs access in offline mode | *** ### Data & Storage Resources | Resource | Count | Purpose | | - | - | - | | Cloud SQL (PostgreSQL 17) | 1 | Main platform database | | Cloud SQL (PostgreSQL 17) | 1 | Temporal workflow database | | GCS Bucket (main) | 1 | Platform object storage | | GCS Bucket (knowledge base) | 1 | KB document storage | | GCS Bucket (monitoring) | 0–1 | Observability data (optional) | | Vertex AI Vector Search Index | 0–N | Vector embeddings (optional) | | Cloud Firestore | 0–1 | Agentex state (optional) | Both Cloud SQL instances are private-only (no public IP) and connected to the VPC via VPC peering. SSL is enforced for all database connections (`ENCRYPTED_ONLY`). *** ### Security Resources | Resource | Count | Purpose | | - | - | - | | Secret Manager Secrets | 10+ | Platform configuration and credentials | | Cloud KMS Key Ring + Key | 0–1 | Customer Managed Encryption Keys (optional) | | Service Account (`sgp--sa`) | 1 | Main SGP workload identity SA | | Service Account (`sgp--node-pool-creator`) | 1 | Node pool creation SA | | Service Account (`sgp-tf-lp-`) | 1 | Terraform execution SA (projectsetup) | | Workload Identity Pool | 1 | GKE pod → GCP SA binding | | IAP Tunnel (Bastion) | 0–1 | Private cluster access | *** ### Artifact Resources (projectsetup phase) | Resource | Count | Purpose | | - | - | - | | Artifact Registry (Docker) | 1 | SGP container images | | Artifact Registry (Helm) | 1 | SGP Helm charts | Both registries are named `sgp--docker-repository` and `sgp--helm-repository`. *** ### Monitoring Resources | Resource | Count | Purpose | | - | - | - | | Cloud Logging | 1 | GKE system components, API server, workloads | | VPC Flow Logs | 0–1 | Network traffic sampling (optional) | | GKE Vulnerability Scanning | 1 | Basic vulnerability mode enabled by default | *** ## Network Architecture ### Address Space ``` VPC: sgp--network └── Subnet: sgp--network-kubernetes-subnet ├── Primary range: 10.0.0.0/16 (65,536 IPs — GKE nodes) ├── Secondary range: 10.2.0.0/16 (65,536 IPs — Kubernetes services) └── Secondary range: 10.4.0.0/16 (65,536 IPs — Kubernetes pods) Cloud SQL VPC Peering: └── Reserved range: /16 block (managed by service networking) GKE Control Plane (private): └── Master CIDR: 10.5.0.0/28 (configurable via private_gke_master_ipv4_cidr_block) ``` ### Traffic Flow **Ingress (External):** ``` Internet → Global Load Balancer (static IP) → Istio Ingress → Pod (TLS terminated at load balancer; SSL policy enforces TLS 1.2+) ``` **Database Access:** ``` GKE Pod → VPC Peering → Cloud SQL (private IP) (SSL enforced; no public IP on Cloud SQL instances) ``` **Google APIs (offline mode):** ``` GKE Pod → Private Google Access → Google APIs (199.36.153.4/30) (Dedicated route; no default internet gateway route created) ``` **Bastion Access:** ``` Engineer → IAP Tunnel → Bastion VM (e2-micro) → GKE API (private endpoint) ``` *** ## GKE Configuration ### Cluster Features | Feature | Value | | - | - | | Datapath provider | `ADVANCED_DATAPATH` (eBPF-based) | | IP stack | Dual-stack IPv4/IPv6 | | Workload Identity | Enabled (`.svc.id.goog`) | | Secret Manager integration | Enabled | | Vulnerability scanning | Basic mode | | Private nodes | Enabled (when `offline_mode = true`) | | Private endpoint | Configurable (`enable_gke_private_endpoint`) | | Master authorized networks | Configurable per deployment | | DNS endpoint | Enabled (allows external cluster DNS access) | ### Workload Identity GKE pods authenticate to GCP services using Workload Identity rather than node-level service account keys. Kubernetes service accounts are bound to GCP service accounts via the workload identity pool: ``` .svc.id.goog[/] → GCP Service Account → Secret Manager, GCS, Artifact Registry, etc. ``` Key bindings provisioned by Terraform: | Kubernetes Identity | GCP Role | | - | - | | `sgp-system-manager` (system-manager namespace) | `secretmanager.viewer`, `secretmanager.secretAccessor` | | `sgp-system-manager-pre-install` | `secretmanager.viewer`, `secretmanager.secretAccessor` | | `egp-api-backend` (egp namespace) | `secretmanager.secretAccessor`, GCS access via SA impersonation | | `egp-api-backend-db-setup` | `secretmanager.secretAccessor` | | `agents` (agents namespace) | `secretmanager.secretAccessor` | *** ## Secret Manager Secrets Key secrets provisioned by Terraform: | Secret Name | Contents | | - | - | | `-system-manager-config` | System Manager runtime configuration (URLs, workspace ID, cloud provider info) | | `-system-manager-desired-state` | Desired state (pack list). Terraform creates it empty; on first start, System Manager fills it from `desired_state.json` and manages it afterwards | | `-identity-service-saml-secret` | SAML IdP configuration | | `-identity-service-oidc-secret` | OIDC IdP configuration | | `terraform-service-account-key-secret-2` | Terraform SA private key created by `projectsetup/`. Your deployment reads it as provider credentials after you set this name in [Step 1 of the deployment guide](/docs/infrastructure/gcp/gcp-sgp-deployment#step-1-bootstrap-the-project) | The `secret_name_prefix` variable (typically `sgp-`) scopes secrets for deployments in shared GCP projects. *** ## Customer Managed Encryption Keys (CMEK) When `useCustomerManagedEncryptionKey = true`, Terraform provisions a Cloud KMS key ring and symmetric encryption key: | Resource | Name | | - | - | | Key Ring | `sgp--key-ring` | | Crypto Key | `sgp--crypto-key` | The key is applied to: * Cloud SQL instances (both main and Temporal) * GCS buckets CMEK is recommended for production deployments to maintain cryptographic control over data at rest. *** ## Optional Capabilities Each optional capability provisions dedicated infrastructure: | Capability | Variable | Additional Resources | | - | - | - | | Agentex | `deployAgentex` (default `true`) | Firestore, dedicated service account, GCS bucket | | Workflows | `deployCompass` | Cloud SQL database, GCS bucket, service account | | Dex (Document Understanding) | `enable_dex` | Cloud SQL (`db-custom-4-15360`), GCS bucket, service account | | Reducto | `enable_reducto` | Cloud SQL (`db-custom-2-7680`), GCS bucket, Vision API key, service account | | Model Engine | `deployModelEngine` (module default `true`; `deployments/prototype` sets `false`) | Vertex AI, additional node pools | | Monitoring | `enable_monitoring` | GCS bucket, IAM bindings | | Cloud Build | `enable_cloud_build` | GCS bucket, Artifact Registry, service accounts | | Vertex AI Search | `vertex_ai_indices` | Vector Search indices and endpoints | | LiveKit | `provision_livekit_infrastructure` | GCS bucket (audio), IAM bindings | # GCP SGP Deployment Source: https://docs.gp.scale.com/docs/infrastructure/gcp/gcp-sgp-deployment End-to-end guide to deploying SGP in your GCP project, together with Scale engineering. ## Overview This guide covers deploying SGP in a GCP project using the SGP GCP Terraform modules. SGP GCP infrastructure is defined by terraform modules managed by Scale. > [!NOTE] > > SGP is deployed jointly with Scale engineering. Scale provides everything under "The following from Scale" in Prerequisites and stays involved for upgrades and ongoing operation. Contact your Scale team before you start. GCP deployment separates the bootstrap identity from the Terraform execution identity: * **`projectsetup/`** — Run once with privileged GCP credentials. Creates a dedicated Terraform service account with broad project administration roles, enables required GCP APIs, and provisions Artifact Registries for SGP images and Helm charts. * **`deployments//`** — Run with the service account created above. Provisions the GKE cluster, Cloud SQL, networking, and all SGP infrastructure. The execution identity still has broad privileges, including `roles/compute.admin`, `roles/container.admin`, `roles/cloudsql.admin`, `roles/secretmanager.admin`, `roles/resourcemanager.projectIamAdmin`, `roles/iam.serviceAccountKeyAdmin`, `roles/artifactregistry.admin`, and `roles/storage.admin`. Review the complete role list in the provided `projectsetup/main.tf` before granting access. ## Prerequisites * Access to a GCP project with permissions to manage IAM, APIs, and compute resources * The following tools installed: * [gcloud CLI](https://cloud.google.com/sdk/docs/install) (`gcloud auth login` completed) * [Terraform](https://www.terraform.io/) (`>= 1.10.0`) * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) * [helm](https://helm.sh/docs/intro/install/) (for cluster bootstrap) * [crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane) or [Docker](https://docs.docker.com/engine/install/) (for copying images) * [sgpctl](https://github.com/scaleapi/sgpctl) (cli tool to execute system manager related commands) * The following from Scale: * The SGP GCP Infrastructure modules (`gcp/`) * A `workspace_id` (8-digit number) and `registration_secret` unique to your deployment * A new application configured in your identity provider to authenticate to the SGP platform (SAML or OIDC) (optional) * A custom domain for your deployment (optional) ## Installation ### Step 1: Bootstrap the Project Navigate to the `projectsetup/` directory. This phase creates the Terraform service account and Artifact Registries. Edit the `locals` block at the top of `projectsetup/main.tf`: ```hcl theme={null} locals { project = "" region = "" # e.g. "us-east1" zone = "" # e.g. "us-east1-a" service_account_name = "-sa" # Suffix for the service account name workspaceID = "" # 8-digit number from Scale, e.g. "90000001" } ``` Then initialize and apply: ```bash theme={null} cd projectsetup terraform init terraform apply ``` This creates: * A Terraform service account (`sgp-tf-lp-`) with the IAM roles needed to provision SGP infrastructure * A service account key stored as `terraform-service-account-key-secret-2` in Secret Manager * Docker and Helm Artifact Registries (`sgp--docker-repository`, `sgp--helm-repository`) * All required GCP APIs enabled on the project The Terraform outputs include the `service_account_email` of the newly created service account, which is useful for auditing. The project bootstrap stores the key in `terraform-service-account-key-secret-2`. Before initializing your copied deployment, set the `service-account-private-key` data source to read that secret: ```hcl theme={null} data "google_secret_manager_secret" "service-account-private-key" { secret_id = "terraform-service-account-key-secret-2" provider = google.google-service-account-private-key-secret-pull-provider } ``` Terraform then reads the key from Secret Manager, so you don't need a local key file. ### Step 2: Copy SGP Images and Helm Charts *This step is only required if your deployment is configured to block internet access from the cluster (`offline_mode` in step 3).* Before provisioning the main infrastructure, copy SGP's Docker images and Helm charts from Scale's registry into your GCP Artifact Registry. **Download the manifest from Scale's control plane**: ```bash theme={null} sgpctl manifest download \ --control-plane-id "" \ --registration-token "" \ --version "" \ --channel \ --output ".manifest.yaml" ``` Note: sgpctl manifest download requires internet access to [https://control.egp.scale.com](https://control.egp.scale.com), so it must be run from a connected machine. Users with air-gapped operator workstations should run it from their laptop and SCP the output to wherever they'll run the mirror commands. **Authenticate to source + destination registries**: ```bash theme={null} # Source: Scale's ECR (credentials from Scale engineer) crane auth login 022465994601.dkr.ecr.us-west-2.amazonaws.com \ --username AWS \ --password-stdin # Destination: customer's GCP Artifact Registry (uses gcloud creds) gcloud auth configure-docker "-docker.pkg.dev" ``` **Mirror images and charts**: ```bash theme={null} # Container images sgpctl mirror-manifest \ --manifest-file ".manifest.yaml" \ --dest-docker-registry "-docker.pkg.dev//sgp--docker-repository" # Helm charts sgpctl mirror-helm-charts \ --manifest-file ".manifest.yaml" \ --dest-helm-repo "oci://-docker.pkg.dev//sgp--helm-repository" ``` **Verify the mirror**: ```bash theme={null} gcloud artifacts docker images list \ "-docker.pkg.dev//sgp--docker-repository" \ --include-tags --limit=20 gcloud artifacts docker images list \ "-docker.pkg.dev//sgp--helm-repository" \ --include-tags --limit=20 ``` ### Step 3: Configure the Deployment Copy the `deployments/prototype/` directory and rename it for your deployment: ```bash theme={null} cp -r deployments/prototype "deployments/" cd "deployments/" ``` Edit the `locals` block at the top of `main.tf`. After you set the secret name in Step 1, the deployment reads its Terraform service account credentials from Secret Manager; no key file is required on disk. ```hcl theme={null} locals { project = "" region = "" # e.g. "us-east1" zone = "" # e.g. "us-east1-b" # ── Deployment identity ──────────────────────────────────────────────── workspaceID = "" # 8-digit number from Scale registrationSecret = "" # From Scale sgp_account_id = "" # Or the 24-character SGP account ID from Scale deploymentURL = ".workspace.egp.scale.com" # Or your custom domain # ── Repositories ────────────────────────────────────────────────────── # baseRepository: used by workloads running inside the cluster to pull images # publicBaseRepository: used for initial image pulls during bootstrap (before private DNS resolves) # Both point to the same registry when using a private Artifact Registry. baseRepository = "-docker.pkg.dev//sgp--docker-repository" publicBaseRepository = "-docker.pkg.dev//sgp--docker-repository" # ── System Manager ──────────────────────────────────────────────────── systemManagerImageTag = "" # From Scale; must match manifest.yaml offline_mode = true # true when using your own Artifact Registry # ── Auth ────────────────────────────────────────────────────────────── authType = "SAML" # "SAML" or "OIDC" # ── Bootstrap ───────────────────────────────────────────────────────── # deploy_system_manager=false: Terraform creates infra only; run manual-helm-install.sh to bootstrap # deploy_system_manager=true: Terraform also installs System Manager into the cluster deploy_system_manager = false # ── DNS ─────────────────────────────────────────────────────────────── createDNSRecords = false # Set true if Terraform should manage Cloud DNS records # ── Encryption ──────────────────────────────────────────────────────── useCustomerManagedEncryptionKey = false # Set true for CMEK (recommended for production) aes256EncryptionKey = "" # Required when CMEK is off: 32 characters, from `openssl rand -hex 16` # ── Networking ──────────────────────────────────────────────────────── gke_config = { private_endpoint = true master_authorized_networks = [ { cidr_block = "10.0.0.0/16", display_name = "cluster vpc primary range" }, { cidr_block = "10.2.0.0/16", display_name = "cluster vpc services range" }, { cidr_block = "10.4.0.0/16", display_name = "cluster vpc pod range" }, ] } # ── Bastion ─────────────────────────────────────────────────────────── security_compliance = { # ... labels, logging and firewall settings from the prototype bastion_enabled = true bastion_iap_members = ["user:"] # Accounts that reach the private cluster through the bastion } } ``` Don't change `aes256EncryptionKey` on a running deployment: data encrypted with the old key becomes unreadable. **If using SAML,** place your IdP's x509 certificate (without BEGIN/END lines) as `x509.cer` in the deployment directory. Update the `samlConfigSecret` in the `module "sgp"` block: ```hcl theme={null} module "sgp" { source = "../../modules/sgp/" samlConfigSecret = jsonencode({ "id" = local.workspaceID "samlConfiguration" = { "entityId" = "https://auth.${local.deploymentURL}" "x509Cert" = file("${path.module}/x509.cer") "ssoUrl" = "" "attributeMappings" = { "email" = "" "firstName" = "" "lastName" = "" } } }) # ... other variables } ``` **If using OIDC,** update `oidcConfigSecret` instead: ```hcl theme={null} oidcConfigSecret = jsonencode({ "id" = local.workspaceID "clientId" = "" "clientSecret" = "" "issuer" = "" "authorizationUrl" = "" "tokenUrl" = "" "userInfoUrl" = "" }) ``` #### Desired State The `desired_state.json` file in your deployment directory defines which SGP packs System Manager installs. `terraform apply` copies it into the System Manager configuration, and System Manager reads it only on first install, while its desired state is empty. Finish it, including the certificate setting from Step 4, before you run Terraform in Step 5. After the first install, editing the local file has no effect; update the live desired state in the System Manager GUI, API, or Secret Manager instead, then install the affected packs. Update the file to reference your GCP Artifact Registry: ```json theme={null} { "version": "0.1", "packs": [ { "name": "flux" }, { "name": "sgp-helm-repository", "properties": { "helm-repo": { "url": "oci://-docker.pkg.dev//sgp--helm-repository" } } }, { "name": "istio" }, { "name": "sgp-base", "properties": { "helm-base": { "value_overrides": { "refreshRegcred": { "enabled": false }, "gcp": { "presharedCertificates": ["sgp--ssl-certificate"] } } } } }, { "name": "spicedb" }, { "name": "identity-service" }, { "name": "temporalf" }, { "name": "sgp-apps" } ] } ``` The `gcp` block above uses a preshared certificate. [Step 4](#step-4-configure-ssl-certificates) covers that option and the other two. ### Step 4: Configure SSL Certificates The options are listed in order of preference. Choose one before you run Terraform in Step 5: | Option | `main.tf` | `gcp` block of the `sgp-base` pack in `desired_state.json` | Certificate | | - | - | - | - | | 1. Google Certificate Manager (recommended) | `enable_certificate_manager = true` and `enable_gateway_api = true` | Remove it | A wildcard certificate for all subdomains, validated through DNS | | 2. Google-managed certificate | Neither flag | Remove it | Per-subdomain certificates that Google renews, for ``, `api.`, `auth.` and `app.` only. DNS must resolve to the load balancer first, and wildcards aren't supported. They don't cover Agentex's `chat.` and `agentex.` hosts, so use Option 1 when Agentex is on | | 3. Preshared certificate | Neither flag | `presharedCertificates`, naming your certificate | A certificate you upload and renew. Use it when DNS isn't publicly resolvable or Certificate Manager isn't available | For each option, edit the existing `sgp-base` entry and preserve its other properties. When you switch certificate options, replace the existing `presharedCertificates` list instead of adding to it. Before switching to a managed certificate, make sure every `presharedCertificates` list in your configuration is empty. #### Option 1: Google Certificate Manager (Recommended) Set both flags in `main.tf`, and add an output for the DNS authorization record, which the prototype doesn't export. Terraform passes the certificate map to the `sgp-base` pack: ```hcl theme={null} module "sgp" { source = "../../modules/sgp/" enable_certificate_manager = true enable_gateway_api = true # ... } output "certificate_manager_dns_auth_record" { value = module.sgp.certificate_manager_dns_auth_record } ``` After `terraform apply` in Step 5, run `terraform output certificate_manager_dns_auth_record` and add that CNAME to your DNS zone for domain validation. The certificate typically provisions within 10–60 minutes of the DNS record being in place. Terraform passes the Gateway API setting and certificate-map name to the `sgp-base` pack as defaults, and desired-state `value_overrides` take precedence over them. If you set overrides explicitly, merge the following `gcp` values into the existing `sgp-base` entry, preserving its other properties, and install that pack. A certificate map requires the Gateway template: ```json theme={null} { "name": "sgp-base", "properties": { "helm-base": { "value_overrides": { "gcp": { "gatewayApi": { "enabled": true }, "certificateMapName": "sgp--cert-map", "presharedCertificates": [] } } } } } ``` #### Option 2: Google Managed Certificate Google provisions the certificates after DNS resolves to the load balancer IP in Step 7. This is the traditional GCE Ingress default when `gcp.gatewayApi.enabled` is `false` and `presharedCertificates` is empty. To switch from another certificate option, merge these values into the existing `sgp-base` entry and install the pack: ```json theme={null} { "gcp": { "gatewayApi": { "enabled": false }, "certificateMapName": "", "presharedCertificates": [] } } ``` Place this object under `properties.helm-base.value_overrides`, preserving the entry's other properties. These values override the Gateway settings that Terraform provides. Make sure every `presharedCertificates` list in your configuration is empty. Omitting a key does not clear a Terraform-provided default. #### Option 3: Preshared Certificate Upload the certificate: ```bash theme={null} gcloud compute ssl-certificates create "sgp--ssl-certificate" \ --certificate=fullchain1.pem \ --private-key=privkey1.pem \ --project "" ``` For traditional GCE Ingress, merge these values into the existing `sgp-base` entry, keeping its other properties. Before the first install, edit `desired_state.json`; after it, edit the live desired state. These values disable Gateway API and select the preshared certificate: ```json theme={null} { "name": "sgp-base", "properties": { "helm-base": { "value_overrides": { "gcp": { "gatewayApi": { "enabled": false }, "certificateMapName": "", "presharedCertificates": ["sgp--ssl-certificate"] } } } } } ``` > [!WARNING] > > Preshared certificates must be renewed manually. Let's Encrypt certificates expire after 90 days. Edits to `desired_state.json` after the first install have no effect. To change the SSL configuration then: * Edit the `sgp-base` pack's value overrides in the System Manager GUI or API. An API update replaces the pack's whole `helm-base` object, so send all of its value overrides, including `refreshRegcred`. * To move to Option 1, also set its two flags in `main.tf` and run `terraform apply`. Then explicitly install `sgp-base` in the System Manager GUI to apply the change. With the default startup configuration, restarting System Manager also applies the live state by reinstalling all packs: ```bash theme={null} kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager ``` ### Step 5: Provision Infrastructure via Terraform From the deployment directory, initialize and apply: ```bash theme={null} terraform init -upgrade terraform plan -out=tfplan # Inspect the plan before proceeding terraform show tfplan terraform apply tfplan ``` *This step can take 30–60 minutes due to GKE cluster provisioning and Cloud SQL setup.* After apply completes, connect to the cluster: ```bash theme={null} gcloud container clusters get-credentials "sgp--kubernetes-cluster" \ --region "" \ --project "" ``` > [!NOTE] > > If `gke_config.private_endpoint = true`, the cluster API server is only accessible from within the VPC. Use the bastion host (via IAP, as one of the `bastion_iap_members` from Step 3) or a network-connected runner to access it. ### Step 6: Bootstrap the Cluster [SGP System Manager](/docs/infrastructure/system-manager) orchestrates SGP service deployment. If you set `deploy_system_manager = false` in Step 3, bootstrap it manually using the provided script. Edit `manual-helm-install.sh` in your deployment directory with the correct values: ```bash theme={null} PROJECT_ID="" PROJECT_NUMBER="" # From: gcloud projects describe "" --format='value(projectNumber)' BASE_REPOSITORY="-docker.pkg.dev//sgp--docker-repository" SYSTEM_MANAGER_IMAGE_TAG="" OFFLINE_MODE=true # Obtain from Cloud SQL instance → "Connect to this instance" in the console, or from Terraform state SPICEDB_IP_ADDRESS="" SPICEDB_PASSWORD="" TEMPORAL_DB_PASSWORD="" ``` Then run the bootstrap script: ```bash theme={null} chmod +x manual-helm-install.sh ./manual-helm-install.sh ``` This installs System Manager into the cluster. On its first start, System Manager deploys the packs listed in `desired_state.json`. Startup installation is enabled by default, so each startup installs all packs in the live desired state. If startup installation is disabled, install the packs in the System Manager GUI. See [System Manager configuration](/docs/infrastructure/system-manager#configuration) for the chart-version requirement. ### Step 7: Configure DNS Get the external IP address of the load balancer: ```bash theme={null} # From inside the cluster: kubectl get ingress external-ingress --namespace istio-ingress -o jsonpath='{.status.loadBalancer.ingress[0].ip}' # With enable_gateway_api = true, the address is on the Gateway instead: kubectl get gateway external-gateway --namespace istio-ingress -o jsonpath='{.status.addresses[0].value}' # Or from gcloud (the global address is created by Terraform): gcloud compute addresses describe "sgp--external-ip" \ --global \ --project "" \ --format="value(address)" ``` Create DNS A records in your DNS provider pointing to this IP for: * `` (apex) * `api.` * `auth.` * `app.` If `createDNSRecords = true` in your `main.tf` locals, Terraform manages a Cloud DNS zone and creates these records automatically. Retrieve the name servers from the Terraform output: ```bash theme={null} terraform output name_servers ``` Then delegate the zone by configuring these as NS records at your domain registrar. ### Step 8: Verify the Deployment Wait for all services to be ready: ```bash theme={null} kubectl get helmreleases -A # All should show Ready=True kubectl get pods -A # All should be Running or Completed ``` System Manager renders installed packs into HelmReleases, which Flux then reconciles. After you change the desired state, install the affected packs to apply it. If a HelmRelease shows `Ready=False`, check its events: ```bash theme={null} kubectl describe helmrelease "" -n "" ``` ## Accessing the Platform Once all HelmReleases are ready and DNS resolves correctly, navigate to `https://.workspace.egp.scale.com` (or your custom domain) and authenticate via your configured identity provider. # System Manager Source: https://docs.gp.scale.com/docs/infrastructure/system-manager Overview of the SGP System Manager deployment orchestrator ## Overview System Manager is a Kubernetes operator used to deploy and manage the SGP platform. It is responsible for deploying the SGP platform services and agents to an existing Kubernetes cluster. > [!NOTE] > > System Manager configuration changes and SGP upgrades are made together with Scale engineering. Contact your Scale team before changing the configuration below. You can deploy your own agents without Scale; see [Cloud Build and Deploy](/docs/v5/agents/agentex/cloud-build-and-deploy/overview). ## Installation System Manager is installed as a Helm chart into the Kubernetes cluster during the deployment of the SGP platform. See your cloud provider's corresponding deployment guide for more information. ## Configuration System Manager is configured via the `system-manager-config` secret in the cloud provider's secret manager: * AWS: `scale-egp-/system-manager-config` in Secrets Manager * GCP: `system-manager-config` in Secret Manager, starting with the Terraform `secret_name_prefix` when one is set, for example `sgp--system-manager-config` * Azure: `system-manager-config` in the deployment's Key Vault Update the secret directly or use the System Manager GUI. After the configuration is loaded, explicitly install the affected packs to render and apply their resources; saving a secret alone does not deploy changes. Startup installation is enabled by default: a restart reloads startup settings and reinstalls all packs in the desired state. To disable startup installation, set `env.INIT_ON_STARTUP` to `""` in the System Manager Helm chart values (chart version `27.18.2` or later). Any other value, including `"false"` or `"0"`, leaves it enabled. To restart System Manager: ```bash theme={null} kubectl rollout restart deployment sgp-system-manager -n sgp-system-manager ``` The following AWS example selects SAML authentication. For OIDC, set `authType` to `OIDC`, `samlSetupEnabled` to `false`, and `oidcSetupEnabled` to `true`. The `aws` block is only present for AWS deployments; GCP and Azure deployments use equivalent `gcp` and `azure` blocks instead. `baseRepository` is the container registry that SGP images are pulled from, as a prefix without a URL scheme; configure the Helm chart URL separately in the `sgp-helm-repository` pack. On AWS and Azure, Terraform sets it to Scale's registry, `022465994601.dkr.ecr.us-west-2.amazonaws.com`, unless you override it. The GCP guide sets it to your Artifact Registry, `-docker.pkg.dev//sgp--docker-repository`. `deploymentURL` is a hostname without `https://`. ```json theme={null} { "cloudProvider": "aws", "baseRepository": "", "samlSetupEnabled": true, "oidcSetupEnabled": false, "deploymentURL": ".workspace.egp.scale.com", "workspaceId": "", "authType": "SAML", "baseDomain": "", "train_tenant_prefix": "", "deployAgentex": true, "deploySae": true, "aws": { "accountId": "", "region": "", "prefix": "", "modelEngineS3Bucket": "scale-egp--ml", "sqsQueuePolicyTemplate": "", "sqsQueueTagTemplate": "", "clusterName": "", "karpenterIrsaArn": "", "targetGroupArn": "", "nodeSubnets": ["", ""], "nodeSecurityGroup": "", "postgresHostTemporal": "", "compassBucketName": "", "compassMongoHost": "", "compassRedisHost": "", "reductoBucketName": "", "reductoDatabaseUrl": "", "reductoIrsaRoleArn": "", "reductoAzureVisionEndpoint": "", "reductoAzureVisionKey": "", "codeBuildProjectName": "", "codeBuildS3Bucket": "", "codeBuildRegistryUrl": "", "codeBuildServiceRoleArn": "", "cloudDeployEnabled": true, "dex": { "irsaRoleArn": "", "prefix": "duc-" }, "train": { "irsaRoleArn": "", "sagemakerExecutionRoleArn": "", "sagemakerSecurityGroupId": "", "databaseHost": "", "dataBucket": "", "checkpointsBucket": "", "outputBucket": "", "stagingBucket": "" }, "registryProxy": { "irsaRoleArn": "", "ecrRepositoryUrl": "", "jwtSecret": "", "workspaceConfig": "" } }, "frontDoorSSLCertB64": "", "frontDoorSSLKeyB64": "", "initialDesiredState": "", "datadog": { "enabled": true, "env": "", "clusterName": "", "secretName": "", "irsaRoleArn": "" } } ``` ## Architecture System Manager runs as a deployment in the `sgp-system-manager` namespace. Its GUI is accessible on port 8000. ```bash theme={null} kubectl port-forward deployment/sgp-system-manager -n sgp-system-manager 8000:8000 ``` Then navigate to `http://localhost:8000` in your browser to access the GUI. ```mermaid theme={null} graph TD ADMIN(["Platform Admin"]) subgraph CLUSTER["Kubernetes Cluster"] subgraph SM_NS["sgp-system-manager namespace"] GUI["Web GUI
:8000"] API["REST API
/api/v3"] WATCHER["Secret Watcher
change notifications"] INSTALLER["Pack Installer"] end subgraph RESOURCES["Cluster Resources"] NS["Namespaces"] SEC["Secrets + Pull Secrets"] HR["HelmRelease CRDs"] end subgraph FLUX_NS["flux-system namespace"] FLUX["Flux Helm Controller"] end PODS["SGP Services
(Helm-managed pods)"] end subgraph CLOUD["Cloud Provider"] SM_SECRET["Secret Manager
desired-state · config"] HELM_REPO["Helm Repository
OCI Artifact Registry"] end ADMIN -->|"browser"| GUI GUI --> API API -->|"read / write"| SM_SECRET WATCHER -->|"polls every 3s when configured"| SM_SECRET API --> INSTALLER INSTALLER --> NS INSTALLER --> SEC INSTALLER --> HR FLUX -->|"watches"| HR FLUX -->|"pull chart"| HELM_REPO FLUX -->|"deploy"| PODS ``` ### Packs System Manager organizes services into "packs". Each pack is a collection of resources that are deployed together. Generally packs are composed of [FluxCD HelmRelease](https://fluxcd.io/flux/components/helm/helmreleases/) as well as other resources necessary to support a particular service. When a pack is installed, System Manager renders its resource templates and writes the resulting Kubernetes resources — namespaces, secrets, and FluxCD HelmRelease CRDs — to the cluster. FluxCD then picks up the HelmRelease CRDs and handles pulling and deploying the Helm charts. ### FluxCD System Manager offloads resource reconciliation to FluxCD. FluxCD is a tool that allows you to manage the lifecycle of your Kubernetes resources. Flux reconciles installed HelmRelease changes and manages chart installation and upgrades. System Manager configures workload drift detection in `warn` mode: direct workload edits are reported rather than automatically corrected, and Deployment replica counts are ignored. For more information on FluxCD, see the [FluxCD documentation](https://fluxcd.io/flux/components/helm/helmreleases/). ### Desired state The collection of packs that System Manager will deploy is defined in the `system-manager-desired-state` secret, stored with the same prefix as `system-manager-config` in the cloud provider's secret manager. To modify the desired state, you can either update the secret directly or use the System Manager GUI, then explicitly install the affected packs via the System Manager GUI. Saving the desired state alone does not apply changes. When startup installation is enabled, a restart reinstalls all packs in the desired state, as described in [Configuration](#configuration). A sample desired state is shown below: ```json theme={null} { "version": "0.1", "packs": [ { "name": "flux" }, { "name": "sgp-helm-repository" }, { "name": "istio" }, { "name": "sgp-base" }, { "name": "spicedb" }, { "name": "identity-service" }, { "name": "temporalf" }, { "name": "sgp-apps" } ] } ``` # Get Project Data Source Source: https://docs.gp.scale.com/reference/dex/data-sources/get-project-data-source https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/data-sources/{data_source_id} # Link Project Data Source Source: https://docs.gp.scale.com/reference/dex/data-sources/link-project-data-source https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/data-sources Link an existing Data Sources connector and create DEX webhook subscription. # List Project Data Sources Source: https://docs.gp.scale.com/reference/dex/data-sources/list-project-data-sources https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/data-sources # Resync Data Source Deltas Source: https://docs.gp.scale.com/reference/dex/data-sources/resync-data-source-deltas https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/data-sources/{data_source_id}/resync # Unlink Project Data Source Source: https://docs.gp.scale.com/reference/dex/data-sources/unlink-project-data-source https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/data-sources/{data_source_id} Unlink a data source from a project. With ``delete_connector=true``, also delete the underlying connector on the Data Sources service — but only when no other project still links it (so a shared connector is never pulled out from under another project). # Create Extract Job Source: https://docs.gp.scale.com/reference/dex/extract/create-extract-job https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/extract Create a new extract job. This endpoint creates an extraction job that runs **single-pass** LLM extraction from a parse result or vector store search context. For agentic (decompose / ReAct) extraction, use ``POST …/research`` instead; requests with ``use_agentic=True`` are rejected. Args: request: Extract job request containing source_id and parameters Returns: JobEntity: The created job entity Raises: HTTPException: If source not found or workflow startup fails # Get Extraction Source: https://docs.gp.scale.com/reference/dex/extract/get-extraction https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/extract/{extraction_id}/result Get an extraction by ID (job ID) within the specified project. Args: project_id: Project ID that the extraction belongs to extraction_id: Extraction ID (job ID) to retrieve Returns: ExtractionEntity: The extraction details Raises: HTTPException: If project or extraction not found, or extraction doesn't belong to project # List Extractions Source: https://docs.gp.scale.com/reference/dex/extract/list-extractions https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/extract/results List all extractions for a project with token-based pagination and filtering. Only SUCCEEDED extraction jobs are returned. Filter by ids, created_at, completed_at, or source_id. Args: project_id: Project ID that the extractions belong to filter_params: Filter parameters (ids, created_at, completed_at, source_id) pagination_params: Pagination parameters (page_size, continuation_token, sort_by, sort_order) Returns: PaginatedItems or list of ExtractionEntity instances Raises: HTTPException: If project not found # Delete File Source: https://docs.gp.scale.com/reference/dex/files/delete-file https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/files/{file_id} Delete a file from both storage and the database. This also cleans up associated parse result artifacts from storage. Parse results are cascade-deleted from the database when the file is removed. # Get By Id Source: https://docs.gp.scale.com/reference/dex/files/get-by-id https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/files/{file_id} Fetch a file by id # Get File Download Url Source: https://docs.gp.scale.com/reference/dex/files/get-file-download-url https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/files/{file_id}/download-url Get a secure, time-limited URL to download the content of a previously uploaded file. This is useful when the client needs to pass a URL to another service or download the file through a browser. # List Files Source: https://docs.gp.scale.com/reference/dex/files/list-files https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/files List all files for a project with optional pagination and filtering. Args: project_id: Project ID that the files belong to filter_params: Optional filter parameters (ids, created_at_start, created_at_end) pagination_params: Optional pagination parameters (page_size, continuation_token, sort_by, sort_order) # Stream File Content Source: https://docs.gp.scale.com/reference/dex/files/stream-file-content https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/files/{file_id}/stream Stream the raw byte content of a file directly. This endpoint provides direct access to the raw byte content of a file, making it suitable for streaming applications where the client might not be able to access the blob storage URL directly. # Upload File Source: https://docs.gp.scale.com/reference/dex/files/upload-file https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/files Upload a file and store it in the configured storage backend. This endpoint provides the simplest method for uploading files up to a service-defined limit (e.g., 100 MB). # Batch Delete Files From Index Source: https://docs.gp.scale.com/reference/dex/indexes/batch-delete-files-from-index https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/indexes/{index_id}/files Delete a batch of files from an existing index. Async via Temporal. # Batch Upsert Files To Index Source: https://docs.gp.scale.com/reference/dex/indexes/batch-upsert-files-to-index https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/indexes/{index_id}/files Add or update a batch of files in an existing index. Async via Temporal. # Create Index Source: https://docs.gp.scale.com/reference/dex/indexes/create-index https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/indexes Create an empty index (synchronous). Allocates the DB record, writes an empty serialized engine to blob storage with the provided config, and marks the index ``ready``. Files are added later via ``POST /indexes/{index_id}/files`` (which kicks off a batch-mutation workflow). # Delete Index Source: https://docs.gp.scale.com/reference/dex/indexes/delete-index https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/indexes/{index_id} Delete an index and its blob storage. # Get Index Source: https://docs.gp.scale.com/reference/dex/indexes/get-index https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/indexes/{index_id} Get a specific index with nodes hydrated from blob storage. # List Indexes Source: https://docs.gp.scale.com/reference/dex/indexes/list-indexes https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/indexes List all indexes for a project. # Navigate Index Source: https://docs.gp.scale.com/reference/dex/indexes/navigate-index https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/indexes/{index_id}/navigate Navigate the index hierarchy. # Search Index Source: https://docs.gp.scale.com/reference/dex/indexes/search-index https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/indexes/{index_id}/search Search the index for relevant document IDs. # Cancel Job Source: https://docs.gp.scale.com/reference/dex/jobs/cancel-job https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/jobs/{job_id}/cancel Cancel a running or pending job. Idempotent: cancelling an already-cancelled job returns 200. For RUNNING jobs, returns 202 since the workflow handles cancellation asynchronously. # Get Job Source: https://docs.gp.scale.com/reference/dex/jobs/get-job https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/jobs/{job_id} Get a job by ID within the specified project. Args: project_id: Project ID that the job belongs to job_id: Job ID to retrieve Returns: JobEntity: The job details Raises: HTTPException: If project or job not found, or job doesn't belong to project # List Jobs Source: https://docs.gp.scale.com/reference/dex/jobs/list-jobs https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/jobs List all jobs for a project with token-based pagination and filtering. Args: project_id: Project ID that the jobs belong to pagination_params: Pagination parameters (page_size, continuation_token, sort_by, sort_order) filter_params: Filter parameters (ids, created_at, completed_at, status, operation) Returns: PaginatedItems or list of JobEntity instances Raises: HTTPException: If project not found # Stream Job Updates Source: https://docs.gp.scale.com/reference/dex/jobs/stream-job-updates https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/jobs/{job_id}/updates Stream job trace spans as Server-Sent Events. Emits three event types: - ``trace_update``: a new or updated ``TraceSpan`` (JSON-serialised span) - ``job_status``: emitted when the database job status changes - ``done``: terminal event; the stream closes after this Clients should reconnect on transient network errors. # Create Batch Parse Job Source: https://docs.gp.scale.com/reference/dex/parse/create-batch-parse-job https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/parse/batch Create a batch parse job that processes multiple documents asynchronously. If file_ids are provided, only those files are parsed. If file_ids are omitted, all uploaded files in the project are parsed. Returns a single batch job that can be polled for progress and results. # Create Chunk Job Source: https://docs.gp.scale.com/reference/dex/parse/create-chunk-job https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/parse/{parse_result_id}/rechunk Rechunk a parse result with a different chunking strategy. This endpoint initiates rechunking of an existing parse result using the specified chunking strategy. The operation is performed asynchronously via Temporal workflows. Args: project_id: Project ID that the parse result belongs to parse_result_id: Parse result ID to rechunk request: Rechunk parse result request with chunking strategy and options Returns: Job entity that can be used to track progress Raises: HTTPException: If the parse result is not found or other errors occur # Create Parse Job Source: https://docs.gp.scale.com/reference/dex/parse/create-parse-job https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/parse Create a parse job that will asynchronously process a document. This endpoint initiates document parsing using the specified engine (default: Reducto). The operation is performed asynchronously via Temporal workflows. Args: request: Parse job request with source document ID and parameters Returns: Job entity that can be used to track progress Raises: HTTPException: If the source document is not found or other errors occur # Get Parse Result Source: https://docs.gp.scale.com/reference/dex/parse/get-parse-result https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/parse/{parse_result_id}/result Get a parse result by ID within the specified project. Args: project_id: Project ID that the parse result belongs to parse_result_id: Parse result ID to retrieve Returns: ParseResultEntity: The parse result details Raises: HTTPException: If project or parse result not found, or result doesn't belong to project # List Parse Results Source: https://docs.gp.scale.com/reference/dex/parse/list-parse-results https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/parse/results List all parse results for a project with token-based pagination and filtering. Args: project_id: Project ID that the parse results belong to filter_params: Filter parameters (ids, created_at, status, engine) pagination_params: Pagination parameters (page_size, continuation_token, sort_by, sort_order) Returns: PaginatedItems or list of ParseResultEntityWithoutContent (without content chunks for performance) Raises: HTTPException: If project not found # Upload Parse Result Source: https://docs.gp.scale.com/reference/dex/parse/upload-parse-result https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/parse/upload-result Ingest a pre-computed parse result directly without creating a job. This endpoint allows direct ingestion of parse results from external tools while storing them in a dedicated parse_results table for better performance and indexing. Args: request: Parse result creation request with source document ID and result payload storage_gateway: Storage gateway for uploading content to object store Returns: ParseResultEntity representing the stored parse result Raises: HTTPException: If the source document is not found or other errors occur # Create Project Source: https://docs.gp.scale.com/reference/dex/projects/create-project https://dex.sgp.scale.com/openapi.json post /v1/projects Create a new project. # Get Project Source: https://docs.gp.scale.com/reference/dex/projects/get-project https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id} Get a project by ID. # List Projects Source: https://docs.gp.scale.com/reference/dex/projects/list-projects https://dex.sgp.scale.com/openapi.json get /v1/projects List all projects with pagination and filtering support. # Update Project Source: https://docs.gp.scale.com/reference/dex/projects/update-project https://dex.sgp.scale.com/openapi.json put /v1/projects/{project_id} Update a project. # Create Research Job Source: https://docs.gp.scale.com/reference/dex/research/create-research-job https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/research Kick off a Dex Research job. Persists a ``research`` job and starts :class:`~dex_api.temporal.research.workflows.research_workflow.ResearchWorkflow` on the research task queue. **Scope:** one ``inputs.vector_stores`` id, **or** one or more ``inputs.parse_results`` ids (no mixing with vector stores). **Empty inputs:** if the project has exactly one vector store, that store is used automatically (empty-input shortcut). # List Research Results Source: https://docs.gp.scale.com/reference/dex/research/list-research-results https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/research/results List research jobs for this project (same persistence as ``GET …/jobs``, forced to ``operation=research``). Returns :class:`~dex_core.models.jobs.JobEntity` rows (operation is always ``research``). # Add files to vector store Source: https://docs.gp.scale.com/reference/dex/vector-stores/add-files-to-vector-store https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/vector-stores/{vector_store_id}/files Add parsed files to a vector store for indexing and search. This endpoint initiates an asynchronous process to add files to a vector store: 1. Creates a background job to track the operation 2. Starts a Temporal workflow to process the files 3. Files are parsed, chunked, and embedded into the vector store 4. Job status can be monitored via the jobs API **Authentication:** - Credentials can be provided either through project-level credentials or request-level credentials - For SGP Knowledge Base, SGP credentials are required **Process:** - The operation runs asynchronously in the background - Use the returned job entity to monitor progress - Files must be previously parsed (have parse_result_ids) **Returns:** - Job entity with ID, status, and metadata for tracking the operation progress **Example Request:** ```json { "parse_result_ids": ["parse_123", "parse_456"], "credentials": { "sgp": { "api_key": "your-api-key", "account_id": "your-account-id" } } } ``` **Example Response:** ```json { "id": "job_123456789", "operation": "VECTOR_STORE", "status": "PENDING", "parameters": { "parse_result_ids": ["parse_123", "parse_456"] }, "source_id": null, "project_id": "proj_987654321", "created_at": "2024-01-15T10:30:00Z", "updated_at": "2024-01-15T10:30:00Z" } ``` # Create a new vector store Source: https://docs.gp.scale.com/reference/dex/vector-stores/create-a-new-vector-store https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/vector-stores Create a new vector store for a project. This endpoint creates a vector store using the specified engine and configuration. Currently supports SGP Knowledge Base engine, but we plan to support more engines in the future. **Authentication:** - For SGP Knowledge Base, SGP credentials are required **Returns:** - Created vector store entity with ID, name, engine, and creation timestamp **Example Request:** ```json { "engine": "sgp_knowledge_base", "name": "My Knowledge Base", "embedding_model": "openai/text-embedding-3-large", } ``` **Example Response:** ```json { "id": "vs_123456789", "project_id": "proj_987654321", "name": "My Knowledge Base", "engine": "sgp_knowledge_base", "created_at": "2024-01-15T10:30:00Z" } ``` # Delete a vector store Source: https://docs.gp.scale.com/reference/dex/vector-stores/delete-a-vector-store https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/vector-stores/{vector_store_id} Permanently delete a vector store and all its associated data. This endpoint performs a complete deletion of the vector store: 1. Deletes the vector store from the third-party engine 2. Removes all associated files and data 3. Deletes the vector store record from the database **Warning:** This operation is irreversible and will permanently delete all data. **Authentication:** - For SGP Knowledge Base, SGP credentials are required **Returns:** - Success message confirming deletion **Example Response:** ```json { "message": "Vector store deleted successfully" } ``` # Get vector store details Source: https://docs.gp.scale.com/reference/dex/vector-stores/get-vector-store-details https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/vector-stores/{vector_store_id} Retrieve detailed information about a specific vector store. This endpoint returns the complete metadata for a vector store including: - Vector store ID and name - Associated project ID - Engine type and configuration - Creation timestamp **Returns:** - Complete vector store entity with all metadata **Example Response:** ```json { "id": "vs_123456789", "project_id": "proj_987654321", "name": "My Knowledge Base", "engine": "sgp_knowledge_base", "created_at": "2024-01-15T10:30:00Z" } ``` # List files in vector store Source: https://docs.gp.scale.com/reference/dex/vector-stores/list-files-in-vector-store https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/vector-stores/{vector_store_id}/files Retrieve a paginated list of files that have been added to a vector store. This endpoint returns all files that are currently indexed in the specified vector store, including their metadata such as file name, size, MIME type, and creation timestamp. **Token-based Pagination:** - New syntax (recommended): Use `page_size`, `continuation_token`, `sort_by`, `sort_order` query parameters - Legacy syntax (deprecated): Use `size` and `continuation_token` query parameters - If no pagination parameters are provided, returns a plain list - Use the `next_token` from the response to fetch subsequent pages - When `next_token` is `null`, you have reached the last page **Returns:** - Paginated list of vector store file entities - Total count of files in the vector store - `next_token` for pagination (null if no more pages) - File metadata including ID, name, size, and MIME type **Example Response:** ```json { "items": [ { "id": "vsf_123456789", "vector_store_id": "vs_123456789", "parse_result_id": "pres_987654321", "file_id": "file_987654321", "file_name": "document.pdf", "file_size": 1024000, "file_mime_type": "application/pdf", "created_at": "2024-01-15T10:30:00Z" } ], "total_count": 1, "next_token": "eyJmaWxlX2lkIjoiZmlsZV85ODc2NTQzMjEifQ==" } ``` # List vector stores Source: https://docs.gp.scale.com/reference/dex/vector-stores/list-vector-stores https://dex.sgp.scale.com/openapi.json get /v1/projects/{project_id}/vector-stores Retrieve a paginated list of vector stores for a project. This endpoint returns all vector stores associated with the specified project, with optional filtering by name and engine type. **Filtering:** - `name`: Filter vector stores by exact name match - `engine`: Filter by vector store engine type (e.g., sgp_knowledge_base) - `ids`: Filter by vector store IDs (list) - `created_at_start`: Filter by creation time (start) - `created_at_end`: Filter by creation time (end) - `project_id`: Filter by project ID (if not provided in path) **Sorting:** - `sort_by`: Field to sort by (id, created_at, engine, project_id) - `sort_order`: Sort order (asc or desc) **Token-based Pagination:** - New syntax (recommended): Use `page_size`, `continuation_token`, `sort_by`, `sort_order` query parameters - Legacy syntax (deprecated): Use `size` and `continuation_token` query parameters - If no pagination parameters are provided, returns a plain list - Use the `next_token` from the response to fetch subsequent pages - When `next_token` is `null`, you have reached the last page **Returns:** - Paginated list of vector store entities - Total count of matching vector stores - `next_token` for pagination (null if no more pages) **Example Response:** ```json { "items": [ { "id": "vs_123456789", "project_id": "proj_987654321", "name": "My Knowledge Base", "engine": "sgp_knowledge_base", "created_at": "2024-01-15T10:30:00Z" } ], "total_count": 1, "next_token": "eyJpZCI6InZlY3Rvcl9zdG9yZV8xMjM0NTY3ODkwYWJjZGVmIn0=" } ``` # Remove files from vector store Source: https://docs.gp.scale.com/reference/dex/vector-stores/remove-files-from-vector-store https://dex.sgp.scale.com/openapi.json delete /v1/projects/{project_id}/vector-stores/{vector_store_id}/files Remove specific files from a vector store. This endpoint removes files and their associated embeddings from the vector store: 1. Removes files from the third-party vector store engine 2. Deletes file associations from the database 3. All embeddings and searchable content for these files are permanently removed **Authentication:** - Credentials can be provided either through project-level credentials or request-level credentials - For SGP Knowledge Base, SGP credentials are required **Warning:** This operation permanently removes files and their embeddings from the vector store. **Returns:** - Success message confirming file removal **Example Request:** ```json { "parse_result_ids": ["parse_123", "parse_456"], "credentials": { "sgp": { "api_key": "your-api-key", "base_url": "https://api.example.com" } } } ``` **Example Response:** ```json { "message": "Files removed from vector store successfully" } ``` # Search in vector store Source: https://docs.gp.scale.com/reference/dex/vector-stores/search-in-vector-store https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/vector-stores/{vector_store_id}/search Perform semantic search within a vector store. This endpoint performs vector-based semantic search to find the most relevant content based on the provided query. The search uses embeddings to find semantically similar content. **Search Parameters:** - `query`: The search query text - `top_k`: Number of results to return (ranked by relevance) - `filters`: Optional filters to narrow search scope (e.g., by file_id) **Authentication:** - Credentials can be provided either through project-level credentials or request-level credentials - For SGP Knowledge Base, SGP credentials are required **Returns:** - Ranked list of relevant content chunks - Each chunk includes content, source blocks, relevance score, and parse result ID - Results are ordered by relevance score (highest first) **Example Request:** ```json { "query": "What are the main features of the product?", "top_k": 5, "filters": { "file_id": "file_123" }, "credentials": { "sgp": { "api_key": "your-api-key", "base_url": "https://api.example.com" } } } ``` **Example Response:** ```json { "chunks": [ { "content": "The product includes advanced analytics, real-time monitoring, and automated reporting features.", "blocks": [ { "id": "block_123", "content": "The product includes advanced analytics...", "type": "text" } ], "score": 0.95, "parse_result_id": "parse_123" } ] } ``` # Search in vector store's file Source: https://docs.gp.scale.com/reference/dex/vector-stores/search-in-vector-stores-file https://dex.sgp.scale.com/openapi.json post /v1/projects/{project_id}/vector-stores/{vector_store_id}/files/{file_id}/search Perform semantic search within a vector store and filter down to a specific file. Provide the file_id to search within the file, don't confuse it with parse_result_id, because the two are connected. This endpoint performs vector-based semantic search to find the most relevant content based on the provided query. The search uses embeddings to find semantically similar content. **Search Parameters:** - `query`: The search query text - `top_k`: Number of results to return (ranked by relevance) - `filters`: Optional filters to narrow search scope (e.g., by file_id) **Authentication:** - Credentials can be provided either through project-level credentials or request-level credentials - For SGP Knowledge Base, SGP credentials are required **Returns:** - Ranked list of relevant content chunks - Each chunk includes content, source blocks, relevance score, and parse result ID - Results are ordered by relevance score (highest first) **Example Request:** ```json { "query": "What are the main features of the product?", "top_k": 5, "filters": { "file_id": "file_123" }, "credentials": { "sgp": { "api_key": "your-api-key", "base_url": "https://api.example.com" } } } ``` **Example Response:** ```json { "chunks": [ { "content": "The product includes advanced analytics, real-time monitoring, and automated reporting features.", "blocks": [ { "id": "block_123", "content": "The product includes advanced analytics...", "type": "text" } ], "score": 0.95, "parse_result_id": "parse_123" } ] } ``` # Ingest Data Sources Change Event Source: https://docs.gp.scale.com/reference/dex/webhooks/ingest-data-sources-change-event https://dex.sgp.scale.com/openapi.json post /v1/webhooks/data-sources Verify HMAC and enqueue processing for a normalized change event.