Capabilities Changelog

​

Latest Updates

​

Major Upgrades to Compass Workflows

​

What’s New

1. Updated: Introduction to Compass
   • Key features and available card types
   • Scheduling and monitoring workflows
   • Roadmap for H1 2026
2. New: Guide to Automating Evals on Compass
   • Step-by-step tutorial for building automated evaluation workflows
   • Connecting to data sources (Snowflake, Postgres, MongoDB, MS SQL)
   • Calling Agentex agents for completions
   • Joining ground truth datasets from other workflows
   • Configuring LLM as Judge evaluations
   • Exporting results to Evaluations and Dashboards
   • Scheduling automated evaluation runs

​

Updates to Key Features

• Call Agent: Generate outputs from your Agentex agents with configurable timeout, retries, and response caching
• Join Workflow: Combine datasets from multiple workflows with Left, Right, Inner, or Outer joins
• LLM as Judge: Evaluate agent outputs with customizable rubrics and judge configurations

• Access sources configured in SGP Data Sources tab in your workflows
• Export datasets directly to SGP Datasets tab
• Export evaluation results directly to SGP Evaluations tab

​

How to see the features

​

Dex SDK: Version 0.4.0

​

🔄 Unified Entity Operations

• Unified API: All entities (files, jobs, parse_results, extractions, vector_stores) now support standardized get and list operations
• Project-scoped operations: Access all entity types through your project instance
• Consistent patterns: Same approach works across all entity types

# List operations available for all entities
files = await project.list_files()
jobs = await project.list_jobs()
parse_results = await project.list_parse_results()
extractions = await project.list_extractions()
vector_stores = await project.list_vector_stores()

​

📄 Pagination, Filtering, and Sorting

• Token-based pagination: Set page size and use continuation tokens to navigate through results
• Entity-specific filters: Available for all entities with dedicated filter types
• Flexible sorting: Sort results by any field in ascending or descending order
• Performance improvements: Process large datasets more efficiently

from dex_sdk.types import PaginationParams

pagination_params = PaginationParams(
    page_size=10,
    sort_by="created_at",
    sort_order="desc",
    continuation_token=None
)

files = await project.list_files(pagination_params=pagination_params)

from dex_sdk.types import FileListFilter, JobListFilter, ParseResultListFilter, ExtractionListFilter, VectorStoreListFilter
from datetime import datetime, timedelta

# Filter for entities created in the last day
file_filter = FileListFilter(created_at_start=datetime.now() - timedelta(days=1))
job_filter = JobListFilter(created_at_start=datetime.now() - timedelta(days=1))
parse_result_filter = ParseResultListFilter(created_at_start=datetime.now() - timedelta(days=1))
extraction_filter = ExtractionListFilter(created_at_start=datetime.now() - timedelta(days=1))
vector_store_filter = VectorStoreListFilter(created_at_start=datetime.now() - timedelta(days=1))

# Apply filters
files = await project.list_files(pagination_params=pagination_params, filter=file_filter)
jobs = await project.list_jobs(pagination_params=pagination_params, filter=job_filter)
parse_results = await project.list_parse_results(pagination_params=pagination_params, filter=parse_result_filter)
extractions = await project.list_extractions(pagination_params=pagination_params, filter=extraction_filter)
vector_stores = await project.list_vector_stores(pagination_params=pagination_params, filter=vector_store_filter)

​

🔧 Enhanced Type Support

• Type overrides: SDK provides different types for paginated vs non-paginated responses
• Backward compatibility: Existing code continues to work without changes
• Better IDE support: Enhanced autocomplete and type checking

​

⚡ Async Job Improvements

• Job monitoring: Easily track job status and progress
• Throttling support: Control job throughput to match your needs
• SGP trace integration: Jobs are now connected to SGP traces for end-to-end observability
• Trace retrieval: Fetch complete trace data for any job

from dex_sdk.types import ReductoParseJobParams, ReductoParseEngineOptions, ReductoChunkingOptions, ReductoChunkingMethod, ParseEngine, JobStatus
import asyncio

# Start a parse job
parse_job = await project.start_parse_job(
    dex_file=dex_file,
    parameters=ReductoParseJobParams(
        engine=ParseEngine.REDUCTO,
        options=ReductoParseEngineOptions(
            chunking=ReductoChunkingOptions(
                chunk_mode=ReductoChunkingMethod.VARIABLE,
            )
        ),
    ),
)

# Monitor job progress
while parse_job.data.status != JobStatus.SUCCEEDED and parse_job.data.status != JobStatus.FAILED:
    await asyncio.sleep(1)
    await parse_job.refresh()

# Get the result
parse_result = await parse_job.get_result()

# Retrieve SGP traces for the job
from scale_gp_beta import SGPClient

sgp_client = SGPClient(...)
spans = [span for span in sgp_client.spans.search(
    sort_by="created_at",
    sort_order="desc",
    extra_metadata={"job_id": "job_parse_eb62b57e8c4940028b82c559903ed003"},
    parents_only=True,
)]

trace_id = spans[0].trace_id
spans = [span for span in sgp_client.spans.search(trace_ids=[trace_id])]

• Better observability: Track your jobs through SGP’s tracing infrastructure
• Easier debugging: Access detailed execution traces for failed jobs
• Performance monitoring: Analyze job performance and identify bottlenecks
• Request correlation: Connect job execution to API requests and traces

​

How to Update

• Add pagination to large list operations for better performance
• Use filters to find specific entities by creation time or other criteria
• Integrate SGP tracing to monitor your async jobs
• Leverage enhanced type support for better IDE experience

​

Dex Document Understanding: Version 0.3.2

​

🔐 Authentication Changes

• SGP credentials are now passed directly to DexClient instead of being stored in project configurations
• Every API request is now authenticated using your SGP API key and account ID
• The ProjectCredentials and SGPCredentials types are deprecated and will be removed in a future version

from dex_sdk import DexClient
from dex_sdk.types import ProjectCredentials, SGPCredentials

dex_client = DexClient(base_url="https://dex.sgp.scale.com")

project = await dex_client.create_project(
    name="My Project",
    credentials=ProjectCredentials(
        sgp=SGPCredentials(
            account_id=os.getenv("SGP_ACCOUNT_ID"),
            api_key=os.getenv("SGP_API_KEY"),
        ),
    ),
)

import os
from dex_sdk import DexClient

# Pass credentials when creating the client
dex_client = DexClient(
    base_url="https://dex.sgp.scale.com",
    api_key=os.getenv("SGP_API_KEY"),
    account_id=os.getenv("SGP_ACCOUNT_ID"),
)

# Create project without credentials parameter
project = await dex_client.create_project(
    name="My Project",
)

• Enhanced Security: SGP credentials are no longer stored in the Dex database
• Simpler API: Credentials are set once at client initialization
• Consistent Authentication: Every request is authenticated the same way

​

🗄️ Data Retention Policies

• Automatic cleanup: Set retention periods for files and result artifacts to automatically delete data after a specified time
• Flexible configuration: Configure different retention periods for files vs. processing artifacts
• Project-level control: Retention policies are configured per project and can be updated at any time

from datetime import timedelta
from dex_sdk.types import ProjectConfiguration, RetentionPolicy

project = await dex_client.create_project(
    name="My Project",
    configuration=ProjectConfiguration(
        retention=RetentionPolicy(
            files=timedelta(days=30),           # Files expire after 30 days
            result_artifacts=timedelta(days=7),  # Parse/extract results expire after 7 days
        )
    )
)

• Compliance: Meet regulatory requirements (GDPR, HIPAA) by enforcing data retention limits
• Cost Management: Reduce storage costs by automatically cleaning up old files and artifacts
• Security: Minimize data exposure by limiting how long sensitive documents are stored

​

📚 Documentation Improvements

• Common Types Section: Added comprehensive documentation for all core data models organized by category:
  • Project Types: ProjectEntity, ProjectStatus, ProjectConfiguration, RetentionPolicy
  • File Types: FileEntity, FileStatus, FileDownloadURL
  • Job Types: JobEntity, JobOperationType, JobStatus
  • Parse Types: ParseResultEntity, ParseResultMetadata, ParseChunk, ParseBlock, BoundingBox, ParseEngine, ReductoChunkingMethod
  • Extraction Types: ExtractionEntity, ExtractionResult, ExtractedField, ExtractionCitation, ExtractionParameters, UsageInfo
  • Vector Store Types: VectorStoreEntity, VectorStoreEngines, VectorStoreChunk, SearchConfig
• Parse Job Parameters: New section documenting parsing configuration options

• Working with Parse Results: How to access parse metadata, iterate through chunks and blocks, access bounding boxes and confidence scores
• Working with Extraction Results: How to access extracted field values, citations, confidence scores, and token usage information

• Introduction: Updated File Management section to mention data retention policies
• Getting Started Guide: Added “Configuring Data Retention” section with practical examples
• API Reference: Enhanced type documentation and examples

​

🔧 Type System Updates

• Request objects now end with *Request (e.g., CustomParseResultRequest)
• Entity objects now end with *Entity (e.g., ParseResultEntity)

# Old import (deprecated)
from dex_sdk.types import CustomParseResult

# New import
from dex_sdk.types import CustomParseResultRequest

• Fixed typing inconsistencies for entities returned by the DEX API
• Better type hints for all SDK methods
• Enhanced autocomplete support in IDEs

​

How to Update

1. Add api_key and account_id parameters to DexClient() initialization
2. Remove credentials parameter from create_project() calls
3. Remove imports of ProjectCredentials and SGPCredentials

​

New Capability: IRIS OCR

​

What’s New

1. Introduction to IRIS
   • Overview of IRIS OCR capability
   • Three-stage pipeline architecture (layout detection, OCR processing, assembly)
   • 15+ supported OCR models including open-source and vision-language models
   • Multi-language support with specialized Arabic models
   • Common use cases and key advantages
2. Getting Started with IRIS
   • Comprehensive guide to using IRIS through Dex SDK
   • Prerequisites and setup instructions
   • Parsing PDFs and images with complete examples
   • Configuration options for parse engine
   • Understanding parse results and chunk structure
   • File management and error handling
   • Batch processing examples
   • Multi-language support details
   • Best practices for production use
   • Performance considerations and optimization tips

​

Key Features

• Layout-Aware Processing: Automatically detects text, tables, and images before OCR
• Multiple OCR Engines: Choose from Tesseract, EasyOCR, PaddleOCR, Surya, GPT-4o, Gemini, and more
• Table-Specific Processing: Specialized models optimized for accurate table extraction
• Multi-Language Support: Process documents in 35+ languages including Arabic
• Dex Integration: Seamless integration with Dex’s document understanding platform
• Async Processing: Non-blocking parse jobs with project-based organization

​

How to Access

from dex_core.models.parse_job import IrisParseEngineOptions, IrisParseJobParams

parse_job = await dex_file.start_parse_job(
    IrisParseJobParams(options=IrisParseEngineOptions())
)

​

Configuration Updates

​

Navigation Structure Improvements

• Added explicit V5 (beta) version tags to all Capabilities navigation groups
• Ensures proper scoping of Capabilities documentation to V5
• Improved navigation organization for better user experience

• Getting Started
• Document Understanding
• OCR
• Workflows

​

Support and Feedback

​

Compass-Specific Questions

• Slack: #compass
• Documentation: Introduction to Compass
• Tutorial: Guide to Automating Evals on Compass

​

Dex-Specific Questions

• Slack: #dex-help
• Documentation: Getting Started with Dex
• API Reference: Dex SDK API Reference

​

General Capabilities Documentation