Flushing Tracing Data

By default, tracing data (spans) are collected in an in-memory queue and sent to the SGP platform asynchronously in batches by a background worker. This batching mechanism is designed for efficiency and minimal performance impact on your application.

However, there are specific scenarios where you might need to force an immediate export of tracing data:

• Critical workflow points: To ensure data is persisted before sensitive operations or transitions in a distributed workflow.
• Debugging: To see tracing data appear on the Traces page immediately during development.

​

How to Flush Tracing Data

The SDK provides methods to manually trigger data export:

​

Flushing the entire queue

Use tracing.flush_queue() to force a synchronous export of all spans currently buffered in the SDK’s internal queue. This method will block the calling thread until all pending spans have been sent.

import scale_gp_beta.lib.tracing as tracing
import time

# Simulate creating some spans
with tracing.create_trace("short_lived_process_trace"):
    with tracing.create_span("task_alpha"):
        time.sleep(0.1)
    with tracing.create_span("task_beta"):
        time.sleep(0.1)

# Force all pending spans to be sent before the application exits
tracing.flush_queue()
print("All pending spans have been flushed to SGP.")

# In a multi-worker or distributed application, you might use this
# before enqueuing a job to ensure parent trace data is sent first.

​

Flushing individual spans or traces

You can also explicitly flush an individual Span object. When you call span.flush(), that specific span will be sent. For Trace objects, flushing (trace.flush()) will send the root span.

import uuid
from datetime import datetime, timezone, timedelta
import scale_gp_beta.lib.tracing as tracing

now = datetime.now(timezone.utc)

# Example: Flushing a single span immediately after creation
single_span = tracing.create_span("critical_single_operation", trace_id=str(uuid.uuid4()))
single_span.start_time = now.isoformat()
single_span.end_time = (now + timedelta(seconds=1)).isoformat()
single_span.output = {"status": "completed"}
single_span.flush() # This span is sent to the backend immediately

# Example: Flushing a trace's root span
my_trace = tracing.create_trace(name="important_trace")
my_trace.root_span.start_time = now.isoformat()
# ... perform operations within the trace ...
my_trace.root_span.end_time = (now + timedelta(seconds=5)).isoformat()
my_trace.flush()

my_span = tracing.create_span("my_async_span", trace_id="...")
my_span.start()
# ... do work ...
my_span.end()
my_span.flush(blocking=False) # Enqueue for background worker, don't block

​

Automatic Shutdown Flush

You typically do not need to manually call flush_queue() at the very end of your program. When the SDK’s background worker is shut down (e.g., when your Python application exits normally), it will automatically attempt to flush any remaining tracing data in the queue before exiting.