Agent Service provides a powerful mechanism to evaluate the performance of individual nodes within a workflow. Evaluations help measure the quality of retrieval, reranking, and LLM-generated responses, ensuring that each component of a workflow meets accuracy and relevance standards.This guide explains how to configure node evaluations using an example YAML configuration.
Ensure High-Quality Responses: Evaluate LLM output against known answers.
Optimize Retrieval & Reranking: Identify how well retrieved documents align with ground truth.
Automate Quality Assurance: Use auto-evaluations to continuously measure AI performance.
Track Model Performance Over Time: Store results in EGP UI for monitoring and improvement.
This configuration provides a fully automated evaluation pipeline for assessing retrieval, reranking, and response quality in Agent Service workflows.