The Scale GenAI Platform empowers modern enterprises to rapidly develop, test and deploy Generative AI applications for custom use cases, using their proprietary data assets.The platform supports two distinct workflows related to Generative AI application development.
The first one is Agent Building and Execution, or the act of having an agent that runs.
The second one is Agent Operations, or the act of managing the agent, reviewing the data that it outputs, and improving that agent.The ultimate goal of SGP is to enable teams of developers to be able to build and improve multiple agents specializing in multiple goals all on the same platform. SGP helps every Enterprise build and maintain a portfolio of agents.
The agent builder view shows a list of all the agents that are currently being built or monitored in your account.Clicking into an agent allows users to interact with an agent, monitor the agent, and view evaluations performed by an agent.
Users can also see the deployment history of the agent.
SGP’s goal for building agents is to provide the most developer friendly framework and environment possible. At the core, building agents is just writing code.
Our goal is for all developers to continue developing in the environment they are most familiar with at the fastest speed possible.
This is an example of a medical assistant agent we built, a medical assistant that helps users find doctors at a specific hospital based on their symptoms. The agent diagnoses their symptom and then directs them to a doctor.
In today’s world, pulling an agent from the internet is easy, most agent frameworks focus on helping users get started on writing an agent. The problem that SGP AgentEx aims to solve is to help users quickly get from a locally developed agent into a fully functional production quality agent other people can use.
Backend - AgentEx provides a simple CLI that enables users to run the agent locally.
Frontend - AgentEx also provides the ability to run a local UI very quickly to allow users to test.
Users can then go to their local machine and run an agent.This experience enables a fast development cycle:
User can test exactly what’s going on.
User can make alterations to the code and change the behavior.
User can test again.
AgentEx has a built a continuous deployment pipeline that takes changes from the agent and pushes them through a series of steps, and automatically deploys your agent for you.After making changes, users will automaticaly be able to see on the platform the deployment history of the agent.
A user can interact with an agent by clicking “Open Agent” on the page of the specific Agent.This will take users to a fully deployed, hosted version of the agent developers and end users can interact with.Since this is a live deployment, all interactions captured from this deployment will show up in our tracing interface (see below as part of Agent Operations) to assist developers with monitoring their application.
All agents built and deployed with AgentEx (like in the example above) will have their interactions automatically show up in the tracing library.The AgentOps Tracing Platform also has an SDK that allows users to upload their own traces to get the same experience.Users can also see specific details on the execution of the agent in the tracing view.Users can see the input, output, and all the steps the agent went through for the execution. Engineers can use this data to understand what tools the agent is calling, what steps the agents are going through for reasoning, and how the agent is performing and make adjustements to the agent.To learn more about tracing, refer to the tracing section.
After identifying areas that can be improved from your agents, the AgentOps platform supports developers in running evaluations on their agents.Users can create evaluation datasets through traces and then run evaluations on the platform.The platform supports two types of evaluations:
LLM-as-a-Judge: Have an LLM evaluate the dataset
Contributor Evaluation: Have humans annotate the dataset
To learn more about evaluations, refer to the evaluations section.
The combination of all these tools supports the agent development lifecycle.
Developing the local agent.
Review and deploy the local agent.
Test the deployed agent.
Monitor the deployed agent through traces.
Segment out evaluations from traces.
Enterprises that scale agents need to stop worrying about the overhead of development and focus on the business logic.
SGP is a platform designed to enable that.
