Agents

Execute Agent

### Description Executes one Agent inference step. Given a list of messages and a list of tools to ask for help from, the Agent will either respond with a final answer directly or ask the user to execute a tool to provide more information. ### Details An Agent is a component that utilizes a Language Model (LLM) as an interpreter and decision maker. Unlike asking an LLM for a direct response, communicating with an agent consists of a running dialogue where an agent can optionally ask the user to execute specialized tools for specific tasks, such as calculations, web searches, or accessing custom data from private knowledge bases. An agent is designed to be stateless, emitting outputs one step at a time. This means that client-side applications are responsible for managing message history, tool execution, and responses. This grants users greater flexibility to write and execute custom tools and maintain explicit control over their message history. #### Message Types - `User Message`: A message from the user to the agent. - `System Message`: An informational text message from the system to guide the agent. It is not a user message or agent message because it did not come from either entity. - `Agent Message`: A message from the agent to the client. It will contain either a final answer as `content` or a request for the user to execute a tool as a `tool_request`. - `Tool Message`: A message from the user to the agent that contains the output of a tool execution. The tool message will be processed by the agent and the agent will respond with either a final answer or another tool request. #### Agent Instructions Instructions are used to guide the agent's decision making process and output generation. Good prompt engineering is crucial to getting performant results from the agent. If you are having trouble getting the agent to perform well, try writing more specific instructions before trying more expensive techniques such as swapping in other models or finetuning the underlying LLM. For example, the default instructions we set for the agent are the following: > You are an AI assistant that helps users with their questions. You can answer questions directly or acquire information from any of the attached tools to assist you. Always answer the user's most recent query to the best of your knowledge.<br> > When asked about what tools are available, you must list each attached tool's name and description. When asked about what you can do, mention that in addition to your normal capabilities, you can also use the attached tools by listing their names and descriptions. You cannot use any other tools other than the ones provided to you explicitly. ### Restrictions and Limits **Message Limits**: - The message list is not limited by length, but by the context limit of the underlying language model. If you are getting an error regarding the underlying model's context limit, try using a memory strategy to condense the input messages. **Model Restrictions**: - Currently, only closed source models like GPT and Claude are supported due to the limitations of open source models when it comes to tool selection, generating tool arguments in valid JSON, and planning out multi-step tool execution. Specialized fine-tuning will likely be required to make open source models compatible with agents.

POST

agents

execute

object

model

enum<string>

memory_strategy

object

name

enum<string>

params

object

integer

tools

array

messages

array

model_parameters

object

temperature

number

stop_sequences

array

max_tokens

integer

topP

number

topK

integer

frequency_penalty

number

presence_penalty

number

instructions

string

Authorizations

x-api-key

string

headerrequired

Body

application/json

model

enum<string>

required

The ID of the model to use for the agent. We only support the models listed here so far.

Available options:

gpt-4,

gpt-4-0613,

gpt-4-32k,

gpt-4-32k-0613,

gpt-3.5-turbo,

gpt-3.5-turbo-0613,

gpt-3.5-turbo-16k,

gpt-3.5-turbo-16k-0613,

claude-instant-1,

claude-instant-1.1,

claude-2,

claude-2.0

tools

object[]

required

The list of specs of tools that the agent can use. Each spec must contain a name key set to the name of the tool, a description key set to the description of the tool, and an arguments key set to a JSON Schema compliant object describing the tool arguments.

The name and description of each tool is used by the agent to decide when to use certain tools. Because some queries are complex and may require multiple tools to complete, it is important to make these descriptions as informative as possible. If a tool is not being chosen when it should, it is common practice to tune the description of the tool to make it more apparent to the agent when the tool can be used effectively.

tools.name

string

required

Name of the tool.

A tool is a function that the client application has at its disposal. The tool name is the name the client wishes the Agent to use to refer to this function when it decides if it wants the user to use the tool or not. Itmust be unique amongst the set of tools provided in a single API call.

tools.description

string

required

Description of the tool.

Because some queries are complex and may require multiple tools to complete, it is important to make these descriptions as informative as possible. If a tool is not being chosen when it should, it is common practice to tune the description of the tool to make it more apparent to the agent when the tool can be used effectively.

tools.arguments

object

required

An JSON Schema-compliant schema for the tool arguments. To describe a function that accepts no parameters, provide the value {"type": ""object", "properties": {}}.

For more information on how to define a valid JSON Schema object, visit https://json-schema.org/understanding-json-schema/reference/object.html

messages

object[]

required

The list of messages in the conversation.

Expand each message type to see how it works and when to use it. Most conversations should begin with a single user message.

memory_strategy

object

The memory strategy to use for the agent. A memory strategy is a way to prevent the underlying LLM's context limit from being exceeded. Each memory strategy uses a different technique to condense the input message list into a smaller payload for the underlying LLM.

We only support the Last K memory strategy right now, but will be adding new strategies soon.

model_parameters

object

Configuration parameters for the agent model, such as temperature, max_tokens, and stop_sequences.

If not specified, the default value are:

temperature: 0.2
max_tokens: None (limited by the model's max tokens)
stop_sequences: None

model_parameters.temperature

number

default: 0.2

What sampling temperature to use, between [0, 1]. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Setting temperature=0.0 will enable fully deterministic (greedy) sampling.

Required range: 0 < x < 1

model_parameters.stop_sequences

string[]

List of up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.

model_parameters.max_tokens

integer

The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model's context length. If not, specified, max_tokens will be determined based on the model used: | Model API family | Model API default | EGP applied default | | --- | --- | --- | | OpenAI Completions | 16 | context window - prompt size | | OpenAI Chat Completions | context window - prompt size | context window - prompt size | | LLM Engine | max_new_tokens parameter is required | 100 | | Anthropic Claude 2 | max_tokens_to_sample parameter is required | 10000 |

model_parameters.topP

number

The cumulative probability cutoff for token selection. Lower values mean sampling from a smaller, more top-weighted nucleus. Available for models from Anthropic, Google, Mistral, LLM Engine, and OpenAI.

model_parameters.topK

integer

Sample from the k most likely next tokens at each step. Lower k focuses on higher probability tokens. Available for models from Anthropic, Google, and LLM Engine.

model_parameters.frequency_penalty

number

Penalize tokens based on how much they have already appeared in the text. Positive values encourage the model to generate new tokens and negative values encourage the model to repeat tokens. Available for models from LLM Engine, and OpenAI

model_parameters.presence_penalty

number

Penalize tokens based on if they have already appeared in the text. Positive values encourage the model to generate new tokens and negative values encourage the model to repeat tokens. Available for models from LLM Engine, and OpenAI.

instructions

string

default: You are an AI assistant that helps users with their questions. You can answer questions directly or acquire information from any of the attached tools to assist you. Always answer the user's most recent query to the best of your knowledge. When asked about what tools are available, you must list each attached tool's name and description. When asked about what you can do, mention that in addition to your normal capabilities, you can also use the attached tools by listing their names and descriptions. You cannot use any other tools other than the ones provided to you explicitly.

The initial instructions to provide to the agent.

Use this to guide the agent to act in more specific ways. For example, if you have specific rules you want to restrict the agent to follow you can specify them here. For example, if I want the agent to always use certain tools before others, I can write that rule in these instructions.

Good prompt engineering is crucial to getting performant results from the agent. If you are having trouble getting the agent to perform well, try writing more specific instructions here before trying more expensive techniques such as swapping in other models or finetuning the underlying LLM.

Response

200 - application/json

action

enum<string>

required

The action that the agent performed.

The context will contain a key for each action that the agent can perform. However, only the key corresponding to the action that the agent actually performed will have a populated value. The rest of the values will be null.

Available options:

tool_request,

content

context

object

required

Context object containing the output payload. This will contain a key for all actions that the agent can perform. However, only the key corresponding to the action that the agent performed have a populated value. The rest of the values will be null.

Synthesize Chunks Create Completion

Knowledge Bases

Chunks

Agents

Completions

Chat Completions

Models

Users

Accounts

Organizations

Question Sets

Evaluations

Evaluation Configs

Evaluation Datasets

Studio Projects

Application Specs

Questions

Knowledge Base Data Sources

Model Templates V3 (Beta)

Model server

API Reference

Fine Tuning Jobs V3 (Beta)

Training Datasets V3 (Beta)

package deployments

Beta

Applications

ChatThreads

Interactions

MonitoringDashboard

Chat Themes

account groups

account

Guardrails

Execute Agent

Authorizations

Body

Response