Generate completion
Description
Interact with the LLM model using a completions interface. The LLM model will generate a text completion based on the provided prompt.
{
"model": "gpt-4o",
"prompt": "What is the capital of France?"
}
Authorizations
Body
Prompt for which to generate the completion.
Good prompt engineering is crucial to getting performant results from the model. If you are having trouble getting the model to perform well, try writing a more specific prompt here before trying more expensive techniques such as swapping in other models or finetuning the underlying LLM.
The name of the model.
Should be formatted as vendor/model_name or vendor/model_name/deployment (ie: openai/gpt-3.5-turbo).
The sampling temperature to use for the completion. Higher values mean the model will take more risks.
Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.
An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.
It is generally recommended to alter this or temperature but not both.
Only sample from the top K options for each subsequent token
Whether or not to stream the response.
Setting this to True will stream the response in real-time.
Customize streaming behavior
Up to 4 sequences where the API will stop generating further tokens.
Modify the likelihood of specified tokens appearing in the completion.
Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.
Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.
An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.
The maximum number of tokens that can be generated.
The token count of your prompt plus max_tokens cannot exceed the model's context length.
How many completions to generate for each prompt.
Note: Because this parameter generates many completions, it can quickly consume your token quota. Use carefully and ensure that you have reasonable settings for max_tokens and stop.
If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result.
The suffix that comes after a completion of inserted text.This parameter is only supported for gpt-3.5-turbo-instruct.
An object specifying the format that the model must output
A unique identifier representing your end-user.
Audio-related parameters for chat completion
Timeout for the API request
Maximum number of tokens to generate in the completion
List of modalities for the chat completion
text, audio Additional params to be included in the API request
Currently only supported for OpenAI and Anthropic models.A list of tools the model may call.Currently, only functions are supported as a tool.Use this to provide a list of functions the model may generate JSON inputs for.
Currently only supported for OpenAI and Anthropic models.Controls which (if any) tool is called by the model.
none, auto, required Currently only supported for OpenAI models.Enables calling tools in parallel.
Generates best_of completions server-side and returns the best (the one with the highest log probability per token). Results cannot be streamed.
Additional headers to be included in the API request
Configure a predicted output to reduce latency
Response
The list of completions.
Usage statistics.
A unique identifier for the completion.
The model used for completion.
The object type, ex text_completion or chat.completion
This fingerprint represents the backend configuration that the model runs with.
The Unix timestamp (in seconds) of when the chat completion was created.