> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gp.scale.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Generate OpenAI chat completion from messages

> Generates a chat completion from an OpenAI-style `messages` array.

Use this endpoint for standard messages-based chat inference; use /v5/completions instead when you
have a raw text `prompt` rather than messages, /v5/responses for the OpenAI Responses API contract,
and /v5/inference when the payload does not follow any OpenAI schema. The request accepts the OpenAI
Chat Completions parameters plus a fixed set of vendor generation parameters such as `top_k`,
`guided_choice`, and `thinking`; other extra fields are dropped, and fields that set the provider
endpoint, credentials, or headers (for example `api_base`, `api_key`, `extra_headers`) are rejected
with a 422. The model is selected from `model` given as `vendor/name`; most vendors are served
through the litellm proxy gateway, while
OpenAI may use a native gateway. When `stream` is set the response is delivered as server-sent events
of `chat.completion.chunk`; otherwise a single `chat.completion` object is returned. Token usage is
recorded for the account, and for streaming responses it is read from the final chunk.



## OpenAPI

````yaml /openapi/v5.yaml post /v5/chat/completions
openapi: 3.1.0
info:
  title: EGP API V5
  description: >-
    This is the parent API for all EGP APIs. If you are looking for the EGP API,
    please go to https://api.egp.scale.com/docs.
  contact:
    name: Scale Generative AI Platform
    url: https://scale.com/genai-platform
  version: 0.1.0
servers:
  - url: https://api.egp.scale.com
security: []
paths:
  /v5/chat/completions:
    post:
      tags:
        - Inference
      summary: Generate OpenAI chat completion from messages
      description: >-
        Generates a chat completion from an OpenAI-style `messages` array.


        Use this endpoint for standard messages-based chat inference; use
        /v5/completions instead when you

        have a raw text `prompt` rather than messages, /v5/responses for the
        OpenAI Responses API contract,

        and /v5/inference when the payload does not follow any OpenAI schema.
        The request accepts the OpenAI

        Chat Completions parameters plus a fixed set of vendor generation
        parameters such as `top_k`,

        `guided_choice`, and `thinking`; other extra fields are dropped, and
        fields that set the provider

        endpoint, credentials, or headers (for example `api_base`, `api_key`,
        `extra_headers`) are rejected

        with a 422. The model is selected from `model` given as `vendor/name`;
        most vendors are served

        through the litellm proxy gateway, while

        OpenAI may use a native gateway. When `stream` is set the response is
        delivered as server-sent events

        of `chat.completion.chunk`; otherwise a single `chat.completion` object
        is returned. Token usage is

        recorded for the account, and for streaming responses it is read from
        the final chunk.
      operationId: POST-V5-/v5/chat/completions
      parameters:
        - name: x-api-key
          in: header
          required: false
          schema:
            anyOf:
              - type: string
              - type: 'null'
            title: X-Api-Key
        - name: x-openai-api-key
          in: header
          required: false
          schema:
            anyOf:
              - type: string
              - type: 'null'
            title: X-Openai-Api-Key
        - name: x-selected-account-id
          in: header
          required: false
          schema:
            anyOf:
              - type: string
              - type: 'null'
            title: Account ID Header
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ChatCompletionRequest'
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                anyOf:
                  - $ref: '#/components/schemas/ChatCompletion'
                  - $ref: '#/components/schemas/ChatCompletionChunk'
                title: Response Post-V5-/V5/Chat/Completions
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      security:
        - APIKeyHeader: []
components:
  schemas:
    ChatCompletionRequest:
      properties:
        top_k:
          anyOf:
            - type: integer
            - type: 'null'
          title: Top K
          description: Only sample from the top K options for each subsequent token
        frequency_penalty:
          anyOf:
            - type: number
              maximum: 2
              minimum: -2
            - type: 'null'
          title: Frequency Penalty
          description: >-
            Number between -2.0 and 2.0. Positive values penalize new tokens
            based on their existing frequency in the text so far.
        function_call:
          anyOf:
            - additionalProperties: true
              type: object
            - type: 'null'
          title: Function Call
          description: >-
            Deprecated in favor of tool_choice. Controls which function is
            called by the model.
        functions:
          anyOf:
            - items:
                additionalProperties: true
                type: object
              type: array
            - type: 'null'
          title: Functions
          description: >-
            Deprecated in favor of tools. A list of functions the model may
            generate JSON inputs for.
        logit_bias:
          anyOf:
            - additionalProperties:
                type: integer
              type: object
            - type: 'null'
          title: Logit Bias
          description: >-
            Modify the likelihood of specified tokens appearing in the
            completion. Maps tokens to bias values from -100 to 100.
        logprobs:
          anyOf:
            - type: boolean
            - type: 'null'
          title: Logprobs
          description: Whether to return log probabilities of the output tokens or not.
        max_completion_tokens:
          anyOf:
            - type: integer
            - type: 'null'
          title: Max Completion Tokens
          description: >-
            An upper bound for the number of tokens that can be generated,
            including visible output tokens and reasoning tokens.
        max_tokens:
          anyOf:
            - type: integer
            - type: 'null'
          title: Max Tokens
          description: >-
            Deprecated in favor of max_completion_tokens. The maximum number of
            tokens to generate.
        metadata:
          anyOf:
            - additionalProperties:
                anyOf:
                  - type: string
                  - items:
                      type: string
                    type: array
              type: object
            - type: 'null'
          title: Metadata
          description: >-
            Developer-defined tags and values used for filtering completions in
            the dashboard.
        modalities:
          anyOf:
            - items:
                type: string
              type: array
            - type: 'null'
          title: Modalities
          description: >-
            Output types that you would like the model to generate for this
            request.
        'n':
          anyOf:
            - type: integer
            - type: 'null'
          title: 'N'
          description: How many chat completion choices to generate for each input message.
        parallel_tool_calls:
          anyOf:
            - type: boolean
            - type: 'null'
          title: Parallel Tool Calls
          description: Whether to enable parallel function calling during tool use.
        prediction:
          anyOf:
            - additionalProperties: true
              type: object
            - type: 'null'
          title: Prediction
          description: >-
            Static predicted output content, such as the content of a text file
            being regenerated.
        presence_penalty:
          anyOf:
            - type: number
              maximum: 2
              minimum: -2
            - type: 'null'
          title: Presence Penalty
          description: >-
            Number between -2.0 and 2.0. Positive values penalize tokens based
            on whether they appear in the text so far.
        reasoning_effort:
          anyOf:
            - type: string
            - type: 'null'
          title: Reasoning Effort
          description: >-
            For o1 models only. Constrains effort on reasoning. Values: low,
            medium, high.
        response_format:
          anyOf:
            - additionalProperties: true
              type: object
            - type: 'null'
          title: Response Format
          description: An object specifying the format that the model must output.
        seed:
          anyOf:
            - type: integer
            - type: 'null'
          title: Seed
          description: >-
            If specified, system will attempt to sample deterministically for
            repeated requests with same seed.
        stop:
          anyOf:
            - type: string
            - items:
                type: string
              type: array
            - type: 'null'
          title: Stop
          description: Up to 4 sequences where the API will stop generating further tokens.
        store:
          anyOf:
            - type: boolean
            - type: 'null'
          title: Store
          description: >-
            Whether to store the output for use in model distillation or evals
            products.
        stream:
          anyOf:
            - type: boolean
            - type: 'null'
          title: Stream
          description: If true, partial message deltas will be sent as server-sent events.
        stream_options:
          anyOf:
            - additionalProperties: true
              type: object
            - type: 'null'
          title: Stream Options
          description: Options for streaming response. Only set this when stream is true.
        temperature:
          anyOf:
            - type: number
              maximum: 2
              minimum: 0
            - type: 'null'
          title: Temperature
          description: >-
            What sampling temperature to use. Higher values make output more
            random, lower more focused.
        tool_choice:
          anyOf:
            - type: string
            - additionalProperties: true
              type: object
            - type: 'null'
          title: Tool Choice
          description: >-
            Controls which tool is called by the model. Values: none, auto,
            required, or specific tool.
        tools:
          anyOf:
            - items:
                additionalProperties: true
                type: object
              type: array
            - type: 'null'
          title: Tools
          description: >-
            A list of tools the model may call. Currently, only functions are
            supported. Max 128 functions.
        top_logprobs:
          anyOf:
            - type: integer
              maximum: 20
              minimum: 0
            - type: 'null'
          title: Top Logprobs
          description: >-
            Number of most likely tokens to return at each position, with
            associated log probability.
        top_p:
          anyOf:
            - type: number
              maximum: 1
              minimum: 0
            - type: 'null'
          title: Top P
          description: >-
            Alternative to temperature. Only tokens comprising top_p probability
            mass are considered.
        audio:
          anyOf:
            - additionalProperties: true
              type: object
            - type: 'null'
          title: Audio
          description: >-
            Parameters for audio output. Required when audio output is requested
            with modalities: ['audio'].
        model:
          type: string
          title: Model
          description: model specified as `model_vendor/model`, for example `openai/gpt-4o`
        messages:
          items:
            additionalProperties: true
            type: object
          type: array
          title: Messages
          description: openai standard message format
      additionalProperties: true
      type: object
      required:
        - model
        - messages
      title: ChatCompletionRequest
    ChatCompletion:
      properties:
        id:
          type: string
          title: Id
        object:
          type: string
          const: chat.completion
          title: Object
          default: chat.completion
        choices:
          items:
            $ref: '#/components/schemas/openai__types__chat__chat_completion__Choice'
          type: array
          title: Choices
        created:
          type: integer
          title: Created
        model:
          type: string
          title: Model
        service_tier:
          anyOf:
            - type: string
              enum:
                - auto
                - default
                - flex
                - scale
                - priority
            - type: 'null'
          title: Service Tier
        system_fingerprint:
          anyOf:
            - type: string
            - type: 'null'
          title: System Fingerprint
        usage:
          anyOf:
            - $ref: '#/components/schemas/CompletionUsage'
            - type: 'null'
      additionalProperties: true
      type: object
      required:
        - id
        - choices
        - created
        - model
    ChatCompletionChunk:
      properties:
        id:
          type: string
          title: Id
        object:
          type: string
          const: chat.completion.chunk
          title: Object
          default: chat.completion.chunk
        choices:
          items:
            $ref: >-
              #/components/schemas/openai__types__chat__chat_completion_chunk__Choice
          type: array
          title: Choices
        created:
          type: integer
          title: Created
        model:
          type: string
          title: Model
        service_tier:
          anyOf:
            - type: string
              enum:
                - auto
                - default
                - flex
                - scale
                - priority
            - type: 'null'
          title: Service Tier
        system_fingerprint:
          anyOf:
            - type: string
            - type: 'null'
          title: System Fingerprint
        usage:
          anyOf:
            - $ref: '#/components/schemas/CompletionUsage'
            - type: 'null'
      additionalProperties: true
      type: object
      required:
        - id
        - choices
        - created
        - model
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    openai__types__chat__chat_completion__Choice:
      properties:
        finish_reason:
          type: string
          enum:
            - stop
            - length
            - tool_calls
            - content_filter
            - function_call
          title: Finish Reason
        index:
          type: integer
          title: Index
        logprobs:
          anyOf:
            - $ref: '#/components/schemas/ChoiceLogprobs'
            - type: 'null'
        message:
          $ref: '#/components/schemas/ChatCompletionMessage'
      additionalProperties: true
      type: object
      required:
        - finish_reason
        - index
        - message
      title: Choice
    CompletionUsage:
      properties:
        completion_tokens:
          type: integer
          title: Completion Tokens
        prompt_tokens:
          type: integer
          title: Prompt Tokens
        total_tokens:
          type: integer
          title: Total Tokens
        completion_tokens_details:
          anyOf:
            - $ref: '#/components/schemas/CompletionTokensDetails'
            - type: 'null'
        prompt_tokens_details:
          anyOf:
            - $ref: '#/components/schemas/PromptTokensDetails'
            - type: 'null'
      additionalProperties: true
      type: object
      required:
        - completion_tokens
        - prompt_tokens
        - total_tokens
      title: CompletionUsage
      description: Usage statistics for the completion request.
    openai__types__chat__chat_completion_chunk__Choice:
      properties:
        delta:
          $ref: '#/components/schemas/ChoiceDelta'
        finish_reason:
          anyOf:
            - type: string
              enum:
                - stop
                - length
                - tool_calls
                - content_filter
                - function_call
            - type: 'null'
          title: Finish Reason
        index:
          type: integer
          title: Index
        logprobs:
          anyOf:
            - $ref: '#/components/schemas/ChoiceLogprobs'
            - type: 'null'
      additionalProperties: true
      type: object
      required:
        - delta
        - index
      title: Choice
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
              - type: string
              - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
        input:
          title: Input
        ctx:
          type: object
          title: Context
          additionalProperties: true
      type: object
      required:
        - loc
        - msg
        - type
      title: ValidationError
    ChoiceLogprobs:
      properties:
        content:
          anyOf:
            - items:
                $ref: '#/components/schemas/ChatCompletionTokenLogprob'
              type: array
            - type: 'null'
          title: Content
        refusal:
          anyOf:
            - items:
                $ref: '#/components/schemas/ChatCompletionTokenLogprob'
              type: array
            - type: 'null'
          title: Refusal
      additionalProperties: true
      type: object
      title: ChoiceLogprobs
      description: Log probability information for the choice.
    ChatCompletionMessage:
      properties:
        content:
          anyOf:
            - type: string
            - type: 'null'
          title: Content
        refusal:
          anyOf:
            - type: string
            - type: 'null'
          title: Refusal
        role:
          type: string
          const: assistant
          title: Role
        annotations:
          anyOf:
            - items:
                $ref: '#/components/schemas/Annotation'
              type: array
            - type: 'null'
          title: Annotations
        audio:
          anyOf:
            - $ref: '#/components/schemas/ChatCompletionAudio'
            - type: 'null'
        function_call:
          anyOf:
            - $ref: '#/components/schemas/FunctionCall'
            - type: 'null'
        tool_calls:
          anyOf:
            - items:
                anyOf:
                  - $ref: '#/components/schemas/ChatCompletionMessageFunctionToolCall'
                  - $ref: '#/components/schemas/ChatCompletionMessageCustomToolCall'
              type: array
            - type: 'null'
          title: Tool Calls
      additionalProperties: true
      type: object
      required:
        - role
      title: ChatCompletionMessage
      description: A chat completion message generated by the model.
    CompletionTokensDetails:
      properties:
        accepted_prediction_tokens:
          anyOf:
            - type: integer
            - type: 'null'
          title: Accepted Prediction Tokens
        audio_tokens:
          anyOf:
            - type: integer
            - type: 'null'
          title: Audio Tokens
        reasoning_tokens:
          anyOf:
            - type: integer
            - type: 'null'
          title: Reasoning Tokens
        rejected_prediction_tokens:
          anyOf:
            - type: integer
            - type: 'null'
          title: Rejected Prediction Tokens
      additionalProperties: true
      type: object
      title: CompletionTokensDetails
      description: Breakdown of tokens used in a completion.
    PromptTokensDetails:
      properties:
        audio_tokens:
          anyOf:
            - type: integer
            - type: 'null'
          title: Audio Tokens
        cached_tokens:
          anyOf:
            - type: integer
            - type: 'null'
          title: Cached Tokens
      additionalProperties: true
      type: object
      title: PromptTokensDetails
      description: Breakdown of tokens used in the prompt.
    ChoiceDelta:
      properties:
        content:
          anyOf:
            - type: string
            - type: 'null'
          title: Content
        function_call:
          anyOf:
            - $ref: '#/components/schemas/ChoiceDeltaFunctionCall'
            - type: 'null'
        refusal:
          anyOf:
            - type: string
            - type: 'null'
          title: Refusal
        role:
          anyOf:
            - type: string
              enum:
                - developer
                - system
                - user
                - assistant
                - tool
            - type: 'null'
          title: Role
        tool_calls:
          anyOf:
            - items:
                $ref: '#/components/schemas/ChoiceDeltaToolCall'
              type: array
            - type: 'null'
          title: Tool Calls
      additionalProperties: true
      type: object
      title: ChoiceDelta
      description: A chat completion delta generated by streamed model responses.
    ChatCompletionTokenLogprob:
      properties:
        token:
          type: string
          title: Token
        bytes:
          anyOf:
            - items:
                type: integer
              type: array
            - type: 'null'
          title: Bytes
        logprob:
          type: number
          title: Logprob
        top_logprobs:
          items:
            $ref: '#/components/schemas/TopLogprob'
          type: array
          title: Top Logprobs
      additionalProperties: true
      type: object
      required:
        - token
        - logprob
        - top_logprobs
      title: ChatCompletionTokenLogprob
    Annotation:
      properties:
        type:
          type: string
          const: url_citation
          title: Type
        url_citation:
          $ref: >-
            #/components/schemas/openai__types__chat__chat_completion_message__AnnotationURLCitation
      additionalProperties: true
      type: object
      required:
        - type
        - url_citation
      title: Annotation
      description: A URL citation when using web search.
    ChatCompletionAudio:
      properties:
        id:
          type: string
          title: Id
        data:
          type: string
          title: Data
        expires_at:
          type: integer
          title: Expires At
        transcript:
          type: string
          title: Transcript
      additionalProperties: true
      type: object
      required:
        - id
        - data
        - expires_at
        - transcript
      title: ChatCompletionAudio
      description: >-
        If the audio output modality is requested, this object contains data

        about the audio response from the model. [Learn
        more](https://platform.openai.com/docs/guides/audio).
    FunctionCall:
      properties:
        arguments:
          type: string
          title: Arguments
        name:
          type: string
          title: Name
      additionalProperties: true
      type: object
      required:
        - arguments
        - name
      title: FunctionCall
      description: >-
        Deprecated and replaced by `tool_calls`.


        The name and arguments of a function that should be called, as generated
        by the model.
    ChatCompletionMessageFunctionToolCall:
      properties:
        id:
          type: string
          title: Id
        function:
          $ref: '#/components/schemas/Function'
        type:
          type: string
          const: function
          title: Type
      additionalProperties: true
      type: object
      required:
        - id
        - function
        - type
      title: ChatCompletionMessageFunctionToolCall
      description: A call to a function tool created by the model.
    ChatCompletionMessageCustomToolCall:
      properties:
        id:
          type: string
          title: Id
        custom:
          $ref: '#/components/schemas/Custom'
        type:
          type: string
          const: custom
          title: Type
      additionalProperties: true
      type: object
      required:
        - id
        - custom
        - type
      title: ChatCompletionMessageCustomToolCall
      description: A call to a custom tool created by the model.
    ChoiceDeltaFunctionCall:
      properties:
        arguments:
          anyOf:
            - type: string
            - type: 'null'
          title: Arguments
        name:
          anyOf:
            - type: string
            - type: 'null'
          title: Name
      additionalProperties: true
      type: object
      title: ChoiceDeltaFunctionCall
      description: >-
        Deprecated and replaced by `tool_calls`.


        The name and arguments of a function that should be called, as generated
        by the model.
    ChoiceDeltaToolCall:
      properties:
        index:
          type: integer
          title: Index
        id:
          anyOf:
            - type: string
            - type: 'null'
          title: Id
        function:
          anyOf:
            - $ref: '#/components/schemas/ChoiceDeltaToolCallFunction'
            - type: 'null'
        type:
          anyOf:
            - type: string
              const: function
            - type: 'null'
          title: Type
      additionalProperties: true
      type: object
      required:
        - index
      title: ChoiceDeltaToolCall
    TopLogprob:
      properties:
        token:
          type: string
          title: Token
        bytes:
          anyOf:
            - items:
                type: integer
              type: array
            - type: 'null'
          title: Bytes
        logprob:
          type: number
          title: Logprob
      additionalProperties: true
      type: object
      required:
        - token
        - logprob
      title: TopLogprob
    openai__types__chat__chat_completion_message__AnnotationURLCitation:
      properties:
        end_index:
          type: integer
          title: End Index
        start_index:
          type: integer
          title: Start Index
        title:
          type: string
          title: Title
        url:
          type: string
          title: Url
      additionalProperties: true
      type: object
      required:
        - end_index
        - start_index
        - title
        - url
      title: AnnotationURLCitation
      description: A URL citation when using web search.
    Function:
      properties:
        arguments:
          type: string
          title: Arguments
        name:
          type: string
          title: Name
      additionalProperties: true
      type: object
      required:
        - arguments
        - name
      title: Function
      description: The function that the model called.
    Custom:
      properties:
        input:
          type: string
          title: Input
        name:
          type: string
          title: Name
      additionalProperties: true
      type: object
      required:
        - input
        - name
      title: Custom
      description: The custom tool that the model called.
    ChoiceDeltaToolCallFunction:
      properties:
        arguments:
          anyOf:
            - type: string
            - type: 'null'
          title: Arguments
        name:
          anyOf:
            - type: string
            - type: 'null'
          title: Name
      additionalProperties: true
      type: object
      title: ChoiceDeltaToolCallFunction
  securitySchemes:
    APIKeyHeader:
      type: apiKey
      in: header
      name: x-api-key

````