> ## Documentation Index > Fetch the complete documentation index at: https://docs.gp.scale.com/llms.txt > Use this file to discover all available pages before exploring further. # Create Completion > ### Description Given a user's input, runs LLM inference to produce the model's response. ### Details LLM [completions](https://scale-egp.readme.io/docs/completions-1) have many use cases, such as content summarization, question-answering, and text generation. The `model` parameter determines which LLM will be used to generate the completion. Keep in mind that different models have varying sizes, costs, and may perform differently across different tasks. The user input, commonly referred to as the "prompt", is a required field in the request body. The quality of the model's response can vary greatly depending on the input prompt. Good prompt engineering can significantly enhance the response quality. If you encounter suboptimal results, consider writing more specific instructions or providing examples to the LLM before trying more expensive techniques such as swapping in other models or finetuning. By default, the endpoint will return the entire response as one whole object. If you would prefer to stream the completion in real-time, you can achieve this by setting the `stream` flag to `true`. ## OpenAPI ````yaml https://app.stainlessapi.com/api/spec/documented/sgp/openapi.yml post /v4/completions openapi: 3.1.0 info: title: EGP API V4 description: >- This is the parent API for all EGP APIs. If you are looking for the EGP API, please go to https://api.egp.scale.com/docs. contact: name: Scale Generative AI Platform url: https://scale.com/genai-platform version: 0.1.0 servers: - url: https://api.egp.scale.com security: [] tags: - name: Models description: Model API. paths: /v4/completions: post: tags: - Completions summary: Create Completion description: >- ### Description Given a user's input, runs LLM inference to produce the model's response. ### Details LLM [completions](https://scale-egp.readme.io/docs/completions-1) have many use cases, such as content summarization, question-answering, and text generation. The `model` parameter determines which LLM will be used to generate the completion. Keep in mind that different models have varying sizes, costs, and may perform differently across different tasks. The user input, commonly referred to as the "prompt", is a required field in the request body. The quality of the model's response can vary greatly depending on the input prompt. Good prompt engineering can significantly enhance the response quality. If you encounter suboptimal results, consider writing more specific instructions or providing examples to the LLM before trying more expensive techniques such as swapping in other models or finetuning. By default, the endpoint will return the entire response as one whole object. If you would prefer to stream the completion in real-time, you can achieve this by setting the `stream` flag to `true`. operationId: POST-V4-/v2/completions requestBody: content: application/json: schema: $ref: '#/components/schemas/CreateCompletionRequest' required: true responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/CreateCompletionResponse' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' x-codeSamples: - lang: Python source: |- import os from scale_gp import SGPClient client = SGPClient( api_key=os.environ.get("SGP_API_KEY"), # This is the default and can be omitted ) for completion in client.completions.create( model="gpt-oss-120b", prompt="prompt", ): print(completion) - lang: Go source: "package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/stainless-sdks/sgp-go\"\n\t\"github.com/stainless-sdks/sgp-go/option\"\n)\n\nfunc main() {\n\tclient := sgp.NewClient(\n\t\toption.WithAPIKey(\"My API Key\"),\n\t)\n\tcompletionsResponse, err := client.Completions.New(context.TODO(), sgp.CompletionNewParams{\n\t\tModel: sgp.F(sgp.CompletionNewParamsModelGptOss120b),\n\t\tPrompt: sgp.F(\"prompt\"),\n\t})\n\tif err != nil {\n\t\tpanic(err.Error())\n\t}\n\tfmt.Printf(\"%+v\\n\", completionsResponse.Completion)\n}\n" components: schemas: CreateCompletionRequest: properties: account_id: title: Account Id description: >- The account ID to use for usage tracking. This will be gradually enforced. type: string model: anyOf: - type: string enum: - gpt-oss-120b - gpt-oss-20b - gpt-4 - gpt-4-0613 - gpt-4-32k - gpt-4-32k-0613 - gpt-4-vision-preview - gpt-4o - gpt-4o-mini - gpt-4o-2024-08-06 - gpt-3.5-turbo - gpt-3.5-turbo-0613 - gpt-3.5-turbo-16k - gpt-3.5-turbo-16k-0613 - gemini-pro - gemini-1.5-pro-001 - gemini-1.5-pro-002 - gemini-1.5-pro-preview-0409 - gemini-1.5-pro-preview-0514 - text-davinci-003 - text-davinci-002 - text-curie-001 - text-babbage-001 - text-ada-001 - claude-instant-1 - claude-instant-1.1 - claude-2 - claude-2.0 - llama-7b - llama-2-7b - llama-2-7b-chat - llama-2-13b - llama-2-13b-chat - llama-2-70b - llama-2-70b-chat - llama-3-8b - llama-3-8b-instruct - llama-3-1-8b-instruct - llama-3-1-70b-instruct - llama-3-70b-instruct - llama-3-2-1b-instruct - llama-3-2-3b-instruct - llama-3-3-70b-instruct - Meta-Llama-3-8B-Instruct-RMU - Meta-Llama-3-8B-Instruct-RR - Meta-Llama-3-8B-Instruct-DERTA - Meta-Llama-3-8B-Instruct-LAT - falcon-7b - falcon-7b-instruct - falcon-40b - falcon-40b-instruct - mpt-7b - mpt-7b-instruct - flan-t5-xxl - mistral-7b - mistral-7b-instruct - mixtral-8x7b - mixtral-8x7b-instruct - mixtral-8x22b-instruct - llm-jp-13b-instruct-full - llm-jp-13b-instruct-full-dolly - zephyr-7b-alpha - zephyr-7b-beta - zephyr-cat-merged - codellama-7b - codellama-7b-instruct - codellama-13b - codellama-13b-instruct - codellama-34b - codellama-34b-instruct - phi-3-mini-4k-instruct - phi-3-cat-merged - dolphin-2.9-llama3-8b - dolphin-2.9-llama3-70b - defense-llama-3-8b-instruct - donovan-combat-llama - type: string title: Model description: >- The ID of the model to use for completions. Users have two options: - Option 1: Use one of the supported models from the dropdown. - Option 2: Enter the ID of a custom model. Note: For custom models we currently only support models finetuned using using the Scale-hosted LLM-Engine API. prompt: type: string title: Prompt description: >- Prompt for which to generate the completion. Good prompt engineering is crucial to getting performant results from the model. If you are having trouble getting the model to perform well, try writing a more specific prompt here before trying more expensive techniques such as swapping in other models or finetuning the underlying LLM. images: title: Images description: >- List of image urls to be used for image based completions. Leave empty for text based completions. items: $ref: '#/components/schemas/ImageCompletionRequests' type: array model_parameters: $ref: '#/components/schemas/ModelParameters' description: >- Configuration parameters for the completion model, such as temperature, max_tokens, and stop_sequences. If not specified, the default value are: - temperature: 0.2 - max_tokens: None (limited by the model's max tokens) - stop_sequences: None default: temperature: 0.2 stream: type: boolean title: Stream description: |- Whether or not to stream the response. Setting this to True will stream the completion in real-time. default: false type: object required: - model - prompt title: CreateCompletionRequest CreateCompletionResponse: properties: completion: $ref: >- #/components/schemas/egp_api_backend__server__api__models__egp_models__Completion description: The actual completion text and the finish reason. token_usage: $ref: '#/components/schemas/TokenUsage' description: >- Token usage numbers. If streaming, this field is null until the stream completes, at which point it will be populated (if supported). type: object required: - completion title: CreateCompletionResponse HTTPValidationError: properties: detail: items: $ref: '#/components/schemas/ValidationError' type: array title: Detail type: object title: HTTPValidationError ImageCompletionRequests: properties: image_url: type: string title: Image Url description: Image URL to run image completion on. detail: title: Detail description: Detail to run image completion with. Defaults to auto default: auto type: string type: object required: - image_url title: ImageCompletionRequests ModelParameters: properties: temperature: title: Temperature description: >- What sampling temperature to use, between [0, 1]. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Setting temperature=0.0 will enable fully deterministic (greedy) sampling. Temperature is ignored for OpenAI reasoning models. default: 0.2 type: number maximum: 1 minimum: 0 stop_sequences: title: Stop Sequences description: >- List of up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence. items: type: string type: array maxItems: 4 max_tokens: title: Max Tokens description: > The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model's context length. If not, specified, max_tokens will be determined based on the model used: | Model API family | Model API default | EGP applied default | | --- | --- | --- | | OpenAI Completions | [`16`](https://platform.openai.com/docs/api-reference/completions/create#max_tokens) | `context window - prompt size` | | OpenAI Chat Completions | [`context window - prompt size`](https://platform.openai.com/docs/api-reference/chat/create#max_tokens) | `context window - prompt size` | | LLM Engine | [`max_new_tokens`](https://github.com/scaleapi/launch-python-client/blob/207adced1c88c1c2907266fa9dd1f1ff3ec0ea5b/launch/client.py#L2910) parameter is required | `100` | | Anthropic Claude 2 | [`max_tokens_to_sample`](https://docs.anthropic.com/claude/reference/complete_post) parameter is required | `10000` | type: integer topP: title: Topp description: >- The cumulative probability cutoff for token selection. Lower values mean sampling from a smaller, more top-weighted nucleus. Available for models from Anthropic, Google, Mistral, LLM Engine, and OpenAI. type: number topK: title: Topk description: >- Sample from the k most likely next tokens at each step. Lower k focuses on higher probability tokens. Available for models from Anthropic, Google, and LLM Engine. type: integer frequency_penalty: title: Frequency Penalty description: >- Penalize tokens based on how much they have already appeared in the text. Positive values encourage the model to generate new tokens and negative values encourage the model to repeat tokens. Available for models from LLM Engine, and OpenAI type: number presence_penalty: title: Presence Penalty description: >- Penalize tokens based on if they have already appeared in the text. Positive values encourage the model to generate new tokens and negative values encourage the model to repeat tokens. Available for models from LLM Engine, and OpenAI. type: number reasoning_effort: title: Reasoning Effort description: >- The effort level for the model to reason. Available for models from OpenAI. type: string enum: - low - medium - high type: object title: ModelParameters egp_api_backend__server__api__models__egp_models__Completion: properties: text: type: string title: Text description: >- Completion text. If streaming, this field will contain each packet of text. finish_reason: title: Finish Reason description: Reason the LLM finished generating text. type: string response_metadata: title: Response Metadata description: Additional metadata returned from the completion response additionalProperties: true type: object type: object required: - text title: Completion TokenUsage: properties: prompt: title: Prompt description: Number of tokens in the prompt. type: integer completion: title: Completion description: Number of tokens in the completion. type: integer total: type: integer title: Total description: Total number of tokens in both the prompt and the completion. type: object required: - total title: TokenUsage ValidationError: properties: loc: items: anyOf: - type: string - type: integer type: array title: Location msg: type: string title: Message type: type: string title: Error Type input: title: Input ctx: type: object title: Context additionalProperties: true type: object required: - loc - msg - type title: ValidationError ````