Recipes
Create Completion and Chat Completions
Use this recipe to use the Scale GenAI Platform SDK to perform completion and chat completions
https://pypi.org/project/scale-egp/
# Prerequisite: pip install -U scale-egp
from scale_gp import SGPClient
from scale_gp.types.completion_create_params import ModelParameters
from scale_gp.types.chat_completion_create_params import (
ModelParameters as ChatModelParameters,
MessageEgpAPIBackendServerAPIModelsEgpModelsUserMessage,
)
Fetch your API Key from: https://gp.scale.com/admin/api-key
Fetch your Account ID from: https://gp.scale.com/admin/accounts
All resources you interact with using this client will belong to this account.
Note: If you are using your own VPC-deployed version of Scale GP, you will have a different endpoint_url. For users of our multi-tenant platform, use https://gp.scale.com
client = SGPClient(api_key=api_key)
Define model name and parameters to perform a completion.
model_parameters = ModelParameters(max_tokens=200, temperature=0.5, top_k=1, top_p=1)
completion = client.completions.create(
model="gpt-4",
prompt="Why is the sky blue?",
account_id=account_id,
model_parameters=model_parameters,
)
print(completion.completion.text)
Call a chat model to perform a completion using a message and the parameters defined above. Chat completion allows for a series of messages to be sent to the model, serving as a conversation between the user and the model
chat_model_parameters = ChatModelParameters(max_tokens=200, temperature=0.5, top_k=1, top_p=1)
message = MessageEgpAPIBackendServerAPIModelsEgpModelsUserMessage(role="user", content="What is the capital of Canada?")
chat_completion = client.chat_completions.create(
model="gpt-4",
messages=[
message
], # messages is a list of historical messages in the conversation, with roles usually alternating between user, assistant and system
account_id=account_id,
model_parameters=model_parameters,
instructions="Answer the question like an elementary school teacher.",
)
print(chat_completion.chat_completion.message.content)