Use Lytix to manage and call your models
LX_API_KEY
environment variable.
LX_API_KEY
when starting your program as an environment variable.
credentials
to the queryModel
function.
credentials
can be any of the following
queryModel
function:
speedPriority
: This can be used to control how OptiModel should prioritize the request. If set to high
it will not focus on cost
validator
: This is a function that will be used to validate the response. If the validator returns False
the request will be retried. Note: You must pass fallbackModels
if you use a validator.
fallbackModels
: This a list of other models to fallback to if the first model fails the validator.
maxGenLen
: This is the maximum length of the response, if the model response is longer than this, it will be truncated. This will check against the contig, so for example if you pass a value of 1 million, and no provider will be able to generate a response of that length, the request will fail.
jsonMode
: This will enable json mode for the request. This is useful if you want to pass in a json object as a prompt.
provider
: You can optionally force a specific provder to be used. This is useful if you have multiple providers setup and want to force a specific one to be used.
userId
: You can optionally pass in a userId to track user requests across the Lytix platform. This is often a unique user identifier.
sessionId
: You can optionally pass in a sessionId to track sessions or workflows. Events with the same sessionId will be grouped together.
workflowName
: You can optionally pass in a workflowName to track sessions or workflows. Events with the same workflowName will be grouped together.