> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lytix.co/llms.txt
> Use this file to discover all available pages before exploring further.
# [Deprecated] OptiModel
> Use Lytix to manage and call your models
**🚨 Note** We have fully transitioned to use the OpenAI SDK as our main interaction point. Please see [here](/Quickstart/openai-integration) for more information.
Using lytix to manage your [OptiModel](https://github.com/Lytix-Labs/optimodel/tree/master) server is as simple as creating a lytix API key.
**Prerequisite** First create a lytix account [here](https://lab.lytix.co/home/login)
## Create a Lytix API Key
Start by creating and noting down a lytix api key. See instructions [here](/api-key-setup)
#### (Reccomended) Set you API key via the environment variable
```sh
export LX_API_KEY=
```
#### Set your API key in the code
```py Python
from optimodel import LytixCreds
LytixCreds.setAPIKey('')
```
```ts Typescript
import { LytixCred } from "@lytix/client";
LytixCreds.setAPIKey("sk-1234");
```
## (Optional) Add a new Provider
Note: You optinally skip this step and can send credentials with every SDK call. See instructions [here](/Quickstart/optimodel-getting-started#using-local-credentials)
Before you can start making LLMs calls, you'll first need to setup a new provider [here](https://lab.lytix.co/home/settings/model-providers)
**Note** Access to models is limited to what providers you have setup. For example, if you only setup OpenAI, you will not be able to call llama3 models.
## Install the SDK
```sh Python
pip3 install optimodel-py
```
```sh JavaScript
npm i @lytix/client
```
## Call The SDK
Now you are ready to make your first call by passing in the `LX_API_KEY` environment variable.
```py Python Example
from optimodel import queryModel, listModels, ModelMessage, ModelTypes
prompt = "Hello How are you?"
response = await queryModel(
model=ModelTypes.llama_3_8b_instruct,
messages=[
ModelMessage(
role="system",
content="You are a helpful assistant. Always respond in JSON syntax",
),
ModelMessage(role="user", content=prompt),
],
)
print("Got response:", response)
```
```ts Typescript
import { ModelTypes, SpeedPriority } from "@lytix/client";
import { queryModel } from "@lytix/client";
/**
* Simple validator to check if the response is JSON
*/
function validator(x: string): boolean {
try {
JSON.parse(x);
return true;
} catch {
return false;
}
}
const main = async () => {
const prompt = "Hello How are you?";
const response = await queryModel({
model: ModelTypes.gpt_4o_mini,
messages: [
{ role: "user", content: prompt },
{ role: "system", content: "Always respond with a JSON object" },
],
speedPriority: SpeedPriority.low,
temperature: 0.5,
fallbackModels: [ModelTypes.gpt_4o],
validator: validator,
maxGenLen: 256,
});
console.log(`Got response: ${response}`);
};
main();
```
Just remember to pass your `LX_API_KEY` when starting your program as an environment variable.
```sh Python
> LX_API_KEY= python3 your_script.py
```
```sh Typescript
> LX_API_KEY= node your_script.js
```
## Using Local Credentials
If you want to use local credentials, you can do so by passing in an array of `credentials` to the `queryModel` function.
```ts Python
response = await queryModel(
...the same parameters as above
credentials: [creds],
)
```
```ts Typescript
const response = await queryModel({
...the same parameters as above
credentials: [creds],
});
```
Where each items in `credentials` can be any of the following
### **AWS Credentials**
```ts Python
from optimodel import AWSBedrockCredentials
creds = AWSBedrockCredentials(
awsAccessKeyId="$AWS_ACCESS_KEY_ID",
awsSecretKey="$AWS_SECRET_KEY",
awsRegion="$AWS_REGION",
)
```
```ts Typescript
import { AWSBedrockCredentials } from "@lytix/client";
const creds: AWSBedrockCredentials = {
awsAccessKeyId: "$AWS_ACCESS_KEY_ID",
awsSecretKey: "$AWS_SECRET_KEY",
awsRegion: "$AWS_REGION",
}
```
### **Open AI**
```ts Python
from optimodel import OpenAICredentials
creds = OpenAICredentials(
openAiKey="$OPENAI_API_KEY",
)
```
```ts Typescript
import { OpenAICredentials } from "@lytix/client";
const creds: OpenAICredentials = {
openAiKey: "$OPENAI_API_KEY",
}
```
### **Anthropic**
```ts Python
from optimodel import AnthropicCredentials
creds = AnthropicCredentials(
anthropicApiKey="$ANTHROPIC_API_KEY",
)
```
```ts Typescript
import { AnthropicCredentials } from "@lytix/client";
const creds: AnthropicCredentials = {
anthropicApiKey: "$ANTHROPIC_API_KEY",
}
```
### **TogetherAI**
```ts Python
from optimodel import TogetherAICredentials
creds = TogetherAICredentials(
togetherApiKey="$TOGETHER_API_KEY",
)
```
```ts Typescript
import { TogetherAICredentials } from "@lytix/client";
const creds: TogetherAICredentials = {
togetherApiKey: "$TOGETHER_API_KEY",
}
```
### **Groq**
```ts Python
from optimodel import GroqCredentials
creds = GroqCredentials(
groqApiKey="$ANTHROPIC_API_KEY",
)
```
```ts Typescript
import { GroqCredentials } from "@lytix/client";
const creds: GroqCredentials = {
groqApiKey: "$ANTHROPIC_API_KEY",
}
```
### **MistralAI**
```ts Python
from optimodel import MistralAICredentials
creds = MistralAICredentials(
mistralApiKey="$MISTRAL_API_KEY",
)
```
```ts Typescript
import { MistralAICredentials } from "@lytix/client";
const creds: MistralAICredentials = {
mistralApiKey: "$MISTRAL_API_KEY",
}
```
## Extra Parameters
The following extra parameters are available to pass to the `queryModel` function:
`speedPriority`: This can be used to control how OptiModel should prioritize the request. If set to `high` it will not focus on cost
```py Python
speedPriority = "low" | "high"
```
```ts Typescript
speedPriority = "low" | "high";
```
`validator`: This is a function that will be used to validate the response. If the validator returns `False` the request will be retried. *Note:* You must pass `fallbackModels` if you use a validator.
```py Python
validator = Callable[[str], bool]
```
```ts Typescript
validator = (response: string) => boolean;
```
`fallbackModels`: This a list of other models to fallback to if the first model fails the validator.
```py Python
fallbackModels = List[ModelTypes]
```
```ts Typescript
fallbackModels = ModelTypes[];
```
`maxGenLen`: This is the maximum length of the response, if the model response is longer than this, it will be truncated. This will check against the contig, so for example if you pass a value of 1 million, and no provider will be able to generate a response of that length, the request will fail.
```py Python
maxGenLen = int
```
```ts Typescript
maxGenLen = number;
```
`jsonMode`: This will enable json mode for the request. This is useful if you want to pass in a json object as a prompt.
```py Python
jsonMode = bool
```
```ts Typescript
jsonMode = boolean;
```
`provider`: You can optionally force a specific provder to be used. This is useful if you have multiple providers setup and want to force a specific one to be used.
```py Python
provider = "openai" | "groq" | "bedrock" | "anthropic" | "together"
```
```ts Typescript
provider = "openai" | "groq" | "bedrock" | "anthropic" | "together";
```
`userId`: You can optionally pass in a userId to track user requests across the Lytix platform. This is often a unique user identifier.
```py Python
userId = str
```
```ts Typescript
userId = string;
```
`sessionId`: You can optionally pass in a sessionId to track sessions or workflows. Events with the same sessionId will be grouped together.
```py Python
sessionId = str
```
```ts Typescript
sessionId = string;
```
`workflowName`: You can optionally pass in a workflowName to track sessions or workflows. Events with the same workflowName will be grouped together.
```py Python
workflowName = str
```
```ts Typescript
workflowName = string;
```