> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lytix.co/llms.txt
> Use this file to discover all available pages before exploring further.
# OpenAI Integration
> How to integrate OpenAI with Lytix
Use Lytix to manage your evaluation and usage diretly with the OpenAI SDK. Gain access to models across providers and manage your usage and billing.
# Quickstart
**Prerequisite** First create a lytix account [here](https://lab.lytix.co/home/login)
### Create a Lytix API Key
Start by creating and noting down a lytix api key. See instructions [here](/api-key-setup)
### Update your OpenAI SDK
With 2 lines you can start using Lytix to manage your evaluation and usage.
```py Python
from openai import OpenAI
client = OpenAI(
# Update your base url to the lytix proxy
base_url=f"https://api.lytix.co/proxy/v2/openai",
# Update your api key to the lytix api key
api_key="$LYTIX_API_KEY",
default_headers={
# Move your openai key to the default headers
"openaiKey": "$OPENAI_API_KEY"
},
)
```
```ts Typescript
import OpenAI from "openai";
const client = new OpenAI({
// Update your base url to the lytix proxy
baseURL: "https://api.lytix.co/proxy/v2/openai",
// Update your api key to the lytix api key
apiKey: '$LYTIX_API_KEY',
defaultHeaders: {
// Move your openai key to the default headers
openaiKey: '$OPENAI_API_KEY',
},
});
```
**🇪🇺 Note** You will need to use `https://eu.api.lytix.co/proxy/v2/openai` if you are in the EU region.
### Optional Fields
Optimodel supports a variety of optional parameters to help you get the best results.
```py Python
response = client.chat.completions.create(
...,
extra_body={
"lytix-fallbackModels": ...
...
},
extra_headers={
"sessionId": "1234567890",
"userId": "sid@lytix.co",
"workflowName": "test-workflow",
},
)
```
```ts Typescript
const chatCompletion = await client.chat.completions.create(
{
...,
// @ts-ignore Add any body parameters here
"lytix-guards": [
{
guardName: "META_LLAMA_PROMPT_GUARD_86M",
jailbreakThreshold: 0.9999,
guardType: "preQuery",
blockRequest: true,
blockRequestMessage: "You are not allowed to respond to this message",
} as LLamaPromptGuardConfig,
],
},
{
headers: {
userId: "sid@lytix.co",
...
},
}
);
```
You will need to use the `optimodel-py`/`@lytix/client` package to use these parameters.
```sh Python
pip3 install optimodel-py
```
```sh Typescript
npm i @lytix/client
```
The following optional parameters are supported:
#### Guards
`lytix-guards`: Pass in a list of fallback models to use
```py Python
from optimodel_server_types import LLamaPromptGuardConfig
extra_body={
"lytix-guards": [LLamaPromptGuardConfig(
guardName="LLamaPromptGuard",
jailbreakThreshold=0.9999,
guardType="preQuery", # You'll likely only want to guard the input here
).dict()]
}
```
```ts Typescript
import { LLamaPromptGuardConfig } from "@lytix/client";
const chatCompletion = await client.chat.completions.create(
{
...,
"lytix-guards": [
{
guardName: "META_LLAMA_PROMPT_GUARD_86M",
jailbreakThreshold: 0.9999,
guardType: "preQuery",
blockRequest: true,
blockRequestMessage: "You are not allowed to respond to this message",
} as LLamaPromptGuardConfig,
],
}
)
```
See [here](https://github.com/Lytix-Labs/optimodel/tree/master?tab=readme-ov-file#guards) for a list of all supported guards
#### Fallback Models
`lytix-fallbackModels`: Pass in a list of extra models to try if the primary model fails. This can be helpful in mitigating provider outages.
```py Python
from optimodel_server_types import ModelTypes
...
extra_body={
"lytix-fallbackModels": [ModelTypes.claude_3_5_sonnet.name, ...]
}
```
```ts Typescript
import { ModelTypes } from "@lytix/client";
...
const chatCompletion = await client.chat.completions.create(
{
...,
"lytix-fallbackModels": [
ModelTypes.claude_3_5_sonnet,
],
}
)
```
#### Speed Priority
`lytix-speedPriority`: Pass in a speed priority to use
```py Python
extra_body={
"lytix-speedPriority": "low"
}
```
```ts Typescript
import { SpeedPriority } from "@lytix/client";
...
const chatCompletion = await client.chat.completions.create(
{
...,
"lytix-speedPriority": SpeedPriority.low,
}
)
```
If set to `low`, optimodel will choose the cheapest possible model across all providers (for example if you have two providers `bedrock` and `anthropic` that both offer `claude-3-opus`, optimodel will choose the `claude-3-opus` model with the lowest price regardless of which provider is faster). If set to `high`, optimodel will choose the fastest possible model across all providers.
#### Provider
`lytix-provider`: Pass in a provider to use
```py Python
from optimodel_server_types import Providers
...
extra_body={
"lytix-provider": ProviderTypes.bedrock.name
}
```
```ts Typescript
import { ProviderTypes } from "@lytix/client";
...
const chatCompletion = await client.chat.completions.create(
{
...,
"lytix-provider": ProviderTypes.bedrock.name
}
)
```
Explicitly specify a provider to use incase you have multiple providers available for a specific model and want to force a specific one.
You can also track workflows, users and sessions to get a better understanding of your users and how they interact with your models.
#### SessionId
`sessionId`: A unique identifier for the session.
```py Python
extra_headers={
"sessionId": "1234567890"
}
```
```ts Typescript
import { LLamaPromptGuardConfig } from "@lytix/client";
...
const chatCompletion = await client.chat.completions.create(
{
...,
},
{
headers: {
sessionId: "1234567890",
},
}
)
```
#### UserId
`userId`: A unique identifier for the user.
```py Python
extra_headers={
"userId": "sid@lytix.co"
}
```
```ts Typescript
...
const chatCompletion = await client.chat.completions.create(
{
...,
},
{
headers: {
userId: "sid@lytix.co",
},
}
)
```
#### WorkflowName
`workflowName`: A unique identifier for the workflow. If this workflow does not exist, it will be created and can be viewed [here](https://lab.lytix.co/home/workflows)
```py Python
extra_headers={
"workflowName": "test-workflow"
}
```
```ts Typescript
...
const chatCompletion = await client.chat.completions.create(
{
...,
},
{
headers: {
workflowName: "test-workflow",
},
}
)
```
# Passing in Images
Passing images to any model uses the OpenAIs syntax. Underneath we'll convert the syntax for the model you're using.
```py Python
import base64
# Encode the image to base64
with open("some.png", "rb") as image_file:
encoded_string = base64.b64encode(image_file.read())
encoded_string = encoded_string.decode('utf-8')
response = client.chat.completions.create(
model="gpt-4",
messages=[
...
{
"role": "user",
"content": [
{"type": "text", "text": "whats this image?"},
{
"type": "image_url",
"image_url": {
"url": f"data:image/jpeg;base64,{encoded_string}"
}
}
]
},
],
)
```
```ts Typescript
...
import fs from 'fs';
// Function to encode image to base64
function encodeImageToBase64(filePath: string): string {
const image = fs.readFileSync(filePath);
return image.toString('base64');
}
// Usage
const imagePath = 'some.png';
const encodedImage = encodeImageToBase64(imagePath);
const chatCompletion = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [
{
role: "user",
content: [
{ type: "text", text: "What's this image?" },
{
type: "image_url",
image_url: {
url: `data:image/png;base64,${encodedImage}`
}
}
]
}
],
max_tokens: 300
});
```
Then you can switch to a model such as `claude-3-5-sonnet` and pass the image in with no code changes.
```py Python
from optimodel_server_types import ModelTypes,
response = client.chat.completions.create(
model=ModelTypes.claude_3_5_sonnet.name,
messages=[
# Same as above
...
],
)
```
```ts Typescript
import { ModelTypes } from "@lytix/client";
const chatCompletion = await client.chat.completions.create(
{
model: ModelTypes.claude_3_5_sonnet.name,
...,
},
)
```
# Using Models From Other Providers
Beyond the models available on the OpenAI API, Lytix also supports a range of other models from different providers. Just add the credentials for the model/provider and you can start using them immediately.
```sh Python
pip3 install optimodel-py
```
```sh JavaScript
npm i @lytix/client
```
Then just update our `model` field to the model you want to use.
```py Python
from optimodel import ModelTypes
from openai import OpenAI
client = OpenAI(
base_url="https://api.lytix.co/proxy/v2/openai",
api_key="$LYTIX_API_KEY",
default_headers={
# Add your lytix api key
"openaiKey": "$OPENAI_API_KEY",
# Add any extra credentials for the providers you want to use
"mistralApiKey": "$MISTRAL_API_KEY"
},
)
response = client.chat.completions.create(
# Specify the model you want to use from the ModelTypes enum (Remember to use the .name attribute)
model=ModelTypes.codestral_latest.name,
messages=[
{"role": "user", "content": "Say this is a test"}
]
)
```
```ts Typescript
import { ModelTypes } from "@lytix/client";
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.lytix.co/proxy/v2/openai",
apiKey: "$LYTIX_API_KEY",
defaultHeaders: {
openaiKey: "$OPENAI_API_KEY",
// Add any extra credentials for the providers you want to use
mistralApiKey: "$MISTRAL_API_KEY",
},
});
const response = await client.chat.completions.create(
{
messages: [{ role: "user", content: "Say this is a test" }],
// Specify the model you want to use from the ModelTypes enum
model: ModelTypes.codestral_latest,
}
);
```
### Passing in Credentials
To pass in credentials for a provider, you can add the credentials to the headers. The following is a list of credentils you can pass in:
* `mistralApiKey`: The API key for the Mistral API.
* `mistralCodeStralApiKey`: The API key for the Mistral CodeStral API.
* `openaiKey`: The API key for the OpenAI API.
* `anthropicApiKey`: The API key for the Anthropic API.
* `groqApiKey`: The API key for the Groq API.
* `togetherApiKey`: The API key for the Together API.
* `geminiApiKey`: The API key for the Gemini API.
**Bedrock** To run models via bedrock, 3 headers are required:
* `awsAccessKeyId`: The access key for the AWS account.
* `awsSecretKey`: The secret access key for the AWS account.
* `awsRegion`: The session token for the AWS account.
### Supported Models & Providers
You can see the list of up to date models and providers [here](https://lab.lytix.co/home/settings/model-providers/) and clicking "Available Models".