Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.lytix.co/llms.txt

Use this file to discover all available pages before exploring further.

Use Lytix to manage your evaluation and usage diretly with the OpenAI SDK. Gain access to models across providers and manage your usage and billing.

​
Quickstart

Prerequisite First create a lytix account here

​
Create a Lytix API Key

Start by creating and noting down a lytix api key. See instructions here

​
Update your OpenAI SDK

With 2 lines you can start using Lytix to manage your evaluation and usage. 
from openai import OpenAI

client = OpenAI(
  # Update your base url to the lytix proxy
  base_url=f"https://api.lytix.co/proxy/v2/openai",
  # Update your api key to the lytix api key
  api_key="$LYTIX_API_KEY",
  default_headers={
    # Move your openai key to the default headers
    "openaiKey": "$OPENAI_API_KEY"
  },
)
🇪🇺 Note You will need to use https://eu.api.lytix.co/proxy/v2/openai if you are in the EU region.

​
Optional Fields

Optimodel supports a variety of optional parameters to help you get the best results. 
response = client.chat.completions.create(
   ...,
    extra_body={
        "lytix-fallbackModels": ...
        ...
    },
    extra_headers={
        "sessionId": "1234567890",
        "userId": "sid@lytix.co",
        "workflowName": "test-workflow",
    },
)
You will need to use the optimodel-py/@lytix/client package to use these parameters. 
pip3 install optimodel-py
The following optional parameters are supported:

​
Guards

lytix-guards: Pass in a list of fallback models to use 
  from optimodel_server_types import LLamaPromptGuardConfig
  extra_body={
      "lytix-guards": [LLamaPromptGuardConfig(
          guardName="LLamaPromptGuard",
          jailbreakThreshold=0.9999,
          guardType="preQuery", # You'll likely only want to guard the input here
      ).dict()]
  }
See here for a list of all supported guards

​
Fallback Models

lytix-fallbackModels: Pass in a list of extra models to try if the primary model fails. This can be helpful in mitigating provider outages. 
  from optimodel_server_types import ModelTypes
  ...
  extra_body={
    "lytix-fallbackModels": [ModelTypes.claude_3_5_sonnet.name, ...]
  }

​
Speed Priority

lytix-speedPriority: Pass in a speed priority to use 
extra_body={
  "lytix-speedPriority": "low"
}
If set to low, optimodel will choose the cheapest possible model across all providers (for example if you have two providers bedrock and anthropic that both offer claude-3-opus, optimodel will choose the claude-3-opus model with the lowest price regardless of which provider is faster). If set to high, optimodel will choose the fastest possible model across all providers.

​
Provider

lytix-provider: Pass in a provider to use 
from optimodel_server_types import Providers
...
extra_body={
    "lytix-provider": ProviderTypes.bedrock.name
}
Explicitly specify a provider to use incase you have multiple providers available for a specific model and want to force a specific one. You can also track workflows, users and sessions to get a better understanding of your users and how they interact with your models.

​
SessionId

sessionId: A unique identifier for the session. 
extra_headers={
  "sessionId": "1234567890"
}

​
UserId

userId: A unique identifier for the user. 
extra_headers={
  "userId": "sid@lytix.co"
}

​
WorkflowName

workflowName: A unique identifier for the workflow. If this workflow does not exist, it will be created and can be viewed here 
extra_headers={
  "workflowName": "test-workflow"
}

​
Passing in Images

Passing images to any model uses the OpenAIs syntax. Underneath we’ll convert the syntax for the model you’re using. 
import base64

# Encode the image to base64
with open("some.png", "rb") as image_file:
    encoded_string = base64.b64encode(image_file.read())
    encoded_string = encoded_string.decode('utf-8')

response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        ...
        {
            "role": "user", 
            "content": [
                {"type": "text", "text": "whats this image?"}, 
                {
                    "type": "image_url", 
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{encoded_string}"
                    }
                }
            ]
        },
    ],
)
Then you can switch to a model such as claude-3-5-sonnet and pass the image in with no code changes. 
from optimodel_server_types import ModelTypes,

response = client.chat.completions.create(
    model=ModelTypes.claude_3_5_sonnet.name,
    messages=[
      # Same as above
      ...
    ],
)

​
Using Models From Other Providers

Beyond the models available on the OpenAI API, Lytix also supports a range of other models from different providers. Just add the credentials for the model/provider and you can start using them immediately. 
pip3 install optimodel-py
Then just update our model field to the model you want to use. 
from optimodel import ModelTypes
from openai import OpenAI

client = OpenAI(
  base_url="https://api.lytix.co/proxy/v2/openai",
  api_key="$LYTIX_API_KEY",
  default_headers={
    # Add your lytix api key
    "openaiKey": "$OPENAI_API_KEY",
    # Add any extra credentials for the providers you want to use
    "mistralApiKey": "$MISTRAL_API_KEY"
  },
)

response = client.chat.completions.create(
  # Specify the model you want to use from the ModelTypes enum (Remember to use the .name attribute)
  model=ModelTypes.codestral_latest.name,
  messages=[
    {"role": "user", "content": "Say this is a test"}
  ]
)

​
Passing in Credentials

To pass in credentials for a provider, you can add the credentials to the headers. The following is a list of credentils you can pass in:
  • mistralApiKey: The API key for the Mistral API.
  • mistralCodeStralApiKey: The API key for the Mistral CodeStral API.
  • openaiKey: The API key for the OpenAI API.
  • anthropicApiKey: The API key for the Anthropic API.
  • groqApiKey: The API key for the Groq API.
  • togetherApiKey: The API key for the Together API.
  • geminiApiKey: The API key for the Gemini API.
Bedrock To run models via bedrock, 3 headers are required:
  • awsAccessKeyId: The access key for the AWS account.
  • awsSecretKey: The secret access key for the AWS account.
  • awsRegion: The session token for the AWS account.

​
Supported Models & Providers

title You can see the list of up to date models and providers here and clicking “Available Models”.