Use Lytix to manage your evaluation and usage diretly with the Langchain SDK. Gain access to models across providers and manage your usage and billing.

​
Prerequisites

Prerequisite First create a lytix account here

​
Create a Lytix API Key

Start by creating and noting down a lytix api key. See instructions here

​
OpenAI

​
Install the Langchain SDK

Python
pip install langchain-openai

​
Initialize the ChatOpenAI client

Python
from langchain_openai import ChatOpenAI

model = ChatOpenAI(
    model="gpt-4o-mini", 
    # Update your api key to the lytix api key
    base_url=f"https://api.lytix.co/proxy/v1/openai", 
    # Update your api key to the lytix api key
    api_key="$LYTIX_API_KEY",
    default_headers={
        # Move your openai key to the default headers
        "openaiKey": "$OPENAI_API_KEY"
    },
)

🇪🇺 Note You will need to use https://eu.api.lytix.co/proxy/v1/openai if you are in the EU region.

​
Continue Using Langchain Normally!

You can now continue using Langchain as you normally would and use this model as you would any other Langchain model.

Python
messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    ("human", "I love programming."),
]

response = model.invoke(messages)
print("Got response: ", response)

​
Optional Fields

You can pass these fields by updating your model as follows:

Python
model = ChatOpenAI(
  model="gpt-4o-mini",  # Update your base url to the lytix proxy
  base_url=f"https://api.lytix.co/proxy/v1/openai",
  # Update your api key to the lytix api key
  api_key=LYTIX_API_KEY,
  default_headers={
      ...,
      # Optional fields
      "userId": "sid@lytix.co",
      "workflowName": "Example workflow",
      "cacheTTL": "60",
      "sessionId": "1234567890",
  },
)

Langchain supports the following optional fields with Lytix:

  • sessionId: You can pass a sessionId to track usage for a specific session.
  • userId: You can pass a userId to track usage for a specific user.
  • workflowName: You can pass a workflowName to track usage for a specific workflow. Learn more about workflows here.
  • cacheTTL: You can pass a cacheTTL to cache the response for a specific amount of time. Learn more about caching here.

​
Anthopic

​
Install the Langchain SDK

Python
pip install langchain-anthropic

​
Initialize the ChatAnthropic client

Python
from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(
   model="claude-3-5-sonnet-20240620",
    # Update your api key to the lytix api key
    base_url=f"https://api.lytix.co/proxy/v1/anthropic", 
    # Update your api key to the lytix api key
    api_key="$LYTIX_API_KEY",
    default_headers={
        # Move your anthropic key to the default headers
        "anthropicApiKey": "$OPENAI_API_KEY"
    },
)

🇪🇺 Note You will need to use https://eu.api.lytix.co/proxy/v1/anthropic if you are in the EU region.

​
Continue Using Langchain Normally!

You can now continue using Langchain as you normally would and use this model as you would any other Langchain model.

Python
messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    ("human", "I love programming."),
]

response = model.invoke(messages)
print("Got response: ", response)

​
Optional Fields

You can pass these fields by updating your model as follows:

Python
model = ChatAnthropic(
  model="claude-3-5-sonnet-20240620",  
  # Update your base url to the lytix proxy
  base_url=f"https://api.lytix.co/proxy/v1/anthropic",
  # Update your api key to the lytix api key
  api_key=LYTIX_API_KEY,
  default_headers={
      ...,
      # Optional fields
      "userId": "sid@lytix.co",
      "workflowName": "Example workflow",
      "cacheTTL": "60",
      "sessionId": "1234567890",
  },
)

Langchain supports the following optional fields with Lytix:

  • sessionId: You can pass a sessionId to track usage for a specific session.
  • userId: You can pass a userId to track usage for a specific user.
  • workflowName: You can pass a workflowName to track usage for a specific workflow. Learn more about workflows here.
  • cacheTTL: You can pass a cacheTTL to cache the response for a specific amount of time. Learn more about caching here.

​
Capturing RAG Metrics

Lytix also supports capturing RAG metrics directly from Langchain.

​
Install lytix SDK

Python
pip install lytix-py

​
Setup A Langchain Callback Handler

Python
from langchain_anthropic import ChatAnthropic
from lytix_py import LytixCreds, LytixLangchainCallback

# Or set it via the `LX_API_KEY` environment variable
LytixCreds.setAPIKey("sk-1234")

# Setup your Langchain Client
# Or ChatOpenaAI
llm = ChatAnthropic(
    model="claude-3-5-sonnet-20240620",
    base_url=f"{LytixCreds.LX_BASE_URL}/proxy/v2/anthropic",
    api_key=LytixCreds.LX_API_KEY,
    default_headers={
        # Move your anthropic key to the default headers
        "anthropicApiKey": ANTHOPIC_API_KEY,
    },
)

# Sudo code, replace with your actual implementation
history_aware_retriever = create_history_aware_retriever(
    llm,  ...
)
qa_chain = create_stuff_documents_chain(
    llm, ... 
)

qa = create_retrieval_chain(history_aware_retriever, qa_chain)

res = qa.invoke(
    {"input": "What is the most important part of the speech?"},
    # Add the Lytix Callback Handler
    {"callbacks": [LytixLangchainCallback()]},
)

​
View Metrics in the Lytix Dashboard

You can view RAG metrics in the lytix dashboard when viewing an IO Event.

Gemini IntegrationImages