Use Lytix to manage your evaluation and usage diretly with the Langchain SDK. Gain access to models across providers and manage your usage and billing.

Prerequisites

Prerequisite First create a lytix account here

Create a Lytix API Key

Start by creating and noting down a lytix api key. See instructions here

OpenAI

Install the Langchain SDK

Python

pip install langchain-openai

Initialize the ChatOpenAI client

Python

from langchain_openai import ChatOpenAI

model = ChatOpenAI(
    model="gpt-4o-mini", 
    # Update your api key to the lytix api key
    base_url=f"https://api.lytix.co/proxy/v1/openai", 
    # Update your api key to the lytix api key
    api_key="$LYTIX_API_KEY",
    default_headers={
        # Move your openai key to the default headers
        "openaiKey": "$OPENAI_API_KEY"
    },
)

🇪🇺 Note You will need to use https://eu.api.lytix.co/proxy/v1/openai if you are in the EU region.

Continue Using Langchain Normally!

You can now continue using Langchain as you normally would and use this model as you would any other Langchain model.

Python

messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    ("human", "I love programming."),
]

response = model.invoke(messages)
print("Got response: ", response)

Optional Fields

You can pass these fields by updating your model as follows:

Python

model = ChatOpenAI(
  model="gpt-4o-mini",  # Update your base url to the lytix proxy
  base_url=f"https://api.lytix.co/proxy/v1/openai",
  # Update your api key to the lytix api key
  api_key=LYTIX_API_KEY,
  default_headers={
      ...,
      # Optional fields
      "userId": "sid@lytix.co",
      "workflowName": "Example workflow",
      "cacheTTL": "60",
      "sessionId": "1234567890",
  },
)

Langchain supports the following optional fields with Lytix:

sessionId: You can pass a sessionId to track usage for a specific session.
userId: You can pass a userId to track usage for a specific user.
workflowName: You can pass a workflowName to track usage for a specific workflow. Learn more about workflows here.
cacheTTL: You can pass a cacheTTL to cache the response for a specific amount of time. Learn more about caching here.

Anthopic

Install the Langchain SDK

Python

pip install langchain-anthropic

Initialize the ChatAnthropic client

Python

from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(
   model="claude-3-5-sonnet-20240620",
    # Update your api key to the lytix api key
    base_url=f"https://api.lytix.co/proxy/v1/anthropic", 
    # Update your api key to the lytix api key
    api_key="$LYTIX_API_KEY",
    default_headers={
        # Move your anthropic key to the default headers
        "anthropicApiKey": "$OPENAI_API_KEY"
    },
)

🇪🇺 Note You will need to use https://eu.api.lytix.co/proxy/v1/anthropic if you are in the EU region.

Continue Using Langchain Normally!

You can now continue using Langchain as you normally would and use this model as you would any other Langchain model.

Python

messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    ("human", "I love programming."),
]

response = model.invoke(messages)
print("Got response: ", response)

Optional Fields

You can pass these fields by updating your model as follows:

Python

model = ChatAnthropic(
  model="claude-3-5-sonnet-20240620",  
  # Update your base url to the lytix proxy
  base_url=f"https://api.lytix.co/proxy/v1/anthropic",
  # Update your api key to the lytix api key
  api_key=LYTIX_API_KEY,
  default_headers={
      ...,
      # Optional fields
      "userId": "sid@lytix.co",
      "workflowName": "Example workflow",
      "cacheTTL": "60",
      "sessionId": "1234567890",
  },
)

Langchain supports the following optional fields with Lytix:

sessionId: You can pass a sessionId to track usage for a specific session.
userId: You can pass a userId to track usage for a specific user.
workflowName: You can pass a workflowName to track usage for a specific workflow. Learn more about workflows here.
cacheTTL: You can pass a cacheTTL to cache the response for a specific amount of time. Learn more about caching here.

Capturing RAG Metrics

Lytix also supports capturing RAG metrics directly from Langchain.

Install lytix SDK

Python

pip install lytix-py

Setup A Langchain Callback Handler

Python

from langchain_anthropic import ChatAnthropic
from lytix_py import LytixCreds, LytixLangchainCallback

# Or set it via the `LX_API_KEY` environment variable
LytixCreds.setAPIKey("sk-1234")

# Setup your Langchain Client
# Or ChatOpenaAI
llm = ChatAnthropic(
    model="claude-3-5-sonnet-20240620",
    base_url=f"{LytixCreds.LX_BASE_URL}/proxy/v2/anthropic",
    api_key=LytixCreds.LX_API_KEY,
    default_headers={
        # Move your anthropic key to the default headers
        "anthropicApiKey": ANTHOPIC_API_KEY,
    },
)

# Sudo code, replace with your actual implementation
history_aware_retriever = create_history_aware_retriever(
    llm,  ...
)
qa_chain = create_stuff_documents_chain(
    llm, ... 
)

qa = create_retrieval_chain(history_aware_retriever, qa_chain)

res = qa.invoke(
    {"input": "What is the most important part of the speech?"},
    # Add the Lytix Callback Handler
    {"callbacks": [LytixLangchainCallback()]},
)

View Metrics in the Lytix Dashboard

You can view RAG metrics in the lytix dashboard when viewing an IO Event. RAG Metrics On IO Event

Get Started

API Keys

Proxy

Image & Video

Async Logging

Testing

Users

Metadata

Workflows

Datasets

Caching

Playground

Alerts

Bot Evaluation

Prompts

Integrations

Guardrails

Custom Errors

Custom LLM Tracing

CLI

Langchain Integration

Prerequisites

Create a Lytix API Key

OpenAI

Install the Langchain SDK

Initialize the ChatOpenAI client

Continue Using Langchain Normally!

Optional Fields

Anthopic

Install the Langchain SDK

Initialize the ChatAnthropic client

Continue Using Langchain Normally!

Optional Fields

Capturing RAG Metrics

Install lytix SDK

Setup A Langchain Callback Handler

View Metrics in the Lytix Dashboard

​Prerequisites

​Create a Lytix API Key

​OpenAI

​Install the Langchain SDK

​Initialize the ChatOpenAI client

​Continue Using Langchain Normally!

​Optional Fields

​Anthopic

​Install the Langchain SDK

​Initialize the ChatAnthropic client

​Continue Using Langchain Normally!

​Optional Fields

​Capturing RAG Metrics

​Install lytix SDK

​Setup A Langchain Callback Handler

​View Metrics in the Lytix Dashboard

Prerequisites

Create a Lytix API Key

OpenAI

Install the Langchain SDK

Initialize the ChatOpenAI client

Continue Using Langchain Normally!

Optional Fields

Anthopic

Install the Langchain SDK

Initialize the ChatAnthropic client

Continue Using Langchain Normally!

Optional Fields

Capturing RAG Metrics

Install lytix SDK

Setup A Langchain Callback Handler

View Metrics in the Lytix Dashboard