After setting up your caching DB, you are ready to start caching your queries. Depending on what client you are using, the process is the same.

​
Optimodel

Prerequisite Please see here to learn more about how to use Optimodel with Lytix.

To set a TTL for your query, just pass the extra cacheTTL parameter. This is a value in seconds

from optimodel import queryModel, listModels, ModelMessage, ModelTypes

prompt = "Hello How are you?"

response = await queryModel(
    ...All the same params
    # Just add this extra param
    cacheTTL=60*1 # 1 minute
)
print("Got response:", response)

​
OpenAI Client

Prerequisite Please see here to learn more about how to use OpenAI with Lytix.

While using the OpenAI client, you can pass an extra cacheTTL header when making your request. This is a value in seconds.

from openai import OpenAI

client = OpenAI(
  base_url=f"https://api.lytix.co/proxy/v1/openai",
  api_key="$LYTIX_API_KEY",
  default_headers={
    "openaiKey": "$OPENAI_API_KEY",
    # Add the cacheTTL header
    "cacheTTL": "60 * 1" # 1 minute
  },
)

​
Viewing Cache Hits

You can view the cache hits on the Lytix dashboard here

title