Prerequisite Please see here to get and set your API Key
environment variable.
Install the Lytix pip package
Make sure to set your API key e.g. export LX_API_KEY=...
Capture Model Trace
Lytix also supports capturing trace information (e.g. duration) for the LLM model. To do this you can use the following
import asyncio
from lytix_py import lytix
userInput = "Whats the capital of France?"
"""
The function to be traced
"""
@lytix.trace(modelName="testModelName")
async def getResponse(logger):
modelOutput = "Paris is the capital of france"
lytix.setMessage(userInput, "user")
lytix.setMessage("You are a helpful assistant", "system")
lytix.setMessage("Ouput of model", "assistant")
lytix.setUserIdentifier("testUser")
lytix.setSessionId("testSession")
return modelOutput
async def main():
response = await getResponse("What is the capital of France?")
print(response)
if __name__ == "__main__":
asyncio.run(main())
You’ll now see the model duration trace in the Lytix platform here.
Examples
To see all examples please check the Github repo here