Custom LLM Tracing
Custom LLM Tracing
Learn what it means to capture custom LLM traces
Lytix supports capturing custom LLM traces (e.g. not associated with any SDK or API call). This is useful if you have custom models you’d still like to track.
Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
Learn what it means to capture custom LLM traces
from lytix_py import lytix
@lytix.trace(modelName="nodeCli")
async def getResponse(logger):
modelOutput = "Paris is the capital of France"
# Set our defined params however we want
lytix.setMessage("Some user input", "user")
lytix.setMessage(modelOutput, "assistant")
# Optional values
lytix.setUserIdentifier("testUser")
lytix.setSessionId("testSession")
return modelOutput
import { MetricCollector } from "@lytix/node-cli";
const response = await MetricCollector.captureModelTrace({
modelName: "testModelName",
systemPrompt: "You are a helpful assistant",
userPrompt: "Some user input",
generateModelOutput: async () => {
const openai = new OpenAI();
...
/**
* This callback is expecting the model output to be returned
*/
return response;
}
});