Prerequisite Please see here to get and set your API Key environment variable.

Install the Lytix NPM package

npm i @lytix/client

​
LLM Model Input Output Collection

Lytix supports automatically collecting the input and output of LLM models. To do this you can use the following

import { MetricCollector } from "@lytix/client";

await MetricCollector.captureModelIO({
  modelName: "testModelName",
  systemPrompt: "You area helpful assistant",
  userPrompt: "Some user input",
  assistantMessage: "Some model output",


  /**
   * Optional fields
   */
  metricMetadata: {
    /**
     * Optional extra data
     */
    env: "prod",
  }
  userIdentifier?: string;
  sessionId?: string;
);

Lytix will automatically process the input and output of the LLM model and push the metrics to the Lytix platform. You can see your model metrics in the Lytix platform here.

​
LLM Model Trace Collection

Lytix also supports capturing trace information (e.g. duration) for the LLM model. To do this you can use the following

import { MetricCollector } from "@lytix/client";

const response = await MetricCollector.captureModelTrace({
  modelName: "testModelName",
  systemPrompt: "You area helpful assistant",
  userPrompt: "Some user input",
  generateModelOutput: async () => {
    const openai = new OpenAI();
    ...
    /**
     * This callback is expecting the model output to be returned
     */
    return response;
  }


  /**
   * Optional fields
   */
  metricMetadata: {
    /**
     * Optional extra data
     */
    env: "prod",
  }
  userIdentifier?: string;
  sessionId?: string;
});

You’ll now see the model duration trace in the Lytix platform here.