Use Lytix to manage your evaluation and usage diretly with the Gemini SDK. Gain access to models across providers and manage your usage and billing.

Note Currently this is only supported for the Typescript Gemini SDK.

​
Quickstart

Prerequisite First create a lytix account here

​
Create a Lytix API Key

Start by creating and noting down a lytix api key. See instructions here

​
Install the Gemini SDK

Typescript
npm install @google/generative-ai

​
Initialize the GoogleGenerativeAI client

Typescript
import { GoogleGenerativeAI } from "@google/generative-ai";

const genAI = new GoogleGenerativeAI(
  "$GEMINI_API_KEY"
);

​
Initialize the Gemini Model with the Lytix Proxy

Typescript
const customHeaders = new Headers({
  "lx-api-key": "$LYTIX_API_KEY",
});

const requestOptions = {
  customHeaders: customHeaders,
  baseUrl: "https://api.lytix.co/proxy/v1/gemini",
};

const model = genAI.getGenerativeModel(
  {
    model: "$GEMINI_MODEL",
  },
  requestOptions
);

🇪🇺 Note You will need to use https://eu.api.lytix.co/proxy/v1/gemini if you are in the EU region.

​
Invoke the Model 🚀

Typescript
const result = await model.generateContent([
  "Hello world!"
]);

const response = result.response;
const text = await response.text();

​
Optional Fields

You can also track workflows, users and sessions to get a better understanding of your users and how they interact with your models.

  • sessionId: A unique identifier for the session.
  • userId: A unique identifier for the user.
  • workflowName: A unique identifier for the workflow. If this workflow does not exist, it will be created and can be viewed here

​
[Coming Soon ⚡] Saving Video Responses

Sometimes you’d like the ability to save video inputs in your events. This can be done to reply and debug events after the fact.

Gemini does not support querying the files passed into the model query (e.g. the result of fileManager.uploadFile cannot be retrieved after the response has been generated).

To get around this, you can point to a URL that will be used to save the video response. This URL can be passed in as a header and will be saved on the event in the Lytix platform.

Typescript
const customHeaders = new Headers({
  "lx-api-key": "$LYTIX_API_KEY",
  // Add the URL to save the video response
  "videoURL": "$FILE_URI:FILE_MIMIE_TYPE:$URL_OF_VIDEO",
});

// ...The rest of your code as normal

A full example if you choose to save the video response to S3:

Typescript
// First upload your field to Gemini
const fileManager = new GoogleAIFileManager(
  "$GEMINI_API_KEY"
);
const uploadResult = await fileManager.uploadFile(...);

// (sudo code) Next save your file to S3
const s3URL = await uploadFileToS3({
  bucket: 'your-s3-bucket-name',
  key: 'path/to/file.mp4',
  body: uploadResult.file, // Assuming uploadResult contains the file data
  contentType: 'video/mp4' // Adjust based on actual file type
});

// Now when creating custom headers, pass in the S3 URL

const customHeaders = new Headers({
  "lx-api-key": "$LYTIX_API_KEY",
  // Add the URL to save the video response
  "videoURL": `${uploadResult.file.uri}:${uploadResult.file.mimeType}:${s3URL}`,
});
// ...The rest of your code as normal

We’ll save this metadata in the Lytix platform and you’ll be able to reply and debug this event in the future.

​
Using Models From Other Providers

Coming soon. ⚡

Vercel AI IntegrationGuardrails