LLM Test Suite
How to test your LLM interactions
Prerequisite Please see here to get and set your API Key environment variable.
Install the Lytix pip package
Make sure to set your API key e.g. export LX_API_KEY=...
LLM Test Suite
Write custom LLM tests and integrate them into your CI/CD environment.
Create the Test
First head over to lab.lytix.co/home/tests/ and create a new test.
Here is an example test that checks if the LLM output contains profanity:
{{ input }}
and {{ output }}
are placeholders that will be replaced with the actual input and output of the LLM interaction.
Run the test
The following is some example code that runs the test we defined
Now we can run our test suite:
We can now see test2
fail in our dashboard here, and reflect in our exit code
Examples
To see all examples please check the Github repo here