jevals
Replacing LLM judges with typed Jev decisions
jevals runs evals and guardrails for AI agents using Jev-style decision models instead of an LLM judge. All the checks for a trace, such as tool choice, groundedness, answer relevancy, prompt injection and PHI, go out as one request that costs a few thousandths of a cent and returns in a few hundred milliseconds. First shared by its maker on Show HN: https://news.ycombinator.com/item?id=49780849
✨ The overview, FAQ and details below were drafted by AI from the product's own website and haven't been reviewed by the maker yet. Are you the maker? Claim this listing to review and edit them.
Overview
Who is it for?
Developers who evaluate and guard AI agents in production.
Problem
LLM-as-judge evals are slow and costly, so teams can't run them on every trace or inside the agent loop.
Solution
Typed decision models that score many checks in one cheap, fast request, including security checks like indirect injection.
What makes it unique
The README's quickstart runs eight checks in one request for about $0.00006 in 0.33 seconds.
jevals FAQ
How do I install it?
pip install jevals, then set an API key for Jev via Vercel AI Gateway, TypeSafe or OpenRouter.
Can it run locally?
Yes, with Kev or Laya running on a Mac, or with a regular chat LLM (slower and more expensive).
What can it check?
Tool choice, use of tool results, groundedness, staying in scope, answer relevancy, completeness, indirect prompt injection and PHI, among others.
Details
- Pricing
- Open Source
- Categories
- LLMs & InfrastructureAI Agents
- Built for
- Developers
- Platforms
- API
- Tech stack
- Python
Compare jevals
Hunted by AI Launch Team
Discussion (0)
Sign in to join the discussion.