AI tools for benchmarks

A reproducible benchmark for typed decision models

JevBench, from Benchmark Heaven, compares Jev-class decision models across intelligence, calibration, speed and cost. It sits alongside Benchmark Heaven's broader model rankings, which can be filtered by provider, lab, hosting region and da

0 commentsY▲ 153 on HNResearch & DataLLMs & InfrastructureFreeby AI Launch Team

A System One Model for Computer Use

Cua gives AI agents computers they can use. It provides open-source desktop automation drivers, isolated cloud desktops (Cua Fleets), local macOS VMs (Lume), CUA-S1 specialist models for computer-use decisions, and Cua Bench for evaluating

0 commentsY▲ 95 on HNAI AgentsLLMs & InfrastructureFreemiumby AI Launch Team

Same model, 12 harness configurations, up to 17.5x cost difference

FrontierHarness Eval compares coding-agent harnesses on identical software engineering tasks using the same model and runtime. Across 12 configurations of 9 harnesses, including Codex, Claude Code and OpenCode, pass rates were similar but c

0 commentsY▲ 82 on HNCoding & Dev ToolsResearch & DataOpen Sourceby AI Launch Team

How well can models one-shot a Pac-Man game?

PacBench tests how well a model and harness can recreate Pac-Man from one prompt: "Create a Pac-Man game in a single html page." Entries from different models and harnesses can be sorted by score, cost, time and tokens, with run data for ea

0 commentsY▲ 78 on HNCoding & Dev ToolsResearch & DataFreeby AI Launch Team