FrontierHarness Eval

Same model, 12 harness configurations, up to 17.5x cost difference

🧑‍💻 Coding & Dev Tools 🔬 Research & DataOpen SourceY▲ 82 on HNLaunched October 2, 2026

FrontierHarness Eval compares coding-agent harnesses on identical software engineering tasks using the same model and runtime. Across 12 configurations of 9 harnesses, including Codex, Claude Code and OpenCode, pass rates were similar but cost per task varied up to 17.5x. First shared by its maker on Show HN: https://news.ycombinator.com/item?id=49538490

✨ The overview, FAQ and details below were drafted by AI from the product's own website and haven't been reviewed by the maker yet. Are you the maker? Claim this listing to review and edit them.

Overview

Who is it for?

Engineers choosing or building a coding-agent harness.

Problem

It's hard to tell how much of an agent's cost and quality comes from the harness rather than the model.

Solution

A controlled evaluation where every run uses the same model, hardware and fresh checkpoint, so only the harness changes.

What makes it unique

All 360 trials start from the same fresh checkpoint restore to avoid warm-cache bias, and you can run it on your own harness.