# BonuslyBench — Final Research Package
**September 2026 · 102 models × 40 GTM tests · deterministic scoring**

This folder contains the complete BonuslyBench research exercise: a benchmark of
frontier and open-source LLMs on real go-to-market operations work, built from
live (redacted) GTM data and graded by code — never by another model.

## Start here
1. **`06_Research/BonuslyBench_Research_Piece.docx`** — the executive thought piece:
   why we ran this, what we found, and what it means for GTM/ops leaders.
   (Import to Google Docs for the final destination.)
2. **`01_Results/BonuslyBench_Final_Results.html`** — the full results matrix
   (102 models × 40 tests, color-coded; click any model to see its answers).
3. **`01_Results/BonuslyBench_Results.xlsx`** — the spreadsheet version
   (upload to Google Sheets): 4 tabs — Leaderboard, Full Matrix, Test Difficulty, Model Info.

## Folder map
| Folder | Contents |
|---|---|
| `01_Results/` | HTML results matrix · Excel/Sheets workbook · summary data (JSON) |
| `02_Tests/` | The 40-test specification (HTML + PDF): every test, its planted traps, scorer checks, grading methodology |
| `03_Answers/reviewer/` | **Response Reviewer** — open `index.html`: every model's complete unedited answer per test, side-by-side with the correct answer and per-check ✓/✗ grading |
| `04_Graphs/` | 6-panel dashboard + 7 analysis charts (value map, category heatmap, score distribution, hardest tests, top-20, speed map, test difficulty) |
| `05_Models/` | Model Reference Guide: every model with provider, release date, pricing, marketing positioning, benchmark characteristics |
| `06_Research/` | The docx research piece (executive-ready) |

## Headline findings
- **Open-source leads operational work**: Muse-Spark 1.1 (0.977), Kimi K3 (0.976),
  MiMo v2.5 Pro, Hy3 — ahead of every frontier model on this bench.
- **Price and quality have decoupled**: the top open models cost $0.05–$2 for the
  full 40-test suite; the most expensive frontier model tested cost $123.62 and
  ranked mid-table.
- **Claude Opus 5 is the frontier's best** (0.967, tied with Hy3 which costs 48× less).
- **Hallucination is rare but catchable**: 18 of ~100 models invented an entity at
  least once; the alias-whitelist detector caught every instance.
- **The grader needed grading**: five scorer defects initially made six tests
  "impossible" for every model — fixed mid-exercise, documented in the research piece.

## Honest caveats
- One company's data, one snapshot (2026-09-05), redacted (aliased company names).
- Some model families (Claude Sonnet/Fable, several Gemini variants) did not route
  via the available credentials and are documented as backlog.
- A handful of slow small models ended with partial coverage; every partial is
  labeled with its exact test count — nothing is silently scored.
- Grok models excluded by decision; their early results were removed from all artifacts.

## How to verify any number in this package
Open `03_Answers/reviewer/index.html` → pick a model → every test shows the correct
answer (gold panel) next to the model's actual answer, with the exact checks that
produced the score. Nothing in this package requires trusting us.