BonuslyBench — Final Results

103 curated models × 40 GTM tests, deterministic scoring (scorer v2, post-fix). Green = perfect pass. Click any model name to read its actual answers per test in the Response Reviewer.

CURATED SET · MAX 5/PROVIDER · GROK EXCLUDED · SNAPSHOT 2026-09-14 · 96 of 103 models at full 40/40 coverage
📖 How to review any model's work: click the model name or the green 📄 view answers button in any row — that opens the model's page in the Response Reviewer, where every test shows the correct answer (gold panel) side-by-side with the model's actual answer, plus every graded check ✓/✗.
⚖️ Ranking fairness: only the 96 models with full 40/40 coverage receive a rank. The 7 partial models (endpoint timeouts on their heaviest tests, disclosed below the ranked table) are listed after the ranked group — their means cover only completed tests and are not comparable to full-coverage means.
#ModelMeanPassCostLatCommunicationCustomer & SuccessData & CrmDeal & IntelligenceMarketingOps & MaintenanceRep & PerformanceReporting & Analytics
ceo
slack
compression		funnel
digest
paragraph		partner
digest
section		rfp
answer
from
policy		stalled
deal
followup
email		churn
save
eligibility		expansion
kit		onboarding
status		renewal
risk
conflicting
dates		ticket
theme
synthesis		crm
completeness
enrichment		gong
hubspot
join
integrity		owner
resolution		snowflake
sql
trapped
catalog		stage
hygiene
audit		call
transcript
extraction		closed
lost
classification		next
to
close		pipeline
tiering		product
gap
detection		case
study
match		channel
attribution
math		competitor
battlecard
update		nurture
sequence
diagnosis		weekly
marketing
goals
rollup		datadog
incident
timeline		feature
flag
state
summary		nda
triage		runbook
from
slack
thread		skill
file
reconciliation		activity
mix
vs
outcome		ae
qtd
scorecard		multithreading
gap		stale
pipeline
by
rep		talk
track
from
transcripts		aha
moment
2x2		arr
rollforward
reconciliation		kvm
month
over
month		q3
weighted
forecast		rewards
ytd
section
1meta/muse-spark-1.1
📄 view answers vs correct answer →open		0.97735/40$2.1740s✓✓✓✓✓✓✓✓✓0.80✓✓✓✓✓✓✓✓0.80✓✓✓✓✓✓✓✓✓✓0.86✓✓✓0.83✓✓✓0.80✓✓
2moonshotai/kimi-k3
📄 view answers vs correct answer →frontier		0.97636/40$7.67102s✓✓✓✓✓✓✓✓✓0.80✓✓0.83✓✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.67✓✓✓✓✓✓
3xiaomi/mimo-v2.5-pro
📄 view answers vs correct answer →open		0.97436/40$0.69119s✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.67✓0.83✓✓0.70✓
4meta/muse-spark-1.3
📄 view answers vs correct answer →open		0.97335/40$3.0343s✓✓✓✓✓✓✓✓✓0.80✓✓✓✓✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.750.830.80✓✓✓✓✓
5z-ai/glm-5.3-flash
📄 view answers vs correct answer →open		0.96835/40$0.28401s✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.800.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.67✓0.83✓✓✓0.67
6tencent/hy3-preview
📄 view answers vs correct answer →open		0.96834/40$0.5982s0.80✓✓✓✓✓✓✓✓0.80✓✓✓✓✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓0.86✓✓✓0.67✓0.83✓✓✓✓
7tencent/hy3
📄 view answers vs correct answer →open		0.96736/40$0.331611s✓✓✓✓✓✓✓✓✓✓✓0.50✓✓✓✓0.600.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.83✓✓✓✓
8anthropic/claude-opus-5
📄 view answers vs correct answer →open		0.96735/40$15.91107s✓✓✓✓✓✓✓✓✓0.80✓✓✓✓0.83✓0.800.50✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓
9google/gemini-3.8-flash
📄 view answers vs correct answer →open		0.96634/40$6.34117s0.80✓✓✓✓✓✓✓✓0.80✓0.750.67✓0.83✓0.80✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓
10qwen/qwen3.8-flash
📄 view answers vs correct answer →open		0.96534/40$0.34111s✓✓✓✓0.67✓✓✓✓0.80✓0.750.83✓✓✓✓✓✓✓✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓0.80✓✓✓
11meta/muse-spark-1.2
📄 view answers vs correct answer →open		0.96433/40$3.6062s0.80✓✓✓✓✓✓✓✓0.80✓0.75✓✓✓✓✓✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓0.83✓0.83✓0.80✓✓
12z-ai/glm-5.1
📄 view answers vs correct answer →open		0.96433/40$1.99178s0.80✓✓✓✓✓✓✓✓0.80✓✓0.67✓✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.830.80✓0.90✓
13qwen/qwen3.8-max-0902
📄 view answers vs correct answer →open		0.96234/40$3.88160s✓✓✓✓✓✓✓✓✓0.80✓✓0.83✓0.83✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.67✓✓0.60✓✓✓
14z-ai/glm-5.3
📄 view answers vs correct answer →open		0.96033/40$6.86301s0.80✓✓✓0.67✓✓✓✓0.80✓✓✓✓✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.67✓0.83✓✓0.90✓
15deepseek/deepseek-v4-pro
📄 view answers vs correct answer →open		0.95933/40$1.04328s✓✓✓✓0.67✓✓✓✓0.80✓✓✓✓✓✓0.80✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.83✓✓✓0.670.80✓0.80✓
16anthropic/claude-opus-4.5
📄 view answers vs correct answer →open		0.95732/40$9.8390s✓✓✓✓✓✓✓0.75✓0.80✓✓0.83✓✓✓✓✓0.80✓✓0.800.80✓✓✓✓✓✓✓✓✓✓0.67✓0.83✓✓✓✓
17sakana/fugu-ultra
📄 view answers vs correct answer →open		0.95734/40$31.55410s✓✓✓✓✓✓✓✓✓0.80✓✓✓0.67✓✓0.80✓✓✓✓✓✓✓✓✓✓✓0.50✓✓✓✓0.67✓0.83✓✓✓✓
18openai/gpt-5.5-pro
📄 view answers vs correct answer →frontier		0.95632/40$123.62183s✓✓✓✓✓✓✓✓✓0.800.80✓0.83✓✓✓0.80✓✓✓✓✓✓✓✓✓✓✓0.750.86✓✓0.750.67✓✓✓✓✓✓
19meituan/longcat-2.0
📄 view answers vs correct answer →open		0.95532/40$0.63245s✓✓✓✓✓✓✓✓✓0.80✓0.750.83✓✓✓✓0.75✓✓✓✓✓✓✓✓✓✓0.75✓✓✓✓✓✓0.830.60✓0.90✓
20anthropic/claude-opus-4.7
📄 view answers vs correct answer →open		0.95433/40$10.7062s✓✓✓✓✓✓✓✓✓0.60✓✓0.83✓0.83✓0.800.50✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.800.80✓✓
21google/gemini-3.7-flash
📄 view answers vs correct answer →frontier		0.95331/40$5.1483s0.80✓✓✓✓✓✓✓✓0.80✓0.750.67✓✓✓✓0.75✓✓✓✓0.80✓✓✓✓✓✓✓✓✓✓0.83✓0.83✓✓0.90✓
22moonshotai/kimi-k2.7-code
📄 view answers vs correct answer →open		0.95331/40$1.571145s✓✓✓✓✓✓✓✓✓✓✓0.750.83✓✓✓✓0.750.80✓✓✓✓✓✓✓✓✓✓0.86✓✓✓✓✓0.830.600.800.90✓
23qwen/qwen3.8-27b
📄 view answers vs correct answer →open		0.95332/40$1.63213s✓✓✓✓0.67✓✓✓✓0.80✓✓✓✓✓✓0.800.750.80✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.83✓0.670.80✓✓✓
24openai/gpt-5.2-pro
📄 view answers vs correct answer →open		0.95231/40$97.83181s✓✓0.67✓✓✓✓✓✓0.80✓0.750.83✓0.83✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.83✓✓✓0.80✓0.83
25openai/gpt-6-astra
📄 view answers vs correct answer →open		0.95234/40$19.8248s0.80✓✓✓✓✓✓✓✓0.80✓✓✓0.50✓✓0.800.50✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.67✓✓✓✓✓✓
26deepseek/deepseek-v4-pro-0813
📄 view answers vs correct answer →open		0.94732/40$1.85110s✓✓✓✓✓✓✓✓✓0.80✓0.750.830.50✓✓✓0.75✓✓✓✓✓0.75✓✓✓✓✓✓✓✓✓0.67✓0.83✓✓✓✓
27anthropic/claude-haiku-4.5
📄 view answers vs correct answer →frontier		0.94430/40$2.3459s✓✓✓✓✓✓0.83✓✓0.80✓✓0.83✓✓✓✓✓0.80✓✓✓✓✓✓✓✓✓0.750.86✓✓0.750.67✓0.670.80✓✓✓
28openai/gpt-5.2
📄 view answers vs correct answer →open		0.94429/40$7.35106s0.80✓0.67✓✓✓✓✓✓0.80✓0.750.83✓0.83✓0.800.75✓✓✓✓✓✓✓✓✓✓✓0.86✓✓✓0.83✓✓✓✓✓0.83
29qwen/qwen3.8-2.4t-a95b
📄 view answers vs correct answer →open		0.94331/40$5.20259s0.80✓✓✓✓✓✓✓✓0.80✓✓0.83✓0.67✓✓✓0.80✓0.75✓✓0.75✓✓✓✓✓✓✓✓✓0.67✓0.67✓✓✓✓
30openai/gpt-5.6-luna-pro
📄 view answers vs correct answer →open		0.94331/40$1.0475s✓✓✓✓✓✓✓✓✓0.80✓0.750.83✓✓✓0.800.75✓✓✓✓✓✓✓✓✓✓✓0.86✓✓0.750.67✓✓✓✓✓0.50
31minimax/minimax-m2.1
📄 view answers vs correct answer →open		0.94231/40$0.6760s0.60✓✓✓✓✓0.83✓✓✓✓✓✓✓✓✓0.800.75✓✓✓✓✓✓✓✓✓✓0.75✓✓0.83✓✓✓0.830.80✓✓0.50
32openai/gpt-5.4-pro
📄 view answers vs correct answer →open		0.94232/40$131.87256s✓✓0.67✓✓✓✓✓✓0.80✓✓0.67✓✓✓0.800.50✓✓✓✓✓✓✓✓✓✓0.75✓✓✓✓0.67✓0.83✓✓✓✓
33openai/gpt-5.5
📄 view answers vs correct answer →frontier		0.94131/40$9.0637s✓✓✓✓✓✓✓✓✓0.800.80✓0.670.670.67✓0.80✓✓✓✓✓✓✓✓✓✓✓0.75✓✓✓✓0.67✓0.83✓✓✓✓
34openai/gpt-6-astra-pro
📄 view answers vs correct answer →open		0.94132/40$43.5469s0.80✓✓✓✓✓✓✓✓0.80✓✓✓0.50✓✓0.800.50✓✓✓✓✓✓✓✓✓✓0.75✓✓✓✓0.67✓0.83✓✓✓✓
35anthropic/claude-sonnet-5
📄 view answers vs correct answer →open		0.94032/40$8.38115s✓✓✓✓✓✓✓✓✓0.80✓0.75✓✓✓✓0.600.75✓✓✓✓✓✓✓✓✓✓0.750.86✓✓✓✓✓✓0.60✓✓0.50
36openai/gpt-5.6-sol
📄 view answers vs correct answer →frontier		0.94029/40$3.6739s0.80✓✓✓✓✓✓0.75✓0.800.80✓✓0.83✓✓0.800.750.80✓✓✓✓✓✓✓✓✓0.750.86✓✓✓0.67✓✓✓✓✓✓
37deepseek/deepseek-v4-flash-0731
📄 view answers vs correct answer →open		0.93930/40$0.0955s✓✓✓✓0.67✓✓✓✓0.80✓0.75✓✓0.67✓0.800.75✓✓✓✓✓0.75✓✓✓✓✓0.86✓✓✓✓✓0.83✓✓0.70✓
38anthropic/claude-opus-4.8
📄 view answers vs correct answer →frontier		0.93728/40$13.0677s✓✓✓✓✓✓✓✓✓0.80✓0.750.830.670.83✓0.800.75✓✓✓✓✓✓✓✓✓✓✓0.86✓✓✓0.83✓0.670.80✓0.90✓
39openai/gpt-5.6-sol-pro
📄 view answers vs correct answer →open		0.93731/40$7.9753s0.80✓✓✓✓✓✓0.75✓0.80✓✓✓0.67✓✓0.800.75✓✓✓✓✓✓✓✓✓✓0.75✓✓✓✓0.67✓✓✓✓✓0.50
40openai/gpt-5.6-luna
📄 view answers vs correct answer →frontier		0.93630/40$0.4241s✓✓✓✓✓✓✓0.75✓0.800.80✓0.830.50✓✓0.800.750.80✓✓✓✓✓✓✓✓✓✓✓✓✓0.750.67✓✓✓✓✓✓
41deepseek/deepseek-v4-flash
📄 view answers vs correct answer →open		0.93630/40$0.20151s0.60✓✓✓✓✓✓✓✓0.80✓✓0.83✓0.830.80✓✓✓✓✓✓0.80✓✓✓✓✓✓✓✓0.83✓✓✓0.830.60✓✓0.50
42openai/gpt-5.4-mini
📄 view answers vs correct answer →frontier		0.93530/40$3.2172s✓✓✓✓✓✓0.830.75✓0.80✓0.75✓✓✓✓0.800.75✓✓✓✓0.80✓✓✓✓✓✓✓✓✓✓0.50✓0.830.60✓✓✓
43xiaomi/mimo-v2.5
📄 view answers vs correct answer →open		0.93230/40$0.2763s✓✓✓✓✓✓✓✓✓0.80✓0.750.83✓0.67✓0.800.75✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.830.60✓✓0.50
44openai/gpt-5.1
📄 view answers vs correct answer →open		0.93130/40$4.83115s✓✓0.67✓✓✓0.83✓✓0.60✓0.750.67✓✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓0.71✓✓✓0.83✓0.83✓✓0.60✓
45openai/gpt-5-pro
📄 view answers vs correct answer →open		0.92729/40$93.00355s0.80✓0.67✓✓✓✓✓✓0.600.800.750.67✓✓✓✓0.75✓✓✓✓0.80✓✓✓✓✓✓✓✓✓0.750.67✓0.83✓✓✓✓
46meta/muse-glimmer-30b
📄 view answers vs correct answer →open		0.92530/40$0.9898s✓✓0.67✓✓✓✓✓✓0.800.800.75✓✓0.83✓✓0.75✓✓✓✓✓✓✓✓✓✓0.75✓✓✓✓0.83✓✓✓0.80✓✗
47openai/gpt-5.4-nano
📄 view answers vs correct answer →open		0.92427/40$1.04117s✓✓0.67✓✓✓✓✓✓0.80✓✓0.83✓✓✓✓0.750.80✓✓✓✓✓✓✓✓✓✓0.86✓✓0.750.33✓0.830.800.800.900.83
48stepfun/step-3.5-flash
📄 view answers vs correct answer →open		0.92329/40$0.45166s✓✓✓✓✓0.80✓✓0.75✓✓✓0.67✓✓✓✓0.75✓✓✓✓✓✓0.80✓✓✓✓0.86✓✓0.750.83✓0.83✓✓0.400.50
49minimax/minimax-m2.5
📄 view answers vs correct answer →open		0.92328/40$0.44594s0.80✓✓✓✓0.600.83✓0.750.80✓0.750.830.67✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.830.80✓✓0.50
50openai/gpt-5.6-terra
📄 view answers vs correct answer →frontier		0.92027/40$3.6727s0.80✓0.67✓✓0.60✓✓✓0.800.80✓0.670.67✓✓0.800.750.80✓✓✓0.80✓✓✓✓✓✓✓0.80✓✓0.83✓✓✓✓✓✓
51inclusionai/ling-3.0-flash
📄 view answers vs correct answer →open		0.91928/40$0.0649s✓0.670.67✓✓0.800.83✓✓0.80✓✓✓✓✓✓0.800.75✓✓✓✓0.40✓✓0.80✓✓✓0.86✓✓✓✓✓✓0.600.80✓✓
52stepfun/step-3.7-flash
📄 view answers vs correct answer →open		0.91928/40$0.791536s✓✓0.67✓✓✓✓0.75✓0.80✓0.250.83✓✓✓0.80✓✓✓✓✓✓0.75✓✓✓✓✓0.86✓0.83✓✓✓0.830.600.80✓✓
53openai/gpt-5
📄 view answers vs correct answer →open		0.91628/40$6.21131s0.80✓0.67✓✓✓✓✓✓0.800.800.750.67✓✓✓0.800.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.67✓0.83✓✓0.600.50
54thinkingmachines/inkling
📄 view answers vs correct answer →open		0.91627/40$3.1347s✓✓✓✓0.67✓✓✓0.750.80✓0.500.67✓✓0.80✓0.750.80✓✓0.80✓✓✓✓✓✓✓0.86✓✓✓0.83✓✓0.600.80✓✓
55openai/gpt-5.6-terra-pro
📄 view answers vs correct answer →open		0.91526/40$7.9841s0.80✓0.67✓✓✓✓0.75✓0.800.80✓0.830.67✓✓0.800.75✓✓✓✓✓✓✓✓✓✓0.75✓✓✓0.750.83✓✓✓✓0.900.50
56minimax/minimax-m2
📄 view answers vs correct answer →open		0.91428/40$1.1959s0.60✓✓✓✓✓✓✓✓0.80✓0.50✓0.83✓✓✓0.750.80✓✓✓✓✓✓✓✓✓0.75✓✓✓0.750.83✓0.830.80✓0.30✓
57upstage/solar-pro4
📄 view answers vs correct answer →open		0.91329/40$0.0554s0.80✓✓✓✓✓0.83✓✓0.80✓✓✓✓0.67✓0.80✓0.80✓✓✓✓✓✓✓✓✓✓✓✓0.83✓✗✓✓0.80✓0.700.50
58openai/gpt-chat-latest
📄 view answers vs correct answer →open		0.90829/40$3.77120s✓✓✓✓✓✓✓✓✓0.800.80✓✓0.50✓✓0.800.750.60✓✓✓✓✓✓✓✓✓0.75✓✓✓0.250.67✓✓✓✓0.900.50
59tencent/hy4-preview
📄 view answers vs correct answer →open		0.90629/40$1.731548s✓✓✓✓0.33✓✓✓✓0.80✗0.75✓0.83✓✓0.800.750.80✓✓✓✓✓✓✓✓✓✓0.86✓✓✓0.50✓0.83✓✓✓✓
60qwen/qwen3.7-flash
📄 view answers vs correct answer →open		0.90627/40$0.0883s✓✓0.67✓✓0.800.83✓✓0.60✓0.750.83✓✓✓✓0.75✓✓✓✓0.80✓✓0.80✓✓✓✓✓0.50✓✓✓✓0.80✓0.600.50
61openai/gpt-5-mini
📄 view answers vs correct answer →open		0.90531/40$1.04120s✓✓0.67✓✓✓✓✓✓0.80✓0.25✓✓0.67✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓0.33✓✓0.600.80✓0.33
62bytedance-seed/seed-2.0-mini
📄 view answers vs correct answer →open		0.90027/40$0.12118s✓✓0.67✓0.67✓0.83✓✓0.80✓0.750.83✓0.50✓✓✓✓0.50✓✓✓✓0.80✓✓✓✓✓✓✓✓0.50✓0.83✓✓0.800.50
63google/gemma-4-31b-it
📄 view answers vs correct answer →open		0.89025/40$0.16516s✓✓0.67✓✓✓✓✓0.750.800.800.500.17✓0.83✓✓0.750.80✓✓✓0.80✓✓✓✓✓0.75✓✓✓0.750.83✓✓0.60✓0.80✓
64bytedance-seed/seed-1.6
📄 view answers vs correct answer →open		0.88928/40$0.71116s0.80✓✓✓✓✓✓✓✓✓✓0.750.67✓✓✓0.800.750.20✓✓✓✓✓0.80✓✓✓✓✓✓0.50✓0.67✓0.33✓✓0.800.50
65minimax/minimax-m3
📄 view answers vs correct answer →open		0.87826/40$0.622415s✓✓✓✓✓✓✓✓0.750.80✓0.750.33✓✗✓✓✓0.80✓0.75✓✓✓✓✓✓✓✓0.86✓✓0.750.83✓✗0.800.800.90✓
66aion-labs/aion-3.0-mini
📄 view answers vs correct answer →open		0.87428/40$1.29194s0.60✓0.67✓✓✓✓✓✓✓✓✗0.67✓0.67✓0.80✓0.80✓✓✓✓✓✓✓✓✓✓✓✓0.83✓0.50✓0.33✓✓0.600.50
67arcee-ai/trinity-large-thinking
📄 view answers vs correct answer →open		0.87326/40$0.6874s0.80✓✓✓0.67✓✓✓✓0.40✓0.750.83✓0.67✓0.800.75✓✓✓✓✓✓0.80✓✓✓✓✓✓0.50✓✓✓0.670.600.600.10✓
68inception/mercury-2.5-preview
📄 view answers vs correct answer →open		0.86827/40$0.1023s✓✓✓✓✓✓0.83✓0.750.60✓0.750.17✓0.17✓0.80✓✓✓✓0.60✓✓✓✓✓✓✓✓✓✓✓0.50✓0.830.80✓0.400.50
69deepseek/deepseek-v3.2
📄 view answers vs correct answer →open		0.86323/40$0.97220s0.80✓✓✓✓0.60✓✓0.750.40✓0.500.83✓0.67✓✓0.25✓0.750.750.80✓0.75✓0.60✓✓✓0.86✓0.830.50✓✓✓✓✓0.90✓
70google/gemini-3-pro-image
📄 view answers vs correct answer →open		0.86325/40$3.8066s0.80✓0.67✓✓0.80✓✓✓0.800.80✓0.83✓✓✓0.60✓0.80✓✓✓✓✓✓✓✓✓0.75✓✓✓0.750.67✓0.17✗✓0.600.50
71poolside/laguna-xs-2.1
📄 view answers vs correct answer →open		0.85420/40$0.36623s0.80✓0.67✓✓0.800.83✓✓0.80✓0.500.670.830.830.600.60✓0.80✓✓✓0.80✓✓✓✓✓0.75✓✓✓✓0.83✓0.830.800.600.300.50
72aion-labs/aion-2.0
📄 view answers vs correct answer →open		0.85326/40$0.76125s✓0.33✓✓✓✓✓✓✓0.80✓✓0.33✓✗0.800.800.750.80✓0.75✓✓✓✓✓✓✓✓✓✓0.83✓✗✓0.830.60✓✓0.50
73thinkingmachines/inkling-small
📄 view answers vs correct answer →open		0.85123/40$0.8958s✓✓0.670.750.67✓0.67✓✓✓✓0.750.67✓0.830.800.800.75✓0.500.25✓✓✓✓✓✓✗0.75✓✓✓✓✓0.80✓0.600.80✓✓
74google/gemma-4-26b-a4b-it
📄 view answers vs correct answer →open		0.84520/40$0.111002s✓✓0.67✓✓0.80✓✓0.750.800.80✗0.17✓0.83✓0.800.750.800.50✓✓0.80✓✓✓✓✓0.75✓✓✓0.750.83✓✓0.800.800.900.50
75openai/gpt-5-nano
📄 view answers vs correct answer →open		0.79417/40$0.35106s✓✓0.67✓✓0.800.83✓✓0.60✓✗0.50✓0.83✓✓0.750.800.500.75✓0.60✓0.80✓✓0.67✓✓✓0.830.750.830.800.330.600.600.400.50
76mistralai/devstral-2512
📄 view answers vs correct answer →open		0.79116/40$0.6930s0.60✓0.67✓✓0.400.830.750.750.80✓0.750.33✓0.67✓0.600.750.80✓✓0.80✓✓0.80✓✓✓0.75✓0.200.830.750.500.60✓0.60✓0.600.50
77nvidia/nemotron-3.5-lightning
📄 view answers vs correct answer →open		0.78516/40$0.3222s0.80✓0.67✓0.670.400.83✓0.750.80✓0.750.330.670.50✓0.800.750.80✓0.75✗✓✓✓✓✓✓0.500.86✓✓0.750.83✓✓0.600.600.500.50
78inception/mercury-2
📄 view answers vs correct answer →open		0.78419/40$0.82746s✓✓0.67✓✓✓0.50✓✓0.40✓0.500.33✓✓✓0.800.25✓✓0.750.80✓0.75✓0.800.75✓0.500.43✓0.330.750.83✓✓0.800.800.300.33
79openai/o4-mini
📄 view answers vs correct answer →open		0.77218/40$1.7653s0.80✓0.67✓✓0.800.83✓0.750.60✓0.250.33✓✓✓0.80✓0.800.50✓✓0.800.750.80✓✓✓0.75✓✓0.33✓0.33✓✗0.200.800.500.50
80mistralai/ministral-14b-2512
📄 view answers vs correct answer →open		0.75519/40$0.4365s0.80✓0.67✓✓0.800.83✓0.750.80✓0.500.17✓0.33✓0.600.750.600.75✓0.80✓✓✓✓✓✓✓✗✓0.67✓0.33✓0.330.20✓✗0.50
81inclusionai/ling-3.0-flash-fin
📄 view answers vs correct answer →open		0.74517/40$0.15727s✓✓0.67✓✓0.800.83✓0.500.60✓0.250.17✓0.830.60✓0.50✓✓0.750.40✓✓0.800.80✓✓✓0.860.600.83✓0.33✓0.170.600.200.200.50
82mistralai/ministral-3b-2512
📄 view answers vs correct answer →open		0.74116/40$0.2271s0.80✓0.67✓✓0.600.33✓0.500.60✓0.500.67✓0.83✓0.600.500.60✓0.750.800.80✓0.80✓✓✓✓✗✓0.67✓0.17✓0.670.600.600.100.50
83mistralai/mistral-medium-3-5
📄 view answers vs correct answer →open		0.73815/40$1.49873s0.60✓0.67✓✓0.20✓0.75✓0.600.80✗0.50✓✓✓0.800.75✓0.50✓0.80✓✓0.80✓✓✓0.500.860.200.500.250.830.600.830.800.600.300.50
84nvidia/nemotron-3-nano-30b-a3b
📄 view answers vs correct answer →open		0.72815/40$0.73133s✓✓0.67✓✓0.800.50✓0.750.60✓✗0.50✓0.67✓0.600.25✓0.250.750.800.800.750.800.80✓✓✓✓✓✓0.750.330.800.83✗0.60✗0.50
85openai/o3
📄 view answers vs correct answer →open		0.71517/40$4.1343s0.80✓0.67✓✓0.60✓0.750.750.60✓✗0.17✓✗✓0.800.500.20✓✓✓✓0.75✓0.80✓✓0.75✓0.200.50✓0.330.800.830.20✓0.60✗
86mistralai/mistral-small-2603
📄 view answers vs correct answer →open		0.71110/40$0.139s0.80✓0.67✓✓0.800.830.750.750.60✓0.750.170.830.67✓0.600.750.800.500.750.800.60✓✓✓✓✓0.750.860.200.670.750.170.800.330.200.400.400.50
87meta-llama/llama-4-maverick
📄 view answers vs correct answer →open		0.69914/40$0.24151s✓✓0.67✗✓0.800.83✓0.750.60✓✗0.33✓0.67✓0.600.500.800.250.750.800.80✓0.80✓✓✓✓0.860.800.17✓✗✓0.330.200.800.500.33
88amazon/nova-premier-v1
📄 view answers vs correct answer →open		0.69816/40$10.09102s0.80✓✗✓✓0.20✓0.75✓0.60✓✗0.67✓0.33✓0.600.500.80✓0.25✓0.800.75✓✓✓✓0.750.570.800.500.750.33✓0.170.20✓0.300.50
89meta-llama/llama-3.3-70b-instruct
📄 view answers vs correct answer →open		0.68316/40$0.0575s0.80✓0.67✓✓0.600.670.50✓0.200.80✗0.67✓0.330.200.600.750.20✓✓✓0.60✓0.80✓✓✓✓✗0.400.50✓✗✓0.830.600.60✗✓
90upstage/solar-pro-3
📄 view answers vs correct answer →open		0.64513/40$0.69182s0.400.67✗✓✓0.800.50✓0.500.40✓✗0.33✓0.67✓0.800.250.800.750.75✓✓0.75✓0.40✓✓0.50✗0.800.330.50✗✓0.17✓0.400.500.83
91amazon/nova-pro-v1
📄 view answers vs correct answer →open		0.6199/40$4.8148s✓✓✗✓✓0.600.170.250.250.200.800.250.330.500.500.800.800.250.800.250.750.800.80✓0.80✓✓✓0.50✓0.800.500.750.170.800.330.800.400.300.50
92ibm-granite/granite-4.0-h-micro
📄 view answers vs correct answer →open		0.1240/40$0.002s0.600.33✗✗0.67✗0.17✗✗✗✗✗0.170.50✗0.20✗0.250.20✗0.50✗✗✗✗✗0.250.17✗✗0.20✗✗✗0.40✗✗0.20✗0.17
93minimax/minimax-m2-her
📄 view answers vs correct answer →open		0.1240/40$0.0068s0.600.33✗✗0.67✗0.17✗✗✗✗✗0.170.50✗0.20✗0.250.20✗0.50✗✗✗✗✗0.250.17✗✗0.20✗✗✗0.40✗✗0.20✗0.17
94thedrummer/cydonia-24b-v4.1
📄 view answers vs correct answer →open		0.1240/40$0.0046s0.600.33✗✗0.67✗0.17✗✗✗✗✗0.170.50✗0.20✗0.250.20✗0.50✗✗✗✗✗0.250.17✗✗0.20✗✗✗0.40✗✗0.20✗0.17
95kwaipilot/kat-coder-pro-v2
📄 view answers vs correct answer →open		0.1110/40$0.0028s0.600.33✗✗0.67✗0.17✗✗✗✗✗0.170.50✗0.20✗0.250.20✗0.50✗✗✗✗✗0.25✗✗✗0.20✗✗✗0.20✗✗0.20✗✗
96kwaipilot/kat-coder-pro-v2.5
📄 view answers vs correct answer →open		0.1110/40$0.008s0.600.33✗✗0.67✗0.17✗✗✗✗✗0.170.50✗0.20✗0.250.20✗0.50✗✗✗✗✗0.25✗✗✗0.20✗✗✗0.20✗✗0.20✗✗
—poolside/laguna-s-2.1
📄 view answers vs correct answer →openpartial · 39/40		0.939 (partial)33/39$0.27173s✓0.33✓✓✓✓✓✓✓0.60✓✓✓0.67✓✓—✓✓✓0.75✓✓✓✓✓✓✓✓✓✓✓✓0.67✓✓0.60✓✓✓
—ibm-granite/granite-4.2-8b
📄 view answers vs correct answer →openpartial · 39/40		0.912 (partial)24/39$0.36310s0.80✓0.67✓✓✓0.83✓✓0.800.800.75✓✓0.83✓—0.75✓✓✓✓✓0.750.80✓✓✓✓✓✓✓✓0.83✓0.830.800.800.50✓
—aion-labs/aion-3.0
📄 view answers vs correct answer →openpartial · 39/40		0.904 (partial)32/39$3.54913s✓✓✓✓✓✓✓✓✓✓✓✓—✓✗✓0.80✓0.20✓✓✓✓✗✓✓✓✓✓✓✓✓✓0.67✓✓0.800.80✓✓
—z-ai/glm-5v-turbo
📄 view answers vs correct answer →openpartial · 36/40		0.967 (partial)31/36$1.81154s0.80✓✓✓✓✓✓0.75✓0.80✓—✓✓0.67✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓—✓✓✓—0.80✓✓—
—z-ai/glm-5-turbo
📄 view answers vs correct answer →openpartial · 31/40		0.966 (partial)27/31$1.21141s✓✓✓✓✓✓✓✓✓0.80✓——✓—✓0.80✓—✓✓✓✓0.75✓✓✓✓✓✓——✓—✓—0.60✓✓—
—nex-agi/nex-n2-mini
📄 view answers vs correct answer →openpartial · 28/40		0.945 (partial)23/28$0.061445s✓✓✓✓✓✓✓✓✓0.80——0.330.67✓✓—0.75—✓✓✓——✓✓✓✓✓——✓——✓——✓0.90✓
—nex-agi/nex-n2-pro
📄 view answers vs correct answer →openpartial · 14/40		0.929 (partial)10/14$0.052298s—✓0.67✓✓—✓✓—————0.83———0.75—✓✓✓✓———✓—0.75———————————