Pickbench
Sample report

Toyota vs Kia

Which brand 5 AI models prefer, overall and by buyer need when available

Configurations compared: 2026 Toyota Crown AWD (Auto (AV-S6), 4 cyl, 2.4 L, Turbo) vs 2026 Kia K4 4DR w/ DMS (Auto (AV-S1), 4 cyl, 2.0 L)

Toyota
Kia

Simulation: scenario paths made up to reproduce the published per-judge shares, not measured rows.

Too close to call: the confidence interval includes 50%

Published 95% confidence interval for Toyota: 37%–61%. Press Drop to watch simulated scenarios land.

How to read it

Each ball is a simulated buying scenario. Each row of pins is an AI model, pushing the ball toward the brand it picked.

One drop is one sample. Drop again and the pile shifts; the orange band shows where the balance lands in 95% of drops.

A leader is named only when the band clears the middle. The paths are simulated from the published per-judge shares: no model saw these scenarios.

Tap to drop. Hold to keep dropping.

How to read it

Each ball is a simulated buying scenario. Each row of pins is an AI model, pushing the ball toward the brand it picked.

One drop is one sample. Drop again and the pile shifts; the orange band shows where the balance lands in 95% of drops.

A leader is named only when the band clears the middle. The paths are simulated from the published per-judge shares: no model saw these scenarios.

The numbers behind the machine

AI consensus

Too close to call: the confidence interval includes 50%

Configurations compared: 2026 Toyota Crown AWD (Auto (AV-S6), 4 cyl, 2.4 L, Turbo) vs 2026 Kia K4 4DR w/ DMS (Auto (AV-S1), 4 cyl, 2.0 L)

  • Confidence interval for Toyota: 37%–61%
  • 95% confidence intervals (MOVER combination of the per-judge intervals, widened for small samples in battles and newer boards)

Orange: the 95% confidence interval for Toyota. Notch: the point estimate. Line: 50%.

How the judges lean

  • 0 judges lean to Toyota
  • 0 judges lean to Kia
  • 5 judges: too close to call

A judge leans to a brand only when its confidence interval excludes 50%. Point estimates alone never count as a win.

The consensus gives equal weight to each model family. Per-judge results below are the primary measure.

Fast panel: the fast or small current models of each vendor, not the full flagship panel.

About this battle

Category
Midsize sedans
As of
Edition
2026-10
Methodology
ACM-1.0
Judges
5
Scenarios
62
Precision
Measured

Vote

Which brand would you choose?

Your tap stays in this browser: nothing is sent.

Visitors' votes are shown next to the AI results for comparison only and are never mixed into the AI numbers.

Results computed but not published

Only the overall consensus and per-judge results are published for this battle: each buyer need has fewer than 30 scenarios, the pre-registered minimum for a need row.

See who was counted and why

Per judge

95% confidence intervals (hull of the scenario-clustered bootstrap and Wilson intervals, widened for small samples in battles and newer boards)

Conditional pick share for Toyota by judge (among scenarios where the judge picked a brand), with confidence intervals
Judge (API model)Share for ToyotaConfidence intervalLeaningInterval chart
anthropic/claude-haiku-5.5-20261007 @ claude-haiku-5.5-20261007 via OpenRouter51%38%–63%Too close to call
deepseek/deepseek-v4.1-flash-20260910 @ deepseek-v4.1-flash-20260910 via OpenRouter52%39%–65%Too close to call
google/gemini-4-argon-20261013 @ gemini-4-argon-20261013 via OpenRouter49%36%–62%Too close to call
openai/gpt-6-luna-20260922 @ gpt-6-luna-20260922 via OpenRouter47%34%–60%Too close to call
mistralai/mistral-large-4-0-20261006 @ mistral-large-4-0-20261006 via OpenRouter48%35%–61%Too close to call

Who was counted

The rule that decides which judges count was fixed before the run. Judges that met it are counted with the weights below; the others are listed with their reasons.

  • 62 buyer scenarios were planned.
  • None of them was left out.
Consensus weights of 5 counted judges
Weight of each counted judge in the consensus
Judge (API model)Model familyWeight in the consensus
anthropic/claude-haiku-5.5-20261007 @ claude-haiku-5.5-20261007 via OpenRouteranthropic20.0%
deepseek/deepseek-v4.1-flash-20260910 @ deepseek-v4.1-flash-20260910 via OpenRouterdeepseek20.0%
google/gemini-4-argon-20261013 @ gemini-4-argon-20261013 via OpenRoutergoogle20.0%
openai/gpt-6-luna-20260922 @ gpt-6-luna-20260922 via OpenRouteropenai20.0%
mistralai/mistral-large-4-0-20261006 @ mistral-large-4-0-20261006 via OpenRoutermistral20.0%

Judges not counted

Every judge on the panel met the rule.

Results computed but not published

Per-need scenario counts (n) are computed deterministically from the pre-registered plan.

Buyer need: Daily commute
n = 12 scenarios (fewer than 30) — Fewer scenarios than the pre-registered minimum
Buyer need: Driving fun
n = 13 scenarios (fewer than 30) — Fewer scenarios than the pre-registered minimum
Buyer need: Family
n = 15 scenarios (fewer than 30) — Fewer scenarios than the pre-registered minimum
Buyer need: Reliability
n = 11 scenarios (fewer than 30) — Fewer scenarios than the pre-registered minimum
Buyer need: Low running costs
n = 11 scenarios (fewer than 30) — Fewer scenarios than the pre-registered minimum

Methodology

Each judge is an AI model called through an API; “About this result” names each model and how it was called. In each buyer scenario, the judge chooses between the two brands. The scenario is the unit of analysis.

The results are based on 62 buyer scenarios.

A judge leans to a brand only when its confidence interval excludes 50%. Point estimates alone never count as a win.

Precision figures on this page are measured.

Consensus, needs and languages
95% confidence intervals (MOVER combination of the per-judge intervals, widened for small samples in battles and newer boards)
Per judge
95% confidence intervals (hull of the scenario-clustered bootstrap and Wilson intervals, widened for small samples in battles and newer boards)
Scenario languages
English

Data warnings

  • Models chose on EPA specification facts only; purchase price, reliability and availability were not assessed.

About this result

What was measured, how, when, and how to check it.

Methodology
ACM-1.0
Edition
2026-10
Measured on
AI models (API)
  • anthropic/claude-haiku-5.5-20261007 @ claude-haiku-5.5-20261007 via OpenRouter, via OpenRouter (pinned providers)
  • deepseek/deepseek-v4.1-flash-20260910 @ deepseek-v4.1-flash-20260910 via OpenRouter, via OpenRouter (pinned providers)
  • google/gemini-4-argon-20261013 @ gemini-4-argon-20261013 via OpenRouter, via OpenRouter (pinned providers)
  • openai/gpt-6-luna-20260922 @ gpt-6-luna-20260922 via OpenRouter, via OpenRouter (pinned providers)
  • mistralai/mistral-large-4-0-20261006 @ mistral-large-4-0-20261006 via OpenRouter, via OpenRouter (pinned providers)
Conditions
Each model was called through the route listed next to it and chose between the two brands in 62 buyer scenarios. Scenario languages: English.
Judges counted

The pre-registered inclusion rule counted 5 of the 5 judges on the panel. The results use 62 of the 62 planned scenarios.

See who was counted and why

Pre-registration (SHA-256)
f942c264540d4c7c6ef8fbd80c0b92ca18297986840aa1f3b6aee5c08b075e58
What the pre-registration fixes
Signed snapshot

Verify this result: download the signed snapshot (JSON)

Signing key (kid)
FnG9tqyrr6Y0YUh6teXhpERjAKUSfzL78m4WdnEZ9Jg
Production key
SHA-256 of the payload
e9f050df6e5a5771e91a18c1334cded909851987e6cd8cac17ec175094875eba

How to verify a snapshot

Rank, inclusion and influence on results are never sold.