Pickbench
Contact

Password managers

Which brand 5 AI models pick, with intervals.

48 scenariosBrand names only

Fast panel · Panel, equal weight

  • anthropic/claude-haiku-5.5 @ claude-haiku-5.5 via OpenRouter (20%)
  • openai/gpt-6-luna @ gpt-6-luna via OpenRouter (20%)
  • google/gemini-4-argon @ gemini-4-argon via OpenRouter (20%)
  • mistralai/mistral-large-4-0 @ mistral-large-4-0 via OpenRouter (20%)
  • deepseek/deepseek-v4.1-flash @ deepseek-v4.1-flash via OpenRouter (20%)

Too closeNo single favourite: 1Password, Bitwarden, Keeper, and Proton Pass share the top tier for these 5 API models.

Choice share · 95% interval
0255075100

Tier 1

  1. Bitwarden59%95% interval 46–73%plausible rank 1–4
  2. 1Password57%95% interval 44–69%plausible rank 1–3
  3. Keeper28%95% interval 15–42%plausible rank 2–10
  4. Proton Pass27%95% interval 15–41%plausible rank 1–10

Tier 2

  1. Dashlane20%95% interval 10–32%plausible rank 3–10
  2. RoboForm18%95% interval 7–33%plausible rank 3–10
  3. Enpass15%95% interval 4–29%plausible rank 3–10
  4. Zoho Vault10%95% interval 4–20%plausible rank 3–10
  5. NordPass8%95% interval 2–18%plausible rank 3–10
  6. LastPass7%95% interval 2–15%plausible rank 3–10

Changes since the previous edition

The judge panel changed since the previous edition, so shares are not compared between the two.

  • google/gemini-4-argon @ gemini-4-argon via OpenRouter: added in this edition
  • mistralai/mistral-large-4-0 @ mistral-large-4-0 via OpenRouter: same model, new judge settings
  • deepseek/deepseek-v4.1-flash @ deepseek-v4.1-flash via OpenRouter: added in this edition

This note was added after publication from the published editions; the signed snapshot is unchanged. Corrections log

How to read this
  • Choice share: how often models pick a brand when it appears in a shortlist of 4. Chance is 25%, the dashed line.
  • The band is the 95% interval; the bright tick is the estimate. Overlapping bands are not a strict order.
  • Tiers are published: a tier groups brands not distinguishable from that tier's leader. Same tier does not mean equal.
  • Rank shows the plausible rank range, not a strict place.
  • Drag across the chart to probe a share and see which intervals contain it.
  • Brand names only: the models saw each option as a brand name, with no product facts, and answered from their own knowledge.
  • Fast panel: the fast or small current models of each vendor, not the full flagship panel.
  • Each page shows the latest edition. Dates are in the downloadable JSON.

Panel and conditions

Each edition puts the brands of the category to a panel of AI models, called through their APIs, in randomized shortlists of 4 (chance is 25%). Each option was a brand name only, with no product facts, prices or any commercial data, so a brand's share measures which brand the models recommend from their own knowledge for the shopper needs of the scenarios. Shares come with 95% intervals; tiers and rank intervals are simultaneous at the same level.

  • anthropic/claude-haiku-5.5 @ claude-haiku-5.5 via OpenRouter (20%)
  • openai/gpt-6-luna @ gpt-6-luna via OpenRouter (20%)
  • google/gemini-4-argon @ gemini-4-argon via OpenRouter (20%)
  • mistralai/mistral-large-4-0 @ mistral-large-4-0 via OpenRouter (20%)
  • deepseek/deepseek-v4.1-flash @ deepseek-v4.1-flash via OpenRouter (20%)

Conditions: closed-book API calls via OpenRouter (pinned providers); web search, grounding, tools and retrieval off; forced choice; 2 paraphrases per scenario in English.

Methodology: ACM-1.0-chart-draft (draft rules for movement, significance and events, awaiting review).

All boards