Arena AI — formerly LMArena, originally Chatbot Arena — is today the most popular public leaderboard ranking large language models (LLMs). 6 million votes cast, a live-updated ranking, and a startup valuation of $1.7 billion after the Series A round in January 2026. Arena is the informal but real arbiter of the question “which AI is best right now” — and that has direct consequences for GEO and SEO strategy in the AI era.

This guide is everything an SEO specialist, copywriter, and brand manager should know about Arena in 2026: how the methodology works, why it rebranded from a university project, what its real controversies look like (the “Leaderboard Illusion” paper from April 2025), and — most importantly — how to read the leaderboard for content strategy.

What Arena AI is

Arena is a public web platform that ranks LLMs based on anonymous head-to-head voting. The mechanism is simple:

  1. You enter a prompt.
  2. You get two responses from two random models — without knowing which is which.
  3. You pick the better one (or tie / “both bad”).
  4. Your vote feeds the central ranking.

After voting, the model identities are revealed. Scale: over 6M votes from users worldwide — making Arena the largest crowdsourced LLM benchmark in 2026 (arXiv 2403.04132 — Chatbot Arena: Open Platform for Evaluating LLMs by Human Preference).

Unlike static benchmarks like MMLU or HumanEval — which test models on a fixed question set — Arena measures subjective user preference on open-ended prompts. That’s its greatest strength and its biggest weakness, which we’ll come back to in the controversy section.

A short history: from a Berkeley project to a $1.7B startup

Arena started as an academic project at UC Berkeley Sky Computing Lab under the name Chatbot Arena, as part of the LMSYS research group. The creators — initially two Berkeley roommates — wanted to build a neutral benchmark for comparing LLMs (founded.com — founders’ story).

Timeline:

Two years from a student project to a unicorn is a typical trajectory in the AI era, but Arena has one peculiarity: the company has retained its research ethos despite commercial scale. It was only in 2026 that this neutrality was first seriously challenged — more on that below.

Founders and capital

The core team:

  • Anastasios N. Angelopoulos — CEO, Berkeley PhD, statistician specializing in hypothesis testing and prediction-powered inference.
  • Wei-Lin Chiang — CTO, one of the authors of the original Chatbot Arena paper.
  • Ion Stoica — co-founder and advisor, Berkeley professor, co-founder of Databricks and Anyscale, one of the most recognizable figures in AI infrastructure.

Series A investors include Andreessen Horowitz (a16z) and others. The team recruits from Google, DeepMind, Discord, Vercel, Berkeley, and Stanford.

How Arena works — battle mode and blind voting

The concrete flow:

  1. The user picks a mode (Battle Mode, Direct Chat, Vision Battle, etc.) — most votes come from Battle Mode.
  2. Enters a prompt in natural language.
  3. The system samples two models from the current pool (50+ commercial and open-source models).
  4. Models respond — the user sees both responses anonymously (labeled “Model A” and “Model B”).
  5. The user votes: A better, B better, tie, or both bad.
  6. After voting, model identities are revealed.

Each vote is one observation in a giant preference matrix. The central piece of methodology — how rankings emerge from those votes — is built on statistics, which we’ll dive into next.

Ranking methodology: Bradley-Terry instead of Elo

Originally, Arena used classic Elo ranking (familiar from chess). Since late 2023 the platform migrated to the Bradley-Terry (BT) model (LMArena blog — Statistical Extensions of Bradley-Terry and Elo Models, arXiv 2412.18407 — Statistical Framework for Ranking LLM-Based Chatbots).

Why not classic Elo

Classic Elo is an online algorithm: after each match, both players’ ratings are updated incrementally. That makes sense in chess, where:

  • Each player plays hundreds of matches across years.
  • Player form changes (better today, worse tomorrow).
  • Centralized access to the full match history may be impossible.

None of those conditions apply to LLMs:

  • Models are static — GPT-5 from June 2026 is the same model as GPT-5 from July, unless OpenAI ships a new version.
  • We have the full match history — an online algorithm isn’t needed.
  • Match order shouldn’t matter — model X lost to Y in January, won against Y in February; both matches are equivalent.

What Bradley-Terry offers

BT is a maximum likelihood estimator (MLE) for pairwise win-rates. The model assumes each player has a fixed but unknown “strength,” and the probability of winning depends only on the strength difference between players. This gives:

  • More stable ratings — less dependent on match order.
  • Real confidence intervals (bootstrap) — you can see whether the difference between model #2 and #3 is statistically significant or noise.
  • Decomposition by category (coding, math, multilingual) without breaking the global ranking.

Practical consequence for the leaderboard reader: don’t look only at single positions — look at confidence intervals. Between positions #2 and #5 there’s often no statistically significant difference.

What you’ll find on the 2026 leaderboard

Arena maintains several rankings in parallel:

  • Overall Leaderboard — generic ranking across all categories. Most often cited in the media.
  • Coding Arena — programming tasks; here Anthropic Claude and specialized models like Cursor or Cognition Devin dominate.
  • Math Arena — math problems; OpenAI reasoning models (o1, o3) usually lead.
  • Vision Arena — comparisons on prompts with images.
  • Hard Prompts Arena — questions filtered as harder, less susceptible to format gaming.
  • Multilingual Arenas — per-language rankings (smaller vote base, so wider confidence intervals).
  • Style Control — experimental ranking with corrections for response length and markdown format.

Practical takeaway: the Overall leaderboard is the least informative if you know what you need a model for. A code-assistant team looks at Coding; an analytics firm at Math; a content agency at Multilingual + Hard Prompts.

”The Leaderboard Illusion” — the April 2025 controversy

In April 2025, a team from Cohere Labs, AI2, Princeton, Stanford, Waterloo, and the University of Washington published a 68-page paper titled The Leaderboard Illusion, alleging systemic irregularities in how Arena tests models.

Key allegations:

  • Selective disclosure: large labs (Meta, OpenAI, Google, Amazon) had the ability to privately test multiple variants of the same model and publish only the best score.
  • Concrete Llama 4 case: Meta privately tested 27 Llama 4 variants between January and March 2025; on launch day, only one score was disclosed publicly — one that happened to land near the top of the leaderboard.
  • Effect at scale: the paper shows large-budget labs systematically use the private testing mechanism, while smaller players get one public shot.

Sources:

LMArena’s response

Ion Stoica called the paper “full of inaccuracies” and “questionable” (LMArena blog — Our response). The platform’s arguments:

  • Any provider can submit as many variants as they want — large labs submit more because they develop more models, not because they got special access.
  • The published ranking is based on disclosed models, so the publication decision belongs to the vendor, not Arena.
  • Since 2025 the platform has introduced some reforms (greater transparency, a clear model testing policy), but has not eliminated private testing — because smaller labs also need it to iterate before a public debut.

What this means in practice

Regardless of which side you take, the fact is: the Overall leaderboard is skewed toward big labs in a way that doesn’t necessarily reflect “the truth about quality.” For the reader, this is an argument to:

  • Look at specialized rankings rather than Overall.
  • Compare Arena with alternative benchmarks (HumanEval, MMLU, SWE-Bench, GAIA).
  • Account for the fact that the latest models are often better tuned to “Arena style” than earlier ones.

Goodhart’s Law — gaming style instead of quality

A second layer of criticism doesn’t concern vendor gaming, but the very nature of crowdsourced voting. Independent analyses (Collinear — Goodhart’s Law in AI Leaderboard Controversy) show that users systematically reward:

  • Bulleted lists over flowing prose.
  • A specific response length (~200–400 words for general queries) — shorter ones are rated as “lazy,” longer ones as “rambling.”
  • Markdown formatting (headings, bolds).
  • Tonal confidence over a nuanced answer.

Consequence: a model that has learned the Arena style gets higher ratings even without a real increase in substantive quality. This is a textbook example of Goodhart’s Law: “when a measure becomes a target, it ceases to be a good measure.”

In addition, independent researchers have shown the ranking can be manipulated with just a few hundred coordinated votes (OpenReview — Improving Your Model Ranking on Chatbot Arena by Vote Rigging). Not cheap manipulation, but for a lab with a marketing budget — fully achievable.

Why an SEO and marketer should check Arena

If the leaderboard has real flaws, why look at it at all?

Three concrete reasons:

1. Choosing the model you optimize GEO for

GEO (Generative Engine Optimization) requires knowing which AI engine actually serves your customers. If the Arena ranking shows that GPT-5 dominates “business inquiries” while Claude leads “technical writing” — your B2B content strategy should be optimized for GPT-5 first, with technical content tuned for Claude.

New models appear on Arena weeks before full commercial launches. Tracking new names on the leaderboard lets you predict which model will be cited in AI Overviews 3–6 months out — and prepare content that AI is willing to cite before competitors notice the shift.

3. Calibrating your own tests

Most content teams use a “walking prompt” — their own queries to evaluate AI quality. The problem: those prompts are usually narrow and may favor one model. Arena, despite its flaws, is a much wider sample of human preference. If your internal evaluation diverges from Arena, that’s a signal — either you have a unique use case (great!) or your evaluation is biased (time to revisit).

How to read the leaderboard for content strategy

Practical rules

1. Skip Overall, go to specialized rankings. Coding, Multilingual (per language), Hard Prompts — those tell you something meaningful about real use cases.

2. Look at confidence intervals, not positions. The difference between #2 and #5 is often within noise. The threshold where one rank step differs statistically from the next is typically 30–50 points of BT score.

3. Watch trends, not snapshots. A single leaderboard snapshot is uninformative — track which model is moving. A model rising fast from #15 to #5 is often a better 6-month bet than the steady leader holding #1 for months.

4. Combine with other benchmarks. Arena + MMLU + SWE-Bench + GAIA gives a four-dimensional view. Arena Overall alone is a summary.

5. Don’t believe the Overall “winner-takes-all” narrative. Even in 2026, the top 3 models have their strengths and weaknesses. Content strategy should be multi-model, not “one true LLM.”

Concrete workflow for a marketing team

  • Weekly: check movement in the Top 10 (any new names).
  • Monthly: update the list of models you test your content against (see new SEO KPIs: AI citation).
  • Quarterly: review specialized categories matching your industry.
  • After every major launch (GPT-X, Claude X, Gemini X): 2–3 weeks of observation on how the new model ranks before strategic decisions.

Limitations and risks of the leaderboard

Summary of the flaws to keep in mind:

  • Subjectivity — voter preference doesn’t always match objective quality.
  • Format gaming — models learning the Arena style get a boost unrelated to substance.
  • Selective disclosure by large labs (the Leaderboard Illusion controversy).
  • Narrow user sample — voters are mostly tech-savvy English-speaking population, not necessarily representative.
  • Volatility — positions shift weekly, especially after launches. Strategic decisions based on a leaderboard snapshot are risky.
  • Specialized categories have smaller samples — confidence intervals in non-English or Vision rankings are wider than in Overall.

Alternative and complementary benchmarks

Arena isn’t the only benchmark — and shouldn’t be your only decision input.

  • MMLU (Massive Multitask Language Understanding) — static general-knowledge benchmark across 57 domains.
  • HumanEval / MBPP — programming tasks (static, automatically evaluated).
  • SWE-Bench / SWE-Bench Verified — real GitHub issues, measures bug-fixing capability.
  • GAIA — agentic benchmark with real internet browsing.
  • GPQA Diamond — extremely hard scientific questions (Google-proof Q&A).
  • MT-Bench — GPT-4-judged multi-turn conversation evaluations.

Each has its drawbacks, but the combination of Arena + MMLU + GPQA + SWE-Bench gives a much fuller picture than any single ranking.

Practical checklist for a marketing team

If you run a marketing agency or a content team, in 2026 the minimum is:

  • An arena.ai account — even just for occasional blind tests of your own.
  • A list of 3–5 models you write AI-citable content for, refreshed quarterly.
  • A subscription to news.lmarena.ai — alerts on major methodology changes.
  • A combination of Arena + at least one static benchmark (MMLU/SWE-Bench) in strategic decisions.
  • A monthly test of your own site’s citation rate across the top 3 models — compare with making your website visible to LLMs.

What’s next for Arena

After Series A, Arena faces several decisions that will shape what it looks like in 2027:

  • Commercialization: how to monetize the platform without losing research trust? Premium features, API, enterprise audits? Each path has trade-offs.
  • Reforms after Leaderboard Illusion: how much transparency around private testing and selective disclosure will they introduce?
  • Internationalization: non-English rankings are under-sampled today. Will Arena invest in voter localization?
  • Competition: alternative benchmarks (e.g., closed enterprise audits) are emerging and could shift some users.

For SEO/GEO practitioners: regardless of Arena’s direction, a public LLM leaderboard now exists permanently. What may change is the brand and methodology — not the fact that such a platform is part of the ecosystem.

Często zadawane pytania

Is Arena AI the same as Chatbot Arena?

Yes. Chatbot Arena (launched 2023) → LMArena → Arena (rebrand January 28, 2026). It's the same platform, the same team, the same leaderboard. Old links to lmarena.ai usually redirect to arena.ai.

Where does Arena get money from if it's free for users?

In 2025 a $100M seed, in January 2026 a $150M Series A at a $1.7B valuation. Investors include Andreessen Horowitz. Monetization plans: enterprise audits, API, premium analytics. Use remains free.

Is Arena's ranking trustworthy after the 'Leaderboard Illusion' controversy?

Trustworthy within limits. The Overall leaderboard has known flaws (selective disclosure by large labs, format gaming). Specialized rankings (Coding, Math, Hard Prompts) are more reliable. Best practice: combine Arena + MMLU + SWE-Bench + GAIA.

Which model is currently winning on Arena?

Positions shift weekly. In April 2026 the Top 3 typically includes GPT-5 variants, Claude, and Gemini, with rotation by category. Check the live ranking on arena.ai — a snapshot at publication time goes stale fast.

How does Bradley-Terry differ from classic Elo?

Classic Elo is an online algorithm: ratings update step by step after each match. Bradley-Terry is a maximum likelihood estimator (MLE) on the full match history, assuming fixed player strength. BT has more stable ratings and real confidence intervals — a better fit for static LLMs where match order shouldn't matter.

Should a small marketing agency check Arena?

Yes — it's a free benchmark that helps pick models for testing your own content's citation rate (GEO/AEO). Minimum workflow: monthly check of the Top 10 and categories matching your clients' industries. Make strategic decisions based on trends, not single leaderboard snapshots.

Conclusions

Arena AI is today the closest public approximation to the question “which LLM is best,” but it’s not an objective verdict — it’s a crowdsourced ranking of human preference with concrete methodological flaws. For an SEO and marketer in 2026, the value of Arena isn’t “who wins,” but what’s changing and how: which models are gaining, how new players are growing share, which categories reward different response styles.

Three things worth remembering:

  1. Specialized rankings > Overall. Coding, Multilingual, Hard Prompts say more than the main leaderboard.
  2. Combine benchmarks. Arena + MMLU + SWE-Bench + GAIA = a credible picture.
  3. Read the leaderboard as a trend, not a snapshot. Strategic decisions based on a single state are risky.

A well-run GEO and AEO strategy uses Arena as one of 4–5 signals — not as a single source of truth.

Sources

Official Arena sources

Encyclopedic and overview references

Funding and business

Academic papers and methodology

Criticism, controversies, and analysis