Agents

What does the agents leaderboard rank?

It follows the LMArena agent leaderboard: models are given tasks that take several steps and require calling tools — searching, reading files, running code — and people judge which model handled the task better. Instead of single answers, whole sessions are compared.

Why is there a Score instead of an Elo?

This leaderboard publishes a score centred on zero rather than an Elo rating. A model above zero was preferred more often than the average model of the board, a model below zero less often. The column is labelled "Score" and shows three decimals, and the votes counted are sessions. Because the scale is different, never compare these numbers with the Elo ratings of the text or coding boards; compare models within this board only.

Are tiers calculated the same way?

Yes. The tier rule depends only on the confidence intervals and the share of the board, not on the scale. A model needs 300 sessions to be tiered. Its place is one plus the number of models whose whole interval lies above its own; S goes to the first 10% of places, A up to 30%, B up to 60%, C up to 85%, D to the rest.

What else does each row show?

• the vendor and whether the weights are open;
• the cheapest price per million tokens among providers, from models.dev and LiteLLM;
• the context window — long contexts matter for agents that read a lot;
• the price trend over 30 days.

What should I be careful about?

The board is younger than the text arena and lists fewer models. The agent framework used in the arena is also not the one you will use: a model that performs well there can behave differently inside your own coding agent or automation. Check the coding agents page for the tools themselves, and try the shortlisted models on a real task before committing.