How the scores work

Star ratings alone are a bad way to rank software. Everything bunches up between 4.2 and 4.8, and a tool with nine glowing reviews outranks one with nine hundred. Head-to-head votes fix that — asking “which of these two is better?” produces sharper separation — but they need volume before they settle.

So every board blends both:

• A trust-weighted average of star ratings, smoothed toward a prior so a handful of reviews cannot spike a score.
• An Elo rating replayed from every head-to-head vote ever cast, the same idea chess uses.

A new tool with one five-star review is not the best AI in the world. Each score starts anchored to a prior of 7.0 carrying the weight of 10 average votes, and real reviews pull it away from there. The more genuine ratings a tool collects, the less the prior matters.

Accounts earn influence by contributing. A brand-new account carries a fraction of the weight of someone with a long, well-regarded history, which makes mass-registering accounts an expensive way to move a number.

Head-to-head votes are weighted the same way: a vote from a trusted account moves Elo further, because the K-factor of 24 is scaled by that same multiplier.

On day one there are almost no reviews, so an average of them is noise. Elo works from the very first comparison, so it carries most of the weight early on. As real reviewers accumulate, the balance tips toward the review average, which is more stable at scale.

With no reviewers the board is 85% head-to-head. By a few thousand it settles at 15%, with reviews carrying the rest.

A task like write a blog post has no reviews of its own, so its ranking starts out derived from the ability scores it depends on — writing, reasoning, speed, value, each with a published weight shown on the task page. That works immediately, from data that already exists.

Then it inverts: as people vote head-to-head on that specific task, those votes progressively replace the derived estimate, down to a floor of 25%. Around 150 votes a task is mostly community-driven. Each task matures at its own pace, and every result is labelled with how much evidence sits behind it.

An AI company can claim its page and correct its own description, tools and pricing — the details they know best and that go stale otherwise. They cannot touch their score, their reviews, or the listed strengths and weaknesses.

That is not a promise, it is a property of the system. The database grants providers write access to exactly four marketing columns; the score, strengths and weaknesses columns are not writable by any signed-in account at all, and the ranking job reads none of the fields a provider can edit. Every edit a provider makes is recorded. We take no payment for placement, and there is no mechanism through which we could deliver it.

Every hour the whole thing is rebuilt: Elo replayed from the first vote onward, averages recomputed, every board re-ranked. Nothing is incrementally patched, so a score can always be reproduced from the raw votes. Snapshots are kept, which is where the movement arrows and sparklines come from.