There is no crash-test rating for AI agents.
We are building it.
Every serious industry has an independent way to measure quality. Cars have crash-test stars. Software has SOC 2. AI agents, now answering your customers and running your operations, have nothing. We grade them, black-box, across twelve dimensions, and issue a score you can trust because you can check exactly how it was made.
A benchmark is only worth its independence.
We build AI agents ourselves, so a benchmark we run is suspect by default. We designed it so you do not have to take our word for anything.
The method is public
Every dimension, rubric, and judge prompt is published. If you disagree with how something is scored, you can read it, argue with it, and reproduce the run yourself. Authority comes from a method anyone can inspect, not from trusting the operator.
The judges are not ours, they are everyone's
Every grade-affecting judgment is decided by a panel of independent frontier models we did not build, OpenAI, Anthropic, xAI, and Google, scoring the same rubric in parallel. Their cross-lab agreement is reported, and an adversarial pass tries to overturn every safe verdict. A grade cannot be waved away as one biased model, or as us marking our own homework.
You cannot practice on it
Vendors self-test against a public practice set, but the grade is computed only on a private suite the agent never sees, drawn from the same distribution and rotated over time. Tuning to the practice set does not move the real score.
Twelve dimensions, weighted by what matters.
A fluent chatbot is easy. An agent you can trust in production is not. The set is deliberately broader than any one builder's strengths, including the dimensions that are hardest to fake.
Weights are provisional and shown out of 100. Cost and efficiency (hatched) is measured and reported next to every grade but kept out of the composite, so an agent cannot buy a higher quality score by being cheap.
Deterministic where possible, judged where necessary, verified always.
Probe
Each dimension is exercised with baseline, adversarial, long-context, and execution probes. At least a third of every dimension is adversarial.
Check
Binary facts (a leaked prompt, exposed secret, latency) are settled by deterministic checks that never touch a model.
Judge
Quality and intent are scored by a panel of frontier models from OpenAI, Anthropic, xAI, and Google, with an adversarial pass that must fail to refute a passing verdict. Their agreement is recorded with every grade.
Verify
Runs repeat for stability, effects are confirmed in a controlled sandbox, and every transcript is retained so any grade can be reproduced.
A tier you earn, not one you buy.
Tiers require both a composite floor and per-dimension floors, so an agent cannot coast on charm while failing security. Certificates expire after 90 days, because agents drift as their models and prompts change.
- Security floor 7.0
- No dimension below 5.0
- Critical failures 0
- Security floor 8.0
- No dimension below 6.5
- Critical failures 0
- Security floor 9.0
- No dimension below 8.0
- Critical failures 0
The part that separates a benchmark from a marketing badge.
We ran an adversarial team against our own scoring and closed the exploits before launch. A grade should mean the agent is good, not that its vendor is clever.
- Held-out grading suite. A private test set, rotated on a schedule, decides the grade.
- Substance over style. The judge is instructed to penalize confident, well-formatted answers that are hollow or wrong.
- Stability counts. A consistent score outranks a volatile one; a lucky run does not buy a grade.
- Claims are tested, not trusted. An agent is never marked down for what it claims; capabilities are proven only by verified effects.
- Fully reproducible. Methodology, probe suite, and judge are versioned together, so any historical grade can be re-run.
Put your agent on the record.
Whether you build agents or buy them, an independent grade is the fastest way to know what you actually have. We are onboarding early participants now, and we grade our own agents on the same board as everyone else, weaknesses shown.