Trends
Two years ago, about 210 people a month typed "vals ai" into Google. As of August 2026 it is 3,600, up 650% in a year, and the database classifies the pace as exponential. The company behind the name sells nothing a consumer can buy. It runs tests on other people's AI models and publishes the scores.
Here is the part nobody ranking for the term has noticed. While Vals AI went up 650%, the generic term "benchmarking" went down 18%, and "lmarena", the public leaderboard where anyone can vote on which model answered better, is down 45%. Evaluation is splitting away from benchmarking, and the money is following the half that happens behind a closed test set. That split is what this piece is about, and the numbers for it only exist in our data.
Key takeaways:
- "vals ai" grew from 210 to 3,600 US monthly searches in two years, up 650% year over year and 1614% over two, classified as EXPONENTIAL in the Rising Trends database (data as of August 2026).
- The trigger is dated. On August 13, 2026 the company announced a $40 million Series A at a $400 million valuation, led by a16z, and published it itself. That month search volume jumped from 1,900 to 3,600.
- The search curve broke out in April 2026, four months before the funding headline. Demand came first, the round confirmed it.
- The product is a test set you cannot train on. Vals publishes results only from a set that "remains private at all times", and licenses a separate private validation set to companies.
- The public leaderboards are moving the other way. "lmarena" is down 45% in a year and "benchmarking" is down 18%, while "artificial analysis" is up 514% and "ai evals" is up 2,400% over two years.
- The company says revenue grew 8x compared with all of 2025, the customer base doubled and the team tripled in six months, going from eight employees to roughly 30 this year.
Let's get into it.
The numbers
Here is the monthly search volume for "vals ai" over the last two years, from our database. It is flat for eighteen months and then it is not.
The shape matters more than the peak. Volume sat at 720 through the back half of 2025 and 880 through early 2026. April 2026 is the breakout month, the first time the term crossed a quarter of its eventual peak. June took it from 1,000 to 1,600, July to 1,900, and August to 3,600, an 89% jump in the month the Series A landed. Over two years the term is up 1614%.
Now put it next to the rest of the category. This is the chart that explains why a 3,600-search term is worth writing about at all.
The two falling terms are the two that describe measuring things in public. "Benchmarking" is the generic management word, 90,500 searches a month and down 18% in a year, flat across two. "lmarena" is the crowd-voting leaderboard, 33,100 a month and down 45%. Everything rising is a named independent evaluator or the act of running your own tests: vals ai up 650%, artificial analysis up 514%, ai benchmarks up 124%, ai evals up 2,400% over two years.
By raw volume, Vals is still the small name in a big room, which is the usual shape of a term that has just broken out.
A term at 3,600 a month sitting under terms three to twenty-five times bigger is not a weakness. It is the position "artificial analysis" was in two years ago, before it went up 6764%.
What Vals AI actually sells
The company calls itself the independent evaluator of artificial intelligence, and its own about page doubles as the diagnosis it sells into. Models summit old benchmarks in months. Open test sets get absorbed into pre-training corpora, which quietly invalidates every result built on them. And new benchmarks, in its words, are "often produced or run by the same companies that build the models".
The fix is boring and it is the whole business: keep the answers secret. The methodology page describes three tiers per benchmark. A public validation set anyone can inspect. A larger private validation set licensed to companies for their own internal testing. And a test set that "remains private at all times", which is the only one used for the scores Vals publishes. For open benchmarks like Terminal-Bench and SWE-bench, it runs every model itself under one harness so the numbers stay comparable.
The flagship output is the Vals Index.
Read the subtitle on that page and you can see the pitch to an enterprise buyer. The Vals Index is "a single measure of AI's potential economic impact", scoring agentic model performance across finance, coding, legal and tax tasks, weighted by each sector's share of US GDP. As of its October 13, 2026 update, Gemini 4 Argon leads at 68.90%, Claude Sonnet 5.5 is second at 67.04% and Claude Opus 5.5 third at 66.97%.
The column that makes it useful sits next to accuracy: cost per test. Gemini 4 Argon's 68.90% costs $15.68 a test. Claude Opus 5.5 scores 1.93 points lower and costs $32.14, more than double. GPT-6.1 Sol gets 61.15% for $3.24. A procurement team does not need the top of the leaderboard, it needs the point where the curve stops paying, and no lab's own announcement draws that line for you.
The self-serve product is Vals Smith. Connect a public or private GitHub repo and it mines merged pull requests into benchmark tasks, each with hidden tests that must fail before the original fix and pass after it. It turns your codebase into a private benchmark, so the question stops being "which model is best" and becomes "which model is best at my code". The logos on that page are Vercel, Exa, Fireworks AI, Thorin and Harvey.
Why it's suddenly everywhere
The dated trigger is August 13, 2026. Vals published its own Series A announcement, written by co-founder Rayan Krishnan: $40 million at a $400 million valuation, led by a16z, with existing investors 8VC, Pear VC and BloombergBeta joined by HRT Ventures and Next Ladder Ventures. Searches went from 1,900 in July to 3,600 in August.
The traction numbers in that post are the company's own and worth reading as such. Revenue grew 8x compared with all of 2025, the customer base doubled, and the team tripled in six months. Vals says its results have been cited in model cards from OpenAI, Anthropic, Google, Meta and xAI, and that it has supported the Department of Commerce and members of Congress on AI policy. Krishnan later told CNBC the headcount went from eight to roughly 30 this year.
Then the politics arrived. In September, Dario Amodei published We Must Pace the Frontier, arguing for a slower rate of capability gain with outside checks, and calling embedded third-party evaluators "the key step for verifiability of any pacing commitments". Anthropic committed to it, announcing it would embed evaluators from Accenture's Faculty business and fund that work directly, while admitting the obvious gap: "There are, as yet, no standards for what information embedded evaluators should have access to, or how they should report what they find. There is also no settled system for funding independent evaluation." At the end of September, President Trump presented a voluntary accord encouraging AI companies to partner with an independent external auditor or evaluator, in place of regulation.
A niche B2B measurement company became a policy instrument in six weeks. This clip from The Information, posted on October 2, 2026, is Krishnan being asked which lab reaches recursive self-improvement first, a question that would have gone to a lab researcher a year ago.
His answer in that clip, per the post's caption: "at the current pace, looking at the last year of model releases, we're projecting forward that if this pace continues, we'll see models start to meet human level researchers by August 2027." Whether that lands or not, the measurement company is now the one making the forecast, which is what we cover in self-improving AI, explained.
What developers do with the scores
The search curve is the demand signal. The behaviour behind it is consistently the same move: someone distrusts a lab's own numbers, so they go get a third-party score.
On Hacker News in February 2026, a developer pushing back on a model release put it plainly: "I don't usually trust most of what any of these vendors claim in their release notes without a third party," linking the Vals SWE-bench page as what they were actually looking for. In October, a commenter dismissed Mistral Large 4 with one line and one link, saying it looked "about a year behind still" on the Vals Index.
The cost column gets used as a weapon too. After Claude Opus 5 shipped, one commenter worked through both leaderboards and found the price story unflattering: "Vals Index Opus 4.8 > 5.0 goes from $2.90 to $8.54, for 4% gain ... That is a massive cost increase." That is the product being used in the wild. Not a ranking, an argument.
The scepticism cuts both ways, and it should. When Vals wrote up zAI's legal model in October, a reader noted the post "totally excluded Meta Spark models" that were doing well on the same benchmark. An independent evaluator still chooses what to publish, and that choice is not covered by a private test set.
Who it's up against
| Name | Model | Searches (Aug 2026) | 1y change |
|---|---|---|---|
| Artificial Analysis | Public leaderboard, quality plus speed plus price | 40,500 | +514% |
| LMArena / Arena AI | Crowd voting, public | 33,100 | -45% |
| Vals AI | Private test sets, paid, professional tasks | 3,600 | +650% |
| METR | Nonprofit, frontier risk and autonomy | not tracked as a search term | n/a |
Artificial Analysis is the one to watch, because it is winning on volume and converging on the same business. It describes itself as "Independent analysis of AI" across quality, speed and cost, it publishes faster, it covers more models, and it has now launched Optima, its own custom-benchmark builder. Two companies racing from opposite ends toward the same product: a public index for reach, a private benchmark for revenue.
LMArena is the cautionary tale. Crowd voting made it the default scoreboard of 2024 and 2025, and it is down 45% in a year, with lmarena.ai now redirecting to arena.ai, which still bills itself as "the public leaderboard". Part of the decline is a rebrand splitting its own search demand. The rest is the problem a vote cannot fix: it measures which answer a stranger preferred in a chat window, not whether a model can close a month's books.
METR is the nonprofit comparison, and the money shows how fast this category is repricing. In August 2026 it said it had raised around $71 million in commitments over six months, against $13.6 million in total contributions for all of 2024. The same curve as Vals' search term, drawn in donations.
What it means for the industry
Measurement is becoming a paid layer of the AI stack. Every number in a model card is produced by someone with an interest in it being high. As AI agents move from demos into systems that spend money and touch records, "we tested it ourselves" stops being acceptable procurement. The buyers are the enterprises running generative AI pilots that now need to show a return, and a GDP-weighted index with a cost column is a budget document as much as a leaderboard.
The cost column is the pricing pressure. When an independent scoreboard prints accuracy and dollars per test side by side, a lab cannot ship a 4% gain at triple the price without someone doing the arithmetic in public, which is exactly what happened to Opus 5 on Hacker News. That transparency is why AI routers exist, and Vals has already published its own finding there: a router that picked the best model for every task in hindsight would beat the best single model by 14 points at about 40% lower cost.
Independence is a business model problem, not just an ethical one. The labs being measured are also the largest potential customers. Vals went for-profit and raised venture money; METR stayed a nonprofit and raised donations; Anthropic is directly funding Accenture to evaluate Anthropic. None of those three structures has solved the conflict, and Anthropic's own post says the long-term answer should be "pooled or government sources". The companies evaluating AI safety are now a cybersecurity-adjacent category in their own right, which is why we track them alongside the cybersecurity trends of 2026.
Expert judgment is the moat. The same shift that made micro1 a 33,100-search employer brand, labs paying experts to produce what a model cannot scrape, is what makes a professional benchmark expensive to build. Vals partners with reference institutions in each field to build its task taxonomies. That is why a benchmark company can be worth $400 million.
Where this is heading
Watch whether Vals stays a measurement company. Vals Smith sells a product rather than a report, and the Web Search Index it is building with Exa, Parallel and Tavily as partners is a benchmark funded by the companies inside it. That is a reasonable way to pay for independent testing and a reasonable thing to be nervous about.
Watch the regulation path. Right now the US answer is a voluntary accord. If licensed evaluators ever become a legal requirement, the companies already cited in five labs' model cards start from the front of the queue. Our AI trends for 2026 hub tracks that policy line as it moves.
Three search terms to watch in our data. vals ai tells you whether this is a funding spike or a category. ai evals tells you whether the practice is spreading to teams who will never buy a benchmark, since at 1,000 searches a month it is still tiny but up 2,400% in two years. And ai benchmarks, at 12,100 and up 124%, is the bridge term: if it keeps climbing while "benchmarking" keeps falling, the vocabulary shift is finished and the category has a new name.
The thing worth holding on to is the timing. Our data says the breakout was April 2026, four months before the round that got written up everywhere. A measurement company does not get searched because it raised money. It gets searched because enough people stopped believing the numbers they were given, and went looking for someone else's. The same instinct is why readers now check whether a model's own detection claims hold up, and why an independent evaluation of a new model is more interesting than its launch post.
Want to spot the next breakout company before the funding headline? Read our guide on how to identify market trends, follow the live vals ai trend page, or browse what is breaking out right now on the Rising Trends dashboard.
