Hermes Index: a score and a cost per task for 14 models in Hermes Agent

The most expensive of the 14 models in the new Hermes Index costs 215 times as much per task as the cheapest, according to the index page on the Nous Research portal, read on the 8th of October. The cheapest task comes to about five cents. The most expensive comes to $11.61 (€10.39).

Nous Research, the company that makes Hermes Agent, announced the index on X on the 6th of October. In the post the company wrote that the index averages several benchmarks of how models perform inside Hermes Agent, that it exists to inform people choosing between models, and that it shows the model makers how their models do inside Hermes. The newsletter AlphaSignal reported the index the same day.

Hermes Agent is the open source agent software from Nous Research, documented on its own site. An agent is a program that takes an AI model and lets it carry out work in steps: read files, run tools, search the web, remember what it learned. The model is the part the user picks, and the index puts a price on that pick.

What Nous Research measured

Each model ran four sets of tasks inside Hermes Agent, the index page says. One set is Hermes Bench, built by Nous Research itself: 150 tasks in 25 categories, covering Hermes skills, research, diagrams and art, memory, tool use and safety. The other three are outside test sets: TerminalBench 4, run without its four tasks that need a graphics card, TerminalBench Science and SkillsBench.

A task is one job handed to the agent. The agent starts in a workspace with real files, the page says, and some tasks come with follow-up turns, the way a person adds a second request after the first answer.

The score is the mean reward across the tasks, so partial credit counts, the page says. In plain words: a task done in full earns full points, a task half done earns half, and the score is the average over every task the model received. The Hermes Index figure is the mean score across the four sets, and the cost per task is the mean amount in US dollars that one task cost in model usage across the same four sets. Makers bill by the token, a piece of a word, so a model that reads and writes more on a task costs more on it.

The 14 models, cheapest first

The index page sorts the table by score. The table below sorts it by cost, cheapest first. Scores and dollar costs are Nous Research's own figures from the index page. Euro figures use the European Central Bank reference rate of the 7th of October, 1 euro = 1.1177 US dollars, rounded to the cent.

Where the costs fall

The chart counts the 14 models from the same page by cost per task. Two of the four in the top group cost more than ten dollars.

The score column and the cost column do not run in the same order. Sorted by price, the page gives one sequence; sorted by score, another. The page attaches no verdict to either column.

What a month would cost

The figures below are arithmetic on Nous Research's per-task costs, for one example: an agent doing 20 such tasks a day for 30 days, which is 600 tasks in the month.

At the cheapest per-task figure in the table, the month comes to $32.40 (€28.99). At the figure in the seventh row of the table, the month comes to $1,692.00 (€1,513.82). At the most expensive figure, the month comes to $6,966.00 (€6,232.44).

The tasks in the index start from a workspace with real files and some carry follow-up turns, the page says, so a user's own tasks may run shorter or longer. The arithmetic shows the spread between models on the same work. A user's own bill follows the user's own tasks.

An agent that remembers needs a machine that stays on

Memory is one of the 25 categories in Hermes Bench, and follow-up turns are part of the test, the index page says. Hermes Agent keeps its conversations, memories and skills from one session to the next, its documentation says, so the person who comes back tomorrow expects the agent to be where they left it, with the files and the memory in place.

OneClickClaw hosts Hermes Agent and OpenClaw on an always-on dedicated server in the EU, at the same price for both, according to its plans page. The customer picks the model on day one and brings a key from that model's maker, so the cost per task in the table above lands on the maker's bill and the server fee stays flat whichever row the customer chooses. The 7-day free trial needs no credit card, and plans start at EUR 14.99 per month (Starter).