You know Opus and Gemini. We tested the models that do nearly the same work at a quarter of the cost.

Run 100 marketing tasks a day on Claude Opus 5 and the model bill comes to $286. Run the same 100 on Kimi K3 and it comes to $68.

a day in model costs for 100 marketing tasks on Claude Opus 5

a day for the same 100 tasks on Kimi K3

Those numbers are findings from Mark-Bench, our marketing benchmark. We built it at Optimizely because nothing like it existed. It's a set of real marketing tasks, and each one comes with a brief, the source files that go with it, and a checklist of criteria the output has to clear.

The strict score is all-pass: a task only counts if the model clears every item on the checklist.

On that score, Opus 5 passed 58.2% of tasks. Kimi K3 passed 55.8%. That's a 2.4-point gap against a price about four times higher ($2.86 a task versus $0.68). Scores get the headlines. The bill rarely does.

Mark runs on closed-weight models today. We're soon introducing open-weight models into Mark, so we ran them through the bench in our marketing lab first. Here's how it shook out.

The models you already know: closed-weight

Today, Mark runs on frontier models (the most capable ones commercially available) from Google, Anthropic, and OpenAI (via Microsoft). The household names. Opus, Sonnet, Gemini, GPT.

They share a trait that rarely comes up in marketing circles. They're all closed-weight. A model's weights are what it learned in training, and the company that built it keeps them private. You can use the model, but only through that company's service. Think of this as a restaurant with a secret recipe. You can order anything on the menu, but you never see the kitchen.

What are open-weight models?

Then there's a whole other group: open-weight models. Their builders publish the weights, so anyone can download the model and run it. Think of this as the chef handing you the finished dish to take home. You can run it in your own kitchen, add your own seasoning (fine-tune it), and, if you host it yourself, your data never leaves the house.

But you still don't get the recipe, so you can't see exactly what went in. (That's different from fully open-source, where the chef also hands over the recipe: the training data and code behind the dish.) You may know the names. GLM. Kimi. DeepSeek.

We took them to the lab and ran them through the same 285 scenarios as the closed models, all on our AI harness, the layer of instructions, context, and tools that sits around a model.

One note before the numbers: we're soon introducing open-weight models into Mark. This is what we learned by testing them first.

What the benchmark says

Score here is the all-pass rate: clear every criterion or the task fails. That's strict on purpose. Cost is what the model costs to run one task.

Clear every criterion or the task fails. Source: Optimizely, Mark-Bench (early version, 285 scenarios)

What the model costs to run one task. DeepSeek V4.1 Flash is approximate (about $0.20). Source: Optimizely, Mark-Bench (early version, 285 scenarios)

Kimi K3 and GLM-5.3 land within 3 points of Opus 5 at roughly a quarter of the price. Against Gemini 4 Argon, which tops the table, they cost about half. Argon does score higher, by 5.6 points over Kimi K3.

The budget end gets more interesting. DeepSeek V4.1 Flash cleared 52.6% for about 20 cents a task. Kimi K3 on a lighter setting scored 54.0% at 37 cents. That's 1.4 more points for nearly double the cost.

Then there's GLM-5.3 Flash: 36.5% at under two cents a task. That's about 63% of what Opus 5 clears, for under 1% of the price.

What 100 runs a day does to a budget

Per-task prices look tiny until marketing work repeats. Back to that $286 versus $68: the difference is $218 a day, or nearly $80,000 a year if the work runs every day.

If you're running it 100 times a day, you're paying a few hundred dollars extra daily. Then take it to a month, and take it to a year. These things pile up.Imran Yousuf|VP of Software Engineering, Optimizely

If you're running it 100 times a day, you're paying a few hundred dollars extra daily. Then take it to a month, and take it to a year. These things pile up.

Imran Yousuf|VP of Software Engineering, Optimizely

We're not the only ones seeing this. OpenRouter, a platform developers use to reach many models through one API, recently posted which models people use most for security audits, ranked by share of spend. GLM 5.3 led at 27.8%, followed by Kimi K3 (10.7%) and DeepSeek V4.1 Flash (10.5%), all three from our results above. The top closed-weight model, Grok 4.7, came fifth at 3.6%.

That's a narrow slice (security audits are 0.9% of all spend) and a developer-heavy one. It says nothing about how well these models handle marketing work, which is what the bench is for.

Right model, right job

Not every task needs the heaviest model.

Say you want to summarize an email. A cent and a half might be more than sufficient. You do not have to use a model that costs $2.86.Imran Yousuf|VP of Software Engineering, Optimizely

Say you want to summarize an email. A cent and a half might be more than sufficient. You do not have to use a model that costs $2.86.

Imran Yousuf|VP of Software Engineering, Optimizely

For complex, multi-step work, Opus 5, Kimi K3, GLM-5.3, and Gemini 4 Argon sit in the same neighborhood on score. Which one you pick comes down to price and how often the task runs.

The catch is knowing what each model can do.

If you do not know how capable a model is, you are not able to use it effectively in the right place for the right kind of job.Imran Yousuf|VP of Software Engineering, Optimizely

If you do not know how capable a model is, you are not able to use it effectively in the right place for the right kind of job.

Imran Yousuf|VP of Software Engineering, Optimizely

That's where marketing-specific testing earns its keep. A general leaderboard can't tell you how a model handles a campaign brief with a brand guide attached. Mark-Bench can, because every scenario is marketing work and the rubrics are informed by real marketers. We test models beyond the ones Mark runs on today, too, because the results help anyone choosing one.

Keep in mind: this is an early version

Treat gaps of a few points as directional.

These results come from an early version of Mark-Bench. Versions 1.5 and 2 are on the way, so expect the numbers to move. The Opus 5 run is the one we showed at Opticon. Opus 5.5 and GPT 6.1 haven't been run yet.

It gives you a relative picture of the models, but it probably doesn't represent reality.Imran Yousuf|VP of Software Engineering, Optimizely

It gives you a relative picture of the models, but it probably doesn't represent reality.

Imran Yousuf|VP of Software Engineering, Optimizely

Where this goes next

The data exists so you don't have to do the sums. Marketers shouldn't have to be token economists, so here's the plan:

Control over which models are switched on in your instance.

Recommendations while you build your own agents. Based on Mark-Bench, a model might be a good fit for the job you've designed, and the platform will suggest you test it.

Where these models run, and what happens to your data when they do (who's running the kitchen, in our metaphor), deserves a post of its own. We'll write it. And if your head is spinning about closed weights, open weights, and open source, we'll write that one too.

Bigger models earn their spot on the hardest work. A lot of everyday marketing tasks can run on something far cheaper, and now we have numbers to prove it.