Closed AI models dominate the price-performance frontier

Closed models might be better generalists because they don't have that much constraint on size and come from well funded labs that can afford to better train for generalization. If a model's actual goal is to be open then smaller is better.

I've had really great results with cheap flash agents using a system that has a parent agent that spawns 2 sub agents. They remain active throughout the whole session and don't lose context.

The 2 sub agents get the exact same prompt and tools as each other and they independently answer but before responding back up to the parent they exchange their findings which must be backed up by a citation and then debate their findings and citations repeatedly until they can agree on correctness of all points. They are also heavily encouraged to never assume and always verify with the prompt and tools and guards where they can be realistically enforced.

It churns through tokens but tokens are crazy cheap with some of the plans out there. Could Opus one shot most of those same queries in like 1/4 time? Yep, but that's not the point for me.

DeepSeek Flash for parent then DeepSeek Flash and GLM 5.3 flash as the debating subagents gives best overall results in my experience.

For context I do equal parts sysadmin/devops and having them write software

Let’s say I want to run qwen 27B or flash next on a machine I own, off solar panels that are also sunk cost.

How did they price the marginal cost of a token? Depreciation on the box + reverse billing opportunity cost with the power company?

I don't think I am going to try to beat them at this game anymore. I've got a machine in my closet that is both collecting dust and accruing value. A very maddening state of affairs.

At this point, trying to beat a major provider is like running your whole house off grid when the alternative is 12c/kWh that is already wired to the premises. You might win if you make it your only job for years on end and have a very specific lifestyle that is tolerant to weird tail risks (assuming you aren't willing to go back to the grid in an emergency).

It is also evidence of commodicisation / brutal competition. I wonder if they cut costs because there is stiff competition from open weight models (and arguably we are in the blitzscaling phase where they try to corner the market, not the enshitification stage).

I'm using Claude all day at work and deepseek privately for very similar tasks. The price/performance difference is extreme.

What's more important is that open models can be inspected, they can be customized, and you don't have to worry that a company will decide that you're not allowed to use them for stuff like reverse engineering things or do security work.

That's why open models, and especially ones you can run locally will always be important. When you're renting closed models, you're basically a serf who has no say in how the model can be used.

For us who spends more than $150K tokens a month with less than $5K spending in subs, it's really hard to justify the switch and it's even senseless (for now).

Open-weights excel primarily at coding.

Let's scale this out 3 years into the future: - Opus 6.7 costs an arm and a leg to run for 2 prompts - Astra 7.89 literally requires you to buy-now-pay-later because it's so expensive - As a bootstrapped dev, your only option is serving local Qwen-4.04-27B that runs on your old RTX (that you bought in 2024)

Sure, it's worse performing. But if you can't afford the best ones in the first place, what's so bad about having a "pretty decent" model that you run essentially free (only paying electricity) if it does what you need it to?

... and if we're all in the permanent underclass anyway, what does it matter that your local model is dumber than the SOTAs? At least you have one! Some people who can't afford GPUs still have to type emails by hand.

(jk, this is actually a cool dataset though ^)

It’s pretty obvious that local AI has a higher TCO if you look only at money. But there are other things to be considered.