We Stopped Using the Mathematics That Works

I found the article confusing. Its premise seems to be that alternative methods to deep learning “work”, and only faded out due to other factors, yet keeps referencing scenarios in which they demonstrably failed to “work”. Such as:

> In 2012, Alex Krizhevsky submitted a deep convolutional neural network to the ImageNet Large Scale Visual Recognition Challenge. It won by 9.8 percentage points over the nearest competitor.

Maybe there’s another definition of “works” that’s implicit and I’m not getting, but I’m struggling to picture a definition relevant to the history-of-deep-learning narrative they are trying to explain.

> This is the VHS-versus-Betamax dynamic, or TCP/IP versus the OSI model, or QWERTY versus every ergonomic alternative proposed since 1936. The technically superior solution loses to the solution that’s easier to deploy, easier to hire for, and good enough for the use cases that pay the bills.

Without commenting on the merit of the claims, the problem with this statement is that in many cases there is no universal "technical superiority", only tradeoffs. E.g. Betamax was technically superior in picture quality while VHS was technically superior in recording time, and more people preferred the latter technical superiority. When people say that the techinically superior approach lost in favour of convenience, what really happened is that their own personal technical preferences were in the minority. More people preferred an alternative that wasn't just "good enough" but technically better, only on a different axis.

Even if we suppose the author is right that his preferred approach yields better outputs, he acknowledges that constructing good inputs is harder. That's not technical superiority; it's a different tradeoff.

> A Bayesian decision-theoretic agent needs explicit utility functions, cost models, prior distributions, and a formal description of the action space. Every assumption must be stated. Every trade-off must be quantified. This is intellectually honest and practically gruelling. Getting the utility function wrong doesn’t just give you a bad answer; it gives you a confidently optimal answer to the wrong question.

I was talking somebody through Bayesian updates the other day. The problem is that if you mess up any part of it, in any way, then the result can be completely garbage. Meanwhile, if you throw some neural network at the problem, it can much better handle noise.

> Deep learning’s convenience advantage is the same phenomenon at larger scale. Why specify a prior when you can train on a million examples? Why model uncertainty when you can just make the network bigger? The answers to these questions are good answers, but they require you to care about things the market doesn’t always reward.

The answer seems simple to me - sometimes getting an answer is not enough, and you need to understand how an answer was reached. In the age of hallucinations, one can appreciate approaches where hallucinations are impossible.

Just because you can analyse it doesn't mean that it is better. Deep learning theory is unbelievably garbage compared to the empirical results.

In particular, please show me a worked example of a decision tree meta learning. Because its trivial to show this for DNNs.

We are at the age of alchemy, wait for the age of chemistry and physics. New mathematical foundations are yet to be found.

>This is the VHS-versus-Betamax dynamic, or TCP/IP versus the OSI model, or QWERTY versus every ergonomic alternative proposed since 1936.

QWERTY has many variants, and every single geopolitical institution have their own odious anti-ergonomic layout, it seems. So this case is somehow different to my mind. As a French native, I use Bépo.

I spent some time in industry working on ML-based credit risk modeling. In my experience, successful shops that have a genuine interest in applying their models to practical decision making with real stakes care deeply about uncertainty quantification and decision theory. Things can get messy very fast though and the challenges faced are often too hyper-specific to one's situation to make sense as part of an academic research program. I think it's been for the best that academic research has tended to focus on the development of general algorithms intended to be broadly useful. Businesses are already well incentivized to take the best of what academia produces and try to get the decision theory right for their particular problems.

I think it's just the time for full acoption of CodeAct in one form or another has not yet arrived. Math should be done using math tools. Just give the tools to model in an easily accessible way to work with data without loss on tokenization and data<->text conversions. That's basically what Anthropic did with their [programmatic tool calling]( https://platform.claude.com/docs/en/agents-and-tools/tool-us... )

LLM-garbage article, ironically.

See also: https://gfrm.in/posts/agentic-ai/

> I’ve spent the last few months building agents that maintain actual beliefs and update them from evidence — first a Bayesian learner that teaches itself which foods are safe, then an evolutionary system that discovers its own cognitive architecture. Looking at what the industry calls “agents” has been clarifying.

> What would it take for an AI system to genuinely deserve the word “agent”?

> At minimum, an agent has beliefs — not hunches, not vibes, but quantifiable representations of what it thinks is true and how certain it is. An agent has goals — not a prompt that says “be helpful,” but an objective function it’s trying to maximise. And an agent decides — not by asking a language model what to do next, but by evaluating its options against its goals in light of its beliefs.

> By this standard, the systems we’re calling “AI agents” are none of these things.

a voice of reason cries out in the howling maelstrom

Heading down the links of this blog ends up at https://github.com/gfrmin/credence, which claims to be an agentic harness that keeps track of usefulness of tools separately and beats LangChain at a benchmark.

LangChain… Now that’s a name I haven’t heard in a long, long time..

Anyway, that’s a cool idea. But also his blog posts include phrases like “That’s not intelligence, it’s just <x> with vibes.” Urg. Slop of the worst sort.

But, like I said, I like the idea of keeping a running tally of what tool uses are useful in which circumstances, and consulting the oracle for recommended uses. I feel slightly icky digging into the code though; there’s a type of (usually brilliant) engineer that assumes when they see success that it’s a) wrong, and b) because everybody’s stupid, and sadly, some of that tone comes through the claude sonnet 4.0 writing used to put this blog together.

Tldr: the author is annoyed at the Bitter Lesson.

Join the crowd dude. It's still true, no matter how inconvenient it is.