Ten advances in mathematics and theoretical computer science

What’s new about LLMs is that you can scalably manipulate people individually. It used to be that you could either have scale (speeches, tweets, interviews, website, etc.) or individual engagement (replying to mail/tweets/town hall questions.)

Now you can pull the history and preferences of an individual, then shape a message—in real time—to them, specifically. You can have conversations on social media with a single person and shape your message specifically to them.

Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.)

My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections.

I wonder if AI will end up being similar. Certain theorems get proven by AI but others do not. We haven't reached the limits of this yet and I haven't found a good argument for where those limits will be (I do doubt that there are no limits).

The results OpenAI demonstrated are impressive, but it also looks like they threw a lot of compute at it just to get results. How many tokens did they waste on problems they couldn't solve? Applying inference infrastructure on a large number of math problems at scale we haven't seen before to me doesn't demonstrate an exponential curve in model abilities.

In general, you can think of the process as generating massive rollouts in generation N, and then compiling in the verifier/human feedback("gradient") signal into generation N+1. The time taken to make the rollout in generation N, and separately the time taken to get the same rollout in generation N+1, each grows constant in some tasks, linear in more, and exponential in some.

In the end, this becomes bottlenecked by time. Today, we can make statements like "I generated all these successful trajectories with 2 weeks of compute, in the next model it will be able to do it in 7 hours of compute", but very soon you'll find yourself making statements like "I generated.... with 8 months of compute, in the next model it can do it in 6 months", which isn't really enticing the same way you can _technically_ brute force passwords but it just needs prohibitive amounts of time and money. That is the "plateau". Note that, this point is quite far away. For example, at any point if we agree it plateaus, today's known hardware techniques such as fixed function accelerators give you a 10-100x timeline reduction immediately allowing for a few more cycles of improvement. This is not to mention future innovations, but of course none of that is helping with the benchmarks where the time needed is growing superlinearly.

In many math and coding benchmarks, we are still in the constant phase. These are the massive improvements we see every few months. I'm not making any prediction of what will plateau and what will not as it's not possible to make an informed prediction about these things IMO. But the observed fact is that some have already plateaud as in, they don't improve with reasonable inference time (likely superlinear growth).

> will we need mathematicians to translate

Let's take a sudoku analogy. The model is initially just doing the random value algorithm, but lets say you the human are watching it. You make one of the usual reductions and interject "hey you can stop trying 8 here because of ....". Over enough examples, you get to a point where the model is _forced_ to learn the logical pattern. Next generation, it will skip that number. After this, you can peak the distribution using simple 1/0 RL. Doing _pure_ 1/0 RL works decent, but its not frontier as its a very sparse signal.

For that lift, human (or even a better LLM, but if you're trying to improve a frontier LLM, there is by definition no better LLM) feedback becomes necessary. This is _why_ it is crucial that these models interface in natural language and is also why the labs are hiring AI tutors by the hundreds. The "better LLM" case is what Kimi etc are doing by "distilling"(bad term for this) claude.

> But the long term is completely bewildering if you believe any of these trends can continue at a similar pace for the next few years.

For math and coding, for now we are in the phase where the times are just ... constant, so there's little reason to think it will stop soon. We still need humans to expand the frontier. It just becomes a matter of if its worth the cost of compute for running this generalized The Algorithm or not.

Given how well chess players internalized _many_ (not all) of alphazero's emergent chess knowledge, I am confident we wont have too much trouble figuring out any new math LLMs come up with, which will let us keep expanding the frontier by giving the LLM the next "lift". Only when we reach the stage where the time growth become exponential will this stop, IMO.

Is there even the tiniest reason to suspect that the people steering this progress will use it for the democratic good of all?

isn't it clearly split between verifiable not verifiable ? what is interesting about that question.

what did you notice ?

Yes.

>I don’t understand it yet. Maybe it’ll take me an afternoon to check all the calculations, but what would still be missing is why this was an approach that would’ve made sense in the first place. Is there some broader context or theory within which this would’ve been the obvious thing to do? What other results can be proven using these techniques? What is it telling us about quantum information or operator theory? I have no idea. I spent about an hour this morning asking ChatGPT these questions, but it’s somewhat frustrating because it speaks with a mishmash of physicist, operator algebraist, quantum information theorist-lingo, plus the usual LLM breezy lilt that annoys everybody.

They certainly seem to have "intuited", in a way that is not immediately obvious to experts in the field, the way to solve at least some of these problems. This was not just simply grinding away at a method that humans already knew would work and just hadn't gotten to yet.

[0] https://nitter.poast.org/henryquantum/status/208362369543662...

That's how they are finding these solutions though, unless we are just going to label intuition as something only humans can do. Like a submarine being unable to swim or whatever that example is.

I disagree. I routinely let LLMs speculate or generate hypotheses along the way of helping with technical research. Sometimes they can prove the correctness of a concrete math idea but other times even an unproven conjecture helps with the numerical algorithm implementation and the result is then simply supported by additional data. I guess that any autoresearch-adjacent application has LLMs intuiting and coming up with hypotheses/conjectures—as do the steps/lemmas along a complex proof. In my opinion the modern LLMs are powerful intuitive thinkers that generate lots of conjectures of varying quality or importance.

Of course computers can grind in a way that humans can't. But now we have systems that convert the human-comprehensible ideas into a computer's plan of attack, in a way that greatly expands the frontier of ideas thus treatable.

People keep saying this. Why?

Surely the AI can complete the prompt “Generate new research questions based on these observations”?

When I read the reasoning traces of coding models they are constantly asking themselves questions and attempting to answer them.

"Excuse me, We demand rigidly defined areas of doubt and uncertainty!"

DT: Might I make an observation at this point?

MT: You keep out of this metal nose.

VF: We demand that that machine not be allowed to think about this problem!

DT: If I might make an observation…

MT: We’ll go on strike!

VF: That’s right. You’ll have a national philosopher’s strike on your hands.

DT: Who will that inconvenience?

MT: Never you mind who it’ll inconvenience you box of black legging binary bits! It’ll hurt, buster! It’ll hurt!

DT: [Booming] If I might make an observation …

“All I wanted to say,” bellowed the computer, “is that my circuits are now irrevocably committed to calculating the answer to the Ultimate Question of Life, the Universe, and Everything.” He paused and satisfied himself that he now had everyone’s attention, before continuing more quietly. “But the program will take me a little while to run.”

Fook glanced impatiently at his watch.

“How long?” he said.

“Seven and a half million years,” said Deep Thought.

Lunkwill and Fook blinked at each other.

“Seven and a half million years!” they cried in chorus.

“Yes,” declaimed Deep Thought, “I said I’d have to think about it, didn’t I? And it occurs to me that running a program like this is bound to create an enormous amount of popular publicity for the whole are of philosophy in general. Everyone’s going to have their own theories about what answer I’m eventually going to come up with, and who better, to capitalize on that media market than you yourselves? So long as you can keep disagreeing with each other violently enough and maligning each other in the popular press, and so long as you have clever agents, you can keep yourselves on the gravy train for life. How does that sound?”

The two philosophers gaped at him.

“Bloody hell,” said Majikthise, “now that is what I call thinking. Here, Vroomfondel, why do we never think of things like that?”

“Dunno,” said Vroomfondel in an awed whisper; “think our brains must be too highly trained, Majikthise.”

So saying, they turned on their heels and walked out of the door and into a life-style beyond their wildest dreams.”

Maybe you’re the one who needs breaking out of your cached beliefs.

There is irony here

not sure how many will get this reference but "AI" for science and math is like super-shoes for runners

at first we are blown away by the impossible improvements including sub-2-hour realworld marathon and every other PR/CR/WR is dialed down

but then the improvements slow and reach a stall point because of the limit of technology and the source of the achievement

ie. sub-2-hour marathon yes, sub-1-hour never happening (rollerblade inline-skate record is 1-hour marathon)

https://garymarcus.substack.com/p/two-critical-updates-re-as...

As always, PR hype. Goalposts have not moved.

Guys, please use critical thinking. The haters don't hate by default, we hate because we're gaslit about this stuff every day and it's annoying. Extraordinary claims require proof, and they're not giving us information that would be essential to knowing if this is actually significant or not.

I heard that Gary Kasparov was impacted by AI chess, but at least he still seems to have a job, so don't give up.

The sofic groups question was the outstanding question about sofic groups. Almost everyone thought that non-sofic groups existed, and there were plausible candidates, but proving a group was non-sofic was out of reach. Now that we know how to do it once, we can probably do it a lot more.

The Connes rigidity conjecture I think people thought was false, but it was a provocative claim to make. The significance of conjectures is frequently not that the answer to the question is "yes", but that we don't know how to answer the question. And now, apparently, we do.

Your worry.... is because they used the word advanced? For marketing? The word is used very appropriately here. There were PhD's who spent a big part of their career tackling these problems.

Mundane incremental research is cobbled from existing citations that already appear nearby in the record.

Basically, innovative research is a measure of bridging thought and domains that were previously not bridged. It's quite concrete as a measure in the citation record.

So we can know pretty conclusively.

Puja Ohlhaver gave a talk on this[1], and ran some experiments (that I had the pleasure to support on)

[1]: https://www.youtube.com/watch?v=guLDNMAOn24

Also on HN front page today: AI's debt binge can't last, hidden borrowing reaches $1.65T (fortune.com)

https://news.ycombinator.com/item?id=49160699

I just want to state that having "lean proofs" that build (checks) does not mean the actual real theorems we care about hold. Ignoring lean kernel bugs, ultimately a human (not an agent) has to verify the lean encoded theorem statements (specs/specifications) that the lean proofs are checked against. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal. AI generated lean encoded theorems can be huge and difficult to understand. I wonder if anyone reputable has audited these specifications.

You seem to overlook a simpler barrier. To make these advances, they have to be possible. A 15% improvement in GPU kernels doesn't evidence that significantly more improvement has been left on the table.

Incredible?

> open ai announced like 15% improvement by fixing gpu kernel issue

That is... ordinary software optimization.

> we are seeing frontier level math breakthroughs (ie performance that would put it in the top 100 or 1000 mathematicians in the world if it were a human, meaning top .00001% or 800/8B)

i think you have misunderstanding of what mathematicians do

OpenAI and Anthropic are too greedy for cash to do anything of the sort.

I don't expect this current economic cycle to bring anything else that will directly greatly improve the life of the average person on the planet, more than it hurts it.

- travel and restaurant recommendations. my last few outings have been entirely LLM-advised and they turned out excellent. LLMs seem to have ingested every single Google review, photo, and menu of every business on Earth and can answer very nuanced questions like "is the garlic chicken at <restaurant, city> garnished with coriander?"

- fitness, nutrition, accounting, therapy, medical, legal, immigration advice (sure it's not a real professional but you know what, it's pretty fucking close, and any capability gap is made up by having perfect two-way communication which you don't get when talking with a human)

- coding (work, side projects, personal tools, documentation & pricing questions, "review this code", etc).

- I start reading most articles with the prompt "Summarize this article: <url>". I just started a non-fiction book by pasting into Claude: "There are 12 chapters in the book <book-name>. Can you give me a 2 sentence synopsis of each chapter?". It reduces the "activation energy" hump and screens if it's worth reading at all.

- I use the LLM in my Tesla for on-the-fly advice for parking and other things. You can simply ask "what's the best Boba place around here?" and it will give you a decent recommendation. You can also follow up with "does this place have ample parking?".

- I use the LLM in YouTube to summarize videos and ask specific questions and/or get timestamps to the parts I care about.

If your critical thinking skills are strong then LLM is a literal superpower.

So I would like to counter your cynicism with a “YMMV” depending on who you work for.

https://x.com/polynoamial/status/2083478171975082334

As a complete guess, it seems like they tested hundreds to thousands of problems with a relatively low per-problem budget

--

The linked tweet from Noam Brown at OpenAI reads:

> And yes we did try other major problems without success. Sadly no Millennium Prize problems (yet).

> But also, we didn’t spend a lot on each problem. It’s possible to push test-time compute much further.

It's not just about requiring to disclose AI use. AI-powered mathematics is a completely valid discipline that doesn't need to be shy, but it should develop its own publication culture.

Even if the cost was $1 mil for these 10 problems, that's maybe 10-20 math researchers for a year.

Do you really think that if you paid that to humans, they will deliver the same results?

I don't think that comparison to p-hacking is fair. I mean not reporting price of all run is nothing like committing scientific fraud and fake results.

Another question I have is whether or not OpenAI 'simply' hired capable combinatorics researchers to work on problems, and they have, and the use of the model is incidental / secondary to their work.

My negative feelings towards AI are about energy use and inequalities, that kind of stuff. It undeniably works well, but whether or not it is better for society or the planet is a lot less clear.

If you translate that to math, then all you get is math competitions, not math as a career. Of course the translation isn't nearly exact ... there's a lot more room for professional mathematicians because the math space is far more vast than the chess space and can't generally be cranked out mechanically (we have proof).

P.S. The response is nonsense ... I explained exactly why it's awful (others have too) and the response doesn't in any way refute the explanation ... rather it offers up a ridiculous strawman.

Thinking of this a little bit with the perspective of every new proof as a burden, dumped for review by actual mathematicians.

The most disappointed folks will simply drop out, but the enthusiastic ones will keep going and with luck make up for the ones who decided to quit. Chess and go certainly went this way.

I'd honestly rather they just automate every job at that point.

I just want to state that having "lean proofs" that build (checks) does not mean the actual real theorems we care about hold. Ignoring lean kernel bugs, ultimately a human (not an agent) has to verify the lean encoded theorem statements (specs/specifications), that the lean proofs are checked against, indeed correctly encode the real theorems. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal. AI generated lean encoded theorems can be huge and difficult to understand. I wonder if anyone reputable has audited these specifications.

Training the model is going to be amortized over other uses.

https://x.com/polynoamial/status/2083470822258467194

Edit: Oh, are you suggesting they just use it to privately improve their models? I imagine a few more correct proofs would have a very marginal benefit, if any. Also, they'll probably just get extracted, meaning it still gets out but OpenAI doesn't get to fancily announce it themselves.

[1] https://x.com/henryquantum/status/2083623695436623915?s=20

If anything it proves more data centres are needed. That's literally the only reasonable conclusion from this news.

Mathematicians will tear it to pieces if any of it is fake!

This was not a problem that was for sale

I’m not sure it will be FOOM, maybe it will require AIs to iterate on hardware to get orders of magnitude more compute/storage/energy which would more likely require months/years, but algorithmic progress would likely saturate quickly. I guess it depends on how much you think further AI progress depends on hardware vs. software.

What I had hoped for was something more ambitious: using AI as an arbiter in economic, political, and social debates, one capable of weighing evidence, exposing trade-offs, and helping us make decisions that produce better outcomes over the medium and long term, even when those decisions conflict with powerful private interests.

I suspect, however, that this is not a particularly urgent goal for the people funding and directing these systems, many of whom live far removed from scarcity and its consequences.

We already know how to solve all of these issues. What we lack is collective political will.

This is division of labor, and it's a good thing. I'm sure OpenAI employees, who are very well paid, donate some money from their salaries to others working on the areas you're citing: probably more than you think, I say that from having attended some EA parties back in the day.

But that isn't my point: my point is that a company which makes brushless motors should put most of its time and money into solving the "make and sell brushless motors" problem, and if they or their investors feel like they need to do more for the world, give money to the people who have the time and ability to do things about that. There are a lot of quality-of-life improvements which need brushless motors.

Next question is how useful their product (OpenAI, I mean) is to more focused do-good-in-the-world professionals. I'm sure that varies quite a bit. For getting the homeless off the street? I conjecture, not very useful. For 'complete the transition off fossil fuels'? Extremely useful, no one in that field knows how to do their job without AI in summer 2026. I'm certain of this.

Just maths isn't really general enough for the G in AGI.

Is there good historical data on some measure of strength across representative populations over time in the modern era? I'm doubtful.

We do know that the introduction of agriculture diminished strength:

"Bone mass was around 20% higher in the foragers - the equivalent to what an average person would lose after three months of weightlessness in space.

After ruling out diet differences and changes in body size as possible causes, researchers have concluded that reductions in physical activity are the root cause of degradation in human bone strength across millennia."

cam.ac.uk/research/news/hunter-gatherer-past-shows-our-fragile-bones-result-from-physical-inactivity-since-invention-of

None of this is anything out of the ordinary; this kind of thing has always happened. The only real story here is that moderators sleep sometimes.

I for one am really fascinated by AI's advances in science and math and would like to talk about it somewhere without the constant flamewars...

For people who have been paying attention to accurate predictions leading to our present state of the world, HN has collectively been reactionary, incorrectly dismissive, and incredibly behind the curve. In public, people are better informed on LessWrong and AI Twitter circles. That's where frontier researchers are. Barely here

I think we've now hit a point where 99.9% of the population gloss over these types of AI advancements because of human competence being insufficient

No human could have published this because it requires paradigm shifts (e. g. Section 5) in multiple mathematical domains. Mastering one of them to this degree is rare, mastering 3+ pretty much non existent for humans.

A slightly smarter highschooler could write these. I could write these. It's clear as day that the LLM, not the human, did the heavy lift. It'd be ridiculous to give full credit to whoever wrote the prompt.

[0]: Not one of the proofs in the linked article, but from OpenAI too.

When the tool is a 3D printer, or any CNC system really, you bet I attribute a build to it.

I could also attribute the operator; there is no contradiction, it's a free choice, just like saying "I am in Berlin" does not contradict "I am in Germany".

What is your mechanistic model of self awareness that yields this conclusion?

> It's a tool

Does your model suggest that tools can't have self awareness?

> Astra, a new model that OpenAI is testing internally, is amazing. No denying that.

To sharpen that, I think he's (obviously) interested in maintaining his own brand as "thought leader" and this necessitates de rigeur defense of particular postures.

Sometimes this is easy because the facts warrant it; other times, a bit of rhetorical license is required to preserve nominal coherence and (at least, for the moment) hold certain lines.

This is one of the latter cases, and it's not subtle.

One of the celebrated properties of many intellectual advances or inventions in whatever domain is precisely that it appears obvious in hindsight. It is quite cynical to leverage consensus distrust of large AI players, warranted but also a popular social construction, to insinuate that these are not "real" advances or "real" hard problems, on the grounds they were in some sense cherry-picked.

Identifying the problems amenable to strategies on the table and intuitions (sic) about where bridges might be, is exactly the discerning work that is the core driver of almost all prior progress, but for celebrated accidents and flashes of insight. Anyone working in any challenging discipline knows that those are celebrated and told around campfires precisely because meaningful durable results arising like that is so uncommon.

These two articles make me think of nothing so much as my own durable reaction to the creeping goalposts of AI critics generally: that they often seem to me not unlike a water color cohort scoffing and jeering at the horse, because it got a D on its tensor calculus exam.

Marcus should be on guard against his own cynicism and take care that his assumptions do not prevent clear sight.

I think that is an oxymoron.

eg. "we spent another 2k on searching for pre-existing proof but found only the weaker result xyz by abc in 1972" would be in the spirit of academics quoting prior work.