Why Erdős Problems Are Falling to AI

Take something like Fermat's last theorem - I'd be curious to hear of any use of the result itself, but there was a massive amount of new mathematics generated by those working on it, whether ultimately successful or not.

These AI math proofs are interesting testament to the power of reinforcement learning applied to math, obviously reflecting the axiomatic self-consistent nature of math itself, but it doesn't seem they have the same value as a humans working on these problems since they are using known math to solve them rather than inventing anything new.

However, it would still be interesting to analyze the LLM lines of reasoning that lead to any of these results, since there may be value there even if no new math, just as human Go players have found value in analyzing computer Go.

Still, as Demis Hassabis has himself said, the real goal with AI is discovery and creativity - you want to create the thing that could design the game of Go in the first place, not just play it. Similarly with math, while there is interest in seeing an AI "play math" using the rules of the game, what would be of much more interest is the AI that can create new math, in the same way as Andrew Wiles did while proving Fermat's last theorem.

Math is an incredibly broad field. I mean, you don't expect a traffic engineer to understand anything about nuclear reactors, do you? Yet, they are all 'career engineers'.

Because it's math, it's all mysterious and genuinely impressive, but in the end, if no human cares about it (apart from attention grabbing "it's so over" tweets and articles), does it really matter?

Since at least 2014, which was my first brush with the phenomenon when someone published a 13GB proof [0].

The consensus is that such a proof is potentially illuminating, though further work is likely required. If for instance, conjecture A is true if and only if conjectures B & C are true, and B is proven false through one of these such proofs, then we can see that A is also false given that we accept the disproof of B.

Though, the sense is that further work is likely required because it is easy to see that further work along the same direction, or in directions depending on the proof will be hard or impossible if there are not enough humans or agents that are capable of understanding and utilizing the proof. Making it 'more elegant' will increase it's utility despite not proving anything new.

This is adjacent to all of the work done to create multiple proofs using different techniques. Having the same information (that X is so) in different languages (algebraic, geometric, via harmonic analysis, etc) allows for researchers not familiar with the original technique to participate in further research.

[0] https://www.newscientist.com/article/1997488-wikipedia-size-...

Making things understandable is mathematics, and more generally a kind of intelligence, and is crucial to continued progress. You couldn't use algebraic geometry to disprove a conjecture if people hadn't organized (what could have been just) a pile of random observations into something called algebraic geometry.

Historically LLMs have done best where it's possible to train using an objectively verifiable reward function. Computer programs are pretty good on this front and so are Lean proofs. (Of course, they don't only do things you can RLVR heavily, but those have progressed fastest.) Not sure where 'making mathematical knowledge more understandable' falls on that spectrum.

Understandability isn't only important for advanced math. Keeping computer programs from becoming a mess is a challenge in high-level organization too, and the chat with the user is an explanation task. If you look online at what people say about large LLM-built codebases (SlopCodeBench is a neat effort to make make it concrete, but common wisdom seems to mostly agree on the general problem) and chatbot prose, I don't think everyone considers those solved problems!

It's hard to tell how thoroughly the labs grasp and care about this at an organization-wide level. I'm sure at least some maybe-results exist inside labs but haven't been published because the humans couldn't verify them and didn't want to be embarrassed with a false result. (Maybe also why counterexamples are a lot of the first results published: often simple to verify, even if hard to obtain.) A good sign would be if results in a few months come out more like what mathematicians consider well-written papers explaining results in a more intuitive way, fewer shocking announcements of bare counterexamples in tweets. It's probably a slow climb to get there.

The way the chess world adapted this was initially to try to understand the machine. After all chess, like math, is complete information - so you can easily see the computers 'thoughts' in terms of the exact moves its saying are best in a variation and how it might respond to any other idea. But it quickly became clear that this wasn't working so well.

Players would regularly get positions that the computer says 'and black wins' and then proceed to lose it convincingly, simply because the positions were so extremely weird and difficult to play that even if it might be technically winning, it's the sort of position where you're walking a fine line with lots of complex moves to find. Humans aren't computers and even the best of us can't play like one in weird positions.

Now a days they're taken more in balance. The computer's evaluation of a position is probably about as good as you can get, but playability matters much more in practical terms. Knowing the eval of a position doesn't really matter if you don't understand the position. Knowing the answer can help with understanding (for instance computers have radically reshaped and improved human understanding of space in chess as we noticed computers obsessing over it) but I think the days of 'oh the computer says it's winning, so I should be able to take it from here' are near to gone.

Example: https://nitter.poast.org/henryquantum/status/208362369543662...

Seems like a disservice to the community that openai put so little effort into producing good writeups...

In all serious "I don't understand any of this it's way over my head."

Huh?

Mathematics is an extremely wide subject, it's perfectly normal even for two professional mathematicians not to understand each other's work. Have you considered that maybe they just don't work in that area?

Are you implying OpenAI's paper (which was, by the way, edited by humans and provided Lean certificates for most of the proofs) is actually gibberish? That's flat-earth levels of conspiracy.

A lot of people were super hyped about OpenAI's 10 discoveries, but I still don't understand what they mean, and even if I did, what are the implications.

Like, what are non-sofic groups, and what follows from the conclusion that they exist?

I mean in the sense that quantum mechanics might make my head spin, but it's because of that that we have stuff like semiconductors, which have been one of the most significant discoveries.

The Fourier transform is one of the reasons we have fast telecommunications and radars.

What practical things are possible or might be possible due to these results?

Isn't that what people from more practical sciences said about math anyways?

All math is eventually applied math.

An unsettling number of people do seem to be more than ready for this. In fact I'd say they seem almost gleeful about it. Thinking is hard and they don't seem to like doing it!

This is not new nor unique. There is plenty of research, especially in math, which can really only be understood by a few people in the entire world. It is not uncommon for a proof to be presented by a mathematician which, initially, is only understood to that mathematician, and it can take a long time for even another mathematician who is an expert in the same field to be able to confidentially say they understood it.

>I saw lots of career research mathematicians say things mostly along the lines of “I don’t understand any of this it’s way over my head”.

This is meaningless in a vacuum. If you give a novel proof in some niche subfield of topology to a competent mathematics researcher who focuses in number theory, they'd say the same thing regardless of if a human or machine wrote the proof. A bunch of "career mathematicians" on Twitter proclaiming this doesn't mean anything other than these people aren't currently equipped to understand the contents of the proofs. That's fine and normal, but the idea that anybody with a PhD in Math should be able to pick up one of these proofs and give it a skim and be able to say, "ahh, yes, quite clever, it seems so obvious in retrospect," is absurd. That's not how this kind of research works.

>Are we ready to just let go of our intellectual faculties and give them to a giant supercomputer nobody understands?

Nobody is blindly accepting these proofs as valid. ChatGPT isn't spitting out a wall of text and proclaiming that they've solved a previously unsolved math problem while everyone is saying, "well if an LLM says it, it must be true!" lol

These proofs are being checked by automated systems (which have been in-use well before LLMs have existed) as well as being checked over by actual experts who are actually capable of (and motivated to) verifying these proofs. But that work still isn't done. There's enough evidence that these companies are confident in saying these proofs are correct, but there's going to be a lot of ongoing work from people to continue to verify and, more importantly, understand these proofs. It's literally some of these people's full-time jobs to do this.

>How do we tell truth from fiction?

When was the last time you verified even a classical, relatively simple mathematical assertion? How often are you just relying on a larger system of experts to ensure that we're not just blindly accepting fiction as truth?

That's not to try to stick it to you personally, but it's just highlight that there's an entire system in-place here that you're not aware of and that you don't have an understanding of that is working just fine including in this context. Real mathematicians aren't going to lazily start letting OpenAI assert whatever they want about their products solving these kinds of problems without heavy scrutiny.

What are the odds that this random FOSS project has solved the problem of building software and nobody noticed. Close to 0. Don't confuse accidental complexity for transcendental depth.

Theoretical science isn't about usefulness per se, but knowledge and understanding for its own sake, so a better way to say this is to point out that you only benefit in such cases if you understand things yourself. If understanding is the goal - which it is in the case of true theory - then the only way to attain that goal is to actually attain understanding. What good is it if an LLM produces a valid proof, but no one grasps it? This isn't like digging a ditch where it doesn't matter who does it or how he does it as long as you have a ditch. Here, the ditch is knowledge, as it were.

I mean, arguably the development of LLMs matches that description.

I can't tell if xkcd #435 is still true, or if math is just as mushy as everything else seems to be. When a math proof can only be understood by a handful of people, what does that mean about that proof? I think the LLMs are pushing a problem that existed already and pushing it further.

[0] https://xkcd.com/435/

The problem is that Paul Erdos was eccentric but he was also Paul Erdos. I'm not saying you're implying that, but I feel it's a similar line of thinking to how popular culture often romanticizes autism and Asperger's because some very smart people are (allegedly) affected. The same group includes people who need 24/7 care.

Computer science at the frontier is as specialized and hard as mathematics, you need years of study to truly understand a field well enough to make meaningful contributions. You can try sponsoring me if you really want, but I don't think you'd be spending your money wisely in expectation.

He died aged 83...

Look at all of the crazy stuff made by unemployed trans people/autists over the past decade, disproportionately from countries with strong welfare states.

There may be some problems of type type "why does X happen?" that appear answerable in terms of known science, but even these would need verification. If you want to make advances in fundamental physics, then a promising AI-generated theory might take a decade and billions of dollars to prove or disprove.

Math is a rather unique field in being entirely theoretical, axiomatic and self-referential. It is basically the best possible case not just for AI to advance without needing experimental verification, but also specifically for today's AI technology of auto-regressive LLMs and RL training, whereby valid reasoning steps learnt in one context will also be valid in another context (i.e. there is some generalizability of learnt reasoning) as long as you have learnt the pertinent aspects of that context that the validity depends on.

But yeah, most problems in physics, chemistry or biology require labs on top of actual hard thinking. You need to be able to design experiments in a certain way. Once you have the funding, the right tools, the right people to use those tools, then you can use LLMs to increase the speed of the calculations and so on.

There have been other discoveries though by deepmind: https://deepmind.google/blog/millions-of-new-materials-disco...

The only two major highlights are weather modeling and folded protein backbone prediction.

Mostly everything else, either lacks enough data, or there are contraits on the size of the foundational models that render them impractical or they just fail to generalize.

Further down the scale are ones that are decidable only by machines that we would never have the wherewithal to construct, even though they could physically be constructed with the material we have to work with.

I mean, we could. But it doesn't really make sense to think that AIs are perfect at finding counterexamples, just because they are good (or even better than us). My general opinion is that they are orthagonally intelligent, that is, they are intelligent in an entirely different way to the way that people are. They are undoubtably clever, but the distance between when they are better than us at their best skill (or even most skills) and when they are better than us in all aspects is going to be MASSIVE.

but even that, if done correctly, is quite impressive. It sure sounds very useful given the number of papers out there.

It would be tempting to assume the commenter here has no idea what Renaissance is or how they made so much money.

Except it's not? It's owned by a private foundation where the only link is both were founded by the same dude who hasn't run either in like 15 years.

That seems like a pretty big thing to just gloss over like it's just set-dressing.