Maxwell: A new kind of model that sits between code and hardware

There's a lot of claim and so little technical details on that page.

How was the formalization of Python accomplished? What formal proof framework do you use?

How do you reproduce the optimizations across several instances / runs of the program? (since the source code doesn't change... assuming you don't redo the same work over and over?)

What visibility, tracking and logging is there into the actual code running? Debugging is an extremely important part of building software, especially if there's a demon switching out binary code constantly underneath. Even if the recompiler is perfect (it won't be), user code never is.

Why is the superoptimization and visibility into the data not surfaced back to the level of the source code? This seems like a waste.

Let me say that I like the idea:

- verification

- superoptimization specialized to a particular piece of hardware

- patching binaries while they are running

all great

That being said, I'm getting some weird vibes from the prose on this site. Little phrases and claims that seem intended for the purposes of marketing, but conflate things with clear technical meaning in confusing ways.

For instance: I watched the video ("a code execution model has to be right every time") which implicitly communicates something like "the model cannot hallucinate" ... of course, the agentic LM is still probabilistic, but it is wrapped with verification tools (to reason about semantic equivalence), and is likely post-trained to be good at using those tools.

This strikes me as a conflation between the model and the harness which is intended to strike a marketing point (like: "it doesn't hallucinate") but which is clearly a ridiculous claim to make (falling down the slope: any harness which does error correction for improper tool calls might also be a "model which makes no mistakes")

Also: "self-improving code"

that's not really it, is it? Self-improving code would be code that runs itself and improves itself. This is code I run, which Maxwell watches and improves ... no?

Related: I dislike the usage of the word "model" in the above discussions on the site. Do we want to call everything a "model" now? Can anyone invent a new frontier "model" by post training an agentic LM with tools?

Also, let's say you are optimizing kernels on GPUs. This can be subtly fraught: for instance, it was well-known for awhile that one should only use certain pieces of software (vLLM, for instance) to reproduce papers (which used vLLM) there are non-deterministic sources of randomness (including, of course, floating point) that would prevent one from recovering the performance claimed in a paper ...

  > Self-improving code would be code that runs itself and improves itself.

I read the entire manifesto thinking the discourse about a code execution model would lead to a revelation that this is a model that can reduce bugs in code in realtime and make sure the code runs to the original spec or blueprint. Instead it turned out to be about a model that keeps on optimising code that can go faster and faster over time. That's very exciting in its own right, don't get me wrong, but I was really starting to hope this would be about creating code that breaks less etc.

Maybe it means both but perhaps the emphasis is on speed.

This was a lot more exciting than I expected.

The homepage does not do a very good job of explaining what we're looking at:

It appears to be a combination of two things:

1) A formalization of the Python language

2) An LLM which hooks into running programs and hot-swaps code with verifiably-equivalent behavior but better runtime performance, found via superoptimization and then verified via the formalization rules

This is brilliant, and in hindsight one of those "I can't believe nobody had thought of this yet" sort of things.

While the people behind this project seem to be real and apparently have credentials, this article is giving me serious whatever the textual version of uncanny valley is called. It hits so many of what reads like assorted buzzwords meant for marketing but guised in programmer-speak. One moment, they’re talking about the Curry-Howard correspondence and then next moment they’re describing what a data type is to people who are presumably more than casually familiar with LLMs. And the worst offense imo is linking to that “November Singularity” bs clearly written by AI to sell hype to people without much of a technical background (blog writer even admits some articles are written with AI “assistance”).

Sorry to be cynical, a lot of smart people seem to have built this.. but I'm failing to see how it's any better than "hey claude, here's my program and a profiling tool, go do your stuff"

Heh, such pushback here. The writeup seems to have gotten under people's skin.

It's refreshing to read human prose about something like this. There's so much you can get about the author's state of mind -- pride at what they've accomplished and how they like to conceptualize it, traces of how they learned things and what those things mean to them... for me, it's a reminder of what we've lost in giving over so much of our writing to AI.

But it's understandable that the writeup would trigger defensiveness. It claims a lot of originality and novelty that seem overreaching. AIUI, it's an optimizing JIT with a superoptimizer and a shared cache, with the search procedure guided by AI. And the equivalence model that the superoptimizer is based on includes all of Python and can handle both CPU and GPU code. It's quite impressive, really.

It's hard to tell what of it is new. Superoptimizers are not, JITs are not, caching JIT code is not, using machine learning to speculate or search is not. Constraining an LLM to produce correct output is no newer than forcing semantically valid JSON output. (You could think them as different type systems.) Different JITs have handled different kinds and amounts of runtime information -- tracing JITs, runtime type inference like SpiderMonkey used to use, speculative compilation and invalidation (global or just in time like inline caches)....

But none of that takes away from Maxwell. There isn't really enough information to compare it to other systems. Random example: many JITs work really hard to balance compilation time with execution time, and the post doesn't talk about that. Is it only for long-running code? Does it reach a steady state, and if so how much overhead is there at that time? What does it make slower or bigger?

At any rate, it seems like there's a lot of interesting work in Maxwell, and it's worth checking out.

Forgive the skepticism: Is this a harness that sits beside a language runtime, looks at traces/logs and uses an LLM to optimize code for given hardware?

It looks interesting but I am really unable to see what is so different from a couple days of prompts and digging into a software to optimize it using LLMs.

Cool. superoptimizing-jit ideas are exciting to me. Auto parallelism, swapping faster code/algos based on data distribution, autovectorization etc. Several years ago I had worked on an idiom based optimiser for assembly (https://github.com/xtellect/osm) and I hoped to extend it to binaries like what this team did. Couldn't find time to work on it further, but what this team achieved looks admirable. Congrats

Suppose I have hundreds or thousands of containers from the same python image running in production, and want Maxwell to speed them up.

How many Maxwells do I need? Maxwell isn’t free; when is it worth it vs when are the gains not enough to justify? How well can we estimate that up front?

Does Maxwell only look at one container, or each of them individually? Do they all get the same patches or are they all treated independently?

Does the profiling or patching cause temporary delays? If so how much, how might it compare to a GC pause or similar?

Sounds very interesting, I've occasionally wondered what it would be like to have a profiler "in the loop" which could direct improvements. Not everything can be known at compile-time, and even one run can differ greatly from another (ETL vs normal interactive use, or day-to-day load).

I don't understand how it works. Is it a tracing JIT? https://en.wikipedia.org/wiki/Tracing_just-in-time_compilati...

How is the compute of the model weighed against the compute that it saves?

Just compile for your specific hardware sort of like what Gentoo does. I don't want you continuously running a model - sort of defeats the purpose of performance gains.

I like how the Manifesto is written, the Turing example at the top got me thinking more holistically about different impressive computer science achievements

Taking my deterministic code and injecting non determinism seems like a nightmare to work with

Can I use Maxwell in my js web app? Or just Python stuff?

I love it, any timeline for general availability?

Sounds cool, but isn't it weird that optimizations from my workload are just for me? What if they could help a another user?

this is an israeli company, use at your own risk (to life, the planet and humanity). plus you dont want an israeli company in your stack swapping code with malware and trojans.

[flagged]

Is it 100% deterministic?

[dead]

The manifesto is so cool!

very cool