PyPy v8.0.0 Release

Posted by lumpa 1 day ago

Counter63Comment15OpenOriginal

Comments

Comment by gomoboo 1 day ago

It always bugs me to think about the Python community not rallying around PyPy. Faster Python is right there and has been there for years. Sure there can be C-extension issues but projects like HPy showed you can remedy that. Why doesn’t everyone just use and support PyPy?

Comment by famouswaffles 1 day ago

"C-extension issues" are not something you can just gloss over. They're a huge chunk of python usage. Users won't rally around that. And Hpy is dead. If CPython embraced PuPy, worked with it to smooth those issues then maybe things would be different.

Comment by pjmlp 1 day ago

To the point that in Python culture those C extensions are seen as Python libraries, not bindings.

Comment by theandrewbailey 23 hours ago

Maybe PyPy needs a Steve Jobs-like figure to write a "no C extension" manifesto, declaring that all Python apps must be 100% Python.

Comment by bastawhiz 23 hours ago

Is there a single example of a language+runtime where this approach has ever been successful

Comment by zbentley 37 minutes ago

I do think Go and Java are strong examples of this. Those languages have native code support, but it’s clunky to use and, more importantly, very rare.

If you removed JNA/JNI/CGo, plenty of people would complain…but the vast majority of uses of those languages would still work, unaltered, because most common tasks on the JVM and Go runtime don’t require external native deps.

Comment by pjmlp 22 hours ago

Well, see popularity of CGO among Go devs, or JNI among Java devs, or the ongoing work to make all the learnings from System C# (from Midori) available in regular C#, while C++/CLI is slowly left in maintenance mode (last updated for partial C++20 support, and not part of FOSS .NET)

Comment by bastawhiz 16 hours ago

All of those are for non native code to call code in those languages. Nobody is saying "all Go code must never invoke native code". That would be wild and to my knowledge, there's no major language that forces such a constraint.

Comment by pjmlp 9 hours ago

All the languages I mentioned compile to native code.

Comment by bastawhiz 2 hours ago

I can't tell if you're being willfully ignorant or not. The comment I was replying to was proposing banning C extensions: invoking native binaries that are not written in the host language and compiled in (at build or runtime). What you're describing has nothing to do with that.

Comment by pjmlp 2 hours ago

Why would a Go binary (to keep the examples short), compiled to native code, with a compiler toolchain that includes support for Assembly if necessary, and a syscall package as well, have to depend on C extensions?

CGO only exists as a matter of convenience.

Comment by bastawhiz 23 hours ago

PyPy makes tradeoffs that might not be acceptable. For anything that isn't a long running server, PyPy can be much, much slower since the JIT doesn't have a chance to warm up. PyPy also implements things differently: repeated string concatenation in a loop is quadratic instead of linear like in CPython. There's also memory tradeoffs, depending on what you're doing.

The list of differences is quite long, and even though most of them are edge cases, it's not hard to get bitten by one on a project of any substantial size:

https://doc.pypy.org/cpython_differences.html

It's not that people are writing weird code, it's that you have a package that does something clever but safe, and then another package using it that calls it in a way that's unexpected but not documented as unsafe, and layers and layers of that compound until you have things like exceptions raised because non-string keys are getting set on type objects.

At the end of the day, Python isn't a terribly well-specified language. CPython behavior is functionally the specification. "Make it match CPython but faster" isn't really an option because it kills your ability to do a lot of the things that make PyPy fast. And if that was all it took, we wouldn't still be arguing over a JIT inside CPython. Pyston was an honest attempt at doing that, but it has been dead for years partially because it hit many of these walls.

Comment by vova_hn2 20 hours ago

> Pyston was an honest attempt at doing that

I'm still kinda sad that Pyston didn't happen

> but it has been dead for years partially because it hit many of these walls

Maybe this is one of the reasons. But I've also heard that Dropbox (which was making Pyston [0]), that used to heavily rely on Python, just decided that gradual migration to Go [1] is a more feasible approach than creating a faster Python implementation.

[0] https://dropbox.tech/infrastructure/introducing-pyston-an-up...

[1] "About a year ago, we decided to migrate our performance-critical backends from Python to Go to leverage better concurrency support and faster execution speed." - https://dropbox.tech/infrastructure/open-sourcing-our-go-lib...

Comment by vova_hn2 20 hours ago

A lot of Python code is just a relatively thin layer of "glue" between C libraries.

An ML pipeline will probably rely on numpy to do all the heavy lifting.

A web backend will probably use something like psycopg, which is a wrapper around libpq (official postgres client lib, written in C).

etc

Comment by duttonw 1 day ago

like bun?