Trials and Tribulations of Native Lua Compilation

Posted by _samt_ 3 days ago

Counter29Comment13OpenOriginal

Comments

Comment by Philpax 17 hours ago

This is of interest to me, but I have no idea why you'd set `height: 720px`. I had to turn that off to make it readable.

Comment by _samt_ 8 hours ago

Thanks for pointing that out, it might be diffu ule to read on some résolutions

Comment by 13 hours ago

Comment by drdexebtjl 13 hours ago

> It also means that libraries written in C or C++ don't need to be translated into some special Lua-specific implementation. They can remain native libraries, with a thin clx binding layer exposing the functionality to Lua.

This is already true of upstream Lua implementations. It is very easy to write thin Lua binding layers to C libraries.

That is the entire point of Lua. It’s easy to call Lua from native code, and it’s easy to call native code back from Lua, so you can have call stacks that seamlessly interweave Lua and native code.

I’m very skeptical of the idea that this can unlock improvements in performance or size. We already have LuaJIT which is plenty fast, and PUC-Lua which is so small, it runs on MCUs.

Comment by _samt_ 8 hours ago

The primary goal of clx is not size or performance but to ship standalone executables without the need of the Lua VM with a native approach rather than the well known (and somewhat hacky) way to embed a script and the LuaVM inside an executable (and when using binary modules it can be quite tricky !)

Comment by drdexebtjl 6 hours ago

There’s nothing hacky about it, it is in fact the way the language was designed to be used, hence why the interpreter ships as a library, and why the module loader looks for package.preload[modname] before reaching for the file system (and can be trivially made to not touch files at all).

Comment by _samt_ 4 hours ago

[dead]

Comment by 5 hours ago

Comment by gucci-on-fleek 4 hours ago

I wouldn't consider static linking + #embed to be hacky though, and that's all that it takes to deploy a standalone Lua program.

Comment by _samt_ 4 hours ago

It will be hacky if you need binary modules. Some platforms (Windows I see you !) don't permit to link binary modules to a static executable. Your only possibility is to embed the Lua dynamic library and the binary modules your need, extract them somewhere and so on...

Comment by gucci-on-fleek 4 hours ago

> Some platforms (Windows I see you !) don't permit to link binary modules to a static executable

If you have the source code for these modules, you can statically link to them too. I've done this myself for Windows a few times, and it works pretty well in my experience.

Comment by _samt_ 4 hours ago

[dead]

Comment by wiradikusuma 17 hours ago

If it has to go through compilation process to make it native, doesn't that defeat the purpose of Lua? I mean, why not just use the same language of the host application, in other words, just another function/module of that app.

Comment by PhilipRoman 16 hours ago

1) You can use interpreted version for rapid development and switch to compiled for release

2) You may like Lua as a language and want to use it for general purpose programming, not necessarily embedding into host program

I will say though that the performance aspect has to be taken with a grain of salt, "native compilation" is not a meaningful improvement if every source line translates to multiple branches or vtable lookups. From my experience plain conversion of Lua to C (assuming you faithfully follow the specification, including coroutines, metatables, etc.) produces equal or slower code with rare exceptions for numerical loops. If it wasn't all AI generated I'd maybe spend time understanding this project.

Comment by le-mark 11 hours ago

> From my experience plain conversion of Lua to C (assuming you faithfully follow the specification, including coroutines, metatables, etc.) produces equal or slower code with rare exceptions for numerical loops.

This has been a long known reality of compiling dynamically typed and functional languages to native code.

Comment by _samt_ 8 hours ago

Totally agree, and that's why clx static analysis and optimizer steps permits some performance gains

Comment by _samt_ 8 hours ago

[dead]

Comment by _samt_ 3 days ago

A technical write‑up about the difficulties of compiling Lua to native code, based on my experience developing clx