We used Luerl, the underlying Lua engine, in production for years as a sandboxed scripting environment for FarmBot devices back in the day. Users could execute scripts directly on the device to control peripherals. It was a solid library and the community support was great. From an ergonomics perspective, developers preferred this approach over API calls. I am surprised more projects don’t provide a Lua scripting layer.
> I am surprised more projects don’t provide a Lua scripting layer.
Completely agree. I've been adding Lua scripting support to pretty much everything I make now. Most recently my programmable SSE server [1]. It's extended the functionality far beyond anything that I would have had the time and patience to do myself. Lua is such a treat.
I've been using Lua professionally and personally for some decades now, and I think that there is no need to be surprised that more projects don't provide a Lua scripting layer, because actually, Lua is everywhere. And the scripting aspect is one thing - but its not the only reason to use Lua.
Lua is so easily embeddable and functionally productive as a development tool, its not always necessarily necessary to have this layer exposed to the user.
Sure, engines and things provide scripting, and that is a need/want of the respective markets. Who doesn't love redis, et al.?
But Lua can be put in places where the scripting aspect is merely a development method, and in the end shipped product, things are tightly bound to bytecode as an application framework beyond scripting.
The point being, Lua is such a treat that it is, literally, in places you might least expect - or at least, will find under a lot of covers.
And after all, thats the beauty of it: as an embedded language/VM/function interface, Lua can sure scale to different territories.
This is not embedding the C Lua runtime and compiler, but rather a
complete implementation of Lua 5.3. This feat is made possible by the
underlying Luerl library, which implements a Lua parser, compiler,
and runtime, all in Erlang.
Okay, that's actually pretty cool (:
Also a sign of Lua's maturity and popularity, that it's got various independent implementations (LuaJIT, this one, perhaps others I don't know about).
This is so cool. A key benefit is that it's not embedding the C Lua runtime and compiler, but rather implements Lua in the host language (Elixir/Erlang).
When sandboxing user code in another runtime, you need to serialize the data to and from that runtime. That comes with a performance penalty.
So, for example, if you sandbox code in WASM, you need to pick a transport data format, like JSON. You need to serialize Elixir data structures into JSON, send it to WASM, and then deserialize the result. For a high-performance data pipeline, this adds up!
But if your sandbox is in the host language, no serialization/de-serialization is required. You can execute the sandboxed language in microseconds.
Very nice, but it is confusing to name a library for using a language the same thing as that language. I suppose this is meant to be a temporary state of affairs while Lua (the library) gets merged into Luerl, but I would have personally called it Luerl++ or such.
More seriously, I considered alternate names, but settled on this because it was short, literal, and given that its in the context of Elixir, makes sense when using it.
As you stated, the hope is to consolidate it into Luerl at some point
I am trying to understand why would anyone prefer to use Lua to create script instead of Elixir, which supports running scripts. While Lua has lots of users the language just have too many wrong design choices. If I had the choice between Elixir and Lua for scripts I would use Elixir every time.
Elixir is not a sandboxed language, so you can't just accept arbitrary Elixir programs from users and execute them inside of your application without security concerns. Lua, on the other hand, can be executed in a sandboxed fashion, limiting and constraining the reach of user programs.
I would also personally prefer Elixir to Lua, but this would be great for allowing isolated/sandboxed BEAM processes to execute Lua code from your end users whom are likely unfamiliar with Elixir. As mentioned in the article, this is precisely what TV Labs uses it for to allow smart tv app developers to write and run tests for their apps.
Lua can help if you're handing this over to someone else not just devs who know Elixir.
Also, as the sibling post mentioned, in this case Lua is completely interpreted in an Erlang process. That allows a good amount of sandboxing, they have isolated heaps already and you can control memory limits and other resource usages, and all that in a relatively tiny footprint of a few KBs.
This is more for your apps users. Like lets say you have a CRM saas written in Elixir. Then you can allow your users to script things in your app with Lua. If you allow them to use Elixir running in your app you might allow someone to hack your app.
I've been considering adding Lua support to Keila for a while already. It seems like a perfect fit for when I'm going to implement email automation and want to allow custom business logic. This would certainly make that plan easier.
Thanks OP for sharing the library with the community!
Readit News
Completely agree. I've been adding Lua scripting support to pretty much everything I make now. Most recently my programmable SSE server [1]. It's extended the functionality far beyond anything that I would have had the time and patience to do myself. Lua is such a treat.
[1] https://github.com/benwilber/tinysse
It sure is.
I've been using Lua professionally and personally for some decades now, and I think that there is no need to be surprised that more projects don't provide a Lua scripting layer, because actually, Lua is everywhere. And the scripting aspect is one thing - but its not the only reason to use Lua.
Lua is so easily embeddable and functionally productive as a development tool, its not always necessarily necessary to have this layer exposed to the user.
Sure, engines and things provide scripting, and that is a need/want of the respective markets. Who doesn't love redis, et al.?
But Lua can be put in places where the scripting aspect is merely a development method, and in the end shipped product, things are tightly bound to bytecode as an application framework beyond scripting.
The point being, Lua is such a treat that it is, literally, in places you might least expect - or at least, will find under a lot of covers.
And after all, thats the beauty of it: as an embedded language/VM/function interface, Lua can sure scale to different territories.
Also a sign of Lua's maturity and popularity, that it's got various independent implementations (LuaJIT, this one, perhaps others I don't know about).
When sandboxing user code in another runtime, you need to serialize the data to and from that runtime. That comes with a performance penalty.
So, for example, if you sandbox code in WASM, you need to pick a transport data format, like JSON. You need to serialize Elixir data structures into JSON, send it to WASM, and then deserialize the result. For a high-performance data pipeline, this adds up!
But if your sandbox is in the host language, no serialization/de-serialization is required. You can execute the sandboxed language in microseconds.
I wrote more about this here: https://blog.sequinstream.com/why-we-built-mini-elixir/
Wish this library existed just a couple months ago!
If you wait long enough before replacing whatever2 with something that would be whatever3, you can go back to whatever (see pg -> pg2 -> pg)
Luerl++ is not a valid module name :)
More seriously, I considered alternate names, but settled on this because it was short, literal, and given that its in the context of Elixir, makes sense when using it.
As you stated, the hope is to consolidate it into Luerl at some point
https://github.com/rvirding/ship-demo
Some instructions to install it on macOS: https://erlangforums.com/t/lt-using-luerl-to-run-spaceships-...
I am trying to understand why would anyone prefer to use Lua to create script instead of Elixir, which supports running scripts. While Lua has lots of users the language just have too many wrong design choices. If I had the choice between Elixir and Lua for scripts I would use Elixir every time.
Check out Anthony Accomazzo's post about Mini-Elixir, which does a great job breaking this down much further https://blog.sequinstream.com/why-we-built-mini-elixir/
https://docs.tvlabs.ai/scripting/introduction/getting-starte...
Also, as the sibling post mentioned, in this case Lua is completely interpreted in an Erlang process. That allows a good amount of sandboxing, they have isolated heaps already and you can control memory limits and other resource usages, and all that in a relatively tiny footprint of a few KBs.
Deleted Comment
https://hexdocs.pm/lua/Lua.html