We use lua 5.3 on an embedded Platform for scripting and it has been a roller coaster ride. Not a fun one, unfortunately.
You want luasocket? The stable one is not compatible with 5.3.
Packages are sometimes outdated (for years no updates) and there is no replacement.
The lua point releases have breaking changes.
The source code itself is a macro hell which is hard to debug. And the code is not very readable. The documentation lacks for some topics of you use the c api. I tried to implement some scheduling for c and lua threads.
Oh and one thing if you must use Windows... you better quit right away. I had to help my co worker Installing it with luarocks and it is a mess. to be fair it was easy on my Ubuntu machine.
The thing is lua on embedded has no rival. but god did it cost me some nerves.
> The documentation lacks for some topics of you use the c api.
I'm surprised to hear this. I'm doing a lot of Lua and I always thought the documentation is very succinct and complete. I really curious when issues you ran into.
> The thing is lua on embedded has no rival. but god did it cost me some nerves.
My info might be a bit out of date, but something that surprised me when I looked into it was how little control I had over the embedded lua interpreter from my outer app. I basically had to hand it control by calling lua_pcall, and either wait for a result or manage a timeout from another thread.
I expected to be able to do things like tell Lua to run for X milliseconds or Y opcodes or whatever, but it didn't seem that anything like this existed.
So perhaps my disappointment with the documentation was really just disappointment that the implementation wasn't set up to do what I wanted in the way I wanted to do it. But in the course of coming to this realization, I also found the documentation frustrating and opaque.
I'm not sure which embedded platform you're using but give AngelScript [0] a look.
I've been using it to extend some C++ code (bsnes emulator) and it's been fantastic. It's so nice to have a C++-like scripting language with natural script-host bindings that I don't have to think hard about creating or spend a lot of time writing marshalling code or worrying about memory layouts. On top of that nice script-host interface you get an actual usable scripting language without silly things like 1-based array indices.
That said, it does not come with any sort of standard packages like luasocket but there is support for defining your own modules.
Depending on the embedded platform you're using, there may not be support for the native calling convention and you may have to fall back to the generic calling convention, or you could roll your own code to accommodate your platform's calling convention.
I share some of your pain. We use Lua 5.1 on our embedded platform and at one point did development on 5.2.
There's a lot of (seemingly) undocumented changes, even between two minor versions that make you question the semver (and your sanity) at times.
For example, the patterns operate just different enough between the two minors, that I had to resort to dumping everything into JSON and back since that was the only way that one odd case worked similarly between the two versions (albeit sharing the same JSON library).
In regards to the new version, coworker of mine glanced at the 5.4 changelog and could not make out how things had changed, only the fact that they had been changed.
Lua point releases, 5.1, 5.2, etc. are major releases. The numbering scheme isn't semver. It is only accidentally that Lua even has an ecosystem. It is meant entirely for embedding within a larger application and not used as a stand alone application server.
I just had a short glimpse, but it seems that this is rather "transpiling" to C, right? We needed something that is being interpreted to allow easy updates of scripts.
Comment on all subjects (I will just link it to other commenters):
A lot of commenters posted some projects with JS interpreter. But we had to chose something that is stable and is being maintained. The last thing is always hard to argue about because you never know how long maintainers stay commited to the project. But the LUA interpreter is widely used for a lot of (commercial) projects. So there is high chance that it stays for some time. We tried out some other projects, sometimes looking into mailing lists and github issues where we would find open bugs where the interpreter leaks under some circumstances or other problems that you just want to avoid.
LUA seemed a perfect fit and i think still is, but there are still things that didn't work as planned. I really love the concept of the interpreter with the stack. But, again, scheduling is a very complex topic and that is where i found the lua documentation lacking. Especially scheduling between LUA <-> C. I found some projects that solve this issue but then we stumbled upon other issues:
Some projects were for 5.1 and didn't work for us. Others used features we didn't have on embedded (pthreads).
So we decided to do a very basic scheduling but had to understand more what happens under the hood. And that is where the source code is not easily readable. Don't get me wrong i am convinced that the devs are great at their work, but holy moly you can't use more readable name for you variables? Or the use of macros, is it really neccessary to use it for every conversion?
Interesting. Didn't Unicode restrict UTF-8 to allow encoding only 21 bits? Does it mean that it can now do 6 byte UTF-8 encodings? What kind of restrictions did it have before?
Just recall that Lua versioning schema allows for breaking changes to point releases.
This has been both great for Lua as a Lua and one of its biggest challenges. It's allowed the language be constantly refined and tweaked; however, at the expense that it's super common for applications to complete break when upgraded to a point release.
LuaJIT is an entirely different VM. It is binary compatible with Lua 5.1.x modules (shared libraries) and can run most Lua 5.1 and 5.2 code. A good amount of the 5.2 changes, specifically those which do not break 5.1 code, have been added to the VM.
I use FFI and LuaJIT is significantly faster on my embedded platform. Also, you can yield in coroutines in places where you can not with the standard Lua VM, such as across pcall, xpcall, iterators, and in metamethods.
I use Lua via OpenResty / lua-nginx-module and in personal C projects for plugins and scripting. Some of these seem useful, like utf8 support and const variables.
For the multiple user data values feature, I've always found lightuserdata more useful than userdata, because it's not often that you need just one bunch of simple memory that can be freed without any other work. Rather, I almost always have more complex data, such as things that need manual cleanup like sockets, handles, etc. Or, the structure has pointers to other things that must also be cleaned up. What I'd like instead is the ability to pass a destructor callback to `lua_pushlightuserdata` so that it get's called when the pointer falls off the stack.
I don't use Lua threads or coroutines currently, so I don't have much to add there. What I do wish is that there were a way to clone or pass objects from one lua state to another to support parallelism. Basically, I'm thinking of it like the work Eric Snow is doing on subinterpreters in Python. Passing values (by copying, no shared memory) between lua instances.
One use case I had is loading a plugin that you then want to run multiple instances of in parallel with different arguments. Ideally, you wouldn't have to load the plugin multiple times (therefore calling the module level code multiple times). So I'd like to copy the entire lua state and then run each one with different arguments. There's no userdata, coroutines, or lua threads, so I don't have to worry about things that aren't possible to copy.
I got as far as looking into how to copy functions and then got busy with other things and stopped. Is there anyone else out there trying to introspect Lua function structures and copy all the opcodes, upvalues, etc?
I think that's what Lua Lanes if for. Haven't used it myself, but the descriptions sounds exactly like what you're looking for: http://lualanes.github.io/lanes/
Wow, thanks for sharing! Not sure why I didn't find this when I was looking. It does look like a very good match to my use case. I'll have to try it out.
It’s a simple, but powerful language that uses basically two features (functions and hashmaps) to implement everything. Ex: script files are implicitly functions. Global variables are in a hashmap. Semantically it is very much JavaScript without all the surprises. Semi-technical users (artists and designers) can figure it out and go on to make surprisingly powerful features.
The binary code is small and fast. The standard library is small and most of it can be removed easily. Parsing source to bytecode is quick. Loading bytecode is very fast.
It is easy to embed. It’s easy to contain it’s memory allocations in an arena. It’s easy to restrict what it can and cannot do. It’s easy to interface with C/C++. It’s extremely portable.
Your last line there is really one of the big keys. No other language is as easy to embed and interface with C/C++. The only real exceptions are some varieties of Lisp, but the "easily embeddable" ones tend not to be as mature, performant and robust as Lua.
I'm no particular fan of the language, but they do the "embed and interface with C/C++" better than anything else I've seen.
I use it as a configuration and data language, mostly for building APIs for other libraries (like SDL2).
The syntax is simple and clean, like JSON but without the most commonly cited warts (no requirement to quote keys, and comments are allowed.) The VM is lightweight enough that including it in an application is practically inconsequential. I like its modular syntax as well. There are also bindings for just about every language out there, making Lua code more easily portable and embeddable.
It's not without its flaws as a language but to me, it's better suited as a config language than TOML, YAML or XML for most use cases where you don't actually need their complexity, or JSON if you're not on the web.
For KVdb [1], my managed key-value store database, I opted to integrate Lua as a scripting language instead of JavaScript because it felt super lightweight and the syntax is easy to learn in a few hours for even newbies. I'll probably end up adding JavaScript, too, since most developers know it, but there is a simplicity about Lua that I haven't seen in many other somewhat popular languages.
I've built a collection of Android musical instruments [0] and Lua was big win for the project. I was previously familiar with Lua and Love2D framework and I wanted to see how far I can push it. The language is expressive and lightning fast. Very few building blocks that fit together nicely. I dig tables as first class objects, and meta-tables to turn them into complex objects or call delegation mechanism.
My favorite feature is that it's interpreted. The Android app is just a runtime. I can edit source files right on the phone and re-execute. It's awesome prototyping platform.
BNF is amazingly short and therefore language is easy to learn [1]
VM is tiny and can communicate bi-directionally with C / C++ code very easyly, makes it the ideal candidate for embedding in a larger app as a scripting language.
I use it as the scripting, and implementation, language for a console-based mail-client. (Think "mutt", but with real scripting instead of the ad-hoc support that mutt has.)
I use it because it's a simple language that is quick to code in, and trivial to embed inside a C, or C++, host application.
If I were to begin the project again I'd have no qualms about a similar approach. (Writing the core in C++ with a notion of display-modes, email-parsing, etc, and the actual control in Lua. The biggest painpoints were dealing with MIME, and similar broken emails which was largely irrelevant for the lua-side.)
(If you haven't heard of that, then I implore you, for the love of God, don't look it up and try it out and get addicted, whatever you do! And if you do, then don't blame me for getting you hooked. Oh dammit, now I made myself want to play it again!)
I use it as a scripting language for my game engine, but also for all the tooling around it. Premake for generating the VS solution/makefiles, but also my own tools using IUPLua [1] and a few other modules (lfs, sqlite etc). Pretty much my whole ecosystem is C++/Lua.
I used it because that's what I needed for configuring Hammerspoon. Been rather pleased with the language overall, but don't have any other uses for it. Hammerspoon has been killer, especially once I figured out some simple metaprogramming.
I embed it as a simple interactive query language for a time system at the local computer club. I wanted to use guile, but my fellow computer clubbers lose all their intelligence when they see parentheses.
Pretty much only for modding software/games that provide a lua API. If I were to create a software like that I would definitely consider implementing a scripting interface with LUA.
Most of it I'm pretty okay with. The one big pain point is lambda definitions: you have to do `function(x) print(x) end` each time, instead of `\x. print(x)` or something.
It sounds small, but with callback heavy or embedded DSL code it very quickly adds up.
I love lua syntax, its the only language I feel I actually dont have to look up syntax when I return to it a year later. Its just stupid simple, and that has a big value for me. Maybe Java < 8 also .. also pretty simple but very verbose :)
Readit News
Oh and one thing if you must use Windows... you better quit right away. I had to help my co worker Installing it with luarocks and it is a mess. to be fair it was easy on my Ubuntu machine.
The thing is lua on embedded has no rival. but god did it cost me some nerves.
I'm surprised to hear this. I'm doing a lot of Lua and I always thought the documentation is very succinct and complete. I really curious when issues you ran into.
> The thing is lua on embedded has no rival. but god did it cost me some nerves.
You might take a look at https://bellard.org/quickjs/ and https://duktape.org/. The latter seems oddly familiar if you've worked with the Lua C API.
I expected to be able to do things like tell Lua to run for X milliseconds or Y opcodes or whatever, but it didn't seem that anything like this existed.
So perhaps my disappointment with the documentation was really just disappointment that the implementation wasn't set up to do what I wanted in the way I wanted to do it. But in the course of coming to this realization, I also found the documentation frustrating and opaque.
I've been using it to extend some C++ code (bsnes emulator) and it's been fantastic. It's so nice to have a C++-like scripting language with natural script-host bindings that I don't have to think hard about creating or spend a lot of time writing marshalling code or worrying about memory layouts. On top of that nice script-host interface you get an actual usable scripting language without silly things like 1-based array indices.
That said, it does not come with any sort of standard packages like luasocket but there is support for defining your own modules.
Depending on the embedded platform you're using, there may not be support for the native calling convention and you may have to fall back to the generic calling convention, or you could roll your own code to accommodate your platform's calling convention.
[0] https://www.angelcode.com/angelscript/
There's a lot of (seemingly) undocumented changes, even between two minor versions that make you question the semver (and your sanity) at times.
For example, the patterns operate just different enough between the two minors, that I had to resort to dumping everything into JSON and back since that was the only way that one odd case worked similarly between the two versions (albeit sharing the same JSON library).
In regards to the new version, coworker of mine glanced at the 5.4 changelog and could not make out how things had changed, only the fact that they had been changed.
What about something like Nim which compiles to C?
https://nim-lang.org/
Comment on all subjects (I will just link it to other commenters): A lot of commenters posted some projects with JS interpreter. But we had to chose something that is stable and is being maintained. The last thing is always hard to argue about because you never know how long maintainers stay commited to the project. But the LUA interpreter is widely used for a lot of (commercial) projects. So there is high chance that it stays for some time. We tried out some other projects, sometimes looking into mailing lists and github issues where we would find open bugs where the interpreter leaks under some circumstances or other problems that you just want to avoid.
LUA seemed a perfect fit and i think still is, but there are still things that didn't work as planned. I really love the concept of the interpreter with the stack. But, again, scheduling is a very complex topic and that is where i found the lua documentation lacking. Especially scheduling between LUA <-> C. I found some projects that solve this issue but then we stumbled upon other issues: Some projects were for 5.1 and didn't work for us. Others used features we didn't have on embedded (pthreads). So we decided to do a very basic scheduling but had to understand more what happens under the hood. And that is where the source code is not easily readable. Don't get me wrong i am convinced that the devs are great at their work, but holy moly you can't use more readable name for you variables? Or the use of macros, is it really neccessary to use it for every conversion?
- new generational mode for garbage collection
- to-be-closed variables
- const variables
- userdata can have multiple user values
- new implementation for math.random
- warning system
- debug information about function arguments and returns
- new semantics for the integer 'for' loop
- optional 'init' argument to 'string.gmatch'
- new functions 'lua_resetthread' and 'coroutine.close'
- coersions string-to-number moved to the string library
- allocation function allowed to fail when shrinking a memory block
- new format '%p' in 'string.format'
- utf8 library accepts codepoints up to 2^31
Interesting. Didn't Unicode restrict UTF-8 to allow encoding only 21 bits? Does it mean that it can now do 6 byte UTF-8 encodings? What kind of restrictions did it have before?
- https://www.lua.org/manual/5.3/manual.html#6.5
- https://www.lua.org/work/doc/manual.html#6.5
Does this mean + can finally be used for string concatenation and .. disabled as an opt-in C flag?
This has been both great for Lua as a Lua and one of its biggest challenges. It's allowed the language be constantly refined and tweaked; however, at the expense that it's super common for applications to complete break when upgraded to a point release.
https://www.lua.org/versions.html
https://www.freelists.org/post/luajit/Port-bitop-to-53,1
It is fantastic software.
https://luajit.org/performance_x86.html
For the multiple user data values feature, I've always found lightuserdata more useful than userdata, because it's not often that you need just one bunch of simple memory that can be freed without any other work. Rather, I almost always have more complex data, such as things that need manual cleanup like sockets, handles, etc. Or, the structure has pointers to other things that must also be cleaned up. What I'd like instead is the ability to pass a destructor callback to `lua_pushlightuserdata` so that it get's called when the pointer falls off the stack.
I don't use Lua threads or coroutines currently, so I don't have much to add there. What I do wish is that there were a way to clone or pass objects from one lua state to another to support parallelism. Basically, I'm thinking of it like the work Eric Snow is doing on subinterpreters in Python. Passing values (by copying, no shared memory) between lua instances.
One use case I had is loading a plugin that you then want to run multiple instances of in parallel with different arguments. Ideally, you wouldn't have to load the plugin multiple times (therefore calling the module level code multiple times). So I'd like to copy the entire lua state and then run each one with different arguments. There's no userdata, coroutines, or lua threads, so I don't have to worry about things that aren't possible to copy.
I got as far as looking into how to copy functions and then got busy with other things and stopped. Is there anyone else out there trying to introspect Lua function structures and copy all the opcodes, upvalues, etc?
It’s a simple, but powerful language that uses basically two features (functions and hashmaps) to implement everything. Ex: script files are implicitly functions. Global variables are in a hashmap. Semantically it is very much JavaScript without all the surprises. Semi-technical users (artists and designers) can figure it out and go on to make surprisingly powerful features.
The binary code is small and fast. The standard library is small and most of it can be removed easily. Parsing source to bytecode is quick. Loading bytecode is very fast.
It is easy to embed. It’s easy to contain it’s memory allocations in an arena. It’s easy to restrict what it can and cannot do. It’s easy to interface with C/C++. It’s extremely portable.
I'm no particular fan of the language, but they do the "embed and interface with C/C++" better than anything else I've seen.
The syntax is simple and clean, like JSON but without the most commonly cited warts (no requirement to quote keys, and comments are allowed.) The VM is lightweight enough that including it in an application is practically inconsequential. I like its modular syntax as well. There are also bindings for just about every language out there, making Lua code more easily portable and embeddable.
It's not without its flaws as a language but to me, it's better suited as a config language than TOML, YAML or XML for most use cases where you don't actually need their complexity, or JSON if you're not on the web.
* Easy embedding. Compared to Python and ruby which I looked at back then, Lua is so much simpler.
* Fairly easy to learn for users. After all, the complete syntax is one page long: https://www.lua.org/manual/5.1/manual.html#8
* Quite expressive with coroutines, tail calls, metatables, ...
* Trivially allows multiple interpreter instances within the same program.
* More than fast enough, especially once LuaJIT was released.
[1] https://kvdb.io/docs/scripting
My favorite feature is that it's interpreted. The Android app is just a runtime. I can edit source files right on the phone and re-execute. It's awesome prototyping platform.
[0] https://github.com/jmiskovic/hexpress
VM is tiny and can communicate bi-directionally with C / C++ code very easyly, makes it the ideal candidate for embedding in a larger app as a scripting language.
[1]http://lua4z.com/doc/manual/syntax.md.html
I use it because it's a simple language that is quick to code in, and trivial to embed inside a C, or C++, host application.
If I were to begin the project again I'd have no qualms about a similar approach. (Writing the core in C++ with a notion of display-modes, email-parsing, etc, and the actual control in Lua. The biggest painpoints were dealing with MIME, and similar broken emails which was largely irrelevant for the lua-side.)
Currently using it to create a tiny Postscript interpreter: https://github.com/wiladams/lj2ps
(If you haven't heard of that, then I implore you, for the love of God, don't look it up and try it out and get addicted, whatever you do! And if you do, then don't blame me for getting you hooked. Oh dammit, now I made myself want to play it again!)
[1] - https://www.tecgraf.puc-rio.br/iup/en/iuplua.html
It sounds small, but with callback heavy or embedded DSL code it very quickly adds up.