I wrote a trading system where the strategies are written in Lua. It has been a delight, fast and simple. The traders have the expressivity of a full programming language and I can add any function to that language and provide any data as well so that their programming (and execution) sandbox is extremely ergonomic and suited to their specific task. In other words, a trading DSL.
Other code outside the sandbox pulls in up to date price data and if certain safety rules are violated will automatically close out positions. So even if the traders code their way into an infinite loop or make other mistakes, the supervisor can step in and automatically prevent disaster.
Using Lua to make a language for others has been a wonderful experience. FYI, it was approx 11K lines of Lua).
Man, I tried at Bloomberg to get my managers to let me incorporate lua into Tradebook's trading system. They just simply couldn't get it. I'd try to explain and they would look at me like I was from mars or something.
Eventually they got tired of me pitching for it and fired me. How did you ever get managerial buy-in for something like this?
I tried a similar approach with a team I was working with. We were building it on top of redis and after some basic benchmarks gave up the idea and figured out from the docs that that eval script is blocking.
I've embedded Lua in my game engine and I'd say that
- performance is a joke
- GC can cause really bad hick-ups
- loose runtime typing makes it hard to build static tooling for developer support
- lack of predetermined more rigid structure makes some things confusing (like you can mess with the internals such as the meta tables and stuff, which can be useful but also confusing)
+ functionality wise it's adequate for a game programmer, perfect for writing (smallish) scripts of game play
+ it's simple to integrate and provide bindings for
+ lots of flexibility how you use it and what kind of code you use to write with it
In my experience I'd say that if you're planning to integrate and use Lua in your projects you need to
- have a fallback for performance sensitive code, either by being able to mix and match native and Lua user (game) code or provide functionality natively in your engine OOTB
- make sure you can run things properly (using real threads not joke threads) in parallel as much as possible
- stay on top of perf at all times and integrate performance tests and bench marks to your development process
Despite the problems I've managed to build a Lua code editor that has
a) auto formatting
b) syntax highlighting
c) auto completion (limited but still)
d) simple navigation support
e) built-in help
The editor is integrated in my game engine's editor.
Good question. But essentially because things started out small and then grew from there.
Going forward being able to have seamless integration between the scripting and rest of the editor is a key differentiator and value added. For example being able to jump to the right API documentation relevant for the function call under the caret, or being able to open and jump to the game asset from the script.
I'm not sure how well a LSP type of integration would work here. As far as I can tell it'd need to evaluate the script in order to have best possible diagnostics and analysis and that won't work without the interpreter having the native game engine code available as well.
Of course my "solution" was mostly about slapping components other people built together.
- Qt already has a text editor component with components for doing stuff such as syntax highlight quite easily.
- I use tree sitter to extract symbol information from the Lua code.
- I use an open source code formatter someone else built.
I read that as "green threads". But it could be anything just short of a thread maybe even futures and promises.
Just about any thread that isn't a full-on proper thread is not ideal for general purpose game development. There are a lot of thread implementations for various user interfaces that have a goal of minimizing latency or maximizing CPU utilization. But if you are in a latency sensitive environment like a game and you're highly likely to be pushing the CPU to 100% you often need real threads not something like them.
Coroutines have their place in game development too, but they're not (in my experience) used as a thread replacement. They are used to manage complex flow control. Similarly, I have seen green threads and event loops used to manage the user interface or Futures and Promises to manage certain IO in a game but not the rest of the general game work scheduling.
I also need elaboration on this. I presumed calling lua from c can be used to implement user-defined callbacks. Launching a thread for lua + locking seems even slower?
It isn't terribly hard to write the bridge code to lua, just monotonous to translate inputs and outputs from C++.
You throw a lua function into a table (usually tables are used as namespaces and globals are just a table) and then make that C++ translate to your normal one and handle the return value correctly.
The hard thing is building a system to pass in a C++ object in general as that required building a table for each instance that matches its class.
I've had some success by tuning the GC with `collectgarbage('setpause', 100)` and then manually calling ` collectgarbage('step')`every once in a while.
That being said this problem isn't unique to Lua, There are horror stories about Unity and their integration of C# for example.
I believe Roblox's Luau would be suitable for this case? It's got pretty good performance and typing support. Still supports metatables I think (they needed backwards compatibility with old Lua 5.1 code), but it's a start ¯\_(ツ)_/¯
I'm wrapping up a multi-year personal project, a game written fully in lua using love2d.
To me, the beauty of lua is the simplicity and lack of learning curve: I can usually accomplish whatever I need to without looking anything up (as the author said, everything is a table so there isn't much to overthink). Also, the community and support around love2d is fantastic.
One thing that's bothered me is that lua silently returns nil when you reference non-existing elements. That's been a pain when debugging, since a line with a typo (`a = typo`) doesn't fail, and the code fails much farther downstream (e.g. when doing arithmetic on `a`). So almost all my errors end up being "trying to do operation on a nil value", and there is no indication of why it's nil.
I think you can add a metatable to _G with an __index function. This should be called when accessing undefined local variables (as they end up trying to access the global scope _G) and you can thrown an error.
You can. In our game engine ~10 years ago we would hook the global table to stop designers from creating globals (the default in Lua without using the local keyword) at certain areas in the game frame, mostly to stop this exact trap.
I toyed with the idea of inverting the semantics of global and local in lua and to remove "local" and instead default to local and have a "global" keyword. Looking at lua.c quickly dissuaded me when I was a much more junior programmer, but now days it might be fun to try.
My biggest gripe with lua was that depending upon the internals of the implementation, it could "swallow" an error entirely. The program would just die in absolute silence and not give an error at all or any indication it was still running.
> One thing that's bothered me is that lua silently returns nil when you reference non-existing elements. That's been a pain when debugging, since a line with a typo (`a = typo`) doesn't fail, and the code fails much farther downstream (e.g. when doing arithmetic on `a`). So almost all my errors end up being "trying to do operation on a nil value", and there is no indication of why it's nil.
I am also making small games with love2d. I've found you can prevent many of such issues if you:
1. Create objects with private fields, using getters and setters to access values (as function calls will crash if you call them and the functions don't exist, unlike fields). I like an approach that is somewhat similar to this: https://www.lua.org/pil/16.4.html
2. Add assertions liberally, especially in constructors.
Use LuaCheck if you don't want to use the LSP. It warns about the use of globals, which is generally what you want. It's perfectly possible to design your program to use no globals.
If you use the lua lsp, you can make type annotations which basically work like jsdoc.
With those annotations, the lsp will warn you about such issues, there is a diagnostic that's called something like `needs-nil-check`.
That has been nice (at least for editing neovim configuration files). But what if I am editing anything else?
For example, I have then tried to edit Wezterm config files and there are no types. I did find some types someone made online but no idea how to instruct my editor/lsp where these types are or what they are for.
You can define a metatable on your objects of interest (or the root table meta table if you don't mind breaking the language's conventions and thus libraries) with __index and __newindex members. Then you can throw in those by calling the `error` function when they'd otherwise normally return nil, should you desire it.
But runtime checks have a cost, and static types that transpile away are a bit better for overhead so long as you don't mind the build step, so using one of the typed lua variants is probably a bit nicer in the long term. Catching those typos early is their bread and butter.
The default should be an error, since that's the only way to prevent issues where the wrong value gets passed around a lot before finally blowing up somewhere else (and good luck debugging that).
In cases where you really want to fetch the value or else get some default if it doesn't exist, there should be a way to do so that is distinct from regular dereferences and element access.
> After that, I went back to Dmitry and asked him if my understanding of “everything is a table” was correct and, if so, why Lua was designed this way. Dmitry told me that Lua was created at the Pontifical Catholic University of Rio de Janeiro and that it was acceptable for Pontifical Catholic Universities to design programming languages this way.
> Semantically, Lua has many similarities with Scheme,
even though these similarities are not immediately clear be-
cause the two languages are syntactically very different. The
influence of Scheme on Lua has gradually increased during
Lua’s evolution: initially, Scheme was just a language in the
background, but later it became increasingly important as
a source of inspiration, especially with the introduction of
anonymous functions and full lexical scoping
Of course, Haskell was also Scheme influenced although it's an ML descendant.
Speaking of this, I wonder what would've happened if they embedded Lua into Netscape instead of writing JavaScript...
I think if Lua were adopted as the web language, it would have lost its spirit / design philosophy of simplicity. I think the web would have exerted an evolutionary pressure on whatever language was the chosen as the exclusive way of writing code. Everyone forced to use a single language as the only option plus too many people clamoring for their own pet language feature to be added plus design by committee will probably always lead to a mess.
I have been having thoughts, wild and incoherent, about the evolution of a given programming language. What does it take to survive and thrive? I might describe my programming career as being kind of a refugee: on the web, I moved from plain .asp to Perl, fled Perl for Python ...
At the start, a language needs the ability to evolve. I recall reading a book on Python before I adopted it, a book which was so off-putting that I ended up delaying my attempts to use Python, and it went on about Python's philosophy when it came to division. I knew that, at some point, Python would have to change away from integer division by default. And it eventually did. It was a mistake because it was non-obvious, and the language changed to fix it. Good.
Now, though, all kinds of things have crept into Python which are decidedly non-Pythonic: the walrus operator, which looks like something that escaped from a Perl dungeon; the docopt module, which has a weird trap of it being a module you must know about, but only if someone has decided to use it; the utter shambles which is the whole environment and library packaging "let entirely too many ways to do it exist;" or the community's preference for Requests while it somehow has yet to be co-opted into the standard library. I am sure I have gored someone's ox here.
Perhaps a language needs a certain window before it is finally frozen in place. Or a more stringent, forceful mandate than the Zen of Python, which I think has been Not Enough. In a way, I am reminded of the Agile Manifesto, which was well-intentioned but had no enforcement, no orthodoxy; it lacked teeth to nip at the heels of those who fail, who go astray. Originally, I had considered that the Network Effect as a way to freeze things, but it seems to only work on protocols and less so on languages.
There are people who just don't know C++ and people who also don't know that they don't know C++
This makes the existence of JTC1/SC22/WG21 aka "The C++ Standards Committee" more akin to the Académie française (which officially defines the French language, but that's not how natural languages actually work) than many C++ proponents seem to grasp.
The analogy isn't very good, because the natural French language isn't a minefield full of undefined behavior.
In Linguistics, natural language, and native speaker intuition is the gold standard. The theory of the language is deemed to be wrong where it fails to capture it.
So right off the bat we know that a prescriptive institution like Académie française is on linguistic shaky ground; it is not aligned with science.
In computing, the specification is the gold standard, followed by documented, committed implementation behaviors.
haskell is great if you enjoy learning haskell. there's always a new build system, a complete redesign of how things are done, etc. i lost the plot around the time everything became an arrow (or was it a lens ?) myself, but I'm sure a few more generations of improvement have happened already!
My favorite thing about Lua is that it trivially builds anywhere with a C compiler. Lots of other languages that bill themselves as "embeddable" are a real pain in the ass to build on weird platforms.
Fun example: there's a project (Lunatik) that embeds Lua inside the Linux kernel, where no userspace C APIs are available. You'd expect that would require extensive patching but they mostly just had to tweak the makefile and a configuration header file.
> However, when it comes to separating code into modules or packages, Python is more convenient.
Kind of curious about this. I find packaging in Lua quite convenient. I make a file and return either a value or a table full of values and then I require that file in the dependent file that uses the package.
Also, wrt to the missing features like an increment operator. It is possible in at most a few thousand lines of lua to implement a lua' -> lua transpiler where lua' adds some missing features, such as increment operators, in an idiomatic way. In other words, the generated code looks almost exactly the same as the input code except in places where say `x++` gets translated to `x = x + 1`. As long as your file is less than a few thousand lines of code, the transpiler can run in < ~200ms (and obviously much less for shorter files or much faster if the transpiler is implemented in a faster language). Of course the tradeoff is that the original source code may break existing tooling for lua, though the generated code will work fine.
makes me think that they haven't used python in a production environment. It's a mess, pip, poetry, stupid eggs.... that's just deployments then there's personal dev environments.
> Kind of curious about this. I find packaging in Lua quite convenient. I make a file and return either a value or a table full of values and then I require that file in the dependent file that uses the package.
While that aspect of it is nice, I think the way paths are handled by default can get annoying.
Since everything has to be relative from the current directory, it's not very convenient to move files around or make a contained module that depend on its own module.
If you're in charge of your own environment, sure, you can roll your own thing, but then you deviate from the norm and your code becomes less portable.
There is also the LuaRocks package manager, which I believe is decent, but it's largely ignored by a big portion of the Lua community.
> Since everything has to be relative from the current directory, it's not very convenient to move files around or make a contained module that depend on its own module.
What? Not in my experience. For instance, I have lpeg install ~/.luarocks/lib/lua/5.4 and LuaXML in /usr/local/share/lua/5.4 (just to name two modules I use). To use them, it's just
local lpeg = require "lpeg"
local xml = require "LuaXml"
Readit News
Other code outside the sandbox pulls in up to date price data and if certain safety rules are violated will automatically close out positions. So even if the traders code their way into an infinite loop or make other mistakes, the supervisor can step in and automatically prevent disaster.
Using Lua to make a language for others has been a wonderful experience. FYI, it was approx 11K lines of Lua).
Eventually they got tired of me pitching for it and fired me. How did you ever get managerial buy-in for something like this?
I'm sorry but your anectode is simply not believable. No one gets fired for suggesting a harmless feature.
You can find more details on my GitHub
https://github.com/ensisoft/detonator
Going forward being able to have seamless integration between the scripting and rest of the editor is a key differentiator and value added. For example being able to jump to the right API documentation relevant for the function call under the caret, or being able to open and jump to the game asset from the script.
I'm not sure how well a LSP type of integration would work here. As far as I can tell it'd need to evaluate the script in order to have best possible diagnostics and analysis and that won't work without the interpreter having the native game engine code available as well.
Of course my "solution" was mostly about slapping components other people built together.
Just about any thread that isn't a full-on proper thread is not ideal for general purpose game development. There are a lot of thread implementations for various user interfaces that have a goal of minimizing latency or maximizing CPU utilization. But if you are in a latency sensitive environment like a game and you're highly likely to be pushing the CPU to 100% you often need real threads not something like them.
Coroutines have their place in game development too, but they're not (in my experience) used as a thread replacement. They are used to manage complex flow control. Similarly, I have seen green threads and event loops used to manage the user interface or Futures and Promises to manage certain IO in a game but not the rest of the general game work scheduling.
You throw a lua function into a table (usually tables are used as namespaces and globals are just a table) and then make that C++ translate to your normal one and handle the return value correctly.
The hard thing is building a system to pass in a C++ object in general as that required building a table for each instance that matches its class.
What version were you using? Recent Lua versions have a smoother GC overall
I've had some success by tuning the GC with `collectgarbage('setpause', 100)` and then manually calling ` collectgarbage('step')`every once in a while.
That being said this problem isn't unique to Lua, There are horror stories about Unity and their integration of C# for example.
Dead Comment
To me, the beauty of lua is the simplicity and lack of learning curve: I can usually accomplish whatever I need to without looking anything up (as the author said, everything is a table so there isn't much to overthink). Also, the community and support around love2d is fantastic.
One thing that's bothered me is that lua silently returns nil when you reference non-existing elements. That's been a pain when debugging, since a line with a typo (`a = typo`) doesn't fail, and the code fails much farther downstream (e.g. when doing arithmetic on `a`). So almost all my errors end up being "trying to do operation on a nil value", and there is no indication of why it's nil.
I toyed with the idea of inverting the semantics of global and local in lua and to remove "local" and instead default to local and have a "global" keyword. Looking at lua.c quickly dissuaded me when I was a much more junior programmer, but now days it might be fun to try.
Deleted Comment
I am also making small games with love2d. I've found you can prevent many of such issues if you:
1. Create objects with private fields, using getters and setters to access values (as function calls will crash if you call them and the functions don't exist, unlike fields). I like an approach that is somewhat similar to this: https://www.lua.org/pil/16.4.html
2. Add assertions liberally, especially in constructors.
Last I checked I couldn't really find a good way to do that; but like I said: it's been a while.
For example, I have then tried to edit Wezterm config files and there are no types. I did find some types someone made online but no idea how to instruct my editor/lsp where these types are or what they are for.
But runtime checks have a cost, and static types that transpile away are a bit better for overhead so long as you don't mind the build step, so using one of the typed lua variants is probably a bit nicer in the long term. Catching those typos early is their bread and butter.
x = f() or "default value"
or maybe you want it to error
x = f() or error"F failed"
or maybe x is an object in which case you can just
setmetatable(x,{__index = function () return "" --[[default value here]] end})
or maybe you don't want nils, in which case
debug.setmetatable(nil, {}) -- actually this one is complicated, but yeah, you can use this.
In cases where you really want to fetch the value or else get some default if it doesn't exist, there should be a way to do so that is distinct from regular dereferences and element access.
> I can usually accomplish whatever I need to without looking anything up"
And "everything is a table"
Makes it sound a lot like SQL.
Edit: I prefer replies to downvotes.
I agree, it reads as something posted by a chatbot tha glitched.
Where's the "well yes but actually no" meme when one needs it.
It's not Haskell, it's Scheme.
https://www.lua.org/doc/hopl.pdf
> Semantically, Lua has many similarities with Scheme, even though these similarities are not immediately clear be- cause the two languages are syntactically very different. The influence of Scheme on Lua has gradually increased during Lua’s evolution: initially, Scheme was just a language in the background, but later it became increasingly important as a source of inspiration, especially with the introduction of anonymous functions and full lexical scoping
Of course, Haskell was also Scheme influenced although it's an ML descendant.
Speaking of this, I wonder what would've happened if they embedded Lua into Netscape instead of writing JavaScript...
At the start, a language needs the ability to evolve. I recall reading a book on Python before I adopted it, a book which was so off-putting that I ended up delaying my attempts to use Python, and it went on about Python's philosophy when it came to division. I knew that, at some point, Python would have to change away from integer division by default. And it eventually did. It was a mistake because it was non-obvious, and the language changed to fix it. Good.
Now, though, all kinds of things have crept into Python which are decidedly non-Pythonic: the walrus operator, which looks like something that escaped from a Perl dungeon; the docopt module, which has a weird trap of it being a module you must know about, but only if someone has decided to use it; the utter shambles which is the whole environment and library packaging "let entirely too many ways to do it exist;" or the community's preference for Requests while it somehow has yet to be co-opted into the standard library. I am sure I have gored someone's ox here.
Perhaps a language needs a certain window before it is finally frozen in place. Or a more stringent, forceful mandate than the Zen of Python, which I think has been Not Enough. In a way, I am reminded of the Agile Manifesto, which was well-intentioned but had no enforcement, no orthodoxy; it lacked teeth to nip at the heels of those who fail, who go astray. Originally, I had considered that the Network Effect as a way to freeze things, but it seems to only work on protocols and less so on languages.
Only language where you can keep having this realization over and over forever. Of course people prefer lua if they can get away with it.
This makes the existence of JTC1/SC22/WG21 aka "The C++ Standards Committee" more akin to the Académie française (which officially defines the French language, but that's not how natural languages actually work) than many C++ proponents seem to grasp.
In Linguistics, natural language, and native speaker intuition is the gold standard. The theory of the language is deemed to be wrong where it fails to capture it.
So right off the bat we know that a prescriptive institution like Académie française is on linguistic shaky ground; it is not aligned with science.
In computing, the specification is the gold standard, followed by documented, committed implementation behaviors.
But yeah, I guess C++ is the one single language where you can keep learning more and more fundamental things, forever.
Kind of curious about this. I find packaging in Lua quite convenient. I make a file and return either a value or a table full of values and then I require that file in the dependent file that uses the package.
Also, wrt to the missing features like an increment operator. It is possible in at most a few thousand lines of lua to implement a lua' -> lua transpiler where lua' adds some missing features, such as increment operators, in an idiomatic way. In other words, the generated code looks almost exactly the same as the input code except in places where say `x++` gets translated to `x = x + 1`. As long as your file is less than a few thousand lines of code, the transpiler can run in < ~200ms (and obviously much less for shorter files or much faster if the transpiler is implemented in a faster language). Of course the tradeoff is that the original source code may break existing tooling for lua, though the generated code will work fine.
While that aspect of it is nice, I think the way paths are handled by default can get annoying.
Since everything has to be relative from the current directory, it's not very convenient to move files around or make a contained module that depend on its own module.
If you're in charge of your own environment, sure, you can roll your own thing, but then you deviate from the norm and your code becomes less portable.
There is also the LuaRocks package manager, which I believe is decent, but it's largely ignored by a big portion of the Lua community.
What? Not in my experience. For instance, I have lpeg install ~/.luarocks/lib/lua/5.4 and LuaXML in /usr/local/share/lua/5.4 (just to name two modules I use). To use them, it's just
I'm confused as to what you mean.