I used to use Lua and later LuaJIT in Lumix Engine. I switched to Luau because of its type system. However, it's apparent it was not meant to be used outside Roblox, as it has many rough corners. The documentation is not great, and the community is basically nonexistent - I got zero results when searching for any issues I encountered. Also, it's huge compared to Lua or LuaJIT, causing my project to compile 7x slower. The API is not great (e.g., an async API that blocks, using STL in the API, leaking STL headers). I encounter bugs with analysis/LSP often. Overall, I consider moving away from it.
We definitely intend on folks being able to use Luau outside of Roblox, and we know of a number of folks doing so quite successfully including Remedy Entertainment (Alan Wake 2), Digital Extremes (Warframe), GIANTS Software (Farming Simulator 25).
That being said, it has been historically hard to get major investment into work actively supporting growth of the language off-platform since our entire team is employed to work on the project by Roblox. We are nevertheless changing this though, and investing in the language outside of the platform. As some folks have already mentioned here, we have a general-purpose standalone runtime that we're developing called Lute that's focused on using Luau outside of Roblox to write general-purpose programs, and we're building a whole suite of Luau-programmable developer tools for the language atop it.
It takes time to build things, and the Luau ecosystem is definitely still very young as you've noted, but it's something that we care a lot about and are investing in considerably going forward. We 100% believe that the best thing for the health of the language and the ecosystem is to support more diverse users and more diverse use-cases.
I use Luau with my Unity game for all gameplay code (60k+ loc so far). I agree the docs could be improved, especially for custom integrations. The new type system + LSP aren't great together yet either, and I've been a little worried about the direction since zeux left the team.
But otherwise I really like it. My Luau code hot reloads almost instantly while the game is running, and my C++ project compiles quickly when I update it. I like that it provides sandboxing if you want to support modding. And there's an active community of people on the official Discord, which is better than nothing.
Fast hot reload is not Luau specific though. Lua and LuaJIT reloads fast too, so does JS and most other scripting platforms. In fact, I have subsecond C++ hot-reloads.
Do you have a link for the official Luau Discord? I can't see anything on their webpage or github and google is not helpful either.
Thanks for sharing your experience. I'm curious why it would cause your project to compile slower though? It's a scripting language, so you don't need to compile anything, right? Do you mean compiling the Luau VM itself?
Luau seems to be significantly more complex than Lua - I'm not sure it can still be called "small". Looking at the relative size of the implementations: Luau's is 120,000 lines of C++ [0], an order of magnitude larger than Lua 5.1's 14,000 lines of C.
But I think that complexity is unavoidable for a gradually- or statically-typed language. Any language with a reasonably-complete type system is inevitably going to be much more complex than a dynamically-typed scripting language.
[0] Counting *.cpp files in the "Analysis", "AST", "Compiler" and "VM" directories
Lua (and to a somewhat lesser extent Luau) are small in terms of the learned surface of the value language, not necessarily in terms of lines of code. That being said, any runtime use of the language needn't depend on Analysis, which is the biggest compilation unit by far.
Probably also worth mentioning that Analysis currently contains two full type system implementations because we've spent the better part of the past three years building a new type system to address a lot of the fundamental limitations and architectural issues that we'd run into after years of working on the original one. The new one is not without issues still, but is definitely very usable, and gets better and better every week. At some point in the future, we will clip the old one altogether, and that'll shave off some 30,000 lines of code or something.
Is the primary goal of the type system performance, or dev productivity, or something else?
If performance, are you comparing against a baseline that does the classic Self-style optimizations (like what JS engines do)?
FWIW I don't consider LuaJIT to be an interesting baseline because of what I've heard about it being overtuned to specific benchmark that the author(s) care about. (Too much of an attitude where if your code doesn't run well then it's your fault.)
I fully agree. Lua and Luau are impressive, sure, but they are not really "small" or "simple", in my view. I don't think the complexity is unavoidable however. There are many programming languages that are much simpler, but at the same time very expressive. I'm working on one of them currently named "Bau" [1], and I started working on a Lua-inspired VM [2] for a subset of this language. There are many languages like mine, most of them incomplete and not really popular, discussed in [3].
I agree that static and especially gradual typing add complexity, but it's a very much smaller amount of complexity than we're talking about here, so in fact it is very common to encounter dynamically-typed scripting languages that are much more complex than some languages with excellent type systems.
I think you can implement a Hindley–Milner type checker in about a page of code, not the 2000 pages of code you're talking about.
I'm not sure what you mean by "complete". H–M is complete in the sense that it's decidable and doesn't leave any holes: programs that check statically are guaranteed not to have type errors at runtime. It handles higher-order functions and parametric polymorphism (generics) out of the box, it doesn't suffer from null pointers, and it can even handle mutability. And it's fully inferrable. There are various extensions to make it more expressive (GADTs, typeclasses, subtyping, linearity, tagged arguments) but even the basic HM system is already a lot more powerful than something like the type system of C or Java 1.7.
> it is very common to encounter dynamically-typed scripting languages that are much more complex than some languages with excellent type systems.
You're right, that statement was too general. Python is a dynamically-typed scripting language (if you exclude external tools like MyPy), and is one of the most complex languages out there.
I should have been more specific: "Any language with a reasonably-complete type system is inevitably going to be much more complex than Lua".
> I agree that static and especially gradual typing add complexity, but it's a very much smaller amount of complexity than we're talking about here [...] I think you can implement a Hindley–Milner type checker in about a page of code
aw1621107's comment shows that Luau's type checker (the "Analysis" directory) is ~60% of the project's code. Maybe there are languages where the equivalent is just a single page, but even then, type checking makes a language implementation more complex in other ways as well.
For example, Luau's AST implementation alone is 75% the size of the whole of Lua 5.1. By deferring type-checking to runtime, Lua can avoid the need to build an AST at all: the compiler can go straight from source code to bytecode.
I have written a Lua type checker in the past and have delved enough into the Lua source code. For that reason I can say that Lua is particularly densely coded; Lua's 14K LoC is something more like 3--40K LoC when coded in the normal, less dense way. Lua is not necessarily small; it's just kinda concise.
To be fair, both `Analysis` (the type-checker, not necessary at runtime or compile time) and `CodeGen` (the optional JIT engine) have no equivalent in PUC-Rio Lua.
If you look purely at the VM and things necessary to compile bytecode (AST, Compiler and VM) then the difference in code size isn't as stark.
Having worked with both Lua 5.1 and Luau VM code, Luau's codebase is a heck of a lot nicer to work on than the official Lua implementation even if it is more complex in performance-sensitive places. I have mixed feelings on the structural typing implementation, but the VM itself is quite good.
Teal compiles teal files into plain Lua just like TS does for JS. So all the advantages and disadvantages apply.
Luau is a backwards compatible superset of Lua that comes with it's own performance-tuned runtime. It offers more than just gradual typing.
So they are very different things. You can use Teal in cases when you don't control the runtime. Like write a Love2d game or your neovim config in it. Anywhere where Lua runs, you can use teal.
On the other hand Luau can offer superior developer experience because you don't have a separate compile step. They can do a lot more things that are impossible with teal as they have their own runtime and types do not get erased by compiling.
Teal transpiles to Lua, but Luau is a fork of Lua. Luau can implement wider ranging changes, like improving interpreter performance and security or adding syntactic sugar.
Roblox has a market cap near $100B and has multiple developers working full-time on Luau.
Luau isn't "Lua but with types," but rather a language in the Lua family that has grown a powerful gradual type system with a _ton_ of type inference (this is in contrast to e.g. TypeScript which takes the approach more of "add annotations to get benefits"), as well as a growing number of additions to the language that we believe make it a more pleasant developer experience to use. We're very focused on preserving the general ability to embed the language readily and with low costs to binary size and so forth, but we're less ruthlessly committed to the language's simplicity (for both better and worse). Overall, the goal is an embeddable language related to Lua with a focus on developer tooling and a positive developer experience such that people actually want to (and enjoy) build(ing) software in the language, rather than e.g. build large C projects and then expose them to Lua.
It's a shame that Lua did not evolve in a more backwards-compatible manner. In addition to Roblox, lots of others projects started adopting Lua 5.1 as a scripting language in the late 00s. Lua itself is now at 5.4, but it did not keep backwards compatibility. LuaJIT and related projects pretty much only support 5.1. It's similar to the situation Python had with 2.x/3.x, except that the majority of Lua users I am aware of are preferring to stay with the older 5.1.
I think it's even worse than that, luau and luaJIT have evolved in different directions than the official lua project, such that they are now all sublty incompatible with each others. They all branch from lua 5.1 but it feels like there isn't an offical standard anymore.
At the very least, there's a common core of Lua 5.1 that works across Luau, LuaJIT, and PUC Lua, so it's not as if there's no standardization here. We definitely aspire to include _more_ than just Lua 5.1 in Luau though.
The huge difference is that the Lua community doesn't attack people publicly for maintaining backward compatibility, so it's generally pretty easy to write code that works across a wide range of Lua versions.
It's hard to get reliable numbers on this but I believe 5.1 and 5.2 are both more popular than 5.4 which has been out for five years now. And I don't think 5.3 ever surpassed either of them. I'm not sure about luajit it gets a lot of attention but I don't see it around all that much.
I learned about Luau via my 13 years old who is looking into Roblox Studio. That's how I ended up visiting luau.org and I'm quite impressed by Roblox's engineering on this.
Arseny Kapoulkine is an amazing engineer. Highly recommend following his blog or social media. Other than working on luau and the rendering engine at Roblox, he's also responsible for meshoptimizer which if you're in graphics you've most definitely heard of, and volk, which now comes packaged with the Vulkan SDK.
Second Life is switching over from Linden Scripting Language to Luau. It's working out well. The existing system compiled to Mono, and with Mono being deprecated (abandoned?) something new was needed. Amusingly, not only is Luau being supported, the LSL compiler is now targeting the Luau execution engine, and that works. Performance has improved slightly.
Second Life runs on hundreds of thousands of tiny programs, all event driven and running on the servers. Managing that is tricky, since server side resources are limited. Yet it works well, even if user programs are compute-bound. Actually, the biggest problem is that each idle program uses about 1us per frame, which adds up.
When they had the Luau beta regions up and running I gave it a whirl and it seemed performance was greatly improved over the old Mono system. Clicking "Save" was near instant to execute, great improvement to clicking save and waiting 10 seconds to know whether or not there's an issue in your script.
The only desire I have is if they could adopt FiveM-style helper functions which help wrap coroutines, namely being CreateThread(fn) and Wait(ms) (wrapper around yield inside that "thread") and Await/Promises (seems like someone already made an implementation for Luau? https://github.com/evaera/roblox-lua-promise)
FiveM adopting these makes it easy to make better performing scripts without having to mangle coroutines, which is vital given the Lua VM has to finish its current task before the frame is allowed to render.
I'm the contractor responsible for SL's Luau VM integration, appreciate the kind words about the Luau integration!
We're still in figuring out our async strategy for user-facing APIs to be honest, so these references are super helpful. We already have preemptive scheduling of execution, but it's most likely to be some kind of wrapper around `coroutine.create()` where an event loop is responsible for driving execution flow and internal `coroutine.yield()`s let you specify what you're `await`ing.
We'll likely have an RFC for how that will all work within the year, but several users have written their own bespoke `async` / `await` implementations for SL in Lua already.
One thing that was immediately apparent upon switching VMs was that a lot of the existing overhead was in scheduling, context switching and the implementation of the actual library functions like `llDoWhatever()`.
We haven't even used Luau's JIT at all yet, but preemptive scheduling of what's typically trivial glue code is much cheaper and easier with a VM that supports it as a natural consequence of its design versus transforming everything into a state machine at the AST or bytecode level for a VM that doesn't.
> Actually, the biggest problem is that each idle program uses about 1us per frame, which adds up.
(For those not familiar with Second Life, every object that does something has a little program in it. Every chair you can sit on, every door you can open, and every clothing item where you can change colors has a small program written by some users. Most of those programs are almost always idle. But there's a tiny amount of CPU time consumed on each frame for each idle program, about 1us to 2us in the Mono implementation. A region can have 10,000 little programs, each eating 1us on each simulation cycle, 45 times a second. This adds up.)
Readit News
That being said, it has been historically hard to get major investment into work actively supporting growth of the language off-platform since our entire team is employed to work on the project by Roblox. We are nevertheless changing this though, and investing in the language outside of the platform. As some folks have already mentioned here, we have a general-purpose standalone runtime that we're developing called Lute that's focused on using Luau outside of Roblox to write general-purpose programs, and we're building a whole suite of Luau-programmable developer tools for the language atop it.
It takes time to build things, and the Luau ecosystem is definitely still very young as you've noted, but it's something that we care a lot about and are investing in considerably going forward. We 100% believe that the best thing for the health of the language and the ecosystem is to support more diverse users and more diverse use-cases.
But otherwise I really like it. My Luau code hot reloads almost instantly while the game is running, and my C++ project compiles quickly when I update it. I like that it provides sandboxing if you want to support modding. And there's an active community of people on the official Discord, which is better than nothing.
Do you have a link for the official Luau Discord? I can't see anything on their webpage or github and google is not helpful either.
Dead Comment
https://github.com/luau-lang/lute
https://github.com/lune-org/lune
i use it for build scripts and automating tasks for Roblox place files, it's pretty good for my use-case
But I think that complexity is unavoidable for a gradually- or statically-typed language. Any language with a reasonably-complete type system is inevitably going to be much more complex than a dynamically-typed scripting language.
[0] Counting *.cpp files in the "Analysis", "AST", "Compiler" and "VM" directories
Probably also worth mentioning that Analysis currently contains two full type system implementations because we've spent the better part of the past three years building a new type system to address a lot of the fundamental limitations and architectural issues that we'd run into after years of working on the original one. The new one is not without issues still, but is definitely very usable, and gets better and better every week. At some point in the future, we will clip the old one altogether, and that'll shave off some 30,000 lines of code or something.
If performance, are you comparing against a baseline that does the classic Self-style optimizations (like what JS engines do)?
FWIW I don't consider LuaJIT to be an interesting baseline because of what I've heard about it being overtuned to specific benchmark that the author(s) care about. (Too much of an attitude where if your code doesn't run well then it's your fault.)
[1] https://github.com/thomasmueller/bau-lang [2] https://github.com/thomasmueller/bau-lang/blob/main/src/test... [3] https://www.reddit.com/r/ProgrammingLanguages/
I think you can implement a Hindley–Milner type checker in about a page of code, not the 2000 pages of code you're talking about.
I'm not sure what you mean by "complete". H–M is complete in the sense that it's decidable and doesn't leave any holes: programs that check statically are guaranteed not to have type errors at runtime. It handles higher-order functions and parametric polymorphism (generics) out of the box, it doesn't suffer from null pointers, and it can even handle mutability. And it's fully inferrable. There are various extensions to make it more expressive (GADTs, typeclasses, subtyping, linearity, tagged arguments) but even the basic HM system is already a lot more powerful than something like the type system of C or Java 1.7.
You're right, that statement was too general. Python is a dynamically-typed scripting language (if you exclude external tools like MyPy), and is one of the most complex languages out there.
I should have been more specific: "Any language with a reasonably-complete type system is inevitably going to be much more complex than Lua".
> I agree that static and especially gradual typing add complexity, but it's a very much smaller amount of complexity than we're talking about here [...] I think you can implement a Hindley–Milner type checker in about a page of code
aw1621107's comment shows that Luau's type checker (the "Analysis" directory) is ~60% of the project's code. Maybe there are languages where the equivalent is just a single page, but even then, type checking makes a language implementation more complex in other ways as well.
For example, Luau's AST implementation alone is 75% the size of the whole of Lua 5.1. By deferring type-checking to runtime, Lua can avoid the need to build an AST at all: the compiler can go straight from source code to bytecode.
- Analysis: 62821 lines of C++ code, 9254 lines of C headers
- Ast: 8444 lines of C++, 2582 lines of C headers
- CodeGen: 21678 lines of C++, 4456 lines of C headers
- Compiler: 7890 lines of C++, 542 lines of C headers
- VM: 16318 lines of code, 1384 lines of C headers
Compare to Lua 5.1, which tokei says has 11104 lines of C and 1951 lines of C headers in the src/ directory.
If you look purely at the VM and things necessary to compile bytecode (AST, Compiler and VM) then the difference in code size isn't as stark.
Having worked with both Lua 5.1 and Luau VM code, Luau's codebase is a heck of a lot nicer to work on than the official Lua implementation even if it is more complex in performance-sensitive places. I have mixed feelings on the structural typing implementation, but the VM itself is quite good.
[0] https://teal-language.org/
Luau is a backwards compatible superset of Lua that comes with it's own performance-tuned runtime. It offers more than just gradual typing.
So they are very different things. You can use Teal in cases when you don't control the runtime. Like write a Love2d game or your neovim config in it. Anywhere where Lua runs, you can use teal.
On the other hand Luau can offer superior developer experience because you don't have a separate compile step. They can do a lot more things that are impossible with teal as they have their own runtime and types do not get erased by compiling.
Roblox has a market cap near $100B and has multiple developers working full-time on Luau.
See https://luajit.org/extensions.html.
To me, this is the more interesting bit of luau
The performance page[1] contains a pretty good explanation of the work they have done. Pretty impressive engineering if you ask me.
[1] https://luau.org/performance
Second Life runs on hundreds of thousands of tiny programs, all event driven and running on the servers. Managing that is tricky, since server side resources are limited. Yet it works well, even if user programs are compute-bound. Actually, the biggest problem is that each idle program uses about 1us per frame, which adds up.
The only desire I have is if they could adopt FiveM-style helper functions which help wrap coroutines, namely being CreateThread(fn) and Wait(ms) (wrapper around yield inside that "thread") and Await/Promises (seems like someone already made an implementation for Luau? https://github.com/evaera/roblox-lua-promise)
FiveM adopting these makes it easy to make better performing scripts without having to mangle coroutines, which is vital given the Lua VM has to finish its current task before the frame is allowed to render.
https://github.com/citizenfx/fivem/blob/master/data/shared/c...
We're still in figuring out our async strategy for user-facing APIs to be honest, so these references are super helpful. We already have preemptive scheduling of execution, but it's most likely to be some kind of wrapper around `coroutine.create()` where an event loop is responsible for driving execution flow and internal `coroutine.yield()`s let you specify what you're `await`ing.
We'll likely have an RFC for how that will all work within the year, but several users have written their own bespoke `async` / `await` implementations for SL in Lua already.
We haven't even used Luau's JIT at all yet, but preemptive scheduling of what's typically trivial glue code is much cheaper and easier with a VM that supports it as a natural consequence of its design versus transforming everything into a state machine at the AST or bytecode level for a VM that doesn't.
> Actually, the biggest problem is that each idle program uses about 1us per frame, which adds up.
More scheduler overhead to resolve :)