Why not Lua?

[dpacbach]

Question for devs -- I believe if I'm not mistaken that you said that you (kovarex?) had regretted using Lua as the scripting language for factorio -- why?
Thanks

[quinor]

Lua is a very crappy and quite slow language (though with lua jit it got better), something like Python would be much more powerful. Mods in factorio are often very big pieces of complex code. Also I don't know whether there were some problems with determinism of Lua for multiplayer.

[Jap2.0]

Post by Rseding91 on Reddit

[DaveMcW]

Lua tables can get very complicated, with references to tables inside tables. Unfortunately references are stored using a local memory address, meaning they are non-deterministic and break multiplayer.

So the modder learning curve goes like this:
1. Build simple mod with simple tables.
2. Build more complicated mod with more complicated tables.
3. Mod breaks in multiplayer.
4. Rebuild more complicated mod with simple tables.

That is a lot of pain for modders and online players to go through.

[dog80]

Post by Rseding91 on Reddit

its bad language cause it uses ~= instead of != kek

[reverse.yellow]

Post by Rseding91 on Reddit

its bad language cause it uses ~= instead of != kek

No, if that's seriously that much of a problem you can just modify the Lua parser to use != instead of ~=.

[dog80]

i was just referencing to rsedings quote - but seriously i dont have any idea of why it would be a bad or good scripting language at all - its propably just the same thing as usual - when you want something to work quick and in a way reliable you use lua - once you get into it and want to maximize performance and control over it - you write your own...

i appreciate rsedings approach and desire to write an own scripting language for factorio, though it has to be concidered if this is more of a personal desire to write it and maximize its functionality - compared to if it is really usefull for the further progression of the game in a whole...

[zOldBulldog]

As a professional developer that used many formal and scripting languages through his career I can guess at their reasons.

For one, Lua is quite obscure and minor compared to others, so it must be hard to find experienced developers they can hire. It is probably also hard to find libraries to enhance its core feature set. I am not familiar with Lua and so I can't speak for how flexible and easy it is, or how fast it performs, but from using the game I would say that it is probably fairly decent or the dev could not have achieved a game that runs so well even in weak and old machines (unless you make a Megabase).

But I will bet the dev are now wishing they wrote the game in Java (or at least C++).

[posila]

But I will bet the dev are now wishing they wrote the game in Java (or at least C++).

Kovarex wrote the first version in Java and after couple of months switched to C++. So the game is written in C++.

Lua is used just for game objects definitions (prototypes) - which are read once on startup, stored in native data structures and after that there is no further interaction with Lua in that part. And it's also used for scripting API, which is used by vanilla only very lightly.

I don't mind Lua, I think it is very reasonable solution for what we need, but I am not the one who's maintaining scripting API for the most part. Also I am the one who objected to Rseding adding !=, &&, || operators (because it would prohibit people from using existing Lua parsers to write tools or apps that read game prototypes)

[zOldBulldog]

But I will bet the dev are now wishing they wrote the game in Java (or at least C++).

Kovarex wrote the first version in Java and after couple of months switched to C++. So the game is written in C++.

Lua is used just for game objects definitions (prototypes) - which are read once on startup, stored in native data structures and after that there is no further interaction with Lua in that part. And it's also used for scripting API, which is used by vanilla only very lightly.

I don't mind Lua, I think it is very reasonable solution for what we need, but I am not the one who's maintaining scripting API for the most part. Also I am the one who objected to Rseding adding !=, &&, || operators (because it would prohibit people from using existing Lua parsers to write tools or apps that read game prototypes)

That is a brilliant setup and it totally explains why the game performs so well. For raw execution speed there is little that will beat C++.

So, Lua is just used by a few things. I assume that it is discussed so much then because it is what users get to interact with? And I imagine it is what mods get written with?

[ratchetfreak]

The default implementation doesn't do any optimizations apart from tail-call

The only native data structure is a hybrid array/hashtable and depending on how you filled the table you are not guaranteed that all numeric keys are in the array part. I believe factorio already changed these to get consistent iteration order.

The C-api is a weird hybrid of stack and indexed based api and it's not always clear why some arguments are on the stack and why some are indexed. For example lua_gettable (lua_State *L, int index) takes the table at the index, pops the key from the stack and pushes the value on the stack.

In older versions the only way to keep lua values along with non-light userdata was through the registry, you cannot have a handle directly to a value stored in the lua vm, but using the registry like that invites circular references and related leaks. In new versions you can add upvalues to userdata tying the lifetime of the userdata properly to the extra lua values' lifetime.

[WeirdConstructor]

Even though Lua has no nifty optimizations, it's still well 10x faster than for example Python. I benchmarked it multiple times and wrote multiple scripting languages implementations myself over the years. Reaching that versatility and raw performance without a JIT compiler in a scripting language is very advanced. There are decades of optimization and well thougt out design in Lua that is often under appreaciated. Yes, the weird 1 based indexing and ~= are warts, and they hurt experienced programmers like me. But on the other hand JavaScript and Python have even more warts if you look closely.

[darkfrei]

Yes, the weird 1 based indexing and ~= are warts, and they hurt experienced programmers like me. But on the other hand JavaScript and Python have even more warts if you look closely.

I found that indexing from 1 is much easier for real practice, but normally I don't use the index, just the values of elements. But here can be also "a-b-c" indexing, when I want it.

[neoc]

Yes, the weird 1 based indexing and ~= are warts, and they hurt experienced programmers like me. But on the other hand JavaScript and Python have even more warts if you look closely.

I found that indexing from 1 is much easier for real practice, but normally I don't use the index, just the values of elements. But here can be also "a-b-c" indexing, when I want it.

At first I felt like any language that doesn't use zero-based indexing is just bad and I wouldn't want to use it, but after a while you realize that it's really no big deal. For example if you switch from zero-based to 1-based in an indexing loop, you only have to change the comparism operator:

Code: Select all

for(i = 0; i < 10; ++i)

Code: Select all

for(i = 1; i <= 10; ++i)

Once you get used to it, it's just a context switch in your thinking.

[quyxkh]

Lua tables can get very complicated, with references to tables inside tables. Unfortunately references are stored using a local memory address, meaning they are non-deterministic and break multiplayer.

So the modder learning curve goes like this:
1. Build simple mod with simple tables.
2. Build more complicated mod with more complicated tables.
3. Mod breaks in multiplayer.
4. Rebuild more complicated mod with simple tables.

That is a lot of pain for modders and online players to go through.

... so, it can't be all nested tables are forbidden in multiplayer `global`s, just the ones with multiple references? trees ok, dags not okay, cycles not okay, or what?

[Rseding91]

Lua tables can get very complicated, with references to tables inside tables. Unfortunately references are stored using a local memory address, meaning they are non-deterministic and break multiplayer.

So the modder learning curve goes like this:
1. Build simple mod with simple tables.
2. Build more complicated mod with more complicated tables.
3. Mod breaks in multiplayer.
4. Rebuild more complicated mod with simple tables.

That is a lot of pain for modders and online players to go through.

... so, it can't be all nested tables are forbidden in multiplayer `global`s, just the ones with multiple references? trees ok, dags not okay, cycles not okay, or what?

Nested tables is fine, recursive tables is fine, multiple references is fine - it's all fine. It's all deterministic because we changed the Lua internals to make it deterministic.

You aren't going to get desyncs in multiplayer as DaveMcW said.

[zOldBulldog]

Lua tables can get very complicated, with references to tables inside tables. Unfortunately references are stored using a local memory address, meaning they are non-deterministic and break multiplayer.

So the modder learning curve goes like this:
1. Build simple mod with simple tables.
2. Build more complicated mod with more complicated tables.
3. Mod breaks in multiplayer.
4. Rebuild more complicated mod with simple tables.

That is a lot of pain for modders and online players to go through.

I doubt I will ever bother to learn Lua, but based in my experience with many scripting and programming language over the years I would imagine one or two more steps:

5. Write a library mod that encapsulates and hides away all the ugly bits (like tables) and makes frequent tasks easier and cleaner.

Or:

5b: Find one or more such libraries written by somebody else and just use them (for example this is what most people do in C++, Java or JavaScript)

Has nobody written a library to encapsulate Lua tables yet? Or better yet, a library to serve as a comprehensive basis for Factorio mod design? (besides the official API)

If no.... That is one great opportunity for a popular (if difficult to write well) mod.

[quyxkh]

"tables" is lua-speak for slightly-augmented dictionaries aka associative arrays. {1,2,3} is a table, {x=1,y=2} is a table, it's how lua structures data. x={1,2,3} x.note="It's probably lua's best feature, though i'm learning to appreciate its iteration and metatables". I don't understand why assignment isn't an expression and it doesn't have the modifiers like += and ++, in a language that goes to such lengths to support brevity in other ways forcing the verbosity of `count = count + 1 if count > #x` rather than `if ++count > #x` seems senseless.

[BenGoldberg]

I wonder why factorio doesn't have a C modding api... that would in turn allow mods to be written in any programming language which has some form of ffi... python, ruby, javascript, java, lisp, perl6, etc

[caHarkness]

Lua is probably the most underappreciated, useful scripting language there is. The fact that Factorio uses it even excites me more because practically program or app I write I find a way to make it extensible with Lua.

I can't think of anything I would put in its place to do the same job. I remember reading in-depth as to why Lua exists, and it was originally designed for making a project highly configurable for someone who wanted to spend more time experimenting with it than learning and writing C code. I tried looking for it on Lua.org but gave up. I don't remember exactly which page it's on, but it explains everything.

Lua isn't exactly a useful programming language on its own, but where it really shines is how differently it is implemented in all sorts of projects. In fact, the Lua interpreter itself is just a CLI implementation of Lua that allows scripting generic I/O and arithmetic calculations. If you want Lua to be able to "draw" a game or calculate game logic, your C code must already be able to do that in order to create Lua bindings.

Unlike Python, Lua's syntax is very flexible and virtually understood by every programmer. The only learning curve that is actually worth mentioning about its syntax is the whole vararg thing. Every evaluated chunk of code or expression in Lua is represented as a value and even a group of values, e.g. a tuple or a table, is still a value. Kind-of. It makes more sense to a programmer who is using it to make something scriptable.

Not to mention it's also easy for a non-programmer to look at Lua, change a thing or two, and produce results.