Factorio Forums

ptx0 wrote: Thu Mar 04, 2021 2:40 pm it's written in C++ and the compiler and libraries it links to are responsible for ensuring compatibility with the OS and CPUs. somehow it's not a problem that Factorio binary is compiled and passed around to many different types of systems!

Modders are not Wube, some can barely write lua, and you are talking about setting up cross-compilation to Mac and Linux.

ikm wrote: Sat Mar 06, 2021 10:48 am setting up cross-compilation to Mac and Linux

In theory, devs can provide ready-made SDK where everything is set up and ready to work. It's a total overkill and Sisyphean labor, though. Unless you're creating a generic engine like UE or Unity.

My prefered solution would be:

- A c++ modding API
- Strict guidelines and rules what libraries are allowed, what pre-compiler and compiler to use and how they must be configured.
- A build server provided by the studio. Only source-code can be uploaded to the mod portal. That way you can test building locally and then upload correct source code.

- Additional to the c++ API a Google V8 integration, so mods can also be written in JavaScript.
- JavaScript, a widely known scripting language with c-syntax (and no bullshit like 1-based indexes). Can be used oo (prototypical) but does not have to be.
- I don't know how the stack works, but maybe it would be possible to implement the V8 integration as a c++ mod itself. So V8 would itself only facilitate Facotrio's c++ modding API.
- Knowing about V8 and reading their introductory docs, V8 has everything Rseding was talking about in the interview and is talked about here in this thread and seems to be every devs dream, who wants to process scripts.

Sugar, cream and champagne on top of the cake
- I don't know how the lua-interpreter works, but V8 can be used to compile js-files into bytecode. So, an alternative for js-mods for even more speed would be to have the build server compile them. Thus the factorio engine would only use compiled mods, while developing js-mods using js-files without compiling them locally, would still be possible.

- My prefered solution would also include to erradicate everything about lua from the internet and humanities memory.

Update:
This article from 3 years back explains how V8 works internally and that it always compiles the script code.

Impatient wrote: Sat Mar 06, 2021 11:50 am JavaScript, a widely known scripting language with c-syntax

Another cool feature of V8 is that you can use other languages like TypeScript if you don't like JS and want stricter type checks.

... if you make non-programmers have to deal with pointers and memory management ...

Dealing with pointers is trivial and can be learned in 5 minutes. Memory management is pretty much always handled by the system or language libraries and most likely, would never be seen in an interpreter. Pointers can be hidden or just not used as well as shown in the implementation of C++ for the Arduino.
So not really an issue. The libraries can hide pretty much all of the 'difficult' things.

- My prefered solution would also include to erradicate everything about lua from the internet and humanities memory.

Yes!

I have to admit, some of my viewpoints may not be common any more. After Basic in college, and Fortran later in the military, I learned to write machine code from scratch without an assembler. That was back before personal computers were common. The first 'computer' I owned was a single board development board that had a telephone like keypad and some LED numeric displays. The machine code was entered one hex or octal value at a time. I think this was before recording on a small cassette recorder was available. I remember typing in the 'bytes' of code from a magazine so that I could play the game 'Hunt the Wumpus'.
Computers were easy for me for some reason. And I am starting to feel really old!

jamiechi1 wrote: Sun Mar 07, 2021 7:09 am Dealing with pointers is trivial and can be learned in 5 minutes.

The rules of chess are trivial and can be learned in 5 minutes.

IMHO a scripting language for something like Factorio should have two things:
Purity
Simplicity

Lua mostly meets the second requirement, but it's lack of purity is what makes it awful for safety, performance and flexibility.

It is somewhat ironic that a project like fCPU, uses combinators internally to get performance improvements, combinators are somewhat of a pure / specialized interpreter, this enables the use of SIMD internally, but they're extremely limited in features.

Most interpreted imperative/OO scripting languages like python, lua, and ruby are on the order of 100x slower than C code, or if you have people who can write a JIT compiler like V8: use stupid amounts of memory

For memory safety you either have to have garbage collection, or a proof system (i.e rust's affine types), a pure language would also help mitigate issues with garbage collection, since we know the life times of said data more reliably, incremental engines also solve the freeze the world issue you get, Unity has this, as well as Haskell.

Based on the way modern Unity games have limited stutter, and even with careful control of GC, and minimal heap allocation for the modding system, I don't think GC is ever an issue for a scripting / modding language.

Statically compiled languages *could* work if Factorio can include the compiler, and only compile the code on mod load, this solves portability issues.

Purity allows effortless parallelism, most scripting languages have terrible threading performance because closures and mutability demand locks everywhere.

I personally think types are great for safety, but understand why Lua, python etc forgoes them, they can be a pain to deal with, But ADT's are extremely powerful, and reduce the allocation heavy "just use an object for everything", and parametric polymorphism plus type inference (like what Haskell has) can make coding feel similar to dynamic languages.

I personally haven't found a scripting language that ticks all the boxes for me (especially since pure languages are almost non-existent), but there's a collection of languages that get a lot closer than Lua, especially if you're willing to sacrifice some simplicity for safety and performance.

You can see here, how a simple dynamically typed language like Lisp without manual memory allocation can perform on par with C if compiled, or well designed interpreted niche language like Erlang can compete with far less engineering/man hours.

Part of me wishes that people could actually learn something like Haskell, a subset of it could be simple enough to do scripting in it, it's one of the few pure languages around which enables huge amounts of safety, not only can you remove the likes of IO with the type system, but you can do pure multithreading with map, fold and par and not worry about non-determinism, you could use some black magic to get seamless parallelism on event callbacks too.

I've been wanting to make a simple pure functional language for a programmable combinator like fCPU, but Lua makes it rather hard to scale well especially for UPS.

I really can't fault Wube for choosing Lua, or at least I don't know well enough to say that it was definitely the wrong choice. However, I imagine something like C or Rust for modding would have been much harder to implement. Back then, there would have also been a little more uncertainty about the project's longevity and playerbase, although there definitely must have been a clear future vision. It's probably fair that they didn't want to go down that rabbit hole, and instead they chose something simpler. On paper, Lua may also appeal to more people in general, even though a large portion of the biggest mod makers probably would prefer the improved performance.

Personally, I am of the opinion that one shouldn't forego a wealth of features (such as C++'s performance) in favor of slight simplicity (not having to worry about pointers - just don't mention Lua tables - and maybe simpler syntax). If you make anything remotely complicated, you still need to learn to deal with Lua and the unexpected or tricky parts of its design. You also still need the right mindset to wrap your head around whatever you're programming, which is also not trivial if you have very little previous programming knowledge. As a result, it's just not really "beginner friendly" unless you're making something very simple.

Combined, from the view point of a modder, I don't see much of a reason to choose Lua. You could deal with the complexity of C, but you cannot deal with the performance limitations of Lua. There is a very hard line beyond which you can no longer do anything at all to improve its performance.

Eh, personally, as a hobby coder/modder, I perfer languages like Lua or VB.NET. I honestly have a harder time with C# and C/C++. I would probably not mod for this game if I had to deal with those languages.

Further, I like the fact that I can download and open another mod and be able to look directly at their code to see how something was done. Not everyone includes or uploads source code somewhere when the mod is compiled... myself included.

Whether or not Lua was the best choice for this, I don't know. But I do feel as though using C/C++ for the modding language would have been wrong.

I'm shocked that Factorio wasn't built in C for performance reasons.

That being said, this Lua stuff doesn't seem to be too bad.

NineNine wrote: Mon Oct 23, 2023 4:22 pm I'm shocked that Factorio wasn't built in C for performance reasons.

It was, or rather C++. Lua is for modding.

NineNine wrote: Mon Oct 23, 2023 4:22 pm I'm shocked that Factorio wasn't built in C for performance reasons.

It's written in C++. Only the modding API uses Lua.

MEOWMI wrote: Sun Oct 22, 2023 4:10 pm I really can't fault Wube for choosing Lua, [...].

Same.
Out of the languages I'm aware of only Lua has the ability to easily restrict and replace parts of it's standard library, both are important to factorio. For example, factorio doesn't allow arbitrary file access, mods can only write into (but not read from) one specific directory. This serves as a kind of sandboxing, limiting what a bad faith modder can do to other players systems, but is also needed due to factorio's deterministic engine (What would/should happen if two players read different contents?). factorio's modding API documentation has more examples.
Due to being an interpreted language mods are distributed as source, allowing us to inspect other mods as FuryoftheStars already mentioned. Also, distribution as binary is infeasible as it has to run exactly the same on Windows, Linux and Mac. (JVM and C#/.NET exist, but they still have an interpreter / JIT compiler.)
On the other hand, an interpreted language with a rather small and simple interpreter looses the benefits generally associated with an optimizing compiler.

Personally, I am of the opinion that one shouldn't forego a wealth of features (such as C++'s performance) in favor of slight simplicity (not having to worry about pointers - just don't mention Lua tables - and maybe simpler syntax). If you make anything remotely complicated, you still need to learn to deal with Lua and the unexpected or tricky parts of its design. You also still need the right mindset to wrap your head around whatever you're programming, which is also not trivial if you have very little previous programming knowledge. As a result, it's just not really "beginner friendly" unless you're making something very simple.

Combined, from the view point of a modder, I don't see much of a reason to choose Lua. You could deal with the complexity of C, but you cannot deal with the performance limitations of Lua. There is a very hard line beyond which you can no longer do anything at all to improve its performance.

While In haven't seriously messed with rust yet, my preferences steadily shift towards languages outright preventing certain kinds of errors and making use of the optimization opportunities gained from these restrictions. I'd love to have a C++ modding API for factorio, but due to the sandboxing, determinism and portability aspects mentioned above, I don't consider it viable.

Ideally, there'd be a language with a large but easily restrictable and replaceable standard library, a rust-like type and lifetime system (or better) and an optimizing compiler that's fast enough to be used in a REPL environment while still having performance comparable to what today's C++ and rust compilers can do. But I'm getting into wishful thinking territory here…