REQUEST: Improving the Modding API and its Documentation?

[andrfgs]

First of all, Factorio's is an almost perfect game. A big thanks to all the developers that helped make this game what it is today. The game is indeed very fun and I can sink lot's of hours on it. But... There's just one thing that's very bad in the game: the modding API which is extremely limited and its documentation.

Could the devs responsible for the documentation please improve it? And also make sure the wiki stays up to date with the API please? I know the game ships the docs but having an up to date wiki would be nice. But the real crucial thing, is to explain the API better. I'd love to contribute If I could, but I also don't know the API.

As it stands, the API is very hard to understand in some parts. For example, I'm trying to develop a procedural generation mod and I'm having a lot of trouble understanding what each thing is.

For example: https://wiki.factorio.com/Types/AutoplaceSpecification, in the Dimensions tab, what is the range of elevation? I know it's a value up to 5000, but what about the lower bound? Is it -5000? This is an example of lack of documentation. And also, I'd like to know what value is it? A float? A short? An int?


Some parts of the API are confusing or have no documentation. It's very hard to use this as it is. If I hadn't found this tutorial: https://togos.github.io/togos-example-noise-programs/, I wouldn't have been able to guess how to do any of this. I understand this sounds like a rant, but the only thing I really want it to make a request for the devs to improve the modding API. And if there's something that us can do to help doing that, say it as well.

[Therenas]

The wiki is up to date on pretty much everything most of the time, they have a dedicated employee looking after it as their job. Also, a general 'make it better pls' is not helpful to anyone. If you could suggest parts that are not very good in your opinion, people could work on them. 'Just make everything better' is not a goal that can be worked towards.

The part you're trying to learn about is indeed a spot where the documentation is not great. The person that implemented it does not work at Wube anymore, and Bilka and various modders are trying to piece it back together at the moment. It's also very complicated, which doesn't help. Either way, it's likely that that part of the docs will improve in the future.

[andrfgs]

I understand, I would like, if possible, variables to be more detailed, such as type, range, a detailed explanation on how it changes what is supposed to change, things like that. Sorry if my feedback wasn't detailed enough.

A good API documentation (IMO), usually has every single possible field documented and explained thoroughly and it also has an usage example for every single method/field. In the above example, It's not even clear how things in the map settings link to any of the API. Why is it that in order to create another map type option I must extend data and introduce:

Code: Select all

type = "noise-expression",
name = "name",
intended_property = "elevation"

First, how would I even know where to browse? There would need to be a page explaining how everything works under the hood (not a documentation per say, but the introductory part). In short, that which I found at https://togos.github.io/togos-example-noise-programs/ should be on the modding documentation, as introductory parts explaining how the process works are equally important. And also, there's a good amount of examples.

I say Factorio's modding API is limited because it seems as you are forced to follow a very careful and limited path that the devs thought it should be needed. It's not at all how many mods are born. Usually memorable mods go beyond what the API was expecting them to do. In Factorio, if you try to do something much much different, you'll soon get into limitations.

Could I additionally ask the devs why don't they introduce a C/C++ modding API? You don't need to expose the full code of the game, only the parts you intend through header files. And then, at runtime, use something like symlink to load the mod. The concerns about this approach would probably be of security and of platform compatibility though. But the short thing is, don't limit the API because you're afraid players will crash your game, let them crash it thousands of times and learn from their mistakes, instead of making something much more restricted.

[Therenas]

Thanks for clarifying what you think could be better. There is always room for improvement for sure.

Regarding the modding API philosophy itself, I don't think you'll find much support from the devs for a C++ modding API. I get that having the ultimate flexibility would be great, and would lead to some very different mods that would expand the scope of what we thought is possible.

But, on the other side of the coin, allowing that kind of access has a ton of disadvantages. First of all, Factorio prides itself on being very stable and bug-free. Having a limited, well-defined lua API allows them to avoid many bugs and crashes that just giving full access would allow for. It also makes compatibility way harder, for obvious reasons. Both these things would shake users trust in mods, or make it very annoying and work-intensive to compile a list of mods that work with each other. See other games such as Skyrim, where people often joke that they spend more time on getting a heavily modded game to load than actually playing. That is not a good experience imo.

Secondly, and even more importantly in my mind, is that this kind of access would break mods all the time, leading to massive work for developers to just keep their mods up to date. In the long run, this leads to fewer and lower-quality/less complex mods, just because so much time is wasted on updating the basics. The best example for this is Minecraft, where so many great mods are stuck on years old versions of the game, because updating them would be too much work. This fragments the community, and many of the advancements that the new official versions bring won't reach everyone, which is sad because of all the effort the developers put in.

All in all, most mods 'allow' this kind of deep access because it's no effort for the developers, people just hack stuff into the game. It has many disadvantages, and a proper, if limited, API just leads to higher-quality mods in general, which I think is just better. Also, the devs spend quite a lot of their time on stuff that is only relevant to modders, be it features, performance or bug fixes. If there's anything you want from the API, and it's a reasonable idea, the chances are good that the devs will provide them at some point.

Anyways, that's my 2 cents, got kinda long, but I hope you understand the philosophy behind it, at least the way I see it.

[andrfgs]

I understand the philosophy, but I have to highly disagree with many of those points. Minecraft's Forge is, in my opinion the best modding API ever created. It was a product of people working tirelessly over the years to support something that Mojang always promised to support but never did. Forge is actually an example of how mod integration can and will work when there's a will to do it. Look at the ore dictionary, for example. I remembered when it first came around. The ore dictionary made sure many mods that created new ores could integrate easier with each other. The modding api makers saw the problem and created a solution for the problem. The only reason modded Minecraft is so heavy is due to Java's VM overhead, the fact that Minecraft itself was never very optimised (the game used to send updates to every single block, even though most didn't need that) and also the fact they use reflection every time you launch the game. Forge's modding API should be regarded as the golden standard, IMO. If you don't agree, I'd love to know more about why, though.

Minecraft mods became outdated because not only wouldn't Mojang support Forge in any way, they'd change the entire API so much, most mods had to be rewritten. But I can tell you I never had any serious bug with many Minecraft mods. And I used to use 120+ of the technical mods all together. When you have 120+ mods that work so well together when the game they run on was never meant to do that, then that means the modding API is really, really good.

I feel bad to say but, Minecraft mods can create a whole new game. Factorio mods not so much. (not saying they're any bad though) And about the efficiency, I highly disagree with that. Because of the current Factorio's procedural generation API implementation, I am forced to use expressions and cannot efficiently use data structures. I wanted to generate continents and islands at hand given coordinates, which could be optimised if I had a data structure where I could simply lookup which expressions were relevant for (x,y). And for those that weren't, I could prune them, Instead, I must use an entire expression all the time. Just an example on how something that may be the best on one scenario might not be so on many others. In general, I highly doubt Lua mods will be more efficient than C++ mods, and besides, I can still try to break the game if I want though. Or make my mod behave in a bad way. I don't agree with this Sandbox everything and limit all developers though. If that was a concern, simply use digital signatures on the mods. You could have a chain of trust by the developers themselves, moderators or even trusted and well known mod makes, which could emit master keys to other moders or could sign them with theirs if the mod was trustworthy.

But I'll sadly accept that this is how the devs intend to make the modding API be. It is a shame, but I guess I'll have to respect the decision. Thanks for that feedback and the time you took to reply! I really have to agree with Factorio being extremely bug free though. I have never found a bug on the game since 0.16, maybe I'm bad at that idk

[Bilka]

Could the devs responsible for the documentation please improve it? And also make sure the wiki stays up to date with the API please? I know the game ships the docs but having an up to date wiki would be nice. But the real crucial thing, is to explain the API better. I'd love to contribute If I could, but I also don't know the API.

I understand, I would like, if possible, variables to be more detailed, such as type, range, a detailed explanation on how it changes what is supposed to change, things like that. Sorry if my feedback wasn't detailed enough.

The wiki is complete and updated. Descriptions for properties are WIP as they are a huge undertaking. If there are specific properties that you need descriptions for, please let me know so I can prioritize them. (This also goes to anyone else reading this thread.)

For example: https://wiki.factorio.com/Types/AutoplaceSpecification, in the Dimensions tab, what is the range of elevation? I know it's a value up to 5000, but what about the lower bound? Is it -5000? This is an example of lack of documentation. And also, I'd like to know what value is it? A float? A short? An int?

As Therenas says, the person who wrote/modified that code is no longer at Wube. So, this is the only page in the entire documentation where I sadly have to say "I don't know and I can't realistically find out"; this is why this page has the note about its textual components potentially being outdated. I want to stress again that this is literally the only thing in the entire API where this is the case. (I did find out the data type though, it's a double.)

Some parts of the API are confusing or have no documentation. It's very hard to use this as it is. If I hadn't found this tutorial: https://togos.github.io/togos-example-noise-programs/, I wouldn't have been able to guess how to do any of this. I understand this sounds like a rant, but the only thing I really want it to make a request for the devs to improve the modding API. And if there's something that us can do to help doing that, say it as well.

The tutorial is about noise expressions, which are part of the autoplace specification but completely unrelated to your question about the dimensions tab. So, the note on the autoplace page doesn't apply to this at all.
The majority of noise expression related things are documented on this page: https://wiki.factorio.com/Types/NoiseExpression. That page also links the tutorial. So, there should be no issue with the noise expression documentation: It exists, is complete, is updated, has examples and even has a tutorial.

A good API documentation (IMO), usually has every single possible field documented

This is the case already.

and explained thoroughly and it also has an usage example for every single method/field.

As said above, that's WIP.

In the above example, It's not even clear how things in the map settings link to any of the API. Why is it that in order to create another map type option I must extend data and introduce:

Code: Select all

type = "noise-expression",
name = "name",
intended_property = "elevation"

That is explained here: Prototype/NamedNoiseExpression

First, how would I even know where to browse? There would need to be a page explaining how everything works under the hood (not a documentation per say, but the introductory part). In short, that which I found at https://togos.github.io/togos-example-noise-programs/ should be on the modding documentation, as introductory parts explaining how the process works are equally important. And also, there's a good amount of examples.

That tutorial is part of the modding documentation, it's linked from there: https://wiki.factorio.com/Tutorial:Modding_tutorial
You know where to browse by starting from one of the modding landing pages: https://wiki.factorio.com/Tutorial:Modding_tutorial or https://wiki.factorio.com/Modding or 60680 or even https://lua-api.factorio.com/latest/. Further guidance for using the documentation is provided in https://wiki.factorio.com/Tutorial:Modd ... al/Gangsir.

[andrfgs]

Thanks for the reply. I have seen all of those URLs, maybe I may have missed something, but as I understood, the NoiseExpression that I want to control, in order to customise the procedural generation, is the one that modifies the property elevation. I had no idea that repository was related with the modding documentation. My bad then.

Some questions:

• What is the difference between a NoiseExpression and a NamedNoiseExpression? I read https://wiki.factorio.com/Prototype/Nam ... Expression multiple times and I still don't get the difference. Is the NoiseExpression meant to be the abstract type and the Named thing meant to be the actual implementation?
• On that same site, what are all possible intended_property's? Is there a list of all of them? And all possible values for each of them? Are the possible values "elevation, temperature, moisture, aux, and cliffiness" by that exact name? And what does each of them do? Is this explained somewhere? I realise some of these might be obvious but I'd very much appreciate if they were listed by all possible values. But perhaps that is the part you say it's a WIP?
• You say that the AutoplaceSpecification is unrelated to the NoiseExpression but I see the same property on both of them? What is going on? Maybe I'm really not getting the picture here. I have already discovered, through experimentation, that elevation controls the terrain height, but it seems negative values (as stated in the wiki) do not necessarily mean is underwater, only that is below sea level. I'm assuming the water property regulates this then?
• Why is it that a NoiseExpression of property elevation creates a new entry for Map Type? NoiseExpression and elevation seem to be, at a fist glance, unrelated to MapType, so why do they create a new entry automatically? Do all NoiseExpression's do this? Or is it just one that changes the elevation property?

[Bilka]

I hope you don't mind the quote-style question answering.

What is the difference between a NoiseExpression and a NamedNoiseExpression? I read https://wiki.factorio.com/Prototype/Nam ... Expression multiple times and I still don't get the difference. Is the NoiseExpression meant to be the abstract type and the Named thing meant to be the actual implementation?

Your last sentence nearly hits the mark. The Types/NoiseExpression is just some lua table you defined. By sticking it in a NamedNoiseExpression, you can tell the game to actually use it: via a map gen preset, runtime map gen settings, the intended_property dropdown or a noise variable (noise.var("my-named-expression")). You are essentially assigning it to a variable that the noise system knows and then you can use that variable inside the noise system. Another way to tell the game to use a Types/NoiseExpression is to supply it in a AutoplaceSpecification (which is used by entities). These uses were only listed as "uses" on those pages, now they are also listed as "used by" on Types/NoiseExpression and Prototype/NamedNoiseExpression.

On that same site, what are all possible intended_property's? Is there a list of all of them? And all possible values for each of them? Are the possible values "elevation, temperature, moisture, aux, and cliffiness" by that exact name? And what does each of them do? Is this explained somewhere? I realise some of these might be obvious but I'd very much appreciate if they were listed by all possible values. But perhaps that is the part you say it's a WIP?

Anything is possible, even "nice-delicious-pizza", an example for that and a usecase is now on Prototype/NamedNoiseExpression. I listed the intended properties the base game uses on the same page and linked Types/NoiseExpression#variable for what they do in base game. However, as stated on both on those pages, that's just what the base game does. Any mod could do whatever they to them, like change what they do (by changing how they are used in the other noise expressions), change their types or delete them completely. For example, afaik https://mods.factorio.com/mod/fractured-world 1.0 uses "aux" for ore spawning, which is completely different from the base game's usage.

You say that the AutoplaceSpecification is unrelated to the NoiseExpression but I see the same property on both of them? What is going on? Maybe I'm really not getting the picture here. I have already discovered, through experimentation, that elevation controls the terrain height, but it seems negative values (as stated in the wiki) do not necessarily mean is underwater, only that is below sea level. I'm assuming the water property regulates this then?

What properties do you mean? The "Dimensions" that were discussed earlier in this thread are part of the peak-based autoplace system. You aren't going to be interacting with that if you use noise expression based autoplacement. Tiles use the noise expression based system in vanilla, afaik only decoratives and trees use the peak-based system at this point.

There may be some relation, for example the "dimensions" might be using the values supplied by noise expressions. However that again hits the "I don't know and I can't reasonably find out" barrier of the peak-based autoplacement.

Why is it that a NoiseExpression of property elevation creates a new entry for Map Type? NoiseExpression and elevation seem to be, at a fist glance, unrelated to MapType, so why do they create a new entry automatically? Do all NoiseExpression's do this? Or is it just one that changes the elevation property?

Map type is just the "nice" name for the elevation generator. It's renamed if you add any other dropdowns to the GUI. I'll quote the lua api docs here:

The 'map type' dropdown in the map generation GUI is actually a selector for elevation generator. The base game sets property_expression_names.elevation to "0_16-elevation" to reproduce terrain from 0.16 or to "0_17-island" for the island preset. If generators are available for other properties, the 'map type' dropdown in the GUI will be renamed to 'elevation' and shown along with selectors for the other selectable properties.

I have now also made this more clear on the wiki.