[Suggestion] Mod manager optimisation

[mcgiwer]

I would like to propose some optimisations for the Mod manager:

• any changes in the mods should not require restarting of Factorio
• when installing a mod, all required dependancies should become automatically installed as well
• an mod with satisfied requirments from with are other mods dependant should become enabled automatically
• when any changes in the mods become done, the window should become refreshed automatically
• optimise it for speed to make it without any unnessecary delays

[Silari]

I would like to propose some optimisations for the Mod manager:

• any changes in the mods should not require restarting of Factorio
• when installing a mod, all required dependancies should become automatically installed as well
• an mod with satisfied requirments from with are other mods dependant should become enabled automatically
• when any changes in the mods become done, the window should become refreshed automatically
• optimise it for speed to make it without any unnessecary delays

The first isn't possible. The game has to apply all the changes the mod made, and they need to be done in a certain order to ensure that mods can overwrite others properly, and all the mods need to run again so they can account for the presence of the new mod(s) and make changes if needed. The game can't know how mods interact with each other, and the fastest way to find out is to just restart and let them run.

Second should already be done when installing new mods from the mod portal. There might be some exceptions when a mod update adds a new dependency, and possibly the Sync mods with save button doesn't grab pre-reqs if it isn't in the list for the save (again, because it was added).

Not sure what you mean by the third. Sounds like you want the game to automatically enable any mod which has all it's dependencies installed, which sounds really annoying to have it auto-enable a bunch of mods all the time. If you're just asking for it to enable dependencies when a mod is enabled, it does that. I don't think it'll INSTALL missing dependencies when enabling a mod though.

Gonna guess the reason the fourth isn't done is because they'd have to relist all the mods in the directory, and there's little point when changes require the game to restart, where it'll update the list anyway.

As to the last, mod loading is already VASTLY optimized from what it used to be. I doubt any delays left are unnecessary, though some might possibly be done faster. There are some hidden options that can make load faster by caching sprite atlases and such, but that uses a bunch of space so isn't the default.

[ssilk]

I wouldn’t say impossible. It’s hard to implement it, and the gameplay value is questionable, but not impossible to implement.

To explain the problem a bit: the game needs a “configuration”, before it can jump into the main-loop. The configuration is created by the mods. Even if no mod is installed, there is the core-game mod for example. The game runs the configuration parts of the mods in some order, each one writes parts of the configuration, or OVERWRITES previous parts. The order of this process is very important. (I left many things away to reduce the information.)

Which means, that if you change something with the mods, the configuration might change. The game cannot know, what changes, until it goes again through this process. And because that could mean, that the graphics change, it means that it is a good idea, to reload all the graphics, too.

So there is this question as developer: why should I write a very complex config-reload-routine, when we already have that routine? My new routine will be slightly faster, maybe 3 or 4 seconds, but we need much more internal memory and restarting the game is very seldom, compared to the time players are in the game, non-restarting. And it is a source of possible errors, we need to maintain that piece of software.

The decision is clear: possible, but very, very unlikely to be implemented as explained here, because the game-value is not increased (just by some seconds for the reload).


The remaining things are useful and I would add, that if the dependencies to a lib are removed (count the number of dependencies for a mod), it shows somehow, that the lib is also removeable. I would implement that so, that if I automatically install other mods, it gets the state of a “dependency”. And if you want to delete such a dependent mod if the dependencies still exists it warns the player.

[mcgiwer]

First, thanks for the answers

At the begining, I would like to refer myself to the post of Silari:

Not sure what you mean by the third. Sounds like you want the game to automatically enable any mod which has all it's dependencies installed, which sounds really annoying to have it auto-enable a bunch of mods all the time. If you're just asking for it to enable dependencies when a mod is enabled, it does that. I don't think it'll INSTALL missing dependencies when enabling a mod though.

Let me give an example (typing as list for better visiblity):

• mod A is already installed, but it's disabled
• the user want to install an mod B with is dependant from the mod A, but because mod A is disabled, the dependenciens of mod B are marked as not fulfilled, even if mod A is already installed
• while installing mod B, the already installed mod A becomes enabled

As to the last, mod loading is already VASTLY optimized from what it used to be. I doubt any delays left are unnecessary, though some might possibly be done faster. There are some hidden options that can make load faster by caching sprite atlases and such, but that uses a bunch of space so isn't the default.

I'm not sure what have you used as base of your comparation, but please take into attention that:

• the version of Windows and Linux version of Factorio work bit different on each of both mentioned operating systems (mainly from the technical side )
• In some countries of the East Europe (because of the occupation in the past and later political situation) most people in the East Europe aren't able to afford themself very good (in comparation to for example US) computers, plus becuse of above, the technological progress of many countries was delayed for many years

Besides of above, how would you explain the lag's while using the mod's manager when nothing except the "core" and mod manager is enabled ?

And finally, refering myself to the message from ssilk:

I think that a dynamical update of the config files could solved the described by you issue

[Silari]

• mod A is already installed, but it's disabled
• the user want to install an mod B with is dependant from the mod A, but because mod A is disabled, the dependenciens of mod B are marked as not fulfilled, even if mod A is already installed
• while installing mod B, the already installed mod A becomes enabled

Yeah that might be one of those exceptions that they missed. If you install a mod, it'll install not installed dependencies. If you enable an installed mod, it'll enable any installed dependencies. Mixing installing/enabling doesn't seem to be caught, in either direction. There might be a bug report about that, but i could be misremembering.


For the last point I thought you were talking about how fast the game loads mods during startup. If the mod manager page is loading slow, that's probably more a function of your internet connection speed than anything in the game. From what I recall of looking through logs the first thing the game does when the page is loaded is query the mod portal to get info on the installed mods, and makes a couple more when you go to the Install tab. If the connection is slow that could take a bit and might be why it. The game could try and cache that info across loads but that leads to problems with stale data. Again, I feel like I saw a bug report about that.


The only way to tell what files you need to reload would be to examine every line of every file to see if it is making a change based on some check and if that check could possibly be effected by ANY of the other enabled mods. It would then need to see if any other mod changed that same thing at a later time in the load order. The time and complexity to do that (essentially writing a LUA parser to catch every possible way a mod can rely on another mod or its changes) would end up being slower than just reloading everything anyway, assuming you could even manage to find every instance in the first place. You'd also have to figure out what portions of their code would need to be rerun, as you can't just run the whole lua file as it might overwrite a change overwritten by a mod that wasn't just installed. And then you need to figure out what requires you need to run the portion of code you need. Except that even a simple require command can run code that could change something in the data, which could overwrite another mods changes.

With how mods work in the game, it is impossible to do it in any way that is faster than simply reloading them all.

[ssilk]

Silari already explained it, but I want to point it out a bit more.

A) Dependencies
They can be calculated by the so called mods.json. It links just to other mods. And as it can already be seen, this is used to check, if a mod has dependencies. It would be easy, to reload this files (and recalculate the link-graph from it), when you change this. But I think this is not done, because it would mean to have two dependency graphs in memory: One of the truly currently loaded mods, and one of the wanted, after it has been reloaded all mods.

It looks, for me, that the code, that handles currently the mods is either extremly complex (but I would wonder why, because some years ago I already wrote this in lua) or - I guess - just not modular enough to use only the part which loads the mods.json and rerenders the internal dependency graph. In any case the code would need some love, and - also said - it doesn't pay back much, as long as the game is, as it currently is.

B) Dynamic Updates
I think I already explained it a bit above. To make this, we need two instances of the mods-handler, one for the current state and one for the wanted. And every time the wanted version changes, it needs to recalculate the whole dependency-graph. This is no problem with 10 or 20 mods, but it might become a problem with more. Me have for example over 100 mods running (hehe, more or less stable). It would be quite ugly to wait some seconds after every click in the mod-handler, until the new depencies are calculated. To avoid this it needs to run this in an own thread. This soon gets quite complex. As developer I would try to avoid this as long as possible, because the outcome is low.

Low, until the game (more complex mods handling; for example paid mods) will change significantly.

[mcgiwer]

For the last point I thought you were talking about how fast the game loads mods during startup. If the mod manager page is loading slow, that's probably more a function of your internet connection speed than anything in the game. From what I recall of looking through logs the first thing the game does when the page is loaded is query the mod portal to get info on the installed mods, and makes a couple more when you go to the Install tab. If the connection is slow that could take a bit and might be why it. The game could try and cache that info across loads but that leads to problems with stale data. Again, I feel like I saw a bug report about that.

it's not exactly what I had meant

I try to be more descriptive:

After the mod manager is fully loaded and the list of mods loaded, it's like it's getting "lags" with self moving of the mouse.

My computer fulfill the recomended system requirments (disk, memory, etc.) and such strange things still happen

[ssilk]

Can you make a video of that and show us, please?