Integrating Mod packs into the Mod portal

[Sanqui]

Hi,

we are thinking about adding a mod pack feature to the Mod portal and subsequently the game.

In essence, a mod pack would be a collection of mods and specific versions with a description, and anybody could make one. The game would allow for downloading a modpack, which would download all mods that are a part of it, and enabling a modpack would enable all the corresponding mods.

There is no guarantee this will happen in a timely manner (or at all) but I would like to hear your thoughts on how it should work or any other stuff we should keep in mind.

[_npo6ka_]

Unfortunately, there are a large number of good but not completed mods. I have already created my mod pack and I know how difficult it is to correct errors in other mods. Leaving the mods unchanged also does not work out, some mods can conflict with each other to some extent, but I would like to see both of these mods in the mod pack. Of course, I created my mod specifically for modpack, but not all errors can be corrected by writing a separate mod.

[Ipsumlljk]

I actually just registered a forums account to reply to this. I totally agree with the previous post that creating a mod pack is always a bit harder than simply grouping packages. I've maintained a fairly small mod pack over 3 major versions now and every version bump has required editing multiple mods to make them all compatible ( especially when using biome changing mods ). I agree that these changes should simply be merged into each mod, but that usually takes a long time.

My other piece of feedback would be to note that generally multiplayer mod packs are tightly coupled with a specific server. Most of the complication in maintaining a mod pack is keeping the server and all the clients in sync with changes. It would be really nice if the client was smart enough to fetch/update the modpack when attempting to connect to the server.

My current process stores all of the mods on github and requires each player to run a git pull in their mods folder when I update the mod pack. It isn't the worst system, but certainly not the most convenient either. It also requires them to be tracking multiple remote git repos if they use multiple mod packs.

It would be really nice if the process of getting the latest version of a modpack was easy for those of us running a command line only headless server as well.

Thanks for all of the great work so far on the mod portal!

[eradicator]

Kinda been procrastinating answering this because i realised that even though i want modpack support, the moment i start to think about the details i realized it's pretty complicated :D. So this is somewhat of a brainstorming monologue which hopefuly is useful ;).

In minecraft modpacks work like this:
The user downloads a "pack" which contains some mods of a specific version that are tested to be compatible to each other, including a config file to make some mods play nice together. After downloading the pack nothing is enforced. The user can en/disable any mods as they see fit, they can also update any mods or even add new mods to the pack. In factorio terms it's really just a glorified zipped up mods folder.

Is that enough for factorio? I don't know.
Being able to automatically download all dependencies listed in a mod would be a great first step towards a simplistic mod "pack" that would work very similar to this. A GUI for "--mod-directory" would also be a pretty good approximation. Also i'd really like to preserve the possibility to install multiple "packs" in parralel though, e.g. having a "bob pack" and an "angel pack" that can be de/enabled independantly. In the minecraft system you can only have one active pack. Ofc packs might be incompatible to each other so multi-pack support might be a step too far.

But as already mentioned: What if two mods are incompatible and need to be fixed?
For this it would be nice to have something i'd like to call a "first level recursive mod", i.e. a mod that is allowed to contain other mods but only one level of depth is allowed. This would allow a mod "pack" to be a zip that has a directory for each contained mod. They would still be loaded as seperate mods normally. As the mods are directly inside the pack it's possible to apply fixes to them directly without needing to try and fix them from the final-fixes scope of another mod. As far as i remember the 0.17 GUI already supports multiple versions per mod, so this would be simple. The "pack" shows one entry for the pack. Enabling the "pack"-mod forces the use of the pack-internal version for all mods, even if there are stand-alone versions of those mods installed. When the pack is "disabled" the player can still select the pack-internal version of each mod, or use the newest one from the portal in case the pack is partially outdated. This poses one problem though: There is now a scenario where the same mod has several installed instances, all with the same version number, e.g. "Example_Mod_A" could have "1.0.0 (stand-alone)", "1.0.0 (Super Mod Pack)", "1.0.0. (Duper Mod Pack)", etc.

It could even be implemented in a way that "packs" contain only the fixed files, instead of the whole mod. I.e. if a pack contains "modA_1.0.0.\prototypes\newthing.lua" then when loading modA all files are taken from modA as normal except for newthing.lua which is instead loaded from the pack. But as you still need to keep compatibility with "full mod is contained in pack" you suddenly have two cases for recursive mods which is probably not worth the effort.

As this opens the possibility for a mod to be contained only in a pack and not be available seperately the mod portal needs the ability to find and install the pack when searching for containd mod names. Also if a pack contains a "fixed" version of a mod whichs license does not allow third-party distribution you might have to deal with some take-down requests (not that this is a completely new problem).

Ofc it would be awesome if the two methods could be combined. Thus a pack contains a) a list of unchanged mods that can be used directly from the portal and b) a folder for each mod that needs pack-specific fixes. This would keep the pack-size down and prevent unnessecary duplicates for mods that don't need pack-internal fixes.

Why is data-final-fixes not as good? The problem is that some mods might be bugged and thus crash if they're loaded after certain other mods, which is not fixable with a singular instance of final-fixes at the end. It would require to allow the mod "pack" to execute code each time another mod is done. I.e. Mod1_data -> modpack_post_mod1_data -> Mod2_data -> modpack_post_mod2_data -> Mod3_data -> modpack_post_mod3_data. This would be an interesting mechanism for non-mod-packs to have too, and could be implemented via a global variable previous_mod and more stages like data-post, data-updates-post, data-final-fixes-post which are called after every mod. But i doubt that's going to happen. And even that wouldn't solve the problem of control stage incompatibilities. For control stage fixes editing mods is mandatory. Ofc now you say "but that's all bugs in the mods that should be fixed in the mods!". Yes, ofc. In an ideal world. But realistically there's going to be mods that aren't updated in time, so the pack maintainer must fix them themselfs.

Also the most important feature required for mod-pack support is the introduction of "presets" for mod settings. I.e. the ability to save/store/edit distinct configurations of mod settings for all mods. Currently mod settings are horribly volatile. Every time you join a server with slightly different settings all your own personal settings are lost. Every time you remove/re-add a mod all settings are lost. Resetting settings to default is only possible for all mods at once, i.e. if i messed up the setttings for mod_31 and don't remember the default values i have to reset and manually enter the changes for mod_01 to mod_30 all over again. A better system would be to have a "save preset" button. Also joining a server with different settings should generate a new preset, either automatically named or with a dialogue "What do you want to save this servers settings as?" with a default name of "ServerName (Date)".

[Qon]

In essence, a mod pack would be a collection of mods and specific versions with a description.

How is this different from a mod with dependencies? Except for the addition of a new tag "mod pack" and for the GUI to show not only dependencies that an installed mod depends on but also show what mods depend on it that you have installed. Would be nice if (for all mods) mods installed as dependencies could be "hidden"/grouped under the mod that depend on them, a bit like a folder structure maybe?

and anybody could make one

Just like anyone can make a mod (freedom but knowledge requirement) or just like anyone can install a mod (easy GUI lets anyone do it)? For the second I'm guessing a modpack feature would make an empty mod that only has dependencies chosen on the portal and the actual mod files are generated on the server?

The game would allow for downloading a modpack, which would download all mods that are a part of it,

Wasn't this a feature for mods with non-optional dependencies when your message was posted? The feature to download optional dependencies is a bit wonky still though. I click the dependency arrow button and I'm only sometimes shown the mod it is dependent on so I can install it and there's no button to install all of the dependencies (including optional ones).

and enabling a modpack would enable all the corresponding mods.

Ok so this is slightly more than what the portal does, but it seems like a good feature for all mods (an added button to set all dependencies to the same enabled/disabled state).

I totally agree with the previous post that creating a mod pack is always a bit harder than simply grouping packages.

Always? Did you mean to use hyperbole or is that literally? Because that's pretty much all there is to it in almost all cases. If you group mods that are very similar and share a feature set an error from incompabilties are more likely (and many packs would indeed pack several similar mods like this) but it is still fairly rare that mods don't work together from my experience of installing hundreds of mods. Any errors that do appear are mostly just regular bugs in the mods that don't or just rarely appear when the mod is installed in isolation. But mods are sometimes buggy, that's something we report to the mod authors as bugs. I think true incompabilities (that are not bugs) are more rare than bugs.

My other piece of feedback would be to note that generally multiplayer mod packs are tightly coupled with a specific server. Most of the complication in maintaining a mod pack is keeping the server and all the clients in sync with changes. It would be really nice if the client was smart enough to fetch/update the modpack when attempting to connect to the server.

Is this different from how any other mod is kept in sync with a server?

[mrudat]

I would be inclined to suggest that a mod pack that patches required mods should be packaged as a set of diffs, rather than replacement files; the idea being that the mod pack should fix incompatibility, which is unlikely to change between versions, and hopefully updating to support newer versions should just involve editing the dependencies.

[Qon]

I believe that bugs in mods should be fixed by the authours of the mods. It is not mod pack authors job to fix bugs in their dependencies.

I would be inclined to suggest that a mod pack that patches required mods should be packaged as a set of diffs, rather than replacement files; the idea being that the mod pack should fix incompatibility, which is unlikely to change between versions, and hopefully updating to support newer versions should just involve editing the dependencies.

Why are you arguing for diffs vs replacement files when a mod can modify the prototypes of its dependencies? What do you think a diff accomplishes that a mod already can't and why should it be doable?

[mrudat]

As far as I can tell, the only reason you should need to edit one of the original mods is if it is so incompatible with one of the other mods in the mod pack, that it causes a crash before the modpack gets to edit the prototypes, or you lose information along the way.

The biggest thing using diffs buys is that it captures the edits, and hopefully allows the same edits to be applied to a newer version of one of the original mods.

Perhaps instead have the modpack replace the data phase for component mods, and explicitly apply the data phase yourself, so you can inject patch steps where you want, something like:

Code: Select all

require('__mod1__/data.lua')
require('patches/patch-mod1.lua')
require('__mod2__/data.lua')

[Qon]

As far as I can tell, the only reason you should need to edit one of the original mods is if it is so incompatible with one of the other mods in the mod pack, that it causes a crash before the modpack gets to edit the prototypes, or you lose information along the way.

I agree.

Perhaps instead have the modpack replace the data phase for component mods, and explicitly apply the data phase yourself, so you can inject patch steps where you want

Handling data.lua loading of dependencies as an added capability to mods seems like a much more reasonable and stable way of handling incompabilities such as name collisions etc. Maybe as an additional flag on dependencies a bit like optional/mandatory flag for dependencies. A "suppress data stage" flag on a dependency.

But what if a suppressed-data-dependency has a control.lua that expects the prototypes to be named a certain way and then they don't exist because a modpack renamed the prototype? I'm assuming the name collisions of prototypes are the main causes of incompability that you have experienced. Maybe that isn't so? I haven't experienced anything like this ever though so I don't know.

[mrudat]

I'm speaking purely in the theoretical, I've never done anything involving mod packs, but I've played around with producing Debian packages before which is a somewhat related activity, in that the upstream program may have bugs and/or incompatibilities you need to address, but you don't want to have to hand-edit things every time the original updates.

On the subject of control.lua, that's something that probably does need to be (effectively) edited in-place, because unlike the data phase, you can't (easily?) override its behaviour short of editing the file. I suspect that you could possibly do some magic with reflection modifying the compiled code, but that's probably a bit too magical for the average programmer.

I suppose you could have replacement control.lua that does something like:

Code: Select all

control_code=some_function_that_reads_the_file('__some_mod__/control.lua')
control_code:replace('amazing-widget','awesome-widget');
eval(control_code)

...but that's effectively an ad-hoc implementation of patch, and unless it just can't do what you need, it's generally less painful to use the existing specialised tool, rather than inventing your own.


A Debian source package of the simplest form involves unpacking the original file, and applying a patch containing the modifications you made; when a new upstream version arrives, the first attempt is to apply the patches and see what happens.

A somewhat niftier version has a set of patch files, each addressing a different reason for patching, for example:

• fix bug with package 1
• make package 1 and package 2 play nice
• adjust balance of package 3 due to also having package 1+2

When package 1 fixes the bug, you can simply choose not to apply that patch.

Equally, if package 3 includes the rebalance due to detecting package 1+2, you no longer need that patch, either.

Also, if any one patch fails because the upstream version has changed too much, only that patch needs to be manually applied, rather than having to edit the replaced files every time the upstream version changes.

Edit: fixed tyop

[steinio]

I fear the mod portal will be flooded with individual mod packs so each user can quick install his own favorites.

At least i would do it.

[Qon]

I fear the mod portal will be flooded with individual mod packs so each user can quick install his own favorites.

At least i would do it.

So? If the mod packs are uploaded under the new tag "mod pack" and not shown unless you filter your search to mod packs then it wouldn't interfere with the regular non-empty mods.

[Qon]

I fear the mod portal will be flooded with individual mod packs so each user can quick install his own favorites.

At least i would do it.

So? If the mod packs are uploaded under the new tag "mod pack" and not shown unless you filter your search to mod packs then it wouldn't interfere with the regular non-empty mods.

Also, this is already possible: https://mods.factorio.com/mod/SeaBlockMetaPack
Adding proper mod pack capabilities will reduce the number of packs you see (if you don't look for them) simply because now they are shown among regular mods.

[Lgk]

Great idea!

[jdogg]

Has any more thought been put into implementing this or even some form of it such as a "Download All Dependencies " and/or "Download Selected Dependencies" including check boxes within the dependencies portion of the portal?

[i_i]

As i Understand The MOD PACKS would work as a Stand Alone and would increase the speed of Loading the Mods.

there should be a monitor list of entries that are being used by each mod and which ones are being re used as a different variable so able to sus out the mods that are re-editing the values or function of each entity (lets say: different stack sizes, value sizes of each entity, flags, etc...?)

To minimize the Conflicts & Major Mods terrain conflict where SeaBlock & SpaceExploration & Caves etc...
or other assigned shortcuts.

[ickputzdirwech]

I like the idea of creating a modpack by combining all the mods. Although I couldn’t create a script for that myself, this are the challenges I can think of:

• creating the same dependency tree as the game
• dealing with stuff like “if mods[…]”
• dealing with changing sprite and file paths
• Combining control.lua since you can only access each event once in every mod
• sort out duplicate localised strings

I think keeping track of what mod changed what table is an optional feature. When there is an issue just check with the separate mods and you quickly know if there is a bug with the mods or the combining script.