Allow settings that present multiple input fields based on prototypes and other data

[sparr]

About once a week I see someone on the Discord in #mod-making asking how to make a dynamic group of settings based on some game data, such as one setting for each turret or tile. Sadly we have to shoot them down every time, and a few folks end up implementing something like a string setting with a delimited list of key value pairs. This post is my proposal for how the game might support such settings, with minimal fundamental changes:

1. Add a new setting property identifying the setting as being an enumerable type. The value of such a setting is of type {{string,setting-type}, ...}
2. Add a new data.raw category for enumerable setting "keys"
3. Modify the settings UI to present enumerable types as multiple inputs, based on the single setting and multiple entries in the enumerable-settings data for that setting.

Here's an example mod using such functionality to set the sizes of containers:

Code: Select all

-- settings.lua
data:extend({{
        type = "int-setting",
        name = "container-size",
        setting_type = "startup",
        default_value = -1,
        minimum_value = -1,
        maximum_value = 512,
        enumerable = true
}})

-- data.lua
for index,entry in pairs(settings.startup["container-size"].value) do
	if data.raw["container"][entry[1]] then
		if entry[2] > -1 then
			data.raw["container"][entry[1]].inventory_size = entry[2]
		end
	else
		table.remove(settings.startup["container-size"].value, index)
	end
end

-- data-final-fixes.lua
local container_ids = {}
for id,entity in pairs(data.raw["container"]) do
	local keep = true
	if entity.flags then
		for _,flag in ipairs(entity.flags)
			if flag == "hidden" then
				keep = false
				break
			end
		end
	end
	if keep then table.insert(container_ids, id) end
end
data.raw["enumerable-settings"]["container-size"] = container_ids

Here's a walk through how using this mod might look to the user and the engine:
1. Install the mod and restart the game
2. Settings stage. A setting is created named container-size. As a new enumerable setting its value is stored as an empty table.
3. Data stage. No containers are modified. A list of non-hidden containers is created and stored in data.raw["enumerable-settings"]["container-size"].
4. Go to the mod settings menu, startup tab. You'll be presented with not one but three input fields. I am fuzzy on how localization can be accomplished for the labels of the fields. Each input field will have been initialized to the setting's default value of -1, and can be interacted with the same way you would interact with three separate int-type settings.
5. Change one of the values and restart the game. Suppose we changed the value for steel chest to 100. Now the value of the setting has been serialized and stored as {{"iron-chest",-1},{"steel-chest",100},{"wooden-chest",-1}}.
6. Settings stage. A setting is created named container-size. The value of that setting is loaded and validated against the type {{string,int},...}. The int values are validated against the min and max for the setting. The string values are not validated.
7. Data stage. The steel chest entity has its inventory size changed. The same list of non-hidden containers from before is created and stored.
8. You'll see the mod settings menu the same way you left it, with values for the three apparent settings (which are really one setting) as -1, 100, -1.
9. Load a mod that removes steel-chest from the game (or unload the mod that provides it, in the hypothetical where it's not a vanilla entity). Restart the game.
10. Settings stage. Same as #6. This stage has no idea what the string value "steel-chest" means and doesn't care.
11. Data stage. No containers are modified, because "steel-chest" doesn't match the name of an entity. The {"steel-chest",100} entry is removed from the setting table value (this is optional; some mods might want to keep "defunct" settings around). The now shorter list of non-hidden containers is created and stored.
12. The mod settings menu will now show just two settings for container size.

[mrvn]

Can you mock up some graphics for this using gimp or photoshop or something? It's really unclear what this is supposed to be. It sounds like you simply want one int setting for every (not blacklisted) container. You can simply do that with a loop.

[sparr]

It sounds like you simply want one int setting for every (not blacklisted) container. You can simply do that with a loop.

No, you can't. You don't have access to data (where the container list exists) during the settings stage, and you can't create settings during the data stage. Making it possible is the whole point of this proposal.

Visually, it would look [almost?] exactly the same as creating N separate settings looks now.

[mrvn]

That makes it clearer. So basically this is about allowing settings to be added in the data phase, although wrapped in a special case.

[robot256]

What is proposed is a procedural way to modify startup settings after the data stage is complete, to avoid executing any Lua code during a new init phase. I don't think this actually solves the sequencing issue, because the new init phase is still created, and it would just as well be a new phase with Lua code like "settings-dynamic.lua".

The part that would be truly confusing to users is caused by allowing those settings to be used in earlier data stages after a restart. What if data.lua makes steel chest with 48 slots and settings-dynamic adds a setting claiming steel chest has 100 slots? How will the game know the setting hasn't been applied yet? They could start a game with the setting equal to 100 but the prototype equal to 48, play a while, then restart and discover it is 100. It would cause multiplayer errors because two players with the same settings file could have different prototypes.

Or, if the game knows it has to restart any time settings-dynamic changes the settings array compare to last time, you are virtually guaranteed that some combination of mods will cause a boot loop. Imagine the steel-chest=100 setting and a mod that deletes all containers with more than 50 slots in data-updates. On the first boot steel-chest has 48 slots and a setting saying 100. On the second boot steel chest has 100 slots, is deleted, and the setting is deleted too. On the third boot, it's back to 48 slots. I'm not saying this would be normal behavior, but it would be very easy to trigger by mod compatibility interactions and frustrating to constantly debug (for users, modders, and devs alike).

No, you have to add another data stage whose purpose is to read the dynamic settings, so that earlier stages remain deterministic. But then you square the current difficulty for mod authors deciding where to place changes. Before settings-dynamic so that other mods can make settings based on them? Or after, so they can use the dynamic settings (or preempt changes another mod might make)? Do you need a second three-part data phase after settings-dynamic? It is a rabbit hole, and simply using a limited-purpose code generator instead of "settings-dynamic.lua" doesn't get us out of it.

[sparr]

The part that would be truly confusing to users is caused by allowing those settings to be used in earlier data stages after a restart. What if data.lua makes steel chest with 48 slots and settings-dynamic adds a setting claiming steel chest has 100 slots?

It's hard to know how to respond when you're asking questions in the context of your own modification to my proposal (which has no settings-dynamic stage). Any mod can change the size of a steel chest any time in the data stage, and they can overwrite each other's changes. This is nothing new. Whether there's a setting, of the normal kind or this new kind, that says a steep chest should have 100 slots, the steel chest prototype could still end up with 48 slots if something else, setting-based or hard coded into a mod, makes it 48.

How will the game know the setting hasn't been applied yet? They could start a game with the setting equal to 100 but the prototype equal to 48, play a while, then restart and discover it is 100. It would cause multiplayer errors because two players with the same settings file could have different prototypes.

The same settings and mods will produce the same prototypes. If the setting says 100 but the prototype says 48 then some other mod changed it from 100 to 48 after my mod changed it to 100, and that's going to happen each time you start with that mods+settings combination.

Imagine the steel-chest=100 setting and a mod that deletes all containers with more than 50 slots in data-updates. On the first boot steel-chest has 48 slots and a setting saying 100. On the second boot steel chest has 100 slots, is deleted, and the setting is deleted too. On the third boot, it's back to 48 slots. I'm not saying this would be normal behavior, but it would be very easy to trigger by mod compatibility interactions and frustrating to constantly debug (for users, modders, and devs alike).

Would this concern be eliminated if we only delete from the enumerable list in the stage where it's set? I could move the cleanup from data-final-fixes to data, in this example. I don't know if that's enforceable though, but we already can't enforce preventing multiplayer desyncs so we just tell mod authors not to do certain things.

[robot256]

Desyncs are one of those "just don't do that" cases, yes, and the fact that they keep happening and frequently require developer help to debug should be a good reason to avoid adding new ones if at all possible.

I was part of the push to fix the two-stage "sync mods and settings with server" process, that was causing other problems with mod loading order and how settings were applied. Also mod mismatch errors resulting from "require" statements that were conditionally executed based on settings. (This is another "illegal" operation that was causung problems in some mods for many years.)

I didn't fully understand the original premise from your first post. You are specifically trying to replace the case where mods have a string setting and the string is "wood-chest=50,steel-chest=100", correct? And you want instead for a setting to act as a heading, and below it have boxes for "wood-chest" and "steel-chest", and store its value as a table.

Table settings by themselves could be very nice. It's the modification of settings in the data phase (table or not) that concerns me.

I better understand how you intended to handle default values. The default value is set in the settings phase, and your code in data is required to treat the default value the same as if that setting entry were not present--because it won't be present until the first load data-final-fixes completes. (So for missing table keys, the default value must be read from the parent setting prototype directly.) And the code that adds an entry to the list cannot add a default value for that particular entry; it takes the default value of the parent setting prototype.

However, a second requirement is implied here. The code in data must treat a setting entry as its default value if the code in data-final-fixes will, in the future, remove the setting entry. Or, as you say, the removal must happen before the setting is read. Thus the requirement is that a setting can never be read until all modifications to the setting list are complete. It is this constraint on when a setting can be *read* that poses the most difficulty, because up until now, settings were considered immutable during the entire data stage and all mods read them all the time.

There will end up being multiple conventions from different authors, similar to how remote interfaces are used in a variety if dissimilar ways. Do you change the list in data before you modify your entities? In data-updates based on other mods' data entities? In data-final-fixes based on other mods data-updates entities, and right before you read them?

New types of mod mismatch errors will be: If a mod treats "no key" differently that "default value"; or if a mod changes a setting list after it is read. If a mod contains such a defect, it may be a long time before it causes an error for any user, and the cause may not be immediately obvious (especially if it is another mod).

Limiting the data stage changes to items in list settings makes the handling of default values possible; the order of operations issue remains just as significant as if arbitrary Lua changes to the settings were allowed.

It's not unsurmountable, but it is a jump in complexity that has to be appreciated before making the leap.

The locale string question is easy, though. When data stage adds a key to the setting list, it should be paired with a localised string structure.