Lua - Persistent Data

[NetArc]

TL;DR: A way to persist data across different instances of maps/scenarios/mods

I know data is currently serialized and persisted between saving and loading of a map but I think it would open up doors for possibilities in the community for mod/scenario authors to be able to persist data across new games (XP trees, paragon, "leveling" up, etc..).

There are several approaches I think this could be handled through and in no particular order here they are:

1. Add a LuaGameScript.read_file method to read a file from the script-output folder. Then the mod/scenario authors could manage the finer details of persisting data themselves and parsing it.

2. Allow require("filename") to also read files from script-output.

3. Add a global_mod and/or global_scenario variable that functions exactly like global but persists at the client level.

4. Add a game.data class (LuaGameDataStore) that provides get / set key-value tools.

Thank you for your time.

[Klonan]

Its probably never going to happen, storing data outside of the game state is a sure fire way to cause a desync

[NetArc]

Ahh yes, desync.. valid concern!

I think there are more complex solutions to that but for the sake of getting "something" over nothing; I do have a thought in that regard.

This hypothetical stored table could be something (pseudo code) like

Code: Select all

{
  ["key-name"] = {
    data = ...table/string/value..,
    version_vector = ...
  }
}

And when it comes times to sync data you simply accept the newest version vector as the truth and sync that data (so you don't need to do any deep merge strategies).

This concept seems like it would fit the global_mod approach or even game.data tool approach and turns it into a key/value based system (since we don't merge values).

Thoughts?

[ssilk]

This is how I understand that: you want to change mod-data by reading a file that fires an event to update all connected clients mod-data in the middle of process.
Never going to be happen.

There is a mod configuration management planned (with config screens etc. it's mention in an FFF from this year). Let's wait and see how that can be used.

[darkfrei]

viewtopic.php?f=6&t=29739

I think you can move your mods folder to script-output folder. Then you have access to it.

[NetArc]

This is how I understand that: you want to change mod-data by reading a file that fires an event to update all connected clients mod-data in the middle of process.
Never going to be happen.

On the contrary, this idea is not changing mod-data in terms of entities/recipes/technology definitions at run time.
This is about storing strings/numbers/tables across new map instances/games (vs Load Game) much like how we store data in global.

A simple concept but more in-depth explanation would be as follows:

Bob + John play a cool new hypothetical mod (RPG Town) on a random generated map. This mod has some neat things like "leveling up".
When Bob/John level up and "beat" the map, its over and done; but a key/value is updated when they level up for each of them say:

Code: Select all

game.data.set("bob-key", {
  level = 2
})
game.data.set("john-key", {
  level = 2
})

Of course theres no issue syncing data since this occurs while they are playing.

Next time Bob sits down but he plays with Alice a new random generated map with the same mod (RPG Town) active.
When they connect and sync data, Alice gets the new values for keys she doesn't have (both bob-key and john-key).

Bob is level 2 in this game and Alice starts off at level 1 and he is able to help Alice since he has some new RPG abilities unlocked because he is level 2; and it ends up being a fun game and both Bob and Alice level up shortly before beating the game which causes a data update:

Code: Select all

game.data.set("bob-key", {
  level = 3
})
game.data.set("alice-key", {
  level = 2
})

At the end of the week Bob sits back down with John and wants to play RPG Town again, but when they connect we have a data conflict since according to Bob's data store he is now lvl 3; Unfortunately we hit "desync" because John has a different (older) value for bob-key.

Factorio looks at some magic under the hood (vector version associated to the root keys for instance) and takes Bob as the source/truth for both alice-key / alice-key since they are "newer" when compared with John.

And they start a new game with the following:

Code: Select all

game.data.get("bob-key")
=> {
  level = 3
}

game.data.get("john-key")
=> {
  level = 2
}

game.data.get("alice-key")
=> {
  level = 2
}

And Bob is able to show off his level 3 skills to John when they play for their second game.

[NetArc]

viewtopic.php?f=6&t=29739

I think you can move your mods folder to script-output folder. Then you have access to it.

Sadly no, when you start a game; the scenario scripts are copied to a

Code: Select all

{ROOT}/temp/currently-playing/...

folder and cannot include/open anything out of that scope.

Mods also cannot include files outside their mod folder scope; and

Code: Select all

game.write_file(...)

only saves to

Code: Select all

{ROOT}/script-output/...

[darkfrei]

Is it possible to use hard links? https://msdn.microsoft.com/en-us/librar ... s.85).aspx

[NetArc]

Is it possible to use hard links? https://msdn.microsoft.com/en-us/librar ... s.85).aspx

This actually does work (at the expense of a pre-hard link setup), not sure using this approach can over come the desync issue though; but I will explore this avenue further (worst case it can be single player persisting data across maps)

[bobingabout]

I think I was having trouble with this sort of thing for 2 reasons.

I wanted a common part of a mod that could be used by another. This was solved by a remote.call function.
The second part is storing config options, I have a config mod, but this is unable to effect script levels because you can't access files outside of your own mod. This one is currently unresolved, but there is a solution in 0.15.

[ssilk]

What is described here cries for a GUI or remote call. That is the right way to change things over all instances. Cause then the developer will handle the changes correctly. Which is not the case, when a value of hundreds that are eventually stored in such a file is changed between saves. That is the guarantee to break things.

[Jürgen Erhard]

Current situation, and it sucks: example mod "Achiever". It forgets its state every time it gets disabled (which is, according to one dev, soooo rare... unless you want to play vanilla sometimes. Or MP with different mod settings. Oops)

Every time you disable it for a game, and then reenable it, it has forgotten all about its custom achievements, well, achieved. Give me a valid reason how this doesn't suck.

Conclusion: either tell people OFFICIALLY that stuff like this is a no-no, never to expect support (EVEN IN SP... why does my SP experience have to be hampered by MP?)

Or you fix it. Maybe with a disclaimer that "X is not supported in MP play", that'd be fine.

Oh, and switching mods in/out sucks. Needs an (easy) fix (which still makes that annoying "restart whole game" thing, but an improvement).

[BenSeidel]

Its probably never going to happen, storing data outside of the game state is a sure fire way to cause a desync

I was thinking about storing "game state" in a blueprint in the players library. Have yet to get around to trying it. Maybe someone with some more time could try?

[kreatious]

Its probably never going to happen, storing data outside of the game state is a sure fire way to cause a desync

What is described here cries for a GUI or remote call. That is the right way to change things over all instances. Cause then the developer will handle the changes correctly. Which is not the case, when a value of hundreds that are eventually stored in such a file is changed between saves. That is the guarantee to break things.

...

Why not have the server store the data (perhaps in JSON) on its file system, separately from the game state, limit it to 5MB (just like HTML5's sessionStorage), and distribute a compressed copy of it to all the clients that connect. The clients will then store this data in {ROOT}/temp/currently-playing/...

Obviously, this wouldn't carry over to single player, but mods could implement an import/export button if they'd like it to.