Desync only on Modded Multiplayer - Unknown Cause

[Rivaryn]

Good evening,

I was creating a new modpack to try, and added an experimental mod that a friend made that alters how ore is generated. The basic idea is to remove the normal oregen, and then use the coordinates of a chunk to determine what level of ore is allowed to spawn there, then repopulate based on chance. However, 0.14 did not have a way to use random seeds, so my friend implemented their own random function, which looks like it would be deterministic.

This mod works fine in single player - that's where I generated the map itself. However, as soon as I loaded it on to my multiplayer server and tried connecting, I got immediate desyncs within maybe 1-2 seconds of entering the game.

I discussed the issue with my friend and they have no idea where the problem lies or how to fix it. The random functions they implemented do not use time, or some other value that changes, just a seed and a math function that should be deterministic.

Any assistance would be appreciated. Below is a pastebin of my desync log, and attached is an archive of my desync files.

Desync client log: http://pastebin.com/BvqRyc1x

[Reika]

I am the friend described, and the custom random implementation has been discussed before here.

As stated above, it is entirely based on simple mathematics; the only inputs are the game state (chunk data) and the configs, both of which are consistent, as should basic operations like arithmetic and lua library calls like bitshifting. Yet the two get different enough outputs for the CRC to flag it (not usefully, mind you, just that the game errored).

I have attached my lua script files in case they are helpful.

[Rseding91]

I am the friend described, and the custom random implementation has been discussed before here.

As stated above, it is entirely based on simple mathematics; the only inputs are the game state (chunk data) and the configs, both of which are consistent, as should basic operations like arithmetic and lua library calls like bitshifting. Yet the two get different enough outputs for the CRC to flag it (not usefully, mind you, just that the game errored).

I have attached my lua script files in case they are helpful.

It's almost certainty desyncing because you're using local variables that are reset every time the game is loaded to control retro gen and the random generator init.

Code: Select all

local ranTick = false
local ranGenInit = false

Then in your on_tick you check if ranTick is false and if not you do some code then set it to false. Someone joins the server and the game is saved, sent to them, then loaded and now on their end ranTick is false again so the next tick runs and it re-does the logic and the 2 games are now different.

You want to just delete those two values and use on_init to do all of the logic you now do. http://lua-api.factorio.com/latest/LuaB ... ap.on_init

[Rivaryn]

Good Evening,

So with Reika's help I simply removed all the retrogen, which is that part with the local variables and ontick handler. Only the on_event function with on_chunk_generated remains, and it only calls controlChunk, all other code is stripped out.

Unfortunately the problem still occurs just as quickly - about 1-2 seconds from loading into the game. Those local variables are now completely unused and unaltered.

Unfortunately I didn't have more time to look into this tonight or I would get the desync reports/logs attached and keep trying various changes.

[Rseding91]

Good Evening,

So with Reika's help I simply removed all the retrogen, which is that part with the local variables and ontick handler. Only the on_event function with on_chunk_generated remains, and it only calls controlChunk, all other code is stripped out.

Unfortunately the problem still occurs just as quickly - about 1-2 seconds from loading into the game. Those local variables are now completely unused and unaltered.

Unfortunately I didn't have more time to look into this tonight or I would get the desync reports/logs attached and keep trying various changes.

Where do you init the random generator? are you persisting state it may have or are you making sure it's using something that's always valid (like seeding it with the chunk position on chunk generated then dumping it after the chunk is generated)?

[Klonan]

I checked your script with an upvalue checker,
And between save and load the values for this function has changed:

Code: Select all

Before:

nextRand = 
  lastlinedefined = 16
  linedefined = 13
  short_src = ...a/Roaming/Factorio/temp/currently-playing/customrand.lua
  upvalue 1: seed = 1539125320
  upvalue 3: MULTIPLIER = 1103515245
  upvalue 4: ADDEND = 12345
  upvalue 5: MASK = 2147483647
  upvalue 6: BITSHIFT = 7
    
After:

  nextRand = 
  lastlinedefined = 16
  linedefined = 13
  short_src = ...a/Roaming/Factorio/temp/currently-playing/customrand.lua
  upvalue 1: seed = 1452030762
  upvalue 3: MULTIPLIER = 1103515245
  upvalue 4: ADDEND = 12345
  upvalue 5: MASK = 2147483647
  upvalue 6: BITSHIFT = 7
  

If you can attach a full mod file (I had to edit the script to make it work in vanilla),
Then it will be easier to debug

But you should store your random generators seed in the 'global' table, so that its preserved between save and load

[Rivaryn]

I checked your script with an upvalue checker,
And between save and load the values for this function has changed:

That sounds like it would be a very handy piece of software for debugging problems related to desync - is that tool available for public use? I personally have not done much modding (looking to get into it at some point) but I am a software engineer so I should be able to pick this up in relatively short order.

I will not be able to implement and test a fix (likely with Reika's help) until this evening, but I will post the results of what I find. Also the modpack has some experimental (non-public release) mods, so I need to defer to him as to whether he wants them posted. I understand why it's helpful, but I also wish to respect his intellectual property rights.

Thanks for all your assistance so far!

[Rivaryn]

Good Evening,

I attempted to work with Reika in setting up the seed value in the global table, and testing it against the server. After a bunch of crashes related to syntax and nil value indexing, I got it to a state that loads but still desyncs. Per request, and after talking to him, I am uploading the mod files.

The full modpack includes bobs ores, but there are no dependancies so it should be fine to test with vanilla. If no desync exists in vanilla then I can post the full modpack to try and see what is causing it, but my guess is the problem will still occur in vanilla.

Thanks.

[Reika]

For what it is worth, the mod has not been tested in a vanilla environment, but it should have no explicit dependencies. If you do have an issue, try commenting out every non-vanilla ore type in config.lua's oreTiers table (lines 13-34).

[prg]

This already breaks in single player after a few seconds with

Code: Select all

Error while running event on_chunk_generated (ID 12)
__Oreverhaul__/functions.lua:178: bad argument #1 to 'pairs' (table expected, got nil)

(when commenting out the modded ores so it starts at all.)

[Reika]

This already breaks in single player after a few seconds with

Code: Select all

Error while running event on_chunk_generated (ID 12)
__Oreverhaul__/functions.lua:178: bad argument #1 to 'pairs' (table expected, got nil)

(when commenting out the modded ores so it starts at all.)

Try these files.

This is what we meant when we said the mod was in development and not ready for release.

[prg]

Code: Select all

__Oreverhaul__/control.lua:2:
__Oreverhaul__/functions.lua:2:
__Oreverhaul__/oreplacement.lua:1:
__Oreverhaul__/constants.lua:40:
attempt to index global 'game' (a nil value)