[1.1.12][Modded] desyncs with seablock

[frayien]

Hello ! Since 1.1.12 we have been experiencing desyncs with our modded factorio server.
We usualy get desync randomly after an hour.
We never had a desync before 1.1.12, (we jumped from 1.1.8 to 1.1.11 and updated to 1.1.12 immediately).
Nothing changed significantly on the map between before and after, except scaling up.
There are between 1 and 3 players playing on the server.

Mods : seablock mod pack + factory planner

desync :
https://www.dropbox.com/s/41o6hi200dpp0 ... 6.zip?dl=1

At the time of this desync i was afk and alone on the server.

No idea if this is due to a mod, but I guess it is better to post it here than to keep it for myself ^^

Hope it can be of any use, if needed I can provide other desync reports.

[boskid]

I looked at the desync report and it was around fluid counts and fluid speeds inside of the fluid system for Mineral Sludge ({-418.5, 88.5}). All the producers (filtration units) have matching amount of products finished, all the consumers (crystalizers) have matching amount of products finished so the only differences are around the single fluid system state. Desync itself was triggered by the electric energy difference but this is to be expected because of the pumps inside of that fluid system. There were none changes around fluid systems in that range of versions so i am highly in favor of a memory issue, either on the client or server side. If you have more desync reports i may look into them but from this one desync report alone i am unable to determine anything.

[frayien]

Thank you, here are some more reports :

https://www.dropbox.com/s/gpe9mgqr9hg1t ... 6.zip?dl=1
https://www.dropbox.com/s/m67bquh0ta47j ... 4.zip?dl=1


maybe more interesting, this one we were 3 connected on the server and were all desync at the same time : (unfortunately the other 2 did not wait for the desync report to be generated...)

https://www.dropbox.com/s/f2puyunsxqh85 ... 3.zip?dl=1

Server is a dedicated machine running Ubuntu 20.04.1 LTS (GNU/Linux 5.4.0-62-generic x86_64)

[boskid]

In the desync-report-2021-01-22_10-00-26.zip the most obvious difference is that one of the belts has different active state and there are 3 items missing in the desynced because they were taken by the inserter they are inside of the chemical plant inside of {-269.5, 660.5}

In desync-report-2021-01-22_10-18-33 there is a lot of differences and i cannot find anything interesting. There are differences around active entities, electric buffers, fluid boxes, targeters.

desync-report-2021-01-22_11-15-14 has differences around the mineral sludge pipes around {-418.5, 62.5} (similar to the original issue).

Maybe those desyncs help a little by pointing into direction of fluid systems but other than that i am still thinking it may be memory related. I will test saves from those desyncs in my own MP environment.

[frayien]

What do you mean by memory related ?

[boskid]

I kept this save file running for 9h (in game time) in MP between 2 different PCs and there were no desyncs.

By memory related i mean, when memory starts to become broken it may flip some bits which causes crashes which are then misattributed to the software.

[frayien]

you mean some instability in the RAM hardware ? ... I will look into it. Sorry if I took your time for an unrelated issue ...

But it still feels unlikely that it would desync two times on the same pipe doesn't it ?

[frayien]

Up : no desync since I updated to 1.1.13, guess 1.1.13 or restarting the server fixed it somehow...
I will update you if the desyncs come back or if I find something relevant investigating the hardware.