Page 1 of 5

Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 1:13 pm
by FactorioBot

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 1:27 pm
by jan1i3
Good Job on finding those desync bugs, desyncs are somehting else.
I'm happy i haven't had to deal with them... yet ;)

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 1:40 pm
by Drison
More NEW official content please, not just bugfixes for mods...

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 1:47 pm
by Hyperbar00
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
Leaving bugs alive just because they don't affect the main game is the best way to mess things up later.
They nailed a bug, there is no reason not to fix it.

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 1:54 pm
by 5thHorseman
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
Yeah where's that new character gui you've been promising or how about some new animations or something geez.

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 3:06 pm
by joseph222
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...

Reminds me of space engineers. Full of DLC but multiplayer with mods is unplayable.

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 4:04 pm
by raiguard
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
Ungrateful much? Imagine if they never fixed bugs, just added features. The game would be unplayable. Bug fixes are important.

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 4:46 pm
by Rseding91
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
We're done with new features. What's left is bug fixes, GUI work, high resolution sprites and icons.

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 5:30 pm
by valneq
Rseding91 wrote:
Fri Mar 27, 2020 4:46 pm
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
We're done with new features. What's left is bug fixes, GUI work, high resolution sprites and icons.
So the NPE will stay at the 0.17-Version? Some FFFs ago, this was announced to change during 0.18 closer to what it used to be around 0.16 and before.

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 8:55 pm
by proclion
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
Why?
A mod is a whole pile of new content that the Factorio devs haven't had to make.
If it takes them a day to find and fix a bug that stopped a 200 man hour mod from working that could still be a decent use of time depending on the uptake of that mod.

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 9:16 pm
by Philip017
Rseding91 wrote:
Fri Mar 27, 2020 4:46 pm
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
We're done with new features. What's left is bug fixes, GUI work, high resolution sprites and icons.
i believe this means that only bug fixes and some cosmetic work until 1.0 release, additional content will be relegated to mods, and any new official included in the base game, content will be worked on post 1.0

thank you for all your hard work devs, enjoy being at home (with family) and stay safe and healthy. i dont expect this to change before may/june. perhaps even as late as august as this virus can be rather nasty.

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 9:24 pm
by Serenity09
im glad everything is okay with you guys, stay safe!

it's really cool that you spent any time at all on bugs that seemed (on their surface) to be not your responsibility or scope. i imagine its partially due to the mod's popularity, but it's still cool that you do this. you've earned the bug fixes that you make off these reports

i think these little fixes are part of the stability that the game is built on, and the core of long term support for it and its mods. these devs fix things, not patch them
The problem, is that serpent chose nil as the placeholder value. In Lua, writing a table value as nil is the same as deleting the key, and the key won't be seen when looping the table in the future. When serpent then goes back to 'fix-up' the placeholder values, it ends up with each peer saving a different table ordering (because of some even deeper technicalities with our custom version of Lua).
i feel like i followed an interesting bug but then lost my grip on it right when it got juicy. so serpent needs to handle cyclic references and implements that via placeholder and post-processing. its original implementation used nil as the placeholder, which instead of serving as a flag actually fully deleted the key entirely. but somehow that wasn't really an issue, serpent was still able to resolve the placeholder with its real data. the issue only came in when your implementation of Lua reordered the tables, for some entirely other reason
With version 0.17 we added 2 (really nice) mod capabilities:
i have no idea what the challenges are when it comes to supporting 3rd party mods on a codebase that's in active development. i tried searching for an existing dev blog post on the topic without seeing one, but i'd love to hear about the challenges, and your counter strategies. how do you even start to deal with quality control or regression testing (without relying on things breaking first)?

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 10:35 pm
by BattleFluffy
Thanks for putting the work in, even where 99% of players will never notice it. :> Factorio is quality to the core.

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 10:46 pm
by MakeItGraphic
5thHorseman wrote:
Fri Mar 27, 2020 1:54 pm
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
Yeah where's that new character gui you've been promising or how about some new animations or something geez.
This is sarcasm right?

In any regard:

viewtopic.php?f=3&t=82431 - Character GUI

viewtopic.php?f=3&t=82796 - Updated Animations for Water/Trees

viewtopic.php?f=3&t=82489 - Production/Electric Network GUI

viewtopic.php?f=3&t=81631 - New Scenario 'Rocket Rush'

viewtopic.php?f=3&t=81393 - Poison Capsule Update

viewtopic.php?f=3&t=80554 - Updated Map Colors

viewtopic.php?f=3&t=80361 - Updated Mini-Tutorial

viewtopic.php?f=3&t=80247 - Animations Introduced for Water/Tree

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 10:58 pm
by bobucles
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
But if you really think about it, fixing a game breaking bug in a mod is effectively adding new functional content to the game.

Re: Friday Facts #340 - Deep desyncs

Posted: Fri Mar 27, 2020 11:06 pm
by havvy
The problem, is that serpent chose nil as the placeholder value. In Lua, writing a table value as nil is the same as deleting the key, and the key won't be seen when looping the table in the future. When serpent then goes back to 'fix-up' the placeholder values, it ends up with each peer saving a different table ordering (because of some even deeper technicalities with our custom version of Lua).
i feel like i followed an interesting bug but then lost my grip on it right when it got juicy. so serpent needs to handle cyclic references and implements that via placeholder and post-processing. its original implementation used nil as the placeholder, which instead of serving as a flag actually fully deleted the key entirely. but somehow that wasn't really an issue, serpent was still able to resolve the placeholder with its real data. the issue only came in when your implementation of Lua reordered the tables, for some entirely other reason

The order of values in lua tables sometimes matters (e.g. pick the first element that has a certain property) where reordering the table can change the state of the game. Factorio's Lua gives an insertion order of elements (with exception to numeric keys up to 256). Serpent seems to not replace data in the same order every time, so it can lead to a desync if it chooses different orders for different clients since setting to nil deletes the element and readding it adds it to the end. Or maybe it just does so when somebody joins a multiplayer game. By not removing the key, the table entries no longer get moved around.

Re: Friday Facts #340 - Deep desyncs

Posted: Sat Mar 28, 2020 9:17 am
by ssilk
Rseding91 wrote:
Fri Mar 27, 2020 4:46 pm
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
We're done with new features. What's left is bug fixes, GUI work, high resolution sprites and icons.
We should stop FFF’s - no more content. :mrgreen:

Re: Friday Facts #340 - Deep desyncs

Posted: Sat Mar 28, 2020 10:07 am
by Blahbla
"Serpent" is mentioned in this blog post in relation to Lua data serialization. What is it?

Re: Friday Facts #340 - Deep desyncs

Posted: Sat Mar 28, 2020 10:27 am
by Bilka
Blahbla wrote:
Sat Mar 28, 2020 10:07 am
"Serpent" is mentioned in this blog post in relation to Lua data serialization. What is it?
A Lua table pretty printer that we also use to serialize (save) the Lua state. You can find it here: https://github.com/pkulchenko/serpent.

Re: Friday Facts #340 - Deep desyncs

Posted: Sat Mar 28, 2020 11:10 am
by _Attila_
Rseding91 wrote:
Fri Mar 27, 2020 4:46 pm
Drison wrote:
Fri Mar 27, 2020 1:40 pm
More NEW official content please, not just bugfixes for mods...
We're done with new features. What's left is bug fixes, GUI work, high resolution sprites and icons.
It was mentioned some time ago that we'll have the ability to put BP books into BP books. Is that not going to be implemented now?