Friday Facts #340 - Deep desyncs

Regular reports on Factorio development.
User avatar
FactorioBot
Factorio Staff
Factorio Staff
Posts: 430
Joined: Tue May 12, 2015 1:48 pm

Friday Facts #340 - Deep desyncs

Post by FactorioBot »

User avatar
jan1i3
Long Handed Inserter
Long Handed Inserter
Posts: 60
Joined: Sun Dec 09, 2018 1:36 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by jan1i3 »

Good Job on finding those desync bugs, desyncs are somehting else.
I'm happy i haven't had to deal with them... yet ;)
Also known as JanSharp. jan1i3 was/is my old name ;)
Drison
Burner Inserter
Burner Inserter
Posts: 7
Joined: Fri Jan 05, 2018 11:21 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by Drison »

More NEW official content please, not just bugfixes for mods...
User avatar
Hyperbar00
Inserter
Inserter
Posts: 24
Joined: Sun Jan 28, 2018 6:47 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by Hyperbar00 »

Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...
Leaving bugs alive just because they don't affect the main game is the best way to mess things up later.
They nailed a bug, there is no reason not to fix it.
User avatar
5thHorseman
Smart Inserter
Smart Inserter
Posts: 1193
Joined: Fri Jun 10, 2016 11:21 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by 5thHorseman »

Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...
Yeah where's that new character gui you've been promising or how about some new animations or something geez.
joseph222
Inserter
Inserter
Posts: 37
Joined: Sun Apr 10, 2016 9:35 am
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by joseph222 »

Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...

Reminds me of space engineers. Full of DLC but multiplayer with mods is unplayable.
User avatar
raiguard
Factorio Staff
Factorio Staff
Posts: 614
Joined: Wed Dec 13, 2017 8:29 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by raiguard »

Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...
Ungrateful much? Imagine if they never fixed bugs, just added features. The game would be unplayable. Bug fixes are important.
Don't forget, you're here forever.
Rseding91
Factorio Staff
Factorio Staff
Posts: 14595
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by Rseding91 »

Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...
We're done with new features. What's left is bug fixes, GUI work, high resolution sprites and icons.
If you want to get ahold of me I'm almost always on Discord.
User avatar
valneq
Smart Inserter
Smart Inserter
Posts: 1262
Joined: Fri Jul 12, 2019 7:43 am
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by valneq »

Rseding91 wrote: ↑Fri Mar 27, 2020 4:46 pm
Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...
We're done with new features. What's left is bug fixes, GUI work, high resolution sprites and icons.
So the NPE will stay at the 0.17-Version? Some FFFs ago, this was announced to change during 0.18 closer to what it used to be around 0.16 and before.
proclion
Burner Inserter
Burner Inserter
Posts: 11
Joined: Sun Mar 17, 2019 12:36 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by proclion »

Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...
Why?
A mod is a whole pile of new content that the Factorio devs haven't had to make.
If it takes them a day to find and fix a bug that stopped a 200 man hour mod from working that could still be a decent use of time depending on the uptake of that mod.
User avatar
Philip017
Filter Inserter
Filter Inserter
Posts: 360
Joined: Thu Sep 01, 2016 11:21 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by Philip017 »

Rseding91 wrote: ↑Fri Mar 27, 2020 4:46 pm
Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...
We're done with new features. What's left is bug fixes, GUI work, high resolution sprites and icons.
i believe this means that only bug fixes and some cosmetic work until 1.0 release, additional content will be relegated to mods, and any new official included in the base game, content will be worked on post 1.0

thank you for all your hard work devs, enjoy being at home (with family) and stay safe and healthy. i dont expect this to change before may/june. perhaps even as late as august as this virus can be rather nasty.
Serenity09
Burner Inserter
Burner Inserter
Posts: 15
Joined: Mon Sep 30, 2019 4:26 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by Serenity09 »

im glad everything is okay with you guys, stay safe!

it's really cool that you spent any time at all on bugs that seemed (on their surface) to be not your responsibility or scope. i imagine its partially due to the mod's popularity, but it's still cool that you do this. you've earned the bug fixes that you make off these reports

i think these little fixes are part of the stability that the game is built on, and the core of long term support for it and its mods. these devs fix things, not patch them
The problem, is that serpent chose nil as the placeholder value. In Lua, writing a table value as nil is the same as deleting the key, and the key won't be seen when looping the table in the future. When serpent then goes back to 'fix-up' the placeholder values, it ends up with each peer saving a different table ordering (because of some even deeper technicalities with our custom version of Lua).
i feel like i followed an interesting bug but then lost my grip on it right when it got juicy. so serpent needs to handle cyclic references and implements that via placeholder and post-processing. its original implementation used nil as the placeholder, which instead of serving as a flag actually fully deleted the key entirely. but somehow that wasn't really an issue, serpent was still able to resolve the placeholder with its real data. the issue only came in when your implementation of Lua reordered the tables, for some entirely other reason
With version 0.17 we added 2 (really nice) mod capabilities:
i have no idea what the challenges are when it comes to supporting 3rd party mods on a codebase that's in active development. i tried searching for an existing dev blog post on the topic without seeing one, but i'd love to hear about the challenges, and your counter strategies. how do you even start to deal with quality control or regression testing (without relying on things breaking first)?
User avatar
BattleFluffy
Fast Inserter
Fast Inserter
Posts: 206
Joined: Sun Mar 31, 2019 4:58 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by BattleFluffy »

Thanks for putting the work in, even where 99% of players will never notice it. :> Factorio is quality to the core.
User avatar
MakeItGraphic
Fast Inserter
Fast Inserter
Posts: 237
Joined: Sat Jan 06, 2018 7:53 am
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by MakeItGraphic »

5thHorseman wrote: ↑Fri Mar 27, 2020 1:54 pm
Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...
Yeah where's that new character gui you've been promising or how about some new animations or something geez.
This is sarcasm right?

In any regard:

viewtopic.php?f=3&t=82431 - Character GUI

viewtopic.php?f=3&t=82796 - Updated Animations for Water/Trees

viewtopic.php?f=3&t=82489 - Production/Electric Network GUI

viewtopic.php?f=3&t=81631 - New Scenario 'Rocket Rush'

viewtopic.php?f=3&t=81393 - Poison Capsule Update

viewtopic.php?f=3&t=80554 - Updated Map Colors

viewtopic.php?f=3&t=80361 - Updated Mini-Tutorial

viewtopic.php?f=3&t=80247 - Animations Introduced for Water/Tree
bobucles
Smart Inserter
Smart Inserter
Posts: 1708
Joined: Wed Jun 10, 2015 10:37 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by bobucles »

Drison wrote: ↑Fri Mar 27, 2020 1:40 pmMore NEW official content please, not just bugfixes for mods...
But if you really think about it, fixing a game breaking bug in a mod is effectively adding new functional content to the game.
havvy
Burner Inserter
Burner Inserter
Posts: 13
Joined: Fri Aug 29, 2014 9:49 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by havvy »

The problem, is that serpent chose nil as the placeholder value. In Lua, writing a table value as nil is the same as deleting the key, and the key won't be seen when looping the table in the future. When serpent then goes back to 'fix-up' the placeholder values, it ends up with each peer saving a different table ordering (because of some even deeper technicalities with our custom version of Lua).
i feel like i followed an interesting bug but then lost my grip on it right when it got juicy. so serpent needs to handle cyclic references and implements that via placeholder and post-processing. its original implementation used nil as the placeholder, which instead of serving as a flag actually fully deleted the key entirely. but somehow that wasn't really an issue, serpent was still able to resolve the placeholder with its real data. the issue only came in when your implementation of Lua reordered the tables, for some entirely other reason

The order of values in lua tables sometimes matters (e.g. pick the first element that has a certain property) where reordering the table can change the state of the game. Factorio's Lua gives an insertion order of elements (with exception to numeric keys up to 256). Serpent seems to not replace data in the same order every time, so it can lead to a desync if it chooses different orders for different clients since setting to nil deletes the element and readding it adds it to the end. Or maybe it just does so when somebody joins a multiplayer game. By not removing the key, the table entries no longer get moved around.
User avatar
ssilk
Global Moderator
Global Moderator
Posts: 12889
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by ssilk »

Rseding91 wrote: ↑Fri Mar 27, 2020 4:46 pm
Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...
We're done with new features. What's left is bug fixes, GUI work, high resolution sprites and icons.
We should stop FFF’s - no more content. :mrgreen:
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...
Blahbla
Burner Inserter
Burner Inserter
Posts: 10
Joined: Sat Jul 08, 2017 2:57 am
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by Blahbla »

"Serpent" is mentioned in this blog post in relation to Lua data serialization. What is it?
Bilka
Factorio Staff
Factorio Staff
Posts: 3310
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by Bilka »

Blahbla wrote: ↑Sat Mar 28, 2020 10:07 am "Serpent" is mentioned in this blog post in relation to Lua data serialization. What is it?
A Lua table pretty printer that we also use to serialize (save) the Lua state. You can find it here: https://github.com/pkulchenko/serpent.
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.
User avatar
_Attila_
Long Handed Inserter
Long Handed Inserter
Posts: 75
Joined: Sun Jan 06, 2019 2:46 pm
Contact:

Re: Friday Facts #340 - Deep desyncs

Post by _Attila_ »

Rseding91 wrote: ↑Fri Mar 27, 2020 4:46 pm
Drison wrote: ↑Fri Mar 27, 2020 1:40 pm More NEW official content please, not just bugfixes for mods...
We're done with new features. What's left is bug fixes, GUI work, high resolution sprites and icons.
It was mentioned some time ago that we'll have the ability to put BP books into BP books. Is that not going to be implemented now?
Attila's QuickBar Mod - Auto-links hand crafted item to first free quickbar slot if not already linked.
Attila's Signals Mod - Alternate signals to use in same circuit as standard signals.
Attila's Zoom Mod - Modifies zoom functionality.
Post Reply

Return to β€œNews”