[0.17.62] Blueprint Steam Sync - All blueprints are gone

Place for things which are bugs but we have no idea how to solve them. Things related to hardware, libraries, strange setups, etc.
Post Reply
vdreams
Inserter
Inserter
Posts: 20
Joined: Thu May 25, 2017 3:25 pm
Contact:

[0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by vdreams »

Hello,

i have enabled blueprint sync on my windows machine. A couple of days later, i startet the ganme on linux. Loaded my save and recognized that my "global" blueprint library is not available. I go to the options (from the loaded game) and also enabled the sync option on linux.

Result:
WIndows: Blueprint library empty
Linux: Blueprint library empty

That's not what i expected!

Nice job :/

posila
Factorio Staff
Factorio Staff
Posts: 4558
Joined: Thu Jun 11, 2015 1:35 pm
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by posila »

Thanks for the report.
Do you happen to still have Factorio log from the run where you enabled the sync option on Linux? Since you had to enable it, it means Steam Cloud wasn't in-sync with your Windows machine; ... but then on the Windows PC you'd get notification about cloud storage being in conflict and it would let you chose which version do you want to keep. So I have no idea what went wrong.

vdreams
Inserter
Inserter
Posts: 20
Joined: Thu May 25, 2017 3:25 pm
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by vdreams »

Yes, i have the logs, but i don't think it will help.

For reproducing:

* Factorio/Steam/WIn -> Enabled cloudsave for blueprints a while ago
* Installed Steam on Linux and installed Factorio (Arch Linux).
* Started Factorio on Linux and load savegame. Local blueprints empty.
* Enabled cloudsave for blueprints on Linux (while running game) and closed game without saving (Autosave happend in between).
* Steam sync...
* Start game on Linux again and load the savegame. Local blueprints empty.
* Closed game without saving (Alt+F4)
* No Steam sync because no changes
* Started Factorio on Windows and load savegame. Local blueprints also empty.
* All blueprints are gone
Attachments
factorio-previous.log
(5.09 KiB) Downloaded 27 times
factorio-current.log
(4.57 KiB) Downloaded 28 times

posila
Factorio Staff
Factorio Staff
Posts: 4558
Joined: Thu Jun 11, 2015 1:35 pm
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by posila »

vdreams wrote:
Tue Aug 06, 2019 4:02 pm
* Factorio/Steam/WIn -> Enabled cloudsave for blueprints a while ago
* Installed Steam on Linux and installed Factorio (Arch Linux).
* Started Factorio on Linux and load savegame. Local blueprints empty.
* Enabled cloudsave for blueprints on Linux (while running game) and closed game without saving (Autosave happend in between).
I tried to reproduce the issue several times on 2 Windows machine and 1 Linux machine. Every time I enabled blueprint cloud sync on one of the computers, then it would be enabled on other computers and blueprint library would be available right away.

vdreams
Inserter
Inserter
Posts: 20
Joined: Thu May 25, 2017 3:25 pm
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by vdreams »

Ok, that's strange. Maybe i was too fast or because steam was running on both sides at the same time.

Yenz
Burner Inserter
Burner Inserter
Posts: 19
Joined: Thu Dec 14, 2017 9:16 am
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by Yenz »

For me the same happened: All bleuprints are gone :(

- I enabled the sync on my windows pc.
- Then downloaded the game on my laptop (also windows).
- started factorio on laptop, noticed the wrong version (stable, not on experimental branch yet)
- closed factorio and switch to latest experimental branch
- started again
- enabled sync on laptop and closed factorio

now both machines have zero blueprints


edit #1:
I've restarted the game a couple of times because I still had a 4MB large blueprint-storage.dat file and was wondering why it doesn't work.
For me the bp's seems to re-appear after hosting a mp game on my local machine (windows pc).
my laptop is at work, so can't check what's the status there.


edit #2:
forget to mention that I disabled the cloud sync in between the several restarts. don't know if that was important

Koub
Global Moderator
Global Moderator
Posts: 5559
Joined: Fri May 30, 2014 8:54 am
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by Koub »

Koub - Please consider English is not my native language.

Yenz
Burner Inserter
Burner Inserter
Posts: 19
Joined: Thu Dec 14, 2017 9:16 am
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by Yenz »

Oh thanks that of course could happened to me. I didn't got a blueprint-storage-backup.dat but the normal 4MB blueprint-storage.dat was still there.

For a moment I thought it was because of my full cloud storage for factorio (steam displayed "only 2MB left", and I know that many older saves aren't included anymore).

hallgeirl
Burner Inserter
Burner Inserter
Posts: 11
Joined: Fri Dec 14, 2018 8:11 pm
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by hallgeirl »

I can confirm this also happening to me, right now. Version 0.17.79.

I have three computers - here's my observations, for what it's worth:
- On my main gaming computer, where I have played Factorio most of the time until now, I enabled blueprint sync.
- On my laptop, I started Factorio and I saw the BP cloud sync setting being enabled, and all my blueprints available. Great.
- Built a new PC yesterday, and after installing Factorio, I noticed the BP cloud sync setting was NOT enabled, and all the blueprint books gone on this computer. Remembering seeing this thread earlier, I created a new save and "backed up" my blueprint books to the inventory. Then I enabled the BP cloud sync setting on my new computer. Restarted the game, allowing Steam to sync in between, and poof, all BPs gone on my other computers as well. Had I not had my little manual backup, this would be quite bad...

Two observations:
- It seems to be an issue with the original sync. The setting is not properly synced, and I would reckon that also prevents Factorio from doing the initial BP sync, ending up with no blueprints. After enabling, it'll gladly sync the empty library.
- There seems to be no conflict handling. There really should be no way to completely delete something through sync that are not explicitly deleted by the user. My suggestion would be to at least backup the local blueprint library if there's an incoming synced library that is in fact a completely different (empty) library. Without knowing the internals here, having an unique ID for the library that is synced, and trigger a conflict (that the USER must resolve) if the incoming synced library's ID is not matching the game's current library ID, might be a solution...

Tl;dr: Syncing stuff blindly is very scary & dangerous and is probably a bad idea... (But I love the feature in general!)
My 2c as a software engineer. :-)

templar4522
Manual Inserter
Manual Inserter
Posts: 3
Joined: Fri Jan 26, 2018 5:10 am
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by templar4522 »

Same happened here. Just freshly installed a new system on linux, savegames synced correctly, I opened a savegame to see if blueprints were there, zero blueprints so I go and enabled the setting, quit the game, let steam do its thing, reopen the game, and the blueprints are still not there, I suspect they might be gone from the cloud too.

There's a 19k file locally so I am going out on a limb here, maybe loading the game initialised or updated the empty blueprint.dat file, and when enabling the sync the file is taken as the most recent, so zero conflicts.

I suspect it'd be easier if there was no setting at all... that way the blueprint library would be downloaded directly with the rest of the stuff instead of being overridden when enabling the setting.

User avatar
TruePikachu
Filter Inserter
Filter Inserter
Posts: 978
Joined: Sat Apr 09, 2016 8:39 pm
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by TruePikachu »

Actually, I think I came up with a possible resolution for this issue.

For background, Steam Auto-Cloud is the system by which Factorio uses the Steam Cloud, and it is really dumb in how it operates; if a file is newer than the one it knows about (e.g. because the blueprint library was just regenerated from scratch), it will happily replace older versions of the file that have the same name (e.g. your comprehensive library on another computer). The issue arises whenever a new library is covered by cloud sync, either by being a new installation when sync was disabled or by being a downgraded installation in any case; if the old library isn't modified, then sync will see the new one was modified more recently, and replace the old one.

The resolution works by using a second, backup copy of the library. The library also gets an additional field in the header that serves as a unique identifier, generated only when the library is initialised and remaining constant thereafter. This way, the game can detect this form of conflict by comparing the identifiers when it loads, and asking the player to intervene if they happen not to match.

The simple implementation of the backup copy is an extra instance of the file in the data directory, but not covered by sync. If it is updated by simple copy after any conflict resolution takes place, it can also represent a snapshot of the blueprints at the beginning of the previous session, allowing recovery in case the library gets corrupted due to a bug or accidentially overwritten by downgrade.

(The complex implementation would be to use the Steam Cloud API instead of Steam Auto-Cloud for the blueprint library, and handle sync manually, but this almost certainly isn't worth it.)

User avatar
Illiou
Long Handed Inserter
Long Handed Inserter
Posts: 51
Joined: Wed Feb 27, 2019 9:35 pm
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by Illiou »

Has there been any progress on this? This is a data loss issue and therefore of critical severity. Especially since blueprints are even more important than saves to some people. I would probably just quit the game if I would lose all my blueprints... :roll:

Also, why is this in 1/0 magic? If the issue arises as described by TruePikachu it is a straightforward problem. And even if not there are both simple and advanced solutions that mitigate such issues.

One example for a simple mitigation would be what Firefox does with it's sessionstore (where all your open tabs are saved). It always keeps two backup copies of that file. I believe one is always the last non-corrupt version and one is the version from the previous time the browser was exited. I think the main purpose for Firefox to do that is possible corruption of that file, so the approach should be slightly tweaked to fit sync issues better.

But that's the least you could do. Something more elegant, possibly along the lines of an unique identifier like what TruePikachu outlined would be even better of course.

terradus
Inserter
Inserter
Posts: 35
Joined: Mon Mar 14, 2016 9:58 am
Contact:

Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone

Post by terradus »

i haved same problem! But only on Win-10 (i not used other OS)

I lost all my blueprints. But it happened after 6 month after switch on synchronization cloud. After that I took out BP storage from my other computer (old data).
For some time this worked fine, but then a new bug appeared - now I cannot save the new BP in my BPstorage (Button "b"). :evil: !
viewtopic.php?f=7&t=82987

p.s.
just for understanding - I created my blueprints for 1 year! It is very important for me!!! more important than saves of the game, more important then new interface, more important then improvements/features!

Before this bug - I really respected the developers.
After this bug, I am very angry!
2,752 hours in factorio.

Post Reply

Return to “1 / 0 magic”

Who is online

Users browsing this forum: No registered users