Hello,
i have enabled blueprint sync on my windows machine. A couple of days later, i startet the ganme on linux. Loaded my save and recognized that my "global" blueprint library is not available. I go to the options (from the loaded game) and also enabled the sync option on linux.
Result:
WIndows: Blueprint library empty
Linux: Blueprint library empty
That's not what i expected!
Nice job :/
[0.17.62] Blueprint Steam Sync - All blueprints are gone
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
Thanks for the report.
Do you happen to still have Factorio log from the run where you enabled the sync option on Linux? Since you had to enable it, it means Steam Cloud wasn't in-sync with your Windows machine; ... but then on the Windows PC you'd get notification about cloud storage being in conflict and it would let you chose which version do you want to keep. So I have no idea what went wrong.
Do you happen to still have Factorio log from the run where you enabled the sync option on Linux? Since you had to enable it, it means Steam Cloud wasn't in-sync with your Windows machine; ... but then on the Windows PC you'd get notification about cloud storage being in conflict and it would let you chose which version do you want to keep. So I have no idea what went wrong.
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
Yes, i have the logs, but i don't think it will help.
For reproducing:
* Factorio/Steam/WIn -> Enabled cloudsave for blueprints a while ago
* Installed Steam on Linux and installed Factorio (Arch Linux).
* Started Factorio on Linux and load savegame. Local blueprints empty.
* Enabled cloudsave for blueprints on Linux (while running game) and closed game without saving (Autosave happend in between).
* Steam sync...
* Start game on Linux again and load the savegame. Local blueprints empty.
* Closed game without saving (Alt+F4)
* No Steam sync because no changes
* Started Factorio on Windows and load savegame. Local blueprints also empty.
* All blueprints are gone
For reproducing:
* Factorio/Steam/WIn -> Enabled cloudsave for blueprints a while ago
* Installed Steam on Linux and installed Factorio (Arch Linux).
* Started Factorio on Linux and load savegame. Local blueprints empty.
* Enabled cloudsave for blueprints on Linux (while running game) and closed game without saving (Autosave happend in between).
* Steam sync...
* Start game on Linux again and load the savegame. Local blueprints empty.
* Closed game without saving (Alt+F4)
* No Steam sync because no changes
* Started Factorio on Windows and load savegame. Local blueprints also empty.
* All blueprints are gone
- Attachments
-
- factorio-previous.log
- (5.09 KiB) Downloaded 164 times
-
- factorio-current.log
- (4.57 KiB) Downloaded 175 times
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
I tried to reproduce the issue several times on 2 Windows machine and 1 Linux machine. Every time I enabled blueprint cloud sync on one of the computers, then it would be enabled on other computers and blueprint library would be available right away.vdreams wrote: ↑Tue Aug 06, 2019 4:02 pm * Factorio/Steam/WIn -> Enabled cloudsave for blueprints a while ago
* Installed Steam on Linux and installed Factorio (Arch Linux).
* Started Factorio on Linux and load savegame. Local blueprints empty.
* Enabled cloudsave for blueprints on Linux (while running game) and closed game without saving (Autosave happend in between).
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
Ok, that's strange. Maybe i was too fast or because steam was running on both sides at the same time.
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
For me the same happened: All bleuprints are gone 
- I enabled the sync on my windows pc.
- Then downloaded the game on my laptop (also windows).
- started factorio on laptop, noticed the wrong version (stable, not on experimental branch yet)
- closed factorio and switch to latest experimental branch
- started again
- enabled sync on laptop and closed factorio
now both machines have zero blueprints
edit #1:
I've restarted the game a couple of times because I still had a 4MB large blueprint-storage.dat file and was wondering why it doesn't work.
For me the bp's seems to re-appear after hosting a mp game on my local machine (windows pc).
my laptop is at work, so can't check what's the status there.
edit #2:
forget to mention that I disabled the cloud sync in between the several restarts. don't know if that was important
- I enabled the sync on my windows pc.
- Then downloaded the game on my laptop (also windows).
- started factorio on laptop, noticed the wrong version (stable, not on experimental branch yet)
- closed factorio and switch to latest experimental branch
- started again
- enabled sync on laptop and closed factorio
now both machines have zero blueprints
edit #1:
I've restarted the game a couple of times because I still had a 4MB large blueprint-storage.dat file and was wondering why it doesn't work.
For me the bp's seems to re-appear after hosting a mp game on my local machine (windows pc).
my laptop is at work, so can't check what's the status there.
edit #2:
forget to mention that I disabled the cloud sync in between the several restarts. don't know if that was important
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
See also viewtopic.php?f=49&t=78011
Koub - Please consider English is not my native language.
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
Oh thanks that of course could happened to me. I didn't got a blueprint-storage-backup.dat but the normal 4MB blueprint-storage.dat was still there.
For a moment I thought it was because of my full cloud storage for factorio (steam displayed "only 2MB left", and I know that many older saves aren't included anymore).
For a moment I thought it was because of my full cloud storage for factorio (steam displayed "only 2MB left", and I know that many older saves aren't included anymore).
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
I can confirm this also happening to me, right now. Version 0.17.79.
I have three computers - here's my observations, for what it's worth:
- On my main gaming computer, where I have played Factorio most of the time until now, I enabled blueprint sync.
- On my laptop, I started Factorio and I saw the BP cloud sync setting being enabled, and all my blueprints available. Great.
- Built a new PC yesterday, and after installing Factorio, I noticed the BP cloud sync setting was NOT enabled, and all the blueprint books gone on this computer. Remembering seeing this thread earlier, I created a new save and "backed up" my blueprint books to the inventory. Then I enabled the BP cloud sync setting on my new computer. Restarted the game, allowing Steam to sync in between, and poof, all BPs gone on my other computers as well. Had I not had my little manual backup, this would be quite bad...
Two observations:
- It seems to be an issue with the original sync. The setting is not properly synced, and I would reckon that also prevents Factorio from doing the initial BP sync, ending up with no blueprints. After enabling, it'll gladly sync the empty library.
- There seems to be no conflict handling. There really should be no way to completely delete something through sync that are not explicitly deleted by the user. My suggestion would be to at least backup the local blueprint library if there's an incoming synced library that is in fact a completely different (empty) library. Without knowing the internals here, having an unique ID for the library that is synced, and trigger a conflict (that the USER must resolve) if the incoming synced library's ID is not matching the game's current library ID, might be a solution...
Tl;dr: Syncing stuff blindly is very scary & dangerous and is probably a bad idea... (But I love the feature in general!)
My 2c as a software engineer.
I have three computers - here's my observations, for what it's worth:
- On my main gaming computer, where I have played Factorio most of the time until now, I enabled blueprint sync.
- On my laptop, I started Factorio and I saw the BP cloud sync setting being enabled, and all my blueprints available. Great.
- Built a new PC yesterday, and after installing Factorio, I noticed the BP cloud sync setting was NOT enabled, and all the blueprint books gone on this computer. Remembering seeing this thread earlier, I created a new save and "backed up" my blueprint books to the inventory. Then I enabled the BP cloud sync setting on my new computer. Restarted the game, allowing Steam to sync in between, and poof, all BPs gone on my other computers as well. Had I not had my little manual backup, this would be quite bad...
Two observations:
- It seems to be an issue with the original sync. The setting is not properly synced, and I would reckon that also prevents Factorio from doing the initial BP sync, ending up with no blueprints. After enabling, it'll gladly sync the empty library.
- There seems to be no conflict handling. There really should be no way to completely delete something through sync that are not explicitly deleted by the user. My suggestion would be to at least backup the local blueprint library if there's an incoming synced library that is in fact a completely different (empty) library. Without knowing the internals here, having an unique ID for the library that is synced, and trigger a conflict (that the USER must resolve) if the incoming synced library's ID is not matching the game's current library ID, might be a solution...
Tl;dr: Syncing stuff blindly is very scary & dangerous and is probably a bad idea... (But I love the feature in general!)
My 2c as a software engineer.
-
templar4522
- Burner Inserter
- Posts: 14
- Joined: Fri Jan 26, 2018 5:10 am
- Contact:
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
Same happened here. Just freshly installed a new system on linux, savegames synced correctly, I opened a savegame to see if blueprints were there, zero blueprints so I go and enabled the setting, quit the game, let steam do its thing, reopen the game, and the blueprints are still not there, I suspect they might be gone from the cloud too.
There's a 19k file locally so I am going out on a limb here, maybe loading the game initialised or updated the empty blueprint.dat file, and when enabling the sync the file is taken as the most recent, so zero conflicts.
I suspect it'd be easier if there was no setting at all... that way the blueprint library would be downloaded directly with the rest of the stuff instead of being overridden when enabling the setting.
There's a 19k file locally so I am going out on a limb here, maybe loading the game initialised or updated the empty blueprint.dat file, and when enabling the sync the file is taken as the most recent, so zero conflicts.
I suspect it'd be easier if there was no setting at all... that way the blueprint library would be downloaded directly with the rest of the stuff instead of being overridden when enabling the setting.
-
TruePikachu
- Filter Inserter
- Posts: 978
- Joined: Sat Apr 09, 2016 8:39 pm
- Contact:
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
Actually, I think I came up with a possible resolution for this issue.
For background, Steam Auto-Cloud is the system by which Factorio uses the Steam Cloud, and it is really dumb in how it operates; if a file is newer than the one it knows about (e.g. because the blueprint library was just regenerated from scratch), it will happily replace older versions of the file that have the same name (e.g. your comprehensive library on another computer). The issue arises whenever a new library is covered by cloud sync, either by being a new installation when sync was disabled or by being a downgraded installation in any case; if the old library isn't modified, then sync will see the new one was modified more recently, and replace the old one.
The resolution works by using a second, backup copy of the library. The library also gets an additional field in the header that serves as a unique identifier, generated only when the library is initialised and remaining constant thereafter. This way, the game can detect this form of conflict by comparing the identifiers when it loads, and asking the player to intervene if they happen not to match.
The simple implementation of the backup copy is an extra instance of the file in the data directory, but not covered by sync. If it is updated by simple copy after any conflict resolution takes place, it can also represent a snapshot of the blueprints at the beginning of the previous session, allowing recovery in case the library gets corrupted due to a bug or accidentially overwritten by downgrade.
(The complex implementation would be to use the Steam Cloud API instead of Steam Auto-Cloud for the blueprint library, and handle sync manually, but this almost certainly isn't worth it.)
For background, Steam Auto-Cloud is the system by which Factorio uses the Steam Cloud, and it is really dumb in how it operates; if a file is newer than the one it knows about (e.g. because the blueprint library was just regenerated from scratch), it will happily replace older versions of the file that have the same name (e.g. your comprehensive library on another computer). The issue arises whenever a new library is covered by cloud sync, either by being a new installation when sync was disabled or by being a downgraded installation in any case; if the old library isn't modified, then sync will see the new one was modified more recently, and replace the old one.
The resolution works by using a second, backup copy of the library. The library also gets an additional field in the header that serves as a unique identifier, generated only when the library is initialised and remaining constant thereafter. This way, the game can detect this form of conflict by comparing the identifiers when it loads, and asking the player to intervene if they happen not to match.
The simple implementation of the backup copy is an extra instance of the file in the data directory, but not covered by sync. If it is updated by simple copy after any conflict resolution takes place, it can also represent a snapshot of the blueprints at the beginning of the previous session, allowing recovery in case the library gets corrupted due to a bug or accidentially overwritten by downgrade.
(The complex implementation would be to use the Steam Cloud API instead of Steam Auto-Cloud for the blueprint library, and handle sync manually, but this almost certainly isn't worth it.)
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
Has there been any progress on this? This is a data loss issue and therefore of critical severity. Especially since blueprints are even more important than saves to some people. I would probably just quit the game if I would lose all my blueprints... 
Also, why is this in 1/0 magic? If the issue arises as described by TruePikachu it is a straightforward problem. And even if not there are both simple and advanced solutions that mitigate such issues.
One example for a simple mitigation would be what Firefox does with it's sessionstore (where all your open tabs are saved). It always keeps two backup copies of that file. I believe one is always the last non-corrupt version and one is the version from the previous time the browser was exited. I think the main purpose for Firefox to do that is possible corruption of that file, so the approach should be slightly tweaked to fit sync issues better.
But that's the least you could do. Something more elegant, possibly along the lines of an unique identifier like what TruePikachu outlined would be even better of course.
Also, why is this in 1/0 magic? If the issue arises as described by TruePikachu it is a straightforward problem. And even if not there are both simple and advanced solutions that mitigate such issues.
One example for a simple mitigation would be what Firefox does with it's sessionstore (where all your open tabs are saved). It always keeps two backup copies of that file. I believe one is always the last non-corrupt version and one is the version from the previous time the browser was exited. I think the main purpose for Firefox to do that is possible corruption of that file, so the approach should be slightly tweaked to fit sync issues better.
But that's the least you could do. Something more elegant, possibly along the lines of an unique identifier like what TruePikachu outlined would be even better of course.
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
i haved same problem! But only on Win-10 (i not used other OS)
I lost all my blueprints. But it happened after 6 month after switch on synchronization cloud. After that I took out BP storage from my other computer (old data).
For some time this worked fine, but then a new bug appeared - now I cannot save the new BP in my BPstorage (Button "b").
!
viewtopic.php?f=7&t=82987
p.s.
just for understanding - I created my blueprints for 1 year! It is very important for me!!! more important than saves of the game, more important then new interface, more important then improvements/features!
Before this bug - I really respected the developers.
After this bug, I am very angry!
I lost all my blueprints. But it happened after 6 month after switch on synchronization cloud. After that I took out BP storage from my other computer (old data).
For some time this worked fine, but then a new bug appeared - now I cannot save the new BP in my BPstorage (Button "b").
!viewtopic.php?f=7&t=82987
p.s.
just for understanding - I created my blueprints for 1 year! It is very important for me!!! more important than saves of the game, more important then new interface, more important then improvements/features!
Before this bug - I really respected the developers.
After this bug, I am very angry!
2,752 hours in factorio.
Re: [0.17.62] Blueprint Steam Sync - All blueprints are gone
Hello,
Bit of a necro for this rather upsetting bug. It is still occuring now, as of [1.1.91].
This has happened to me before (in ver 1.0) but I lacked info to update this bug report.
In the unreported instance I had dualbooted from win10 to linux and had my blueprints removed.
It happens when installing a fresh factorio on a fresh OS. When you first boot the game it does not have cloud sync enabled (despite it being enabled in the OS you were playing from before). When enabling it, it seems to overwrite the existing cloud settings and blueprints with the blank install. When you go back to the original install that you last played with your blueprints, they will also be overwritten with the blank install from syncing from the cloud.
On to the issue I experienced right now: I purchased a steam deck in the last few days, and have again encountered this nasty bug.
When I left my home machine, dualbooting both windows and linux, cloud sync was enabled and I very recently played factorio on both installs. They sync fine between each other. No sync errors since last exit(s). I always let them finish uploading before closing steam.
-I installed factorio on the completely fresh steam deck. I later launched the game. Note I was connected to the internet during launch. No cloud sync errors or anything.
-I loaded a recent save
-Turned off controller support in favour of mouse and keyboard.
-Noticed my blueprints were missing (shit, here we go again), so went back into the options.
Blueprint sync was off.
-I enabled the blueprint sync, but no blueprints were available.
-Relaunched the game. Still no blueprints.
-Toggled the option on/off with a combination of starting the game again, still no blueprints.
-I then checked my steam cloudsave files:
https://store.steampowered.com/account/ ... pid=427520
No blueprint-storage.dat file to be found. It appears to be blown away from the cloud sync. Not even an empty one was synced back.
player-data.json and config.ini were updated in cloud. all other files I can see (save game zip files) are dated from before I launched factorio on the deck and did sync correctly.
When I get home and launch factorio on either win10 or linux, they will likely sync the blank install and remove my blueprints, just like the first time it happened when I started dualbooting.
[EDIT] confirming now that I'm home, blueprints are nuked on my home machine too. [/EDIT]
Until we get to the bottom of this issue, could you please include the last 5 or so blueprint backup files on both the current local machine and in the cloud save? Out of everything I could have lost from the cloudsync, blueprints hurt the most. Having an automated backup could save others from also losing everything.
Bit of a necro for this rather upsetting bug. It is still occuring now, as of [1.1.91].
This has happened to me before (in ver 1.0) but I lacked info to update this bug report.
In the unreported instance I had dualbooted from win10 to linux and had my blueprints removed.
It happens when installing a fresh factorio on a fresh OS. When you first boot the game it does not have cloud sync enabled (despite it being enabled in the OS you were playing from before). When enabling it, it seems to overwrite the existing cloud settings and blueprints with the blank install. When you go back to the original install that you last played with your blueprints, they will also be overwritten with the blank install from syncing from the cloud.
On to the issue I experienced right now: I purchased a steam deck in the last few days, and have again encountered this nasty bug.
When I left my home machine, dualbooting both windows and linux, cloud sync was enabled and I very recently played factorio on both installs. They sync fine between each other. No sync errors since last exit(s). I always let them finish uploading before closing steam.
-I installed factorio on the completely fresh steam deck. I later launched the game. Note I was connected to the internet during launch. No cloud sync errors or anything.
-I loaded a recent save
-Turned off controller support in favour of mouse and keyboard.
-Noticed my blueprints were missing (shit, here we go again), so went back into the options.
Blueprint sync was off.
-I enabled the blueprint sync, but no blueprints were available.
-Relaunched the game. Still no blueprints.
-Toggled the option on/off with a combination of starting the game again, still no blueprints.
-I then checked my steam cloudsave files:
https://store.steampowered.com/account/ ... pid=427520
No blueprint-storage.dat file to be found. It appears to be blown away from the cloud sync. Not even an empty one was synced back.
player-data.json and config.ini were updated in cloud. all other files I can see (save game zip files) are dated from before I launched factorio on the deck and did sync correctly.
When I get home and launch factorio on either win10 or linux, they will likely sync the blank install and remove my blueprints, just like the first time it happened when I started dualbooting.
[EDIT] confirming now that I'm home, blueprints are nuked on my home machine too. [/EDIT]
Until we get to the bottom of this issue, could you please include the last 5 or so blueprint backup files on both the current local machine and in the cloud save? Out of everything I could have lost from the cloudsync, blueprints hurt the most. Having an automated backup could save others from also losing everything.