Page 1 of 2

CloudSync blueprint ON by default

Posted: Mon May 02, 2022 9:14 am
by Heebra
Game with so many QoL doesn't have this option marked as default on. Why? Now i lost so many HOURS of playing only for improve advanced blueprints only to find out this option deleted my blueprints. I'm taking break of this sad joke. :evil:

Re: CloudSync blueprint ON by default

Posted: Mon May 02, 2022 1:02 pm
by Silari
IIRC issues with it deleting blueprint libraries is why it isn't on by default. Steam's cloud sync isn't great, and even the save one loves to delete saves.

As long as you have a save file from before the library was deleted you can still get your blueprints back: viewtopic.php?f=5&t=94427

Re: CloudSync blueprint ON by default

Posted: Mon May 02, 2022 3:18 pm
by robot256
IMO, it would be nice if the BP lib had a bit of version control or auto backup when you change it. That way there might be an actual advantage to use it instead of the in-game blueprints, besides as an ephemeral clipboard between save files.

It's also a little confusing to new players that there are two different blueprint libraries: "game" blueprints that are shared with other players in the same map, and "my" blueprints that are stored on your machine and visible when you load a different map.

Re: CloudSync blueprint ON by default

Posted: Mon May 02, 2022 4:34 pm
by Heebra
Silari wrote: Mon May 02, 2022 1:02 pm IIRC issues with it deleting blueprint libraries is why it isn't on by default. Steam's cloud sync isn't great, and even the save one loves to delete saves.

As long as you have a save file from before the library was deleted you can still get your blueprints back: viewtopic.php?f=5&t=94427
I have more than 930 hours of play, reinstalling windows/linux many times and i lost my blueprints ONLY by this option off because game see i have 0 blueprint and when i turn it on it overwrite blueprints in cloud sync to 0...

Re: CloudSync blueprint ON by default

Posted: Mon May 02, 2022 6:40 pm
by FuryoftheStars
Heebra wrote: Mon May 02, 2022 4:34 pm
Silari wrote: Mon May 02, 2022 1:02 pm IIRC issues with it deleting blueprint libraries is why it isn't on by default. Steam's cloud sync isn't great, and even the save one loves to delete saves.

As long as you have a save file from before the library was deleted you can still get your blueprints back: viewtopic.php?f=5&t=94427
I have more than 930 hours of play, reinstalling windows/linux many times and i lost my blueprints ONLY by this option off because game see i have 0 blueprint and when i turn it on it overwrite blueprints in cloud sync to 0...
I'm one of those where having the sync on has been what deleted things. I've lost many save games between Factorio and other games from Steam cloud sync screwing up. I now always run with it off, and if I really feel the need to share something between machines, I use other methods.

I would not support having it on by default, but I would say that turning it from the off to on state should prompt you which one (cloud or local) you want to keep.

Re: CloudSync blueprint ON by default

Posted: Mon May 02, 2022 6:51 pm
by robot256
I would not support having it on by default, but I would say that turning it from the off to on state should prompt you which one (cloud or local) you want to keep.
Yes, exactly. That turning it on can ever save a blank library over an existing one makes it more of an April Fool's prank than a usable feature.

Re: CloudSync blueprint ON by default

Posted: Tue May 03, 2022 4:10 am
by ssilk
One of those things wube should fix very fast. I think about moving this to bugs, because it’s unexpected behavior; what do you think?

The possible explanation why this isn’t working as expected: Factorio was for a long time not invented to play it under Steam, the main focus was to sell it directly under their website. No cloud feature was planned.

Re: CloudSync blueprint ON by default

Posted: Tue May 03, 2022 10:07 am
by SoShootMe
ssilk wrote: Tue May 03, 2022 4:10 am The possible explanation why this isn’t working as expected: Factorio was for a long time not invented to play it under Steam, the main focus was to sell it directly under their website. No cloud feature was planned.
Yes; it seems to me the basic problem stems from the combination of the Steam cloud sync API and the blueprint library being a single file, with the latter no issue in the absence of Steam. Blueprints (etc) as files and blueprint books as directories (containing a metadata file defining eg order) might work quite nicely and opens other possibilities, but not without downsides (which may or may not be significant).

Much simpler (at least in relative terms), perhaps it would work to make it so that an empty library means no blueprint-storage.dat file.

Both these ideas depend on details of how cloud sync works that I couldn't find quickly.

Re: CloudSync blueprint ON by default

Posted: Tue May 03, 2022 11:00 am
by FuryoftheStars
ssilk wrote: Tue May 03, 2022 4:10 am I think about moving this to bugs, because it’s unexpected behavior; what do you think?
While it is kind of a bug, it’s not one with Factorio so much. Any automatic sync system that does not verify with the user first has the potential of screwing up and syncing the wrong files. Tbh, I don’t think I’ve found one yet that got it right 100% of the time (though some come close).

Re: CloudSync blueprint ON by default

Posted: Tue May 03, 2022 11:07 am
by robot256
There's no point in syncing to the cloud if the ways of retrieving that data are few or unreliable. If this is a problem with the Steam side of things, then Factorio should include a warning like they do with non-blocking saves.

Right before turning on cloud sync would be a great time to make a blueprints.bak copy just in case.

Re: CloudSync blueprint ON by default

Posted: Tue May 03, 2022 3:20 pm
by Tertius
I remember that if Steam isn't able to cloud sync at game start, it gives a warning that ingame saves and sync items may be missing and could be lost. It gives the player the option to continue loading anyway (items might get lost) or abort loading or wait for sync completion.

This happened to me just after I installed Factorio on a new PC and started Factorio immediately after it finished installing. So I waited for sync completion and nothing was lost.

To avoid complete loss in this situation if a player starts Factorio without waiting for successful sync and Steam later sync an empty state to the cloud, the only thing Factorio can do is probably to determine the sync wasn't successful and then don't save to the usual save directory. Or ask the player (again) if he wants to save to the usual directory that will overwrite anything in the cloud, or if he wants to save to some alternate directory that is not cloud synced.

For me, the current behavior is fine. I appreciate cloud sync default on. I take care of messages and don't click everything away. No additional safeguards required. I also do regular backups, so even if there was some data lost, I can restore it. May be Factorio can also do machine-local (not cloud synced) backup of the blueprint library, since it's a fixed filename that will always be overwritten by a sync and not just added as with new savegame names.

Re: CloudSync blueprint ON by default

Posted: Tue May 03, 2022 6:03 pm
by FuryoftheStars
Tertius wrote: Tue May 03, 2022 3:20 pm I remember that if Steam isn't able to cloud sync at game start, it gives a warning that ingame saves and sync items may be missing and could be lost. It gives the player the option to continue loading anyway (items might get lost) or abort loading or wait for sync completion.

This happened to me just after I installed Factorio on a new PC and started Factorio immediately after it finished installing. So I waited for sync completion and nothing was lost.

To avoid complete loss in this situation if a player starts Factorio without waiting for successful sync and Steam later sync an empty state to the cloud, the only thing Factorio can do is probably to determine the sync wasn't successful and then don't save to the usual save directory. Or ask the player (again) if he wants to save to the usual directory that will overwrite anything in the cloud, or if he wants to save to some alternate directory that is not cloud synced.

For me, the current behavior is fine. I appreciate cloud sync default on. I take care of messages and don't click everything away. No additional safeguards required. I also do regular backups, so even if there was some data lost, I can restore it. May be Factorio can also do machine-local (not cloud synced) backup of the blueprint library, since it's a fixed filename that will always be overwritten by a sync and not just added as with new savegame names.
It can also happen in situations where cloud sync is disabled on one or both, or internet wasn't available on one or both, then both machines are used before cloud sync is re-enabled / internet comes back.

Re: CloudSync blueprint ON by default

Posted: Wed May 04, 2022 5:08 am
by Heebra
ssilk wrote: Tue May 03, 2022 4:10 am One of those things wube should fix very fast. I think about moving this to bugs, because it’s unexpected behavior; what do you think?

The possible explanation why this isn’t working as expected: Factorio was for a long time not invented to play it under Steam, the main focus was to sell it directly under their website. No cloud feature was planned.
I think best option for now is to create an capability to easy backup/import blueprints somewhere in pc if online backup is so broken.
About moving to bugs, i think you're right.

Re: CloudSync blueprint ON by default

Posted: Wed May 04, 2022 5:42 am
by ssilk
Ok.

And this show also the weak point in argumentation: those players which have problems with that don’t have a proper backup. A reliable backup where you can restore your yesterdays blueprints nowadays will cost around 50-150€. Sounds much money, but nobody wants backup, everyone wants just restore. :mrgreen:

Well perhaps I’m the only one, that now thinks: “is it really a problem only of Factorio?”

I know that’s in contrast to my first post.

Re: CloudSync blueprint ON by default

Posted: Wed May 04, 2022 1:54 pm
by robot256
The biggest trouble happens when someone thinks "cloud sync" is the same as "backup". A backup service would keep historical versions; cloud sync only overwrites with whatever it thinks is newest.

The users who post questions about lost blueprints often do not know where the blueprints file is located, or that manual backups are even possible.

While still a manual operation, adding a main menu button to export and import the blueprint library itself would make users aware that this can and should be done, and make it easy to bypass Steam cloud if they desire.

Re: CloudSync blueprint ON by default

Posted: Wed May 04, 2022 3:52 pm
by SoShootMe
ssilk wrote: Wed May 04, 2022 5:42 am And this show also the weak point in argumentation: those players which have problems with that don’t have a proper backup.
Sure, software that misbehaves and causes data loss shouldn't be a major problem because you should have backups to recover from it. I don't think that in any way weakens the argument that it shouldn't happen, much as I wouldn't say a ship design with a flaw that occasionally results in ships of that design sinking is OK because they have lifeboats.
robot256 wrote: Wed May 04, 2022 1:54 pm The biggest trouble happens when someone thinks "cloud sync" is the same as "backup". A backup service would keep historical versions; cloud sync only overwrites with whatever it thinks is newest.
The trouble happens when "whatever [cloud sync] thinks is newest" is not what the user expects. The lack of historical versions compounds the effect but is not the source of it.

Re: CloudSync blueprint ON by default

Posted: Wed May 04, 2022 5:13 pm
by robot256
My Steam cloud currently refuses to sync from either of my machines, so I don't have a horse in this race. Pretty sure I ran the Steam version simultaneously on two machines one too many times, with a different major version installed on each. Manual syncing makes more sense to me anyways (too much mod testing to bother with steam).

Re: CloudSync blueprint ON by default

Posted: Wed May 04, 2022 5:25 pm
by Tertius
External backup is a bit off topic, so I keep it short: You get 128 GB USB sticks for 10-15 EUR. Plug one in, use the built in Windows File History to configure directories to backup (there are defaults; add c:\users\<username>\appdata\roaming\Factorio for Factorio), configure hourly or bi-hourly backup, done. Restore from Explorer context menu or with File History app. Not limited to Factorio; include all your important files created by yourself as well, default c:\users\<username>\documents is already included as default.

With Factorio itself, some copies of blueprint-storage.dat (blueprint-storage.dat-backup0, blueprint-storage.dat-backup1, ...) created by Factorio would be nice.
Could also be included in saves and restored separately on demand.

Re: CloudSync blueprint ON by default

Posted: Wed May 04, 2022 6:23 pm
by netmand
robot256 wrote: Wed May 04, 2022 1:54 pm The biggest trouble happens when someone thinks "cloud sync" is the same as "backup".
This!

If your blueprints are so important; export them and put the exported strings into a another system, like a spreadsheet. Losing a copy of precious data is one of those things we all get burned on until we get it and start putting the time and effort to create our own methods to back it up.

I don't expect my copy of Factorio to baby-sit my blueprints, maps, and game-state saves.

and oh yeah, not all of us play Factorio via Steam. (was this already said? sorry if I'm repeating)

Re: CloudSync blueprint ON by default

Posted: Thu May 05, 2022 5:26 am
by ssilk
SoShootMe wrote: Wed May 04, 2022 3:52 pm
ssilk wrote: Wed May 04, 2022 5:42 am And this show also the weak point in argumentation: those players which have problems with that don’t have a proper backup.
Sure, software that misbehaves and causes data loss shouldn't be a major problem because you should have backups to recover from it. I don't think that in any way weakens the argument that it shouldn't happen, much as I wouldn't say a ship design with a flaw that occasionally results in ships of that design sinking is OK because they have lifeboats.
Nice comparison. :) yes, of course needs wube to fix this, so that’s clear it’s not their fault. But only, because no players will think when they lost their blueprint “shit, I’m such an idiot, why wouldn’t I install a backup?” But more like “Factorio, it’s your fault that I lost now my blueprint!!”

This needs to be fixed because of that.
robot256 wrote: Wed May 04, 2022 1:54 pm The biggest trouble happens when someone thinks "cloud sync" is the same as "backup". A backup service would keep historical versions; cloud sync only overwrites with whatever it thinks is newest.
The trouble happens when "whatever [cloud sync] thinks is newest" is not what the user expects. The lack of historical versions compounds the effect but is not the source of it.
Cloud sync is per se not just overwriting. There are some quite innovative cloud sync solutions, which really do their job. I remember for example https://transfercloud.io which did a great job syncing four of my computers without any failure. Dunno how it’s nowadays.