[0.18.18] Blueprint Book File Symlink Overwritten

[dpacbach]

OS: Linux

I have my blueprint-storage.dat file as a symlink into another folder (it's a symlink into a git repo so that I can version my blueprint book). This has always worked fine in that Factorio, when saving blueprint book changes, would follow the symlink. But in recent versions (I jumped from 0.16 to 0.18) this is no longer the case, and the symlink is replaced with a file. Why is this and can it be restored to previous behavior? Thanks

[invisus]

I would love to do this too. However, my guess is that this changed sometime in 0.17.x around when 68499 was reported.

[posila]

Hmm, I thought it was also caused by our update to newer C++ which as it turned out has more nightmarish implementation of statndard filesystem library, but it turns out it is because of our WriteFileGuard, which saves file under temporary name, renames old file, renames new file to final name and then deletes the old file. We do this in order to not corrupt your good old file in case something goes wring while saving new one. But we use this for blueprint storage since 0.16.33

[dpacbach]

I think that you could continue using that copy/tmp methodology -- all you have to do is first resolve the symlink pointed to by blueprint-storage.dat. So logic to fix this would be like this:

Code: Select all

let f = "blueprint-storage.dat"
if on platform supporting symlinks {
  f = realpath(f) // resolve symlink if it is a symlink, otherwise no-op.
}
// Now just write the file using the existing tmp/copy methodology.
WriteFile(f)

@posila do you think it could be done?
David

[SilentWarrior]

Symlinks in Windows do exist. I wonder if this is a "thing" people do. As a programmer, this sounds ideal. Version your blueprint book. Although.. perhaps it should be part of the game itself.

[Koub]

Symlinks in Windows do exist. I wonder if this is a "thing" people do.

Symlinks were added to unix (BSD) somewhere in the mid 80s, and unix have always been a command line interfaced OSes. People using UNIX-like OSes have symlinks in their blood.
Microsoft-side, they were only added in Windows XP (2001), the command to create them with Windows Vista (2006) AND they only work on NTFS (I wouldn't be surprised some people still play on potatoes with FAT32 filesystem). Moreover, the command is command-line only, can't create it directly from an explorer Window.
People using Windows OSes are not "culturally" accustomed to fiddling with command-lines in a shell. I work in a moderately sized IT support team, and I'd be surprised anyone apart me in the ~100 techies has ever heard of symbolic links. And I only know of them because 1) I'm old, and 2) I've already used UNIX/linux OSes.

[ssilk]

In the moment when you export a Unix filesystem with many symlinks to a Windows, the dark rises.

[mrwacko]

I have observed this behavior on my two windows machines (10 and 11) as well, with the most recent stable version of factorio (1.1.60 as of this writing). I use symlinks to point at my relocated blueprint-storage.dat file, which is in a sync'd drive. Any time changes are made, the symlink is broken. If this could be changed back to intended symlink behavior that would be amazing, and thanks.

[ssilk]

When I understand this correct, this is — as already mentioned by Posila — because a file is not written to its real path, it’s written as tmpfile, then the old file is renamed, the new file is renamed to the old file and the old file is removed.

Correct way is explained by dpacbach.

I’m not sure, if this works in any case. I think this is not so simple. If the file is symlinked into a directory where it has no write-rights it cannot do this.

IMHO the better way to make a copy of the blueprint book so that a player can use it from anywhere would be to have some kind of cloud functionality.

[Nidan]

There are basically two options how to replace an existing file with a new version: a) truncate the old file and write the new content or b) write the new content into a new file and then move it on top of the old one.

a) preserves symlinks, but risks corruption in case of a crash. b) can recover from a crash since the old version is still around, but breaks symlinks. Therefore b) is often seen as the better approach.

Instead of only putting the blueprint file elsewhere, why not adjust the write-data path in config.ini so saves are included as well? Or configure your backup or versioning tool to only grab the one file and ignore the rest.

[SoShootMe]

There are basically two options how to replace an existing file with a new version: a) truncate the old file and write the new content or b) write the new content into a new file and then move it on top of the old one.

a) preserves symlinks, but risks corruption in case of a crash. b) can recover from a crash since the old version is still around, but breaks symlinks. Therefore b) is often seen as the better approach.

There is a bit more to the latter approach than just crash recovery. The fundamental benefit is that it allows the new file to be completely written before anything else happens. This obviously provides safety (eg if the writer crashes) - which is all that matters for Factorio. More generally, it also allows you to ensure a reader will never see a partially-written file and, with POSIX rename(), will see either old or new file (never no file).

Instead of only putting the blueprint file elsewhere, why not adjust the write-data path in config.ini so saves are included as well? Or configure your backup or versioning tool to only grab the one file and ignore the rest.

I think these are better (and actually more obvious) solutions than expecting Factorio to be changed to write through a symbolic link (risks corruption as mentioned) or use realpath() (why should it, when it "owns" the file?).