Bikeshedding about non-blocking saving on Windows

[ptx0]

Non-blocking saving isn't possible to implement on windows in the same way it works on the other platforms. Windows does not have process forking.

it has been implemented via Cygwin (and i presume subsystem for Linux but you cant guarantee its presence or featureset), and the details of those two implementations are probably pretty equally shitty: http://cygwin.com/cygwin-ug-net/highlig ... hi-process

[Jap2.0]

Non-blocking saving isn't possible to implement on windows in the same way it works on the other platforms. Windows does not have process forking.

I'm not an expert on Windows APIs by any means, but apparently there is ZwCreateProcess which is similar. (example 1, example 2 (original) - first is GPLv2 and the second is some obscure French license so I'm not sure if you can directly use either of them, but they exist.)

Granted it's technically undocumented (except for some random book that seems to have a lot of Windows API documentation from ~2000 that I keep seeing mentioned) so that's not ideal but Windows tends to have pretty stable APIs (...for the most part, see below).

Edit: example 1 was released under MIT so that should be fine (1/2)

Edit 2: Apparently at least the first has issues on newer than Vista. But it might be possible. Maybe. Probably still not something you want to get into, but just in case there's enough interest.

[ssilk]

Non-blocking saving isn't possible to implement on windows in the same way it works on the other platforms. Windows does not have process forking.

I would not say impossible - just as said: much more complicated to implement, because reimplementing fork is hard. But maybe there are other ways...

[Optera]

Almost all non-blocking windows backups use the built in shadow copy feature.

[Oktokolo]

Almost all non-blocking windows backups use the built in shadow copy feature.

Shadow copy can create snapshots of files and volumes. To use that with a Factorio game state, it has first to become a file...

[Optera]

Almost all non-blocking windows backups use the built in shadow copy feature.

Shadow copy can create snapshots of files and volumes. To use that with a Factorio game state, it has first to become a file...

Ah so it's writing memory to file blocking in Windows. Nevermind me then.

[posila]

Ah so it's writing memory to file blocking in Windows. Nevermind me then.

It is blocking, because the update thread can't be doing changes to game state while saving thread is traversing it and serializing it. On POSIX OSes, the game process is forked - the process is duplicated with almost exact state - only difference is that the original (parent) and duplicate (child) know which is which. Parent and child process share all parent's existing memory in copy on write mode, which means the OS will make new copies of memory pages for the parent and the child process as either of them starts making changes in the shared memory.

[ptx0]

Ah so it's writing memory to file blocking in Windows. Nevermind me then.

It is blocking, because the update thread can't be doing changes to game state while saving thread is traversing it and serializing it. On POSIX OSes, the game process is forked - the process is duplicated with almost exact state - only difference is that the original (parent) and duplicate (child) know which is which. Parent and child process share all parent's existing memory in copy on write mode, which means the OS will make new copies of memory pages for the parent and the child process as either of them starts making changes in the shared memory.

yeah the problem i mentioned in my original post was that fork() reimplementations on windows can not be copy-on-write, the windows kernel does not have any mechanism for this. it *must* copy the entire process memory space into a new address space and memcpy is expensive - it gets worse the larger the working set is. it's a problem that Windows must solve, not Wube.

[Oktokolo]

yeah the problem i mentioned in my original post was that fork() reimplementations on windows can not be copy-on-write, the windows kernel does not have any mechanism for this. it *must* copy the entire process memory space into a new address space and memcpy is expensive - it gets worse the larger the working set is. it's a problem that Windows must solve, not Wube.

The naive way to solve it, is to treat game states as immutable. The game state update code would take the old game state and create a new one including the updates.
After the update, the old state normally is discarded.
The game state save code would take an existing game state and just keep it until it is done with saving it to disk.

And if the last N game states would be kept in memory, a game state debugger could go back N game states in time. But that obviously only becomes real useful with key game states and incremental game state change overlays - or obscene amounts of RAM...

[ptx0]

yeah the problem i mentioned in my original post was that fork() reimplementations on windows can not be copy-on-write, the windows kernel does not have any mechanism for this. it *must* copy the entire process memory space into a new address space and memcpy is expensive - it gets worse the larger the working set is. it's a problem that Windows must solve, not Wube.

The naive way to solve it, is to treat game states as immutable. The game state update code would take the old game state and create a new one including the updates.
After the update, the old state normally is discarded.
The game state save code would take an existing game state and just keep it until it is done with saving it to disk.

that's what it means when i say it's not copy-on-write.

windows has to make a complete copy of all memory space for a process, and maps can use under vanilla gameplay about 6-8GiB memory meaning duplicating all of that every time you save the game.

it's also not a free process, copying memory is expensive. it's going to cause a pause so that the game state can be made immutable while it is copied into new address space. you can't avoid the pause, because it will corrupt the game state, without adding some stupid form of journaling into the game state that will undoubtedly impact performance for just one very small reason.

i mean, something you could argue is that you might want an option to enable saving under any exit condition to a temporary save file, like $SAVENAME"-corrupted-$(date +%s)". why do we save? to revert to autosave if the game crashes, mostly, for me - not like vanilla gameplay crashes much, but with mods, whoo boy.

sometimes it'll exit without any option to save. sometimes it allows you to save. i'm not sure why it does sometimes and doesn't others - and sometimes it allows SOME players to save the map, but not the one who experienced the error - and this isn't useful in single player.

so maybe if we can get that resolved and the game state can be saved any time the game unexpectedly exits then autosave can be simply disabled in more cases since there will be some form of obtaining the new map when Things Go Wrong.

[coppercoil]

Ok, copying RAM is expensive. Which is less expensive: full serialization and compressing like now, or copying the memory? If there’s no non-blocking save, can we have a shorter freeze?

[Oktokolo]

Ok, copying RAM is expensive. Which is less expensive: full serialization and compressing like now, or copying the memory? If there’s no non-blocking save, can we have a shorter freeze?

Copying RAM is orders of magnitudes faster than storing stuff to disk.
So yes, just copying the gamne state, then continue running the game while also doing the actual saving of the copy in background is a viable strategy to reduce the freeze to less than a second.

[ptx0]

Ok, copying RAM is expensive. Which is less expensive: full serialization and compressing like now, or copying the memory? If there’s no non-blocking save, can we have a shorter freeze?

Copying RAM is orders of magnitudes faster than storing stuff to disk.
So yes, just copying the gamne state, then continue running the game while also doing the actual saving of the copy in background is a viable strategy to reduce the freeze to less than a second.

i dunno, copying once is less expensive. but i see your point. nothing's really going to get around the serialisation pause other than a full memory copy, and sometimes a memory copy pause might be preferred. of course, they tell me this is never going to happen before v1.0.

[Rseding91]

Ok, copying RAM is expensive. Which is less expensive: full serialization and compressing like now, or copying the memory? If there’s no non-blocking save, can we have a shorter freeze?

Copying RAM is orders of magnitudes faster than storing stuff to disk.
So yes, just copying the gamne state, then continue running the game while also doing the actual saving of the copy in background is a viable strategy to reduce the freeze to less than a second.

Currently the "write to disk" happens in parallel with copying. So it doesn't actually impact saving that much - about 15-20% savings if you just skip the write-to-disk completely. And a portion of that comes from not having to save the copied memory anywhere - meaning it would be slower still if it *did* actually have to save it fully in memory for it to finally get flushed to disk.

Reading all of those random bits of memory is that expensive (AKA: copying memory is expensive).

[coppercoil]

Currently the "write to disk" happens in parallel with copying.

Great, it seems it’s already optimized. Of course, I won’t stop thinking
What if the copy/serialization would be performed slowly in the background without locking the game, leaving empty placeholders for some "hot" values, which would be stored separately in the final general lock? If copying is expensive, let’s copy just a fraction.
In other words, there can be frequently-changing values and rarely-changing values.

[ssilk]

Then you need to manage what you already have copied at which state and how actual it is.

Nothing more to say.

[ptx0]

it's really not hard to run Linux, that's the real solution here. running a server on windows is a bit of a novelty since it doesn't even support basic server functionality..

[CheeseMcBurger]

running a server on windows is a bit of a novelty since it doesn't even support basic server functionality..

What an ignorant statement.

[ssilk]

I think ptx0 means it in context of Factorio. Why do you mean he is wrong with it?

[ptx0]

running a server on windows is a bit of a novelty since it doesn't even support basic server functionality..

What an ignorant statement.

i would love for you to back that "insult" up with some evidence, as i've been trying to do all along.

if you ask someone who is used to POSIX programming standards - not having COPY ON WRITE makes Windows fundamentally broken for efficient memory management.