Factorio Forums

fork sounds like a good idea on the surface... but then it marks ALL pages in source and destination as copy on write, so as the game continues to run and update the state in memory, all of those pages have to get a new map and get a copy. It's not just the child process that copies on write, otherwise it would get a invalid page.

Synchronizing it such that nothing is happening ( nothing in any lua scripts doing anything at the time I would think would be more tricky.

Even if the child process is mostly a read only process, once it exits, all of the pages in the parent are still marked copy on write. (except those that the parent updated while updating the game state) you'd get a significant stutter in the parent process after the fork.

whether sounds are playing or not is really irrelavent to the child and none of the threads should really have to be forked... from its own entry point it could just start scanning memory and building an image to save... probably internaly there is a point like beginning to render the display where things have completed and would be a good time to fork for consistent game-state data.

---
slowness is probably due to 'good coding practices' that associate serialization methods with their classes; this would cause lots of calls that move tiny tiny bits instead of having a separate state loop that can just iterate through the data without calls. A call is much likely to be a cpu cache hit than a jump in a loop; though even jumps cause pipeline flushses.

maybe once it's closer to 1.0 things can be migrated to more of a dedicated serialize-everything sort of routine instead of billions of tiny calls.

My own image library started as something resembling allegro too; one of the first things I did was to replace 'target any pixel type' to 'everything internally is 32 bit pixels' and then instead of calling 'compute pixel' for every operation, moved to for( y) for( x ) (*output++) = (*input++) (to even remove the computed index into the arrays, or even simple offset index into arrays) to copy an image for instance... a minor varation to test if( input alpha is 255 copy, else do a computation for alpha) instead of calling a routine to compute the alpha increased performance immensely. Hooray for #define.

I think that using virtual memory's copy on write functionality within the same process is a better approach, it has less overhead, plus both Linux & Windows have APIs to manipulate the virtual memory protection flags within your own process (mmap/VirtualProtect).

Implementing this copy on write thing could involve setting the entire heap as COW, and having the background save thread add a fixed offset to every single pointer involved, to avoid accidentally reading the live data. I suggest implementing a custom pointer wrapper template class that uses the address where the pointer is stored (or use thread local storage) to determine what offset to add when dereferencing. Desync bugs from forgetting to use the pointer class can be caught by deleting the main thread's copy of the game state (and suspending that thread as well) - any segmentation faults will quickly isolate the bug. The game state can then get restored from the save.

The cost of setting a range of pages to copy on write amounts to a write to a single entry within the Virtual Address Descriptor - its O(1). On Windows Server 2008 and below (IIRC), reserving memory pages is O(N), Windows 7 and above (again, IIRC), its O(1).

d3x0r wrote: Synchronizing it such that nothing is happening ( nothing in any lua scripts doing anything at the time I would think would be more tricky.

There is no need for thread synchronization with copy-on-write. The main thread and the background save thread each get their own private copy of the memory storing the game state. The first thread that makes a change to the copy on write memory causes the operating system to make private copy of it first before committing the write to memory. Since the background saving thread won't ever see any of the writes done by the main thread, and the data its reading is unchanging, there is no possibility of a race condition. And since the two threads are in the same process, signalling the main thread can be accomplished by writing to a volatile bool... without synchronizing.

d3x0r wrote: Even if the child process is mostly a read only process, once it exits, all of the pages in the parent are still marked copy on write. (except those that the parent updated while updating the game state) you'd get a significant stutter in the parent process after the fork.

In Windows, the Virtual Address Descriptor also contains information on the # of shared copies of a section of memory. If there's only 1 copy (because the background save is done and its memory view was unmapped), then there's no longer a need for the operating system to copy the page before writing. And during the actual saving, it's a lot faster for the processor to copy 4KB of contiguous memory at a time than it is for it to randomly access memory while serializing the game state. With RAM speeds being in excess of 10 GB/s, and Factorio's memory usage being under 100MB, you can make a full copy of everything in under 10ms - in practice it'll be much faster since most of that memory isn't actually changed often, if at all (like graphics & sounds). The only thing to beware of is that the copies of the pointers will all point into the main thread's game state and not the copy.