Factorio Forums

Been watching new streamers and people being effected by this. The game is already overwhelming and some people are unable to find the settings and get very annoyed by this. A lot them end up just disabling once they find it. If it was adjusted to say 15min I think it would be a lot less painful for new players.

every 2min auto save has spared me from lot of trouble, sometimes it's amazing how much you can do just 2 mins. Will adjust auto-save to 5 min or so since my current big map takes time to save... sadly.

Kane wrote:Been watching new streamers and people being effected by this. The game is already overwhelming and some people are unable to find the settings and get very annoyed by this. A lot them end up just disabling once they find it. If it was adjusted to say 15min I think it would be a lot less painful for new players.

These settings are pretty much in the place where every single game has them: under the Options choice in the game's pause menu. Maybe the 'other settings' menu could do with a bit of work (perhaps splitting off multiplayer related settings to its own menu, and hiding a few under an advanced button), but I don't think changing the autosave interval is the way to go. That will most certainly lead to a lot more frustration when someone dies at the 14 minute mark and loses a LOT of progress.

Making autosave unremarkable into background is per sure a wish of all devs, especially Kovarex. That's in my eyes the way to go and the devs also want it to go.

If you think, that the autosave-option is difficult to find, then make a suggestion where it should be found better. But I really think it's a bad idea to reduce the default auto-save time, because why even have saves then? 15 minutes is quite a lot of time and one biter can kill you.

(Off-topic
And to be honest: someone, who makes videos of how he plays games doesn't find a simple option? Come on: Is that worth watching - I mean: What did he want us to show, unprepared, knowing obviously nothing, and why does he expect us watching him searching that?
)

Here is a spin around. With recent corruption save fixes is 6 mins to access all your backups of that save consider appropriate?
Sometimes you want more or longer backups in age. I think 2min is just crazy too long that is all but it's just opinion based hehe. It seems more frustrating having it come up a lot but then again threaded background saving is the future so who knows

I actually personally run 30 min 10 diff auto saves hehe.

If I also die I don't restore the characters lost items.

Making autosave unremarkable into background is per sure a wish of all devs, especially Kovarex. That's in my eyes the way to go and the devs also want it to go.

It certainly won't be easy. The entire game state has to be captured at once, processed into a saveable format, and then written to disk. The latter two tasks can happen on their own, but the first task HAS to be done with the game in a synchronized state. That means being paused while the data is captured.

If you don't mind blowing a little extra RAM, the game state can be cloned and split off to a separate task. This would allow the game to resume fairly quickly, but it can be a huge surge of extra RAM and baby systems might not like that.

bobucles wrote: If you don't mind blowing a little extra RAM, the game state can be cloned and split off to a separate task. This would allow the game to resume fairly quickly, but it can be a huge surge of extra RAM and baby systems might not like that.

The amount of ram required really depends on your save and mods so factorio can easily use +3GB of memory and even a good pc might not be able to handle that when doubled.

Save files aren't 3GB. They're closer to a few dozen megs uncompressed. But the challenge of figuring out if cloning is the way to go is a matter for the devs, I suppose.

I change my autosave time depending on the stability of the version. When a new major version is released, I set it to like 2 min. As the version becomes more stable, I increase the time. As ssilk stated, if the youtuber doesn't even know how to change this setting, shame on him.

bobucles wrote:

Making autosave unremarkable into background is per sure a wish of all devs, especially Kovarex. That's in my eyes the way to go and the devs also want it to go.

It certainly won't be easy. The entire game state has to be captured at once, processed into a saveable format, and then written to disk. The latter two tasks can happen on their own, but the first task HAS to be done with the game in a synchronized state. That means being paused while the data is captured.

If you don't mind blowing a little extra RAM, the game state can be cloned and split off to a separate task. This would allow the game to resume fairly quickly, but it can be a huge surge of extra RAM and baby systems might not like that.

Factorio running on my system right now is ~1.5 Gb Virtual, ~1.3 Gb Working Set. So that means that most of the game is already in chip memory, as opposed to being paged out to the swap file. Making a clone would only make sense if the bulk of the time required for the pause is currently needed to "serialize" the data, i.e. convert it to a writeable format and / to actually write it to disk. If the working state can be cloned to a buffer in a relatively short length of time, then this would be worth doing. That's something that the Devs would need to profile before even thinking about doing this.

Sure, while that second buffer was present the game's memory footprint could get as large as doubling (i.e. ~3.0Gb Virtual), and the Working Set would almost certainly spike, although most likely not by as much. Once the copy was done, the convert and write to disk could be done lazily in another thread. Any machine with 8Gb or more of memory should be able to handle this, Windows is remarkably good at managing memory, better than some people give it credit for.

dgnuff wrote:

bobucles wrote:

Making autosave unremarkable into background is per sure a wish of all devs, especially Kovarex. That's in my eyes the way to go and the devs also want it to go.

It certainly won't be easy. The entire game state has to be captured at once, processed into a saveable format, and then written to disk. The latter two tasks can happen on their own, but the first task HAS to be done with the game in a synchronized state. That means being paused while the data is captured.

If you don't mind blowing a little extra RAM, the game state can be cloned and split off to a separate task. This would allow the game to resume fairly quickly, but it can be a huge surge of extra RAM and baby systems might not like that.

Factorio running on my system right now is ~1.5 Gb Virtual, ~1.3 Gb Working Set. So that means that most of the game is already in chip memory, as opposed to being paged out to the swap file. Making a clone would only make sense if the bulk of the time required for the pause is currently needed to "serialize" the data, i.e. convert it to a writeable format and / to actually write it to disk. If the working state can be cloned to a buffer in a relatively short length of time, then this would be worth doing. That's something that the Devs would need to profile before even thinking about doing this.

Sure, while that second buffer was present the game's memory footprint could get as large as doubling (i.e. ~3.0Gb Virtual), and the Working Set would almost certainly spike, although most likely not by as much. Once the copy was done, the convert and write to disk could be done lazily in another thread. Any machine with 8Gb or more of memory should be able to handle this, Windows is remarkably good at managing memory, better than some people give it credit for.

Saving already works this way:

20 x 1 MB buffers are allocated and then the save file is serialized into a buffer 1 at a time until it's full. When a buffer is full it's swapped over to another thread which in parallel with the main saving thread compresses and writes it to disk.

The time for a 50 MB zipped save file is roughly 5 seconds. That time is split like this:

• 2.8 seconds copying the game state into buffers
• 2.2 seconds compressing and writing to disk

The time difference being measured by commenting out the compressing and writing to disk portion and seeing how long it takes to do the main copy-game-state-to-buffer.
The game state is scattered across memory and in random order compared to what ends up getting saved which means a ton of random access and it ends up waiting on RAM.

We can't just "clone" the main programs memory and write that to disk because the vast majority of it is not something that should be saved else your save file would end up roughly the same size as the RAM it takes to run.

No problem with the 2 min auto save here. Currently have used this feature to save my bacon way too often

Rseding91 wrote:
We can't just "clone" the main programs memory and write that to disk because the vast majority of it is not something that should be saved else your save file would end up roughly the same size as the RAM it takes to run.

My understanding of the suggestion was to prepend a full imaging process onto the existing system. That is, you quickly blit an image of the entire game-state, then use that as a reference for doing a proper serialization procedure. This does have the down-side of doubling the memory requirement of the game, as well as potentially taking a non-trivial amount of time, especially if you have memory fragmentation issues and can't just copy a huge contiguous block.

Your outlined process does give me an interesting thought though:

1. Can the game resume while buffers are still in the compressing & writing stage? (but the filling process has completed).
2. If the above is a "yes" (which I would expect it should be), would it work to increase the number of buffers -- like, by a lot? If the system in question has enough memory (Both my desktop and laptop I play Factorio on are 16GB, and that's not too unusual), I can't see a reason not to allocate enough buffers to allow the game-state-copy process to run unblocked. At that point the compress-and-write process can asynchronously complete.

From your quickie-benchmark, it would appear that such a process could nearly double the speed of game-saving.

Probably but it would take a lot of reworking that we don't want to do right now.

Mostly because if you hit save, it finished and then clicked exit it would still be saving and would have to block until that finished else it would corrupt the save.

Rseding91 wrote:Probably but it would take a lot of reworking that we don't want to do right now.

Mostly because if you hit save, it finished and then clicked exit it would still be saving and would have to block until that finished else it would corrupt the save.

Fair enough. I suspect your architecture makes this a little more complex than throwing a semaphore-wait before quitting.

I'll put it with "make the game gracefully handle attempting to save to a suddenly unrwitable target" on my save-game-handling wishlist

Wow, Factorio is actually remarkably resilient against save-file write issues.

E: I partially take that back. With my "too small FS" test, it segfaulted after a few times of well-mannered "out of disk space" errors. I'm not going to file a proper bug report though (unless you would like it), because "user decided to mount a 1MB tmpfs onto saves/" is probably not something you all want to waste time on.

On Linux you can in principle just fork() at the point the save occurs, and have the child process do the actual saving while the main process immediately carries on playing the game - letting the kernel deal with doing the copy-on-write work to duplicate the state where necessary.

This would require that the point where you trigger the save is very controlled (no other threads actively running or holding locks, for a start), and it's going to have an impact on the performance and memory usage of the game during the save since every page it writes to will have to first trigger a page allocation and copy. It's likely to be simpler and use less memory than actually creating a second copy of the state inside the game, though. Debatable whether this is useful or not

On OSX you can probably do the same since the kernel also implements fork(), but I've never actually programmed for OSX and don't know if there's any reason why this kind of program can't easily fork that's mac-specific. On Windows there's not really a sane way to fork; it's possible to fake it in certain cases but it comes with a lot of caveats.

We had that already discussed. And other aspects of this thread.
Here the threads; interested should read before continuing:

viewtopic.php?f=6&t=24242 Use fork() on *nix systems for doing save game
viewtopic.php?f=6&t=25640 Save in Background
And of course this topic.

(And there is a large list of ideas about save/autosave etc. - indeed it's so much written it's worth an own topic)

15 is extreme for a default. I would recommend a 4 at the max.

It barely takes any time for my save to go through, but I pointed my Windows Documents to be on my SSD..
I've noticed quite a few gamers will have their My Documents on their HDD vs the SSD, If you want faster saves, you'll want to fix that.

Additionally you could Setup a Ram Disk if you really want extra performance for the save, but that's only if you do not have an SSD

OkariDraconis wrote:15 is extreme for a default. I would recommend a 4 at the max.

It barely takes any time for my save to go through, but I pointed my Windows Documents to be on my SSD..
I've noticed quite a few gamers will have their My Documents on their HDD vs the SSD, If you want faster saves, you'll want to fix that.

Additionally you could Setup a Ram Disk if you really want extra performance for the save, but that's only if you do not have an SSD

The time spent waiting on the disk is negligible in how long it takes to actually save the game. The OS already buffers writes in RAM.