Save game file size + autosave lags

[squisher]

Even on my screamin' gaming rig + SSD drives, each autosave lags my computer momentarily and it gets annoyingly noticeable when the save file passes about the 10 MB mark.

Any tips / mods to clean up the save file? Or plans to fix this in the future? (Turning off autosave doesn't count!)

As an experiment, I tried scooping up all the doodads lying around on my conveyor belts, there must have been 10s of thousands of them, and put them in boxes. I figured storing the individual position of each doodad on the belts would be using up the most space, but it barely made a difference. Possible bug in the growing save file size?

[Jobeo]

Wowww is that what that periodic lag is... I was wondering all along. I was really thinking it was periodic garbage collection.

Now this is really making me think. Is it possible to customize the directory it is using for these saves? If so I will mount a ramdrive and point it to save there. How can I customize the path?

Thanks

[squisher]

Ha, didn't even think of trying a ramdisk. Saves are local to the factorio folder in the .zip version of factorio. (you can actually tweak it in the [path] section of config.ini) So then you can copy the whole Factorio folder to a ramdisk and run it from there, and saves will be on the ramdisk.

Bad news though, I got the same lag when saving/autosaving a game to ramdisk. A bunch of in-memory stuff going on before the save to disk, then.

[edit, woohoo, i'm now a burner inserter..]

[jerbearx]

I don't remember when, but one of the devs mentioned that they are looking into parallelizing the save process. I think it's needed to maintain determinism for the multiplayer.

[Jobeo]

Well I think this would be a very important thing to parallelize. I am thinking of turning down the frequency of autosaves to compensate in the meantime.

[Teurlinx]

That would be amazing. Currently it's still not too bad with a 30MB save and a momentary pause, but it's only going to get worse obviously. Would be great if it could be in the background entirely.

[ssilk]

Well. Look at the roadmap and think: When sometimes multiplayer should be possible, it can't work, if every player has to wait for those, who are currently saving. So it must and will be changed.

[squisher]

Well, think I figured out the primary cause of the save file size growing. I ran in a straight line with max exoskeletons and game speed 3, and checked out my save game size periodically. It made a sudden jump from 15 to 20 MB, then another leap from 20 to 29 MB. I would assume as I reached a certain threshold distance, the map would generate a new quadrant or whatever and store all of it. Looks like the preventative measure for now is.. turtle up in a tight little space and don't explore

[Chthon]

Assuming the game uses 3 bytes of data per block, 1 for the mineral value, 1 for the terrain type, and 1 for the features, this means that a 1024 x 1024 tile area is 9 megabytes in size. Without some means of compression this can get prohibitively large fast.

I think that along with their moving the save feature to a different thread, they need to find some way to compress the saves. It would make it more processor intensive to save, but in a separate thread it won't be an issue for most, and would greatly reduce the file size.

[Programmdude]

The saves are already slightly compressed, they are stored in a .zip file.
The easiest way to speed it up would be to save it in ram, then compress and save to the hdd on a separate thread. Not as quick as saving it all asynchronously, but the two major bottlenecks (hdd access and compression) would be asynchronous.

[GewaltSam]

Kind of a workaround for the time being: I just set the autosave interval to five minutes (somewhere in the settings, you can't miss it). In my opinion, that's more than enough; Factorio is such a drawn-out game, I don't need a new save every two minutes. When I do something dangerous, I save manually anyway. And a bit of a hiccup every 5 minutes isn't as noticeable and intruding as it is every 2 minutes.

Maybe that helps some of you guys.

[Jobeo]

Ya I bumped it down to once every 8 minutes and now it seems like much less of an issue

[CobraA1]

I'm not so much concerned about the file size (a few dozen MB in an era of GB and TB drives isn't gonna kill anybody's disk space) as much as the lag. Perhaps some sort of snapshot / copy-on-write system would allow saving the game's state while avoiding locking the entire system while the save is in progress.