Excessive savefile load times

[MisterFister]

I initially posted at 113149. I dropped Factorio in April of 2023, back at base v1.1.76, with a savefile that was 210mb and had 415 hours on it. Earlier this year, I'd performed a system re-format, and was running Factorio v1.1.107, with roughly a dozen or so mods all also freshly reinstalled by syncing to the savefile. The blueprint library wouldn't load, but BrainGamer_ [user_id=155933] (thank you) kindly pointed me to 94427 which did almost-entirely solve my blueprint library corruption issue.

Even having solved the initial baseball-bat-to-the-face symptom of the corrupted bp library, something squirrelly is still definitely going on with my game install. Relevant here is the fact that there was nothing wrong with the savefile, and I'm essentially certain that nothing was corrupted as I migrated through a system reformat. To be clear, my savefile was backup up on at least 2 different clouds (Steam and Dropbox) and I was able to verify that both of those clouds had identical copies of not only the savefiles, but also of blueprint-storage.dat.

TLDR, extremely long load times on savefiles that ran approx 225mb before and after the reformat, before game runtime ram footprint was ~5-8gb with 50+ fps, whereas now it runs consistently with 7-10gb of ram used, often averaging 25-30 fps but frequent dips into low-single-digit fps. Savefile had 415 hours pre-reformat, and almost 35 hours since recovering my bp library. Admittedly, I'd revealed the map to a pretty sizeable radius, I'm guessing ~300 chunks in all four directions from spawn, but that was true both before and after the reformat so no idea why the change in game-client performance upon my return. To eliminate as many variables as possible, I'd culled my modlist pretty significantly and even opted to reroll a new map, and I'm ~75 minutes into the new save (new mapseed, everything) and even that savefile is already 100mb in size, factorio is using 3.5gb of ram, and it takes almost 8-10 minutes to load into this new save, again with <90 minutes on the save.

System:

• Lenovo IdeaPad3 15IAH7, Windows 11-Home-64bit v23H2
• CPU i7-12650H 2.30GHz
• RAM 16.0gb, 15.7gb usable
• SSD 421gb, currently 50+gb free, currently work assignment soon to conclude this month and when it does that should free up 85-100gb of ssd space

Factorio: according to savefile load-selection screen for brand new savefile

• base v1.1.107-0
• EvoGUI 0.4.601
• miniloader 1.15.7 (*strong preference to keep but can be persuaded if performance impact is sizeable)
• VehicleSnap 1.18.5
• angelsaddons-cab 0.2.9
• angelsaddons-storage 0.0.10
• AutoDeconstruct 0.3.12 (*)
• blueprint_flip_and_turn 101.8.6 (**see below)
• camedo-snapmine 1.0.0
• CleanedConcrete 1.0.4 (I really like this mod!)
• creative-mod 1.8.2 (**)
• LandfillEverything 1.1.2 (**)
• nixie-tubes 1.1.5 (*)
• rso-mod 6.2.26 (*)
• textplates 0.6.10 (*)
• TransformationProcess 1.0.1 (**)
• VehicleGrid 1.4.0 (*)

This very visit to the forums was in reaction to realizing that some mods weren't listed here:

• LTN - Logistic Train Network 1.18.6 (*, **)
• Nanobots: Early Bots 3.2.19 (*, **)
• Picker Dollies 1.2.6 (*, **)
• SnapMines 1.0.0
• VehicleWagon2 3.1.2 (**)

This was due to having disabled certain dependency-library mods which I could've sworn I'd re-enabled prior to creating my new save:

• Factorio Library 0.14.1 (req'd by LTN)
• Factorio Standard Library 1.4.8 (req'd by Nanobots, Picker Dollies
• Robot256's Library 1.1.5 (req'd by VehicleWagon2)
• NFC why SnapMines isn't appearing on the load-select screen, as the mod page indicates no dep's

After typing this, I'll return to that troubleshooting, but I'm shallow enough into the new save that I'm perfectly willing to nuke it and re-roll yet again with a verified mod list.

** -- I'm vaguely aware of some UPS issues with creative-mod, but I'm nervous to part with it, as I run a parallel creative-save for blueprinting and design-tinkering purposes, which is kind of a medium-sized deal for a multiplayer session being planned. I may end up disabling this mod for the multiplayer sessions, only enabling it when I need to access the creative save. Same for blueprint_flip_and_turn and LandfillEverything as those are almost never used in survival gameplay.

Group first-time convenes online late this upcoming week, hence my troubleshooting now. For that multiplayer map, the strong preference is to add:

• AfraidOfTheDark
• Atomic Artillery
• ExpensiveRepairPacks
• NaturalEvolution
• Rampant
• Recipe Recurrence (weak preference)
• Recursive Science
• Science Bottles

Mods I kept during the recent wave of removals, but am still considering removing:

• EvoGUI
• Nanobots
• SnapMines
• Text Plates

I'm aware of f4 and f5, but I genuinely don't know how to identify which mod might be causing in-game issues (save and autosave times are slower, but nowhere near the problem for load times.) And of course, f4/f5 info isn't available during game load anyway... Any suggestions on how to identify what, precisely, is the issue with loading these saves would be appreciated. And again, I'll also note that I have no idea why my new save, with <90 of playtime, is already 110mb in size.

[FuryoftheStars]

If you happen to have a second computer, you can try loading the save on that one, too, just as a way of determining if it's something up with your computer, install, or the save itself. Alternatively, post your save for someone else to download and test on their own computer. Anything further without a save, unless someone happens to know the answer, I believe would just be stabs in the dark.

If it can be determined that it is the save and no one else has any ideas, you could try posting your save to here: viewtopic.php?f=5&t=17501

There was just recently someone else who did this trying to figure out why their save was using 22 gb of ram and the devs found one of the players from it had an excessive number of blueprints nested in their inventory (potential griefing).

[Rseding91]

Can you please post he save file(s) mentioned here? Or post on google drive, dropbox, onedrive and link here.

[Silari]

** -- I'm vaguely aware of some UPS issues with creative-mod, but I'm nervous to part with it, as I run a parallel creative-save for blueprinting and design-tinkering purposes, which is kind of a medium-sized deal for a multiplayer session being planned. I may end up disabling this mod for the multiplayer sessions, only enabling it when I need to access the creative save.

Those are runtime UPS costs, not save/load time costs. It's not saving that much stuff to global so it shouldn't be affecting that much, and even less if you're using 1.8.1 or newer. Most of the UPS issues should be fixed too. Most of the old entities that relied heavily on scripting were switched to using in-engine items that did the same thing - ie using infinity chests for all the creative/void chests. Most of the ones that are still there have alternatives available - ie using a loader and void chest to remove items from belts instead of the matter void.

[MisterFister]

** -- I'm vaguely aware of some UPS issues with creative-mod, but I'm nervous to part with it, as I run a parallel creative-save for blueprinting and design-tinkering purposes, which is kind of a medium-sized deal for a multiplayer session being planned. I may end up disabling this mod for the multiplayer sessions, only enabling it when I need to access the creative save.

Those are runtime UPS costs, not save/load time costs. It's not saving that much stuff to global so it shouldn't be affecting that much, and even less if you're using 1.8.1 or newer. Most of the UPS issues should be fixed too. Most of the old entities that relied heavily on scripting were switched to using in-engine items that did the same thing - ie using infinity chests for all the creative/void chests. Most of the ones that are still there have alternatives available - ie using a loader and void chest to remove items from belts instead of the matter void.

Good to know, thanks. Related question: disabling a mod, but leaving the files installed and available to be re-enabled -- does that fully eliminate troubleshooting variables, i.e. does this prevent "contamination" of the install for any troubleshooting purpose that might or might not be relevant for my issue here? In other words, is there any benefit to fully removing the mod as opposed to simply un-enabling it within the game client mod portal? And for that matter, is there any difference between hard-removing-uninstalling the mod through the game-client, as opposed to manually removing the relevant subfolders from outside of the game-client?

Can you please post he save file(s) mentioned here? Or post on google drive, dropbox, onedrive and link here.

I did anticipate this request, but I apologize for my delay in getting back to you.

https://www.dropbox.com/scl/fo/2orboj53 ... xeaj9&dl=0

This links to a dedicated folder that I've set aside for this instance. Files are actively uploading and syncing to dropbox now. Note the subfolder organization:

• 01 Old Creative -- copies of my creative-save, recovered from cloud from before my system reformat. Dropbox and Steam-cloud versions were identical and indistinguishable, so I cannot determine precisely which source yielded these here, but I do know that they all came as a group from the same cloud. This save was only used occasionally, but usually for extended sessions, as I'd prefer to accrue a list of things to address in creative mode all in the same batch before applying them to my solo gameplay. My hiatus from Factorio was from April2023 to April2024; note filenames indicating before and after that hiatus. May2024 gameplay progress was after having succeeded in addressing my original issue of blueprint library corruption.
• 02 New Creative -- Even after addressing library corruption, I was dissatisfied with game performance. The symptoms were roughly equal as between my solo and creative saves, but more of a hindrance to creative-save gameplay was some sort of periodic momentary or intermittent screen-scrambling. Never for more than an instant, frankly it took me a while to realize that I wasn't imagining it. I decided that maybe the next step was simply to create a new map, with the current game version, on a clean install with a pruned mod list where every mod was freshly installed and known to be up to date. With 400+ hours invested in the solo save, the obvious choice was to re-roll the creative save first, as it was a simpler map to recreate and get back up to the state that I'd had it already. My process was to roll the map with no terrain, no resources, no trees, no grass, no biters, no expansion, no pollution, nothing. Then, with creative-mod, to cheat-radar-reveal a 50-chunk radius around spawn, replace all land tiles with deep-water tiles (to make it easier to record blueprints with embedded landfill-tile information) and then to array the blueprint builds as I wanted them laid out, with the focus being on a large plot of featureless landfill in the middle for "whiteboarding" purposes. After doing all of this, I noted that the savefile was <10mb in size!!! It only grew to 125mb in size after adding my blueprint strings and build-entities into the map.
• 03 Old Solo Survival -- Same as 01, for the solo map. 2024 progress, again, was after addressing library corruption. This save did have the intermittent screen-scramble issue, but do to in-game activity bringing baseline fps to ~30, it was easy to initially presume that I was either imagining it entirely, or that it was a result of savefile size / age independent of other issues.
• 04 New Solo -- With 400+ hours invested in the solo save, I was very reluctant to abandon and re-roll. This, coupled with the fact that early burner-stage gameplay is less exciting to me (no offense to the devs!) Both rerolls were to brand new mapseeds, and the two new rerolls do not share a mapseed.
• 05 New Settings -- I am 90% certain that rerolling did eliminate my screen-scramble issues in both savefiles, but this only brought further emphasis to my excessive load times. I will note, however, that gameplay in the new rerolls is indeed much smoother now, with that earlygame crisp 55-60 fps baseline that is very nice to have of course, despite these load times. It was at this point that I noted the savefile sizes, and I recalled how small the new savefile was for my creative reroll. The solo reroll, however, consistently generated ~100mb sizes, regardless of map settings. These various attempts were all recorded with <5seconds gameplay each, all intentionally using the same mapseed base as the new solo save that I had accrued ~90 minutes into. Load times on these "pristine" saves is also frustratingly slow.
• 06 Blueprint Library -- For the sake of thoroughness, this is a copy of my appdata\roaming\factorio files, one copied from my cloud (again, assessed to have been identical between the two cloud platforms, unsure which platform furnished this copy) and the other as a current snapshot.

Thank you.

[Rseding91]

The issue is your blueprint library. It's *HUGE* - 613 megabytes. Every time you load a save file (or create one) the game starts transferring the library into the save so the contents can be used.

You can resolve the issue by trimming some 95% of what you have in the library. Or just delete it completely.

[FuryoftheStars]

Related question: disabling a mod, but leaving the files installed and available to be re-enabled -- does that fully eliminate troubleshooting variables, i.e. does this prevent "contamination" of the install for any troubleshooting purpose that might or might not be relevant for my issue here? In other words, is there any benefit to fully removing the mod as opposed to simply un-enabling it within the game client mod portal? And for that matter, is there any difference between hard-removing-uninstalling the mod through the game-client, as opposed to manually removing the relevant subfolders from outside of the game-client?

In this game, a mod must be enabled to have any affect on your game (I don't believe an enabled mod can even access the resources (graphics, scripts, etc) of a disabled mod). It's not like other games where simply installing the mod enables it (well, except a game setting that is for literally enabling a mod after installing, which you can disable) or makes it available, etc. As such, too, disabling it completely removes its affects from your game, though this may leave your game in a weird or broken state, depending on what it was doing, though mind you technically it's not unfixable by simply playing and redesigning everything to accommodate the loss of the disabled mod (ie, a mod that makes locomotives use fluid fuels would leave trains stranded after being disabled and would require redesigning your fuel depot and then possibly going around to manually insert some fuel into each until they can right themselves). Disabling a mod also cannot affect a save file itself. The changes that must be (un)done from disabling a mod happen during game load and can only affect your save file if you then save, overwriting the original.

[MisterFister]

The issue is your blueprint library. It's *HUGE* - 613 megabytes. Every time you load a save file (or create one) the game starts transferring the library into the save so the contents can be used.

You can resolve the issue by trimming some 95% of what you have in the library. Or just delete it completely.

I knew it was big, but I didn't know it was THAT big. Wow.

Is there any way to sock it away offline, so as to delete it from my Factorio directly, but retaining the possibility of re-accessing it in the future if need be?

... wait just a damn second. Before my migration it was <65mb. wtf did I do...? (I'll continue looking into this to report back on what I find, but thank you. Bit of a forehead slapper moment, that info was staring me in the face.)

[FuryoftheStars]

Is there any way to sock it away offline, so as to delete it from my Factorio directly, but retaining the possibility of re-accessing it in the future if need be?

Create a copy of the file and rename the copy.

[fusionfan]

The issue is your blueprint library. It's *HUGE* - 613 megabytes. Every time you load a save file (or create one) the game starts transferring the library into the save so the contents can be used.

You can resolve the issue by trimming some 95% of what you have in the library. Or just delete it completely.

A naive question - is the blueprint library read from disk every time a save is loaded? I.e. when I'm in a game, and the blueprint library is already loaded, and I load a save from within a save (e.g. because I got killed by biters), is the blueprint library reloaded from disk? I could understand big BP libraries impacting load times at first, but not repeatedly...?