Extreme RAM usage and memory leak

[AndrolGenhald]

Pic says it all, a little over 20 GiB just by loading the save. Mods attached, save is too large to upload, caused nginx error page, it's at http://www.AndrolGenhald.com/factorio_ram_bug.zip for the time being.

RAM.png (62.58 KiB) Viewed 4885 times

Wow, I didn't realize how bad XFCE's default screenshot software was till I used it on a moving graph.

[Rseding91]

I can't reproduce the issue. I loaded the save file on my computer and it used 21~ GB of RAM but when I clicked "exit" and it went back to the main menu it released all of it.

[AndrolGenhald]

Well since there's no crash I don't see anything useful in the log. Is there a way to force verbose logging, or is there anything else I can do to help?
Regardless of the memory leak though, might there be anything that's using excess RAM as well? I realize the uncompressed save is almost a gig, and I expect it to use quite a bit of RAM to run it, but 20 GiB seems kind of high. If it can't be fixed I'm going to have to figure out how to run Factorio in a cgroup so the thrashing doesn't kill my system until I build a new one with 64 GiB RAM.

[Smarty]

Install this mod https://mods.factorio.com/mods/orzelek/undecorator
I was able to reduce save file size by almost 66%
It might take some loading time for the mod to do its thing

[Twinsen]

Are you sure it's not some OS caching? Does loading the large save again cause your memory usage to reach maximum or make the game or computer unusable?

[Oxyd]

It's glibc keeping free'd memory to itself, instead of making it available for the rest of the OS.

Fixed in 0.14.9.

[AndrolGenhald]

You guys are great!

I take it that the amount of RAM being used is not currently considered an issue though?
Since exploration seems to be the largest factor in increased file size I'm assuming that it's also a significant factor for RAM usage? Could passive entities be unloaded after RAM reaches an optional limit based on LRU or furthest from the player (I'm thinking trees and decorations that might only be used when drawing them on the screen)? I realize it could end up being a lot of work, so I don't expect much, just throwing some thoughts out there.

[AndrolGenhald]

Figured out why I couldn't get cgroups to work last night, apparently even when running Factorio from a shell, it's the steam process that actually starts it, so I was just putting the messenger in the cgroup instead of Factorio itself. After putting Steam in a cgroup and fiddling around with memory.limit_in_bytes I can get it to run pretty well with a few minor lag spikes due to paging with as low as 2 GiB memory and the rest in swap.

Edit: I've increased the limit to 16 GiB so that autosaves don't take forever, but I think the fact that it runs reasonably well with 2 GiB shows that most of the memory is used very infrequently.