[0.17.62] Is there a way to lower RAM usage (headless)

Anything that prevents you from playing the game properly. Do you have issues playing for the game, downloading it or successfully running it on your computer? Let us know here.
Post Reply
Ringk
Burner Inserter
Burner Inserter
Posts: 8
Joined: Sat Nov 17, 2018 1:29 pm
Contact:

[0.17.62] Is there a way to lower RAM usage (headless)

Post by Ringk »

Hi,
Is there a way to reduce RAM usage of the factorio server?

I tested loading my save with 2GB (ram) server and it wouldn't load the save (OS would kill it due to attempting to use too much ram). I also tested with 4GB server and it ran smoothly the factorio headless with my save (htop showed around 3.5GB usage for factorio), for a while until it kinda crashed the server (didn't have any swap memory as backup memory so couldn't recover):

Code: Select all

$ ls
-bash: fork: Cannot allocate memory
I do not have any mods installed.

My save isn't that huge yet, 244 rocket launches, filesize 41MB.

Obviously I could just upgrade to 6+ GB ram server.

Edit: Out of mem again, this time ran it in screen instaed of as daemon so I could send SIGINT to recover. Also uploaded the save to google drive. Let me know if the memory usage is totally normal in this case and I just should either run shorter games or upgrade the server to keep playing. Also I am interested in specific game settings (like disabling biters) that could significantly reduce the ram footprint of the game.
Second time the factorio server stopped responding. Had htop on other display.
Second time the factorio server stopped responding. Had htop on other display.
Screenshot from 2019-08-03 12-26-02.png (2.96 MiB) Viewed 2476 times
I guess autosaving (non-blocking) and logging silently fails when not enough memory available, following snippet is from screen-window. Kinda just describes what happens when too low ram for the application.

Code: Select all

2381.006 Verbose ServerSynchronizer.cpp:801: Peer 1 decreasing latency from 9 to 8 as buffer was always bigger than 2
2387.129 Verbose ServerRouter.cpp:158: Sending keepalive nat punch request to pingpong server.
2389.660 Info HttpSharedState.cpp:54: Downloading https://multiplayer.factorio.com/post-game-heartbeat/5744880
2404.581 Verbose ServerRouter.cpp:158: Sending keepalive nat punch request to pingpong server.
2409.973 Verbose ServerSynchronizer.cpp:814: Peer 1 increasing latency from 8 to 9 as it has skipped tick closures and empty buffer in the last period
2410.033 Verbose ServerSynchronizer.cpp:378: Peer 1 sent IncreasedLatencyConfirm tick: 9827405 increasedBy: 1
2419.680 Info HttpSharedState.cpp:54: Downloading https://multiplayer.factorio.com/post-game-heartbeat/5744880
2421.981 Info AppManager.cpp:267: Saving to _autosave43 (non-blocking).
2422.038 Info AsyncScenarioSaver.cpp:144: Saving process PID: 2929
2422.039 [2929] Verbose Scenario.cpp:834: Saving game as /opt/factorio/0.17.62/servers/public/saves/_autosave43
2422.887 Verbose ServerRouter.cpp:158: Sending keepalive nat punch request to pingpong server.
2503.519 Info HttpSharedState.cpp:54: Downloading https://multiplayer.factorio.com/post-game-heartbeat/5744880
^C2529.748 [2929] Received SIGINT, shutting down
2529.748 Received SIGINT, shutting down
2549.264 Error HttpSharedState.cpp:136: CURL failed: code:28, 1c; Timeout was reached
2549.426 Info HttpSharedState.cpp:145: Downloading https://multiplayer.factorio.com/post-game-heartbeat/5744880
2549.686 Info HttpSharedState.cpp:146: Status code: 0
2550.693 Info MatchingServer.cpp:170: Failed to sendHeartbeat to the multiplayer matching server: Download failed (status code 520): failed to parse JSON response: Unexpected character (T) at ?:1
JSON: Timeout was reached
2551.845 Verbose MatchingServer.cpp:98: Sending update request for game(5744880) to matching server
2553.429 Info HttpSharedState.cpp:54: Downloading https://multiplayer.factorio.com/update-game/5744880
^C2606.167 [2929] Received second SIGINT, terminating immediately
2606.167 Received second SIGINT, terminating immediately  
Server closed: /opt/factorio/0.17.62/servers/old_public

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2339
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: [0.17.62] Is there a way to lower RAM usage (headless)

Post by Jap2.0 »

I'm fairly certain that non-blocking saving forks the process. Some looking through stack overflow shows that forking doesn't immediately copy all of the memory, but I'm guessing that would have some memory impact (hence leading to it failing). I'd guess you'd know more than I would.

I'm seeing a little over 4GB memory (with graphics), which isn't unreasonable for a save of that size.

Your pollution reaches out of the map a little, unless something changed that I don't know about, every chunk that is touched by pollution (and iirc a couple chunk radius outside) is generated (and as a result becomes part of the game states and uses a little bit of ram). It looks like at most pollution reaches ~10 chunks outside of your explored area.

Removing decoratives (the little decorative objects on the ground) used to be a big thing for save file size, but I believe that was improved significantly since. As a result most mods that remove them have disappeared; I was eventually able to find a command to remove them, and it resulted in no memory improvement.

There are mods to remove chunks that you haven't touched, but in reality that's at best going to remove the chunks outside of your pollution range (which aren't a ton), regenerate 95% of them, and make it so radar has to rediscover everything.

Unfortunately I'm not coming up with any good ways to reduce memory usage, so getting a server with more memory may be your best/only bet.
There are 10 types of people: those who get this joke and those who don't.

Ringk
Burner Inserter
Burner Inserter
Posts: 8
Joined: Sat Nov 17, 2018 1:29 pm
Contact:

Re: [0.17.62] Is there a way to lower RAM usage (headless)

Post by Ringk »

I was afraid that might be the case. Thank you very much for your feedback! I certainly will use normal blocking saving in the future unless I happen to have excess ram to use.

slippycheeze
Filter Inserter
Filter Inserter
Posts: 587
Joined: Sun Jun 09, 2019 10:40 pm
Contact:

Re: [0.17.62] Is there a way to lower RAM usage (headless)

Post by slippycheeze »

Jap2.0 wrote:
Sat Aug 03, 2019 6:09 pm
I'm fairly certain that non-blocking saving forks the process. Some looking through stack overflow shows that forking doesn't immediately copy all of the memory, but I'm guessing that would have some memory impact (hence leading to it failing). I'd guess you'd know more than I would.
You are correct, fork is lazy, and only copies on write -- but it'll copy on a write from either side, so as the game continues every change that mutates a Lua or C++ object will have to copy the entire page.

Even more unfortunately for you, saving a game also triggers a full GC in Lua, which is likely to modify a non-trivial amount of that memory, which would help explain why it is going to be less effective than you might hope.

TL;DR here is you are gonna need either more memory, or more swap space. On the plus side, swap space should be a fine solution to this problem if you save infrequently enough, so if you can do that with your system, it might help.

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2339
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: [0.17.62] Is there a way to lower RAM usage (headless)

Post by Jap2.0 »

Again, can't say this for certain, but I have heard that Factorio plays very nicely with swap (to a point, obviously).
There are 10 types of people: those who get this joke and those who don't.

slippycheeze
Filter Inserter
Filter Inserter
Posts: 587
Joined: Sun Jun 09, 2019 10:40 pm
Contact:

Re: [0.17.62] Is there a way to lower RAM usage (headless)

Post by slippycheeze »

Jap2.0 wrote:
Wed Aug 07, 2019 3:24 am
Again, can't say this for certain, but I have heard that Factorio plays very nicely with swap (to a point, obviously).
A better way to put it is: if you don't have 2GB or so of swap, you are causing yourself pain, as it has significant value above and beyond simply letting you use more memory that you don't have. :)

...but, seriously, unless you are swapping in at a high rate, swap will only help. Having a bazillion gig of it also turns "out of memory" into "swaps forever", so 2-4GB is a good size, but you lose performance not having that much.

Post Reply

Return to “Technical Help”