Page 1 of 1

Overloading the server

Posted: Mon Feb 13, 2017 2:19 am
by bman212121
Hi guys,

I can't seem to find anything quite relevant to what I'm seeing, but I was just curious if anyone was aware of, or had an idea if maybe a fix was going to happen for an issue. The long / short is that we've noticed it's very possible to queue up a huge amount of work that can overload the dedicated server, which is turn causes it stop communicating with the clients until it catches up (Server not responding). If you attempt to drop something like a blueprint that has 5,000 items in it, the server CPU will peg at 100% and if it stays there too long drops all of the players out of the game. Once the "work" is done and the cpu goes back down to normal, everyone can rejoin the game again. The obvious answers to the problem itself is "don't do that" and "look for a way to increase the server timeout wait period", but curious if there was a way to either prioritize client connections or maybe move it off into a separate thread so that even if the server is completely loaded down with work, people don't get dropped out of the game (Maybe it's overloading the client side as well, I'm not sure). I'm seeing this with a dedicated server on an i7 and a lan client with an i7 that's even faster than the dedicated server, so I'd imagine it's probably more frequent for lower end hardware.

Re: Overloading the server

Posted: Mon Feb 13, 2017 6:55 pm
by Rseding91
A blueprint with 5000 items is nothing. The server will process that in a fraction of a millisecond - there's no way it would cause it to hang unless you've severely limited the CPU power or amount of RAM the server can use.

Does the server have sufficient RAM? What are the specs and can you reproduce it on your local machine?

Re: Overloading the server

Posted: Tue Feb 14, 2017 3:17 am
by bman212121
Server is a VM with 2 VCPUs and 2GB Ram. (Headless Ubuntu) Factorio was using around 1.5GB at the time and there was still 130MB free of RAM and no swap. CPU is i7 3770.

I can in fact reproduce the issue on a local pc in single player, and I'd definitely say the issue is exponentially bad because of a mod. To try a scenario that is not utilizing the functionality of the mod, but with the mod still active in game, I simply tried to blueprint a solar farm that had around 45K items in it. If you shift + click to drop the blueprint, even on the local pc in single player the game will freeze solid for 5+ seconds while it's running the code for that. (i7 2600K with 16GB RAM on Windows 10). All of debugging code halts on the UI and nothing occurs until the processing is done. If I grab even more items I can get it pretty close to 10 seconds where the entire game is basically locked solid. I checked and I have right around 7K construction robots on the map, so a decent chunk of robots are probably being assigned tasks at the same time the blueprint is being dropped.

Now if I try using the "upgrade planner" mod, to simply swap out items from say yellow belts / undergrounds / splitters to their blue counterpart, when you shift + click for their code to "blueprint upgrade" I'd guess their code takes longer per item to process. It was pretty easy to freeze the game solid for 24 seconds (Timed) which would be more than enough time that anyone connected to my local machine would likely have been dropped because their client thought the server was down. I can't see the number of items I had processing, but I'm guessing it was less than the 45K from the solar farm test. My 5K guess does look a bit low after looking at the solar test. (More than half of the items in the blueprint's single row are off screen)


As a side note the blueprint code has an interesting effect I didn't notice until now. The more items you select while blueprinting, the lower the FPS goes despite the UPS staying relatively high. If I had like 100K items under the blueprint selection I can make the FPS go into single digits, while UPS is still around 40, and continues to do so even if the only thing being rendered on screen is trees. The FPS also drops when trying to hold the blueprint after it's created or simply trying to hover over it while it's in your inventory. If you deselect the blueprint the fps goes right back to normal.