Option to make server wait for slowest client

Post your ideas and suggestions how to improve the game.

Moderator: ickputzdirwech

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 12889
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Mitigating multiplayer lag spikes by server performance tuning

Post by ssilk »

What if the server-owner just wants players, that are fast enough to cope up with the server? What if the lag of a player is so much, that he should go?

Too much seen from one direction.
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...
oloklir
Burner Inserter
Burner Inserter
Posts: 5
Joined: Mon Dec 17, 2018 6:09 pm
Contact:

Re: Mitigating multiplayer lag spikes by server performance tuning

Post by oloklir »

ssilk wrote: Sat Nov 09, 2019 8:07 am What if the server-owner just wants players, that are fast enough to cope up with the server? What if the lag of a player is so much, that he should go?

Too much seen from one direction.
May be.

But if there is an option/command to control simulation performance or switch automatic simulation performance tuning, it will be handy for server moderators.
Not like decreasing [game.speed], which limits maximum ups and doesn't really prevents lag spikes and disables achievement.
Koub
Global Moderator
Global Moderator
Posts: 7784
Joined: Fri May 30, 2014 8:54 am
Contact:

Re: Option to make server wait for slowest client

Post by Koub »

[Koub] Merged into older suggestion topic about the same issue.
Koub - Please consider English is not my native language.
AIAndy
Manual Inserter
Manual Inserter
Posts: 2
Joined: Mon Nov 02, 2020 6:02 pm
Contact:

Re: Multiplayer adaptive game speed (slow down for slow clients)

Post by AIAndy »

Since I just got a duplicate notification when trying to add it as a new topic in this forum, I'll add it here instead (it fits after all, but I hope it does not just get lost).
This is a serious issue. While not technically a bug, it will cause problems for most groups of friends playing Factorio together as how likely is it that they use the slowest computer to host if not specifically told to. The issue does not happen right away but only once the factory has grown to a certain size and it starts spiky because not every update takes the same time so most mistake it for lag or graphic issues. Once it progresses to the fall behind state the issue becomes clearer but people tend to give up before that.
TL;DR
Add a feature that slows down the Multiplayer Server/Host to the slowest connected client up to a certain percent.

What ?
Add a new multiplayer server option that when set to a certain percent allows the server to slow down to adapt to slower clients by adding up to that percent of his own update loop time as waiting time after the end of the update loop. The amount of time that is actually added should be the difference between the maximum of the update loop times of all clients and its own update loop time in the previous time tick up to the limit given by the option.
This will cause the server to slow down enough for clients to keep up without sacrificing their render loop when computation load gets too high for 60 UPS.
Why ?
This is the solution to a common problem in Factorio Multiplayer when the host is a normal gaming PC. When the computation load of the game goes above what is sustainable at 60 UPS, any clients slower than the host PC sacrifice their render loop to keep up, causing massive FPS loss even if only slightly slower than the host. This usually starts with spikes during attacks of larger enemy groups. To the players this seems like lag so the cause of the problem is usually overlooked. Hosting on the slowest PC can fix that but especially among small groups of friends there is often no clear slowest PC. The other solution is manually slowing down the game, but then it always runs slower instead of only during the high load frames and it is not a very user friendly fix (if they even realize that this can fix what looks like lag or GPU issues).
User avatar
Tesse11ation
Fast Inserter
Fast Inserter
Posts: 186
Joined: Sat Mar 05, 2016 12:59 am
Contact:

Re: Option to make server wait for slowest client

Post by Tesse11ation »

Can't game speed be manually adjusted down? What if the host does this when the clients complain about this problem? The downside of doing this is that it's a lua command, so it disables achievements. But, it is possible.
ImageImage
System Specs
OS: Windows 10 Professional 64 Bit
CPU: AMD Ryzen 5 3600X (@~3.8 gHz)
GPU: Nvidia RTX 2080
RAM: 32GB DDR4 (2400)
DRIVES: 2x 1TB NVMe SSD, 1x 6TB HDD
AIAndy
Manual Inserter
Manual Inserter
Posts: 2
Joined: Mon Nov 02, 2020 6:02 pm
Contact:

Re: Option to make server wait for slowest client

Post by AIAndy »

KoblerMan wrote: Tue Nov 03, 2020 2:44 pm Can't game speed be manually adjusted down? What if the host does this when the clients complain about this problem? The downside of doing this is that it's a lua command, so it disables achievements. But, it is possible.
There are two problems with that.
The first is realizing what the problem is. It feels like lag so if you look on Reddit at posts complaining about lag you will see that it is often actually this problem (if a group of 5 friends plays and one randomly hosts there is an 80% chance that at least one player will suffer from this issue once the factory grows). If they know enough to look at the debug view they will see something like 60 UPS / 10 FPS so next guess will be graphics. That the issue is actually being slightly slower than the server is very hard to see (and slightly slower is enough for serious frame drops).
The second problem is that not every single tick takes the same time to calculate. It spikes quite a lot when the pathfinder is heavily used, mostly when biter groups start to attack (several mods amplify the problem). If you want to fix the problem this way, you have to reduce the game speed enough that the worst case tick calculation on the slowest client is doable in time while with a proper solution from the game it could run full speed most of the time.

So fixing it that way requires:
  • Realizing that this is the issue
  • Knowing the lua command to set the speed
  • Sacrificing achievements
  • Communicating with the clients to find out if they have this issue and how much slowdown is needed
  • Probably try around a bit until it fits
  • Repeat it every x minutes when the factory has grown enough that the slowdown is no more sufficient
User avatar
kizrak
Long Handed Inserter
Long Handed Inserter
Posts: 77
Joined: Thu Jul 19, 2018 1:27 am
Contact:

Re: Option to make server wait for slowest client

Post by kizrak »

Maybe this could be regulated via some type of permission (if anti-griefing is desired). The server could yield to users/clients with a permission, but not others. I'm not sure if this, or probably technical limitations make this prohibitive. 🤔

Out of curiosity (I know it isn't likely to change the outcome) could someone shine some insight as to what makes this difficult, technically or otherwise?

Thanks, 🧡
-Kizrak
foamy
Filter Inserter
Filter Inserter
Posts: 432
Joined: Mon Aug 26, 2019 4:14 am
Contact:

Re: Option to make server wait for slowest client

Post by foamy »

There's a mod I've found handy for addressing this, called Speed Control, which effectively gives everyone on the server the ability to adjust the sim rate up or down.

Unfortunately, it can't address the 'cannot join at all' aspect of a speed mismatch, short of a side-channel communication to someone already in the game. The automated option for a server admin would be useful.
adam_bise
Filter Inserter
Filter Inserter
Posts: 450
Joined: Fri Jun 08, 2018 10:42 pm
Contact:

Re: Option to make server wait for slowest client

Post by adam_bise »

I've ran into issues where I couldn't connect to my own server because it was faster than my client on a large map. Client would hang at "catching up". Seemed ridiculous that the server would not budge on speed even for one player. But then, it also seems unusual that multiplayer processes on server and all clients instead of just the server.
User avatar
NotRexButCaesar
Smart Inserter
Smart Inserter
Posts: 1133
Joined: Sun Feb 16, 2020 12:47 am
Contact:

Re: Option to make server wait for slowest client

Post by NotRexButCaesar »

adam_bise wrote: Wed Dec 02, 2020 3:17 am I've ran into issues where I couldn't connect to my own server because it was faster than my client on a large map. Client would hang at "catching up". Seemed ridiculous that the server would not budge on speed even for one player. But then, it also seems unusual that multiplayer processes on server and all clients instead of just the server.
Something for Factorio 2
—Crevez, chiens, si vous n'étes pas contents!
Ida-Marie
Burner Inserter
Burner Inserter
Posts: 12
Joined: Tue Jul 25, 2017 5:54 pm
Contact:

Re: Multiplayer adaptive game speed (slow down for slow clients)

Post by Ida-Marie »

AIAndy wrote: Mon Nov 02, 2020 7:26 pm Since I just got a duplicate notification when trying to add it as a new topic in this forum, I'll add it here instead (it fits after all, but I hope it does not just get lost).
This is a serious issue. While not technically a bug, it will cause problems for most groups of friends playing Factorio together as how likely is it that they use the slowest computer to host if not specifically told to. The issue does not happen right away but only once the factory has grown to a certain size and it starts spiky because not every update takes the same time so most mistake it for lag or graphic issues. Once it progresses to the fall behind state the issue becomes clearer but people tend to give up before that.
TL;DR
Add a feature that slows down the Multiplayer Server/Host to the slowest connected client up to a certain percent.

What ?
Add a new multiplayer server option that when set to a certain percent allows the server to slow down to adapt to slower clients by adding up to that percent of his own update loop time as waiting time after the end of the update loop. The amount of time that is actually added should be the difference between the maximum of the update loop times of all clients and its own update loop time in the previous time tick up to the limit given by the option.
This will cause the server to slow down enough for clients to keep up without sacrificing their render loop when computation load gets too high for 60 UPS.
Why ?
This is the solution to a common problem in Factorio Multiplayer when the host is a normal gaming PC. When the computation load of the game goes above what is sustainable at 60 UPS, any clients slower than the host PC sacrifice their render loop to keep up, causing massive FPS loss even if only slightly slower than the host. This usually starts with spikes during attacks of larger enemy groups. To the players this seems like lag so the cause of the problem is usually overlooked. Hosting on the slowest PC can fix that but especially among small groups of friends there is often no clear slowest PC. The other solution is manually slowing down the game, but then it always runs slower instead of only during the high load frames and it is not a very user friendly fix (if they even realize that this can fix what looks like lag or GPU issues).
I will add that this in combination with other issues is a serious problem for me and my friends. Just filed three bug reports regarding problems arising around this
viewtopic.php?f=6&t=94793
viewtopic.php?f=49&t=94789
viewtopic.php?f=49&t=94791

My solutions so far have been using the slowest computer as headless server, and using cpulimit temporarily to handle joins. This is most needed on first joining when someone in-game cannot push the pause button.

Running cpulimit permanently on a server is a really bad idea since it cuts multithread performance, which clients can be expected to handle well. Then it's better with a dedicated slow computer.

So for today it needs an advanced handling strategy to keep a server with a megabase running open and usable for friends. I wish the Factorio team looks more seriously at multiplayer performance tweaking. (not to forget all good they have done with the game though...)
User avatar
ptx0
Smart Inserter
Smart Inserter
Posts: 1507
Joined: Wed Jan 01, 2020 7:16 pm
Contact:

Re: Multiplayer adaptive game speed (slow down for slow clients)

Post by ptx0 »

Ida-Marie wrote: Fri Jan 22, 2021 7:10 pm ...
use /c game.speed=.8
Phoenix27833
Long Handed Inserter
Long Handed Inserter
Posts: 52
Joined: Mon Aug 27, 2018 12:46 pm
Contact:

Slow down server setting

Post by Phoenix27833 »

A common issue with factorio multiplayer is that when the map gets bigger and you begin to reach UPS limits the slower players get dropped as they cannot keep up.
The current solutions to this are either to:
  • host the game on the slowest pc so that everyone can keep up. This is not always possible or feasible as that person might not have good internet. It is also annoying that when they are not available to play, someone else has to host and when they come back you have to move the host around.
  • or lower the game speed via command. This is not great either as the game then runs slower even during periods when everyone would be able to keep up with 60UPS, and if there is a big UPS dip due to fighting biters etc, then the slow players still start being dropped. Also as the base increases in size the game speed constantly has to be adjusted to allow the slowest pc to keep up.
What I suggest is to add a setting which applies a multiple of the tick time as a delay after the tick on the host. This would dynamically slow the host to the approximate tick rate of the slower pc, letting them keep up even with spikes in tick time.
I would make the setting set an inverse value to the tick delay, ie. set at 1 means run at full speed, 0.5 half speed (add the tick time as delay), 0.33 adds double the tick time as delay, etc. This would probably feel more natural to the players.

If you want to be fancy you could have the clients report their tick times to the host and have it automatically calculate the slowdown factor needed to allow everyone to keep up.
Koub
Global Moderator
Global Moderator
Posts: 7784
Joined: Fri May 30, 2014 8:54 am
Contact:

Re: Option to make server wait for slowest client

Post by Koub »

[Koub] Merged to older thread with same suggestion.
Koub - Please consider English is not my native language.
Post Reply

Return to “Ideas and Suggestions”