[1.1.59][Multiplayer][SOLVED] UPS jump up followed by a disconnect of a player

[sejbr]

Hi I have setup a headless server with mods on

OVH Comfort VPS 4 vCore 8GB Ram
Linux 5.15.0-25-generic #25-Ubuntu SMP Wed Mar 30 15:54:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Description: Ubuntu 22.04 LTS
Release: 22.04
Codename: jammy

The issues are as follows:
1. My buddy joins the game. Loads the map and is able to move.
2. The instant he is able to move UPS jump up to values from 80 to 240 visible both on my end and his

ups.png (22.5 KiB) Viewed 2341 times

3. After a few seconds with UPS above 60 he is getting disconnected.
4. The second he starts to get dropped the ups goes back to 60.

Sometimes he gets the "Your pc couldn't keep up" message, sometimes just "Lost connection"

server-settings.json were mostly left the same as in the example file.

What we've tried:
1. Changing settings listed below with no success

Code: Select all

max_upload_in_kilobytes_per_second
minimum_latency_in_ticks
non_blocking_saving

2. Changing the game speed to 0.5
3. On his end - Removing the whole mods folder
4. On his end - Removing Factorio alltogether along with %appdata%/factorio

Server logs usually just say this:

Code: Select all

1360.192 Info ServerMultiplayerManager.cpp:1061: Disconnect notification for peer (8)
1360.193 Info ServerSynchronizer.cpp:627: nextHeartbeatSequenceNumber(68439) removing peer(8).

His logs from one of the tries:

Code: Select all

0.343 Info ClientMultiplayerManager.cpp:610: UpdateTick(32278) changing state from(ConnectedLoadingMap) to(TryingToCatchUp)
 120.914 Info ClientMultiplayerManager.cpp:610: UpdateTick(32460) changing state from(TryingToCatchUp) to(WaitingForCommandToStartSendingTickClosures)
 121.074 Info ClientMultiplayerManager.cpp:610: UpdateTick(32460) changing state from(WaitingForCommandToStartSendingTickClosures) to(InGame)
 121.141 Info GameActionHandler.cpp:5015: UpdateTick (32462) processed PlayerJoinGame peerID(13) playerIndex(1) mode(connect) 
 138.973 Info ClientSynchronizer.cpp:335: Latency changed to (15)
 149.290 Info ClientSynchronizer.cpp:335: Latency changed to (17)
 168.973 Info ClientSynchronizer.cpp:335: Latency changed to (21)
 188.975 Info ClientSynchronizer.cpp:335: Latency changed to (75)
 195.173 Warning ClientSynchronizer.cpp:106: nextHeartbeatSequenceNumber(1930822503) server wants a heartbeat(1930821301) that we don't have.
 195.741 Error ClientMultiplayerManager.cpp:98: MultiplayerManager failed: "Utracono połączenie z grą."
 195.742 Info ClientMultiplayerManager.cpp:610: UpdateTick(34694) changing state from(InGame) to(WaitingForUserToSaveOrQuitAfterServerLeft)
 195.742 Info ClientMultiplayerManager.cpp:930: Disconnect notification for peer (13)
 209.017 Info ClientMultiplayerManager.cpp:210: Quitting multiplayer connection.
 209.017 Info ClientMultiplayerManager.cpp:610: UpdateTick(34694) changing state from(WaitingForUserToSaveOrQuitAfterServerLeft) to(Disconnected)

When I host the game locally from my Factorio and invite him via steam with the same mods installed - everything works perfectly.
I was thinking that either his computer is not fast enough or the server is not fast enough for that heavily modded game or OVH firewall has issues but in all these cases.
1. His game works fine when connected to my game hosted on windows with the same mods.
2. Server works for me and he has issues regardles if he's the only player or not
3. OVH would usually send an email if they blocked any packets.

Any ideas?
Thank you for your time.

[Loewchen]

Post complete logs of server and client of the same session.
Does your friend have the issue only on your server?
Does the issue persist on a new map without mods?

[sejbr]

Complete logs attached.
Unfortunately for us, the game is fine unmodded on a new map so I assume it's either an issue with a mod on linux or with a mod on a headless server?
I have tried watching show-time-usage as he connected but nothing changed.

[Loewchen]

Sounds similar to 102530. At least one packet is not getting to the server for some reason, how this could effect the UPS counter on a different client I do not understand. You could try to enable verbose logging on server and client and see if that gives more info in the logs.

[sejbr]

Verbose logs from both linux server and the client attached.

We've checked another thing.
I have taken my steam factorio installation which we know worked since when I start multiplayer in game he is fine.
Started the windows binary with --start-server, default configs and the same savegame we tried when I launched multiplayer via GUI.

The same issue. He joins, UPS goes out of wack and he gets disconnected with the only log around that time being the "wants the heartbeat that we don't have".
Looks like an issue with headless server? Can a mod be compatible with GUI and incompatible with headless?

Also his latency jumps up before disconnect but only on headless server.

[sejbr]

There has been an odd development...
My buddy's game now works and I'm the one having the issue.
Nothing else changed at the time.

[Bilka]

I see you're running flib (Factorio library mod). Try setting its translation per tick value lower in the mod settings. I cannot find the whole thing I'm remembering about this and what I can find is old-ish, but it might help.

[sejbr]

Will this setting be in mods/mod-settings.dat that I synchronize to the server and work after startup or do I need to generate a new map alltogether since it's in map settings?

[FuryoftheStars]

Will this setting be in mods/mod-settings.dat that I synchronize to the server and work after startup or do I need to generate a new map alltogether since it's in map settings?

Yes, it will be in the mod-settings.dat file. All mod settings are held in there. There are none that cannot be adjusted and changed at some point, though it does depend on how it’s used within the mod on the affect during a running game (ie, RSO, changing resource generation settings won’t affect anything existing, but will work on anything new found).

[sejbr]

I see you're running flib (Factorio library mod). Try setting its translation per tick value lower in the mod settings. I cannot find the whole thing I'm remembering about this and what I can find is old-ish, but it might help.

Man You are a God sent. Changed this value to it's minimum along with some other setting in LTN described with "Lower if you have performance issues" and still after two server reboots we have no issues and the UPS is at a stable 60.
Thank you so very much. After two weeks of battles with this issue we were close to just not starting the session at all. You single handedly managed to take 300 hours of our future time from us! Cheers!