[Optimization] Improve map download netcode

Post your ideas and suggestions how to improve the game.
Post Reply
luziferius
Inserter
Inserter
Posts: 24
Joined: Thu Jun 01, 2017 12:59 pm
Contact:

[Optimization] Improve map download netcode

Post by luziferius » Tue Mar 12, 2019 1:06 pm

The network code used to download multiplayer maps can be optimized, resulting in generally faster map downloads.

Here in Germany, many ISPs offer highly asymmetric DSL lines, like 16MBit/s down, 0.5MBit/s up. Because the factorio network code uses about 4 times the number of packages in upload direction than needed, factorio quickly saturates the upload bandwidth available. For me, this leads to degraded map download speeds, even if the server is hosted on an uncongested symmetrical Gigabit connection.

To show the current state, I’ve created two screenshots of my system monitor, while it displays network metrics:

Purely as a reference, this is my current download speed when loading a regular, big file somewhere from the internet. From top to bottom it is data rate, packet rate, error rate. Here, the download line is saturated, while the upload line is not.
Data rate regular download.png
Data and packet rate of a regular file download.
Data rate regular download.png (99.21 KiB) Viewed 216 times
The same situation, but using factorio to download a larger multiplayer map.
Data rate factorio map download.png
Data and packet rate during factorio map download.
Data rate factorio map download.png (2.31 MiB) Viewed 216 times
For me, the download rate is capped at about 660 KiB/s, regardless of the used server. I have never seen higher download rates in factorio.

When you compare the packet rates, you can see that a regular download uses about 540-580 packets/s for TCP ACKs. Factorio somehow takes over 1800 packets/s, which is even more than the received package rate.

My suggestion is to reduce the number of acknowledgement packages sent, if you can control this. This will lead to less upload congestion and in turn, improve the map download speed for everyone with such asymmetrical network connections.

weaknespase
Long Handed Inserter
Long Handed Inserter
Posts: 59
Joined: Sat Mar 24, 2018 8:19 am
Contact:

Re: [Optimization] Improve map download netcode

Post by weaknespase » Tue Mar 12, 2019 9:36 pm

Probably too small payload for each transmitted packet. 660K/1800 ~ 320 bytes of payload per packet, i'm not sure they even using TCP for bulk data transfer, and I think they should.

luziferius
Inserter
Inserter
Posts: 24
Joined: Thu Jun 01, 2017 12:59 pm
Contact:

Re: [Optimization] Improve map download netcode

Post by luziferius » Tue Mar 19, 2019 3:58 pm

weaknespase wrote:
Tue Mar 12, 2019 9:36 pm
Probably too small payload for each transmitted packet. 660K/1800 ~ 320 bytes of payload per packet, i'm not sure they even using TCP for bulk data transfer, and I think they should.
Its actually 1300 packages/s incoming, and 1800 packages/s sent.
While downloading maps, Factorio sends more packages than it receives.

SyncViews
Filter Inserter
Filter Inserter
Posts: 261
Joined: Thu Apr 21, 2016 3:17 pm
Contact:

Re: [Optimization] Improve map download netcode

Post by SyncViews » Tue Mar 19, 2019 4:31 pm

weaknespase wrote:
Tue Mar 12, 2019 9:36 pm
Probably too small payload for each transmitted packet. 660K/1800 ~ 320 bytes of payload per packet, i'm not sure they even using TCP for bulk data transfer, and I think they should.
If I recall they said previously that they don't use TCP because it is another thing for firewalls and NAT to break when not using port forwarded / public IP servers.

It has been some years since I programmed mixed TCP/UDP, but I also recall there are issues with TCP sending more and more until it causes packet loss, probably for a home connection on the servers upload side. You dont want packet loss there as it also impacts the UDP for all the players already connected.

luziferius
Inserter
Inserter
Posts: 24
Joined: Thu Jun 01, 2017 12:59 pm
Contact:

Re: [Optimization] Improve map download netcode

Post by luziferius » Tue Mar 19, 2019 10:05 pm

SyncViews wrote:
Tue Mar 19, 2019 4:31 pm
It has been some years since I programmed mixed TCP/UDP, but I also recall there are issues with TCP sending more and more until it causes packet loss […]
Yeah, thats the TCP congestion control algorithm (https://en.wikipedia.org/wiki/TCP_congestion_control). Many algorithms use packet loss as an indicator.
Newer TCP stacks tend to use algorithms that quickly saturate the line and don’t back off that far, if they hit packet loss conditions.
If they break UDP streams, that may be bad for servers with low upload bandwidth.

----

Increasing the overall packet size for incoming packets should improve the situation here. Currently, the packet size is about 500 bytes (660KiB/1300 packets). A normal network MTU is about 1500 bytes (including headers). so there is at least some room for improvement. Going too large may introduce packet fragmentation during the transport. Especially, if there is some network tunneling involved.

----

But I think, Factorio shouldn’t send that many requests per second, at all. TCP can acknowledge the data using a quarter of the packages Factorio uses.

SyncViews
Filter Inserter
Filter Inserter
Posts: 261
Joined: Thu Apr 21, 2016 3:17 pm
Contact:

Re: [Optimization] Improve map download netcode

Post by SyncViews » Wed Mar 20, 2019 12:12 am

luziferius wrote:
Tue Mar 19, 2019 10:05 pm
Increasing the overall packet size for incoming packets should improve the situation here. Currently, the packet size is about 500 bytes (660KiB/1300 packets). A normal network MTU is about 1500 bytes (including headers). so there is at least some room for improvement. Going too large may introduce packet fragmentation during the transport. Especially, if there is some network tunneling involved.
1500 is for Ethernet, so LAN. WiFi I think has even more. Finding the MTU over the internet is more tricky, I recall that it is at least 500 and something bytes for IPv4 internet, but not sure on what the current state is.
luziferius wrote:
Tue Mar 19, 2019 10:05 pm
But I think, Factorio shouldn’t send that many requests per second, at all. TCP can acknowledge the data using a quarter of the packages Factorio uses.
Yes this seems to be the bottleneck. Not looked at Factorio under say Wireshark (normally dealing either with regular stuff like HTTP, or audio/video streams), but would expect it to combine ACK packets

Not that TCP is perfect, a lot of the "optimisations" can end up seeming counter productive in the protocols I have worked with, at least without being careful anyway.

Post Reply

Return to “Ideas and Suggestions”

Who is online

Users browsing this forum: jgilmore42