Very slow downloads from the website

Anything that prevents you from playing the game properly. Do you have issues playing for the game, downloading it or successfully running it on your computer? Let us know here.
Crysesa
Manual Inserter
Manual Inserter
Posts: 3
Joined: Sat Jan 15, 2022 11:30 am
Contact:

Re: Very slow downloads from the website

Post by Crysesa »

Yes, downloads in-game cancel after a while for large mods.
download.PNG
download.PNG (16.69 KiB) Viewed 2750 times

But! Using a dedicated downloader like jdownloader I get somewhat faster speeds, around 300 kbps which is still way slower than my actual connection speed, and I can sometimes successfully download mods.

Crysesa
Manual Inserter
Manual Inserter
Posts: 3
Joined: Sat Jan 15, 2022 11:30 am
Contact:

Re: Very slow downloads from the website

Post by Crysesa »

vinzenz wrote:
Fri Jan 14, 2022 3:54 pm
ptx0 wrote:
Fri Jan 14, 2022 3:32 pm
as i'm not the only person living in BC, I imagine there's more people who experience this
BC as in British Columbia? To troubleshoot this further I would like you to do some network path testing with a tool like https://en.wikipedia.org/wiki/MTR_(software) or https://en.wikipedia.org/wiki/PathPing

Could you please do traces for us1.factorio.com, eu1.factorio.com, jp1.factorio.com, mods-data.factorio.com and cdn.factorio.com

Reports of these tools are a bit verbose and contain somewhat private information (IPs of your home network, ISP gateway, etc) so feel free to send them to support@factorio.com instead of posting them here.
Using winmtr I tried to test mods-data.factorio.com. I am not a network specialist, but I believe the packet loss rate is quite high?

winmtr_mods-data_downl.png
winmtr_mods-data_downl.png (15.2 KiB) Viewed 2732 times

orzelek
Smart Inserter
Smart Inserter
Posts: 3911
Joined: Fri Apr 03, 2015 10:20 am
Contact:

Re: Very slow downloads from the website

Post by orzelek »

Crysesa wrote:
Sat Jan 15, 2022 5:58 pm
vinzenz wrote:
Fri Jan 14, 2022 3:54 pm
ptx0 wrote:
Fri Jan 14, 2022 3:32 pm
as i'm not the only person living in BC, I imagine there's more people who experience this
BC as in British Columbia? To troubleshoot this further I would like you to do some network path testing with a tool like https://en.wikipedia.org/wiki/MTR_(software) or https://en.wikipedia.org/wiki/PathPing

Could you please do traces for us1.factorio.com, eu1.factorio.com, jp1.factorio.com, mods-data.factorio.com and cdn.factorio.com

Reports of these tools are a bit verbose and contain somewhat private information (IPs of your home network, ISP gateway, etc) so feel free to send them to support@factorio.com instead of posting them here.
Using winmtr I tried to test mods-data.factorio.com. I am not a network specialist, but I believe the packet loss rate is quite high?


winmtr_mods-data_downl.png
That first 87% loss is strange since packets seem to pass further up anyway.
But those losses near end that can cut up to 50% of packets will wreak havoc with factorio transfer mechanics (and will badly mess even tcp/ip) since transfer acknowledges and packets will get lost a lot and it might get stuck trying to retransmit packets. So in the end it will be doing lots of transmit/retransmit and your speed will plummet.

From what I recall I had issues in online games when I had 5-10% drops on one stpe and after reporting that to ISP (when I managed to reach properl technical person) it was considered a problem and traced to defective network equipment.

For comparison my trace to mods url doesn't go through twelve99 at all:
image_2022-01-16_205234.png
image_2022-01-16_205234.png (39.91 KiB) Viewed 2709 times

User avatar
ptx0
Smart Inserter
Smart Inserter
Posts: 1507
Joined: Wed Jan 01, 2020 7:16 pm
Contact:

Re: Very slow downloads from the website

Post by ptx0 »

your link doesn't go through twelve99 because your ISP peers with Cogent, which is a far better ISP than GTT or Twelve99.

donny
Manual Inserter
Manual Inserter
Posts: 4
Joined: Tue Jan 08, 2019 10:41 pm
Contact:

Re: Very slow downloads from the website

Post by donny »

I also just tried downloading mods via the ingame downloader. It takes minutes until it shows the modlist and minutes until the download button can be seen. the download itself didn't take long. I'm located in Germany and use Vodafone DSL. Do you guys still have problems?
*Update* I read somewhere that Avast causes problems. This solved my problem. But one mod stubbornly doesn't want to be downloaded: https://mods.factorio.com/mod/rewire-tool

User avatar
ptx0
Smart Inserter
Smart Inserter
Posts: 1507
Joined: Wed Jan 01, 2020 7:16 pm
Contact:

Re: Very slow downloads from the website

Post by ptx0 »

donny wrote:
Thu Feb 03, 2022 6:45 pm
I also just tried downloading mods via the ingame downloader. It takes minutes until it shows the modlist and minutes until the download button can be seen. the download itself didn't take long. I'm located in Germany and use Vodafone DSL. Do you guys still have problems?
*Update* I read somewhere that Avast causes problems. This solved my problem. But one mod stubbornly doesn't want to be downloaded: https://mods.factorio.com/mod/rewire-tool
try it again. some changes hadn't fully gone through yet, i think. it works perfectly for me now.

nuhll
Filter Inserter
Filter Inserter
Posts: 922
Joined: Mon Apr 04, 2016 9:48 pm
Contact:

Re: Very slow downloads from the website

Post by nuhll »

might i ask why use a cdn instead of just getting like 5 servers in different regions?

e.g. germany for europe, something asian, some australian, us left, us right... something like this? (i mean you see better where ppl are from)

that way you can easier switch and or control how your stuff gets served? (and if you dont want to fiddle around with "auto select correct server" just make it adjustable, you could preselct via language tho)

i also had crazy problems with your downloads, but thankfully never since you changed the first time (atleast not that bad i expiericnced it at that time)

User avatar
BlueTemplar
Smart Inserter
Smart Inserter
Posts: 2420
Joined: Fri Jun 08, 2018 2:16 pm
Contact:

Re: Very slow downloads from the website

Post by BlueTemplar »

just getting like 5 servers in different regions
Which would then be a CDN ?
BobDiggity (mod-scenario-pack)

User avatar
ptx0
Smart Inserter
Smart Inserter
Posts: 1507
Joined: Wed Jan 01, 2020 7:16 pm
Contact:

Re: Very slow downloads from the website

Post by ptx0 »

sorry for the necro-bump. i've got follow-up information from investigating issues this morning.

first off, i'm so sorry for forcing Wube to waste time on something I should have spent more time validating. even opening up Wireshark would have shown me a few questionable details.

secondly, i have a fever. I might not be coherent. however, this morning I had the idea to investigate TCP Window Scaling and its function in my life, after having to restart a download every 30 seconds so that it completed.

basically, disabling TCP Window Scaling forces (on Linux, anyway) the window size to a fixed 65535 which means we send as much data as a TCP packet naturally allows, before considering the 'scaling factor'. This results in a CONSTANT throughput of 10mbps, but it can't go any higher on a single TCP session. Not with my current RTT latency, anyway. For the same Window size on different latency connections, you can get varying throughput. 10mbps is somewhat ideal, i've had 5mbps this way to further distant networks.

but then I went into Windows to figure out whether it's a Linux-specific problem. Something to note here, is, if I'm tethered to my phone that uses the same LTE provider, it works fine! My theory is there's something wrong with the modem they provide, that I can not change. I've put it into Bridge mode and turned off any kind of filtering/QoS and alas, nothing changes this behaviour...

except...

putting these values into sysctl.conf:

Code: Select all

net.core.wmem_max = 4194304
net.core.rmem_max = 12582912
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 87380 4194304
and reenabling TCP Window Scaling. Now, my downloads start out "slow" and gain speed as they go, instead of beginning deceptively fast, and cliff-diving.

I should note that using `iperf3` to test the Window size is what led me to searching how to increase these values, as, the default max window size on Linux, is a mere 250kB

For what it's worth, the limits are the same on Windows.

anyone with a very fast network connection (pretty much any modern ISP) should revisit their TCP tunings, because despite praise that things "Just Work" a lot more often these days, sometimes, you still have to biff it.

nuhll
Filter Inserter
Filter Inserter
Posts: 922
Joined: Mon Apr 04, 2016 9:48 pm
Contact:

Re: Very slow downloads from the website

Post by nuhll »

BlueTemplar wrote:
Tue Mar 08, 2022 1:35 pm
just getting like 5 servers in different regions
Which would then be a CDN ?
No.
Static servers in different regions. Like steam does. (i know each region is not only one server)

But the speed problems never came back after they fixed it, so im happy anyway.

Post Reply

Return to “Technical Help”