sorry for the necro-bump. i've got follow-up information from investigating issues this morning.
first off, i'm so sorry for forcing Wube to waste time on something I should have spent more time validating. even opening up Wireshark would have shown me a few questionable details.
secondly, i have a fever. I might not be coherent. however, this morning I had the idea to investigate TCP Window Scaling and its function in my life, after having to restart a download every 30 seconds so that it completed.
basically, disabling TCP Window Scaling forces (on Linux, anyway) the window size to a fixed 65535 which means we send as much data as a TCP packet naturally allows, before considering the 'scaling factor'. This results in a CONSTANT throughput of 10mbps, but it can't go any higher on a single TCP session. Not with my current RTT latency, anyway. For the same Window size on different latency connections, you can get varying throughput. 10mbps is somewhat ideal, i've had 5mbps this way to further distant networks.
but then I went into Windows to figure out whether it's a Linux-specific problem. Something to note here, is, if I'm tethered to my phone that uses the same LTE provider, it works fine! My theory is there's something wrong with the modem they provide, that I can not change. I've put it into Bridge mode and turned off any kind of filtering/QoS and alas, nothing changes this behaviour...
except...
putting these values into sysctl.conf:
Code: Select all
net.core.wmem_max = 4194304
net.core.rmem_max = 12582912
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 87380 4194304
and reenabling TCP Window Scaling. Now, my downloads start out "slow" and gain speed as they go, instead of beginning deceptively fast, and cliff-diving.
I should note that using `iperf3` to test the Window size is what led me to searching how to increase these values, as, the default max window size on Linux, is a mere 250kB
For what it's worth, the limits are the same on Windows.
anyone with a very fast network connection (pretty much any modern ISP) should revisit their TCP tunings, because despite praise that things "Just Work" a lot more often these days, sometimes, you still have to biff it.