Multiplayer lag

[Tongs]

A friend and I have been playing a multiplayer map together for a while, and want to push this to megabase size, although it is not there yet. It is in that nebulous territory between "we have a main bus" and "we measure rocket launches in tens or hundreds per minute." The save game size is around 150 MB, although a good portion of those chunks are in "train-over" territory between the base and various mining outposts.

We have been hosting this on a headless server with good specs all along, and everything was fine until it grew to its current size. At first it was hosted on home internet, then it graduated to dedicated hosting - Amazon EC2, Google Cloud - and these are are mid-range hosts, with 2+ CPUs and 10GB+ of RAM, connected to fat cloud internet pipes.

None of this matters. While ping times are fine, the map can download and the client can catch up quickly enough, and UPS holds steady at 60, the game is very laggy. For example, I get sub-100ms ping to the current server, but at times my character is about as responsive as I am after a bottle of whiskey: standing around doing nothing despite instructions to move, then slamming into walls uncontrollably.

If either one of us loads the map single player, it plays fine at 60 UPS and good frame rate. If either one of us host on a LAN and connect to the local headless server, it plays fine. Once we cross over into internet territory, it seems like nothing will fix the lag.

My theory is that the clients and server all have plenty of resources to run the game well, but the sheer quantity of data feeding the game updates between multiple time zones is simply too much for Factorio to handle once a game grows past a certain point. I know 0.14 was the multiplayer tuning release and it certainly is better in 0.15, but we all know this game is very different from a typical FPS game. When there are tens of thousands of complex entities that need to update, that ends up being a ton of data flying across the wire.

Is there any hope to build a megabase in multiplayer if the players are not all on a LAN?

[orzelek]

Game doesn't send entity updates over multiplayer - they are simulated in parallel. Only player actions are sent so bandwith requirements shouldn't go up with base size.
You should check performance stats (F5 or in F4 menu) because it might be related to performance anyway.