Friday Facts #75 - False hopes

[slpwnd]

Better late than never: http://www.factorio.com/blog/post/fff-75

[Drury]

Hydrogen it is

[beny166]

engines powered by hydrogen fuel... hydrogen fuel... hydrogen... hydrogen bomb MUHAHAHA

[Drury]

Careful Kovarex... If you don't listen to doc you might end up not being able to code for good. I have my experience, my hands are a mangled crippled mess because I thought I knew better. Thankfully I can still type and game just fine, but manual labor is quite painful on the left hand and somehow I can lick both of my elbows as well as twist my arms in ways not imaginable.

I got lucky but it's not a rule.

[darci480]

Keep up the good work guys

I like the left version off the engine better!

Btw, when are the high-def graphics comming?

[VanVeenGames]

Keep up the good work guys

I like the left version off the engine better!

Btw, when are the high-def graphics comming?

Also, maybe we could have a downsampled texturepack as well? I think less pixels could yield a nice effect as well.
Refactoring Factorio must hurt so good. It's full of complex systems and there's probably always a way to make the code more elegant and faster. The bugs that arise from that are like "whack em" where you push one down to make another one pop up.

Good luck, looking forwards to doing something with all my materials :]
Also, can our robots come with us to space, or do we have to leave them behind Q.Q?

[Drury]

You can lower texture res for pixely effect.

[VanVeenGames]

You can lower texture res for pixely effect.

Not quite. It does create a nice pixely effect, but you can clearly see that most items are not rendered at the same resolution at all. High definition x128 copper plates in a 12x pixel assembler is really ugly. Also, the inserter hands are rendered as diagonal pixels in low resolution.

[Fatmice]

I see our desync bug still does not want to die.

[kovibear]

I was excited when I saw mention of rewriting some of the network code and hoped it meant we might see a change to a host-based system, rather than the CRC/P2P method.. and was a bit disappointed to see it was just a refactoring of other parts.

Oh well.. hopefully the quirks with the chosen method will be ironed out with time. Other than that, the updates having been looking great.

[58MadMax]

Ironic... the players keep restarting factorio because we think we can redo our factories and optimize it just a little better next time. The development team decided the same thing with the code.

It IS addictive.

[FishSandwich]

In the beginning of this week he went to see the doctor and actually got a kind of bondage for his left arm.

Today we learned that kovarex likes S&M.

[kovarex]

I was excited when I saw mention of rewriting some of the network code and hoped it meant we might see a change to a host-based system, rather than the CRC/P2P method.. and was a bit disappointed to see it was just a refactoring of other parts.

Oh well.. hopefully the quirks with the chosen method will be ironed out with time. Other than that, the updates having been looking great.

As it was discussed earlier, in factorio, the host system isn't really possible.

[Drury]

Ironic... the players keep restarting factorio because we think we can redo our factories and optimize it just a little better next time. The development team decided the same thing with the code.

It IS addictive.

Factorio is a pretty good parallel to actual coding.

My factories make an excellent illustration of my coding habits - unoptimized spaghetti mess, gives people who know what they're doing a heart attack.

[ssilk]

What I've learned some weeks ago: Factorio multiplayer architecture is quite similar or eventually it is a Shared Nothing. Currently the nodes have all the same data. The system tries to compute and display one simulation on many different hosts/screens. The system still works, if all but one host disconnects.

[Drury]

Yeah, as with all peer-to-peer systems.

Peer-to-peer used to be the norm, then Doom came about and showed how horrible it is for action games. Carmack then opted for server-client in Quake and since then, it has become the norm for gaming in general (only strategy games stayed on P2P).

P2P - lots of actions performed at once (thousands), low amount of users (generally up to 6)(game runs no faster than the slowest peer can communicate with every other peer+little to no safety)
Client/server - fewer actions per user (character movement), high amount of users (hundreds)(game speed independent of client performance, server handles safety of all clients)

The main difference between P2P and client/server is that in client/server, every client acts as a "remote control" - very little actual logic is being done on client hardware. When you press W in Quake, your PC does absolutely nothing to send your character forward, instead it tells the server that you pressed W, then the server itself actually moves your character forward in itself and sends a message back to your PC that your character moved forward, which then shows your character moving forward on the screen. Everything you do, every key you press, is sent to the server for processing, which then sends it's state back to you so you get feedback on your actions.

You can't have this in Factorio. Imagine each time a miner mined a piece of coal, the server would have to send information to all clients that a chunk of coal got mined. Imagine you have 300 miners on the map. Miners are just a random example - there's thousands of tasks being done in a factory at once. No way you can send all of that info to all clients without serious slowdowns (talking a few seconds/minutes per frame here) and clog up all bandwidth.

Factorio runs peer-to-peer which is vastly different. There is no server which holds all info about the game and no "dumb" clients that only transmit keyboard input. There are peers - every peer is a basic PC that runs the factory and everything that happens in it. Say there are two identical PCs that run the same map. Even without any type of connection going on between them, you could, in theory, have both PCs load the map at the exact same time and have them run the map simultaneously. When you look at the two screens, you can see the same actions being done - a chunk of coal reaches the boiler at the same time on both, inserter picks it up on both PCs at once, and places it in the boiler at the exact same time. Of course, eventually, they will desynchronize; microlag occurs on either of the PCs at various points in time and they end up running the same steps with delays. They will still make the same steps though; the same chunks of coal get put in the same boiler, just with a bit of delay on either PC. Peer-to-peer is based on this - every peer runs it's own game, and since it's the very same predictable game running at a constant speed for both, there is no need to send much info between. No information about chunks of coal mined is sent over the network, there's no need as both peers mine the same chunk at once anyway. Only if one of the peers stutters behind, the other peer detects a desynchronization and waits up while he catches up, then they go on each on their own again. Then there's player interaction - it's kind of similar to client-server, except now when you press W, you send information to all peers and each and every one of them sends back confirmation that you moved before you actually move on your screen. Similarly, you send all info about you placing stuff in the world to all peers.

As I said, Doom had this, and it was bad for Doom because if somebody decided to connect from Mars (Doomguy himself presumably), the game suddenly slowed to crawl for all players since every peer had to wait for their info to reach Doomguy on Mars and then wait for Doomguy to confirm every peer down on Earth. What's more, you couldn't have a lot of players participating at once because you can't afford sending information to 3000 peers on your crappy Wi-Fi connection. In server-client, when the Quakeguy connects from another realm of existence, he'll be stuttering but the server won't wait for him, it sends info at a constant rate to all clients and if Quakeguy's connection can't keep up, it simply won't send any info regarding him (Quakeguy seemingly stands still doing nothing for a while on every other client). You can have a lot of players connected to one server, provided the server is sufficiently powerful. As far as basic home PC clients are concerned, information is only being sent to a single machine and back no matter how many players are connected to that machine.

[bobingabout]

Before I start, a quick point. You've mentioned 0.11.17 quite a few times, talking as if it has been released. There is no 0.11.17 in the releases section of this forum.


You having said that (Host-Server being the norm for everything, except stratagy that uses P2P), Spring (A game I earlier sugested you take a look at) is a server-host based multiplayer stratagy game system, that works flawlessly for massive quantities of units.

so the excuse that host-server isn't possible to do with factorio, is simply that, an excuse, because you don't want to do it.

When you press W in Quake, your PC does absolutely nothing to send your character forward

I don't know how it is for other games... but this is dumb. this is an obsolete concept, for a simple reason. Reaction time. when you press W, you want to move forward, you don't want to be sitting for half a second waiting for the confirm command to return to you, before you move forward (this is actually how it apears to function in factorio Multiplayer right now) What you have in most games is a degree of "client trust", your char is where your computer says it is, not where the server says it is, which is why in older games in a "laggy" game, charactors apear to teleport from position to position. this is caused by a low quallity connection. but this doesn't apply to factorio so...


Factorio is Procedural and Determinalistic, this means that what happens on one computer is almost certinately the same that happens on all others running the same simulation from the same set of variables, there is just one thing that is not constant between computers. what the player does.

this actually gives you a MASSIVE boost for any type of multiplayer system, being P2P or Host-Server.



Lets first assume P2P relationship, because, many of the posts here, and even you, the devs, insist is the only options.
From the experience I have with multiplayer, you have a pre-defined server "latency" setting, lets say 500ms (because, anything else is stupid when trying to play with someone on the oposite side of the planet) You press W to move north, you wait half a second, then you start to move north. Why? Because you have those 30 ticks worth of action pre-processed, so when it sends the "I pressed W" command to the other peers, the commands are all syncronised.

This is actually the single-most annoying thing about Factorio Multiplayer, I do understand why you've done it, but it is a very nasty solution to the problem.

Introduce client trust. Press W, you should instantly see yourself starting to move on screen, with no lag caused by latency. The command is send to all other peers that the player moved, and sends a projection of where the player will be in 30 ticks from now, using known information. on the other peers, your charactor will apear to teleport to the new position, and continue moving as projected. Press A too to move north west, and a similar effect will ripple though.

This will only cause problems in instances where you may recieve damage, in which case, these scenarios need to be factored in as well.

Placing of entities can quite easilly use either method, and be perfectly fine, one of the biggest irritations at the moment though is trying to place a line of belts, the first has the 30 tick delay, the second cannot be placed untill the first has been placed. This needs to change, so that a command to place a second may be given while the first hasn't actually been placed yet. this will require comparing your placement command in the currently vissible location to the active latest tick, rather than the vissible tick, to prevent mass placement commands given within the same tile while a player is trying to place entities while in motion.


Now lets add the Host-Client relationship. What's changed? Firstly, you have an extra computer added to the eqasion, simulating everything, just as all the clients are, just as all the peers did in the previous scenario. The host MAY be a non-player instance of the game that other players can connect to, or MAY be one of the actual players (Which is actually as it is now, the player whom creates the internet game is the host, he is who the other players have to connect to, and when desyncs happen, this "Host" peer sends out the map, etc). What else has changed? When you press W, in a true Host-Client relationship, this information would first be relayed to the server, and then from the server to every client. this is not mandatory behaviour however, this W can just as easilly be transmitted to all users without the host relay.

One of the existing features of favtorio is the CRC, now I've mentioned before that this happens WAY too often, there is so much traffic going on over the internet that the game is unplayable, you get 1 tick every couple of seconds. tone it down. one of the differences of host-client relationship would be... All CRC checks get sent to a single user, the host, not to other peers. if one of these is off on any one client, the game is paused and ONLY the one client has to redownload the map. This is a degree of traffic control. Why should an ADSL using peer be forced to transmit as much data as everyone else, when his download is 10, 20, 50 times faster than his upload, as is standard on most home based internet connections. This brings us back to why have a host? It may be hosted on a server somewhere designed with a high output bandwidth, where the standard peer is not.



One more important point on the traffic thing... Okay, you may assume people have good internet connections, in reality this is not the case, The reason why I can't play factorio multiplayer with my friend is internet, the internet connection is too slow. on almost every other game we play together, any issues are usually caused by the lack of processing power of one of our computers. Even the Spring engine I mentinoed earlier that has thousands of syncronised units, our lag and issues are caused by CPU and GPU deficciencies, not slow internet. Factorio can be quite a CPU hog, we should not be bottlenecked by a slow internet connection.



So, a closing point, A challenge to the devs, one that you really should take quite seriously. play Factorio over a 56k modem connection. Does it play? if the answer is yes, well done. if the answer is no, Go back to the drawing board, because you fail at net-code. In the realm of internet of the modern age, you might think I'm being silly to sugest such a thing, but in reality, not everyone has the super awesome speeds of american internet, 56k is being VERY generous. You don't even have Latency to worry about, because the cable is probably only going to be 3 foot long, you only have speed to worry about.

A more practical test for this problem, use a net speed limiting program, such as net limiter, to limit the speed of factorio to 6 Kilobytes a second. Similar end result. if you can't play the game over your LAN with a 6 kBps limit, fix it so you can.

[Drury]

I mentioned Quake specifically because there was no client prediction yet, for the sake of simplicity/clarity. I believe Counter Strike came up with that and pretty much every modern game uses it. Quake didn't and most strategy games don't either.

What would that bring to Factorio? Well other than no delays, you'd get the nasty stuff that comes with it. Rubberbanding, visual glitching. It'd add a whole lot of bugs to fix some of which would be plainly unfixable. I've frankly never played a modern MP game where I never got sniped through an impenetrable wall while I obviously turned a corner just a moment ago. Can't imagine getting teleported a few tiles and get ran over by a train while I never actually crossed the tracks or dying to biters who already died, to name some potential issues. The benefits? Woooo smooth movement most of the time (when there's no rubberbanding and the connection is actually good). Client prediction isn't for complex strategy games. It doesn't add much value really in this context. Biters aren't Unreal Tournament pros, they move predictably and fire homing projectiles, there's really no incentive to make that interaction any smoother.

[ssilk]

A more practical test for this problem, use a net speed limiting program, such as net limiter, to limit the speed of factorio to 6 Kilobytes a second. Similar end result. if you can't play the game over your LAN with a 6 kBps limit, fix it so you can.

You forget, that 0.11 is not thought for that. The devs mentioned it several times. You can read it everywhere: Not made for that.

0.11 is made for a LAN, 1 Mbit/sec minimum, low latency.

What you anger about will be made for 0.12, from the Roadmap:

Working Multiplayer.

This means, that you need to wait. 0.11 is playable over internet, even if that wasn't planned. That is a wonder, if you look at the current architecture.
0.12 will solve the problems with this architecture.

[GewaltSam]

Before I start, a quick point. You've mentioned 0.11.17 quite a few times, talking as if it has been released. There is no 0.11.17 in the releases section of this forum.

And if you read the newest blog entry in an observing manner, you'll get that the whole post is about why the release of 0.11.17 this week was merely "false hope" (see blog entry title) by the devs, and what the reasons are for a further delay of said version

You having said that (Host-Server being the norm for everything, except stratagy that uses P2P), Spring (A game I earlier sugested you take a look at) is a server-host based multiplayer stratagy game system, that works flawlessly for massive quantities of units.

so the excuse that host-server isn't possible to do with factorio, is simply that, an excuse, because you don't want to do it.

Spring. You're kidding, right? From their homepage:
"- Large battles limited only by the power of your computer; support for up to 5000 units."
5000 units? That may be "massive" for a standard RTS. Pal, we're talking Factorio here, 5000 units is what you have moving in the third tutorial mission

I don't know why that is still up for discussion. Even if it would be possible to go for host-based multiplayer at this point, they decided otherwise. I am no expert in the matter (and after what I read, neither are most people here who want a host-based system), but it's pretty clear why that would be at least difficult and highly impracticable, if not impossible (they would have to rewrite virtually every piece of code for MP, maybe some other more general systems, and there wouldn't even be any form of guarantee that it would run better - on the contrary, I believe).

Now lets add the Host-Client relationship. What's changed? Firstly, you have an extra computer added to the eqasion, simulating everything, just as all the clients are, just as all the peers did in the previous scenario. The host MAY be a non-player instance of the game that other players can connect to, or MAY be one of the actual players (Which is actually as it is now, the player whom creates the internet game is the host, he is who the other players have to connect to, and when desyncs happen, this "Host" peer sends out the map, etc). What else has changed? When you press W, in a true Host-Client relationship, this information would first be relayed to the server, and then from the server to every client. this is not mandatory behaviour however, this W can just as easilly be transmitted to all users without the host relay.

So, the old system is kinda too slow and too laggy for your standards (see next quote), but you really wish that every piece of input should first go from client to a host, and then to every other client? And you would need to do that; clients sending input to other clients is in direct contradiction to the suggestion of host-based gameplay to begin with...

One more important point on the traffic thing... Okay, you may assume people have good internet connections, in reality this is not the case, The reason why I can't play factorio multiplayer with my friend is internet, the internet connection is too slow. on almost every other game we play together, any issues are usually caused by the lack of processing power of one of our computers. Even the Spring engine I mentinoed earlier that has thousands of syncronised units, our lag and issues are caused by CPU and GPU deficciencies, not slow internet. Factorio can be quite a CPU hog, we should not be bottlenecked by a slow internet connection.

Have you thought about how much information you have to receive if the whole simulation needs to be sent and updated on every client by a host in a constant fashion? Simulating that on your CPU isn't that hard (the determinism is), but sending all of those simulation results to every client would be a PRETTY hefty chunk of data every 1/60th of a second... Just sayin'. Imagine this: In an FPS, the host sends you updates about the xyz-coordinates of every player (let's say 10-50 other players), their keystrokes (shooting, jumping, skills, etc.), the same thing for NPCs, maybe the position of projectiles like rockets and such. There isn't "that" much more which needs to be send to you by the host.

Now let's imagine for a second that Factorio would get such a host-based system: You have for example a 200 tile long belt with, say, 2000 parts on it (and you know that this really isn't much, and you got a LOT more stuff even in a small-ish factory). Now, if the Factorio mechanics wouldn't be changed very much from what they are now, the host would simulate the movement and position of those parts 60 times per second. This information (let's say a simple xy-coordinate for every part) needs to be send to EVERY client who participates. Every time. Maybe it would be possible to work with less (sent) updates per second, which would be on the expense of an exact simulation (or at least on the expense of an exact representation of said simulation). The content of a belt wouldn't be visualized exactly, but more in an "approximately this many stuff" kind of way. Imagine a Sim City road (in one of the older games), where not every car is really a car in the game, but part of the traffic density visualization on said road. Factorio would need to adopt similar systems, because, like I said, 60 updates per second sent to every client is simply way too much information. I guess it would play a lot more like, say, the Anno games than what it is now.

I can't believe that your internet connection really is the problem. May I ask what kind of connection you and your friend are having? I am playing with someone literally (at least virtually ) on the other side of the world (Central Europe to California). We both have passable connections, but nevertheless 200ms just because of the distance. 200ms lag isn't great to begin with, but it's fine after one gets used to it. And other than the map download, I can't say that there is that much traffic going on between us (although the map is 40MB big, and the simulation itself is getting so complex that my CPU can't keep up without some engine performance tweaks; that's another story though).

So, to conclude this thing, or tl;dr:
If you look at the facts, there really is no good reason to change the system from P2P to host based. Most people aren't used to P2P multiplayer, but it absolutely makes sense to do Factorio that way. If you would change it to host-based, the amount of traffic you get would get bigger, not smaller; only thing that would change is that you only had connection to the host, and not directly to every other client. But this host would need to send you WAY more data than all clients need to do now - because they send you only their player input, not the whole simulation. There are a lot of discussions about this in the forum, and I hope that I haven't started something off topic here, but I really needed to say those things

Bob, I am sorry if it seems like I try to bash you here. I can assure you that this is not my intention. But I think that a lot of your arguments aren't thought out very well, and I read similar in other threads. So I took the chance and added my two cents here. No hard feelings bro