Friday Facts #43

[slpwnd]

This time with a bit of vegetable flavor: http://www.factorio.com/blog/post/fff-43

[FishSandwich]

The multiplayer sounds more and more awesome every week. I can't wait to try it myself.

[TheDemonic]

Can't wait for multiplayer! This is only a small thing and not important at all but I would like buildings to have a streak of color to match the person who put it down. Eg. work arms would have the color of the player on their base but their arms would be the same so you can tell what type they are or the other way around. its only a small thing but it would be nice to see.

[Xterminator]

Multiplayer is sounding pretty awesome! And can't wait for the new character models
This is probably a dumb question, but how do 2 people play one the same computer at the same time? O.o

[MisterSpock]

Will there be a respawn for players?

[Nova]

Looks good. I like.

[bobingabout]

You mention player D being a relay for player A who disconnected. There is another method of handling this.

A Host.

In fact any serious multipler game would not be peer to peer as the way you described it is now, you'd have a dedicated server instances, either with player A on it, or no actual players at all. (Which is what I'd hope to see eventually, because I run instances of Minecraft and Terraria using a dedicated server on a nettop computer sat next to my router.)

Lets say in your example, player B is the host. in this case, A talks to B, and B alone, B talks to all 3 of the other players. Still assuming player D is 1 tick ahead, the "Tree was cut down in tick 101" would not be an issue, because B is still in tick 100, and the tree is not cut down, and therefore D never gets the "Tree is cut down" command in the first place.

But then you might end up with other problems, which would be real issues even in your existing setup. Internet lag, expect latencies as high as 2 seconds, I'm not kidding, I game with a guy in oz, and I'm in england, literally oposite ends of the earth, and it is impossible to get a ping to him below 400, pings as high as 650 are common, and 1000 isn't unheard of.

That's 60 ticks behind. So, what would happen if one player was in tick 260, and the other was still in tick 200, and sent a command saying "I mined a tree in tick 200", what'd happen then?

[Mark]

Looks awesome! I can't wait

[Zequez]

I'm really eager to try multiplayer!

[Redominus]

I'm really eager to try multiplayer!

Automatic gates, don't forget the automatic gates.

[SHiRKiT]

Please don't forget the gates.

[devedse]

You mention player D being a relay for player A who disconnected. There is another method of handling this.

A Host.

In fact any serious multipler game would not be peer to peer as the way you described it is now, you'd have a dedicated server instances, either with player A on it, or no actual players at all. (Which is what I'd hope to see eventually, because I run instances of Minecraft and Terraria using a dedicated server on a nettop computer sat next to my router.)

Lets say in your example, player B is the host. in this case, A talks to B, and B alone, B talks to all 3 of the other players. Still assuming player D is 1 tick ahead, the "Tree was cut down in tick 101" would not be an issue, because B is still in tick 100, and the tree is not cut down, and therefore D never gets the "Tree is cut down" command in the first place.

But then you might end up with other problems, which would be real issues even in your existing setup. Internet lag, expect latencies as high as 2 seconds, I'm not kidding, I game with a guy in oz, and I'm in england, literally oposite ends of the earth, and it is impossible to get a ping to him below 400, pings as high as 650 are common, and 1000 isn't unheard of.

That's 60 ticks behind. So, what would happen if one player was in tick 260, and the other was still in tick 200, and sent a command saying "I mined a tree in tick 200", what'd happen then?

I'm a developer that also has quite some experience in network programming. From my experience the guy I quoted is right. Some really strange things will start happening when you keep sending everything p2p if everyone is at different latencies (since there is not one true state in the game). Just one of the many examples could be that player A and B place a building on the same location on tick 101. Now player C receives the information from A first while player D receives the information from B first. Now both states are equally likely to be "the true state".
When you would use a Host instead this is what would happen: Player A and B build a building at the same location at tick 101. They both send their message to the server. Now whoever's package is received first (based on ping or just random luck) will have the building placed. Next the server will broadcast to all Players that a building has been build on position ... by player ..., when the messages send by player B is then received afterwards the servers can easily check if the building can actually be build on this place or not (in this case not, so this request will be ignored). This way you can also quite easily wait with substracting a guys items from the inventory untill the server actually tells the building has been build (or even just let the resources per player just be handles by the server completely)
(Also imagine problems related to taking resources out of a chest when doing P2P, you will also run into desync issues).

I know it will be quite a hassle to rebuild all the networking you've already been working on, but hell, it will save you so much time in the future (mainly because the concept of a "host" means that the game can't desync by design (if implemented correctly ofcourse)).
What you could still do for example is keep the players moving locally so you don't have problems with pressing the "Left" key and having to wait 500ms before you start walking. If you want you can do some verification on the server to see wether the player can actually move to this position at this specific time but I would not concern too much about this, if people want to abuse it, they will.

So summarizing, using the concept of a "HOST" the host would basically handle all commands of the players while the clients only "view" and send "requests" to the server instead of sending "actions" (Requests as in, "Place a building if possible" instead of "Place a building")

As far as my experience goes is that the only way I can see p2p networking work is in a LAN environment where you can literally keep all ticks going at the same speed (maybe a few ticks apart but you can then re-iterate through the information you received and rebuild the current state). Though even in this scenario a "host" would be much easier since he determines the true state of the game.


Looking forward to seeing multiplayer released so much, can't wait to play this with a few friends of mine.

[HeilTec]

P2P - Arbiter : Central or distributed ?

All this host talk got me thinking.
To avoid de-synchronization of the clients game states an arbiter is needed.
The classic client/server architecture puts all the arbitrating load on the server.
The clients could (randomly) take turns on the arbiter role.
This could be called a peer-to-peer distributed-arbiter system.

Since I've never designed such a thing my ideas may be far fetched.

Code: Select all

Initial p2p conections select random Arbiter
for each Tick
    p2p exchange of game state changes
    if designated Arbiter 
      send corrections for any inconsistencies
      select new arbiter
    else
       receive arbiter corrections and new designation 
       if arbiter times-out try to become new arbiter
   end
end

[DaveMcW]

The clients could (randomly) take turns on the arbiter role.
This could be called a peer-to-peer distributed-arbiter system.

Since I've never designed such a thing my ideas may be far fetched.

You really want the person with the best computer to be always be the arbiter.

[FreeER]

You really want the person with the best computer to be always be the arbiter.

I'm by no means an expert but wouldn't it actually the person with the lowest latency (and highest bandwidth if that's an issue) to everyone else that should be the arbiter? The biggest portion of the host's job is simply receiving and forwarding data, not doing intensive computations on that data, especially with the current implementation of Factorio's multiplayer. At least, from what I understand.

Also, I imagine switching Arbiters would have relatively significant overhead and should be minimized (though by no means should there not be any checks for a better arbiter). As for the "if arbiter times-out try to become new arbiter", there should always be a preselected backup arbiter for such situations, otherwise it's a 'free-for-all' as to who it ends up being (and also very likely conflicts since it's probable at times multiple peers would attempt to become arbiters simultaneously because they noticed the arbiter's absence simultaneously).

[Rakshasa]

I'm a developer that also has quite some experience in network programming. From my experience the guy I quoted is right. Some really strange things will start happening when you keep sending everything p2p if everyone is at different latencies (since there is not one true state in the game). Just one of the many examples could be that player A and B place a building on the same location on tick 101. Now player C receives the information from A first while player D receives the information from B first. Now both states are equally likely to be "the true state".

Except the state is never "the true state" before all clients send a message to end their updates for a particular tick.

Before all clients have sent the 'turn end' message, D will have received the building placement message from both A and B. Now by using a pseudo-random algorithm, with some known values as seed (message or user id's, etc), to pick which of A's or B's building placement messages is the authoritative. All clients now know that it is in this case B's message that should be part of "the true state" at the time changes are committed on tick 101.

[kovarex]

You mention player D being a relay for player A who disconnected. There is another method of handling this.

A Host.

In fact any serious multipler game would not be peer to peer as the way you described it is now, you'd have a dedicated server instances, either with player A on it, or no actual players at all. (Which is what I'd hope to see eventually, because I run instances of Minecraft and Terraria using a dedicated server on a nettop computer sat next to my router.)

Lets say in your example, player B is the host. in this case, A talks to B, and B alone, B talks to all 3 of the other players. Still assuming player D is 1 tick ahead, the "Tree was cut down in tick 101" would not be an issue, because B is still in tick 100, and the tree is not cut down, and therefore D never gets the "Tree is cut down" command in the first place.

But then you might end up with other problems, which would be real issues even in your existing setup. Internet lag, expect latencies as high as 2 seconds, I'm not kidding, I game with a guy in oz, and I'm in england, literally oposite ends of the earth, and it is impossible to get a ping to him below 400, pings as high as 650 are common, and 1000 isn't unheard of.

That's 60 ticks behind. So, what would happen if one player was in tick 260, and the other was still in tick 200, and sent a command saying "I mined a tree in tick 200", what'd happen then?

I'm a developer that also has quite some experience in network programming. From my experience the guy I quoted is right. Some really strange things will start happening when you keep sending everything p2p if everyone is at different latencies (since there is not one true state in the game). Just one of the many examples could be that player A and B place a building on the same location on tick 101. Now player C receives the information from A first while player D receives the information from B first. Now both states are equally likely to be "the true state".
When you would use a Host instead this is what would happen: Player A and B build a building at the same location at tick 101. They both send their message to the server. Now whoever's package is received first (based on ping or just random luck) will have the building placed. Next the server will broadcast to all Players that a building has been build on position ... by player ..., when the messages send by player B is then received afterwards the servers can easily check if the building can actually be build on this place or not (in this case not, so this request will be ignored). This way you can also quite easily wait with substracting a guys items from the inventory untill the server actually tells the building has been build (or even just let the resources per player just be handles by the server completely)
(Also imagine problems related to taking resources out of a chest when doing P2P, you will also run into desync issues).

I know it will be quite a hassle to rebuild all the networking you've already been working on, but hell, it will save you so much time in the future (mainly because the concept of a "host" means that the game can't desync by design (if implemented correctly ofcourse)).
What you could still do for example is keep the players moving locally so you don't have problems with pressing the "Left" key and having to wait 500ms before you start walking. If you want you can do some verification on the server to see wether the player can actually move to this position at this specific time but I would not concern too much about this, if people want to abuse it, they will.

So summarizing, using the concept of a "HOST" the host would basically handle all commands of the players while the clients only "view" and send "requests" to the server instead of sending "actions" (Requests as in, "Place a building if possible" instead of "Place a building")

As far as my experience goes is that the only way I can see p2p networking work is in a LAN environment where you can literally keep all ticks going at the same speed (maybe a few ticks apart but you can then re-iterate through the information you received and rebuild the current state). Though even in this scenario a "host" would be much easier since he determines the true state of the game.


Looking forward to seeing multiplayer released so much, can't wait to play this with a few friends of mine.

The problems you present would only occur in some very naive implementation of the p2p. In our implementation, the true state exists and is very well defined. If A and B order to build the building on the same location in tick 101, the tick is executed in 101+latency (lets say latency is 5), so in tick 106 it is executed, but every player executes tick 106 only when it has inputs from tick 101 from all players confirmed. In that moment, it executes the actions of all players in well defined order.

The host models doesn't avoid desynchronisation by design, still the input actions have to lead to the same game state, and the main problem with host model is, that it doubles the latency, as the message has to go to the host and back to the players.

[Cerus]

If A and B order to build the building on the same location in tick 101, the tick is executed in 101+latency (lets say latency is 5), so in tick 106 it is executed, but every player executes tick 106 only when it has inputs from tick 101 from all players confirmed. In that moment, it executes the actions of all players in well defined order.

I'm curious, what does that look like from each player's point of view?

Do Players A and B see their respective buildings placed instantly, then one sees it replaced by the "winner" of the tick evaluation, or does the building not appear until the result of the tick is agreed upon?

[cube]

If A and B order to build the building on the same location in tick 101, the tick is executed in 101+latency (lets say latency is 5), so in tick 106 it is executed, but every player executes tick 106 only when it has inputs from tick 101 from all players confirmed. In that moment, it executes the actions of all players in well defined order.

I'm curious, what does that look like from each player's point of view?

Do Players A and B see their respective buildings placed instantly, then one sees it replaced by the "winner" of the tick evaluation, or does the building not appear until the result of the tick is agreed upon?

We didn't work on this yet, so result of every action appears only after the safety window has successfully elapsed. Later we might add virtual results, that exist until the action is officially applied and then possibly disappear or change (this is what FPS games do)

[Rakshasa]

Is latency a (possibly dynamic) value, or does it rely on confirmation from all clients that they've completed every tick?