Player API: latency buffer jumps

[kizrak]

I've been unable to google or search these forums for my question, so maybe it isn't in the API, so...

Is there a way for a mod (via api) to get each players (debug settings show-multiplayer-statistics) latency, buffer(s) and jumps?

show-multiplayer-statistics : latency, buffer(s) and jumps (Thanks @Koub !)

debug_settings2.png (971.48 KiB) Viewed 8801 times

With this data, I'd like to make a softmod (and/or hardmod) that would automatically slow the game.speed to accommodate the slowest players (and their computers) on the server (I happen to often be one of them).

If there was a way for me to collect this data, the server could automatically slow the server to allow those people to catch up when the megabases start to get really intense.

Additionally, there there any api calls that could tell me which players are downloading and/or catching up with the server? I'd also like to be able to slow/pause the game for new players joining the map to be able to catch up.

Sorry if this already exists in the API and I'm just too slow to find it.

Thanks for any help you can provide!
-Kizrak the Hokie

[Koub]

kizrak, unrelated, but I fixed your image tags (you linked to the page containing the images, not the images themselves). However, they seem very similar to me, maybe did you try to upload the same one to different hosting services.
Pro tip : upload it to the forum it's quick, easy, and the image will be kept as long as the forum exists .

[Bilka]

Is there a way for a mod (via api) to get each players (debug settings show-multiplayer-statistics) latency, buffer(s) and jumps?

No.

Additionally, there there any api calls that could tell me which players are downloading and/or catching up with the server?

No.

None of these api requests will ever be added because the information is not deterministic. You are not allowed to change the gamestate (so, do anything) based on such information.

[eradicator]

This makes me wonder what the usual "teleporting backwards" that happens when playing on a really bad connection looks from inside the game engine. Maybe you can "detect" lag by weird player movement :p.

[BenSeidel]

makes me wonder what the usual "teleporting backwards" that happens when playing on a really bad connection looks from inside the game engine. Maybe you can "detect" lag by weird player movement :p.

No, that is a graphic-only issue related to the way the lag-hiding works. The simulation engine is unable to access this information, so there is no way for the scripts that run in it to access it.

None of these api requests will ever be added because the information is not deterministic

All non-determinism is strictly prohibited by Factorio including, but not limited to, system times, file systems and user input.

[kizrak]

From what I read, chat messages can be sent (by player, or api) and then deterministically parsed by server (and clients). Is this true?

Also, user input is not deterministic? So there are mechanisms in the game that allow non-deterministic data to be processed in a deterministic way. For example, when I tell my character to move left (I, the user, am non-deterministic), but the game moves this into the deterministic game-state, which can then be processed by all other clients (and server). Is this also correct?

Evidence for above assumptions:

on_console_chat
Called when someone talks in-game either a player or through the server interface.

Contains
player_index :: uint (optional): The player if any.
message :: string: The chat message.

defines.events.on_console_chat

on_console_command
Called when someone enters a command-like message regardless of it being a valid command.

Contains
player_index :: uint (optional): The player if any.
command :: string: The command as typed without the preceding forward slash ('/').
parameters :: string: The parameters provided if any.

Is there not a way for a mod/script to trigger the above events?

Assuming these are correct, then, assume that clients can get their own latency, buffer & jumps data; then couldn't a client then decide (non deterministically) when it is experiencing too much lag and send a message (at particular tick in the game deterministically) that when the other clients/servers receive, they determinically act on that message (for example by reducing the game.speed)?


And thank you all for the help so far!

[kizrak]

This problem has come to my attention because my client was falling behind the server, but that gets me thinking:
How do clients response when it is the server that falls behind, and isn't able to maintain the desired UPS? How is that non-deterministic behavior (quality and performance of the server CPU) received by the clients?

This appears to me to artificially change the game.speed, which then makes me thing that game.speed isn't really part of the game state (or at least the deterministic part of it). Does each tick remember how fast it was suppose to go? Is that information (game.speed) use to determine the state of the next tick? Shouldn't the order/behavior/structure/response of the deterministic game state of the map be independent of game.speed? If I run the game faster or slower, will I get different results?

Could someone let me know where my logic is failing?

Thanks!
-Kizrak

[eradicator]

makes me wonder what the usual "teleporting backwards" that happens when playing on a really bad connection looks from inside the game engine. Maybe you can "detect" lag by weird player movement :p.

No, that is a graphic-only issue related to the way the lag-hiding works. The simulation engine is unable to access this information, so there is no way for the scripts that run in it to access it.

Let me explain. Lag hiding on the client pretends that every input action (i.e. "move forward") will succeed. But when some of the commands are rejected due to being too late (lag), or lost (package loss) then on the client the character behaves like it is being "teleported backwards". Sure the game state does not know why this happend. But what i mean was that a script could take an edjucated guess that a player might be lagging if they constantly move in a stop-go-stop-go-stop-go pattern due to rejected input.

None of these api requests will ever be added because the information is not deterministic

All non-determinism is strictly prohibited by Factorio including, but not limited to, system times, file systems and user input.

Only until they make it deterministic though, like i.e. player screen resolution ;p. I've already seen (or did i write that myself?) a proposel to sync the average UPS of the last minute once a minute. This would make scripts like described in the OP possible without causing lots of network traffic.

Ofc the optimal solution would be if the nice guys at wube simple implemented UPS throttling to match the weakest machine as a game hosting option... (I say that from the perspective of the guy that is usually the one suffering from it due to bad internet and old pc :p.)

[Bilka]

The person you are replying to seems to have misunderstood me, but I'll try to explain user input to you.

From what I read, chat messages can be sent (by player, or api) and then deterministically parsed by server (and clients). Is this true?

Also, user input is not deterministic? So there are mechanisms in the game that allow non-deterministic data to be processed in a deterministic way. For example, when I tell my character to move left (I, the user, am non-deterministic), but the game moves this into the deterministic game-state, which can then be processed by all other clients (and server).

For players, this is done using input actions. (Script actions are not synced - this is why mods can cause desyncs). An input action is something like moving, mining, sending a chat message, etc. These actions are saved in the map when they happen, and are sent to the other clients when playing multiplayer, meaning that they are deterministic. This is also how the replays work, the simply replay the input actions.

Assuming these are correct, then, assume that clients can get their own latency, buffer & jumps data; then couldn't a client then decide (non deterministically) when it is experiencing too much lag and send a message (at particular tick in the game deterministically) that when the other clients/servers receive, they determinically act on that message (for example by reducing the game.speed)?

Yes, that would work if implemented in the game, but is not within the capability of mods. The message concept that you describe is basically what an input action is.

[Loewchen]

Shameless plug: Option to make server wait for slowest client.

[kizrak]

Shameless plug: Option to make server wait for slowest client.

Where is the proper place to show my support for such an amazing awesome improvement!?!

[Loewchen]

Shameless plug: Option to make server wait for slowest client.

Where is the proper place to show my support for such an amazing awesome improvement!?!

A comment in the topic is all you can do. I doubt it will get implemented though, it had been in discussion before and after the MP rewrite and nothing came from it so I doubt someone dares to pick it up when development concentrates on polishing existing features, also the new rendering mechanic in 0.17 will make it look even less necessary.

[eradicator]

Shameless plug: Option to make server wait for slowest client.

Where is the proper place to show my support for such an amazing awesome improvement!?! 8-)

A comment in the topic is all you can do. I doubt it will get implemented though, it had been in discussion before and after the MP rewrite and nothing came from it so I doubt someone dares to pick it up when development concentrates on polishing existing features, also the new rendering mechanic in 0.17 will make it look even less necessary.

The new rendering at best will shift "when it happens" a bit further back. But imho the auto-throttling option is definetly a feature that must be in the release version. Currently it's one of the weird things where it "feels" correctly in single player, but in mp feels broken. And having ample experience with being "the weakest link", it is incredibly annoying/difficult/exhausting to explain to the other people with better computers that i'm unable to meaningfully participate if they don't use command to down the speed. Because from their perspective everything runs fine. And they don't want to lose achievements............*sigh*.

[BenSeidel]

The person you are replying to seems to have misunderstood me, but I'll try to explain user input to you.

I'm assuming that I'm "The person" in this statement. If so, I didn't misunderstand you at all. It was a remark designed to show how extremely ridiculous the idea that it being "not deterministic" is the reason why some piece of information is inaccessible to the game engine.

There are MANY reasons why this information isn't accessible to the game engine, as well as why the game speed doesn't slow down to the lowest common denominator. The one that pops into my head immediately is multi-player trolling. Join a server, then to annoy everyone, throttle your game. It's even better to do it when someone else joins, then everyone will blame that person!

Some other questions that you just can't answer, but need to be answered: At what stage do you increase the speed (after it's been slowed)? Do you ensure that everyone gets a smooth performance, or is allowing clients to drop frames OK?

"Every feature starts of at -100 points. It must get above 0 before it can be considered" - Raymond Chen
This feature really can't be justified when the solution is currently available: Get a better computer OR lower the game speed OR start a new game OR save it and play single player. Would you like developer time spent on this issue OR would you like them to spend that time on other things?

The whole "It's not deterministic" thing is getting old...

[Rseding91]

The whole "It's not deterministic" thing is getting old...

That, and "I refuse to do it" are the only 2 reasons why information isn't exposed in the mod API. You might not like it but that's simply how it is: if information isn't deterministic then mods may never have access to it. In order to make information deterministic it has overhead which most of the time we aren't willing to pay just for mods to get access to it.

[mrvn]

The whole "It's not deterministic" thing is getting old...

That, and "I refuse to do it" are the only 2 reasons why information isn't exposed in the mod API. You might not like it but that's simply how it is: if information isn't deterministic then mods may never have access to it. In order to make information deterministic it has overhead which most of the time we aren't willing to pay just for mods to get access to it.

Also if the mod had access to it it could put an iron plate into a chest whenever a player looses a buffer or something else that alters the game state. There would have to be serious restrictions on what a hook for this information can do for it to work.

[BenSeidel]

"I refuse to do it"

See, some honesty

Being this late in the development and being so close to version 1.0, I doubt you would want to attempt what I would assume to be a complete re-write of the user input/state synchronisation system. Especially given the amount of attack surface that would expose.

Also if the mod had access to it it could put an iron plate into a chest whenever a player looses a buffer or something else that alters the game state

Isn't that the mod's choice? If you don't like it, don't play with that mod.

[mrvn]

"I refuse to do it"

See, some honesty

Being this late in the development and being so close to version 1.0, I doubt you would want to attempt what I would assume to be a complete re-write of the user input/state synchronisation system. Especially given the amount of attack surface that would expose.

Also if the mod had access to it it could put an iron plate into a chest whenever a player looses a buffer or something else that alters the game state

Isn't that the mod's choice? If you don't like it, don't play with that mod.

It's more a problem that 99.9% of anything the mod does is an instant desync.