Dealing with large operations in a single tick.

[Danacus]

Hello

Let's say I have a mod that allows users to teleport to new surfaces. When a player enters a portal, a new surface might have to be generated, a new portal must be built on the new surface and the player must be teleported. Now let's say you're playing a multiplayer game and 10 players do this in the same tick (unlikely, but not impossible).

A large amount of stuff would need to be done within that tick. Could this be problematic and possibly cause desyncs in multiplayer or create large UPS drops?

The way I've been dealing with this stuff is to make some kind of task queue and make the game execute the next task every couple of ticks, to spread the load. Every task would do a rather small amount of work, like "Create surface A" or "Teleport player B to surface C". This however creates a couple of new difficulties like storing the state of these tasks in the queue between saves and loads (and sending this state to players joining the game), because every task refers to a function that needs to be called, which cannot be serialized. I've been able to work around this be re-registering the functions that run for each task in a table every time a client loads the world, which works fine. (the table maps task names to functions) A similar problem arises when I want to create some kind of callback system for each task, as these callback functions cannot be serialized either.

My question is whether it is indeed necessary to do these kinds of things. If so, how do you deal with this?

[Klonan]

A large amount of stuff would need to be done within that tick. Could this be problematic and possibly cause desyncs in multiplayer or create large UPS drops?

In your case, probably not. It could create UPS drops, but i would assume it only happens once in a while, not every 10 seconds or so, so its fine.
Desyncs can't occur from large script usage, but potentially clients ca be dropped, if the time to execute the script is move than 30 seconds or something (highly unlikely)

My question is whether it is indeed necessary to do these kinds of things. If so, how do you deal with this?

Not really, just keep it simple and readable. Do it all at the same time, players won't really know the difference.

[Danacus]

Thanks for your advice! It's good to know I was most likely overengineering things. UPS drops aren't a big deal as long it doesn't cause any other issues. I thought desyncs could also be caused by very large changes in the game's state, but I guess this is not the case.

The reason I was confused is because I saw a mod using this kind of task system to supposedly increase stability in multiplayer. Maybe that's something from before 0.14, as it was an old mod.

[mrvn]

For anything that involves player action, especially player movement I would do it all in the same tick. The player will be waiting for the result and having the result delayed for a number of ticks can cause inconsistencies. For example the player walks into the teleport portal, you put the teleport into the queue and the player keeps walking, walking out of the portal. Suddenly 2 seconds later the player teleports.

That said in multiplayer if anyone walks into portal all other players game will freeze. Even more annoying maybe? Helmod does (did?) this every time it filters recipes, which is basically every third click. It can become annoying to other players when you use helmod extensively.

If you can why not generate a (number of) surface and portal ahead of time in a global cache. Then when the player needs one you check if any are cached and take one. The head of time generation can be split across many ticks so as to be unnoticeable.

[Danacus]

For anything that involves player action, especially player movement I would do it all in the same tick. The player will be waiting for the result and having the result delayed for a number of ticks can cause inconsistencies. For example the player walks into the teleport portal, you put the teleport into the queue and the player keeps walking, walking out of the portal. Suddenly 2 seconds later the player teleports.

That's one of the issues I'm having right now is that it takes some time to generate the chunks on a surface, so the player has to wait. For that I still have to use some sort of queue, which could be an annoyance like you mentioned. Every 10 ticks I check all surfaces in the queue to check if the surface is generated at the target position.

That said in multiplayer if anyone walks into portal all other players game will freeze.

I would like to avoid that, but I don't think that will be an issue, since I'm not actively waiting for a surface to be generated.

If you can why not generate a (number of) surface and portal ahead of time in a global cache. Then when the player needs one you check if any are cached and take one. The head of time generation can be split across many ticks so as to be unnoticeable.

That could be an option, but it would be a little unreliable, as it would not always be the case that the surface a player wants to teleport to is cached. One way would be to generate the target surface when a portal is placed, not when a player enters a portal. I could get rid of the above mentioned player queue to prevent players being teleported 2 seconds after they enter a portal. A visual way to show if a portal is active or not might also be a good idea.

[mrvn]

For anything that involves player action, especially player movement I would do it all in the same tick. The player will be waiting for the result and having the result delayed for a number of ticks can cause inconsistencies. For example the player walks into the teleport portal, you put the teleport into the queue and the player keeps walking, walking out of the portal. Suddenly 2 seconds later the player teleports.

That's one of the issues I'm having right now is that it takes some time to generate the chunks on a surface, so the player has to wait. For that I still have to use some sort of queue, which could be an annoyance like you mentioned. Every 10 ticks I check all surfaces in the queue to check if the surface is generated at the target position.

You can force the generation of the chunk(s) the portal is on. Unless your portal is awfully huge that's a maximum of 4 chunks. You can block that long. Then the player can teleport and the remaining chunks will appear around the player over time by itself. Maybe not the best look but better than delaying the teleport I think.

If you can why not generate a (number of) surface and portal ahead of time in a global cache. Then when the player needs one you check if any are cached and take one. The head of time generation can be split across many ticks so as to be unnoticeable.

That could be an option, but it would be a little unreliable, as it would not always be the case that the surface a player wants to teleport to is cached. One way would be to generate the target surface when a portal is placed, not when a player enters a portal. I could get rid of the above mentioned player queue to prevent players being teleported 2 seconds after they enter a portal. A visual way to show if a portal is active or not might also be a good idea.

Maybe give the portals a glow effect that turns on after the surface is generated as visual feedback that the portal is now active. I like that idea the best.

[Danacus]

For anything that involves player action, especially player movement I would do it all in the same tick. The player will be waiting for the result and having the result delayed for a number of ticks can cause inconsistencies. For example the player walks into the teleport portal, you put the teleport into the queue and the player keeps walking, walking out of the portal. Suddenly 2 seconds later the player teleports.

That's one of the issues I'm having right now is that it takes some time to generate the chunks on a surface, so the player has to wait. For that I still have to use some sort of queue, which could be an annoyance like you mentioned. Every 10 ticks I check all surfaces in the queue to check if the surface is generated at the target position.

You can force the generation of the chunk(s) the portal is on. Unless your portal is awfully huge that's a maximum of 4 chunks. You can block that long. Then the player can teleport and the remaining chunks will appear around the player over time by itself. Maybe not the best look but better than delaying the teleport I think.

I know this is possible, but I'm generating a cave surface with perlin noise and it's quite heavy. I might have to do some optimizations. I divide chunks into subchunks and throw them on a queue, so I can generate them one by one. I don't know if generating the surface in a blocking way would even be an option in this situation.

If you can why not generate a (number of) surface and portal ahead of time in a global cache. Then when the player needs one you check if any are cached and take one. The head of time generation can be split across many ticks so as to be unnoticeable.

That could be an option, but it would be a little unreliable, as it would not always be the case that the surface a player wants to teleport to is cached. One way would be to generate the target surface when a portal is placed, not when a player enters a portal. I could get rid of the above mentioned player queue to prevent players being teleported 2 seconds after they enter a portal. A visual way to show if a portal is active or not might also be a good idea.

Maybe give the portals a glow effect that turns on after the surface is generated as visual feedback that the portal is now active. I like that idea the best.

I like that idea too, I'll see what fits best.

[mrvn]

For anything that involves player action, especially player movement I would do it all in the same tick. The player will be waiting for the result and having the result delayed for a number of ticks can cause inconsistencies. For example the player walks into the teleport portal, you put the teleport into the queue and the player keeps walking, walking out of the portal. Suddenly 2 seconds later the player teleports.

That's one of the issues I'm having right now is that it takes some time to generate the chunks on a surface, so the player has to wait. For that I still have to use some sort of queue, which could be an annoyance like you mentioned. Every 10 ticks I check all surfaces in the queue to check if the surface is generated at the target position.

You can force the generation of the chunk(s) the portal is on. Unless your portal is awfully huge that's a maximum of 4 chunks. You can block that long. Then the player can teleport and the remaining chunks will appear around the player over time by itself. Maybe not the best look but better than delaying the teleport I think.

I know this is possible, but I'm generating a cave surface with perlin noise and it's quite heavy. I might have to do some optimizations. I divide chunks into subchunks and throw them on a queue, so I can generate them one by one. I don't know if generating the surface in a blocking way would even be an option in this situation.

I made something similar for a maze generator. But I divided the surface into 32x32 tile chunks. Exactly matching the factorio chunks. And then I put the generation of chunks into the on_chunk_generated event. My maze is infinite (to the limits of the game) so I had to use an on-demand approach. Maybe the same works for you.

[Danacus]

For anything that involves player action, especially player movement I would do it all in the same tick. The player will be waiting for the result and having the result delayed for a number of ticks can cause inconsistencies. For example the player walks into the teleport portal, you put the teleport into the queue and the player keeps walking, walking out of the portal. Suddenly 2 seconds later the player teleports.

That's one of the issues I'm having right now is that it takes some time to generate the chunks on a surface, so the player has to wait. For that I still have to use some sort of queue, which could be an annoyance like you mentioned. Every 10 ticks I check all surfaces in the queue to check if the surface is generated at the target position.

You can force the generation of the chunk(s) the portal is on. Unless your portal is awfully huge that's a maximum of 4 chunks. You can block that long. Then the player can teleport and the remaining chunks will appear around the player over time by itself. Maybe not the best look but better than delaying the teleport I think.

I know this is possible, but I'm generating a cave surface with perlin noise and it's quite heavy. I might have to do some optimizations. I divide chunks into subchunks and throw them on a queue, so I can generate them one by one. I don't know if generating the surface in a blocking way would even be an option in this situation.

I made something similar for a maze generator. But I divided the surface into 32x32 tile chunks. Exactly matching the factorio chunks. And then I put the generation of chunks into the on_chunk_generated event. My maze is infinite (to the limits of the game) so I had to use an on-demand approach. Maybe the same works for you.

I tried that, but I felt like the UPS drops were too large. I might try it again though, as it would make things a lot easier.

[mrvn]

I made something similar for a maze generator. But I divided the surface into 32x32 tile chunks. Exactly matching the factorio chunks. And then I put the generation of chunks into the on_chunk_generated event. My maze is infinite (to the limits of the game) so I had to use an on-demand approach. Maybe the same works for you.

I tried that, but I felt like the UPS drops were too large. I might try it again though, as it would make things a lot easier.

Oh yeah, the UPS drops when you start a new surface and it generates like 25 chunks in seconds and 100+ over the next minutes. Generating chunks simply takes time. So if you can generate the surface ahead of time when the player places a new portal it will be smoother than hving the game generate it in a panic when the player is teleported. The two can work together though.

[Danacus]

I made something similar for a maze generator. But I divided the surface into 32x32 tile chunks. Exactly matching the factorio chunks. And then I put the generation of chunks into the on_chunk_generated event. My maze is infinite (to the limits of the game) so I had to use an on-demand approach. Maybe the same works for you.

I tried that, but I felt like the UPS drops were too large. I might try it again though, as it would make things a lot easier.

Oh yeah, the UPS drops when you start a new surface and it generates like 25 chunks in seconds and 100+ over the next minutes. Generating chunks simply takes time. So if you can generate the surface ahead of time when the player places a new portal it will be smoother than hving the game generate it in a panic when the player is teleported. The two can work together though.

Combining the two might be the best solution, it's a good balance between responsiveness and performance. Thanks for your ideas!

[mrvn]

I look forward to testing it. Give a shout when you have something.

[Hermitude]

If spawning and creating the surface takes quite a few operations, do you have the option of making the operation less "instant"?

For example, can you spawn a portal in a cosmetic "constructing" phase, then later a few ticks later (maybe 15) change its sprite to indicate to the user that it is fully active?

[mrudat]

Perhaps a 'charging' animation as the teleporter winds up, giving you time to do stuff. You'd need to check the player is still present when the charging phase is done, of course.

[Honktown]

Are all the teleporters attached to the same surface on the other side? There's no reason a generated surface has to be "bound" to a teleporter until someone goes through it - if at most, people are going through 2-3 new teleporters in a short time, you can have a pool of available surfaces generated, and just grab the first in the pool, which would become the permanent surface to which that teleporter goes. Even with 100 teleporters placed, if people can only go through a few at a time, only a few surfaces need to be ready every so many ticks.

If it's a matching surface like a cave and the mapgen is expensive... then you have to make some compromises. Anything that can be off-loaded to the noise expression system would be a lot faster than doing it lua-side, but using the noise expressions can be like herding cats.