I'm a trained/experienced programmer, but I prefer C and other compiled languages. I haven't dabbled with lua scripting outside of things done during game load (so, nothing using ontick()).
I have a mod I want to modify. Fundamentally it places down chest entities and then every second it scans those chests to determine whether it should move the contents somewhere (think factorissimo outside/inside chest transfers; similar to that). The problem is in how the mod's current ontick() handler works. It causes stuttering because it is doing too much work during an ontick() call (around 300ms of work). I have a few ideas of how to fix it, but fundamentally is it possible to have an ontick call go through a LUA list, update a FIFO queue, and then have a blocking FIFO queue somewhere else that can operate outside of the ontick() operation? Or is all runtime code for a mod always running in the same thread no matter what?
I fear it's the latter, but I'd love if there was some way that I could have the ontick() call only do the two things it needed to do (periodically scan and move contents) and have all the other work (picking a destination, scanning the destination status, etc) happen outside of ontick() to keep update times reasonable.
Ontick scripting that offloads non-critical work to non-ontick worker?
Re: Ontick scripting that offloads non-critical work to non-ontick worker?
There are a lot of issues that would have to be solved for this idea to be even considered to be implemented:
1/ how worker accesses to game state would be synchronised so they do not happen during entity update?
2/ how to guarantee such accesses would happen in the same game tick when they would happen in MP game?
3/ how worker lua state would be saved into save file? control.lua works as handling events and so between events there is no code running and state can be saved by doing dump of a global
4/ how to synchronize taking data out of worker back to the control.lua so it is deterministic in case of MP game.
Only sane way i can think of would be to have some kind of work-request where some data would be passed to be processed and a tick would be given at which result is to be received back, then worker would be able to only touch such data and then return new data that would be passed back to control.lua at at the tick it was requested. If no data would be available at that tick, code would become blocking until worker would finish. Not sure if such approach would be of any use for you since it would not be able to access any parts of game state and would not be able to communicate with other workers. As for saving, all pending work requests would have to block until they would be completed so the results could be stored within save file.
Practical way of doing long computation right now is to split such computation into smaller incremental pieces that can be computed along multiple ticks.
1/ how worker accesses to game state would be synchronised so they do not happen during entity update?
2/ how to guarantee such accesses would happen in the same game tick when they would happen in MP game?
3/ how worker lua state would be saved into save file? control.lua works as handling events and so between events there is no code running and state can be saved by doing dump of a global
4/ how to synchronize taking data out of worker back to the control.lua so it is deterministic in case of MP game.
Only sane way i can think of would be to have some kind of work-request where some data would be passed to be processed and a tick would be given at which result is to be received back, then worker would be able to only touch such data and then return new data that would be passed back to control.lua at at the tick it was requested. If no data would be available at that tick, code would become blocking until worker would finish. Not sure if such approach would be of any use for you since it would not be able to access any parts of game state and would not be able to communicate with other workers. As for saving, all pending work requests would have to block until they would be completed so the results could be stored within save file.
Practical way of doing long computation right now is to split such computation into smaller incremental pieces that can be computed along multiple ticks.
Re: Ontick scripting that offloads non-critical work to non-ontick worker?
Yeah...see, I'm used to the C-style of thinking which says "You can absolutely blow your own head off. Don't aim at your face!" I'd only ever update entity contents inside ontick() itself, because that's the only place synchronization is guaranteed, but you wouldn't be able to enforce that at a language level. I think your goals with LUA were to not require them to be quite so wary of...breaking things. Not that I expect this to get implemented (I was purely asking what was possible), but since the questions are interesting:
1. Access can be anywhere, it doesn't need to be synchronized. Updates must happen in ontick(). The hidden gotcha being that if you access outside of ontick, cache that value, then try to use it in ontick() bad things (tm) could happen. I'm ok with the onus being on me to know that, but I totally get that breaks what Wube is trying to do wrt having robust validation that a mod will (probably) work if the game loads to menu.
2. Yea, that's a huge PITA. I only develop/test my mods in single player because none of my friends want to play multiplayer with me.
Besides, I like the peace of working on my factory alone.
3. While I could store whatever state I needed to recover in global myself, you are right, there are *far* more problems than are apparent, and this is not really solvable in any realistic way. Solvable yes. Solvable with reasonable runtime? No.
4. eh, anything that solves 2 should also solve 4 (a step of joining a MP game is exchanging a globally agreed upon starting tick ID for all clients, then you could exchange 'on this tick, X happens'? That would have a significant performance penalty though, as you'd be checking 'if tick ==' constantly, and that'd get expensive fast!)
But yea, overall, your idea of a work-request setup is exactly what I had in mind. You are right though, there are more problems than I initially thought about. This is why I don't program for a living. I'm a good problem solver, but my imagination is rather lacking when it comes to scoping and problem hunting ahead of time.
1. Access can be anywhere, it doesn't need to be synchronized. Updates must happen in ontick(). The hidden gotcha being that if you access outside of ontick, cache that value, then try to use it in ontick() bad things (tm) could happen. I'm ok with the onus being on me to know that, but I totally get that breaks what Wube is trying to do wrt having robust validation that a mod will (probably) work if the game loads to menu.
2. Yea, that's a huge PITA. I only develop/test my mods in single player because none of my friends want to play multiplayer with me.
Besides, I like the peace of working on my factory alone.3. While I could store whatever state I needed to recover in global myself, you are right, there are *far* more problems than are apparent, and this is not really solvable in any realistic way. Solvable yes. Solvable with reasonable runtime? No.
4. eh, anything that solves 2 should also solve 4 (a step of joining a MP game is exchanging a globally agreed upon starting tick ID for all clients, then you could exchange 'on this tick, X happens'? That would have a significant performance penalty though, as you'd be checking 'if tick ==' constantly, and that'd get expensive fast!)
But yea, overall, your idea of a work-request setup is exactly what I had in mind. You are right though, there are more problems than I initially thought about. This is why I don't program for a living. I'm a good problem solver, but my imagination is rather lacking when it comes to scoping and problem hunting ahead of time.
Re: Ontick scripting that offloads non-critical work to non-ontick worker?
Just a small thing: you couldn't just "read from the game whenever" in a thread. The game is mutating all the time and that's just called "a race condition" and how you get all kinds of broken crashing.
If you want to get ahold of me I'm almost always on Discord.