[0.15.12] [Oxyd] LuaEntity units randomly removed

[folk]

I'm currently rewriting https://mods.factorio.com/mods/kyranzor/robotarmy just for my own benefit, and the whole LuaEntity (unit) and LuaUnitGroup system is shockingly hard to work with.

Once I create a group of 20 units with hitboxes no bigger than small biters and send them off to do anything, a whole host of seemingly random (though entirely predictable in the sense that they happen every time) things can happen:

1. Units, with or without being in a unitgroup, simply randomly vanish from the game if they are bumping into either eachother, or trees, or anything else that blocks their path - not all the time, but with a unit group of 20, it happens every play test session
2. LuaUnitGroups are sometimes just destroy()'d if for example (this seems to be the most common thing) units in the group are "too far" apart - and I can't quantify "too far", but certainly a maximum zoomed screen-width away seems to be enough
3. Units that are given compound commands, either with or without a luaunitgroup, will in all cases I have observed simply stop executing commands after 5-6 of them.
4. LuaEntity/UnitGroup.has_command isn't very useful, because there's no way to remove commands. Can you make .set_command() simply accept nil to make an entity stop immediately what it's doing?

These problems occur in every single session, every time I playtest something.

#1 and #2 would not be so bad if the engine told me in some way that it had happened.

I mean, for LuaEntity I can track on_entity_died and check on_tick if my units are no longer valid. And if one is invalid, and has not been catched by on_entity_died, it was probably randomly vanished by the game - but I don't know in which edge cases this way of tracking it will NOT work. And it also seems like it shouldn't be necessary.

#1 would not be so bad if the pathing engine didn't give all units the exact same path to get to the destination. In all cases when I send a group (with or without a luaunitgroup) of units going, they immediately go into "duckling mode" and path directly behind eachother, and this is causing most of the problem simply because they crash into eachother wanting to get to the "line" from where they are wandering.
I just think of ducklings and "Walk the Line" when I watch this behavior every single play test.

To work around this, I've started giving each unit their own random destination positions within a radius of where I want them to end up as a group. But obviously this isn't doable if I use a LuaUnitGroup, and also it adds a lot of complexity, and the further away you are from the destination the less chance that it will do any good, because the given path calculated by the engine smooths out into the same path the further away you are.

I can give you my rewrite of robotarmy if you want to check out this stuff for yourselves, but frankly setting up the test scenario would be nontrivial - I presume it'll be easier and faster for you to spawn groups of biters in game and test this some other way.

[folk]

screenshot 1, duckling mode

duckling.jpg (179.13 KiB) Viewed 2602 times

screenshot 2, tangled

tangle.jpg (142.72 KiB) Viewed 2602 times

[folk]

I presume that the game is vanishing units to prevent infinite loops, that it has a while/for loop with a counter and after X iterations it simply says "we better remove this unit or we will loop forever".

If it is so, the first thing I'd try on the engine side (because it would be less work than redoing the whole pathing engine to account for units and real path interpolation) would be a small teleport-jerk in a random direction after a few ticks, for example, and then try again and if it continues looping then all you need to do is give us some way of detecting the fact that a unit has been removed due to invalid pathing; an event or LuaEntity property, or something.

Then we can spawn in a new one some distance away to counteract the engine.

I can do this already - like I mentioned above - by detecting invalid units that haven't had a on_entity_died event, but it's difficult for me to know which edgecases this wont catch.
And, of course, it's not going to look good for the player, but it's better than losing expensive units.

[Oxyd]

You've said many things but gave me nothing to actually work with. I'm going to need a way to reproduce the problems you're talking about – preferrably a save where the problem is happening, but a list of commands is also fine. If I need your mod to reproduce it, then you'll have to post the mod as well.

Units, with or without being in a unitgroup, simply randomly vanish from the game if they are bumping into either eachother, or trees, or anything else that blocks their path - not all the time, but with a unit group of 20, it happens every play test session

This may or may not be a problem. Units are indeed sometimes just removed when they get completely stuck, but that shouldn't happen simply because of trees or other units. Biters bump into everything all the time and they don't vanish like that (much). So I need a way to reproduce it.

LuaUnitGroups are sometimes just destroy()'d if for example (this seems to be the most common thing) units in the group are "too far" apart - and I can't quantify "too far", but certainly a maximum zoomed screen-width away seems to be enough

Yes, unit groups drop members when they get stuck, so that the whole group doesn't get stuck because of a few guys. I suppose I can make this toggleable via the Lua API. Although it's not really a bug but a modding interface request.

Units that are given compound commands, either with or without a luaunitgroup, will in all cases I have observed simply stop executing commands after 5-6 of them.

Never seen that. Give me a way to reproduce it.

LuaEntity/UnitGroup.has_command isn't very useful, because there's no way to remove commands. Can you make .set_command() simply accept nil to make an entity stop immediately what it's doing?

Yes, we can, but it's a modding interface request, not a bug report.

These problems occur in every single session, every time I playtest something.

Great. So just make a save when you see it happening and post it here.

#1 and #2 would not be so bad if the engine told me in some way that it had happened.

I suppose I could add another event type when that happens. Not a bug report, though.

#1 would not be so bad if the pathing engine didn't give all units the exact same path to get to the destination. In all cases when I send a group (with or without a luaunitgroup) of units going, they immediately go into "duckling mode" and path directly behind eachother, and this is causing most of the problem simply because they crash into eachother wanting to get to the "line" from where they are wandering.
I just think of ducklings and "Walk the Line" when I watch this behavior every single play test.

That's the point of unit groups. Instead of wasting CPU by finding a different path for each unit, you find a single path for the entire group. All members of the group also try to keep a certain offset from the real path when following it – this is to prevent them from bumping into each other all the time. This is the intended behaviour and not a bug.

To work around this, I've started giving each unit their own random destination positions within a radius of where I want them to end up as a group. But obviously this isn't doable if I use a LuaUnitGroup, and also it adds a lot of complexity, and the further away you are from the destination the less chance that it will do any good, because the given path calculated by the engine smooths out into the same path the further away you are.

That's only going to make it worse. First, you have to wait extra time for all the paths to be found. Second, when the units are individuals, they try to follow their paths exactly, instead of keeping an offset from it. This mean they're all going to try to be in the same place at the same time. That's only going to cause issues.

[folk]

Thank you, I'm going to make a reproducable test save as soon as I can.

I realise that most of the things I mentioned are modding interface requests. The reason I made one big post is that I wanted a more detailed conversation on the whole topic.

And I figured whoever ended up replying might give me some pointers as to how I should be doing things to avoid the vanishing issue, so I didn't want to presume the need for a modding interface change when I might be entirely at fault for misusing the existing API.

Also, I've gone through several iterations on trying to get the groups of luaentities to work, with and without unitgroups, and some of the problems - specifically with vanishing LuaUnitGroups - do not occur in the current state of the code. But from your answer that seems to be intended behavior and we should simply file a separate modding interface request to toggle that behavior that anyway. So when I get around to making a test case, I'll focus on the vanishing units.

It's just bad timing, because I'm leaving for my hometown tomorrow because my father is being admitted for an operation. I'll have my laptop with me and I'll see what I can do, but I haven't even tried running factorio on it yet. And I don't have a timeframe for my fathers recovery (doctors say anything from 3 days to 3 months, which seems crazy), so if coding on my laptop ends up being more irritating than productive you might not have a response for a long while.

[Oxyd]

Okay, we're not going anywhere, so just take your time.