[0.18.37] Biters still do not group before attack

[darklich14]

During artillery firing, Pathfinder goes to 10ms and I end up with sub 10 FPS and get dropped. When an arty upgrade happens, Pathfinder goes to 100 (!!!) ms. I get sub-1 FPS and get dropped. Here is a screenshot illustrating what is going on. This is just a manual assault but the same happens in auto as well. I've only targeted one nest and the game because unplayable for everyone on the server:

Screenshot from 2020-07-29 19-44-10.png (999.94 KiB) Viewed 5581 times

[Loewchen]

Post a link to the save please.

[darklich14]

Post a link to the save please.

https://gofile.io/d/gEz6zc

I was able to recreate this from a blank map. I used /cheat and /editor and /command to advance artillery to absurd levels (though this is just to recreate what we normally build in a week on AssemblyStorm's weekly reset. The goal there is to launch as many rockets as possible in a week and we kill the server by day 3 and spend the rest of the week UPS hunting (which mostly involves conquest to clear pollution cloud and build solar). When artillery upgrades, the exact same thing happens as in these test maps.

Pathfinder has completely lost it

Screenshot from 2020-07-30 11-46-15.png (1009.97 KiB) Viewed 5511 times

Strings of units (view from map)

Screenshot from 2020-07-30 11-54-34.png (148.36 KiB) Viewed 5511 times

Strings of units (with debug)

Screenshot from 2020-07-30 11-55-00.png (3.5 MiB) Viewed 5511 times

Strings of units in plain view

Screenshot from 2020-07-30 11-55-06.png (3.84 MiB) Viewed 5511 times

Pathfinder usage before `/c game.forces['enemy'].kill_all_units()`

Screenshot from 2020-07-30 11-55-40.png (19.47 KiB) Viewed 5511 times

Pathfinder usage after `/c game.forces['enemy'].kill_all_units()` (though it quickly rises from sub 1ms to multiple milliseconds in mere seconds)

Screenshot from 2020-07-30 11-56-03.png (20.94 KiB) Viewed 5511 times

Please fix this! I've been playing the game in this play-style for 2 years and 0.18.30something broke this and has made end-game literally impossible to play. There have always been lazy unit groups which is also a major problem and probably a contributing factor to this because you can see many of the path requests just go unserviced and stack and stack and stack until the simulation can't even run. Artillery upgrades are literal server suicide. The only fix is to kill all units, but the problem immediately returns! Even conservative use of artillery can cause one large nest to elicit this problem. I don't know what the answer is, but I'm not willing to spam unit kill commands or turn biters off just because it's end game. It should be the time at which there's the most bug killing of all!

[Oxyd]

Yeah, I was just fixing this. We were trying some changes, but now went back to the pre-.36 way, so it should work just as it did before.

[darklich14]

Yeah, I was just fixing this. We were trying some changes, but now went back to the pre-.36 way, so it should work just as it did before.

Thank you kindly! I assume you mean .39 will have it fixed? Because the save and analysis I did was in .38...

I've been quite depressed with the performance/behavior of the biters since then. Are you floating any ideas for how to keep biters moving rather than sitting idle waiting for a turn with the pathfinder (pre-.36 behavior)? I understand the computational challenge and balance between "making biters attack aggressively enough" and "making all biters attack at once."

Are there commands we can run to increase the pathfinder work queue size or the path cache size? I wonder what the tradeoffs between memory/cpu and churn would look like. Perhaps the pathfinder could afford to be more aggressive in order to increase challenge and work off more biters. This has been a constant problem for me for 2 years now (as described as my, and *many* others', playstyle above). Is there any way I could get more insight into how this works and/or how to change settings in this regard? It would be very cool if vanilla just needed some already-exposed parameter modification to square this up.

[traycer]

Thank you kindly! I assume you mean .39 will have it fixed? Because the save and analysis I did was in .38...

Yeah, looks like @Oxyd typed out his reply after .38 was already released. I can confirm that the behaviour remains in .38. I put up another comparison video showing the pathfinding differences between .35 and .38:

https://youtu.be/SRbpJmQvsMk

Also, did artillery automatic targeting change too? I just noticed that the firing pattern in my example has also changed, despite both starting at the same tick with the same game save. Artillery only cares about worms and nests, right? Not about where biters/spitters are?

[darklich14]

Thank you kindly! I assume you mean .39 will have it fixed? Because the save and analysis I did was in .38...

Yeah, looks like @Oxyd typed out his reply after .38 was already released. I can confirm that the behaviour remains in .38. I put up another comparison video showing the pathfinding differences between .35 and .38:

https://youtu.be/SRbpJmQvsMk

Also, did artillery automatic targeting change too? I just noticed that the firing pattern in my example has also changed, despite both starting at the same tick with the same game save. Artillery only cares about worms and nests, right? Not about where biters/spitters are?

Beautiful. I love your side-by-side. It's incredible!

[traycer]

Beautiful. I love your side-by-side. It's incredible!

Thankfully Factorio is a nice, compact game so it doesn't take long to switch from one version to another.

[Oxyd]

Yeah, I was just fixing this. We were trying some changes, but now went back to the pre-.36 way, so it should work just as it did before.

Thank you kindly! I assume you mean .39 will have it fixed? Because the save and analysis I did was in .38...

Yep, the fix will be in .39.

I've been quite depressed with the performance/behavior of the biters since then. Are you floating any ideas for how to keep biters moving rather than sitting idle waiting for a turn with the pathfinder (pre-.36 behavior)? I understand the computational challenge and balance between "making biters attack aggressively enough" and "making all biters attack at once."

Well, they'll form groups again, which means they only need to find their way to the group (which is usually not a problem), and then the entire group needs to find only a single path. This reduces the workload on the pathfinder a lot.

Which is how it was intended to be after .36 as well, but we played with maximum distraction distance – currently there's a hard limit on how far a unit in a group can see and be distracted by entities like turrets. So we wanted to remove that limit, but then biters started being distracted by artillery turrets and tried to go for them individually. The fix is to just put the hard limit back in.

Are there commands we can run to increase the pathfinder work queue size or the path cache size? I wonder what the tradeoffs between memory/cpu and churn would look like. Perhaps the pathfinder could afford to be more aggressive in order to increase challenge and work off more biters. This has been a constant problem for me for 2 years now (as described as my, and *many* others', playstyle above). Is there any way I could get more insight into how this works and/or how to change settings in this regard? It would be very cool if vanilla just needed some already-exposed parameter modification to square this up.

Code: Select all

/c game.map_settings.path_finder.max_work_done_per_tick = 8000

8000 is the default, so if you increase the number, the pathfinder will do more work each tick. There's also max_steps_worked_per_tick, but usually the limiting factor is the work limit. (And I probably should remove the step limit eventually.)

You can also turn on show-detailed-info in the F4 menu to see the current pathfinder usage.

[darklich14]

Oxyd, thank you for all the clarification and suggestions. For (my) future reference, here is the documentation for other pathfinder options: https://wiki.factorio.com/Prototype/Map ... ath_finder

You guys have a lot going on trying to get to 1.0 so thank you for taking the time to make sure that these regressions are fixed! As always, thank you for all you do. You guys are an example to the world of how to develop software.

[traycer]

Quick first impression with 0.18.40 (0.18.39 had a show-stopper bug in the Windows installer): looks like the attack rallying behaviour has been restored. My test scenario does not quite behave exactly the same as in 0.18.35 and earlier, but that may be due to other changes in the meantime. I have not tested the effect on UPS yet, but it is apparent that the grouping is back. Will look into it further.

[EhrN]

Hey !

Coming with my current game.
Vanilla - Deathworld - hudge resources.

Trying to recover some FPS by pushing biters out of my pollution cloud.

This scenario happend at least two times :
- Build some artillery spots and fire ON
- Some spots are easy to reach for biters, sometime artillery reach an other side of hudge water lake.
- If artillery reach other side of water, many biters failed to come.
- After a moment, no biters come at all. They regroup and don't move.
- After looking my FPS down (~20FPS) with attack and biters not coming, I save and quit.

Two times, when I launch my save an other day, HUDGE WAVES from all biters who wasn't come before.

GLAD to see them coming ))

You have change something with this update (0.18.43) ?
Or when biters are totally broken, we should save and launch again the game ?