[Rseding91] [0.18.44] Terrible performance of unsaturated furnace entities with many possible recipes.

[Sopel]

PyIndustry adds entities that can void fluids/gasses/items (ingredient is converted to 0 water). They are of type "furnace". There is a in the order of a few hundred possible recipes for each such entity. After changing the type of the voiding entity to "assembling-machine" the entity update stage takes 25x less time. This only happens on unsaturated furnace entities, i.e. such that they often show "no recipe" in the status. I believe the problem is rooted in the base game and only the amount of recipes in pymods allows it to manifest visibly.

The attached archive contains:
- furnace.png - the screenshot of the debug when the voiding entities are of type "furnace" (look at the entity update time)
- furnace.zip - the save in which the above screenshot was made
- furnace.log - the log from the session in which the above screenshot was made

- assembling-machine.pgn - the screenshot of the debug when the voiding entities are of type "assembling-machine" with the respective recipes set
- assembling-machine.zip - the save in which the above screenshot was made
- assembling-machine.log - the log from the session in which the above screenshot was made
- pyblock_tweaks_1.0.0.zip - the mod that changes the type of voiding entities to "assembling-machine". Required for loading assembling-machine.zip. All other mods are on the mod portal.

[mrvn]

This has come up before with the conclusion that searching all those recipes simply takes time and with the assembling machine the recipe is chosen by the user avoiding this.

Couldn't the furnace cache (if it doesn't already)

a) what items/fluids it can accept when empty
b) the last recipe used so it can use it again if the same input happens again

In most cases a furnace will be dedicated to one recipe. Especialy with the fluid/gas voiding where fluid mixing isn't possible. So the last used recipe will be used again in nearly all cases.

[Sopel]

I believe it already caches the previous recipe. The problem is that when it's not saturated it looks for others (because there may exist a recipe that takes 100 water and voids it and also a recipe that takes 10 water and voids it. Imagine the first one being executed and leaving 20 water in. The second one would be chosen next). Now I'm not sure how this searching is done, but since there is only one recipe per given ingredient (in the voiding case, and in a pretty much any other reasonable case) I would expect that search to be O(1) with a reasonable data structure; or simply no search at all unless the ingredient changed (unless there are more than one recipe for that ingredient).

[mrvn]

I believe it already caches the previous recipe. The problem is that when it's not saturated it looks for others (because there may exist a recipe that takes 100 water and voids it and also a recipe that takes 10 water and voids it. Imagine the first one being executed and leaving 20 water in. The second one would be chosen next). Now I'm not sure how this searching is done, but since there is only one recipe per given ingredient (in the voiding case, and in a pretty much any other reasonable case) I would expect that search to be O(1) with a reasonable data structure; or simply no search at all unless the ingredient changed (unless there are more than one recipe for that ingredient).

Is that even possible to have such overlapping recipes? How does it pick the recipe to use then? The one listed first or the one with largest amounts or what?

And yes, the furnace should have a way to lookup recipes by ingredient, especially for single ingredient recipes. But I believe it simple searches through all.

[Rseding91]

Started looking at this and... after loading up the furnace save my only thoughts are; whoever made that sound for the electrolyzer needs to not ever do that again It's just super loud TV static... but somehow worse... terrible to listen to.

[Rseding91]

After running the profiler against it I found the issue. The slow part wasn't picking the recipe - it was checking if the module effects were valid for the recipe that got chosen. That check is O(N) on the number of recipes the module is configured to allow (linear scan to check if it exists in the array).

Which normally isn't a big deal (so is the selection of the recipe to use) - except there's 11262 recipes the module is configured to allow. Oh also; there are exactly 11262 recipes with all the mods together. So it was configured to allow all recipes and did that by adding all of them to the module instead of... setting the allowed list to nil which means 'all'.

I have a fix for the linear performance issue (changes it to constant time) but it won't be released until after the 1.0 release date and then an experimental release.

[Sopel]

Thanks for looking into it. That's something I have not expected. With that information I'll look into this again tomorrow, because I still think there is some noticable cost to such entities even if they are not moduled - I made the change on a pymods community map where no modules were used and the entity update time dropped by ~30%. I'll post an update here if I get some more data on this.

[mrvn]

Looking forward to testing this fix with angel, bobs and pyaddon mods.

[Sopel]

So I've looked at the performance without modules. I placed around 400 electrolyzers, each with 1200 voids at the outputs. Each void is saturated at about 50%, meaning it works for a second, then doesn't for a second, and it loops. That gives 600 transitions from ingredient shortage -> working per second, so on average 10 per tick. I created 3 settings:
1. Not using voiding entities. Gasses are removed by a tailings pond which is scripted (therefore the update time is moved from entity update to mod update).
2. 1200 voids as assembling-machines.
3. 1200 voids as furnaces.

On my PC (i7-920, 1066MHz DDR3 RAM) I get the following entity update times:
1. 0.550ms - 0.600ms
2. 0.950ms - 1.000ms
3. 1.650ms - 1.900ms

This indicates that there is an issue (albeit smaller) even when mo modules are being used. So it seems like furnace entities (with this particular set of recipes) are about 3 times as costly as assembling-machine entities in this usage scenario (which is relatively tame on working <-> not working switching frequency).

I attach the saves for each of the setting. I allowed myself to skip the screenshots and logs this time. If they are needed I can provide them at a later date.
note: this set of saves has a few more mods as dependencies but they shouldn't be heavy. I really didn't want to tinker with my mod setup too much again. The pymods_tweaks mod is the same as provided in the top post, but I'll attach it here too.

[Rseding91]

I'll look into it more at a future point but there will always be *some* extra cost as it has to dynamically pick recipes and detect all that logic. 3x more expensive doesn't seem that bad; considering how small of numbers we're talking about compared to the huge impact building 10% more inserters will have on the overall performance.

Also having that many of those 'destroy-anything' entities already is unrealistic. If that's what the mod is saying you should be doing; the mod needs to re-balance itself so you don't have to do that.

[mrvn]

I'll look into it more at a future point but there will always be *some* extra cost as it has to dynamically pick recipes and detect all that logic. 3x more expensive doesn't seem that bad; considering how small of numbers we're talking about compared to the huge impact building 10% more inserters will have on the overall performance.

Also having that many of those 'destroy-anything' entities already is unrealistic. If that's what the mod is saying you should be doing; the mod needs to re-balance itself so you don't have to do that.

Doesn't the same cost apply to any furnace? Try deadlocks stacking and beltboxes. Having 1200 beltboxes doesn't sound too far out there and there are probably as much stacking/unstacking recipes as fluid voiding.

[ptx0]

it would be cool if after n dynamic operations being the same to then cache the result for n ticks. then the furnace wouldn't need to work as hard when its inputs are not changing.

this would break sushi furnace belts but those are stupid anyway.

[Rseding91]

it would be cool if after n dynamic operations being the same to then cache the result for n ticks. then the furnace wouldn't need to work as hard when its inputs are not changing.

this would break sushi furnace belts but those are stupid anyway.

It already caches the last recipe. But really it's not a big deal; it's not an area to save a huge amount of performance on. Virtually any save where you start running into performance issues will because of belts, inserters, biters, robots, or trains. They are just what dominate the update time because of how many exist and are running in any given save.

[Sopel]

Seems to not measurably affect vanilla, so probably not high priority now before 1.0.

I set up furnaces to process iron equivalent to 12k SPM with a primed inserters to make the belts compressed and furnaces to not be saturated (it requires 6.8 per to compress a blue belt lane, I used 7). I benchmarked over 6000 ticks. The diffence is immesurable.

furnaces as furnaces:

Code: Select all

Performed 6000 updates in 43158.780 ms
avg: 7.193 ms, min: 4.199 ms, max: 26.423 ms
checksum: 1708472842
49.194 Goodbye

furnaces as assembling machines:

Code: Select all

Performed 6000 updates in 43132.459 ms
avg: 7.189 ms, min: 4.105 ms, max: 42.969 ms
checksum: 1336565620
49.519 Goodbye

I will note that it did affect a pyblock community map by ~20-30% in my limited testing, so it is visible in real situations with heavy modded saves.

[mrvn]

it would be cool if after n dynamic operations being the same to then cache the result for n ticks. then the furnace wouldn't need to work as hard when its inputs are not changing.

this would break sushi furnace belts but those are stupid anyway.

It already caches the last recipe. But really it's not a big deal; it's not an area to save a huge amount of performance on. Virtually any save where you start running into performance issues will because of belts, inserters, biters, robots, or trains. They are just what dominate the update time because of how many exist and are running in any given save.

Earlier someone mentioned having recipes taking 10 items and 100 items. The idea being that if you fill the furnace fast enough it would use the bigger recipe (which I assume would work around 60 recipes/s limit).

Does that even work with the caching? Or work only once and then the caching keeps the recipe stuck to what it picked the first time?

[ptx0]

Earlier someone mentioned having recipes taking 10 items and 100 items. The idea being that if you fill the furnace fast enough it would use the bigger recipe (which I assume would work around 60 recipes/s limit).

Does that even work with the caching? Or work only once and then the caching keeps the recipe stuck to what it picked the first time?

it doesn't really work that way at least not with Schall Scaled Recipes, where the recipes are completely overridden based on a given string pattern to match the entity name (i.e. furnace-*).

[Rseding91]

I never was able to reproduce any super slowdown with furnaces outside of the module effects that is fixed for the next release.

If anyone has any kind of furnace example where it's not artificial copy-paste I can take a look at it. In the meantime I consider this solved.