[boskid][1.1.0] Belt performance regression

[zer0t]

1. What did you do?
- Loaded my 1.0.0 save into 1.1.0

2. What happened?
- FPS/UPS dropped from 17 (at 1.0.0) to 3 (at 1.1.0)
- I let it run for about 10 minutes, but nothing changed

3. What did you expect to happen instead? It might be obvious to you, but do it anyway!
- at least equal performance like in 1.0.0
- maybe even better performance than in 1.0.0 (according to changelog there were some belt multithread improvements and this save has a lot of belts)

Save file / logfiles: https://nx7119.your-storageshare.de/s/KC6CWJfgK6wyqTS
Mods: None
Screen 1.0.0:

1.0.0.png (3.84 MiB) Viewed 11143 times

Screen 1.1.0:

1.1.0.png (4.53 MiB) Viewed 11143 times

Offtopic: Amazing game, amazing devs, pls keep on going

edit: updated link

[boskid]

Thanks for the report. Issue is now fixed for the next release.

Due to multithreaded update, transport lines have to be grouped. In your save file there is at least one huge group that contains 200k+ transport lines and any changes around it (adding or removing belts, splitter, rotating them etc..) will be causing lag spikes when that group is checked for possible split. Issue was that those groups were also being checked periodically in some other unnecesary cases. This save file will not benefit from multithreaded update because of that one huge group. To remove all the remaining lag spikes (when interacting with belts) i would have to disable the whole groupping logic and i will not sacrifice that just because of one extremally unfortunate save file.

[orzelek]

What are the criteria for grouping?

[boskid]

Transport lines are in the same group if they are connected in any way. Easiest way to explain that is if you have 2 points on belts, if an item can go from first to the second only using belts then those points are in the same group. Special exception is made for splitters as they are connecting all 8 transport lines (4 input and 4 output) into single group.

Sometimes by interacting with belts it may happen that group contains not connected components (as in the graph) and the group can be split, but checking for possible split takes some tiny amount of time.

[orzelek]

Thanks.
So for best performance we should not build huge interconnected webs of belts. I guess with main bus and train deliveris it won't get so bad and any entity like loader will also break the lines.

[Jap2.0]

Interesting, if done correctly this could be a good reason to avoid using balancers.

[boskid]

Yes, only belts (+undergrounds) and splitters are grouping, inserters and loaders are not forcing anything to group (loader has 2 transport lines but they connect only from 1 side). It may even happen that both transport lines going through a single belt may belong to 2 separate groups. I would suggest not worrying too much about it as almost every save file i was testing had enough groups to make them benefit from threaded update. Save file in this topic is an exception because there is almost half of the map covered with belts that are interconnected using splitters for filtering and balancing purposes.

[zer0t]

Kinda sad news for my save, but as long as this save is only one unique specific edge case I completely understand your decision. Guess I will have to stick to 1.0.0 or change my belts.
Thanks for the fast response!

[Illiou]

Thanks for the report. Issue is now fixed for the next release.

Due to multithreaded update, transport lines have to be grouped. In your save file there is at least one huge group that contains 200k+ transport lines and any changes around it (adding or removing belts, splitter, rotating them etc..) will be causing lag spikes when that group is checked for possible split. Issue was that those groups were also being checked periodically in some other unnecesary cases. This save file will not benefit from multithreaded update because of that one huge group. To remove all the remaining lag spikes (when interacting with belts) i would have to disable the whole groupping logic and i will not sacrifice that just because of one extremally unfortunate save file.

I mean, even though this save might be an extreme case, having one main bus with everything connected via splitters is more of a default rather than an edge case... So technically any such (common) save would have this problem as the scale grows.

[orzelek]

Thanks for the report. Issue is now fixed for the next release.

Due to multithreaded update, transport lines have to be grouped. In your save file there is at least one huge group that contains 200k+ transport lines and any changes around it (adding or removing belts, splitter, rotating them etc..) will be causing lag spikes when that group is checked for possible split. Issue was that those groups were also being checked periodically in some other unnecesary cases. This save file will not benefit from multithreaded update because of that one huge group. To remove all the remaining lag spikes (when interacting with belts) i would have to disable the whole groupping logic and i will not sacrifice that just because of one extremally unfortunate save file.

I mean, even though this save might be an extreme case, having one main bus with everything connected via splitters is more of a default rather than an edge case... So technically any such (common) save would have this problem as the scale grows.

I guess it should be able to "break up" groups on one belt. I guess that leads to other.. annoying issues since that one belt could get updated out of order and lead to strange behavior.
Tbh it's hard for me to imagine buss that has one interconnected setup with 200k+ transport lines. Have to load the save and check it out

[Rseding91]

Tbh it's hard for me to imagine buss that has one interconnected setup with 200k+ transport lines. Have to load the save and check it out

Imagine a giant pyramid of splitters all merging into one place then splitting off again. that's what it is

[orzelek]

Tbh it's hard for me to imagine buss that has one interconnected setup with 200k+ transport lines. Have to load the save and check it out

Imagine a giant pyramid of splitters all merging into one place then splitting off again. that's what it is

Save file was removed so I'll need to only imagine that one
It's a developers curiosity.. something like "oh carp how did they do that we had no idea it could be used like this".

[robot256]

I mean, even though this save might be an extreme case, having one main bus with everything connected via splitters is more of a default rather than an edge case... So technically any such (common) save would have this problem as the scale grows.

All I see is a new mechanic for the UPS optimizers to exploit! Or it will make miniloaders mandatory so you can make "bus buffers" with a line of loaders into chests and back out again.

[zer0t]

Argh sry thought there is no need anymore for the link. new link: https://nx7119.your-storageshare.de/s/KC6CWJfgK6wyqTS

Imagine a giant pyramid of splitters all merging into one place then splitting off again. that's what it is

you forgot the endless numbers of intersections
Basically I'm distributing all "base" products in a giant matrix and at the end I store anything not need right now in chests.

[AartBluestoke]

I would suggest use miniloaders: miniloader->chest->miniloader should break the belt chain into segments (eg:https://mods.factorio.com/mod/miniloader) you would just need to put an 'inline buffer' for all belts in a parallel bus to cut the segment at that point.

[eradicator]

Interesting. Sounds overall like loader+wide-chest+loader balancers are now vastly better for performance, and huge spaghetti[Edit: sushi] spirals start lag spikes on every interaction?

[Jap2.0]

huge spaghetti spirals start lag spikes on every interaction?

I don't know, sounds more like a problem for lasagna people. I like my spaghetti in separate strands.

[TheRaph]

To remove all the remaining lag spikes (when interacting with belts) i would have to disable the whole groupping logic and i will not sacrifice that just because of one extremally unfortunate save file.

Would it be possible to make it an option in "settings"?
So one who have such corner-case base could disable belt-multi-shredding so no check for grouping is necessary.

I think in Factorio should be a good game for everyone ... independent of his/her playstyle.

[boskid]

Would it be possible to make it an option in "settings"?

No. It would have to be an option that would be part of the game state and shared between all the players and i am not going to duplicate all the tests only to check belts when this logic is turned on and off. If you would have a fluid system with 200k fluid boxes then removing pipe may also lag because it is also doing checks for possible fluid system split. There are no options for fluid and noone is complaining. Only part that i could do is find more cases that are obvious to not cause any group splits, like when placing new belts (for which currently there is a check for possible split).

[eradicator]

huge spaghetti spirals start lag spikes on every interaction?

I don't know, sounds more like a problem for lasagna people. I like my spaghetti in separate strands.

Eh, i meant to say sushi, not sphagetti. Like this where everything is one humongous spiral.