Page 1 of 1
[boskid][1.1.0] Belt performance regression
Posted: Mon Nov 23, 2020 10:38 pm
by zer0t
1. What did you do?
- Loaded my 1.0.0 save into 1.1.0
2. What happened?
- FPS/UPS dropped from 17 (at 1.0.0) to 3 (at 1.1.0)
- I let it run for about 10 minutes, but nothing changed
3. What did you expect to happen instead? It might be obvious to you, but do it anyway!
- at least equal performance like in 1.0.0
- maybe even better performance than in 1.0.0 (according to changelog there were some belt multithread improvements and this save has a lot of belts)
Save file / logfiles:
https://nx7119.your-storageshare.de/s/KC6CWJfgK6wyqTS
Mods: None
Screen 1.0.0:
- 1.0.0.png (3.84 MiB) Viewed 10468 times
Screen 1.1.0:
- 1.1.0.png (4.53 MiB) Viewed 10468 times
Offtopic: Amazing game, amazing devs, pls keep on going
edit: updated link
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 1:08 am
by boskid
Thanks for the report. Issue is now fixed for the next release.
Due to multithreaded update, transport lines have to be grouped. In your save file there is at least one huge group that contains 200k+ transport lines and any changes around it (adding or removing belts, splitter, rotating them etc..) will be causing lag spikes when that group is checked for possible split. Issue was that those groups were also being checked periodically in some other unnecesary cases. This save file will not benefit from multithreaded update because of that one huge group. To remove all the remaining lag spikes (when interacting with belts) i would have to disable the whole groupping logic and i will not sacrifice that just because of one extremally unfortunate save file.
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 2:21 am
by orzelek
What are the criteria for grouping?
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 2:28 am
by boskid
Transport lines are in the same group if they are connected in any way. Easiest way to explain that is if you have 2 points on belts, if an item can go from first to the second only using belts then those points are in the same group. Special exception is made for splitters as they are connecting all 8 transport lines (4 input and 4 output) into single group.
Sometimes by interacting with belts it may happen that group contains not connected components (as in the graph) and the group can be split, but checking for possible split takes some tiny amount of time.
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 2:32 am
by orzelek
Thanks.
So for best performance we should not build huge interconnected webs of belts. I guess with main bus and train deliveris it won't get so bad and any entity like loader will also break the lines.
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 2:36 am
by Jap2.0
Interesting, if done correctly this could be a good reason to avoid using balancers.
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 2:42 am
by boskid
Yes, only belts (+undergrounds) and splitters are grouping, inserters and loaders are not forcing anything to group (loader has 2 transport lines but they connect only from 1 side). It may even happen that both transport lines going through a single belt may belong to 2 separate groups. I would suggest not worrying too much about it as almost every save file i was testing had enough groups to make them benefit from threaded update. Save file in this topic is an exception because there is almost half of the map covered with belts that are interconnected using splitters for filtering and balancing purposes.
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 9:44 am
by zer0t
Kinda sad news for my save, but as long as this save is only one unique specific edge case I completely understand your decision. Guess I will have to stick to 1.0.0 or change my belts.
Thanks for the fast response!
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 7:10 pm
by Illiou
boskid wrote: ↑Tue Nov 24, 2020 1:08 am
Thanks for the report. Issue is now fixed for the next release.
Due to multithreaded update, transport lines have to be grouped. In your save file there is at least one huge group that contains 200k+ transport lines and any changes around it (adding or removing belts, splitter, rotating them etc..) will be causing lag spikes when that group is checked for possible split. Issue was that those groups were also being checked periodically in some other unnecesary cases. This save file will not benefit from multithreaded update because of that one huge group. To remove all the remaining lag spikes (when interacting with belts) i would have to disable the whole groupping logic and i will not sacrifice that just because of one extremally unfortunate save file.
I mean, even though this save might be an extreme case, having one main bus with everything connected via splitters is more of a default rather than an edge case... So technically any such (common) save would have this problem as the scale grows.
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 7:15 pm
by orzelek
Illiou wrote: ↑Tue Nov 24, 2020 7:10 pm
boskid wrote: ↑Tue Nov 24, 2020 1:08 am
Thanks for the report. Issue is now fixed for the next release.
Due to multithreaded update, transport lines have to be grouped. In your save file there is at least one huge group that contains 200k+ transport lines and any changes around it (adding or removing belts, splitter, rotating them etc..) will be causing lag spikes when that group is checked for possible split. Issue was that those groups were also being checked periodically in some other unnecesary cases. This save file will not benefit from multithreaded update because of that one huge group. To remove all the remaining lag spikes (when interacting with belts) i would have to disable the whole groupping logic and i will not sacrifice that just because of one extremally unfortunate save file.
I mean, even though this save might be an extreme case, having one main bus with everything connected via splitters is more of a default rather than an edge case... So technically any such (common) save would have this problem as the scale grows.
I guess it should be able to "break up" groups on one belt. I guess that leads to other.. annoying issues since that one belt could get updated out of order and lead to strange behavior.
Tbh it's hard for me to imagine buss that has one interconnected setup with 200k+ transport lines. Have to load the save and check it out
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 7:23 pm
by Rseding91
orzelek wrote: ↑Tue Nov 24, 2020 7:15 pm
Tbh it's hard for me to imagine buss that has one interconnected setup with 200k+ transport lines. Have to load the save and check it out
Imagine a giant pyramid of splitters all merging into one place then splitting off again. that's what it is
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 7:25 pm
by orzelek
Rseding91 wrote: ↑Tue Nov 24, 2020 7:23 pm
orzelek wrote: ↑Tue Nov 24, 2020 7:15 pm
Tbh it's hard for me to imagine buss that has one interconnected setup with 200k+ transport lines. Have to load the save and check it out
Imagine a giant pyramid of splitters all merging into one place then splitting off again. that's what it is
Save file was removed so I'll need to only imagine that one
It's a developers curiosity.. something like "oh carp how did they do that we had no idea it could be used like this".
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 7:55 pm
by robot256
Illiou wrote: ↑Tue Nov 24, 2020 7:10 pm
I mean, even though this save might be an extreme case, having one main bus with everything connected via splitters is more of a default rather than an edge case... So technically any such (common) save would have this problem as the scale grows.
All I see is a new mechanic for the UPS optimizers to exploit! Or it will make miniloaders mandatory so you can make "bus buffers" with a line of loaders into chests and back out again.
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 7:58 pm
by zer0t
Argh sry thought there is no need anymore for the link. new link:
https://nx7119.your-storageshare.de/s/KC6CWJfgK6wyqTS
Rseding91 wrote: ↑Tue Nov 24, 2020 7:23 pm
Imagine a giant pyramid of splitters all merging into one place then splitting off again. that's what it is
you forgot the endless numbers of intersections
Basically I'm distributing all "base" products in a giant matrix and at the end I store anything not need right now in chests.
Re: [boskid][1.1.0] Belt performance regression
Posted: Tue Nov 24, 2020 9:45 pm
by AartBluestoke
I would suggest use miniloaders: miniloader->chest->miniloader should break the belt chain into segments (eg:
https://mods.factorio.com/mod/miniloader) you would just need to put an 'inline buffer' for all belts in a parallel bus to cut the segment at that point.
Re: [boskid][1.1.0] Belt performance regression
Posted: Wed Nov 25, 2020 11:11 am
by eradicator
Interesting. Sounds overall like loader+wide-chest+loader balancers are now vastly better for performance, and huge spaghetti[Edit: sushi] spirals start lag spikes on every interaction?
Re: [boskid][1.1.0] Belt performance regression
Posted: Wed Nov 25, 2020 7:45 pm
by Jap2.0
eradicator wrote: ↑Wed Nov 25, 2020 11:11 am
huge spaghetti spirals start lag spikes on every interaction?
I don't know, sounds more like a problem for lasagna people. I like my spaghetti in separate strands.
Re: [boskid][1.1.0] Belt performance regression
Posted: Wed Nov 25, 2020 8:22 pm
by TheRaph
boskid wrote: ↑Tue Nov 24, 2020 1:08 am
To remove all the remaining lag spikes (when interacting with belts) i would have to disable the whole groupping logic and i will not sacrifice that just because of one extremally unfortunate save file.
Would it be possible to make it an option in "settings"?
So one who have such corner-case base could disable belt-multi-shredding so no check for grouping is necessary.
I think in Factorio should be a good game for everyone ... independent of his/her playstyle.
Re: [boskid][1.1.0] Belt performance regression
Posted: Wed Nov 25, 2020 8:36 pm
by boskid
TheRaph wrote: ↑Wed Nov 25, 2020 8:22 pm
Would it be possible to make it an option in "settings"?
No. It would have to be an option that would be part of the game state and shared between all the players and i am not going to duplicate all the tests only to check belts when this logic is turned on and off. If you would have a fluid system with 200k fluid boxes then removing pipe may also lag because it is also doing checks for possible fluid system split. There are no options for fluid and noone is complaining. Only part that i could do is find more cases that are obvious to not cause any group splits, like when placing new belts (for which currently there is a check for possible split).
Re: [boskid][1.1.0] Belt performance regression
Posted: Thu Nov 26, 2020 8:31 am
by eradicator
Jap2.0 wrote: ↑Wed Nov 25, 2020 7:45 pm
eradicator wrote: ↑Wed Nov 25, 2020 11:11 am
huge spaghetti spirals start lag spikes on every interaction?
I don't know, sounds more like a problem for lasagna people. I like my spaghetti in separate strands.
Eh, i meant to say sushi, not sphagetti. Like
this where everything is one humongous spiral.