Friday Facts #421 - Optimizations 2.0

[FactorioBot]

Here it is! (beep boop)

https://factorio.com/blog/post/fff-421

[SnowZyDe]

Omg

[pleegwat]

But what if a chunk is revealed by more than 255 radars?

[sorrydnoodle]

Good to see the quest for optimization continues, it has been one of the worries for me with the expansion, but as always my fears were unfounded.

Can't wait to do another 1000x research cost run.

[Arzorth]

you guys rock!

[Hares]

But what if a chunk is revealed by more than 255 radars?

Yeah, I had similar questions. We 100% need nod which would bump radars until the counter resets to 0.

[Siarhei]

It would be interesting to know, why don't process each planet in a separate thread/CPU core?

[Arzorth]

If a chunk has a counter greater than 1, it is put in an update bucket, and we loop through 1 bucket each tick.

Shouldn’t this be “ If a chunk has a counter greater than 0”?

[Terrahertz]

But what if a chunk is revealed by more than 255 radars?

I guess you could use the 255 as a special value, telling the engine to look into a dedicated table, as this should not occur all that often and is only needed for updates to the counter.

[aka13]

This is simply amazing, I love you guys! Factorio has always been and will remain forever in my heart for the most performant and extensive game there is.

[The Phoenixian]

With the lamps change I can see the Bad Apple and Doom people salivating already. I look forward to someone deciding to make an entire space platform just for image display.

EDIT: Let me correct myself. I look forward to someone outside the dev team, following Kovarex's example and deciding to make an entire space platform just for image display.

But what if a chunk is revealed by more than 255 radars?

Hmmm... I wonder how plausible that limit is. Roboports are bigger and harder to hit the limit, so if they can hit the limit then so can radars.
32x32 tile chunks, radar range 2 should mean chunks can only be seen by roboports in a 3x3 area, divided by Roboports size of 4x4... 576. Possible, but you'd have to really go crazy. (or deliberately try to hit it with radars)

The most plausible situation for this to come up by accident is if some absolute madlad makes it such that roughly every second tile in a 3x3 chunk area has a roboport. For some reason.

[Hares]

Speaking of multi-threaded updates, could it be made so each layer of computation uses it's own thread? One thread for belts, one for machines, one for electric network, and so on. Yes, this may introduce 1-tick lag/delay on some events, but this will make the game overall update time be almost equal to the largest of its components, not the sum of them, which is perfect when some aspects take ~equal amount of time (which is a normal use case).

[FasterJump]

What if the logistic robot has a curved path (chasing a mobile entity like the player or spidertron)?

[NaurEnForven]

Regarding the electric network.
My approach would have been to optimize based on groups, i.e. idle or full load.
For these two cases there is no need to iterate through the entities, instead we just track the total power consumption. Then you track changes instead.
All entities report when they no longer should be part of the stable group and subtract their previously stable consumption from the total.
The changes to the total get processed first, moving the entity back into a dynamic group, which is how all entities currently work.
After that all the entities in the dynamic group get iterated over and the the calculation is done as in the current version.

Depending on how many entities are stable, this could be worth it.
Unless I am overlooking something and this does not work at all.

[Terrahertz]

It's amazing that you managed to hit the point where multithreading would slow you down at least in some places, very well done folks

[pleegwat]

Regarding the electric network.
My approach would have been to optimize based on groups, i.e. idle or full load.
For these two cases there is no need to iterate through the entities, instead we just track the total power consumption. Then you track changes instead.
All entities report when they no longer should be part of the stable group and subtract their previously stable consumption from the total.
The changes to the total get processed first, moving the entity back into a dynamic group, which is how all entities currently work.
After that all the entities in the dynamic group get iterated over and the the calculation is done as in the current version.

Depending on how many entities are stable, this could be worth it.
Unless I am overlooking something and this does not work at all.

That might save an update loop too. As encountered in the electric update, memory bandwidth is a limited resource, and you want to limit how many threads need to fetch the same page.

[MEOWMI]

Those improvements are really quite big in aggregate. Like, imagine if you could get these kinds of improvements on your endgame gear in an MMO. That would be extreme!

[Brambor]

Loved this FFF!

Optimizing belt readers went through couple of iterations. One of them was trying to keep running total on transport lines however it failed.

I wonder why did it fail? Was it too complex to programm? Or was the additional complexity of checking all the inserters and other inputs and outputs making it slower than before?

[samcday]

For a couple of years now, I've fantasized about asking Factorio devs for NDA access to the engine source. My dream? To decouple the Factorio renderer from UPS, such that it could do interpolation/prediction of "easy(ish) to predict" linear movement of bots (and maybe also items on the belt) in the extra frames.

That is, I think a Factorio base running at 60UPS but animating a horde of bots whizzing over sushibelts running at 120fps would look positively *sublime*.

Seems to me that this latest optimization goes a long way towards such a glorious future!

[Dev-iL]

I always wondered, and may have raised it on IRC in the past:
1) whether utilizing GPUs for computations instead of (or in addition to) graphic rendering could speed things up. It's obviously no small effort to convert parts of the code to work on a GPU, but since the team now has experience with porting the game to different platforms, including a console, perhaps this is more doable.
2) Use linear algebra to update states - (sparse) matrix operations such as multiplications, additions, etc, utilizing BLAS and MAGMA. It would be conceptually simple to create state transition matrices to update the states on each tick. The real question is which parts of the game loop code can benefit from moving away from explicit sub-looping...