Friday Facts #421 - Optimizations 2.0

[Tertius]

Why do I find the optimisations so exciting? I don't think any of my bases have become noticeably slow. My mega bases were never really very mega.

What you consider mega base might not be the same for everyone. My last base is down to ~30 ups on a i5-13600k. I didn't design for ups, I just built. It would be extremely nice, if it's possible to build bigger and still being able to ignore any engine limitation.

[POPISowyNumer]

"as they only read belt contents and their output is not used by other belt readers"

This is foreshadowing

[gnutrino]

But what if a chunk is revealed by more than 255 radars?

Where did they say the value was a single byte?

[JoshuaJosephson]

What are the chances of getting higher FPS / decoupling the FPS from the UPS?

Something like DLSS3 with its frame generation would be incredibly useful for higher refresh rate gamers

[GregoriusT]

I hope that the Energy Net Code at least handles quantity of individual Networks a bit less laggy.

I once had a few hundred networks going on when removing Solar Fields from my Base, and it somehow lagged a LOT more than when those power poles were connected. They did not even have any Batteries or Solar Panels in range, it was just the 2x2 power poles which formed hundreds of squares of wires going nowhere.

Now obviously this was an inbetween state of things, and happened while i was deconstructing a shitload of stuff, but it was very noticeable.

As for how those wire squares happened specifically? I removed Substations but not the Power Poles, resulting in tons of wires being disconnected by accident. xD

[Kinson25]

These are my favorite. I love the optimization and getting a little bit into the weeds.

[Miroro]

In case I have all my base area covered by logistic network (roboports) I don't need radars anymore? How big a chunk is?

[Terrahertz]

In case I have all my base area covered by logistic network (roboports) I don't need radars anymore? How big a chunk is?

32x32 Tiles, you can press F5 ingame to see the chunks. If you want to align your rails with them for instance.

[Terrahertz]

Btw, Roboports revealing an area of 2 chunks means the port is revealing a bit more than its logistics area right?

How is this even calculated? Does this mean it reveals 4 or 9 chunks?

[Nidan]

I hope that the Energy Net Code at least handles quantity of individual Networks a bit less laggy.

I once had a few hundred networks going on when removing Solar Fields from my Base, and it somehow lagged a LOT more than when those power poles were connected. They did not even have any Batteries or Solar Panels in range, it was just the 2x2 power poles which formed hundreds of squares of wires going nowhere.

Now obviously this was an inbetween state of things, and happened while i was deconstructing a shitload of stuff, but it was very noticeable.

As for how those wire squares happened specifically? I removed Substations but not the Power Poles, resulting in tons of wires being disconnected by accident. xD

Looks like this will be fixed/improved in 2.0, take a look at the thread where this post comes from:

Looking at the provided save; I can't test them in 2.0 because the mods won't load there but the area taking the time has been changed to where manual testing shows it takes almost no time for empty electric poles.

In 1.1 the cost for an empty electric pole is O(Number-Of-Entity-Prototypes * 3)

In 2.0 the cost for an empty electric pole is O(Number-Of-Types-Connected-To-Pole * 3)

In the case of an empty pole that's 0 so it's effectively O(0 * 3) or O(0)

--------------------

Btw, Roboports revealing an area of 2 chunks means the port is revealing a bit more than its logistics area right?

How is this even calculated? Does this mean it reveals 4 or 9 chunks?

The current radar reveals 7x7 chunks, with the prototype specifying "max_distance_of_nearby_sector_revealed = 3". If roboports have this set to 2, they'll reveal a 5x5 chunk area; that's guaranteed to cover the construction area regardless of positioning within the chunk. If Rseding meant to include the center chunk in his range, that's 3x3 chunks revealed; guaranteed to cover the logistics range, but not enough for the construction area (92x92 tiles revealed vs 110x110 construction area).

[j16sdiz]

I always wondered, and may have raised it on IRC in the past:
1) whether utilizing GPUs for computations instead of (or in addition to) graphic rendering could speed things up. It's obviously no small effort to convert parts of the code to work on a GPU, but since the team now has experience with porting the game to different platforms, including a console, perhaps this is more doable.
2) Use linear algebra to update states - (sparse) matrix operations such as multiplications, additions, etc, utilizing BLAS and MAGMA. It would be conceptually simple to create state transition matrices to update the states on each tick. The real question is which parts of the game loop code can benefit from moving away from explicit sub-looping...

Different video card do math slightly differently...
Trust me, you won't want to debug desync between different video card / drive version combinations.

[yinyang117]

Bam!

Sound of jaw hitting floor. Amazing. This optimization journey is so much fun to read about and so inspiring!

[dragon_gawain]

Wow! Those look to be some amazing optimizations! Our computers will be very pleased with your efforts!

Also, about those robots.. in the (admittedly very rare) cases where you want to pick up a flying robot, if it's not synced with its visual, wouldn't that cause problems?

[DrakeyC]

I've only ever had like, a few hundred bots flying around, largely delivering fuel for trains. Another reminder that I'm not just thinking on the right scale for this game.

[crystalcallie]

Question about the performance improvement statistics given in this post: You say that the radar improvement had a 3.6% performance increase, the circuit logic improved things by 9.5%, and that the robot optimization improved things by 15%. Are each of those improvements separately calculated (so that by the end of the optimizations things were roughly ~25% faster overall), or is that a cumulative total (so that the 15% improvement is between 'before making any of these changes' and 'after making all of the changes')?

[Koub]

I love the FFFs in general, but amongst the various FFF subjects, my favourite is optimisation. I love to see how thinking things differently can lead to overall improvements, and also the failed attempts, where a seemingly obvious amelioration leads to worse performance because the bottleneck was not where we expected.

[BraveCaperCat]

a registration style system where anything that wants to reveal an area of the map simply registers the chunks as "keep revealed" by increasing a counter for that chunk in the map system. As long as the counter is larger than zero the chunk stays visible. Things can overlap as much as they want and it simply increases the counters.

Does this mean that I can make a mod adding a radar which adds -1 to this system? So, like a Radar Scrambler? Would be useful in PVP. Also pollution should only be visible for chunks in radar range. Thank you for your time, Developers and Community!

[usafphoenix]

Question about robot movement logic and the "faking it" approach with regards to combat / biter attacks.

The technical aspect of "faking smooth movement" can have a big performance improvement, and that's always great, but will this have unintended consequences with regard to robot survivability in terms of combat or wall defense/repair? If actual location is updated less often, will this increase the robot losses during combat because the robots are, in some respect....stationary?
(or does the teleportation every second make them actually harder for biters to target/predict/hit?)

[eternaleye]

I love the FFFs in general, but amongst the various FFF subjects, my favourite is optimisation. I love to see how thinking things differently can lead to overall improvements, and also the failed attempts, where a seemingly obvious amelioration leads to worse performance because the bottleneck was not where we expected.

Speaking of lateral thinking in optimizations, I'm curious as to both why electric network updates do have such a heavy memory bandwidth impact, and whether that impact could be avoided in the common case.

In thinking about how the electric network behaves, machines exist at a given priority, operate in one of two modes (source and sink), and function in one of two rate regimes (full or partial). Sources operate at partial rates when over-provisioned; sinks operate at partial rates when under-provisioned (brownout).

I can see why, when a machine is operating at partial rates, per-instance information will need computed based on the network info (as things like recipe, modules, beacons, etc. all impact power consumption and machine output, and the derived values need recomputed).

However, would it be possible to make the network capable of recognizing when all connected machines of a given (priority, mode) pair are operating at full rate, and if they were also operating at full rate in the previous tick, not recompute the electric network's impact on that entire class of machines? More generally, if the electric network is partial for that class with the same percentage as the previous tick, I think the same property (of not needing an update) holds, though I expect the performance benefit over the simpler "still full" logic to be minimal.

This would result in - in almost all cases - only updating one of the two cases per tick. In a base not experiencing brownouts (which is true the vast majority of the time), only the sources would be updated per tick, as well, which would simplify things further. Accumulator-heavy builds would, admittedly, act as a periodic brownout of the Tertiary priority, but specifically how accumulators work could allow (almost) all accumulators on the network to be updated with a single memory location as a further optimization.

Is this viable, or are there other factors necessitating the power network updating many memory locations even when machines are operating at full rate?

[Schmendrick]

Question about robot movement logic and the "faking it" approach with regards to combat / biter attacks.

The technical aspect of "faking smooth movement" can have a big performance improvement, and that's always great, but will this have unintended consequences with regard to robot survivability in terms of combat or wall defense/repair? If actual location is updated less often, will this increase the robot losses during combat because the robots are, in some respect....stationary?
(or does the teleportation every second make them actually harder for biters to target/predict/hit?)

I came here to bring this up. The optimization treats bots as though they have no relevance other than their function, but they're targetable entities with a "real physical" presence in the game. Maybe (and I hope) this has been handled and just wasn't addressed in the update, but anything that cares about positions (like an explosion or a turret, or anything like a beam that renders to a position) needs to be able detect when and where a robot is supposed to be.

The inconsistency is small (-ish, bots can end up pretty fast making 20 ticks a significant distance), but it's there and it's a break in simulation fidelity; it feels wrong for any situation in the game to have a different outcome from an identical one that just happens to occur on a different tick. I've taken shortcuts like that (using every 3rd tick) in modding but that's just modding; it's already kinda dirty.

In theory I guess you could have some kind of pseudo-sleep system where bots move into or out of a 1 or 20 tick update but I get the feeling that might end up messy enough to be self defeating, depending on the checks involved.


Edit:
Rereading the post, I hope this

Because of that, I just decided to define some hardcoded magic numbers of maximum 20 ticks of movement without update and 60 ticks of stationary robot without updates.

means that this is handled and I misinterpreted the system (and, presumably, the checks weren't that bad).