Friday Facts #421 - Optimizations 2.0

Regular reports on Factorio development.
User avatar
Dev-iL
Filter Inserter
Filter Inserter
Posts: 303
Joined: Thu Jul 02, 2015 2:48 pm
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by Dev-iL »

samcday wrote: ↑
Fri Jul 26, 2024 11:59 am
For a couple of years now, I've fantasized about asking Factorio devs for NDA access to the engine source.
Theoretically not impossible. People are known to have been given acces to the source. Typically, said people were well-known modders and/or those who wanted to work on features that were low priority for the devs, but they found important enough.
Leading Hebrew translator of Factorio.

Galista
Burner Inserter
Burner Inserter
Posts: 10
Joined: Mon May 17, 2021 7:52 am
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by Galista »

Rseding wrote:I settled on a registration style system where anything that wants to reveal an area of the map simply registers the chunks as "keep revealed" by increasing a counter for that chunk in the map system. As long as the counter is larger than zero the chunk stays visible. Things can overlap as much as they want and it simply increases the counters.
I may be missing something here, but why even bother to count reveal requests? Aren't boolean operations faster?
Last edited by Galista on Fri Jul 26, 2024 1:30 pm, edited 1 time in total.

samcday
Burner Inserter
Burner Inserter
Posts: 5
Joined: Thu Apr 13, 2023 6:39 am
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by samcday »

Dev-iL wrote: ↑
Fri Jul 26, 2024 12:10 pm
samcday wrote: ↑
Fri Jul 26, 2024 11:59 am
For a couple of years now, I've fantasized about asking Factorio devs for NDA access to the engine source.
Theoretically not impossible. People are known to have been given acces to the source. Typically, said people were well-known modders and/or those who wanted to work on features that were low priority for the devs, but they found important enough.
Aye, I'm aware that notable non-Wube folks have/had access. That it could actually happen is what makes my little fantasy such quality daydream material :D Similarly, I like to daydream about winning the lottery (also theoretically possible, though I've never bought a lottery ticket myself....) more than having superpowers (because Compound V sadly/thankfully does not exist in our particular local timeline).

samcday
Burner Inserter
Burner Inserter
Posts: 5
Joined: Thu Apr 13, 2023 6:39 am
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by samcday »

Galista wrote: ↑
Fri Jul 26, 2024 12:24 pm
Rseding wrote:I settled on a registration style system where anything that wants to reveal an area of the map simply registers the chunks as "keep revealed" by increasing a counter for that chunk in the map system. As long as the counter is larger than zero the chunk stays visible. Things can overlap as much as they want and it simply increases the counters.
I may be missing something here, but why even bother to count reveal requests? Aren't boolean operations faster?
It needs to be a counter because otherwise how do you know when the last registration was cleaned up? That is, a boolean would be "lossy" here - you would be throwing away important information. Another way of thinking about this, is that what was described in the blog post is very much like a "semaphore", and you're wondering why not use a "mutex" instead.

JKopter
Manual Inserter
Manual Inserter
Posts: 1
Joined: Sat Jun 22, 2024 5:01 am
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by JKopter »

FasterJump wrote: ↑
Fri Jul 26, 2024 11:41 am
What if the logistic robot has a curved path (chasing a mobile entity like the player or spidertron)?
I was also concerned about this. Also if the location for the robot is removed entirely - it would need to choose a different direction, possibly even exact opposite. I'm thinking it would glitch and jump around.

Perhaps swapping the order would make things smoother? So the 'real' robot moves, and *then* the 'ghost robot' moves from where it was, to the new real robot's position.

Ieuan
Manual Inserter
Manual Inserter
Posts: 1
Joined: Fri Oct 09, 2020 11:44 am
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by Ieuan »

Can't wait to try this out on my most recent (SE) save, my world has gotten rather big with lots of roboports and radars;
screenshots
I don't know if the optimizations to radars and robots would solve it 1:1, but the game is almost entirely unplayable now lol

mmmPI
Smart Inserter
Smart Inserter
Posts: 3151
Joined: Mon Jun 20, 2016 6:10 pm
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by mmmPI »

I think many FFF's are about optimization, but i can understand there's so much of it that it needs to have its own FFF, to point at for when someone ask again "what about multithreading the electric network".I think biters must have been optimized too, given the level of detail everything receive. But i don't think there is any mention of some, just a single base on map view that has disappeared after the rechart. I noticed while searching for some, that there is a nice interchange visible on the first picture, despite the numerous (low-quality?) robotport.

Very interesting to read the technical as always. Not as thrilling as new things ( like biters ) , but it's a way to learn things that we won't discover when playing the expansion as in a regular game and it gives little insight on how to be more efficient when building. I was expecting to read that lamps can transmit power to neighbouring lamps for a moment when seeing the pictures, but i think it's funny to have them work like this on the space platform only, as if it's a better place to send light signals to other planets/far away.

I'm not sure i understand the part about the control behavior apart from " (a benchmark time went down from 131s to just about 8.2s to complete). " how it also relate to lamps, but for both i'm curious to see how it will apply on editor maps with only combinators. I had forgotten the RGB for lamps were implemented for 2.0, i thought it was only mentionned as a possibility. It's a pleasant reminder !

The bot logic part was easier to understand for me, it's also pleasant to hear, none of those optimizations seems like trade-off on anything on the game itself. It's time spent by devs to allow me to build larger crappy design before my machine give up ? I say YES THANK YOU !

X3KJ
Burner Inserter
Burner Inserter
Posts: 14
Joined: Mon Feb 08, 2021 6:45 pm
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by X3KJ »

With the radar update - what happens, if one radar is out of power? Is the registering calculated every tick?

adam_bise
Filter Inserter
Filter Inserter
Posts: 405
Joined: Fri Jun 08, 2018 10:42 pm
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by adam_bise »

Just want to say thanks for all of the work that has gone into optimization over the years. It is things like this that really make Factorio stand out.

Galista
Burner Inserter
Burner Inserter
Posts: 10
Joined: Mon May 17, 2021 7:52 am
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by Galista »

samcday wrote: ↑
Fri Jul 26, 2024 12:36 pm
It needs to be a counter because otherwise how do you know when the last registration was cleaned up? That is, a boolean would be "lossy" here - you would be throwing away important information. Another way of thinking about this, is that what was described in the blog post is very much like a "semaphore", and you're wondering why not use a "mutex" instead.
Oh I see, I'm stupid! Thank you for the explanation! Time to get my coffee... :D

morse
Inserter
Inserter
Posts: 42
Joined: Fri Oct 16, 2015 9:35 am
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by morse »

Hm, this attempt at optimizing the power update make me think. So, if the memory read is a limiting factor, could it be beneficial to compactly store the relevant values separate from the main objects? Modern memory is good at reading large chunks, so iterating over an array should be much faster than dereferencing a list of objects.

Regardless of the outcome, I hope the code for multithreaded update will be kept in the game. Hardware evolves, and what is a limiting factor now might not be so in 10 years.

User avatar
cynicalwanderer
Burner Inserter
Burner Inserter
Posts: 5
Joined: Fri Jan 17, 2020 5:01 pm
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by cynicalwanderer »

morse wrote: ↑
Fri Jul 26, 2024 1:24 pm
Hm, this attempt at optimizing the power update make me think. So, if the memory read is a limiting factor, could it be beneficial to compactly store the relevant values separate from the main objects? Modern memory is good at reading large chunks, so iterating over an array should be much faster than dereferencing a list of objects.

Regardless of the outcome, I hope the code for multithreaded update will be kept in the game. Hardware evolves, and what is a limiting factor now might not be so in 10 years.
Yeah this is what I was thinking too. The cure for memory limited when iterating over an O(n) list is usually to try to pack the relevant data more tightly in contiguous memory so as to reduce cache misses. Kind of like what DOTS in Unity tries to do

morse
Inserter
Inserter
Posts: 42
Joined: Fri Oct 16, 2015 9:35 am
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by morse »

Siarhei wrote: ↑
Fri Jul 26, 2024 11:30 am
It would be interesting to know, why don't process each planet in a separate thread/CPU core?
Because it won't be enough. Modern CPUs have tens of cores, and pin just one of them to a whole planet is just wasteful.

keks244
Burner Inserter
Burner Inserter
Posts: 7
Joined: Thu Jan 07, 2021 1:39 pm
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by keks244 »

It would be great if lamps could be powered without the need for power poles on the planets too.

Perhaps you could add a special ground tile that can distribute the power between the entities.
A planet version of the space platform if you will.

Nice side effect:
You can use your space-platform blueprints even on planets, if you place the "energy tiles" underneath. :)
Last edited by keks244 on Fri Jul 26, 2024 1:56 pm, edited 1 time in total.

Tertius
Filter Inserter
Filter Inserter
Posts: 833
Joined: Fri Mar 19, 2021 5:58 pm
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by Tertius »

Optimizing the game this way is great! I really don't like it to always have in mind: "If I build it this way, what would be the ups impact?" instead of: "If I build it this way, is this nice enough for my technical/aesthetic senses?"

My biggest time consumer is always the inserters. They require 1/4 to 1/3 of my update time. Since they are one of the earliest entities built into Factorio, revisiting them might perhaps give some potential. For example the arm swing. Once the arm starts swinging, it's similar to the robots: the speed is known and doesn't change, so the inserter could sleep up to the tick where the arm reached its target location and wants to grab or release items. Displaying the swing is just something for the visuals.

User avatar
Silari
Filter Inserter
Filter Inserter
Posts: 506
Joined: Sat Jan 27, 2018 10:04 pm
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by Silari »

Siarhei wrote: ↑
Fri Jul 26, 2024 11:30 am
It would be interesting to know, why don't process each planet in a separate thread/CPU core?
Already been discussed multiple times in other threads. That only works if surfaces are truly independent, and they aren't. Linked belts/chests exist, and scripts can touch anything anywhere at any time, and connect things like power poles across surfaces IIRC, so you can't just cut surfaces apart.

Terrahertz wrote: ↑
Fri Jul 26, 2024 11:45 am
It's amazing that you managed to hit the point where multithreading would slow you down at least in some places, very well done folks :D
IIRC they already had done that. Specifically, heat pipe updates were attempted to be multithreaded but it turned out it was slower due to RAM cache misses, mentioned in an FFF a while ago. I think it was before the heat logic was changed though, to the current form parallelized with the fluid and electric networks. Not sure on that.

Agamemnon
Inserter
Inserter
Posts: 37
Joined: Fri Jun 29, 2018 9:48 pm
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by Agamemnon »

Ok first: Nice job!

Second: I wonder if we see some similar improvements for Biters. Their pathing and swarm mechanics are still something that has potential for improvement - most notably in the cases where they (for whatever reason) just accumulate over time and eat your UPS instead of your ammo.

samcday
Burner Inserter
Burner Inserter
Posts: 5
Joined: Thu Apr 13, 2023 6:39 am
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by samcday »

Agamemnon wrote: ↑
Fri Jul 26, 2024 2:22 pm
Ok first: Nice job!

Second: I wonder if we see some similar improvements for Biters. Their pathing and swarm mechanics are still something that has potential for improvement - most notably in the cases where they (for whatever reason) just accumulate over time and eat your UPS instead of your ammo.
Don't forget that the original Space Age announcement included that tantalizing sneak peek at some kind of new life form (and other than our engineer, those things exist purely for us to gleefully tear apart with laser energy, bullets, missiles, poison gas, nuclear warheads and who knows what else?). Since then we've heard exactly NOTHING about any form of organic life in any subsequent FFF, besides the non-sentient Fulgora flora ;)

So my prediction is our heroes at Wube are saving some of the best/juiciest updates for last, right before launch :)

idgod
Burner Inserter
Burner Inserter
Posts: 15
Joined: Tue Oct 04, 2016 11:12 am
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by idgod »

samcday wrote: ↑
Fri Jul 26, 2024 2:33 pm
So my prediction is our heroes at Wube are saving some of the best/juiciest updates for last, right before launch :)
Kovarex:
Image

kerrin
Burner Inserter
Burner Inserter
Posts: 6
Joined: Mon Feb 25, 2019 5:56 pm
Contact:

Re: Friday Facts #421 - Optimizations 2.0

Post by kerrin »

Why do I find the optimisations so exciting? I don't think any of my bases have become noticeably slow. My mega bases were never really very mega. :D

Post Reply

Return to β€œNews”