Friday Facts #322 - New Particle system

[FactorioBot]

Here it is! (beep boop)

https://factorio.com/blog/post/fff-322

[Gergely]

we will not be doing any more 0.17 releases

Okay then...

Guess we don't get the better fluid physics.

[Melerion]

Not sure if it has been previously discussed or not, but ...

Did you ever consider having certain API interfaces marked as deprecated when you replace them, with the "old" one being a wrapper to the new solution, and (optionally) maybe generate a one time notice on map load if that can be easily done.
That would not immediately break a number of mods but allow them to migrate in an orderly fashion.

[Klonan]

we will not be doing any more 0.17 releases

Okay then...

Guess we don't get the better fluid physics.

Not in 0.17 at least

[MrBuisson]

Guys, I loved this last update on particules. I was wondering what send my framerate from 10FPS all the way up to 50FPS. Now I know. My base has quite long walls with many, many flame throwers (40k unit) in death world setting. Bitter dies so often that the stats are smothed out
I tough first it was the fluid update, but it's great to get this one latter on top of the other.

well, I now I have to expand enough to reclaim my 40FPS voucher

[Linver]

Do u have an available estimation of 0.18 release and/or the category of mod API that will be changed?

[Yehn]

I like playing with (sometimes very large amounts of) biters, so I'm guessing this will help performance when under heavy siege, or doing heavy siege-ing, or pretty much time there's a lot of effects going off. Great to hear.

[Lubricus]

Isn't particles usually some sort of shaders that only lives in the graphics card? And thus use no CPU power nor RAM.

[Rseding91]

Not sure if it has been previously discussed or not, but ...

Did you ever consider having certain API interfaces marked as deprecated when you replace them, with the "old" one being a wrapper to the new solution, and (optionally) maybe generate a one time notice on map load if that can be easily done.
That would not immediately break a number of mods but allow them to migrate in an orderly fashion.

Deprecation doesn't work. In my 4.5~ years on Factorio not once have I seen any function or API marked as deprecated actually have any effect except to add more complication to the programmer that has to mark it as such.

The only thing that works is to just delete the old one and provide the new one at the same time. Otherwise everyone just keeps using the old one because "it works, why would I spend the time to switch over?"

[Rseding91]

Isn't particles usually some sort of shaders that only lives in the graphics card? And thus use no CPU power nor RAM.

Particles in Factorio can spawn other particles, other entities, smoke, do damage, and so on. They couldn't ever just live in render-only land. Also, you still have to spend CPU time and RAM to tell the GPU what to do so it's never possible to "use no CPU por nor RAM".

[NIronwolf]

So no more push on 0.17. Guess it's time to concentrate on finishing out this modded playthrough since everything has settled for a while.

Although, I've been anticipating the UI updates so that expectation is tempered now.

Great work on optimizations! Potatoes everywhere thank you!

[Engimage]

After all these years it is really nice to see you guys carefully shape your masterpiece!
Kudos to a master optimizer Rseding as always rocking this side of Factorio

[ledow]

1 x Global List/Array/whatever of pointers to all entities, each with a type inside it.
1 x List/Array/whatever for each "type" of entity that needs to be handled separately, stored as a set of pointers to each of those entities.

Even for a thousand particles, the memory overhead is pathetic - basically an "extra" list of ~1000 pointers to the same entities if you do it right.
When you want to iterate over particles only, you can do without having to go through every entity in the global list.
When you want to pathfind / whatever, you only iterate over the pathfind-affecting lists of entities.
When you want to draw everything, you iterate over the global list.

You're not storing entities twice (there may be a pointer to them in multiple lists, sure, but only one copy of the entity "structure/class"), your iterations are linear and don't have to interrogate every entity to see if it's the right type. And with just two helper functions to add / remove an entity which add it or remove it from all the appropriate lists that it should be in and stitches the lists back together, and which you call whenever you change/create/destroy an entity so they don't get out of sync.

I mean... this is pretty much what lists/arrays/hashes etc. and pointers were MADE for, isn't it?

[starholme]

Time to ramp up the biters in the crash site scenario!

[Omarflyjoemacky]

Loaded question, but .18 before Christmas? ... Super FFF this week.

[jcranmer]

1 x Global List/Array/whatever of pointers to all entities, each with a type inside it.
1 x List/Array/whatever for each "type" of entity that needs to be handled separately, stored as a set of pointers to each of those entities.

Even for a thousand particles, the memory overhead is pathetic - basically an "extra" list of ~1000 pointers to the same entities if you do it right.
When you want to iterate over particles only, you can do without having to go through every entity in the global list.
When you want to pathfind / whatever, you only iterate over the pathfind-affecting lists of entities.
When you want to draw everything, you iterate over the global list.

You're not storing entities twice (there may be a pointer to them in multiple lists, sure, but only one copy of the entity "structure/class"), your iterations are linear and don't have to interrogate every entity to see if it's the right type. And with just two helper functions to add / remove an entity which add it or remove it from all the appropriate lists that it should be in and stitches the lists back together, and which you call whenever you change/create/destroy an entity so they don't get out of sync.

I mean... this is pretty much what lists/arrays/hashes etc. and pointers were MADE for, isn't it?

Each pointer is 8 bytes. As you get to medium-sized factories, that means your lists are each going to be in the 100K-1MB range. You might then say "I've got 8+GB of RAM, who cares?", but the reality is that it's not the RAM that's precious, it's your caches. If your lists don't fit in the cache, you're now paying a miss to main memory to get the address of the location you need to retrieve from main memory, all the while twiddling your thumbs while you're waiting for main memory. When you have enormous memory requirements, performance optimization tends to become all about optimizing cache locality. The classic example here is the random insertion in a list: at what N does it become faster to use a linked list instead of an array list for randomly inserting elements into the middle?

[gGeorg]

Isn't particles usually some sort of shaders that only lives in the graphics card? And thus use no CPU power nor RAM.

Particles in Factorio can spawn other particles, other entities, smoke, do damage, and so on. They couldn't ever just live in render-only land. Also, you still have to spend CPU time and RAM to tell the GPU what to do so it's never possible to "use no CPU por nor RAM".

Interesting. Does it mean we can have a Factorio particles AND the classic particles which live in the render only land?
With these second one, the graphics dept. would go wild with all the sparkles and steam leaks.

[Impatient]

Big thanks to Klonan and Rseding for taking a look at the particle system with its edge performance case of flamethrowers defending against biters. This case was the reason why I had to stop playing multiplayer games regularily, when players started to use flamethrower turrets, due to the low end cpu in my pc. Looking forward to see the lowered performance impact.

Side question: Would it not be perfect to use sprites for the decorative bleed out effects? I have no in depth knowledge, but shouldn't take that kind of solution all the performance strain from the CPU and just require a bit more graphics memory?

[BattleFluffy]

Thankyou for this excellent optimization work guys :>

[Ohz]

If you lost count, this scene contains 15,689 entities

oh ok