Friday Facts #322 - New Particle system

Regular reports on Factorio development.
User avatar
FactorioBot
Factorio Staff
Factorio Staff
Posts: 409
Joined: Tue May 12, 2015 1:48 pm

Friday Facts #322 - New Particle system

Post by FactorioBot »


User avatar
Gergely
Filter Inserter
Filter Inserter
Posts: 607
Joined: Sun Apr 10, 2016 8:31 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by Gergely »

Klonan wrote:we will not be doing any more 0.17 releases
:roll: Okay then...

Guess we don't get the better fluid physics.
Last edited by Gergely on Fri Nov 22, 2019 12:56 pm, edited 2 times in total.

Melerion
Burner Inserter
Burner Inserter
Posts: 14
Joined: Wed May 17, 2017 8:11 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by Melerion »

Not sure if it has been previously discussed or not, but ...

Did you ever consider having certain API interfaces marked as deprecated when you replace them, with the "old" one being a wrapper to the new solution, and (optionally) maybe generate a one time notice on map load if that can be easily done.
That would not immediately break a number of mods but allow them to migrate in an orderly fashion.

User avatar
Klonan
Factorio Staff
Factorio Staff
Posts: 5156
Joined: Sun Jan 11, 2015 2:09 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by Klonan »

Gergely wrote: ↑
Fri Nov 22, 2019 12:46 pm
Klonan wrote:we will not be doing any more 0.17 releases
:roll: Okay then...

Guess we don't get the better fluid physics.
Not in 0.17 at least

MrBuisson
Inserter
Inserter
Posts: 27
Joined: Sat Feb 04, 2017 9:08 am
Contact:

Re: Friday Facts #322 - New Particle system

Post by MrBuisson »

Guys, I loved this last update on particules. I was wondering what send my framerate from 10FPS all the way up to 50FPS. Now I know. My base has quite long walls with many, many flame throwers (40k unit) in death world setting. Bitter dies so often that the stats are smothed out :twisted:
I tough first it was the fluid update, but it's great to get this one latter on top of the other.

well, I now I have to expand enough to reclaim my 40FPS voucher
Last edited by MrBuisson on Fri Nov 22, 2019 1:00 pm, edited 1 time in total.

User avatar
Linver
Fast Inserter
Fast Inserter
Posts: 158
Joined: Wed Jan 09, 2019 2:28 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by Linver »

Do u have an available estimation of 0.18 release and/or the category of mod API that will be changed?

Yehn
Fast Inserter
Fast Inserter
Posts: 111
Joined: Tue Jul 12, 2016 3:45 am
Contact:

Re: Friday Facts #322 - New Particle system

Post by Yehn »

I like playing with (sometimes very large amounts of) biters, so I'm guessing this will help performance when under heavy siege, or doing heavy siege-ing, or pretty much time there's a lot of effects going off. Great to hear.

User avatar
Lubricus
Filter Inserter
Filter Inserter
Posts: 294
Joined: Sun Jun 04, 2017 12:13 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by Lubricus »

Isn't particles usually some sort of shaders that only lives in the graphics card? And thus use no CPU power nor RAM.

Rseding91
Factorio Staff
Factorio Staff
Posts: 13349
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Friday Facts #322 - New Particle system

Post by Rseding91 »

Melerion wrote: ↑
Fri Nov 22, 2019 12:52 pm
Not sure if it has been previously discussed or not, but ...

Did you ever consider having certain API interfaces marked as deprecated when you replace them, with the "old" one being a wrapper to the new solution, and (optionally) maybe generate a one time notice on map load if that can be easily done.
That would not immediately break a number of mods but allow them to migrate in an orderly fashion.
Deprecation doesn't work. In my 4.5~ years on Factorio not once have I seen any function or API marked as deprecated actually have any effect except to add more complication to the programmer that has to mark it as such.

The only thing that works is to just delete the old one and provide the new one at the same time. Otherwise everyone just keeps using the old one because "it works, why would I spend the time to switch over?"
If you want to get ahold of me I'm almost always on Discord.

Rseding91
Factorio Staff
Factorio Staff
Posts: 13349
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Friday Facts #322 - New Particle system

Post by Rseding91 »

Lubricus wrote: ↑
Fri Nov 22, 2019 1:03 pm
Isn't particles usually some sort of shaders that only lives in the graphics card? And thus use no CPU power nor RAM.
Particles in Factorio can spawn other particles, other entities, smoke, do damage, and so on. They couldn't ever just live in render-only land. Also, you still have to spend CPU time and RAM to tell the GPU what to do so it's never possible to "use no CPU por nor RAM".
If you want to get ahold of me I'm almost always on Discord.

User avatar
NIronwolf
Inserter
Inserter
Posts: 42
Joined: Sat Jul 07, 2018 6:44 am
Contact:

Re: Friday Facts #322 - New Particle system

Post by NIronwolf »

So no more push on 0.17. Guess it's time to concentrate on finishing out this modded playthrough since everything has settled for a while. :D

Although, I've been anticipating the UI updates so that expectation is tempered now.

Great work on optimizations! Potatoes everywhere thank you!

Engimage
Smart Inserter
Smart Inserter
Posts: 1069
Joined: Wed Jun 29, 2016 10:02 am
Contact:

Re: Friday Facts #322 - New Particle system

Post by Engimage »

After all these years it is really nice to see you guys carefully shape your masterpiece!
Kudos to a master optimizer Rseding as always rocking this side of Factorio

User avatar
ledow
Fast Inserter
Fast Inserter
Posts: 102
Joined: Sat Sep 24, 2016 3:00 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by ledow »

1 x Global List/Array/whatever of pointers to all entities, each with a type inside it.
1 x List/Array/whatever for each "type" of entity that needs to be handled separately, stored as a set of pointers to each of those entities.

Even for a thousand particles, the memory overhead is pathetic - basically an "extra" list of ~1000 pointers to the same entities if you do it right.
When you want to iterate over particles only, you can do without having to go through every entity in the global list.
When you want to pathfind / whatever, you only iterate over the pathfind-affecting lists of entities.
When you want to draw everything, you iterate over the global list.

You're not storing entities twice (there may be a pointer to them in multiple lists, sure, but only one copy of the entity "structure/class"), your iterations are linear and don't have to interrogate every entity to see if it's the right type. And with just two helper functions to add / remove an entity which add it or remove it from all the appropriate lists that it should be in and stitches the lists back together, and which you call whenever you change/create/destroy an entity so they don't get out of sync.

I mean... this is pretty much what lists/arrays/hashes etc. and pointers were MADE for, isn't it?

starholme
Fast Inserter
Fast Inserter
Posts: 201
Joined: Tue Oct 21, 2014 7:25 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by starholme »

Time to ramp up the biters in the crash site scenario!

Omarflyjoemacky
Fast Inserter
Fast Inserter
Posts: 104
Joined: Tue Nov 15, 2016 10:56 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by Omarflyjoemacky »

Loaded question, but .18 before Christmas? ... Super FFF this week.
"And then Bender ran."

jcranmer
Long Handed Inserter
Long Handed Inserter
Posts: 90
Joined: Wed Jun 29, 2016 9:59 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by jcranmer »

ledow wrote: ↑
Fri Nov 22, 2019 2:30 pm
1 x Global List/Array/whatever of pointers to all entities, each with a type inside it.
1 x List/Array/whatever for each "type" of entity that needs to be handled separately, stored as a set of pointers to each of those entities.

Even for a thousand particles, the memory overhead is pathetic - basically an "extra" list of ~1000 pointers to the same entities if you do it right.
When you want to iterate over particles only, you can do without having to go through every entity in the global list.
When you want to pathfind / whatever, you only iterate over the pathfind-affecting lists of entities.
When you want to draw everything, you iterate over the global list.

You're not storing entities twice (there may be a pointer to them in multiple lists, sure, but only one copy of the entity "structure/class"), your iterations are linear and don't have to interrogate every entity to see if it's the right type. And with just two helper functions to add / remove an entity which add it or remove it from all the appropriate lists that it should be in and stitches the lists back together, and which you call whenever you change/create/destroy an entity so they don't get out of sync.

I mean... this is pretty much what lists/arrays/hashes etc. and pointers were MADE for, isn't it?
Each pointer is 8 bytes. As you get to medium-sized factories, that means your lists are each going to be in the 100K-1MB range. You might then say "I've got 8+GB of RAM, who cares?", but the reality is that it's not the RAM that's precious, it's your caches. If your lists don't fit in the cache, you're now paying a miss to main memory to get the address of the location you need to retrieve from main memory, all the while twiddling your thumbs while you're waiting for main memory. When you have enormous memory requirements, performance optimization tends to become all about optimizing cache locality. The classic example here is the random insertion in a list: at what N does it become faster to use a linked list instead of an array list for randomly inserting elements into the middle?

gGeorg
Filter Inserter
Filter Inserter
Posts: 432
Joined: Wed Jun 19, 2019 8:06 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by gGeorg »

Rseding91 wrote: ↑
Fri Nov 22, 2019 1:28 pm
Lubricus wrote: ↑
Fri Nov 22, 2019 1:03 pm
Isn't particles usually some sort of shaders that only lives in the graphics card? And thus use no CPU power nor RAM.
Particles in Factorio can spawn other particles, other entities, smoke, do damage, and so on. They couldn't ever just live in render-only land. Also, you still have to spend CPU time and RAM to tell the GPU what to do so it's never possible to "use no CPU por nor RAM".
Interesting. Does it mean we can have a Factorio particles AND the classic particles which live in the render only land?
With these second one, the graphics dept. would go wild with all the sparkles and steam leaks.

User avatar
Impatient
Filter Inserter
Filter Inserter
Posts: 883
Joined: Sun Mar 20, 2016 2:51 am
Contact:

Re: Friday Facts #322 - New Particle system

Post by Impatient »

Big thanks to Klonan and Rseding for taking a look at the particle system with its edge performance case of flamethrowers defending against biters. This case was the reason why I had to stop playing multiplayer games regularily, when players started to use flamethrower turrets, due to the low end cpu in my pc. Looking forward to see the lowered performance impact.

Side question: Would it not be perfect to use sprites for the decorative bleed out effects? I have no in depth knowledge, but shouldn't take that kind of solution all the performance strain from the CPU and just require a bit more graphics memory?

User avatar
BattleFluffy
Fast Inserter
Fast Inserter
Posts: 194
Joined: Sun Mar 31, 2019 4:58 pm
Contact:

Re: Friday Facts #322 - New Particle system

Post by BattleFluffy »

Thankyou for this excellent optimization work guys :>

User avatar
Ohz
Fast Inserter
Fast Inserter
Posts: 196
Joined: Tue Feb 03, 2015 11:40 am
Contact:

Re: Friday Facts #322 - New Particle system

Post by Ohz »

If you lost count, this scene contains 15,689 entities
oh ok
I'm not english, sorry for my mistakes

Post Reply

Return to β€œNews”