Friday Facts #326 - Particle emitter & Data cache

[MakeItGraphic]

Personally love the data stage updates, it will make it a lot easier testing out mod pack compatibility issues.

With regards to the above comment, I agree there should be an expanded settings screen that reflect the running parameters visually.

There’s a lot of optimisation features there that are easily overlooked if not a wiki/computer buff. Such as the shader toggle, and my favourite the audio toggle to disable it.

Just dropping a config tool into the install dir would be awesome, and have shortcut in the settings or something.

[scooter010]

How about define particels as a function f(t)?
The emitter has just to define all initial values as angle, and velocity, rest ist function of time und u can calculate the partical at every point of time. No problems with visibility area.

[posila]

Personally love the data stage updates, it will make it a lot easier testing out mod pack compatibility issues.

I am not sure if people are understanding the data cache correctly. If you change anything about the data game is loading on startup (enable/disable/update a mod, make a change in a lua file, change an startup setting or update the game itself), the cache will be invalidated and the game will have to reload everything the regular, slow way again.

[Elyviere]

Judging from the chaotic debris caused by the assembler's exploding, I really think things in a small proximity outside your screen should be generating particles as well. There is debris flying well outside of the visible screen in the gif, so reasonably there should also be debris flying into view from anything exploding just beyond what you can see (i.e. the assembler just outside the visible screen).

[Unknow0059]

The effect of the particles hitting the water is a tad overdone, but this was a nice FF.

[BlueTemplar]

Judging from the chaotic debris caused by the assembler's exploding, I really think things in a small proximity outside your screen should be generating particles as well.

Not sure about these new "simple particles" (maybe need better names to clearly separate them from "complex particles"?), but from my testing in 0.16, the game already "draws" most of the things slightly outside of your screen !

[Honktown]

wew just saw the post.

Curious on the caching:

Does it only work if mods don't change at all, or does it still use the cache if the graphics-relevant bits are the same or less than they used to be? I'm thinking if the graphics were cached on the individual level (state from last update), then one wouldn't have to update all of the graphics when the game loads, so only new graphics ever have to be dealt with. Totally gonna try it out either way.

Edit: @Posila, sorry I missed your post. Guess that answers my question.

[orzelek]

Personally love the data stage updates, it will make it a lot easier testing out mod pack compatibility issues.

I am not sure if people are understanding the data cache correctly. If you change anything about the data game is loading on startup (enable/disable/update a mod, make a change in a lua file, change an startup setting or update the game itself), the cache will be invalidated and the game will have to reload everything the regular, slow way again.

Do changes to control.lua and/or files it depends on also invalidate the cache? Or only lua files that are used in data stages and their dependants?

[eradicator]

Personally love the data stage updates, it will make it a lot easier testing out mod pack compatibility issues.

I am not sure if people are understanding the data cache correctly. If you change anything about the data game is loading on startup (enable/disable/update a mod, make a change in a lua file, change an startup setting or update the game itself), the cache will be invalidated and the game will have to reload everything the regular, slow way again.

Do changes to control.lua and/or files it depends on also invalidate the cache? Or only lua files that are used in data stages and their dependants?

Talking about caches, is it considered "working as intended" that changing the content of a graphic file (*.png) without any changes to the data code does not invalidate the atlas cache? I've always found that quite annoying.

Other than that data caching sounds like awesome news :).

[Honktown]

Personally love the data stage updates, it will make it a lot easier testing out mod pack compatibility issues.

I am not sure if people are understanding the data cache correctly. If you change anything about the data game is loading on startup (enable/disable/update a mod, make a change in a lua file, change an startup setting or update the game itself), the cache will be invalidated and the game will have to reload everything the regular, slow way again.

Do changes to control.lua and/or files it depends on also invalidate the cache? Or only lua files that are used in data stages and their dependants?

Talking about caches, is it considered "working as intended" that changing the content of a graphic file (*.png) without any changes to the data code does not invalidate the atlas cache? I've always found that quite annoying.

Other than that data caching sounds like awesome news .

Hehe. Sounds like there was a reason this option was not "user facing" for a long time. Developers might not've considered someone editing an art file without updating the mod or data info during their prototyping of the feature. Your post answers the question of "who would do such a thing??"

I can understand in context: you're editing your art and don't want to update everything else. Out-of-context: "I randomly edited a png and the game didn't know "

Mod checksums are calculated, but does it do that if they're in folder too? Do you know? I swear I've had mods come up as 0 before in the log, which didn't make sense, idk.

[Allaizn]

Talking about caches, is it considered "working as intended" that changing the content of a graphic file (*.png) without any changes to the data code does not invalidate the atlas cache? I've always found that quite annoying.

As far as I know that's intended - cache invalidation needs to be detected, say via checksum. But getting a checksum from the currently existing images in the game folder requires... loading all the graphics. It would be faster to load them just for the checksum (no texture atlas layouting, no cropping etc), but the whole point of the atlas is to make texture loading fast and it's thus not that great of an idea to forgo the speedup in 99.9% of cases just because someone messes with just the graphics contents in 0.1% of cases - especially since the atlas cache is opt-in only.

[Honktown]

Talking about caches, is it considered "working as intended" that changing the content of a graphic file (*.png) without any changes to the data code does not invalidate the atlas cache? I've always found that quite annoying.

As far as I know that's intended - cache invalidation needs to be detected, say via checksum. But getting a checksum from the currently existing images in the game folder requires... loading all the graphics. It would be faster to load them just for the checksum (no texture atlas layouting, no cropping etc), but the whole point of the atlas is to make texture loading fast and it's thus not that great of an idea to forgo the speedup in 99.9% of cases just because someone messes with just the graphics contents in 0.1% of cases - especially since the atlas cache is opt-in only.

On file systems which support metadata, the last modified date could be used instead of a checksum. If there's a time mismatch, the graphics would be loaded. Worst case: someone's graphics didn't get refreshed because they modified the graphics in an epsilon smaller than the system tracked (somehow loaded the game, made the cache, and updated the graphic in like... less than 2 seconds for FAT32, one of the notable offenders). Systems without metadata could revert to the present behavior.

Edit: time mismatch for detecting the file changed, NOT for anything with regards to how the game caches the data

[obuw]

I was updating my mod and stumbled onto some old issues I was having with the particle system, it looks like this would be the perfect time to make a feature request!

The issue arises when creating particle trails for high velocity objects (for example fast vehicles or high-velocity rockets). You want to create a continuous stream of particles trailing the object, but the emitter is moving too fast, so the particles are created like this:

o___o___o___o___o___o___o___o=>

If you increase the particle frequency, it just spawns more particles at the same spot, so you get an effect like this:

o___o___o___o___o___o___o___o=>

How I solve this issue in my own particle system I wrote for my own games is that if the frequency is above 1, I create additional particles to "fill in the blanks" so to speak. For example, if the frequency is 4, then I'd create 4 particles every update, evenly spaced between the emitters previous location and its current one.

oooooooooooooooooooooooooooo=>

If you want to go one step further, ideally, these particles should be created with some of their lives already expended, to simulate the fact that (assuming a frequency of 4) they were supposed to be created 0.25 updates ago, 0.50 updates ago, and 0.75 updates ago. So for example, if the particles normally begin with 24 life, and lose 8 life per update, then you create 4 particles with 24, 22, 20, 18 life. And this is what you end up with:

02 04 06 08 10 12 14 16 18 20 22 24 =>

instead of:

08 08 08 08 16 16 16 16 24 24 24 24 =>

I hope this makes sense. If you could add something like this to the particle system, it would be a huge improvement for high velocity objects with smoke trails!

[DoubleThought]

Of course, if you create a particle with 10.9 and 10.7 frames worth of life, they will both be alive after 10 ticks and both be dead after 11 ticks. The life difference is then only useful for tracking the age of the particle to perform some added effects, like extremely fine animations.

A uint16_t counting life in frames can count up to 18 minutes and 12.25 seconds, which is longer than a particle would likely last. A uint8_t would last only 4.25 seconds, counting in frames. Having more than one life reduced per frame (I would recommend 12: it would let particles last up to 1 minute and 31.0208333... seconds, though rounding up in frames would bring it to 1 minute and 31.0333... seconds, and 12 is a highly composite number).

In my opinion, it's more likely that counting particle time to live is currently done with a uint16_t counting frames.

As for the particle trails, I think it's entirely possible to store the last position of the rocket and calculate the position of the particles based on sub-frame timing. You could even store current and previous velocity of the rocket and use bezier curves and a couple differential equations to create a realistic-looking path the rocket could have taken (that sounds like an interesting and fun project).

Seriously, though, how hard is it to create a function "create_particle_line"? I can write code right now:

Code: Select all

function create_particle_line(particle_type, x1, y1, x2, y2, count)
    if count == 1 then
        create_particle(x2, y2, particle_type)
    end
    dx = (x2 - x1)/(count - 1)
    dy = (y2 - y1)/(count - 1)
    x = x1
    y = y1
    for i in 0, count do
        create_particle(x, y, particle_type)
        x = x + dx
        y = y + dy
    end
end

[obuw]

Not sure if the first part about particles with fractional life values was aimed at me, or someone else, but the difference between a particle with 0.5 frames of life and one with 1.0 frames of life would be their alpha values. Even if they will both expire next frame, they can have different alpha (and tint, scale, hue, etc) values. But it would probably not be very noticeable in either way, except maybe in screenshots.

And yeah, creating the function to draw a line of particles is pretty straightforward, once you put your mind to it. I'm guessing the devs never encountered the issue to begin with, since pretty much every projectile in game is rather slow by default.

[vesoljc]

Seriously, though, how hard is it to create a function "create_particle_line"? I can write code right now:

it's not... but particle emission is not that simple
let's say that you have a simple forward moving emitter:
- with velocity of 1.0 units per second
- emitter emission rate is 1.7 particles per second
- your pc is currently having a hard time and your framerate is not ideal 60fps, but varies from 10-15 each frame

now draw a 3 second timeline on paper, mark frame times, and mark particle spawn times.

[obuw]

now draw a 3 second timeline on paper, mark frame times, and mark particle spawn times.

It's much easier to write an algorithm to calculate this than try to calculate it on paper yourself. This is the reason computers were invented in the first place.

As for frame skipping, I'm sure the particle system in the game already has ways to compensate for it, but if it doesn't, there are various ways to do it if deemed necessary. (I personally have no framerate issues whatsoever even on my potato laptop, so I never actually saw whether system compensates for it or not).

[Oktokolo]

- your pc is currently having a hard time and your framerate is not ideal 60fps, but varies from 10-15 each frame

FPS and frame skipping should be irrelevant for drawing accurate particles. Whenever particles get drawn, their creation paramters and age in game ticks is known. Everything else should be derived from that.
But it would be nice if the engine would automatically disable the particle system and other eye candy when UPS/FPS risk dropping below 60.

[DoubleThought]

it's not... but particle emission is not that simple
let's say that you have a simple forward moving emitter:
- with velocity of 1.0 units per second
- emitter emission rate is 1.7 particles per second
- your pc is currently having a hard time and your framerate is not ideal 60fps, but varies from 10-15 each frame

now draw a 3 second timeline on paper, mark frame times, and mark particle spawn times.

The game has no concept of actual seconds. All durations are in game-seconds, which are ideally equal to real seconds, but not always. This is by design, as having a mechanism to determine the duration of a frame in the Modding API can destroy the determinism that allows for multiplayer.

With this in mind, an emitter with a velocity of 1.0 units per second moves 1/60 units per frame. It also emits a particle every 600/17 frames (= ~35.3 frames).

Frame skipping is a graphics concept, and no logical frames (updates) can be skipped without breaking determinism. If I understand correctly, particles are created, among other times, during triggers. This would likely occur only during updates, so frame skipping does not affect the creation of particles at all. If the game is running at 10-15 UPS, then the emitter will be moving 1.0 units per game-second, or 4-6 units per real second. Its emission rate will be 1.7 particles per game-second, or a particle every ~35.3 frames, so one particle per a duration between 2.3 to 3.5 real seconds.

All of this is irrelevant, of course, because the line-emission function is written for the use case of creating a more than one particle per frame.

[AndrewIRL]

Startup time - Data Cache

Sounds awesome, would love to see a speedup in modded games. With the wrong settings in:
https://mods.factorio.com/mod/WideChests
eg 255 wide by 255 high chests allowed

the game startup time can be as long as 20-30 minutes. Not seconds, that's not a typo, minutes.