Add "overlay" as a supported blending mode

[kirazy]

Is it possible to add overlay as a supported blending mode under https://wiki.factorio.com/Types/Animation#blend_mode?

Specifically the formula as presented here: https://en.wikipedia.org/wiki/Blend_modes under overlay.

[posila]

Unfortunatelly, it is not possible(*) implement in reasonably efficient way.

You may have seen me rant about developing 2D game for hardware optimized for particular 3D workload, and people expecting 2D games to not be demanding on GPU. This is one of the reasons where my frustration comes from. Overlay is very reasonable bledning mode to have when dealing with layered images. But GPUs are limited to setting up factor for backround color, factor for outputted color and 5 operations that can be done between them. In notation the wiki uses it would be F1 * a # F2 * b, where # is one of the operators +, -, min, max, reverse substraction. Factors can be zero, one, a, b, (1 - a), (1 - b), alpha_a, alpha_b, (1 - alpha_a), (1 - alpha_b) and that's about it. See for example http://ycpcs.github.io/cs470-fall2014/labs/lab09.html for better explanation with some pictures etc.

It is impossible to express the overlay formula in the blend state form. It is also impossible to read current render target in pixel shader, so it would have to be done following way: bind target buffer as a second texture, bind temporary render target and clear it, render the sprite to the temporary framebuffer with special shader, bind back the actual target buffer as render target and copy rasterized pixels from the temporary buffer to the target buffer (by drawing the same quad as we used for the sprite). Overlay blending mode is not even associative, meaning we couldn't batch multiple sprites together. And that is just not fast enough for volume of sprites Factorio is dealing with per frame.

(*) In 2020, there is a way, of course. We could circumvent GPU's fixed-function blending stage and use unordered access view (UAV) to have ability to read and write to the same texture in pixel shader. Which would put us to minimum DirectX feature level 11.0 or OpenGL 4.2, and given macOS is stuck on OpenGL 4.1 forever, we are not planning to up minimum requirements again for a while.

Sorry for getting triggered over your simple request.

[Deadlock989]

Out of interest - is there a similar issue with a "light" blending mode, with similar behaviour to rendering.draw_light()?

[posila]

Out of interest - is there a similar issue with a "light" blending mode, with similar behaviour to rendering.draw_light()?

I am not sure if I understand what you mean. Something like dest' = dest + dest * source? (so it brightens background depending on how bright it was to begin with?)

Lights are rendered additively to a light map that is then used to postprocess the game view. The brighter pixel in the light map, the "less post-processing is applied to it" - technically, the game view pixel is looked up in the current LUT, and light map pixel is used as linear interpolation factor between the new game view pixel color and original one.

[Deadlock989]

Out of interest - is there a similar issue with a "light" blending mode, with similar behaviour to rendering.draw_light()?

I am not sure if I understand what you mean. Something like dest' = dest + dest * source? (so it brightens background depending on how bright it was to begin with?)

Lights are rendered additively to a light map that is then used to postprocess the game view. The brighter pixel in the light map, the "less post-processing is applied to it" - technically, the game view pixel is looked up in the current LUT, and light map pixel is used as linear interpolation factor between the new game view pixel color and original one.

Yes, something like that. How does rendering.draw_light() do it?

The reason I ask is that I seem to frequently want to add layers of blinkenlights or fire to things (usually in the working visualisation but not everything has a working visualisation) - which then "shine darkly" during the night. I use additive blending so that the reflection of the lights on the entity's "hull" look nice, but it's still the default game view so that "light" is still treated as a non-emissive material.

The workaround is to spam the entity with actual lights so it's always "lit" - assuming the entity supports lights. There are probably more clever shenanigans in the control stage with draw_light() but I haven't gotten around to exploring that - and doubt it could be used for anything animated.

It is not urgent or I would probably have already asked - more curious than anything else. I would probably use it immediately if it arrived tomorrow though.

Edited to add: just realised I could be using the same sprites as custom light definitions for those entities that support lights in working visualisations ...

Edited again: no you can't, because lights take a picture and not an animation.

[kirazy]

Unfortunatelly, it is not possible(*) implement in reasonably efficient way.

...

Sorry for getting triggered over your simple request.

xD Not a problem.

Are sprites that are composited of multiple layers during the startup phase of the game baked? My use case is tinting various sprites during startup.

[Deadlock989]

Are sprites that are composited of multiple layers during the startup phase of the game baked? My use case is tinting various sprites during startup.

No, they're not baked. You can see all the separate layers in the atlas if you press F3. Mask layers are just another sprite in the atlas with their colours as defined in the PNG (usually white or greyscale), ready for tinting or runtime tinting.

[kirazy]

Ah. Guess it's separate colored layers for me then. :c

Or actually, what's more performance friendly here?

Case 1: Single base-game sprite with all the details, then an image that's only the colored parts laid over it.

Case 2: Single colored image.

If they're not baked, I'm guessing that Case 2 is more friendly. I have been doing 1 since it's significantly more filesize friendly, case 2 causes the mod filesize to explode by a factor of 10 or so.

[Deadlock989]

posila can give you the proper answer but I have asked about this sort of thing before, e.g. belts with layered masks, and it's basically a tradeoff. If you are going to make a lot of coloured variants of stuff then by using masks you are paying a tiny rendering performance cost per frame in exchange for not ballooning the sprite atlas / GPU texture memory use.

And remember that the game automatically trims any dead transparent space around the edges of each frame, so the size masks end up as is usually very small. Also, masks tend to losslessly compress a lot - any kind of monochrome image does. I use PNGGauntlet.

So I would go for case 1 every time unless the number of coloured variants was literally smaller than the number of sprites needed to make a mask. And maybe not even then. I have one entity with 8 layers, 2 of which are shadows, not counting the lights. It has one variant. Have fielded complaints and gripes about all kinds of things but FPS was never one of them.

[posila]

Out of interest - is there a similar issue with a "light" blending mode, with similar behaviour to rendering.draw_light()?

I am not sure if I understand what you mean. Something like dest' = dest + dest * source? (so it brightens background depending on how bright it was to begin with?)

Lights are rendered additively to a light map that is then used to postprocess the game view. The brighter pixel in the light map, the "less post-processing is applied to it" - technically, the game view pixel is looked up in the current LUT, and light map pixel is used as linear interpolation factor between the new game view pixel color and original one.

Yes, something like that. How does rendering.draw_light() do it?

The reason I ask is that I seem to frequently want to add layers of blinkenlights or fire to things (usually in the working visualisation but not everything has a working visualisation) - which then "shine darkly" during the night. I use additive blending so that the reflection of the lights on the entity's "hull" look nice, but it's still the default game view so that "light" is still treated as a non-emissive material.

The workaround is to spam the entity with actual lights so it's always "lit" - assuming the entity supports lights. There are probably more clever shenanigans in the control stage with draw_light() but I haven't gotten around to exploring that - and doubt it could be used for anything animated.

It is not urgent or I would probably have already asked - more curious than anything else. I would probably use it immediately if it arrived tomorrow though.

Edited to add: just realised I could be using the same sprites as custom light definitions for those entities that support lights in working visualisations ...

Edited again: no you can't, because lights take a picture and not an animation.

Ah, so it seems you want equivalent of draw_as_shadow but for lights, not a new blend mode. That might be doable.
draw_light() queues the sprites to "light" sprites instead of game view sprites, so it's rendered into the light map I mentioned earlier.

@kirazy: Pretty much what Deadlock said. I've written couple technical FFFs (https://www.factorio.com/blog/post/fff-281 https://www.factorio.com/blog/post/fff-264 https://www.factorio.com/blog/post/fff-251), and since I have no idea what are you doing, I don't have generic advice what to do.
There are 3 things to consider:
1) How much video memory does it takes up. (or how many pixels do your spritesheets have)
2) How many times does the game overwrite each pixel on average per frame.
3) How many sprites are rendered per frame.

You want all of these to be as low as possible.
If you are making particles, don't use layers and just create different spritesheets for your variations. Same for decoratives and trees. Try to limit number of layers for item icons; for basic resources and intermediates that will be in large quantities on belts, don't use layers. If you are making large machine with long animation sequence and you want to reuse the machine for 3 or more color variations, layers are probably way to go (even better if the colored layer can be on static part, that doesn't animate, so it adds just one sprite). Same for units and turrets ...

[Deadlock989]

Ah, so it seems you want equivalent of draw_as_shadow but for lights, not a new blend mode. That might be doable.
draw_light() queues the sprites to "light" sprites instead of game view sprites, so it's rendered into the light map I mentioned earlier.

"draw_as_light" would be awesome.

[kirazy]

@kirazy: Pretty much what Deadlock said. I've written couple technical FFFs (https://www.factorio.com/blog/post/fff-281 https://www.factorio.com/blog/post/fff-264 https://www.factorio.com/blog/post/fff-251), and since I have no idea what are you doing, I don't have generic advice what to do.
There are 3 things to consider:
1) How much video memory does it takes up. (or how many pixels do your spritesheets have)
2) How many times does the game overwrite each pixel on average per frame.
3) How many sprites are rendered per frame.

You want all of these to be as low as possible.
If you are making particles, don't use layers and just create different spritesheets for your variations. Same for decoratives and trees. Try to limit number of layers for item icons; for basic resources and intermediates that will be in large quantities on belts, don't use layers. If you are making large machine with long animation sequence and you want to reuse the machine for 3 or more color variations, layers are probably way to go (even better if the colored layer can be on static part, that doesn't animate, so it adds just one sprite). Same for units and turrets ...

In general I'm only working with buildings, but the basic approach is to add a layer to an existing structure entity, reusing the vanilla asset as a base layer, inserting a colored sprite layer (with my preferred blending mode baked in, so rather than a white mask + 5x tints, I have 5x colored sprites).

So for example with the east-facing Boiler:

Base layer:


Select the appropriate color from a set of 5:


Shadow layer:


Result:


This is the case for most entities.

In the case of animated entities, usually the colored images have the same number of frames as the animated entity, either because I'm coloring what's moving, or what's moving is traversing a static area that's colored.

For example:


Each of these are made the same way as the boiler.

If it's an issue I can use a white mask + tint in-game, but the available blending modes don't produce results as nice as prebaking Overlay with photoshop. :c

[Deadlock989]

That does seem like the worst of both worlds to me, but it's just an opinion, YMMV. The way the base game handles that kind of thing is the base is white/greyscale under the mask (see the vanilla gun turret for example), then if there are any low-saturation areas of the tinted mask, the mask is more transparent there - so you aren't restricted to the monochrome black-to-full-saturated-colour dimension you'd get from a tinted mask with no variance in alpha values. You can separate out that kind of thing easily in Blender which is where (or so I believe) the vanilla art originates and where I make my stuff.

e.g. layers of additive masks for tints that appear to vary in lightness:

additive-layers.png (199.5 KiB) Viewed 3402 times

That way you could reduce the number of mask spritesheets loaded per entity type down to 2 from 5 (or whatever number of tiers you're showing) at the cost of rendering one extra layer per game world entity, but at the gain of having any of 16 million tints available with a line of code.

But it's swings and roundabouts.

[kirazy]

You can separate out that kind of thing easily in Blender which is where (or so I believe) the vanilla art originates and where I make my stuff.

I have Photoshop and CAD skills, and I've found CAD to not be very transferable to modeling progrems e.g. Blender/Maya.

I can get decent results by adjusting the mask directly, but it still doesn't get the highlights I want and I'm not sure how to use the in-game methods to get the result I'm looking for (thought you've given me some ideas):

White mask with tints, with normal blending mode:


Separate color masks:

[Deadlock989]

Nothing is transferable to Blender. Not even other versions of Blender.

What I meant is you would separate out your lightness from your tint. You'd draw the coloured tint mask first (greyscale, tinted) and then the white-highlights mask (greyscale, not tinted, low brightness = high alpha) on top of that.

[kirazy]

What I meant is you would separate out your lightness from your tint. You'd draw the coloured tint mask first and then the white-highlights mask on top of that.

I gathered, but I'll have to figure out how to do that when all I have access to are the baked vanilla assets.

There's probably a way to scrape off the whitest pixels using Photoshop.

[Deadlock989]

There's probably a way to scrape off the whitest pixels using Photoshop.

My legit copy of PS dates from 2003. But from memory you could do that kind of thing with channels. e.g. make your mask red; the red channel is where the tintable part of the mask is, anything in the blue or green channels is where the white highlights will be. So you use e.g. the green channel as a selection mask for the greyscale version and presto, you have your white highlights separated out with the right alpha values. I am probably not using exactly the right terminology because I am completely self-taught on this stuff.

I mean, if it was just 5 boilers, I probably wouldn't bother. 5 different spritesheets for turrets with 64 directions when the only difference is a single hue mask is different. I even broke my turret animations up into separate lower body, upper body, barrel and ammo-belt sheets because only the upper body needed a runtime mask and breaking the rest up meant the sprite sizes were down to very small cropped images indeed - all at the cost of rendering five small layers instead of two larger ones.

[kirazy]

I mean, if it was just 5 boilers, I probably wouldn't bother. 5 different spritesheets for turrets with 64 directions when the only difference is a single hue mask is different. I even broke my turret animations up into separate lower body, upper body, barrel and ammo-belt sheets because only the upper body needed a runtime mask and breaking the rest up meant the sprite sizes were down to very small cropped images indeed - all at the cost of rendering five small layers instead of two larger ones.

I mean my preference is to use in-game tint method, but I just can't get the results I want and it's frustrating.



This is close, but jaksldj;aklsdjlaks...

So if I can't get it to work, the question is what's the least-worst way to go?... Just to bake the colors and not use layers?

[Deadlock989]

So if I can't get it to work, the question is what's the least-worst way to go?... Just to bake the colors and not use layers?

Leaving turrets and their runtime masks and many directions aside for a moment. If you have 5 tiered entities and you can get the results with 6 sprites (5 variations + 1 shadow shared among the 5) and just 2 layers per entity (main sprite, 8-bit shadow) then that is the optimal way to do it. There is no real point in having a separate colour-baked mask in that situation because that would be 7 sprites in the atlas (a base, a shadow, 5 masks with baked-in colours) and 3 layers per entity (base, mask, shadow). So it's worse both for rendering performance and atlas use - though probably only on a potato. If there is something about that workflow that makes life a lot easier for you, it may still be worth doing because the performance implication is probably not serious. Multiply that by 64 or 128 directions though and it's starting to look shakier. But then, most people's computers will likely cope ... Technically, though, it's not optimal.

If you could get the tintable mask as you wanted it then you'd have just 3 sprites in the atlas and 3 layers per entity (a base, a tintable mask, possibly a highlights mask, and a shadow), so still very much worth the extra layer considering it produces 5+ different entities.

[kirazy]

If you could get the tintable mask as you wanted it then you'd have just 3 sprites in the atlas and 3 layers per entity (a base, a tintable mask, possibly a highlights mask, and a shadow), so still very much worth the extra layer considering it produces 5+ different entities.

Alright, I have a workflow to make masks that seems to produce good results:

Baked on the left, mask on the right:


Takes us from 5x to 2x, an additive highlight and a tintable mask.