Set Hue for data stage (and for control stage; and saturation and value)

[Qon]

@Qon:
Except full GLSL support would be instantly abused and impossible to make secure.

GLSL can't access a filesystem though? Any examples of how it would be abused or insecure?
Why would it be worse than Lua? GLSL runs on the GPU so to me it seems harder to access the wrong things even in case of a fatal buffer overflow in some implementation. Maybe I'm overlooking something?
And even if the language has support for 'bad' functions you can 'just' not include those in the implementation that mods can access.

[eradicator]

@Qon:
Except full GLSL support would be instantly abused and impossible to make secure.

GLSL can't access a filesystem though? Any examples of how it would be abused or insecure?
Why would it be worse than Lua? GLSL runs on the GPU so to me it seems harder to access the wrong things even in case of a fatal buffer overflow in some implementation. Maybe I'm overlooking something?
And even if the language has support for 'bad' functions you can 'just' not include those in the implementation that mods can access.

Or maybe i know less about GLSL than you think and i just talk without thinking sometimes p:. If you can "just exclude" low level access i'm wrong. My main "worry" is that graphics card drivers are not known for being the spear head of security oriented programming. Memory leaks, privilidge escalation, ye know, the usual stuff. So it'd be yet another sandbox (alongside lua) that wube would have to worry about. And as it's also "yet another thing to learn" most modders would be confronted with "do i learn Photoshop™®💀 or GLSL", which probably wouldn't end favourible to GLSL.

[darkfrei]

By the first starting the stage "Checking sprites" needs about 57 seconds. The second game starting in the same day needs for "Checking sprites" and "Loading sprites" only 9 seconds.
The game checks sprites and computes some stuff for using after. It can be another graphics calculation time.

[Qon]

Or maybe i know less about GLSL than you think and i just talk without thinking sometimes p:. If you can "just exclude" low level access i'm wrong. My main "worry" is that graphics card drivers are not known for being the spear head of security oriented programming. Memory leaks, privilidge escalation, ye know, the usual stuff. So it'd be yet another sandbox (alongside lua) that wube would have to worry about.

Well you wouldn't interact with graphics card driver directly. You just make mathematical functions that operate on vectors (colors) that are executed for every pixel.

And as it's also "yet another thing to learn" most modders would be confronted with "do i learn Photoshop™® or GLSL", which probably wouldn't end favourible to GLSL.

It takes about as many seconds to learn GLSL as Lua. It has C-like syntax and some mathematical functions and syntax sugar for the vector types but is otherwise simpler and lacks some features.

Here's realtime fullscreen color rotation in GLSL:

Code: Select all

// hsv2rgb() taken from https://www.shadertoy.com/view/XslGzl
vec3 hsv2rgb(vec3 c) {
    vec4 K = vec4(1.0, 2.0 / 3.0, 1.0 / 3.0, 3.0);
    vec3 p = abs(fract(c.xxx + K.xyz) * 6.0 - K.www);
    return c.z * mix(K.xxx, clamp(p - K.xxx, 0.0, 1.0), c.y);
}
// http://lolengine.net/blog/2013/07/27/rgb-to-hsv-in-glsl
vec3 rgb2hsv(vec3 c) { 
    vec4 K = vec4(0.0, -1.0 / 3.0, 2.0 / 3.0, -1.0);
    vec4 p = mix(vec4(c.bg, K.wz), vec4(c.gb, K.xy), step(c.b, c.g));
    vec4 q = mix(vec4(p.xyw, c.r), vec4(c.r, p.yzx), step(p.x, c.r));

    float d = q.x - min(q.w, q.y);
    float e = 1.0e-10;
    return vec3(abs(q.z + (q.w - q.y) / (6.0 * d + e)), d / (q.x + e), q.x);
}
vec4 colorize(float c) {
    float hue = mix(0.0, 1.0, 1.0 - c);
    float sat = 1.0;
    float lum = 1.0;
    vec3 hsv = vec3(hue, sat, lum);
    vec3 rgb = hsv2rgb(hsv);
    return vec4(rgb, 1.0);
}
void mainImage( out vec4 fragColor, in vec2 fragCoord ) {
    vec2 uv = fragCoord.xy / iResolution.xy;
    vec3 texColor = texture(iChannel0,uv).rgb;
    float a = (texColor.r + texColor.g + texColor.b)/3.0;
    hsv.x = mod(hsv.x + 0.5, 1.0);
    vec3 rgb = hsv2rgb(hsv);
    fragColor = vec4(rgb, 1.0);
}

Try it in your browser. Click iChannel 0 in the bottom and select a video (Claude Van Damme has colours) or your webcam. The demo has several filters, like greyscale and 'thermal camera'. Click and drag the output video with your mouse, mouse.x is used to determine how to choose which part gets which filter.

[Qon]

I meant "on the fly" as in while being loaded in prototype stage, not in Factorio runtime. Though that would be even cooler.

I've changed by mind. I want it for prototype and runtime. Real-time ray marching displays inside Factorio? Real-time animated icons? Turn Factorio into 3D? Why not?

There are so many cool things we could do with proper shaders. The modern Factorio night vision is an example. What if we could do things like that in mods?

Posila, do you accept bribes to change your mind?

Back to topic (to other things, I might respond in the hue thread)...

Posila is ready to take our bribes! Let's cooperate and make an irresistible offer!
(I'm broke so I'm relying on you!)

[posila]

Not so fast Qon! I'll try to start on positive note before inevitably spiraling down to my cynical asshole self and start ranting

Hue shift as preprocess step is less likely to happen than preprocessing layered icons. But I have been thinking how to make runtime hue shift and desaturation happen. The problem is following. Because of our art-style, which utilizes semi-transparency a lot, we must render sprites from back to front. Proper way to do rendering is to group meshes by material and render many objects with common material at once, then switch to another material and draw all objects that have that material at once, etc. (Material is essentialy, tuple of texture(s), shader, shader parameters and possibly some other pipeline states). Because we don't have a choice of order in which we draw sprites, only option to keep number of draw calls low, and batches of work sent to GPU high, is to use the same material for all sprites, which means put all sprites into single texture (which is not possible anymore), use the same shader and parameters, etc.

So, runtime hue shift would esentially had to be calculated on all sprites, regardless if you want to hue shift it or not (same as currently tint is applied to all sprites, but often times it is just {1,1,1,1}). While it would be breeze for modern GPUs that are often not running in the highest power mode when running Factorio anyway, some people run the game on old garbage integrated GPUs, because their dedicated GPU is overheating for whatever reason (wink wink), or simply just don't have better hardware available at all, and maybe even bought Factorio because it is a 2D game and expected to not have high GPU requirements. We need to be very careful and make sure we don't add something that would degrade performance on not-so-good hardware while not being possible to be disabled in graphics options (if art requires hue-shift, we can't just decide it's not going to be hue-shifted on some computers).

Maybe guarding it by branch in the shader would be good enough, but I am afraid that just presence of the branch would increase shader register pressure and reduce on-GPU paralellization. I liked this formula (https://cmwdexint.com/2015/02/03/awkwar ... ft-shader/) it is much cheaper than proper RGB->HSV->RGB conversion and might provide similar capabilities (I have not tested it yet).

----------------

The above is also one of the reasons there won't be access to shaders in prototypes. When doing the rendering rewrite, I thought there could at least definitions for fullscreen post-process passes, but no. The reality is, the graphics programing feels extremely fragile, as every minor change or feature that should just work causes problems for someone (e.g. 71649). I think our new rendering backend is major improvement for almost everybody and is quite stable, but it certainly doesn't feel that way. I can't even imagine support hell that would begin if we allowed mods to introduce their own shaders.

I hope our next game will use 3D rendering, or at least 2D art style that is very light on transparency. At this point, I want to push Factorio to 1.0 state and want to avoid adding unecessery complexity or complications that might delay that goal further.

[Qon]

So what I'm hearing is that custom per-sprite shaders can't run in runtime. But it's possible in prototype stage?

The above is also one of the reasons there won't be access to shaders in prototypes.

What is a "shader in prototype"? Is that a shader that is static (set in data stage) but gives dynamic results because it executes in runtime (IE no shader compilation in runtime)? Or is it shader full-stop in prototype stage? Shaders that are executed once in prototype stage to generate static sprites should work, right? The sprites that are made are loaded via a shader instead of a file (and give the shader access to a list of files) and doesn't affect render order or make the graphics pipeline any different.

[eradicator]

So what I'm hearing is that custom per-sprite shaders can't run in runtime. But it's possible in prototype stage? :D

The above is also one of the reasons there won't be access to shaders in prototypes.

What is a "shader in prototype"? Is that a shader that is static (set in data stage) but gives dynamic results because it executes in runtime (IE no shader compilation in runtime)? Or is it shader full-stop in prototype stage? Shaders that are executed once in prototype stage to generate static sprites should work, right? The sprites that are made are loaded via a shader instead of a file (and give the shader access to a list of files) and doesn't affect render order or make the graphics pipeline any different.

:)

Yea. I don't need them to run at runtime. It'd already be super-awesome if they could be used to generate/change static sprites that behave as if they were loaded from a file. I.e. similar to how pre-composing layered icons would work if it existed.

[Qon]

Yea. I don't need them to run at runtime. It'd already be super-awesome if they could be used to generate/change static sprites that behave as if they were loaded from a file. I.e. similar to how pre-composing layered icons would work if it existed.

Yeah, we wouldn't have to ask for hue rotations, saturation/hue masking, (de)saturation, contrast changes and so much more. One feature to rule them all!

[BlueTemplar]

Another issue is that AFAIK hue rotations aren't "perceptually linear" in (s?)RGB ?

I guess that it's too late to switch Factorio from sRGB = Rec. 709 (RGB,HSL,HSV...) to Rec. 2020 by now,
but what about SPIR-V (or equivalent) support once Factorio supports Vulkan, way down the line ?
(Also, would you drop DirectX and OpenGL support by then ?)

[posila]

Another issue is that AFAIK hue rotations aren't "perceptually linear" in (s?)RGB ?

I think that hardly matters for the use cases in Factorio. Our renderer is not even gamma correct (treats PNG sources - which are usually sRGB, as if they were linear). Modders who want nice art make new art, modders who don't care just tint the existing art. Any kind of hue shift would be major improvement for them.

I guess that it's too late to switch Factorio from sRGB = Rec. 709 (RGB,HSL,HSV...) to Rec. 2020 by now,
but what about SPIR-V (or equivalent) support once Factorio supports Vulkan, way down the line ?

I am not familiar with Rec. 2020, based on wiki it looks like you are asking for higher bitdepth for colors?
Not sure what SPIR-V has to do with anything. Yes, Vulkan backend would use SPIR-V for shaders.

(Also, would you drop DirectX and OpenGL support by then ?)

No. Vulkan is not supported on GTX 500 Series and older, pre-GCN Radeons (Radeon HD 7xxx–76xx Series and older) and pre-Skylake Intels, at least on Windows. Unless eradicator got new laptop on Christmas, he wouldn't be able to run the game on Vulkan. (Also I don't think we'll have Vulkan backend for 1.0)

EDIT: What I don't like about idea of running scripts to create new sprites from existing sprites at load time, is that it makes it very easy to just write a loop and multiply total size of sprites on a few lines. Now if you want to make 100 assembling machine variants, you either just tint the base one, or start making separate spritesheets. Hopefully, very soon you realize have new spritesheet for each machine is not a way to go and try to solve it differently (i.e. by making color mask instead).

[Optera]

Hopefully, very soon you realize have new spritesheet for each machine is not a way to go and try to solve it differently (i.e. by making color mask instead).

Sounds a bit hypocritical considering most base game entities like assemblers or fluid wagons don't have color masks.

For mods it's often simpler to recolor sprites than create a properly working color mask.

[posila]

Hopefully, very soon you realize have new spritesheet for each machine is not a way to go and try to solve it differently (i.e. by making color mask instead).

Sounds a bit hypocritical considering most base game entities like assemblers or fluid wagons don't have color masks.

For mods it's often simpler to recolor sprites than create a properly working color mask.

There is just one variant of fluid wagon in vanilla, so making it have more layers is worse than having less layers, from my perspective. (Each assembling machine has different model and animation, so masking them can offer very little gains also). Some of my concerns for vanilla is overdraw and total number of sprites drawn in a scene (so again, less layers better) - I don't quite care about these in modded games, but I do care about VRAM usage not exploding easily (i.e. by generating sprites procedurally).

Anyhow, I do realize recoloring is the desired ability and that runtime tint is not good enough often times. That's why I said some kind of runtime hue shift (but maybe not exactly hue shift) is not out of the question, few post ago. Sadly there has been very little progress made on this so far.

[BlueTemplar]

"With great power comes great responsibility"
Is this such a big issue with mods like these ?
https://mods.factorio.com/mod/Natural_Evolution_Enemies

I am not familiar with Rec. 2020, based on wiki it looks like you are asking for higher bitdepth for colors?

No, wider gamuts.
(Though, which tend to also require higher bitdepth, to avoid banding !)
Pushed by the attempt to keep selling lots of TVs, consumer hardware has finally started moving towards wider gamut colors,
so 1996's sRGB* (and it's LCD equivalent, Rec. 709) is being phased out for Rec. 2020 :

Rec. 2020 area-coverage ratios, calculated separately for cyan, magenta, and yellow (Magenta % seem off ?)
(Which is basically called "HDR" in marketing-speak.)

*the term "gamma" itself is a throwback to how CRTs were designed. CRTs use a power law (gamma being the exponent), which makes them perceptually uniform. Whereas LCDs have a linear response (hence, no gamma), but this also make them NOT perceptually uniform ! (since human vision is not linear)

Not sure what SPIR-V has to do with anything. Yes, Vulkan backend would use SPIR-V for shaders.

This is in reference to the talk about GLSL <-> OpenGL.

No. Vulkan is not supported on GTX 500 Series and older, pre-GCN Radeons (Radeon HD 7xxx–76xx Series and older) and pre-Skylake Intels, at least on Windows. Unless eradicator got new laptop on Christmas, he wouldn't be able to run the game on Vulkan. (Also I don't think we'll have Vulkan backend for 1.0)

Yeah, I'm talking waaay down the line, when current hardware is considered old !

Our renderer is not even gamma correct (treats PNG sources - which are usually sRGB, as if they were linear).

Yeah, looks like PNG might have to be abandoned, as it only seems to support sRGB ? EDIT : Nope ?

[posila]

"With great power comes great responsibility"
Is this such a big issue with mods like these ?
https://mods.factorio.com/mod/Natural_Evolution_Enemies

That's fine. Scaling and tinting is done runtime, so for all those only once copy of the spritesheet exists in memory. I just want to kill dreams of "let us make a copy of a sprite and modify it however we want during startup". Had those spitters have copy for each scale and color, that could be massive problem.

Thanks for the info on Rec. 2020.

[Optera]

Hopefully, very soon you realize have new spritesheet for each machine is not a way to go and try to solve it differently (i.e. by making color mask instead).

Sounds a bit hypocritical considering most base game entities like assemblers or fluid wagons don't have color masks.

For mods it's often simpler to recolor sprites than create a properly working color mask.

There is just one variant of fluid wagon in vanilla, so making it have more layers is worse than having less layers, from my perspective. (Each assembling machine has different model and animation, so masking them can offer very little gains also). Some of my concerns for vanilla is overdraw and total number of sprites drawn in a scene (so again, less layers better) - I don't quite care about these in modded games, but I do care about VRAM usage not exploding easily (i.e. by generating sprites procedurally).

Anyhow, I do realize recoloring is the desired ability and that runtime tint is not good enough often times. That's why I said some kind of runtime hue shift (but maybe not exactly hue shift) is not out of the question, few post ago. Sadly there has been very little progress made on this so far.

That's a moot point.
Cargo wagons do have color mask while there's only one in base. The point here is, having color masks opens up interesting mods like coloring a wagon according to inventory.
Assemblers share the same foundation, that part could be a shared sprite colored differently depending on tier. Is it pointless in base? maybe, but again it can provide the scaffolding for interesting mod ideas for little to no cost.

[posila]

Cargo wagon masks were added with locomotive masks back when kovarex's plan was to have wagons also custom tintable by players. Removing things is always sensitive issue, so we are stuck with cargo wagon masks.

I am trying to keep some balance between performance, game's look and modability. Sometimes I'll favor modability even if it makes my life harder, or makes it so the game can't perform as well as it could otherwise. I often say yes to things that I think will make the (vanilla) game look better, and if we can add an option to turn them off they don't need to be that optimized. Sometimes I'll go for performance. 2x speedups Rseding does, he does them by optimizing 100 small things. And that works both ways. 100 minor ineffencies in the name of better modding opportunities, and suddenly people can't play the game on their old laptop - so I'd better pick 100 really good ones. (not being able to run the game on old laptop is not necesserily bad thing, but we already sold the game to those people ... it's not like we have not dropped compatibility with some HW before, but I don't want to make habbit out it, especially if it is not necessary)

I often see people write things like "it's a 2D game, it will run on anything, even potato" making it sound like it'll run on anything because it's 2D. Unfortunatelly, that's not quite how it works. It runs on "a potato" because we made it that way.

[BlueTemplar]

Graphic cards, even integrated ones, being rather focused on 3D acceleration these days, doesn't make your life easier, does it ?

[eradicator]

Modders who want nice art make new art, modders who don't care just tint the existing art. Any kind of hue shift would be major improvement for them.

"Don't care" sounds like "care" is the only factor involved in creating new art. We all know that's not true. Even the really nice mods (angel, pyanodon, etc) have a hard time mimicing vanilla style, many nice models look like plastic toys when mixed with the dirty/gritty vanilla art. If a modder lacks skill/time (or proper hardware *cough*) to make a really good new model it's often better to recolor an old one than to use a sloppy new one. And that is the part that would be improved.

Unless eradicator got new laptop on Christmas

Still waiting for DDR5 to be released ..

Hopefully, very soon you realize have new spritesheet for each machine is not a way to go and try to solve it differently (i.e. by making color mask instead).

I actually made an official request to include color masks once under the assumption/hope that vanilla art at least internally (in blender) uses some sort of masking layout that makes it cheap enough to maintain an extra mask just for modding.

I never cared how "let me make nicer color variants easier" would be implemented. Runtime tint - as long as it's not also runtime scriptable - doesn't produce different visual effects than adding new spritesheets in data stage does. So i'd take whatever is deemed to be best for performance.

[darkfrei]

If a modder lacks skill/time (or proper hardware *cough*) to make a really good new model it's often better to recolor an old one than to use a sloppy new one.

Offtopic:
The blender 3d needs much slower computer as you expected.