Friday Facts #264 - Texture streaming

[arrow in my gluteus]

I have an idea that could be used to improve texture streaming.
For many animations in factorio it doesn't actually matter which frame is shown first, it only matters that they are animated. (Think of the assemblers, it doesn't matter what part of the animation is shown, the animation only shows that the assembler is working).
So you could afford not having all sprites in RAM, but only the first few frames and only load the later frames in RAM once the first few are being used.
Or you could decrease VRAM usage syncing up animations.

[sthalik]

Can we make it good enough, so it wouldn't be an experimental option any more, but something that could be enabled by default? Let's see. The problem is texture allocations, so let's allocate one texture for the entire bitmap cache - it would be a sprite atlas that we would dynamically update. That would also improve sprite batching, but when I started to think how to implement it, I quickly ran into a problem dealing with the fragmentation of space. I considered doing "defragmentation" from time to time, but it started to seem like an overwhelming problem, with a very uncertain result.

I'm assuming "fragmentation" means that you're really-big VRAM texture has sparse areas. It's not entirely clear.

The Radeon driver re-uploads the entire (mega)texture if it's ever been swapped from VRAM back to sysmem. I advise against using just a single texture (buffer object) for this purpose. GL and D3D9 drivers do pretty decent memory management in low-VRAM situations. Of course using one allocation per-sprite is bad, but you can still use multiple max-size atlases (as it is now) with lazy uploads.

The problem with lazy uploads isn't as much bandwidth but latency. Try see what's right outside the player's screen (and just had been researched) and issue uploads with fences basing on that.

In 0.16 the behavior is that the atlases are already filled with tons of stuff that isn't needed for a long time (see, artillery) but also the same atlases contain stuff that's needed basically always. So they can't be unloaded without nuking perf: unload and reupload means uploading the entirety of it.[1]

As for "defragmentation"/"compaction", it's interesting especially given it's done using the GPU only (no sysmem) with LRU or something more involved. But it seems like a bad hack.[2]

What if you just lazy-loaded everything to limited-size megatextures? Keep each megatexture for objects of given size then you can avoid fragmentation and just use a bitmap.[3]

[1] On borderline low-VRAM cases it actually helps to lower atlas size due to aforementioned mismanagement. I'd advise you to assume that only a fraction of the VRAM allocations are actually in VRAM. Hope for the best, assume the worst.
[2] You're considering rewriting a compacting garbage collector for VRAM.
[3] Note how Windows' "operator new" or "malloc" aligns everything to 16 bytes as an implementation detail.

I have an idea that could be used to improve texture streaming.
For many animations in factorio it doesn't actually matter which frame is shown first, it only matters that they are animated. (Think of the assemblers, it doesn't matter what part of the animation is shown, the animation only shows that the assembler is working).

I'd put each "object not working" texture in a common pool. Put animations separately. Don't precache animations for stuff far away in the tech tree like it's done now, see (1) and the paragraph.

[posila]

Or you could decrease VRAM usage syncing up animations.

But that would not trigger people's "OCDs"

The ideas are not bad, difficulty of implementing them is that rendering subsystem and game logic are quite well separated. Entities know how to create sprite draw orders to draw themselves (and don't care much about what happens to them), and renderer doesn't have any context about sprite draw orders it got - for example if it is a frame of an animation or static sprite. There is no unified system of drawing entities - each entity has harcoded logic of creating draw orders, so any changes in this area need to be done for every entity separately, which is lot of work. First experiment with pre-caching sprites will definitelly be something like "there is assembling-machine-1 on the screen, preload every sprite it uses".

[voddan]

Anyway, a GTX 1060 renders a game scene like this in 1080p in 1ms. That's fast, but it means in 4K it would take 4ms, 10ms on integrated GPUs, and more that a single frame worth of time (16.66ms) on old, non-gaming GPUs. No wonder, scenes heavy on smoke or trees can tank FPS, especially in 4K. Maybe we should do something about that

Are you saying you want to optimize for "old, non-gaming GPUs" with 4K displays? Who has those?! LOL!

[gamah]

but factorio appears to just drop frames instead of allowing tearing to happen, even with vsync off...

That's because of borderless fullscreen, the rendered image is not presented to screen directly, but is sent to desktop compositing manager, which then presents it with v-sync anyway. With DirectX 11 we can set flags to allow tearing (on Windows 10 only, though) even when rendering in window or borderless fullscreen. We also plan to add option for exclusive fullscreen.

What... I... You....!!!!!

I never noticed this was the only available fullscreen mode! I just assumed it took over the display... but now it makes sense... display never re-jiggers due to a context switch like other games and the cursor seamlessly leaves the game onto my other monitors and applications.

Kudos to you and the rest of the devs for making me love a game that only runs in my most hated display mode.... now please fix!


Side note: So fun to interact directly with a developer with less than 5 posts on a forum! Such a cool game, and done entirely from scratch instead of using a boxed up engine! I can only imagine what your code base and build systems are like... Being in software myself I often find myself upset with minute details thinking something like "bah this would probably be a 2 line fix, assholes!" only to resent the thought after a few moments of contemplation of the complexity of factorio and my only game-dev knowledge being based entirely on the hand-holding framework that is Unity... You guys are rockstars, can't wait to see the future gameplay changes and optimizations!

[H8UL]

What I haven't got my head around, is whether I should be using some non-default setting on a high end rig, or is that going to improve in 0.17 so I just wait?

Right now I have gtx1080 + i7 7700K @1440p with GSync. Most games are smooth as butter out of the box, but Factorio tends to struggle when I walk past electric miners for example?

[malventano]

but factorio appears to just drop frames instead of allowing tearing to happen, even with vsync off...

That's because of borderless fullscreen, the rendered image is not presented to screen directly, but is sent to desktop compositing manager, which then presents it with v-sync anyway. With DirectX 11 we can set flags to allow tearing (on Windows 10 only, though) even when rendering in window or borderless fullscreen. We also plan to add option for exclusive fullscreen.

PLEASE be sure to add exclusive fullscreen. It's the only way to reliably get VRR displays (operating at a refresh not evenly divisible by 60) to not judder like crazy when playing Factorio.

[Durabys]

https://www.factorio.com/blog/post/fff-264

No wonder, scenes heavy on smoke or trees can tank FPS, especially in 4K. Maybe we should do something about that...

FUCK YES MR. OBVIOUS YOU SHOULD HAVE DONE SOMETHING ABOUT THAT!!!
We have been screaming at you for the last two years to do something about that.

Sorry. Had to get it out. Sorry again.

No you hadn't :
"Yes please".
That way, you'd have looked like a polite interested community member, instead of a spoiled brat (and you wouldn't have had to apologize for being one)

Yes, I had.
Because I never do get social nuances from text alone. Sue me.
PS: I am saying it as someone who spent several Reddit threads arguing that overdraw is the main culprit for Steam Lag and was believed to be an idiot while everyone screamed at me why I am not signing praise to the Devs and kissing the ground where they walk 24/7 (I am doing it 18/5, I want weekend free time, you know).

And Posila puts it better than me:

No wonder, scenes heavy on smoke or trees can tank FPS, especially in 4K. Maybe we should do something about that...

FUCK YES MR. OBVIOUS YOU SHOULD HAVE DONE SOMETHING ABOUT THAT!!!
We have been screaming at you for the last two years to do something about that.

Sorry. Had to get it out. Sorry again.

I am sorry my tease caused you such a negative emotional response. As I said

Our assumption was that the problems were caused by the game wanting to use more video memory than available (the game is not the only application that wants to use video memory) and the graphics driver has to spend a lot of time to optimize accesses to the textures.

we assumed problems come from using to much video memory and so we were adding options to change how sprites are loaded and grouped, so people can find which configuration works for them best. When I changed something it always break performance for some people, so that's we added these changes as options.

Anyway, overdraw is part of the problem, ... other problems (also besides overcommiting of video memory) are how we allocated our render targets, nonoptimal use of texture filtering, ... material for some future friday facts.

I over-reacted to the tease. Pent-up anger over a period of two years. So. I am. Again. Sorry.

[eradicator]

@posila:
Thanks for the post. You need to have more pride in your work and not feel like it's something to apologize for ;).
Btw, what happend to the idea of requiring modders to pre-process textures (from this thread)?
______

I have an idea that could be used to improve texture streaming.
For many animations in factorio it doesn't actually matter which frame is shown first, it only matters that they are animated. (Think of the assemblers, it doesn't matter what part of the animation is shown, the animation only shows that the assembler is working).

From a modders perspective: I'm not sure about assemblers, but for mining drills the animation is linked to the sound, ofc just for that it doesn't matter where you start...but, as far as i remember thanks to the animation being fixed you can theoretically make a drill where the animation is linked to the production cycle. Ofc that's a fairly limited usecase to sacrifice if everything draws much faster instead, so i'm just saying that there are things to lose.
______

Or you could decrease VRAM usage syncing up animations.

But that would not trigger people's "OCDs" :( :)

Hm..if by "synching animations" you mean that every $machine-x entity animates exactly the same frame at the same tick i don't like that at all, so i hope i'm just misunderstanding. While i'm not sure how that would even work for different-speeded machines (irregular beacons/modules), even for same speed machines i find it visually highly desirable to have "well distributed chaos" for lack of a better description (i'm sure one of your in-house artists can explain the importance of visual "chaos" far better than i can). For example if i were looking at an oil field where all oil jacks were pumping visually and auditory in perfect sync i'd feel like everything was looking the same, i'd have no desire to just watch the oil field doing it's thing, because regardless of where i'd look (on that oil field) it'd all be the same. Just think of a fireworks display where every "wave" is exactly the same 3 rockets launched in perfect sync ;).

[arrow in my gluteus]

Hm..if by "synching animations" you mean that every $machine-x entity animates exactly the same frame at the same tick i don't like that at all, so i hope i'm just misunderstanding. While i'm not sure how that would even work for different-speeded machines (irregular beacons/modules), even for same speed machines i find it visually highly desirable to have "well distributed chaos" for lack of a better description (i'm sure one of your in-house artists can explain the importance of visual "chaos" far better than i can). For example if i were looking at an oil field where all oil jacks were pumping visually and auditory in perfect sync i'd feel like everything was looking the same, i'd have no desire to just watch the oil field doing it's thing, because regardless of where i'd look (on that oil field) it'd all be the same. Just think of a fireworks display where every "wave" is exactly the same 3 rockets launched in perfect sync .

well because it's only to optimize VRAM it could be optional. And it doesn't have to be completely in sync. You could have for example only odd animation frames for the first frame, then only even animation frames for the next frame. This still limits the amount of sprites that need to be in VRAM. It could even only be triggered when the game detects that it will run out of vram and switch to software rendering if it doesn't do this.

[sthalik]

Hm..if by "synching animations" you mean that every $machine-x entity animates exactly the same frame at the same tick i don't like that at all, so i hope i'm just misunderstanding. While i'm not sure how that would even work for different-speeded machines (irregular beacons/modules), even for same speed machines i find it visually highly desirable to have "well distributed chaos" for lack of a better description (i'm sure one of your in-house artists can explain the importance of visual "chaos" far better than i can). For example if i were looking at an oil field where all oil jacks were pumping visually and auditory in perfect sync i'd feel like everything was looking the same, i'd have no desire to just watch the oil field doing it's thing, because regardless of where i'd look (on that oil field) it'd all be the same. Just think of a fireworks display where every "wave" is exactly the same 3 rockets launched in perfect sync .

well because it's only to optimize VRAM it could be optional. And it doesn't have to be completely in sync. You could have for example only odd animation frames for the first frame, then only even animation frames for the next frame. This still limits the amount of sprites that need to be in VRAM. It could even only be triggered when the game detects that it will run out of vram and switch to software rendering if it doesn't do this.

It won't work anyway since the latency to upload from sysmem to VRAM is too high for that. You can't upload that much animation data each frame.

Also note that essential sprites are clumped together with stuff like pumpjacks you'd visit 10% of the time. The atlas can't unload because of that. It has to be unused fully to unload from VRAM.

[sthalik]

[...] rendering subsystem and game logic are quite well separated. Entities know how to create sprite draw orders to draw themselves (and don't care much about what happens to them), and renderer doesn't have any context about sprite draw orders it got - for example if it is a frame of an animation or static sprite. [...]

There's a simple solution in PyPy, a tracing Python JIT compiler: they can't reasonably hold each function specialization in RAM. For many versions the VM consumed an unreasonable amount of memory longer it ran. So they added a counter. Prune least-used function-type-specializations first, then divide the existing counters by some constant.

So for workloads where e.g. most inserters are idle, the animations will be flagged as not used too much. With most inserters at work, they'd get a uniform spread.

This doesn't solve your own compaction problem. But as long as you pad sizes to uniform values, VRAM management is reduced to bitmap-per-sprite (sorry, I'm repeating myself).

[Oktokolo]

If for some reason, VRAM use ever becomes an issue, you could give players the option to only use any 2nd, 3rd or 4th frame of each animation, so that only animation FPS suffers instead of having the entire scene FPS drop. That way, animations would lag but player movement would still be butter smooth. Optimally the player could also select groups of stuff to have the full or at least half the animation frames (so they would still get 30 FPS for them). One obvious exemption group would be belts another one inserters. But apart from that i don't know what groups would make sense.

Factorio also has a lot of shadows. If they are only polygons filled with a single RGBA color they could probably be stored as vectors and rendered as geometry somehow to make them occupy less VRAM.

I have an iGPU with 32 GiB of RAM, so VRAM use is not an issue for me. But excessive overdraw probably is - so let's talk about smoke...

I am no GPU coder, but would expect stuff like smoke (maybe also clouds) to be faster if rendered by a procedural shader in the GPU instead of layering tons of bitmaps above each other. That stuff probably is mathematically generated (instead of modelled or painted by hand) already. There are also almost always clusters of many smoke sources in a factory (steam power plants, smelter arrays...).
The smoke shader would probable be able to use a single list or bitmap of smoke source locations and state. It would render all the smoke in a single pass, so that it would cause only a single overdraw per scene pixel.

Maybe, power/circuit cables could also be drawn procedurally for the full scene at once. But they probably do not cause much overdraw anyway.

[arrow in my gluteus]

If for some reason, VRAM use ever becomes an issue, you could give players the option to only use any 2nd, 3rd or 4th frame of each animation, so that only animation FPS suffers instead of having the entire scene FPS drop. That way, animations would lag but player movement would still be butter smooth. Optimally the player could also select groups of stuff to have the full or at least half the animation frames (so they would still get 30 FPS for them). One obvious exemption group would be belts another one inserters. But apart from that i don't know what groups would make sense.

Factorio also has a lot of shadows. If they are only polygons filled with a single RGBA color they could probably be stored as vectors and rendered as geometry somehow to make them occupy less VRAM.

I have an iGPU with 32 GiB of RAM, so VRAM use is not an issue for me. But excessive overdraw probably is - so let's talk about smoke...

I am no GPU coder, but would expect stuff like smoke (maybe also clouds) to be faster if rendered by a procedural shader in the GPU instead of layering tons of bitmaps above each other. That stuff probably is mathematically generated (instead of modelled or painted by hand) already. There are also almost always clusters of many smoke sources in a factory (steam power plants, smelter arrays...).
The smoke shader would probable be able to use a single list or bitmap of smoke source locations and state. It would render all the smoke in a single pass, so that it would cause only a single overdraw per scene pixel.

Maybe, power/circuit cables could also be drawn procedurally for the full scene at once. But they probably do not cause much overdraw anyway.

only using 2nd frame already is an option: low quality rotations

[super_aardvark]

...on old, non-gaming GPUs. No wonder, scenes heavy on smoke or trees can tank FPS, especially in 4K. Maybe we should do something about that...

If someone is running your game on 4k with an old, non-gaming GPU... they've made a poor choice. I don't think you need to spend time covering that kind of scenario.

[arrow in my gluteus]

...on old, non-gaming GPUs. No wonder, scenes heavy on smoke or trees can tank FPS, especially in 4K. Maybe we should do something about that...

If someone is running your game on 4k with an old, non-gaming GPU... they've made a poor choice. I don't think you need to spend time covering that kind of scenario.

maybe for recording timelapses with screenshots?

[sthalik]

If for some reason, VRAM use ever becomes an issue, you could give players the option to only use any 2nd, 3rd or 4th frame of each animation, so that only animation FPS suffers instead of having the entire scene FPS drop. That way, animations would lag but player movement would still be butter smooth.

It doesn't solve the suboptimal memory management problem. Worst-case scenario is that the memory management problem is pushed back to lower priority

I am no GPU coder, but would expect stuff like smoke (maybe also clouds) to be faster if rendered by a procedural shader in the GPU [...]
The smoke shader would probable be able to use a single list or bitmap of smoke source locations and state. It would render all the smoke in a single pass, so that it would cause only a single overdraw per scene pixel.

Both smoke and clouds are frequently implemented via perlin noise. In particular, it's only few individual values that need to be stored for this ordered random noise - scale, octaves, random seed. That's it. You can experiment with various combinations and it all fits in the end. It's even used for terrain height generation.

See that it has no branching and it's a pushover to do in the fragment shader.

...on old, non-gaming GPUs. No wonder, scenes heavy on smoke or trees can tank FPS, especially in 4K. Maybe we should do something about that...

If someone is running your game on 4k with an old, non-gaming GPU... they've made a poor choice. I don't think you need to spend time covering that kind of scenario.

A sprite-based game? I played many recent RPG's keeping the 6970 till it burned down. They weren't shovelware games. Actually many people waited for the 90's RPG revival and the hardware wasn't an issue at all.

[Oktokolo]

only using 2nd frame already is an option: low quality rotations

According to the option tooltip that is only for trains while driving on curved rail.
There are tons of other animations with high frame counts in the game.

[posila]

I'm assuming "fragmentation" means that you're really-big VRAM texture has sparse areas. It's not entirely clear.

Yeah. Each sprite has different size, so one sprite can't replace another without leaving some extra space, these unused regions would accumulate over the time and eventually the game would want to upload a sprite for which there would be enough total number of unused pixels, but none of the empty regions would be large enought for fit the sprite on its own.
Virtual texture mapping doesn't have this problem, because atlas is divided into tiles of the same size, so any tile can replace any other. Minor inconvenience is that a single sprite can span over multiple tiles, and sprite you want takes up just fraction of area of a tile you need to upload, but it's not as big problem as it might seem to be. Streaming RAM -> VRAM is pretty fast, it might not have good latency, but as long as there is no synchronization where CPU would have to wait on GPU, the work should be nicely pipelined (as in, while rendering current frame, we are queueing texture updates for next frame, so they can be done as soon as current frame is finished)

The Radeon driver re-uploads the entire (mega)texture if it's ever been swapped from VRAM back to sysmem. I advise against using just a single texture (buffer object) for this purpose. GL and D3D9 drivers do pretty decent memory management in low-VRAM situations. Of course using one allocation per-sprite is bad, but you can still use multiple max-size atlases (as it is now) with lazy uploads.

Well, point is to have texture that is never unused and all draw calls use just this texture. We would just make sure it contains all parts of mega atlas that is needed. Up until 0.16 we were leaving everying on driver to figure out, but as you said, driver works with entire texture object at once. We tried to split sprites into more atlases based on which layers they were used in. Other ways of sorting out sprites into atlases don't make much sense, because we can't predict what will players build and what sprites will be needed at the same time (+ how mods will modify the game).

Anyway, a GTX 1060 renders a game scene like this in 1080p in 1ms. That's fast, but it means in 4K it would take 4ms, 10ms on integrated GPUs, and more that a single frame worth of time (16.66ms) on old, non-gaming GPUs. No wonder, scenes heavy on smoke or trees can tank FPS, especially in 4K. Maybe we should do something about that

Are you saying you want to optimize for "old, non-gaming GPUs" with 4K displays? Who has those?! LOL!

No, no, no ... it's 4ms in 4K, 10ms on iGPU (in FullHD), more than 16.66ms on media GPUs (in FullHD)

What I haven't got my head around, is whether I should be using some non-default setting on a high end rig, or is that going to improve in 0.17 so I just wait?

Right now I have gtx1080 + i7 7700K @1440p with GSync. Most games are smooth as butter out of the box, but Factorio tends to struggle when I walk past electric miners for example?

Hmm, default settings should be fine (the game detects VRAM size and RAM size and sets defaults based on that), but it's possible you get better performance if you tinker with them. It will be better in 0.17 though.

[Oktokolo]

If for some reason, VRAM use ever becomes an issue, you could give players the option to only use any 2nd, 3rd or 4th frame of each animation, so that only animation FPS suffers instead of having the entire scene FPS drop. That way, animations would lag but player movement would still be butter smooth.

It doesn't solve the suboptimal memory management problem. Worst-case scenario is that the memory management problem is pushed back to lower priority

It might indeed solve the memory management problem. After reducing the number of animation sprites, the remaining sprites could fit into the VRAM all at once. That would remove the need to constantly load and unload sprites. Without deallocation, there is no fragmentation. Without fragmentation they do not need to implement the defragmentation.
Better memory management comes with a runtime cost and is nontrivial to implement and test.
Dropping half or more of the animation sprites could be done once at loading the game and is easy to implement and test.
If i would have a VRAM-limited system, i would want more complicated memory management pushed back in favor of implementing something that is fast to implement and guaranteed to reduce the VRAM load dramatically...