Friday Facts #264 - Texture streaming

[posila]

You said:

~0.1ms on GeForce GTX 1060.

but then wrote:

Anyway, a GTX 1060 renders a game scene like this in 1080p in 1ms. That's fast, but it means in 4K it would take 4ms

But would it not mean, that it takes 0.1 ms for a 4K game scene? Or did I miss understand it?

What I measured (or wanted to measure) is how long does it take to alpha-blend 1080p worth of pixels (so that what the ~0.1ms is), and the 1ms is how long did it take to render entire game (so when we render, the total time is as if we were overwriting every pixel 10 times ... which might not be far from reality)

[HurkWurk]

At this point, I think it would be simpler and faster to render actual 3d models instead of using 2d graphics at all.

I agree ... I wish we had realized it 3 years ago, before we started doing high-res sprites. Turning the game to 3D is not really an option now.

why not? you already created all the models in 3D anyway?
another game i play, Creeper World, he basically did much the same you guys did, a sprite game, but then for the new one hes working on, he switched to 3D, and it works out ok, just simplified his models some to keep the poly count down. (and lord could he benefit from your optimizations, i love the game, but it lags badly after you build a lot of things)

and you could leave the game tile based with basic 3D. (i think one of the sim cities did this?) so really only converting models and the base texture. i dont know... seems like it may be worth at least creating a project plan to determine actual time. my guess would be ~6-8 months to convert the existing version to 3D.

[eradicator]

At this point, I think it would be simpler and faster to render actual 3d models instead of using 2d graphics at all.

I agree ... I wish we had realized it 3 years ago, before we started doing high-res sprites. Turning the game to 3D is not really an option now.

why not? you already created all the models in 3D anyway?
another game i play, Creeper World, he basically did much the same you guys did, a sprite game, but then for the new one hes working on, he switched to 3D, and it works out ok, just simplified his models some to keep the poly count down. (and lord could he benefit from your optimizations, i love the game, but it lags badly after you build a lot of things)

and you could leave the game tile based with basic 3D. (i think one of the sim cities did this?) so really only converting models and the base texture. i dont know... seems like it may be worth at least creating a project plan to determine actual time. my guess would be ~6-8 months to convert the existing version to 3D.

You'd also instantly kill 100% of all mods that use recolored base assets, and even the ones that use custom 3D models would be thrown back into the stone age. I use blender to make my models, but i only have a bare minimum understanding. I can make them render nicely from a single perspective by using all sorts of hacks, like a transparent floor that hides everything that's below the floor (e.g. to make things that look like they're holes in the floor). But i have no clue how to make them usable as game assets, and i've certainly never cared about poly-count, as that is the main strength of 2D. You can add as much detail as you want, once it's baked into png it all costs the same, which is especially nice for expensive things like particle effects, smokes, full scene lighting, etc. Also while many people are familiar with basic 2D image editing tools i think many potential beginning modders would be scared away if 3D was mandatory.

That said i'm certainly not against 3D factorio if it helps, even if it wasn't rotatable, (would give me a push to learn some new things in blender :p). Good old Total Annihilation used 3D models for all the units, but it still "felt" like a 2D game for most parts.

[Jap2.0]

You said:

~0.1ms on GeForce GTX 1060.

but then wrote:

Anyway, a GTX 1060 renders a game scene like this in 1080p in 1ms. That's fast, but it means in 4K it would take 4ms

But would it not mean, that it takes 0.4 ms for a 4K game scene? Or did I miss understand it?

From my understanding, 0.1ms is copying the image to the screen, and 1ms is rendering the entire scene (which takes longer because you have overlap, many sprites, etc.)

[Frightning]

https://www.factorio.com/blog/post/fff-264

No wonder, scenes heavy on smoke or trees can tank FPS, especially in 4K. Maybe we should do something about that...

FUCK YES MR. OBVIOUS YOU SHOULD HAVE DONE SOMETHING ABOUT THAT!!!
We have been screaming at you for the last two years to do something about that.

Sorry. Had to get it out. Sorry again.

I get this feeling too, as I also pointed this issue out to them a long time ago. It was obvious to me that something about how they handled smoke and tree rendering was inefficient from a 'way that it makes use of system resources' standpoint. Nice to know that they are now aware of what the bottleneck is. I don't know enough about rendering to know what to do about it, but I'm sure they could ask people that have expertise on what to do instead of overdrawing a lot to get good looking smoke/trees. I get a pretty solid 60+fps even with max res. sprites *most* of the time. It tends to dip with smoke/trees and now some of the grasses as well, and I only have 1GB of VRAM on my GTX550 Ti.

[bobingabout]

At this point, I think it would be simpler and faster to render actual 3d models instead of using 2d graphics at all.

I agree ... I wish we had realized it 3 years ago, before we started doing high-res sprites. Turning the game to 3D is not really an option now.

why not? you already created all the models in 3D anyway?
another game i play, Creeper World, he basically did much the same you guys did, a sprite game, but then for the new one hes working on, he switched to 3D, and it works out ok, just simplified his models some to keep the poly count down. (and lord could he benefit from your optimizations, i love the game, but it lags badly after you build a lot of things)

and you could leave the game tile based with basic 3D. (i think one of the sim cities did this?) so really only converting models and the base texture. i dont know... seems like it may be worth at least creating a project plan to determine actual time. my guess would be ~6-8 months to convert the existing version to 3D.

I scribble in paintshop pro. 3D only would mean I wouldn't mod it. There would be no bob's mods in a 3D factorio.

[TheYeast]

The bitmap cache is replaced by virtual texture mapping with the texture data being kept in RAM, next big challenge is to load them from disk on demand, but different streaming technique would not make that easier. The big question is - who does it help? How much is it needed? How much time is worth spending on it. The thing is, when I started experimenting with streaming, I didn't know the latest Steam HW survey stats, and I was under an impression high video memory usage is big problem for large portion of players. When I was testing the streaming on low end HW, I found out I can load 0.16 with high-res sprites and mods that increate total VRAM usage to 4GB just fine on old GPU with 1GB VRAM, even with "Low VRAM mode" disable, and it doesn't even run that much worse than in 0.17 with streaming, so maybe we should optimize other areas first before trying to push streaming tech forward?

My system has serious issues with running high-res sprites on a 1GB card.... I'll try tuning the settings some more.

So one thing is not entirely clear to me: will the new Virtual Textures technique only be used for the low-priority sprites that did not make it into the atlas, or will the entire atlas be replaced with Virtual Textures?
Even 512MB can easily hold a hundred 1024x1024 RGBA textures, so you could cache at least 6400 128x128 tiles at the same time if all VRAM is used for the tile cache. I can't help but think that that should be more than enough for playing with high-res sprites, as only a fraction of the sprites will be in the same scene (unless you build an extremely dense factory). Indeed, if the cached tiles are replaced with low-res versions when zooming out, the increased number of different objects in view will be countered by that.

My estimate may be totally off though I tried to verify this with debug info on, but could not find data on that. Would it be worth the effort to add counters for the number of different sprites used during the drawing of a frame, and the total pixel area of all these combined? That might indicate whether it should be considered replacing the entire atlas with Virtual Textures.

[raidho36]

I scribble in paintshop pro. 3D only would mean I wouldn't mod it. There would be no bob's mods in a 3D factorio.

Nothing's stopping you from scribbling in Sketchup 3D. And pretty much all CAD software out there has architecture design tools and can export 3d models.

Besides, 3d modelling is easy. You can make pretty much anything in just two simple steps:

15105055634553.jpg (31.81 KiB) Viewed 6771 times

[bobingabout]

I scribble in paintshop pro. 3D only would mean I wouldn't mod it. There would be no bob's mods in a 3D factorio.

Nothing's stopping you from scribbling in Sketchup 3D. And pretty much all CAD software out there has architecture design tools and can export 3d models.

Besides, 3d modelling is easy. You can make pretty much anything in just two simple steps:
15105055634553.jpg

I've tried. I can't do it. I have Aphantasia. if I can't actually see it, I can't visualise it. I can't even draw very well from scratch either, but if something exists and it doesn't look right, I know it isn't right and can problem fix it. with 3D that level of "It's not right" is more complex, because I can only see a 2D plane of a 3D image, I know it's not right, but can't figure out what to do to fix it. and the less complete an image is, the more difficult it is to do. I could probably start with a blank 2D image and come out with something that looks okay given enough time, but not really in 3D.
That's one reason why when I play building games, pretty much everything comes out in square zones, or cuboids, because my brain can understand the hard rigid logic of straight lines, but has difficulty with curves and fancy patterns.
The latest space building game I've been playing is Avorion. for most of the game I've been flying a brick.

I pretty much have to scribble things down to think a lot of the time, unless it can be done entire in words and sounds. To note: I SPEAK programming languages(in my head), because if I didn't, I wouldn't be able to program in them.
It's not just Factorio either, there's plenty of games in the past that are very modder friendly, but because they're 3D, not 2D, I didn't even try.

Back on the note of "Speaking" programming languages. my brain TALKS to my hands when I'm typing. If I type too fast, or my hands lag behind what's going on in my head, then I often end up with one word replaced with the next word, so you get the same word twice. Also, again, audible words, so even if I'm thinking of say... more, my hands might actually end up typing maw instead (It's a furry word for mouth that sounds the same as more). I often have to read and reread everything I type, because if I don't, I often find that I've just typed a load of bullshit that hardly makes sense.

[Oktokolo]

I scribble in paintshop pro. 3D only would mean I wouldn't mod it. There would be no bob's mods in a 3D factorio.

Nothing's stopping you from scribbling in Sketchup 3D. And pretty much all CAD software out there has architecture design tools and can export 3d models.

Besides, 3d modelling is easy. You can make pretty much anything in just two simple steps:
15105055634553.jpg

I've tried. I can't do it. I have Aphantasia. if I can't actually see it, I can't visualise it. I can't even draw very well from scratch either, but if something exists and it doesn't look right, I know it isn't right and can problem fix it.

I am pretty sure, the cube-to-owl image was a joke. Modeling is hard for most people (like is coding or painting in 2D or higher math...).

You could still edit the 2D images (textures) that are wrapped around the 3D models. You could maybe not change the geometry, but you could still recolor vanilla stuff like you did for most new stuff in your mods.
And there seem to be modders, that can model in 3D (Angel's stuff looks like done in 3D and rendered onto 2D sprites like Wube does it). So you could also use models contributed by other modders in your mod and concentrate on code and balancing. Specialization is okay. You don't have to be good at everything.

[raidho36]

Try using clay or playdoh or something. Just add and remove chunks of clay until it looks right. Then it's pretty easy to translate that to 2d or 3d, compared to making it up from scratch digitally.

Also, perfection is a pretty high mark. Sometimes you just got to say "eh, that'll have to do" and move on.

[posila]

Mmmh, Vega 64 50% slower than GTX1060 in your render test even though it should be exactly the other way round. Do you know why?

I didn't think anything of it, since what I measured was essentialy glorified mem-copy and the result was in a ballpark of what I expected. Maybe I made a mistake. I shall investigate.

We have updated the blog post.
Update: Several people wondered how come Vega 64 ended up slower than GTX 1060. I originally ran the tests with 60 FPS cap, so I re-ran the tests without the cap and got ~0.04ms on Vega, and ~0.07ms on GTX 1060. So the cards were probably operating in some kind of low-power mode, since they were idle for huge part of the frame. You should still take my measurements with big grain of salt, I didn't attempt to be scientific about it, I just wanted to illustrate huge performance difference between different GPUs people might want to use to play the game.

[bobingabout]

I scribble in paintshop pro. 3D only would mean I wouldn't mod it. There would be no bob's mods in a 3D factorio.

Nothing's stopping you from scribbling in Sketchup 3D. And pretty much all CAD software out there has architecture design tools and can export 3d models.

Besides, 3d modelling is easy. You can make pretty much anything in just two simple steps:
15105055634553.jpg

I've tried. I can't do it. I have Aphantasia. if I can't actually see it, I can't visualise it. I can't even draw very well from scratch either, but if something exists and it doesn't look right, I know it isn't right and can problem fix it.

I am pretty sure, the cube-to-owl image was a joke. Modeling is hard for most people (like is coding or painting in 2D or higher math...).

You could still edit the 2D images (textures) that are wrapped around the 3D models. You could maybe not change the geometry, but you could still recolor vanilla stuff like you did for most new stuff in your mods.
And there seem to be modders, that can model in 3D (Angel's stuff looks like done in 3D and rendered onto 2D sprites like Wube does it). So you could also use models contributed by other modders in your mod and concentrate on code and balancing. Specialization is okay. You don't have to be good at everything.

Although it is true that I don't actually need to make graphics myself, even in 3D... having a 3D engine instantly puts me off wanting to even attempt to mod it at all.

[Oktokolo]

Although it is true that I don't actually need to make graphics myself, even in 3D... having a 3D engine instantly puts me off wanting to even attempt to mod it at all.

That makes as much sense as not modding a game because it has been released on a tuesday.

[eradicator]

Although it is true that I don't actually need to make graphics myself, even in 3D... having a 3D engine instantly puts me off wanting to even attempt to mod it at all.

That makes as much sense as not modding a game because it has been released on a tuesday.

Yea, becaues it is well known that every kid learns to draw 3d pictures with 3d pens on 3d paper in the first year of elementary school arts classes.

Oh wait...wrong reality. In this one we only learn 2d drawing at school. (And now i'm actually looking forward to the argument that claims that past experiences have no influence on future preferences...)

[posila]

My system has serious issues with running high-res sprites on a 1GB card.... I'll try tuning the settings some more.

So one thing is not entirely clear to me: will the new Virtual Textures technique only be used for the low-priority sprites that did not make it into the atlas, or will the entire atlas be replaced with Virtual Textures?
Even 512MB can easily hold a hundred 1024x1024 RGBA textures, so you could cache at least 6400 128x128 tiles at the same time if all VRAM is used for the tile cache. I can't help but think that that should be more than enough for playing with high-res sprites, as only a fraction of the sprites will be in the same scene (unless you build an extremely dense factory). Indeed, if the cached tiles are replaced with low-res versions when zooming out, the increased number of different objects in view will be countered by that.

How much RAM do you have? According to Steam survey, over 90% of our players has 8GB+

So we discussed it a little bit in the office and approach we decided to take was this: "Video memory usage" option should change to have values like 0.5 GB, 1 GB, 2 GB, ... and based on those the game should decide which sprites should be put into non-streamed atlases and which ones into the virtual atlas. For 0.5 GB value I can imagine everything would be streamed, or everything except for GUI related graphics. And for something like 4 GB, nothing would be streamed (in vanilla). The value should also include offscreen framebuffers, so if you set 0.5 GB it doesn't necessarily mean all 0.5 GB will be used for sprites.

My estimate may be totally off though I tried to verify this with debug info on, but could not find data on that. Would it be worth the effort to add counters for the number of different sprites used during the drawing of a frame, and the total pixel area of all these combined? That might indicate whether it should be considered replacing the entire atlas with Virtual Textures.

I added something like this while starting with virtual texture mapping just to figure out how big portion of atlas do we need to render a frame. It was usually 10% or less, maximum was about 20% when I zoomed out on huge biter base with lots of biters and spitters.

Disadvantage of replacing entire atlas system with streaming, is that then all sprites would have to be in RAM (unless streaming from HDD works well), which is waste of RAM is you have GPU with large VRAM (this could be solved by storing part of the virtual atlas in VRAM though). Also at the moment we don't want people who have enough VRAM to pay for performance overhead of the streaming (which might be on the other hand quite minimized if no-overhead when not streaming wouldn't be required)

[meganothing]

We have updated the blog post.
Update: Several people wondered how come Vega 64 ended up slower than GTX 1060. I originally ran the tests with 60 FPS cap, so I re-ran the tests without the cap and got ~0.04ms on Vega, and ~0.07ms on GTX 1060. So the cards were probably operating in some kind of low-power mode, since they were idle for huge part of the frame. You should still take my measurements with big grain of salt, I didn't attempt to be scientific about it, I just wanted to illustrate huge performance difference between different GPUs people might want to use to play the game.

Many thanks. Since the new numbers are much lower, your illustration of huge performance differences is even more effective now

[Mike5000]

How much RAM do you have? According to Steam survey, over 90% of our players has 8GB+

This reminds me of an observation.

I have 3GB RAM and 128MB VRAM Lenovo/Nvidia laptop running Linux - pretty cool when I bought it in 2007 but overdue for replacement. Nevertheless Factorio runs acceptably for non-mega bases provided I first close Firefox and except for slowdowns when running near trees.

However I noticed that sometimes after a long Factorio session there is a minute of intense paging when quitting Factorio. This suggests the possibility that stuff is being kept in RAM throughout the game which is little used or possibly unused, and that it is deallocated in some expensive fashion on termination.

[bobingabout]

According to Steam survey, over 90% of our players has 8GB+

I think the main reason for this is... have you even TRIED running windows 10 on 4GB or less? it just doesn't do it well at all.
so anyone with a reasonably new computer will have 8GB+ because anything less is just a shitbox of a computer.

to note: my previous gaming computer has a 6GB RAM. it had 8GB, but 1 stick had a faulty byte and it was causing issues, so I removed it. It's DDR2, max stick size is 2GB.

[raidho36]

Just because you have a lot of RAM available, doens't mean you can be sloppy with RAM usage. Because that would mean huge memory bandwidth and compared to CPU power it's been getting worse and worse since the 70s. The memory was given name "random access" to accentuate the fact that you didn't need to access it in sequential fashion like a magnetic tape to get peak performance. Nowdays you need to specifically design your program to access memory in sequential fashion, because every time you try to address a random location, the entire process will stall for several hundreds of cycles. Bigger memory bank basically just means that you don't need to access the hard drive as often, since larger memory means increased likelyhood that the relevant data is already loaded.

That said, 8 gigs is already beyond huge. The only way you can use it up is if you're being incredibly wasteful. Go on, find a good use for every single of 2 000 000 000 32-bit numbers you could store in 8 gigs of memory, I dare you.