Friday Facts #333 - Terrain scrolling

[Klonan]

I would like to go back to previous FFF where we discussed new sounds and there were many critical voices. Would you mind commenting on that?

We got a lot of feedback, and will take our time digesting it and working on it, aswell as fixing all the issues and bugs that have been found since the new sounds were introduced.

[blackbat]

I would like to go back to previous FFF where we discussed new sounds and there were many critical voices. Would you mind commenting on that?

We got a lot of feedback, and will take our time digesting it and working on it, aswell as fixing all the issues and bugs that have been found since the new sounds were introduced.

Now, there's a reply I appreciate and respect.
Hear the feedback and think it over, don't react immediately, and make sure not to make it work.
Great job guys!

[bobingabout]

Your new method reminds me of how a NES/SNES works.

Heck, most of the old consoles of that era use that kind of screen drawing.

So... if thats how they used to do it, it makes sense that it would be more efficient.

[AshenSwift]

I would like to go back to previous FFF where we discussed new sounds and there were many critical voices. Would you mind commenting on that?

We got a lot of feedback, and will take our time digesting it and working on it, aswell as fixing all the issues and bugs that have been found since the new sounds were introduced.

This is a sign of a good game dev. Other devs say this and it means, "We don't care." Factorio game devs say it, and you know they're actually looking at it and working on it.

[bw_mutley]

...and then instead of copying all the terrain back to the buffer, we can just adjust the offset and update the parts that changed.

Just like 90% of video files do. When I started reading this FFF, this question just pop up in my eyes: Why they don't do just like MP4 files?

Still, Factorio video rendering where still better than a lot of 2D games, taking into consideration the properties carried over each tile.

[valneq]

...and then instead of copying all the terrain back to the buffer, we can just adjust the offset and update the parts that changed.

Just like 90% of video files do. When I started reading this FFF, this question just pop up in my eyes: Why they don't do just like MP4 files?

The video compression algorithms used in MP4 use lossy compression: they throw away some information in order to be able to strongly compress the remaining information. They especially exploit how our vision works: individual pixels being slightly wrong don't bother our visual system that much. Additionally, many videos have larger areas that have similar colors for an extended amount of time – allowing for large scale comparison of many pixels in the same frame and even across several consecutive frames in the video stream. This compression helps in situation when you want to store video data on hard disks, or transmit them through data channels with a limited bandwidth. But both the compression and later decompression for rendering on screen require computation time.

With that in mind, it should become clear that this is absolutely not what a video game rendering engine should be doing. In a video game, you want to update the information for the screen as quickly as possible, and not lose any time on analyzing all the pixels in the generated frame to figure out how to compress all that. After all, the connection between graphics card and screen is designed for direct output of uncompressed video data.

In this week's FFF, the topic is figuring out how to minimize the number of changes that need to be done on the video buffer before it is being drawn on the screen. For terrain this is especially efficient because the terrain does not change when you walk around. What changes is which part of the terrain should be drawn on the screen. The parts that don't change in between frames can stay exactly where they are in the video buffer. You only need to take care of which pixels in the video buffer should be drawn where onto the screen.

[maximm]

I was playing with NEE, AAI, Space Exploration and all mods they depend on (namely Alien Biomes), all in HR. When zoomed out, I've seen FPS drops down to 30
And now, this optimization brought my FPS back to 60 when you zoom out (i7-6600k with 1070 GTX and 32 GB RAM)

Thank you for this improvement.

[leadraven]

All ingenious is simple. Really impressive optimization.

[liwers]

So uh..

What's this optimization may really enhance on Kitty's computer, as it's i9-9900K+Z-390E+2080Ti..

Meow..!

[eradicator]

@posila:
Did you measure the difference on high end cards? Even if it's "only" 0.1ms that's still some extra SPM before dropping below 60 UPS.
_________

I've heard from so many people with self-described "potato" computers who are so happy they can play Factorio

Yea, the poor potato people. I'm so happy that i have a proper Intel HD Graphics 4000 toaster.
_________

...and then instead of copying all the terrain back to the buffer, we can just adjust the offset and update the parts that changed.

Just like 90% of video files do. When I started reading this FFF, this question just pop up in my eyes: Why they don't do just like MP4 files?

The video compression algorithms used in MP4 use lossy compression: they throw away some information in order to be able to strongly compress the remaining information.

Lossyness isn't really the problem. The problem that factorio needs to solve is a movement of *all* screen pixels at the same time. This is something that video compression is infact quite bad at because it is optimized for small localized changes. I.e. if you watch the news then most of the time the only different between two frames would be the mouth of the speaker and all the other pixels would simply stay the same.

[mrudat]

Depending on how complicated the tile transition rules are, could you not have a texture with 1 pixel/tile that stored the tile variation number, then have a shader use that to index into the tile texture during rendering? More or less a sharp-edged displacment map.

I suspect that it might go faster, if only for moving less bits, on the other hand, there's less locality of reference, so it might go slower?

[eradicator]

Depending on how complicated the tile transition rules are

Alien Biomes hits the tile limit of 255 tile types. So if all of these can transition to every other tile in eight directions that's about half a million possible tile combinations i think (≈ 8*255^2)?

[DoubleThought]

Alien Biomes hits the tile limit of 255 tile types. So if all of these can transition to every other tile in eight directions that's about half a million possible tile combinations i think (≈ 8*255^2)?

Each of the eight adjacent tiles can be any of 255 tile types. The central tile can also be any of 255 tile types. Assuming that all nine of these tiles contribute to the tile transition, that's a total of 255^9 = 4,558,916,353,692,287,109,375 (4 sextillion or 4 * 10^21).

So, a bit worse than half a million tile combinations.

It's this exponential growth that would require that the transitions are programmatically generated as needed. If all of the images were prerendered, and each image took a millisecond to generate (a mild overestimation), it would take a whooping 144,466,308,206 (144 billion) years to pretender all the tile transitions, not to mention the frankly absurd amount of memory required. For comparison, the Earth is estimated at a mere 4,540,000,000 (4.54 billion) years of age, less than a third of how long it would take to prerender each tile transition.

On the other end of the scale, a modern GPU has a clock rate of around 1000 MHz. A tile has around 1000 pixels. Assuming rendering each pixel takes a single clock cycle (a mild underestimation), a tile would take a microsecond to render. With this estimation, it would take only 144,466,308 (144 million) years to render the tile transitions. This is approximately how long ago the Cretaceous period started (144 million years ago).

It's worse if there's an additional built-in no-tile type, which brings the total number of tile transitions to 256^9; there would be an additional 163 quintillion images needed to be rendered.

Prerendering all of tile transitions would take between the age of Earth to the age of dinosaurs.

Luckily for us, the Factorio devs are amazing; otherwise, starting up Factorio with Alien Biomes could take longer than it would take for the sun to scour all life from earth (7.5 billion years).

[conn11]

.
Luckily for us, the Factorio devs are amazing; otherwise, starting up Factorio with Alien Biomes could take longer than it would take for the sun to scour all life from earth (7.5 billion years).

This should actually happen much faster. The sun is slowly but continuously increasing in luminosity and therefore believed to shift earth out of the habitable zone and more into Venus territory in about a billion years. Meaning no liquid water, no life as we know it.
Luckily for all of us, Factorio is, even with a more constrained timeframe, efficient enough to start with Alien Biomes long before the surrounding landscape becomes an alien biome.

[daifeiya]

Thank you so much for this optimization! Don't think this is less than 1 millisecond optimization is insignificant, this is a significant improvement for the super factory! Thank you so much!

[BattleFluffy]

I would like to go back to previous FFF where we discussed new sounds and there were many critical voices. Would you mind commenting on that?

We got a lot of feedback, and will take our time digesting it and working on it, aswell as fixing all the issues and bugs that have been found since the new sounds were introduced.

Now, there's a reply I appreciate and respect.
Hear the feedback and think it over, don't react immediately, and make sure not to make it work.
Great job guys!

Agreed, this is the right approach and I'm really glad to hear this after a week or two of silence on the subject. :>

The optimization to terrain scrolling is great! I have to confess I don't quite follow how it's working, but the benchmark time doesn't lie.. :>

[posila]

I am pleasantly surprised by all the positive feedback on this FFF, thank you.

I feel like I need to clear some things up - the described optimization affects only GPU. GPU is already working in parallel to CPU. And in Factorio, render thread (which sends commands to GPU) runs in parallel to update thread. So if you have dedicated GPU, this has no benefit on UPS whatsoever. There might be some UPS improvement on integrated GPUs, which live on the same die as CPU, and they compete for resources, but I have not tried to measure that.

[fiery_salmon]

Thank you for making possible to play this game also on a bit older hardware (like my 13 year old laptop). It is result of actually caring about it and many optimizations like this one.

[MakeItGraphic]

It's funny, I bought my laptop 3 weeks ago, it is old 2013 and was over a few hundred. Anyway I actually burnt it out (red lining the hardware from over heat) playing this game ahahaa. Nothing to do with the game/developers, probably too many mods and what not. And I know .18 runs a lot better.

But it's pretty amazing what this game can run on, before 64bit I was using my last computer which was windows XP/ubuntu 512kb DDR2 x 2 Pentium 4 and it was the only modern game that would actually run on the damn thing. It saved me a lot of nights of boredom, and pain. You can just turn on this game zone out for 12 hours at a time, takes your mind off everything.

Absolute god send, and it's great to see the continuation of effort to optimize this game in a equal opportunity way for all players regardless of their budget for hardware. Being from a background of low income you're not using modern hardware, and often it is what it is. Unfortunately with that a lot of developers have vied out from optimizing their software with older hardware, which is fine and understandable economically speaking. However pride of ownership should still be a thing. Irregardless if we have Intel Core i9-9900KS, or DDR4 hitting the 5,000MHz barrier and the standard amount of ram now is nearing 16gbs. Not everyone is going to be running that, so just saying oh it's only using 500mb of ram to load a webpage I mean it's a lot. I've configured windows 7 even to clock under 400mb from desktop. Apples and oranges.

It's just nice to be able to follow a company with efficiency, and clean/optimized coding practices in mind.

[drgarantia]

I bought this game a little over two years ago, and I get amazed how great this game gets every single patch since then .

About a year ago I got a 4k monitor and as expected, at the time, my 750ti had a really hard time pushing all those pixels at anything over 40ish fps in the best case, then the texture compression patch came in and since them I get 60 frames at 4k. It is just amazing how well this game looks and performs now compared to when I bought it, and it was already very well optimized compared to most other "2Dish" games!

Thanks for the awesome job.