Any game-dev that could answer the stackexchange question?

[kovarex]

This is the question:
http://gamedev.stackexchange.com/questi ... in-2d-game

[Tyrithe]

Registered to comment, Hi

I read through the site you linked. I'm not a game dev but I was wondering why you can't just use max zoomed out area + 2-3 chunks round the edge(that you are loading information for) to tell you what exactly you need cached.

To be honest, I don't really understand why using the textures from two 8192x8192 files is faster than using several smaller files, Is this some kind of strange limitation to allegro? If all the files are loaded into memory anyway they should have the same access speed right?

Sorry if these are rather obvious but I'm not a programmer.

I'd post more but unfortunately I have to go work now.

[muzzy]

Although I don't have an answer to your question, I have some other random thoughts to reduce memory requirements. Basically, I just went through the graphics folder looking at each image and writing down my thoughts:

Use shaders to produce some details that currently exist in the bitmaps. For example, simplify the water textures significantly and make a water shader.

Shaders for dirt too, maybe use grayscale bitmaps for displacement/bump effects on top of a smaller conservative set of ground textures to create variations.

Implement grass with multiple layers. Simple ground layer on bottom and then transparent grass on top, create dry/medium/normal variations by altering opacity of the grass layer. One more layer for extra foliage perhaps, the larger things that are similar in all grass variation textures.

Instead of the inner/outer corner transparent bitmaps per terrain, have grayscale masks and use them instead. Less memory + potential for reuse.

Partially replace splash animations with particle systems, extra clutter away to minimize bounding boxes.

Write a script to analyze things like assembly machines to find which portion of the animation actually animates, then combine the assembly machine as a base sprite and animation with a tighter box. It will cut perhaps 50% of the assembly machine 1 bitmap data away, 20% for assembly machine 2, maybe 5% of assembly machine 3 as well (the left edge).

Separate shadows into different bitmaps, they're 1bpp data and there might be more efficient way to render that and you'd significantly cut bounding boxes of high structures such as the beacon.

Biters ... now these are seriously huge ... Separate the death splatters into different bitmaps without the rotation, should make bounding boxes smaller for the actual biter parts.

Consider a hybrid 2d/3d engine such as the one used by Total Annihilation back in '97, where ground textures were bitmaps and units were 3d models. Since you're using 16-way rotation for enemies right now, the bitmap requirements for adding more enemy types would end up being outright gigantic. Rendering everything else as 2d but enemies and weapon turrets as 3d might be a viable choice if you cannot find any other way around the issue.

The car would benefit even more from a hybrid 3d renderer since it only has 64-way directional frames and no animation, same for locomotive and etc.

Shader to make color variations would help reduce footprint of different splitters since they all look the same except for one color. It doesn't actually even need a shader for the splitters, just make the colorless animation the base and then draw the colored front piece on top of it.

Oh dear, I just opened the player textures. The poses differ so much there aren't many 2d choices here. If you aren't going for 3d rendering of the player, at least go for variable speed animations. There are many redundant frames that could be pruned in mining. There are just too damn many frames in these animations. The "mine by hand" animation is something you only see in the beginning of the game and almost never again, it's a complete waste of VRAM.

Consider splitting the character into separate parts; head, torso, legs, arms. Combine these to cut idle animations so you don't waste VRAM when legs don't move between two frames. You'll need to write a tool for authoring the animations though, because it'll become a pain in the ass to manually define which leg frame goes for which animation frame.

Also, simplify the character shadows. Nobody cares, especially since I only just now noticed that the character shadow rotates so it points in the wrong direction, plus it's rendered as gray instead of black and doesn't feel like it's even blended... nobody will miss the player shadow, this will cut the bounding boxes smaller.

[kovarex]

To be honest, I don't really understand why using the textures from two 8192x8192 files is faster than using several smaller files, Is this some kind of strange limitation to allegro? If all the files are loaded into memory anyway they should have the same access speed right?

I have no deep understanding of other engines, so I don't know if it is only allegro related. But in allegro, it slows down, when you draw from different source bitmap than the last time. It is related to some overhead related to switching contexts.

[kovarex]

Use shaders to produce some details that currently exist in the bitmaps. For example, simplify the water textures significantly and make a water shader.

Shaders for dirt too, maybe use grayscale bitmaps for displacement/bump effects on top of a smaller conservative set of ground textures to create variations.

Implement grass with multiple layers. Simple ground layer on bottom and then transparent grass on top, create dry/medium/normal variations by altering opacity of the grass layer. One more layer for extra foliage perhaps, the larger things that are similar in all grass variation textures.

Instead of the inner/outer corner transparent bitmaps per terrain, have grayscale masks and use them instead. Less memory + potential for reuse.

The terrain doesn't take much memory and the transitions are custom, the water/grass is the most visible example.

Partially replace splash animations with particle systems, extra clutter away to minimize bounding boxes.

This is certainly planned, we have procedural particle-based animation in the current 0.10 WIP version, and not only it saves space, it also is looking much better, and every explosion is different.

Write a script to analyze things like assembly machines to find which portion of the animation actually animates, then combine the assembly machine as a base sprite and animation with a tighter box. It will cut perhaps 50% of the assembly machine 1 bitmap data away, 20% for assembly machine 2, maybe 5% of assembly machine 3 as well (the left edge).

Yes we will do such things more in the future. We did this for roboport, in previous version, the whole roboport was animation, now just the small moving part is animation.

Separate shadows into different bitmaps, they're 1bpp data and there might be more efficient way to render that and you'd significantly cut bounding boxes of high structures such as the beacon.

This is certainly the most interesting idea. The 1bpp and smaller bounding boxes could save a lot in some cases. All I have to do is to find out, if there are any performance penalties when drawing 1bpp bitmap to 32bpp surface.

Biters ... now these are seriously huge ... Separate the death splatters into different bitmaps without the rotation, should make bounding boxes smaller for the actual biter parts.

Yes, we plan to do something along these lines.

Consider a hybrid 2d/3d engine such as the one used by Total Annihilation back in '97, where ground textures were bitmaps and units were 3d models. Since you're using 16-way rotation for enemies right now, the bitmap requirements for adding more enemy types would end up being outright gigantic. Rendering everything else as 2d but enemies and weapon turrets as 3d might be a viable choice if you cannot find any other way around the issue.

The car would benefit even more from a hybrid 3d renderer since it only has 64-way directional frames and no animation, same for locomotive and etc.

I was condidering this as well, but the problem is not only that we would have to probably change the graphics engine to make this hybrid possible. 3d render modelling would probably not integrate well with the cycles based (raytracing) modelling of our prerendered pictures.

Shader to make color variations would help reduce footprint of different splitters since they all look the same except for one color. It doesn't actually even need a shader for the splitters, just make the colorless animation the base and then draw the colored front piece on top of it.

Yes, this is planned for the player as well, as player will need to have adjustable colors.

Oh dear, I just opened the player textures. The poses differ so much there aren't many 2d choices here. If you aren't going for 3d rendering of the player, at least go for variable speed animations. There are many redundant frames that could be pruned in mining. There are just too damn many frames in these animations. The "mine by hand" animation is something you only see in the beginning of the game and almost never again, it's a complete waste of VRAM.

That is certainly possible, we might remove every odd frame for some particular animations.

Consider splitting the character into separate parts; head, torso, legs, arms. Combine these to cut idle animations so you don't waste VRAM when legs don't move between two frames. You'll need to write a tool for authoring the animations though, because it'll become a pain in the ass to manually define which leg frame goes for which animation frame.

Also, simplify the character shadows. Nobody cares, especially since I only just now noticed that the character shadow rotates so it points in the wrong direction, plus it's rendered as gray instead of black and doesn't feel like it's even blended... nobody will miss the player shadow, this will cut the bounding boxes smaller.

The character will be completely redone in the future anyway, so we will try to find different solution during that process.


Anyway thank you for the info, I appreciate that.
There are 2 things we need to done 1 is to try to save the consumption of individual entities, which is always good and 2 is to enlarge the raw possibilities of the game engine.

[Drury]

Separate shadows into different bitmaps, they're 1bpp data and there might be more efficient way to render that and you'd significantly cut bounding boxes of high structures such as the beacon.

This is certainly the most interesting idea. The 1bpp and smaller bounding boxes could save a lot in some cases. All I have to do is to find out, if there are any performance penalties when drawing 1bpp bitmap to 32bpp surface.

Would this make dynamic shadows possible? The way I imagine it the separated shadow sprites would warp (via post-processing) depending on light sources present (each one of which would spawn a new shadow sprite, so each light source could cast it's own shadow). Also, if I'm not mistaken, the light sources themselves can be handled via post-processing rather than spritemasks (which might save that teeny bit of memory as well).

[kovarex]

Separate shadows into different bitmaps, they're 1bpp data and there might be more efficient way to render that and you'd significantly cut bounding boxes of high structures such as the beacon.

This is certainly the most interesting idea. The 1bpp and smaller bounding boxes could save a lot in some cases. All I have to do is to find out, if there are any performance penalties when drawing 1bpp bitmap to 32bpp surface.

Would this make dynamic shadows possible? The way I imagine it the separated shadow sprites would warp (via post-processing) depending on light sources present (each one of which would spawn a new shadow sprite, so each light source could cast it's own shadow). Also, if I'm not mistaken, the light sources themselves can be handled via post-processing rather than spritemasks (which might save that teeny bit of memory as well).

It could and it couldn't.

The problem is, that the shadows look different depending on the angle of the source. (Imagine shadow of cargo wagon for example from top vs from side).
But the truth is, that it doesn't have to be super precise and player would probably not notice. This would allow us to have effects, like shadows away from explosion and such ...

[Drury]

It's not super-realistic at this point either and you don't hear anyone complaining

Well, za pokus nič nedáš.

[muzzy]

Separate shadows into different bitmaps, they're 1bpp data and there might be more efficient way to render that and you'd significantly cut bounding boxes of high structures such as the beacon.

This is certainly the most interesting idea. The 1bpp and smaller bounding boxes could save a lot in some cases. All I have to do is to find out, if there are any performance penalties when drawing 1bpp bitmap to 32bpp surface.

You could create a new 1bpp texture and render shadows into it to act as a screenspace shadow map. You could then have a shader for terrain rendering that uses the shadow texture to bake the shadows in during rendering, or do it in two passes much like traditional shadow mapping technique.

Consider a hybrid 2d/3d engine such as the one used by Total Annihilation back in '97, where ground textures were bitmaps and units were 3d models.

I was condidering this as well, but the problem is not only that we would have to probably change the graphics engine to make this hybrid possible. 3d render modelling would probably not integrate well with the cycles based (raytracing) modelling of our prerendered pictures.

It will indeed be a challenge to match the visual styles, but it shouldn't be impossible to make it look reasonably nice.

Rewriting the graphics engine might be necessary if you aren't using the opengl approach for 2d rendering currently. If you are using opengl already, you can probably set up a projection that works for both 2d and isometric 3d rendering at the same time. Just have X and Y map to screenspace directly and project Z axis to 2/3 Y, then draw tiles as 1.5 longer along z-axis to make it appear square in screenspace but to get proper depth buffer values. Or something, maybe it'd be better to have separate pass for the gui layer with different projection afterall. This is making my head hurt already, I'll just stop pretending to know anything about graphics programming now.

[Tyrithe]

Honestly the only way I can think of reducing memory usage is by having less 'clutter'. But that's not a solution in any way at all. That or a 'simple' mode that doesn't load alternate versions of tiles, In case you have an lower vram cap.

Will zooming out more put enough strain on GPU usage that a graphics card that doesn't have more than 512mb VRAM won't be able to handle it?

I'm using a geforce 760, I'm at about 40% at game.player.zoom = 0.1(Not that you can even see the player). About 30-35% otherwise, I imagine about 5% of that is from background apps as I'm in windowed mode.
This is with 0 activity no biters running around no belts moving items and so on.
Vram usage is about 560mb~, 140mb of that used by system.

I'm only guessing that you want to have the game playable on older graphics cards and/or computers though.

Anyway.. I hope you find a solution. I find these sorts of things pretty interesting.

[ssilk]

If you don't count it, it doesn't exist.

Create a statistic about the usage of every stupid bitplane. For example: every frame add 1 to a counter of each used bitplane.

Now that stats you make per second enables you to see how probable it is, that a bitplane is used.

If you write the stats for every second into a database, you can create very good statistics. The median/quantile usage of a bitplane, normal distribution. The "worth" : multiply with the number of pixels it uses. Very seldom usages. Very high usages.

From the normal distribution you can guess: if I need this bitplane, I need also that. Reduce the randomness.

I know that performance is not the problem yet, but it gives you a better understanding of the process and maybe a new idea. I made that kind of statistics one time for SQL-queries on a database with more than 30,000 queries/second and found out, that the cache-locking slowed it down. After shutting off the cache, it made 60,000 q/s. I spared a whole server with a simple change of a runtime-option. Such ideas can only be found if you have the stats.

I know this is no fast solution, but I fear, that for this problem there is no fast solution. But having numbers gives you some kind of safety what kind of solution really can bring you forward. This is especially then useful, if you are not sure and /or trying out don't gives you the guarantee to have the best solution.

[Darthlawsuit]

States tend to move from 1 to lets say 15 then back to state 1 with a possible state 0 where its destroyed. Is it possible to store only what changes from one state to the next?

I am curious. If I am standing in Factory A and Factory B is a 5 minute train ride from factory A are the textures for factory B being processed or do you only process textures X distance from your character?

[raidho36]

Only models that show up on the screen are rendered, so no.

The question was about memory management, because with large resolution animated sprites it's quickly getting out of hand. Not only you are limited with how many textures can GPU store at a time, you're also limited with how often you can switch texture to draw another object, hence use of atlases where you can draw multiple different sprites without switching a texture. Not to mention, loading data to GPU takes a while. Larger atlas means less texture changes, but at the same time it's harder to manage memory carefully because you must keep the atlas in memory if you have to draw any sprite from it. This is why it's a complicated issue and there's no obvious solutions.

One suggested using multiple smaller atlases. The other suggested building atlas in run-time dynamically. The problems with those solutions are readily obvious. The other guys suggested to use animation diff-maps to crop sprites, use pallettes, and use lower color depth textures to save memory directly, but at expense of quality or performance.

The core issue is that there may not be enough GPU RAM to hold all of your sprites. And no matter what you do, you may still face the situation where you have to render all of them. Any memory management methods would only delay that moment, but with or without, once it happens, the game will slow down to a crawl - if it doesn't crash.

What I suggest is keeping track of used video memory, and when it's past some value, then only start uploading compressed graphics to the GPU, with DXT compression and maybe even lower resolution if memory is exceptionally short, while also unloading "full" graphics and replacing them with compressed graphics as well in a separate thread. It's largely the same as previously suggested use of lower color depths, except you don't always have to use low fidelity graphics. It will not produce best looking visuals under stress, but it beats having the game running at a turn based strategy pace, let alone simply crashing on you randomly.

Whichever method you stick to, however, you can always get away with keeping DXT-compressed low resolution minimally-/non-animated graphics at all times so even if you fail to fetch the proper graphics, something will be displayed and it will even resemble the sprite you were going to render.

[Smarty]

by Darthlawsuit » 25 Apr 2014, 06:55


thank you

[raidho36]

Eh whatever, man. Unanswered questions have no expiration date.