Image Optimization (I cut 59% of graphics folders file size)

[miLL3rTime]

I am sure I am not the first person to have thought of this, but I was unable to find any discussion about it here. I originally started looking at the graphic folders (one in core and one in base) to see how things were organized and whatnot. I took an image and threw it into an Alfred workflow that uses a few optimization algorithms and found that it's file size decreased significantly. I should note that you may be asking if this optimization is lossy, technically yes, visually, I would pay you to find a difference in visual quality. Here is a link to the Alfred workflow I used: packaldotorg/workflow/image-optimization

I proceeded to run all 2,269 image files through and batch process them. When done, graphics folder's file size decreased 59%! See screenshot of folder info below.

Now, I wasn't able to do a controlled experiment for performance implications, but I personally got a 5FPS boost on a dumb little factory that I have. If someone else has a mega factory with tons of action and moving robots and production I would be happy to do some testing. 5FPS increase is pretty significant for a little factory.

At any rate, I have provided links to some sprites so you can see the difference between original and optimized. If you have a specific request I can upload different ones. I chose to provide copper ore, roboport door, and coal. Obviously the alpha channel has been preserved with all images.

Developers, with the announcement of the games release sometime this year, was image optimization something you had considered doing at the very end? - Or is this something new? I would love to know what sort of performance boost can be achieved by the 59% cut in file size. I am no expert, so it could very well be a moot point, but until I am informed otherwise I would find that hard to believe.

Please let me know if you have any questions, and also please let me know if you think this would help with performance.

Thanks!

Links:
I am sure I am not the first person to have thought of this, but I was unable to find any discussion about it here. I originally started looking at the graphic folders (one in core and one in base) to see how things were organized and whatnot. I took an image and threw it into an Alfred workflow that uses a few optimization algorithms and found that it's file size decreased significantly. I should note that you may be asking if this optimization is lossy, technically yes, visually, I would pay you to find a difference in visual quality. Here is a link to the Alfred workflow I used: http://www.packal.org/workflow/image-optimization

I proceeded to run all 2,269 image files through and batch process them. When done, graphics folder's file size decreased 59%! See screenshot of folder info below.

Now, I wasn't able to do a controlled experiment for performance implications, but I personally got a 5FPS boost on a dumb little factory that I have. If someone else has a mega factory with tons of action and moving robots and production I would be happy to do some testing. 5FPS increase is pretty significant for a little factory.

At any rate, I have provided links to some sprites so you can see the difference between original and optimized. If you have a specific request I can upload different ones. I chose to provide copper ore, roboport door, and coal. Obviously the alpha channel has been preserved with all images.

Developers, with the announcement of the games release sometime this year, was image optimization something you had considered doing at the very end? - Or is this something new? I would love to know what sort of performance boost can be achieved by the 59% cut in file size. I am no expert, so it could very well be a moot point, but until I am informed otherwise I would find that hard to believe.

Please let me know if you have any questions, and also please let me know if you think this would help with performance.

Thanks!

Links:

[sillyfly]

If I understand sprite-based graphics correctly (and this may be entirely wrong!), the graphics are basically converted to bitmaps and saved as such in the VRAM, so the size the game engine/graphics engine sees is entirely dependent upon the actual size (in pixels) of the image. This means there shouldn't be any in-game performance boost, so I tend to doubt the improvement was due to these changes - at least until it can be reproduced and verified to rule out any other possible causes.

The savings in file sizes are probably not too critical - when downloading the files are compressed, which probably gives similar savings. It may help in the loading times (when converting the image files to atlas bitmaps), but then again it may not or may even hurt it - if it is mostly a performance (CPU/GPU) bound task, rather than I/O bound.

All in all it's an interesting experiment, but unless I completely misunderstand sprite-based graphics (which I concede is absolutely possible!) it probably doesn't have any real value for Factorio.

[mrvn]

Afaik bitmaps can be compressed or send compressed over the bus.

[mp0011]

The problem in Factorio, with under 2GB o VRAM is quantity of textures/variants.

Are You able to reduce number of animation frames to 1/2 or 1/4?
If I have to choose to keey 60fps/60UPS with choppy boiler animation vs 30FPS/UPS with full animation, the answear is obvious.
Boiler (for example) has 32 animation frames. It may have 16 or 8 for older machines.

Also, can You reduce train/car/tank/enemies/player/player weapon rotations to 1/2 or 1/4?

It could be usefull for someone as well...
My problem with Factorio was not the quality of textures, but constant swapping of them, making my GPU to run at 0%, but lagging to 20FPS...

[mrvn]

The problem in Factorio, with under 2GB o VRAM is quantity of textures/variants.

Are You able to reduce number of animation frames to 1/2 or 1/4?
If I have to choose to keey 60fps/60UPS with choppy boiler animation vs 30FPS/UPS with full animation, the answear is obvious.
Boiler (for example) has 32 animation frames. It may have 16 or 8 for older machines.

Also, can You reduce train/car/tank/enemies/player/player weapon rotations to 1/2 or 1/4?

It could be usefull for someone as well...
My problem with Factorio was not the quality of textures, but constant swapping of them, making my GPU to run at 0%, but lagging to 20FPS...

Maybe one could cut down the number of frames when not enough VRAM is available but offset that by drawing intermediate frames that are 50% of the previous texture and 50% of the next. Blend the two images. Obviously this would increase the GPU usage but if the problem is not enough VRAM that might not be a problem.

[mp0011]

Do not blend between animation frames. Just, skip them and repeat 2 or 4 times. So instead playing 32 frames animation in 32 ticks, You play 8 frames, 4 tick each.

[5thHorseman]

Beta is not the time to optimize graphics, as they could change several times before release.

[Rseding91]

Runtime performance is all about the total sprite resolution of the loaded textures, the amount of VRAM you have and the max atlas texture size your system supports. In the ideal setup you load all sprites into 1 giant atlas in VRAM and you never need to swap textures when rendering.

If you use texture compression, mipmapping, trilinear filtering, any VRAM usage option that isn't "all" or your system doesn't support 16k atlases then they get split up into multiple atlases and the game has to switch between them as it renders.

Compressing them on disk doesn't change any of that. It would just reduce runtime quality by what ever information is lost in the lossy format.

[kreatious]

I ran optipng, a program that tries out lots of different lossless compression algorithms.

And it made the graphics folder a whopping 0.2% smaller.

Results

124652 KiB - original
124636 KiB - no bit depth, palette, or color reductions (16 KiB saved)
124400 KiB - w/ reductions (252 KiB)

Code: Select all

// C#
// script takes 15 minutes to finish
using System.Threading;

var semaphore = new SemaphoreSlim(Environment.ProcessorCount);
await Task.WhenAll(Directory.EnumerateFiles("C:/Users/kreatious/factorio-temp/graphics", "*.png", SearchOption.AllDirectories)
    .Select(async png =>
    {
        await semaphore.WaitAsync();
    
        var taskCompletionSource = new TaskCompletionSource<object>();
        var process = new Process
        {
            StartInfo =
            {
                FileName = "C:/Users/kreatious/Downloads/optipng.exe",
                Arguments = $"-o7 -clobber -quiet \"{png}\"",
                UseShellExecute = false
            },
            EnableRaisingEvents = true
        };
        process.Exited += (sender, args) =>
        {
            taskCompletionSource.SetResult(null);
            process.Dispose();
            semaphore.Release();
        };
        process.Start();
        return taskCompletionSource.Task;
    }));