Page 1 of 1

Save file size: Use 7zip instead of zip

Posted: Mon Jul 04, 2016 11:17 am
by DjTeo
I wont suggest to use 7zip that will take time to zip-unzip, just use zip with high compression even 1-2 mb smaller size than a 11mb zip file its better, cause I dont have good Connection for upload, but my PC its high-end enough. Best will be to have option for that or a 7zip compression in the sending file in Multiplayer, so the 11mb file will be 7.6mb :D

Re: Save file size: Use 7zip instead of zip

Posted: Mon Jul 04, 2016 10:16 pm
by Zaflis
With my quick test at least 18mb zip was reduced to 13mb. But packing that 7zip took 20 seconds, whereas zip just 1.

Re: Save file size: Use 7zip instead of zip

Posted: Tue Jul 05, 2016 9:18 am
by bobingabout
Okay, and in the case where your zip compression only took 1 second, what happens if you set the compression ratio to maximum? The default is somewhere in the middle, if 7Zip took linger, then the better compression might simply just be down to the default being maximum compression, unlike zip.

Re: Save file size: Use 7zip instead of zip

Posted: Tue Jul 05, 2016 9:33 am
by Zaflis
Some more tests with different compressions:
7-zip normal took 19 seconds with 9.9mb, and fastest 3 seconds for 15mb.
Zip normal took 5 seconds for 15mb, and fastest 1 second for 16mb.
Factorio had it 18mb zip (i uncompressed the contents for the test, not compressing a compressed file again).

Re: Save file size: Use 7zip instead of zip

Posted: Tue Jul 05, 2016 11:05 am
by ratchetfreak
frankly a domain-specific compression format would be better, it would allow the devs to eliminate redundant information they know is redundant instead of having the general compression library discover it on each save.

I'm pretty sure a 50% reduction over the 10mb is possible.

Re: Save file size: Use 7zip instead of zip

Posted: Tue Jul 05, 2016 11:52 am
by ssilk
IMHO - to be it worth to think about it - the reduction should be much more than 50%. It's not worth discussing if we spare 5 MB, when normal harddisks have 1.000.000 MB. Let's say only 50% of such a disk is used, then the saving of 5 MB is equivalent to 0.01 Promille.
So much more important (currently) is the saving of time. But that can also not be compared, cause under normal circumstances the compression is handled in a second thread, the limiting factor currently is the RAM- and CPU-speed, which is needed to copy and serialize the world. The CPU which makes compression will be bored, cause of that.

Re: Save file size: Use 7zip instead of zip

Posted: Wed Jul 06, 2016 9:54 am
by ratchetfreak
ssilk wrote:IMHO - to be it worth to think about it - the reduction should be much more than 50%. It's not worth discussing if we spare 5 MB, when normal harddisks have 1.000.000 MB. Let's say only 50% of such a disk is used, then the saving of 5 MB is equivalent to 0.01 Promille.
So much more important (currently) is the saving of time. But that can also not be compared, cause under normal circumstances the compression is handled in a second thread, the limiting factor currently is the RAM- and CPU-speed, which is needed to copy and serialize the world. The CPU which makes compression will be bored, cause of that.
Well it's still helpful for multiplayer map transfers though.

Re: Save file size: Use 7zip instead of zip

Posted: Fri Jul 08, 2016 6:25 pm
by DjTeo
ssilk wrote:IMHO - to be it worth to think about it - the reduction should be much more than 50%. It's not worth discussing if we spare 5 MB, when normal harddisks have 1.000.000 MB. Let's say only 50% of such a disk is used, then the saving of 5 MB is equivalent to 0.01 Promille.
So much more important (currently) is the saving of time. But that can also not be compared, cause under normal circumstances the compression is handled in a second thread, the limiting factor currently is the RAM- and CPU-speed, which is needed to copy and serialize the world. The CPU which makes compression will be bored, cause of that.
My problem is the slow transfer rate, not how much of my Hard drive it takes. Just use high compression at the file (savegame) it sends to clients not the autosaves. Or have it as option beacause some server has fast connection but not so fast cpu.