[0.16.30] zip headers error

[studmuffin]

The game runs fine, but when extracting the files from a game save zip, 7zip is giving a warning for "Headers Error". This issue first became prevalent in our factorio web control, which checks a zip file for consistency when uploaded to the server. Factorio appears to be slightly corrupting the zip files on save.

[Rseding91]

Can you upload one of the saves with the issue? Also do you have any reliable way to reproduce the problem from a new save? When I test myself it works without any issues.

[Ps7cho]

Its in our web server that is using php to check the zip file its finding inconsistency in the save file that factorio gives
https://puu.sh/zI7zf/affc78511b.png
when i unzip it and rezip with 7zip the php takes it just fine.
when i test the save file with 7zip it gives this error on our web server: https://puu.sh/zI7MA/153bf18330.png
the save file with the errors is attached.

[Rseding91]

Its in our web server that is using php to check the zip file its finding inconsistency in the save file that factorio gives
https://puu.sh/zI7zf/affc78511b.png
when i unzip it and rezip with 7zip the php takes it just fine.
when i test the save file with 7zip it gives this error on our web server: https://puu.sh/zI7MA/153bf18330.png
the save file with the errors is attached.

Are you sure you uploaded the correct file? When I test it with 7z it reports no errors:

[Ps7cho]

https://puu.sh/zI88A/e3d2b0417f.png

[kitcat]

7zip versions 9.04 (on Ubuntu 11.04) and 9.20 (on Ubuntu 16.04) report no errors for that file for me and neither does UnZip version 6.00 (on Ubuntu 11.04 and 16.04).

The file is identical to the previously uploaded file, btw.

Edit: Included OS info.

[Rseding91]

What OS do you have? (both of you)?

[Rseding91]

It almost looks like the latest version of 7z for Win10 is broken. I can reproduce the error on 7z 17 on win10 but 16 on win10 works correctly saying no errors and 18 on win10 gives no errors for the in-app "test" but does give errors for the right click -> test option.

[studmuffin]

After checking a few more things with the same zip file Ps7cho uploaded,

win10 - winrar 5.60 beta1 64-bit does not detect errors
win10 - 7zip 18.01 x64 gives the warning "Headers Error"
win10 - 7zip 16.04 x64 does not detecterror
ubuntu16.04 - php with zip version 1.15.2, below is the code using to call the zip library to check the file

Code: Select all

$fileTmp = $_FILES['file']['tmp_name'];
$zip = new ZipArchive();
$res = $zip->open($fileTmp, ZipArchive::CHECKCONS);
if ($res !== TRUE) {
    switch($res) {
        case ZipArchive::ER_NOZIP:
        unlink($fileTmp);
        die('Error z'.__LINE__.': Not a zip archive');
        case ZipArchive::ER_INCONS :
        unlink($fileTmp);
        die('Error z'.__LINE__.': Zip consistency check failed');
        case ZipArchive::ER_CRC :
        unlink($fileTmp);
        die('Error z'.__LINE__.': Zip checksum failed');
        default:
        unlink($fileTmp);
        die('Error z'.__LINE__.': Zip error ' . $res);
    }
}

fails on case ZipArchive::ER_INCONS


"client" is the same file that Ps7cho uploaded - fails php and 7zip check
"server" is the started on the client file then immediately stopped, triggering the server to save the loaded game -fails php and 7zip check
"7zip" is the original save unzipped, and re archived - passes php check
"winrar" is the original save unzipped, and re archived - passes php check

The server save is a different size than ps7cho's upload. This could be different compression levels between the server and the client configurations. 7zip and winrar filesize can be almost completely disregarded due to potential compression methods. However it is interesting that the same version of factorio (albeit, ps7cho is running windows 10, and the server is running ubuntu 16.04) is producing different sized files.

if I take the server's file, extract it, and re-archive with 7zip, the file size is further reduced to 1.17mb. start/stopping this file returns the save to the same 1.28mb as before.

Another thing to note, is that the server and a client will start fine with any of these files. It's mainly a problem for us since the server is checking for zip inconsistency, which we will likely disable soon so we can upload files again. But this is also concerning that factorio is doing something to cause inconsistency. Downloading the server generated, and attempting to re-upload also triggers the inconsistency error.

--edit: correction
The php zip check is failing on validating ZipArchive::CHECKCONS, indicating there is an error with the zip file. we'll have to remove both the consistency error and the default error case to enable file uploads on our server again. This check is being use to try and ensure only actual zip files are uploaded to the server. The consistency check was in case the zip was legitimately corrupted; ie, a server force stopped before a save completed, or some other situation where a factorio save might be interrupted. These files would prevent the server from starting.

[primedead]

I had just had the same error with a save. It was saved on a linux headless server. 7-zip version 18.01 windows 10 build 17120.1. This is the save file uploaded to google drive: https://drive.google.com/open?id=17p2wu ... bu3k7O4cxv I tested it by downloading and extracting and it still had the header error. I expect phpBB to rezip the file removing the error.

[Chocolatetthunder]

Yes I am using windows 10 but the php check was also having errors as stud muffin explains better. The php is on a Linux vm

[posila]

I think it might be related to us using zip64, so the file has ZIP headers and ZIP64 headers and I guess the library we use (minizip) writes out ZIP header with some unexpected or unusual values. I am not really in favour of looking into this much further.