I recently had a 320 hour save file get corrupted due to an old server's RAM going out. Backups mysteriously stopped at like the 50 hour mark. Took me about a day, but I managed to rescue it on my own and I wanna share how I did it. If you find yourself in a similar spot, hopefully this helps you recover things too.
The fault signature was during loading the error message "Corrupt map: unknown decorative prototype ID 16467. File has likely been corrupted due to failing hardware" would appear.
Moved the save file to local machine directory (find yours here)
Decompressed the zip file into the same directory and deleted it (Factorio will also load out of folders, handy for doing manual edits like this)
Most data appears to be stored in the "level" files. level-init.dat, level.datmetadata, and script.dat are uncompressed raw binary files. The level.dat# files are zlib compressed files. When uncompressed each file is 1MB
During runtime Factorio seems to load in the level.dat# numbers sequentially, decompresses them on the fly and inserts them into one large holding buffer. It then starts a processing pass keeping track of where it is in this large buffer.
If you run Factorio with the "-v" flag in the arguments list, the log located one directory level up from the saves folder (factorio-current.log) will print out the exact byte location in the holding buffer when the error occurs. (e.g. 38.536 Verbose ThreadedBufferedReadStream.cpp:20: ThreadedBufferedReadStream destroyed due to exception. totalBytesRead=101887170, fileSize=215887603)
Using the totalBytesRead value as the fault location, divide that number out by 1048576 (1MB) to get the file number (e.g. 97.167) and take the floor of that (97). This is the level.dat# that has the bad data. Use a zlib decompression tool (I run Debian so I did "zlib-flate -uncompress < ./level.dat97 > 97_uncompressed"") to get the full data. Open up the file in a binary/hex editor (I used VSCodium with the "Hex Editor" extension installed), then take the file number, multiply it by 1048576, subtract that quantity from totalBytesRead to get the index into the file that needs editing (e.g. 101887170 - (97 * 1048576) = 175298).
The modding? menu in game with Ctrl-Shift-E gave me enough information to figure out that ID numbers are just 16-bit integers, so if you decode the binary at that index as a little-endian it should match the ID number printed out in decimal in the original error message (e.g. 0x5340 -> 0x4053 = 16467). I don't know the structure and meaning of the bytes located nearby, but they seem to be part of a repeating pattern and there were several instances of what looked like ID 0x0053 so I just overwrote the 0x40 byte with 0, recompressed the file, reloaded the save file and I was in. (For my case "zlib-flate -compress=0 < 97_uncompressed > level.dat97)
Some other helpful forum posts that I came across
viewtopic.php?t=47014
viewtopic.php?t=3979
Save File Editing Tips
Re: Save File Editing Tips
You seem to be quite close to my workflow when fixing corrupted save files posted on forums. My usual workflow is following:
1/ Extract zip into directory (game can still load unpacked save files)
2/ In game go to "load game" and only click on the save so the preview opens. If you have `consolidate-chunked-data-in-directory-package` hidden setting enabled and you were looking at unpacked save file, then game will create a level.dat file in its directory.
3/ Verify that level.dat exists and then delete all of the level.dat# files (where # are numbers or "metadata")
4/ Try loading this save while having a verbose logging enabled. Those `totalBytesRead` were added primarily for the purpose of identifying where in the save file the error happened
5/ Modify the level.dat using whatever knowledge you have access to using whatever binary editing tools you have.
6/ Try loading save file. If it does not work, go to step 5
7/ When everything works fine, repackage the directory into zip. Game will happily load a save that is non chunked zipped save.
`consolidate-chunked-data-in-directory-package` exists purely because of me fixing save files. I am also using `max-save-compression-threads` of 0 so my regular saves created by the game have level.dat inside and not the chunked save (chunked saves are there just to make saving faster even if it results in slightly larger zip files).
Main gotcha is that in only 80% of times when fixing saves it is as simple as fixing a single bit flip. Some corrupted save files experience multiple errors, and some corrupted saves need additional access to game sources to understand what structure is corrupted and how to fix it. Because corruption is non deterministic, fixing usually needs to involve a human factor and not all saves can be fixed.
My settings. Max compression threads so my saves have level.dat in it. Consolidate so i can convert other saves to have level.dat in it. Verbose logging to get details where the error is. Consistency check makes the game run extra checks on load.
1/ Extract zip into directory (game can still load unpacked save files)
2/ In game go to "load game" and only click on the save so the preview opens. If you have `consolidate-chunked-data-in-directory-package` hidden setting enabled and you were looking at unpacked save file, then game will create a level.dat file in its directory.
3/ Verify that level.dat exists and then delete all of the level.dat# files (where # are numbers or "metadata")
4/ Try loading this save while having a verbose logging enabled. Those `totalBytesRead` were added primarily for the purpose of identifying where in the save file the error happened
5/ Modify the level.dat using whatever knowledge you have access to using whatever binary editing tools you have.
6/ Try loading save file. If it does not work, go to step 5
7/ When everything works fine, repackage the directory into zip. Game will happily load a save that is non chunked zipped save.
`consolidate-chunked-data-in-directory-package` exists purely because of me fixing save files. I am also using `max-save-compression-threads` of 0 so my regular saves created by the game have level.dat inside and not the chunked save (chunked saves are there just to make saving faster even if it results in slightly larger zip files).
Main gotcha is that in only 80% of times when fixing saves it is as simple as fixing a single bit flip. Some corrupted save files experience multiple errors, and some corrupted saves need additional access to game sources to understand what structure is corrupted and how to fix it. Because corruption is non deterministic, fixing usually needs to involve a human factor and not all saves can be fixed.
My settings. Max compression threads so my saves have level.dat in it. Consolidate so i can convert other saves to have level.dat in it. Verbose logging to get details where the error is. Consistency check makes the game run extra checks on load.
consistency-check=always
verbose-logging=true
consolidate-chunked-data-in-directory-package=true
max-save-compression-threads=0
Re: Save File Editing Tips
Few things in life can make a programmer happier than being told their reverse engineering efforts yield similar results to what those with the source code can do.
Thank you
To be honest, I was fearing my save file was in that category of multiple faults and was amazed that the single bit flip fixed it. My day job is writing software for spacecraft, so I'm no stranger to radiation flipping a bit and the hell that causes but I'm legitimately shocked to see something similar on a terrestrial system. Granted the RAM I work with is orders of magnitude slower than the DDR4 of modern systems but I'm still surprised that a single bit-flip like this could happen (or be often enough that your team has pre-existing procedures for dealing with it).
Thank you
To be honest, I was fearing my save file was in that category of multiple faults and was amazed that the single bit flip fixed it. My day job is writing software for spacecraft, so I'm no stranger to radiation flipping a bit and the hell that causes but I'm legitimately shocked to see something similar on a terrestrial system. Granted the RAM I work with is orders of magnitude slower than the DDR4 of modern systems but I'm still surprised that a single bit-flip like this could happen (or be often enough that your team has pre-existing procedures for dealing with it).