Corrupt map: Unknown tile 246

[mephir]

So as usual I finished playing yesterday night and save my game. I tried to load it today and I got message from save game. Is there anything what could be done so I can go back to my 300+ hs save? I'm using nicefill and waterfill mods is one of them which caused issue? My log looks like this:

Code: Select all

  13.655 Info HttpSharedState.cpp:54: Downloading https://updater.factorio.com/updater/get-available-versions?username=mephir&token=<private>&apiVersion=2
  14.724 0 packages available to download (experimental updates disabled).
  16.142 Loading map C:\<private :) >\jazda - w drodze do 45sps.zip: 76092522 bytes.
  16.174 Loading level.dat: 196775798 bytes.
  16.179 Info Scenario.cpp:197: Map version 1.1.35-1
  17.328 Warning Map.cpp:346: Map loading failed: Corrupt map: unknown tile 246.
  17.747 Error AppManagerStates.cpp:1591: Corrupt map: unknown tile 246. 

Here are autosaves as well as uploaded save game:
https://rt.mephir.io/factorio/_autosave1.zip
https://rt.mephir.io/factorio/_autosave2.zip
https://rt.mephir.io/factorio/_autosave3.zip
https://rt.mephir.io/factorio/jazda.zip - savegame with changed filename only
https://rt.mephir.io/factorio/mods.zip

[azesmbog]

Oh how! So I'm not alone with my problem ))))))))

[KeepGridButton]

Oh how! So I'm not alone with my problem ))))))))

No you're not alone. There's an entire section "0/1 Magic" of people with similar issues. People are losing all their saves and the staff are in disagreement about the problem. I recommended making backups of saves on successful load(no more than one per save so it won't take up alot of space) to avoid people losing all their playthrough saves from this kind of issue. The staff person who comes around and fixes these says it's random memory corruption they can't prevent, which seems reasonable. However, the other staff person who rejects the mitigation strategy disagrees, and says rather than mitigate with backups the underlying bug should be fixed. So one says it's not a bug, the other says it is bug.

[mephir]

Oh how! So I'm not alone with my problem ))))))))

No you're not alone. There's an entire section "0/1 Magic" of people with similar issues. People are losing all their saves and the staff are in disagreement about the problem. I recommended making backups of saves on successful load(no more than one per save so it won't take up alot of space) to avoid people losing all their playthrough saves from this kind of issue. The staff person who comes around and fixes these says it's random memory corruption they can't prevent, which seems reasonable. However, the other staff person who rejects the mitigation strategy disagrees, and says rather than mitigate with backups the underlying bug should be fixed. So one says it's not a bug, the other says it is bug.

I don't think it is bit flip. I can agree it is difficult to find cause. Bit flip appears quite often, during data copy, there is many causes why it might happen. The thing is that save was created actually on machine which has ecc memory, yeah i use basically server as my desktop, quite niche, but i don't have a lot of choice when it comes to my work. memory is not the only thing which might cause bit flip. bit flip maybe would cause one save corrupted but not last 9, i used to save every hour. I'm just disappointed a little, because it was my first playthrough, game is amazing and something i like the most does not require top line graphic card. There is only one more game which can bring my attention for so long, it is ttd(openttd currently), most i finish playing after less than 10-20 hours. I'm hoping I can continue.

[orzelek]

Oh how! So I'm not alone with my problem ))))))))

No you're not alone. There's an entire section "0/1 Magic" of people with similar issues. People are losing all their saves and the staff are in disagreement about the problem. I recommended making backups of saves on successful load(no more than one per save so it won't take up alot of space) to avoid people losing all their playthrough saves from this kind of issue. The staff person who comes around and fixes these says it's random memory corruption they can't prevent, which seems reasonable. However, the other staff person who rejects the mitigation strategy disagrees, and says rather than mitigate with backups the underlying bug should be fixed. So one says it's not a bug, the other says it is bug.

I don't think it is bit flip. I can agree it is difficult to find cause. Bit flip appears quite often, during data copy, there is many causes why it might happen. The thing is that save was created actually on machine which has ecc memory, yeah i use basically server as my desktop, quite niche, but i don't have a lot of choice when it comes to my work. memory is not the only thing which might cause bit flip. bit flip maybe would cause one save corrupted but not last 9, i used to save every hour. I'm just disappointed a little, because it was my first playthrough, game is amazing and something i like the most does not require top line graphic card. There is only one more game which can bring my attention for so long, it is ttd(openttd currently), most i finish playing after less than 10-20 hours. I'm hoping I can continue.

Problem with those bit flips is that they can corrupt all the autosaves when they happen in a way that doesn't affect normal game processing. So game won't crash etc due to one but data is corrupted in memory. And every subsequent save will save that corrupted bit which will render save file unloadable.
From what I recall some of validation has been added to saving process (to catch those earlier) but full validaiton is done on load and only then you can find out that save file you are loading is corrupted (and that all of them might be also).

It doesn't make sense here if you were playing on machine that has ecc memory. It might not be perfect in preventing errors it should be guaranteed to catch them (one bit flip can be corrected, multiple bit flips will cause system error).

I would recommend checking basic vitals of system like tempreatures during operation and voltages. That could also cause similar issues and it can change with time (PSU's degrade, it's summer so temps are up etc)

[mephir]

Problem with those bit flips is that they can corrupt all the autosaves when they happen in a way that doesn't affect normal game processing. So game won't crash etc due to one but data is corrupted in memory. And every subsequent save will save that corrupted bit which will render save file unloadable.
From what I recall some of validation has been added to saving process (to catch those earlier) but full validaiton is done on load and only then you can find out that save file you are loading is corrupted (and that all of them might be also).

It doesn't make sense here if you were playing on machine that has ecc memory. It might not be perfect in preventing errors it should be guaranteed to catch them (one bit flip can be corrected, multiple bit flips will cause system error).

I would recommend checking basic vitals of system like tempreatures during operation and voltages. That could also cause similar issues and it can change with time (PSU's degrade, it's summer so temps are up etc)

The bit flip errors may happen in many ways in many places. Outside temperature is not valid in my case. Im not living in Poland/europe, but middle east. I could put 300hs in 2,5 month only because "outside" is unbearable hot. Temperatures close to 40 or more during a day, my home has 4 hvac zones and keep fixed 22 degrees(celsius) all year round.

I don't know what "Unknown tile" error means, i don't have access to code of game, but most likely simple fix would be to change it to something known and tool could be easy provided or done by game with some warning message, that save was recovered due corruption. I won't be going with argumentation why in my opinion it is not bit flip, because most likely it won't bring things and actual bug might be hard to isolate and fix. In my career I had to, for different reasons, mark many tickets as 'wont fix', so I'm not to blame devs. Anyway I happily restored mine blueprints so I will start today new playthrough and just dump memory after each save.

[Silari]

It means that the value it found in the save file doesn't match the list of tiles it has saved - ie it doesn't know what the tile is supposed to be, it is unknown. IIRC from dev posts the game takes all the tiles and maps their name to a numeric id, so that it can save that info as a single byte rather than the text string of the name. When it loaded the save the value it found for that tile wasn't in their map so it can't know what tile it is.

There's no way to fix that automatically since the game would have to know what that tile is supposed to be, and the only way to know that is for that byte to not have been corrupted. It could guess at what it might be using surrounding tiles and/or terrain gen but there wouldn't be any way to verify if it's correct, which isn't a particularly good automatic tool.

[mephir]

It means that the value it found in the save file doesn't match the list of tiles it has saved - ie it doesn't know what the tile is supposed to be, it is unknown. IIRC from dev posts the game takes all the tiles and maps their name to a numeric id, so that it can save that info as a single byte rather than the text string of the name. When it loaded the save the value it found for that tile wasn't in their map so it can't know what tile it is.

There's no way to fix that automatically since the game would have to know what that tile is supposed to be, and the only way to know that is for that byte to not have been corrupted. It could guess at what it might be using surrounding tiles and/or terrain gen but there wouldn't be any way to verify if it's correct, which isn't a particularly good automatic tool.

It can always put fill tile and marking during loading it was corrupted. I don't have time to reverse engineer save, so i can't really do it myself. considering complexity it will take a lot of time. after 300hs and unfinished target of getting each science blue belt, i would rather have one tile of fill somewhere which mostly i wont spot anyway, than have this feeling of unfinishing something. It is even not feeling of being defeated, because I would accept it and start new game.

[DarkShadow44]

Agreed, noone said the game should fix the bitflip, but an option to continue loading with an error tile set at that position would be great. Better than losing the entire save, and should be easy to implement as well.

[posila]

The problem is, the tile it erred on, is not the place that is corrupted. It is just the first place the game has detected the value it read from the binary stream is not valid.

It is most likely repairable, but definitelly not automatically ... some human investigation to find what value is actually the corrupted one and guessing what the correct or at least valid value could be needs to take place.

[DarkShadow44]

The problem is, the tile it erred on, is not the place that is corrupted. It is just the first place the game has detected the value it read from the binary stream is not valid.

Sorry, I don't understand. It's a bitflip in the tile data, no?

[posila]

Sorry, I don't understand. It's a bitflip in the tile data, no?

Well, I don't know what exactly it is yet. I know there were 373 tiles saved on chunk (-37, 32) and after loading 373 tiles, some of the tiles on the chunk were not initialized yet, so it continued loading tiles. Either one of the tiles has incorrect size, or some tiles were not saved at all.

In general though, the same error can be also caused by corrupted entity on previous chunk, which would happen to end loading of the chunk, and the game would start loading tiles on the next chunk, but would read something that is not tile data at all.

[KeepGridButton]

Agreed, noone said the game should fix the bitflip, but an option to continue loading with an error tile set at that position would be great. Better than losing the entire save, and should be easy to implement as well.

Backing up a save after a successful load would at least ensure if the game started saving corrupted saves, you'd have at least one valid uncorrupted save from prior to the corruption. You'd have no more than one copy of each save backed up so it's not going to be something that grows out of control.

[Sonderlinger]

So for the not so super tech-savvy pleaple like myself (no idea what a bit flip is...):

On some save one save game gets currupted. This doen't initally prevents it from loading. So on every save after that this corruptions get's carried over and on some point it prevents ALL saves from loading?

I have the same problem: viewtopic.php?f=49&t=99376

So in conclusion, nothing can be done from our side and we have to hope that someone can find the time to fix this by hand like in this case viewtopic.php?f=35&t=98417 ?

This sounds kind of like a horrible problem, I'm totally down to starting a new game again, but after my last couple of hundred hours save game this sounds scary...