Corrupt map: invalid entities marker

[Polyphobia]

Hello

Game version: Factorio 2.0.43 (build 82163, linux64, steam, space-age)

The manual and auto saves have this error "Corrupt map: invalid entities marker" when I try to load them.
I have attached the save files and the log when trying to load the save.

If anyone can help me fix the save I would appreciate it.

[eugenekay]

Corrupt maps are almost always an indicator of failing Hardware somewhere in your system. The general advice is: Revert to an earlier Autosave; and turn up the # that you keep, or use date-based Save Game file names.

0.010 System info: [CPU: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz, 8 cores, RAM: 31919 MB]

This is a 2017 era Kaby Lake CPU, so it is unlikely to be a "Raptor Lake" BIOS/CPU defect. The usual next-cause is a Memory defect (run Memtest), or Thermal problem under intense load.

Good Luck!

[pioruns]

Tried to load your save. Result:

Code: Select all

  24.485 Loading map /home/pioruns/GOG Games/Factorio/game/saves/2025.zip: 41937040 bytes.
  24.493 Loading level.dat: 115285148 bytes.
  24.496 Info Scenario.cpp:153: Map version 2.0.43-3
  25.280 Warning Map.cpp:362: Map loading failed: Corrupt map: invalid entities marker
  25.417 Error AppManagerStates.cpp:1649: Corrupt map: invalid entities marker
  32.295 Quitting: user-quit.
  32.536 Goodbye

Can you revert to manual save day before the corruption?

[Polyphobia]

Corrupt maps are almost always an indicator of failing Hardware somewhere in your system. The general advice is: Revert to an earlier Autosave; and turn up the # that you keep, or use date-based Save Game file names.

0.010 System info: [CPU: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz, 8 cores, RAM: 31919 MB]

This is a 2017 era Kaby Lake CPU, so it is unlikely to be a "Raptor Lake" BIOS/CPU defect. The usual next-cause is a Memory defect (run Memtest), or Thermal problem under intense load.

Good Luck!

Thanks for the pointers.

Now it's just me that is curious, but how can a thermal problem be an issue with all the saves? As I see it, if the system was running too hot, then it will just shutdown and corrupt the save in progress (and not all of them).


Regardless, after a Memtest it showed 2 addresses with errors... so that's probably what it was that corrupt the save file. But I find it funny that I don't have any system instability.

[Polyphobia]

Tried to load your save. Result:

Code: Select all

  24.485 Loading map /home/pioruns/GOG Games/Factorio/game/saves/2025.zip: 41937040 bytes.
  24.493 Loading level.dat: 115285148 bytes.
  24.496 Info Scenario.cpp:153: Map version 2.0.43-3
  25.280 Warning Map.cpp:362: Map loading failed: Corrupt map: invalid entities marker
  25.417 Error AppManagerStates.cpp:1649: Corrupt map: invalid entities marker
  32.295 Quitting: user-quit.
  32.536 Goodbye

Can you revert to manual save day before the corruption?

I appreciate you giving it a try. But unfortunately I only have the one manual save you have tried. I guess, that I am just gonna have to start a new game, when I get a new computer.

[pioruns]

I appreciate you giving it a try. But unfortunately I only have the one manual save you have tried. I guess, that I am just gonna have to start a new game, when I get a new computer.

Very sad to hear. Next time, make manual saves, every day when you start or finish your session. That way, you never losing more than one session of gaming.

Regardless, after a Memtest it showed 2 addresses with errors... so that's probably what it was that corrupt the save file. But I find it funny that I don't have any system instability.

Faulty RAM with only two addresses with errors is not enough to crash the system regularly, system does not use much resources, so its not that exposed to these tiny damaged RAM cells. But a game like Factorio, loads several gigabytes of data into memory. And that data got corrupted during your session, and made way into all your saves from that session, and it happened to be something important, unfortunately. At least, that's how I understand it, correct me if I am wrong.

[Rseding91]

… But I find it funny that I don't have any system instability.

Well, you do! The system instability corrupted a save file in a game you were playing.

[eugenekay]

Now it's just me that is curious, but how can a thermal problem be an issue with all the saves? As I see it, if the system was running too hot, then it will just shutdown and corrupt the save in progress (and not all of them).


Regardless, after a Memtest it showed 2 addresses with errors... so that's probably what it was that corrupt the save file. But I find it funny that I don't have any system instability.

High system temperatures cause the silicon to degrade in many parts - either directly through electrical switching action (causing heating resistance at a Silicon junction that is just barely working, pushing it over the edge) or very rarely by causing a mis-read of a memory cell, which is caught by ECC RAM. Whether the degradation was inevitable or due to Temperatures is hard to tell without Microscopic inspection of the silicon die; but you can usually find a brown “hot spot” after delidding the chip. However, seeing persistent memtest errors at a specific Address tells you where to look. Or just change the memory stick - it’s cheaper than resoldering equipment

Most applications will never notice a single bad byte - you might get a single bad pixel in a video frame for a tenth of a second.

In a game like Factorio which stores all of the data in C style memory structures in the save game file, a single “bad byte” in Memory can go undetected while running, then get Saved to disk. With a large amount of system Memory it is unlikely that you will have anything “important” assigned to a particular Faulty memory address…. Until you launch a high-memory usage game, multiple web browsers, etc.

Good luck!

[Polyphobia]

I appreciate you giving it a try. But unfortunately I only have the one manual save you have tried. I guess, that I am just gonna have to start a new game, when I get a new computer.

Very sad to hear. Next time, make manual saves, every day when you start or finish your session. That way, you never losing more than one session of gaming.

Regardless, after a Memtest it showed 2 addresses with errors... so that's probably what it was that corrupt the save file. But I find it funny that I don't have any system instability.

Faulty RAM with only two addresses with errors is not enough to crash the system regularly, system does not use much resources, so its not that exposed to these tiny damaged RAM cells. But a game like Factorio, loads several gigabytes of data into memory. And that data got corrupted during your session, and made way into all your saves from that session, and it happened to be something important, unfortunately. At least, that's how I understand it, correct me if I am wrong.

You make an excellent point, in the future I will make more backups of my saves.


And thanks for the advice.

[Polyphobia]

… But I find it funny that I don't have any system instability.

Well, you do! The system instability corrupted a save file in a game you were playing.

I apologize for not making myself more clear. I meant to say that I haven't experienced any other problems... I guess that it has just been dumb luck.

[Polyphobia]

Now it's just me that is curious, but how can a thermal problem be an issue with all the saves? As I see it, if the system was running too hot, then it will just shutdown and corrupt the save in progress (and not all of them).


Regardless, after a Memtest it showed 2 addresses with errors... so that's probably what it was that corrupt the save file. But I find it funny that I don't have any system instability.

High system temperatures cause the silicon to degrade in many parts - either directly through electrical switching action (causing heating resistance at a Silicon junction that is just barely working, pushing it over the edge) or very rarely by causing a mis-read of a memory cell, which is caught by ECC RAM. Whether the degradation was inevitable or due to Temperatures is hard to tell without Microscopic inspection of the silicon die; but you can usually find a brown “hot spot” after delidding the chip. However, seeing persistent memtest errors at a specific Address tells you where to look. Or just change the memory stick - it’s cheaper than resoldering equipment

Most applications will never notice a single bad byte - you might get a single bad pixel in a video frame for a tenth of a second.

In a game like Factorio which stores all of the data in C style memory structures in the save game file, a single “bad byte” in Memory can go undetected while running, then get Saved to disk. With a large amount of system Memory it is unlikely that you will have anything “important” assigned to a particular Faulty memory address…. Until you launch a high-memory usage game, multiple web browsers, etc.

Good luck!

Appreciate you taking the time to break that down for me, is really insightful. Makes a lot more sense now, especially with the Factorio example.

[boskid]

Just for reference if other people would find this topic searching for "invalid entities marker" - this is just a set of known good values that are put inside of a save file in known locations so that if there are values corrupted in a way that make loading confused about which bytes are what, if at the known position there is a wrong value read, that means that definitely loading is misaligned with what was saved and is now interpreting bytes incorrectly and it is better to abort loading as it would not load anything meaningful anymore. The "entities marker" (and 3 other types of known good values) are just ways to detect that something has happened really wrong when loading.

In the provided save file, i looked at the position of misalignment and the 2 bytes marked red are where game thinks there are no more entities saved on the current chunk and it checks for a known bytes indicating it is indeed an end of entities, however the 2 bytes read are not what was supposed to be there. Marked with blue there are 2 bytes where a valid entities marker exists, clearly showing there are some bytes in between that should still be interpreted as entities but due to some corruption the game no longer reads them causing save to not load properly.

04-21-2025, 20-16-10.png (41.75 KiB) Viewed 512 times

[boskid]

As an excercise i tried to fix this save file and it is indeed corrupted save file, but it was in a repairable state. Nearby the faulty spot there was a valid tiles marker "/T" so i decided to take a look at how this save file was interpreted. There were some structures loaded from chunk that looked correctly, then there were 2 fishes and then "00 00" happened which was interpreted as "no more entities on this chunk". I did a blind guess and changed it into "D0 00" making the following bytes be recognized as a fish and the save file loaded correctly.

128224-2025-fixed-notes.png (29.26 KiB) Viewed 485 times

A "fixed" save file:

128224-2025-fixed.zip

(34.03 MiB) Downloaded 17 times

[eugenekay]

As an excercise i tried to fix this save file and it is indeed corrupted save file, but it was in a repairable state. Nearby the faulty spot there was a valid tiles marker "/T" so i decided to take a look at how this save file was interpreted. There were some structures loaded from chunk that looked correctly, then there were 2 fishes and then "00 00" happened which was interpreted as "no more entities on this chunk". I did a blind guess and changed it into "D0 00" making the following bytes be recognized as a fish and the save file loaded correctly.

As a Computer Science academic exercise, I wonder how often a real-world Memory Mis-Read ends up as the “null byte”, compared to “all high bits” or “one mixed bit”? It may be possible to detect the Null corruption more intelligently if you can encapsulate the problem to a single “Entity” that could be repaired based on context… Or at least catch it earlier by adding a routine to “sanity check” Autosave files by reading them back from disk immediately - BEFORE deleting the oldest (presumably-still-good) Autosave file.

Most of my programming experience with Memory errors comes from Distributed Transactional systems, wherein the Correct behavior is to abort the Process entirely so it can be re-played on a different Database Node: not a feasible pattern here.

[boskid]

As a Computer Science exercise, I wonder how often a real-world Memory Mis-Read ends up as the “null byte”, compared to “all high bits” or “one mixed bit”?

I do not know that, but given that there are 4 save files all corrupted the same way (not verified), what i predict has happened is that the faulty fish instance had a corrupted prototype pointer: entities in memory have couple of important values on them, one being pointer to a prototype from which it was created, and when saving an entity, the ID of the entity is obtained from the prototype. Corrupted prototype pointer could be pointing at some other block of bytes and at the exact position where ID was expected were just zeros (it could be any other value at this point). This is the only reasonable explanation why this specific fish was corrupted and the issue was persistent. If it would be corruption while saving then it would only affect one save file.

[eugenekay]

If it would be corruption while saving then it would only affect one save file.

Ahhh so it’s not the exact runtime Memory structure being dumped/ restored to do Save/Load; there is de/referencing step each time. That makes sense to allow for Mod changes etc…

I guess a more distilled thought is: Can you check for (all types of) Runtime Corruption at Autosave time, by doing the first half of the Verification—>Load process on the about-to-be-saved Autosave.zip? This would tell the player immediately that their Running game is corrupted, and they wouldn’t be able to Load it anyway.

I realize that this is now a full “Ideas & Suggestions” post.

[Rseding91]

We generally want auto-save to be as fast as possible and so don't do slow verification logic on saving. It's also expected that players computers are not broken so spending extra time for the < 1% case where they are isn't great.

[boskid]

I guess a more distilled thought is: Can you check for (all types of) Runtime Corruption at Autosave time, by doing the first half of the Verification—>Load process on the about-to-be-saved Autosave.zip? This would tell the player immediately that their Running game is corrupted, and they wouldn’t be able to Load it anyway.

We can, but we wont. At least we could do the consistency check, but whenever possible we try to make saving as fast as possible and this type of checks would slow this process down significantly. Recently i added the consistency scraper which visits random chunks and checks consistency of small subset of entities each tick but the amount is extremally small so there are no huge performance penalties at runtime. For 2.0.46 i extended the entity consistency check to verify the prototype pointer is valid, aiming to find exactly that type of corruption that happened in this topic.

[eugenekay]

We can, but we wont. At least we could do the consistency check, but whenever possible we try to make saving as fast as possible and this type of checks would slow this process down significantly. Recently i added the consistency scraper which visits random chunks and checks consistency of small subset of entities each tick but the amount is extremally small so there are no huge performance penalties at runtime. For 2.0.46 i extended the entity consistency check to verify the prototype pointer is valid, aiming to find exactly that type of corruption that happened in this topic.

Good insights! Thank you for sharing. The Consistency Scraper sounds like it is solving the exact same problem from the other side: excited to see how that works. I don’t have any suspicious hardware myself - I am just Paranoid about data loss.

Maintaining real-time responsiveness is indeed critically important in a User-facing application, so that makes a lot of sense for the single-threaded case. Adding the complexity for the experimental-fork-Autosave for the extreme edge case is indeed a little crazy, but I am in it for the Academic question anyway.