is save consistency checks run when loading a map downloaded from a server?

[ilikehackinggames]

is save consistency checks perforemed when downloading and loading a map from a server? or can a server send a borked map and posibly cause more issues?

[Rseding91]

It is not. It's assumed if the save is running the consistency check has already happened at load time and it should remain consistent during the runtime.

[ilikehackinggames]

It is not. It's assumed if the save is running the consistency check has already happened at load time and it should remain consistent during the runtime.

noted. and what gets checked durring the consistencey checks when they do run?

[Rseding91]

A ton of things. Way too many to list here.

[ilikehackinggames]

A ton of things. Way too many to list here.

and the main point of the checks is to prevent bugs like null refrences, memory curruptson, overreads, and general unintended behavor correct?

[eugenekay]

A ton of things. Way too many to list here.

and the main point of the checks is to prevent bugs like null refrences, memory curruptson, overreads, and general unintended behavor correct?

Interesting angle, but I do not think you will get very far here towards modifying shared Game State. The server-client protocol depends upon the Client being able to "run" the game and report back a Checksum to the Server periodically. If these checks fail you get a Desync - and the Client gets kicked.

You can (possibly?) get a Client Remote-Code-Execution from this methodology, but I have my doubts that it gains you anything beyond exploting the Lua Sandbox in-game already does. Factorio is not the type of program to run untrusted as root...

[Rseding91]

Consistency checks are what ever we think is worth checking. The most common one being: we expect that if condition A and condition B are true, condition C should be true. Or things such as cached values always being equal to the computed values. It's not about memory corruption, overreads, or null dereferences - those should never happen regardless of things being consistent.

An example around caching:

You are told to keep track of how many people are inside a house. You sit at the front door and every time someone goes in you increase the counter by 1. Every time someone leaves you decrease it by 1.

A consistency check for this would be going over the entire house and manually adding up how many people are in it to verify it's the same as the counter you've been maintaining. If it's wrong - you either forgot to account for someone coming or going through the front door - or people are getting in/out of the house through other means than the front door. Maybe people are coming/going through the back door and you need to also keep track of that. Or maybe some super edge case happened like someone giving birth or dying while in the house and you need to add a special rule for that happening.

[ilikehackinggames]

Consistency checks are what ever we think is worth checking. The most common one being: we expect that if condition A and condition B are true, condition C should be true. Or things such as cached values always being equal to the computed values. It's not about memory corruption, overreads, or null dereferences - those should never happen regardless of things being consistent.

An example around caching:

You are told to keep track of how many people are inside a house. You sit at the front door and every time someone goes in you increase the counter by 1. Every time someone leaves you decrease it by 1.

A consistency check for this would be going over the entire house and manually adding up how many people are in it to verify it's the same as the counter you've been maintaining. If it's wrong - you either forgot to account for someone coming or going through the front door - or people are getting in/out of the house through other means than the front door. Maybe people are coming/going through the back door and you need to also keep track of that. Or maybe some super edge case happened like someone giving birth or dying while in the house and you need to add a special rule for that happening.

wait so if thats true why cant it just fix them automaticly by for exsample making condition C true or recounting the people in your exsample. also why does the game still crash in certen cases with a invalid save then eg the most commen one being trying to remove something in chunk x but marked as in chunk y (at least from reading outer currupted save threads) is it some check that terminates the program early? what about cases like a inserter having over 5 filters for exsample? is that handled somewhere outer then consistency checks? eg when creating the inserter?

[ilikehackinggames]

thanks for ansering these questons btw

[ilikehackinggames]

A ton of things. Way too many to list here.

and the main point of the checks is to prevent bugs like null refrences, memory curruptson, overreads, and general unintended behavor correct?

Interesting angle, but I do not think you will get very far here towards modifying shared Game State. The server-client protocol depends upon the Client being able to "run" the game and report back a Checksum to the Server periodically. If these checks fail you get a Desync - and the Client gets kicked.

You can (possibly?) get a Client Remote-Code-Execution from this methodology, but I have my doubts that it gains you anything beyond exploting the Lua Sandbox in-game already does. Factorio is not the type of program to run untrusted as root...

to be clear for Remote-Code-Execution to be useful it doesnt need to be root. in fact most malware (or at least windows malware idk about linix malware) doesnt need admin/root to do damage.
also so far all ive found when it comes to exsploting the lua sandbox is overflowing the lua stack with 0s (in thery posibly currupting the save if your unlucky but likely crashing if anything. but nothing useful without controaling what its overwriting witch would requre heap fung sway witch im pritty sure needs the ability to realocate the buffer witch as far as i can tell theres no way to realocate the lua stack. plus even then the lack of ability to overflow anything but 0s would make it much harder to find something useful to overide that doesnt get freed (witch would crash the game due to heap validatson/null refrence on both linux and windows. plus the lack of a info leak to bypass alsr is the seal in the coffen. not to mencon this bug is over 10 years old now.) also now that i think about it doesnt the server get a desync report when it desyncs for debuging. doesnt that desync report include like the entire game state? i feel like that could be useful if you cause a desync using a oob read. as you could likely see the value read from oob.

[Rseding91]

wait so if thats true why cant it just fix them automaticly by for exsample making condition C true or recounting the people in your exsample. also why does the game still crash in certen cases with a invalid save then eg the most commen one being trying to remove something in chunk x but marked as in chunk y (at least from reading outer currupted save threads) is it some check that terminates the program early? what about cases like a inserter having over 5 filters for exsample? is that handled somewhere outer then consistency checks? eg when creating the inserter?

In the count example, if the cache value is wrong it means we missed some case and it needs to be fixed. If we just "reset the count" we never get notified it was wrong and never know to fix it. That also leads to desyncs.

In the other cases - it's simply "this should never be possible but some how it did, likely hardware bit flips corrupted things"

[ilikehackinggames]

wait so if thats true why cant it just fix them automaticly by for exsample making condition C true or recounting the people in your exsample. also why does the game still crash in certen cases with a invalid save then eg the most commen one being trying to remove something in chunk x but marked as in chunk y (at least from reading outer currupted save threads) is it some check that terminates the program early? what about cases like a inserter having over 5 filters for exsample? is that handled somewhere outer then consistency checks? eg when creating the inserter?

In the count example, if the cache value is wrong it means we missed some case and it needs to be fixed. If we just "reset the count" we never get notified it was wrong and never know to fix it. That also leads to desyncs.

ah makes sence.
still doesnt anser why the game still crashes even with consistency checks disabled for many broken saves. as based on your last message you said any crashes from bad map data is intended (ex biter on chunk a but regesterd to chunk b crashes when trying to kill it with consistency checks disabled) is it hitting a assert in cases like this or what?