Page 1 of 1
level.dat format
Posted: Mon Feb 09, 2015 4:41 pm
by barkface
Hello!
In my quest to create a mod-managing utility, I'm trying to read information about the mods used from a savegame.
Looking at level.dat, the header seems to have what I want.
I can see the mod names stored in a single-width character set (One of the forms of ISO-8859-1?).
Thanks to Rseding, I know the 4 bytes in front of the mod names are an uint32 describing the length of the mod name in byte.
So the section with the mod names looks like
4 bytes uint length_of_name
length_of_name string modname
6 byes unknown
and repeating.
I can't make heads or tails of the header before the mod names, though.
Can a dev shed some light on the level.dat format?
What's in that header, and how do I find the length of the entire thing? Is my assumption about the mod names section correct?
Thanks~
Re: level.dat format
Posted: Mon Feb 09, 2015 4:44 pm
by dee-
AFAICT the format is still WIP.
For example there has been a major data redesign (float -> fixed int) some weeks ago.
Somewhere in the header must also be the game version that created the save as older game versions refuse to open newer savegames and can tell which game version created the save.
Re: level.dat format
Posted: Mon Feb 09, 2015 4:50 pm
by barkface
The game version could actually be the very first thing. Assuming uint16, the first 12 bytes spell out 0 11 15 for my recent saves, and 0 11 10 for an old one I had lying around (could easily have been 0.11.10).
Re: level.dat format
Posted: Mon Feb 09, 2015 5:25 pm
by dee-
barkface wrote:The game version could actually be the very first thing. Assuming uint16, the first 12 bytes spell out 0 11 15 for my recent saves, and 0 11 10 for an old one I had lying around (could easily have been 0.11.10).
one more down
Re: level.dat format
Posted: Tue Feb 10, 2015 9:28 am
by kovarex
The 6 unknown bytes is version of the mod (2 bytes per number).
Code: Select all
void ScenarioExecutionContext::save(MapSerialiser& output) const
{
output << this->location;
output << this->difficulty;
output << this->finished;
output << this->playerWon;
output << this->nextLevel;
output << this->savingReplay;
output << ApplicationVersion::currentVersion;
output << this->allowedCommands;
output << this->stats;
output << global->modManager->getActiveModThatChangeTheGameStateIDList();
}
Re: level.dat format
Posted: Tue Feb 10, 2015 9:22 pm
by barkface
Squeee, Kovarex-senpai noticed me~
Thanks a bunch, with some more pointers from Rseding, I made it up to and including allowedCommands. I'm pretty sure I have those values right, and my offset is correct after I've read those (at least the version number always comes out matching the one at the very beginning of the file)
The stats are a mystery to me, though. According to Rseding, it's a sequence of three variable-length complex types, but I cannot make heads or tails of how to determine their length. Can you help out once more?
Re: level.dat format
Posted: Fri May 19, 2017 10:14 pm
by mickael9
After some reverse engineering I wrote this parser in Python, it should work with all saves that can be loaded by Factorio 0.15.
https://gist.github.com/mickael9/5dbdb9 ... f0cc1d5a9f
Sample output:
Code: Select all
/home/mickael/.factorio/saves/test.zip:
base_mod: base
campaign: ''
current_version: 0.15.11
difficulty: 1
finished: false
mods:
base: 0.15.11
creative-mode: 0.3.3
cut-and-paste: 0.1.2
name: freeplay
next_level: ''
player_won: false
stats: {}
version: 0.15.11.2
This only parses mods and everything that comes before in the save file.
Re: level.dat format
Posted: Sun Apr 15, 2018 1:20 pm
by mickael9
Updated for 0.16
Code: Select all
from zipfile import ZipFile
from struct import Struct
class Deserializer:
u16 = Struct('<H')
u32 = Struct('<I')
def __init__(self, stream):
self.stream = stream
self.version = tuple(self.read_u16() for i in range(4))
def read(self, n):
return self.stream.read(n)
def read_fmt(self, fmt):
return fmt.unpack(self.read(fmt.size))[0]
def read_u8(self):
return self.read(1)[0]
def read_bool(self):
return bool(self.read_u8())
def read_u16(self):
return self.read_fmt(self.u16)
def read_u32(self):
return self.read_fmt(self.u32)
def read_str(self, dtype=None):
if self.version >= (0, 16, 0, 0):
length = self.read_optim(dtype or self.u32)
else:
length = self.read_fmt(dtype or self.u32)
return self.read(length).decode('utf-8')
def read_optim(self, dtype):
if self.version >= (0, 14, 14, 0):
byte = self.read_u8()
if byte != 0xFF:
return byte
return self.read_fmt(dtype)
def read_optim_u16(self):
return self.read_optim(self.u16)
def read_optim_u32(self):
return self.read_optim(self.u32)
def read_optim_str(self):
length = self.read_optim_u32()
return self.read(length).decode('utf-8')
def read_optim_tuple(self, dtype, num):
return tuple(self.read_optim(dtype) for i in range(num))
class SaveFile:
def __init__(self, filename):
zf = ZipFile(filename, 'r')
datfile = None
for f in zf.namelist():
if f.endswith('/level.dat'):
datfile = f
break
if not datfile:
raise IOError("level.dat not found in save file")
ds = Deserializer(zf.open(datfile))
self.version = self.version_str(ds.version)
self.campaign = ds.read_str()
self.name = ds.read_str()
self.base_mod = ds.read_str()
# 0: Normal, 1: Old School, 2: Hardcore
self.difficulty = ds.read_u8()
self.finished = ds.read_bool()
self.player_won = ds.read_bool()
self.next_level = ds.read_str() # usually empty
if ds.version >= (0, 12, 0, 0):
self.can_continue = ds.read_bool()
self.finished_but_continuing = ds.read_bool()
self.saving_replay = ds.read_bool()
if ds.version >= (0, 16, 0, 0):
self.allow_non_admin_debug_options = ds.read_bool()
self.loaded_from = self.version_str(ds.read_optim_tuple(ds.u16, 3))
self.loaded_from_build = ds.read_u16()
self.allowed_commands = ds.read_u8()
if ds.version <= (0, 13, 0, 87):
if not self.allowed_commands:
self.allowed_commands = 2
else:
self.allowed_commands = 1
self.stats = {}
if ds.version <= (0, 13, 0, 42):
num_stats = ds.read_u32()
for i in range(num_stats):
force_id = ds.read_u8()
self.stats[force_id] = []
for j in range(3):
st = {}
length = ds.read_u32()
for k in range(length):
k = ds.read_u16()
v = ds.read_u32()
st[k] = v
self.stats[force_id].append(st)
self.mods = {}
if ds.version >= (0, 16, 0, 0):
num_mods = ds.read_optim_u32()
else:
num_mods = ds.read_u32()
for i in range(num_mods):
name = ds.read_optim_str()
version = ds.read_optim_tuple(ds.u16, 3)
if ds.version > (0, 15, 0, 91):
ds.read_u32() # CRC
self.mods[name] = self.version_str(version)
@staticmethod
def version_str(ver):
return '.'.join(str(x) for x in ver)
if __name__ == '__main__':
import sys
try:
from yaml import safe_dump
except ImportError:
print('Install PyYAML for pretty printing')
def safe_dump(s, **kw):
return repr(s)
for name in sys.argv[1:]:
sf = SaveFile(name)
print('%s:' % name)
print()
print(safe_dump(sf.__dict__, default_flow_style=False))
print('---')
Re: level.dat format
Posted: Fri Jan 04, 2019 4:16 am
by Siorai
Oh -awesome- ! Great job on this btw. ( Verified working on Python3.7 in case you were wondering ). By chance gonna update it for .17 ? ( Once it's out that is... )
mickael9 wrote: ↑Sun Apr 15, 2018 1:20 pm
Updated for 0.16
Code: Select all
from zipfile import ZipFile
from struct import Struct
class Deserializer:
u16 = Struct('<H')
u32 = Struct('<I')
def __init__(self, stream):
self.stream = stream
self.version = tuple(self.read_u16() for i in range(4))
def read(self, n):
return self.stream.read(n)
def read_fmt(self, fmt):
return fmt.unpack(self.read(fmt.size))[0]
def read_u8(self):
return self.read(1)[0]
def read_bool(self):
return bool(self.read_u8())
def read_u16(self):
return self.read_fmt(self.u16)
def read_u32(self):
return self.read_fmt(self.u32)
def read_str(self, dtype=None):
if self.version >= (0, 16, 0, 0):
length = self.read_optim(dtype or self.u32)
else:
length = self.read_fmt(dtype or self.u32)
return self.read(length).decode('utf-8')
def read_optim(self, dtype):
if self.version >= (0, 14, 14, 0):
byte = self.read_u8()
if byte != 0xFF:
return byte
return self.read_fmt(dtype)
def read_optim_u16(self):
return self.read_optim(self.u16)
def read_optim_u32(self):
return self.read_optim(self.u32)
def read_optim_str(self):
length = self.read_optim_u32()
return self.read(length).decode('utf-8')
def read_optim_tuple(self, dtype, num):
return tuple(self.read_optim(dtype) for i in range(num))
class SaveFile:
def __init__(self, filename):
zf = ZipFile(filename, 'r')
datfile = None
for f in zf.namelist():
if f.endswith('/level.dat'):
datfile = f
break
if not datfile:
raise IOError("level.dat not found in save file")
ds = Deserializer(zf.open(datfile))
self.version = self.version_str(ds.version)
self.campaign = ds.read_str()
self.name = ds.read_str()
self.base_mod = ds.read_str()
# 0: Normal, 1: Old School, 2: Hardcore
self.difficulty = ds.read_u8()
self.finished = ds.read_bool()
self.player_won = ds.read_bool()
self.next_level = ds.read_str() # usually empty
if ds.version >= (0, 12, 0, 0):
self.can_continue = ds.read_bool()
self.finished_but_continuing = ds.read_bool()
self.saving_replay = ds.read_bool()
if ds.version >= (0, 16, 0, 0):
self.allow_non_admin_debug_options = ds.read_bool()
self.loaded_from = self.version_str(ds.read_optim_tuple(ds.u16, 3))
self.loaded_from_build = ds.read_u16()
self.allowed_commands = ds.read_u8()
if ds.version <= (0, 13, 0, 87):
if not self.allowed_commands:
self.allowed_commands = 2
else:
self.allowed_commands = 1
self.stats = {}
if ds.version <= (0, 13, 0, 42):
num_stats = ds.read_u32()
for i in range(num_stats):
force_id = ds.read_u8()
self.stats[force_id] = []
for j in range(3):
st = {}
length = ds.read_u32()
for k in range(length):
k = ds.read_u16()
v = ds.read_u32()
st[k] = v
self.stats[force_id].append(st)
self.mods = {}
if ds.version >= (0, 16, 0, 0):
num_mods = ds.read_optim_u32()
else:
num_mods = ds.read_u32()
for i in range(num_mods):
name = ds.read_optim_str()
version = ds.read_optim_tuple(ds.u16, 3)
if ds.version > (0, 15, 0, 91):
ds.read_u32() # CRC
self.mods[name] = self.version_str(version)
@staticmethod
def version_str(ver):
return '.'.join(str(x) for x in ver)
if __name__ == '__main__':
import sys
try:
from yaml import safe_dump
except ImportError:
print('Install PyYAML for pretty printing')
def safe_dump(s, **kw):
return repr(s)
for name in sys.argv[1:]:
sf = SaveFile(name)
print('%s:' % name)
print()
print(safe_dump(sf.__dict__, default_flow_style=False))
print('---')