[0.13.13] Save file size increased ~5 times

[madpav3l]

I don't know why but all of a sudden my loading and saving times took very long time than usual. Instead of 1-2s loading is now 6s loading and instead of 1s saving time is now 4s... and before you ask, it's not my slow computer (i7-6700K@4.6GHz and SSD 500+ MB/s read/write speed, 90000+ IOPS).

So I discovered that the save file is all of a sudded almost 70 MB big (Temp - download) while not so long ago it was 15 MB or something like that. Bad is I don't have a backup before this happened
I am using some mods (0.9 MB download) mostly quality of life improvements nothing game changing, below is a list, but even if I disable all of them nothing changes.

Code: Select all

autofill_1.4.1.zip
AutoTrash_0.1.7.zip
Enhanced_Map_Colors_1.3.0.zip
EvoGUI_0.4.107.zip
FARL_0.5.36.zip
filtered-deconstruction-planner_0.4.6.zip
KBlueprints_1.1.8.zip
long-reach_0.0.5.zip
mod-list.json
ModuleInserter_0.2.1.zip
More signal color_0.0.1.zip
rso-mod_2.0.11.zip
RSORadar_0.0.1.zip
ScoreExtended_1.0.14.zip
SmallFixes_0.1.21.zip
TheFatController_0.4.18.zip
tree_collision_0.3.0.zip
upgrade-planner_1.2.4.zip

And if I compare it to my old base, which is a 41 MB save (__1st Base - download) and the base approx. 5 times bigger (1.2 GW energy consumption vs 260 MW) it does not make any sence... something must be wrong.
So I hope you can find something in the save what makes it so big and make it small again.

[Klonan]

Thanks for the report,

Your map has a huge area generated, which im guessing is due to the RSO radar mod,
And you have over 23,000 chunks on your map

As for the size difference, your Temp map has a lot more entities compared to your first base, 12,638,881 in Temp and 3,347,332 in first base

So i don't think this is a bug, you just have a big map

[madpav3l]

If I may ask what is classified as entity, it does not make any sence to me... the map is big empty of nothing.

The map revealed is smaller in Temp vs 1st base (27,335 vs 23,024) but there is still more entities 12,000,000 vs 3,150,000 while I have build allmost nothing and there are no biters, I have killed them all while exploring on foot, that's why the map is revealed a lot even RSO radar does not scan that far.

And yesterday when I played it was all fine... normal loading time, autosave almost unnoticeable with the same map size.

[Klonan]

If I may ask what is classified as entity, it does not make any sence to me... the map is big empty of nothing.

The map revealed is smaller in Temp vs 1st base (27,335 vs 23,024) but there is still more entities 12,000,000 vs 3,150,000 while I have build allmost nothing and there are no biters, I have killed them all while exploring on foot, that's why the map is revealed a lot even RSO radar does not scan that far.

And yesterday when I played it was all fine... normal loading time, autosave almost unnoticeable with the same map size.

Everything that isn't a tile or an item on the ground is an entity, this includes trees, biters, doodads, decorations, rocks, fish, resources and player entities

[orzelek]

If I may ask what is classified as entity, it does not make any sence to me... the map is big empty of nothing.

The map revealed is smaller in Temp vs 1st base (27,335 vs 23,024) but there is still more entities 12,000,000 vs 3,150,000 while I have build allmost nothing and there are no biters, I have killed them all while exploring on foot, that's why the map is revealed a lot even RSO radar does not scan that far.

And yesterday when I played it was all fine... normal loading time, autosave almost unnoticeable with the same map size.

Everything that isn't a tile or an item on the ground is an entity, this includes trees, biters, doodads, decorations, rocks, fish, resources and player entities

I went into curious mode and this map shows a bit of a map gen issue.
If you have a lot of desert... your decorations or doodads (not sure which are the shrubs/grass like plants) will count in 10's of thousands at least. It seems... excessive. You can get 5 of those per tile on quite big parts of desert.
And player has no in game means to remove those I think - maybe concrete paving?

[madpav3l]

If I may ask what is classified as entity, it does not make any sence to me... the map is big empty of nothing.

The map revealed is smaller in Temp vs 1st base (27,335 vs 23,024) but there is still more entities 12,000,000 vs 3,150,000 while I have build allmost nothing and there are no biters, I have killed them all while exploring on foot, that's why the map is revealed a lot even RSO radar does not scan that far.

And yesterday when I played it was all fine... normal loading time, autosave almost unnoticeable with the same map size.

Everything that isn't a tile or an item on the ground is an entity, this includes trees, biters, doodads, decorations, rocks, fish, resources and player entities

I went into curious mode and this map shows a bit of a map gen issue.
If you have a lot of desert... your decorations or doodads (not sure which are the shrubs/grass like plants) will count in 10's of thousands at least. It seems... excessive. You can get 5 of those per tile on quite big parts of desert.
And player has no in game means to remove those I think - maybe concrete paving?

Well you are not wrong, the map generation causes this problem.
I did some tests and I you can generete a map in 0.12.35 and if you reveal the map with

/c game.player.force.chart(game.player.surface,{{x = -2500, y = -2500}, {x = 2500, y = 2500}})

which is 26,244 chunks, it will always have around 3,000,000 entities and save file around +/- 25 MB and if you do the same in 0.13.13 you get a map with 13,000,000+ entities and save file +/- 75MB.
The map settings are not that important, see screens below.

No offence Klonan, but I think someone with a knowledge of the game code should look at it...

[Loewchen]

I did some tests and I you can generete a map in 0.12.35 and if you reveal the map with

/c game.player.force.chart(game.player.surface,{{x = -2500, y = -2500}, {x = 2500, y = 2500}})

which is 26,244 chunks, it will always have around 3,000,000 entities and save file around +/- 25 MB and if you do the same in 0.13.13 you get a map with 13,000,000+ entities and save file +/- 75MB.
The map settings are not that important, see screens below.

No offence Klonan, but I think someone with a knowledge of the game code should look at it...

As orzelek said, the amount of desert is crucial for the savegame size and as the .13 map creation of yours has those and the .12 one does not it explains the difference while not being a bug.

[orzelek]

I would still treat it as very suspicious behavior.

Why player with desert map should have 2-3 times save file size vs player with mostly grass map?
Terrain in game had no that big function till now and with this issue it means that desert is now both faster pollution spread and much bigger save file. This will also negatively affect multiplayer map uploads.

[madpav3l]

As orzelek said, the amount of desert is crucial for the savegame size and as the .13 map creation of yours has those and the .12 one does not it explains the difference while not being a bug.

It is not really just desert related in my opinion... if you watch the entities number while the map is revealed by the command, it rises gradually revealing green land or desert.
Below is a desert map from 0.12.35 - 1,600,000 entities.

2016-08-05_00-43-43.png (248.3 KiB) Viewed 6393 times

Also scary is that now if you wanna build a megabase with RSO the save will be huge.
Map from below is from 0.12.XX, 85 MB save (not mine) 1 min. rocket base with 96,000 chunks and 4,600,000 entities. With the new behavior it would be like 50,000,000+ entites and +/- 300 MB save...

2016-08-05_00-41-41.png (1.12 MiB) Viewed 6393 times

I would still treat it as very suspicious behavior.

Why player with desert map should have 2-3 times save file size vs player with mostly grass map?
Terrain in game had no that big function till now and with this issue it means that desert is now both faster pollution spread and much bigger save file. This will also negatively affect multiplayer map uploads.

That's a good point!

[Loewchen]

I absolutely agree that the increase is a unfavourable development. My point is, that I don't see that it is caused by an error in the compression algorithm or the map generator, but rather a side effect of deliberate change of the later, so we might have to wait until savegame size reduction gets on the agenda again.

[orzelek]

I'll give an example then from the save thats attached to this bug report.
It's initial entity count is about 12 mil. After removal of all decorations this number drops to about 1.7 mil. Size of save file is reduced to about 25MB.

I also made small mod (named undecorator) that will do the cleanup and prevent more decorations from spawning.

[madpav3l]

I'll give an example then from the save thats attached to this bug report.
It's initial entity count is about 12 mil. After removal of all decorations this number drops to about 1.7 mil. Size of save file is reduced to about 25MB.

I also made small mod (named undecorator) that will do the cleanup and prevent more decorations from spawning.

That's great, thanks orzelek!