Friday facts #157 - We are able to eat paper, but we don't do it

Regular reports on Factorio development.
kovarex
Factorio Staff
Factorio Staff
Posts: 8078
Joined: Wed Feb 06, 2013 12:00 am
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by kovarex »

There will be 1 liquid container with 3 points to load/unload it. The graphics might be little confusing.

Xeteth
Fast Inserter
Fast Inserter
Posts: 158
Joined: Tue Feb 17, 2015 6:06 am
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by Xeteth »

kovarex wrote: We changed to 4 bytes:
x (1 byte, 5 bits for tile, 3 bits for the rest)
y (1 byte)
decorative id (1 byte)
count (1 byte)

With the count added, we could save small chunks (I estimate 5-10) of decoratives as one.
Sounds like a great optimization! Out of curiosity since the entities are no longer saved in the save file I am assuming this will reduce save file size (which is great for MP!) - are you able to say approximately how much the save file might be reduced by (as a percentage on average perhaps)?

kovarex
Factorio Staff
Factorio Staff
Posts: 8078
Joined: Wed Feb 06, 2013 12:00 am
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by kovarex »

Xeteth wrote:
kovarex wrote: We changed to 4 bytes:
x (1 byte, 5 bits for tile, 3 bits for the rest)
y (1 byte)
decorative id (1 byte)
count (1 byte)

With the count added, we could save small chunks (I estimate 5-10) of decoratives as one.
Sounds like a great optimization! Out of curiosity since the entities are no longer saved in the save file I am assuming this will reduce save file size (which is great for MP!) - are you able to say approximately how much the save file might be reduced by (as a percentage on average perhaps)?
We decided that it stays as part of the save file, but the decoratives itself should be reduced at least 10 times in the save file and more than 30 times in the memory.
The overall save file reduction depends on how many of those are there.

henry1992
Manual Inserter
Manual Inserter
Posts: 1
Joined: Sat Oct 08, 2016 6:39 am
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by henry1992 »

How fast exactly? What fps do you have? (When you hit F5, it should be in the top right corner)

I will be honest, the game speed is determined by the refresh rate of display (yes lame, but this is temporary solution), until we do multithreading.
Multithreading will not only let us set the game speed more properly, it will also allow having MUCH larger factories without lag.

bman212121
Fast Inserter
Fast Inserter
Posts: 141
Joined: Mon Apr 18, 2016 8:08 pm
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by bman212121 »

kovarex wrote:
Xeteth wrote:
kovarex wrote: We changed to 4 bytes:
x (1 byte, 5 bits for tile, 3 bits for the rest)
y (1 byte)
decorative id (1 byte)
count (1 byte)

With the count added, we could save small chunks (I estimate 5-10) of decoratives as one.
Sounds like a great optimization! Out of curiosity since the entities are no longer saved in the save file I am assuming this will reduce save file size (which is great for MP!) - are you able to say approximately how much the save file might be reduced by (as a percentage on average perhaps)?
We decided that it stays as part of the save file, but the decoratives itself should be reduced at least 10 times in the save file and more than 30 times in the memory.
The overall save file reduction depends on how many of those are there.
A bit late to the party for this one, but I'm catching back up with all of the changes. Thank you very much for seeing and making this change! I was really wondering why the saves were starting to balloon out so much and figured there had to be something to do to make this better. Even for servers with only 4 or 5 players this is a substantial difference. As people stated I put the map size around 80MB if you were to just walk around the perimeter of one screen on a map. (Hit M for map, zoom all the way out, and outline one full screen by exploring around the edge) Unless you have fiber, a 10mbit upload means each person would have to wait over a minute just to download the map. Thankfully it's async now so it's not as big of a problem as it used to be. If that same map size is now cut in half as people are reporting, then that is a significant help and I can't wait to test to see if the game runs smoother with a map of that size.

AndrolGenhald
Inserter
Inserter
Posts: 47
Joined: Tue Mar 22, 2016 6:35 pm
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by AndrolGenhald »

Don't know if people are still reading this, but while you're doing the decoratives optimizations would you consider optimizing paths (as in brick and concrete) as well? Given the save file size changes after placing down a ton of concrete it seems like they're implemented as entities like the decoratives are, and I can't be the only one who likes to go a bit crazy placing concrete in gigantic rectangles. Theoretically, given that concrete tends to be placed in rectangles, and checking the intersection of rectangles is pretty easy, storage size for concrete could be nearly constant if it were possible to store a set of rectangles instead of individual entities. Even doing something like having a boolean for each 32x32 block to indicate that it's covered in concrete instead of storing each tile individually would be amazing.

Rseding91
Factorio Staff
Factorio Staff
Posts: 13175
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by Rseding91 »

AndrolGenhald wrote:Don't know if people are still reading this, but while you're doing the decoratives optimizations would you consider optimizing paths (as in brick and concrete) as well? Given the save file size changes after placing down a ton of concrete it seems like they're implemented as entities like the decoratives are, and I can't be the only one who likes to go a bit crazy placing concrete in gigantic rectangles. Theoretically, given that concrete tends to be placed in rectangles, and checking the intersection of rectangles is pretty easy, storage size for concrete could be nearly constant if it were possible to store a set of rectangles instead of individual entities. Even doing something like having a boolean for each 32x32 block to indicate that it's covered in concrete instead of storing each tile individually would be amazing.
Paths aren't implemented like entities. For each path you put down it adds 9 bytes (before compression) to the save file. So, even if you pave a 1000x1000 area that's only 8789 KB or 8.58 MB of space before compression reduces it to something below 1 MB. Decorative entities on the other hand add something like 144~ bytes per one in the world.
If you want to get ahold of me I'm almost always on Discord.

AndrolGenhald
Inserter
Inserter
Posts: 47
Joined: Tue Mar 22, 2016 6:35 pm
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by AndrolGenhald »

Rseding91 wrote:
AndrolGenhald wrote:Don't know if people are still reading this, but while you're doing the decoratives optimizations would you consider optimizing paths (as in brick and concrete) as well? Given the save file size changes after placing down a ton of concrete it seems like they're implemented as entities like the decoratives are, and I can't be the only one who likes to go a bit crazy placing concrete in gigantic rectangles. Theoretically, given that concrete tends to be placed in rectangles, and checking the intersection of rectangles is pretty easy, storage size for concrete could be nearly constant if it were possible to store a set of rectangles instead of individual entities. Even doing something like having a boolean for each 32x32 block to indicate that it's covered in concrete instead of storing each tile individually would be amazing.
Paths aren't implemented like entities. For each path you put down it adds 9 bytes (before compression) to the save file. So, even if you pave a 1000x1000 area that's only 8789 KB or 8.58 MB of space before compression reduces it to something below 1 MB. Decorative entities on the other hand add something like 144~ bytes per one in the world.
I don't know how you guys manage to respond so quickly to everything and make such a great game at the same time. Thanks for the info though, I'll do a little more thorough research next time before posting.

kovarex
Factorio Staff
Factorio Staff
Posts: 8078
Joined: Wed Feb 06, 2013 12:00 am
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by kovarex »

Rseding91 wrote:
AndrolGenhald wrote:Don't know if people are still reading this, but while you're doing the decoratives optimizations would you consider optimizing paths (as in brick and concrete) as well? Given the save file size changes after placing down a ton of concrete it seems like they're implemented as entities like the decoratives are, and I can't be the only one who likes to go a bit crazy placing concrete in gigantic rectangles. Theoretically, given that concrete tends to be placed in rectangles, and checking the intersection of rectangles is pretty easy, storage size for concrete could be nearly constant if it were possible to store a set of rectangles instead of individual entities. Even doing something like having a boolean for each 32x32 block to indicate that it's covered in concrete instead of storing each tile individually would be amazing.
Paths aren't implemented like entities. For each path you put down it adds 9 bytes (before compression) to the save file. So, even if you pave a 1000x1000 area that's only 8789 KB or 8.58 MB of space before compression reduces it to something below 1 MB. Decorative entities on the other hand add something like 144~ bytes per one in the world.
Are you mixing ram usage and save usage? One decorative entity certainly doesn't take 144 of save space, while one tile takes more than 9 bytes of ram.
I would actually remove the "under the tiles" data from the save file.

hoho
Filter Inserter
Filter Inserter
Posts: 677
Joined: Sat Jan 18, 2014 11:23 am
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by hoho »

kovarex wrote:I would actually remove the "under the tiles" data from the save file.
You mean replacing the ground tile from sand/grass/dirt to concrete/stone instead of storing both?

Wouldn't it cause issues when player deconstructs concrete/stone floors or can you just re-create the data on demand?

AndrolGenhald
Inserter
Inserter
Posts: 47
Joined: Tue Mar 22, 2016 6:35 pm
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by AndrolGenhald »

hoho wrote:
kovarex wrote:I would actually remove the "under the tiles" data from the save file.
You mean replacing the ground tile from sand/grass/dirt to concrete/stone instead of storing both?

Wouldn't it cause issues when player deconstructs concrete/stone floors or can you just re-create the data on demand?
They clearly have a way to deterministically generate terrain when exploring, I'm guessing all they really have to do is re-generate it from the same seed. It's actually a pretty good idea, wish I'd have thought of it. Paths would basically be another terrain type (storage wise, not necessarily implementation wise), when you place a path instead of placing it on top of terrain you just replace the terrain, so you don't have to store any extra data.

kovarex
Factorio Staff
Factorio Staff
Posts: 8078
Joined: Wed Feb 06, 2013 12:00 am
Contact:

Re: Friday facts #157 - We are able to eat paper, but we don't do it

Post by kovarex »

AndrolGenhald wrote:
hoho wrote:
kovarex wrote:I would actually remove the "under the tiles" data from the save file.
You mean replacing the ground tile from sand/grass/dirt to concrete/stone instead of storing both?

Wouldn't it cause issues when player deconstructs concrete/stone floors or can you just re-create the data on demand?
They clearly have a way to deterministically generate terrain when exploring, I'm guessing all they really have to do is re-generate it from the same seed. It's actually a pretty good idea, wish I'd have thought of it. Paths would basically be another terrain type (storage wise, not necessarily implementation wise), when you place a path instead of placing it on top of terrain you just replace the terrain, so you don't have to store any extra data.
This is how it always worked, concrete is tile like any other. The problem is, that the tile that appeared under the concrete could be completely different due to the tile correction logic. But I would say that this is not important at all.

Post Reply

Return to “News”