[0.15.28] Headless does not release temp/mp-save*.zip files

[psihius]

We have a long-term 24/7 server that we run and maps are big, so MP saves way a lot, and being the FactorioMMO server - there is a lot of traffic.
Today I runned into the issue that the server does not release the mp-save-xxx.zip files, so after deletion they still take space.

I have a cron command running that clears out temporary file data.

Code: Select all

*/10 * * * * find /var/factorio/instances/*/temp -name '*.zip' -mmin +60 -delete

This is my current state:

Code: Select all

root at factoriommo in /var/factorio/instances/rpg
$ lsof | grep "/var" | grep deleted
factorio  11597         factorio    9r      REG                8,1 740927424     392999 /var/factorio/instances/rpg/temp/mp-save-204.zip (deleted)
factorio  11597         factorio   12r      REG                8,1 733000903     392991 /var/factorio/instances/rpg/temp/mp-save-200.zip (deleted)
factorio  11597         factorio   14r      REG                8,1 739463086     393892 /var/factorio/instances/rpg/temp/mp-save-214.zip (deleted)
factorio  11597         factorio   15r      REG                8,1 740622323     394298 /var/factorio/instances/rpg/temp/mp-save-208.zip (deleted)
factorio  11597         factorio   16r      REG                8,1 148645745     394309 /var/factorio/instances/rpg/temp/mp-save-209.zip (deleted)
factorio  11597         factorio   17r      REG                8,1 739485805     394140 /var/factorio/instances/rpg/temp/mp-save-218.zip (deleted)
factorio  11597         factorio   18r      REG                8,1 148248114     394327 /var/factorio/instances/rpg/temp/mp-save-219.zip (deleted)
factorio  11597         factorio   19r      REG                8,1 743812915     393998 /var/factorio/instances/rpg/temp/mp-save-226.zip (deleted)
factorio  11597         factorio   20r      REG                8,1 153675729     394355 /var/factorio/instances/rpg/temp/mp-save-253.zip (deleted)
factorio  11597         factorio   21r      REG                8,1 784528542     392495 /var/factorio/instances/rpg/temp/mp-save-265.zip (deleted)
factorio  11597         factorio   22r      REG                8,1 787961774     393041 /var/factorio/instances/rpg/temp/mp-save-274.zip (deleted)
factorio  11597         factorio   23r      REG                8,1 787689839     393045 /var/factorio/instances/rpg/temp/mp-save-275.zip (deleted)
factorio  11597         factorio   24r      REG                8,1 786289608     393046 /var/factorio/instances/rpg/temp/mp-save-277.zip (deleted)
factorio  11597         factorio   25r      REG                8,1 794639232     394445 /var/factorio/instances/rpg/temp/mp-save-283.zip (deleted)
factorio  11597         factorio   26r      REG                8,1 798340746     393054 /var/factorio/instances/rpg/temp/mp-save-286.zip (deleted)
factorio  11597 11605   factorio    9r      REG                8,1 740927424     392999 /var/factorio/instances/rpg/temp/mp-save-204.zip (deleted)
factorio  11597 11605   factorio   12r      REG                8,1 733000903     392991 /var/factorio/instances/rpg/temp/mp-save-200.zip (deleted)
factorio  11597 11605   factorio   14r      REG                8,1 739463086     393892 /var/factorio/instances/rpg/temp/mp-save-214.zip (deleted)
factorio  11597 11605   factorio   15r      REG                8,1 740622323     394298 /var/factorio/instances/rpg/temp/mp-save-208.zip (deleted)
factorio  11597 11605   factorio   16r      REG                8,1 148645745     394309 /var/factorio/instances/rpg/temp/mp-save-209.zip (deleted)
factorio  11597 11605   factorio   17r      REG                8,1 739485805     394140 /var/factorio/instances/rpg/temp/mp-save-218.zip (deleted)
factorio  11597 11605   factorio   18r      REG                8,1 148248114     394327 /var/factorio/instances/rpg/temp/mp-save-219.zip (deleted)
factorio  11597 11605   factorio   19r      REG                8,1 743812915     393998 /var/factorio/instances/rpg/temp/mp-save-226.zip (deleted)
factorio  11597 11605   factorio   20r      REG                8,1 153675729     394355 /var/factorio/instances/rpg/temp/mp-save-253.zip (deleted)
factorio  11597 11605   factorio   21r      REG                8,1 784528542     392495 /var/factorio/instances/rpg/temp/mp-save-265.zip (deleted)
factorio  11597 11605   factorio   22r      REG                8,1 787961774     393041 /var/factorio/instances/rpg/temp/mp-save-274.zip (deleted)
factorio  11597 11605   factorio   23r      REG                8,1 787689839     393045 /var/factorio/instances/rpg/temp/mp-save-275.zip (deleted)
factorio  11597 11605   factorio   24r      REG                8,1 786289608     393046 /var/factorio/instances/rpg/temp/mp-save-277.zip (deleted)
factorio  11597 11605   factorio   25r      REG                8,1 794639232     394445 /var/factorio/instances/rpg/temp/mp-save-283.zip (deleted)
factorio  11597 11605   factorio   26r      REG                8,1 798340746     393054 /var/factorio/instances/rpg/temp/mp-save-286.zip (deleted)
factorio  11597 11606   factorio    9r      REG                8,1 740927424     392999 /var/factorio/instances/rpg/temp/mp-save-204.zip (deleted)
factorio  11597 11606   factorio   12r      REG                8,1 733000903     392991 /var/factorio/instances/rpg/temp/mp-save-200.zip (deleted)
factorio  11597 11606   factorio   14r      REG                8,1 739463086     393892 /var/factorio/instances/rpg/temp/mp-save-214.zip (deleted)
factorio  11597 11606   factorio   15r      REG                8,1 740622323     394298 /var/factorio/instances/rpg/temp/mp-save-208.zip (deleted)
factorio  11597 11606   factorio   16r      REG                8,1 148645745     394309 /var/factorio/instances/rpg/temp/mp-save-209.zip (deleted)
factorio  11597 11606   factorio   17r      REG                8,1 739485805     394140 /var/factorio/instances/rpg/temp/mp-save-218.zip (deleted)
factorio  11597 11606   factorio   18r      REG                8,1 148248114     394327 /var/factorio/instances/rpg/temp/mp-save-219.zip (deleted)
factorio  11597 11606   factorio   19r      REG                8,1 743812915     393998 /var/factorio/instances/rpg/temp/mp-save-226.zip (deleted)
factorio  11597 11606   factorio   20r      REG                8,1 153675729     394355 /var/factorio/instances/rpg/temp/mp-save-253.zip (deleted)
factorio  11597 11606   factorio   21r      REG                8,1 784528542     392495 /var/factorio/instances/rpg/temp/mp-save-265.zip (deleted)
factorio  11597 11606   factorio   22r      REG                8,1 787961774     393041 /var/factorio/instances/rpg/temp/mp-save-274.zip (deleted)
factorio  11597 11606   factorio   23r      REG                8,1 787689839     393045 /var/factorio/instances/rpg/temp/mp-save-275.zip (deleted)
factorio  11597 11606   factorio   24r      REG                8,1 786289608     393046 /var/factorio/instances/rpg/temp/mp-save-277.zip (deleted)
factorio  11597 11606   factorio   25r      REG                8,1 794639232     394445 /var/factorio/instances/rpg/temp/mp-save-283.zip (deleted)
factorio  11597 11606   factorio   26r      REG                8,1 798340746     393054 /var/factorio/instances/rpg/temp/mp-save-286.zip (deleted)
factorio  11597 11607   factorio    9r      REG                8,1 740927424     392999 /var/factorio/instances/rpg/temp/mp-save-204.zip (deleted)
factorio  11597 11607   factorio   12r      REG                8,1 733000903     392991 /var/factorio/instances/rpg/temp/mp-save-200.zip (deleted)
factorio  11597 11607   factorio   14r      REG                8,1 739463086     393892 /var/factorio/instances/rpg/temp/mp-save-214.zip (deleted)
factorio  11597 11607   factorio   15r      REG                8,1 740622323     394298 /var/factorio/instances/rpg/temp/mp-save-208.zip (deleted)
factorio  11597 11607   factorio   16r      REG                8,1 148645745     394309 /var/factorio/instances/rpg/temp/mp-save-209.zip (deleted)
factorio  11597 11607   factorio   17r      REG                8,1 739485805     394140 /var/factorio/instances/rpg/temp/mp-save-218.zip (deleted)
factorio  11597 11607   factorio   18r      REG                8,1 148248114     394327 /var/factorio/instances/rpg/temp/mp-save-219.zip (deleted)
factorio  11597 11607   factorio   19r      REG                8,1 743812915     393998 /var/factorio/instances/rpg/temp/mp-save-226.zip (deleted)
factorio  11597 11607   factorio   20r      REG                8,1 153675729     394355 /var/factorio/instances/rpg/temp/mp-save-253.zip (deleted)
factorio  11597 11607   factorio   21r      REG                8,1 784528542     392495 /var/factorio/instances/rpg/temp/mp-save-265.zip (deleted)
factorio  11597 11607   factorio   22r      REG                8,1 787961774     393041 /var/factorio/instances/rpg/temp/mp-save-274.zip (deleted)
factorio  11597 11607   factorio   23r      REG                8,1 787689839     393045 /var/factorio/instances/rpg/temp/mp-save-275.zip (deleted)
factorio  11597 11607   factorio   24r      REG                8,1 786289608     393046 /var/factorio/instances/rpg/temp/mp-save-277.zip (deleted)
factorio  11597 11607   factorio   25r      REG                8,1 794639232     394445 /var/factorio/instances/rpg/temp/mp-save-283.zip (deleted)
factorio  11597 11607   factorio   26r      REG                8,1 798340746     393054 /var/factorio/instances/rpg/temp/mp-save-286.zip (deleted)
factorio  11597 11608   factorio    9r      REG                8,1 740927424     392999 /var/factorio/instances/rpg/temp/mp-save-204.zip (deleted)
factorio  11597 11608   factorio   12r      REG                8,1 733000903     392991 /var/factorio/instances/rpg/temp/mp-save-200.zip (deleted)
factorio  11597 11608   factorio   14r      REG                8,1 739463086     393892 /var/factorio/instances/rpg/temp/mp-save-214.zip (deleted)
factorio  11597 11608   factorio   15r      REG                8,1 740622323     394298 /var/factorio/instances/rpg/temp/mp-save-208.zip (deleted)
factorio  11597 11608   factorio   16r      REG                8,1 148645745     394309 /var/factorio/instances/rpg/temp/mp-save-209.zip (deleted)
factorio  11597 11608   factorio   17r      REG                8,1 739485805     394140 /var/factorio/instances/rpg/temp/mp-save-218.zip (deleted)
factorio  11597 11608   factorio   18r      REG                8,1 148248114     394327 /var/factorio/instances/rpg/temp/mp-save-219.zip (deleted)
factorio  11597 11608   factorio   19r      REG                8,1 743812915     393998 /var/factorio/instances/rpg/temp/mp-save-226.zip (deleted)
factorio  11597 11608   factorio   20r      REG                8,1 153675729     394355 /var/factorio/instances/rpg/temp/mp-save-253.zip (deleted)
factorio  11597 11608   factorio   21r      REG                8,1 784528542     392495 /var/factorio/instances/rpg/temp/mp-save-265.zip (deleted)
factorio  11597 11608   factorio   22r      REG                8,1 787961774     393041 /var/factorio/instances/rpg/temp/mp-save-274.zip (deleted)
factorio  11597 11608   factorio   23r      REG                8,1 787689839     393045 /var/factorio/instances/rpg/temp/mp-save-275.zip (deleted)
factorio  11597 11608   factorio   24r      REG                8,1 786289608     393046 /var/factorio/instances/rpg/temp/mp-save-277.zip (deleted)
factorio  11597 11608   factorio   25r      REG                8,1 794639232     394445 /var/factorio/instances/rpg/temp/mp-save-283.zip (deleted)
factorio  11597 11608   factorio   26r      REG                8,1 798340746     393054 /var/factorio/instances/rpg/temp/mp-save-286.zip (deleted)
factorio  11597 11609   factorio    9r      REG                8,1 740927424     392999 /var/factorio/instances/rpg/temp/mp-save-204.zip (deleted)
factorio  11597 11609   factorio   12r      REG                8,1 733000903     392991 /var/factorio/instances/rpg/temp/mp-save-200.zip (deleted)
factorio  11597 11609   factorio   14r      REG                8,1 739463086     393892 /var/factorio/instances/rpg/temp/mp-save-214.zip (deleted)
factorio  11597 11609   factorio   15r      REG                8,1 740622323     394298 /var/factorio/instances/rpg/temp/mp-save-208.zip (deleted)
factorio  11597 11609   factorio   16r      REG                8,1 148645745     394309 /var/factorio/instances/rpg/temp/mp-save-209.zip (deleted)
factorio  11597 11609   factorio   17r      REG                8,1 739485805     394140 /var/factorio/instances/rpg/temp/mp-save-218.zip (deleted)
factorio  11597 11609   factorio   18r      REG                8,1 148248114     394327 /var/factorio/instances/rpg/temp/mp-save-219.zip (deleted)
factorio  11597 11609   factorio   19r      REG                8,1 743812915     393998 /var/factorio/instances/rpg/temp/mp-save-226.zip (deleted)
factorio  11597 11609   factorio   20r      REG                8,1 153675729     394355 /var/factorio/instances/rpg/temp/mp-save-253.zip (deleted)
factorio  11597 11609   factorio   21r      REG                8,1 784528542     392495 /var/factorio/instances/rpg/temp/mp-save-265.zip (deleted)
factorio  11597 11609   factorio   22r      REG                8,1 787961774     393041 /var/factorio/instances/rpg/temp/mp-save-274.zip (deleted)
factorio  11597 11609   factorio   23r      REG                8,1 787689839     393045 /var/factorio/instances/rpg/temp/mp-save-275.zip (deleted)
factorio  11597 11609   factorio   24r      REG                8,1 786289608     393046 /var/factorio/instances/rpg/temp/mp-save-277.zip (deleted)
factorio  11597 11609   factorio   25r      REG                8,1 794639232     394445 /var/factorio/instances/rpg/temp/mp-save-283.zip (deleted)
factorio  11597 11609   factorio   26r      REG                8,1 798340746     393054 /var/factorio/instances/rpg/temp/mp-save-286.zip (deleted)
factorio  11597 11613   factorio    9r      REG                8,1 740927424     392999 /var/factorio/instances/rpg/temp/mp-save-204.zip (deleted)
factorio  11597 11613   factorio   12r      REG                8,1 733000903     392991 /var/factorio/instances/rpg/temp/mp-save-200.zip (deleted)
factorio  11597 11613   factorio   14r      REG                8,1 739463086     393892 /var/factorio/instances/rpg/temp/mp-save-214.zip (deleted)
factorio  11597 11613   factorio   15r      REG                8,1 740622323     394298 /var/factorio/instances/rpg/temp/mp-save-208.zip (deleted)
factorio  11597 11613   factorio   16r      REG                8,1 148645745     394309 /var/factorio/instances/rpg/temp/mp-save-209.zip (deleted)
factorio  11597 11613   factorio   17r      REG                8,1 739485805     394140 /var/factorio/instances/rpg/temp/mp-save-218.zip (deleted)
factorio  11597 11613   factorio   18r      REG                8,1 148248114     394327 /var/factorio/instances/rpg/temp/mp-save-219.zip (deleted)
factorio  11597 11613   factorio   19r      REG                8,1 743812915     393998 /var/factorio/instances/rpg/temp/mp-save-226.zip (deleted)
factorio  11597 11613   factorio   20r      REG                8,1 153675729     394355 /var/factorio/instances/rpg/temp/mp-save-253.zip (deleted)
factorio  11597 11613   factorio   21r      REG                8,1 784528542     392495 /var/factorio/instances/rpg/temp/mp-save-265.zip (deleted)
factorio  11597 11613   factorio   22r      REG                8,1 787961774     393041 /var/factorio/instances/rpg/temp/mp-save-274.zip (deleted)
factorio  11597 11613   factorio   23r      REG                8,1 787689839     393045 /var/factorio/instances/rpg/temp/mp-save-275.zip (deleted)
factorio  11597 11613   factorio   24r      REG                8,1 786289608     393046 /var/factorio/instances/rpg/temp/mp-save-277.zip (deleted)
factorio  11597 11613   factorio   25r      REG                8,1 794639232     394445 /var/factorio/instances/rpg/temp/mp-save-283.zip (deleted)
factorio  11597 11613   factorio   26r      REG                8,1 798340746     393054 /var/factorio/instances/rpg/temp/mp-save-286.zip (deleted)
factorio  11597 11626   factorio    9r      REG                8,1 740927424     392999 /var/factorio/instances/rpg/temp/mp-save-204.zip (deleted)
factorio  11597 11626   factorio   12r      REG                8,1 733000903     392991 /var/factorio/instances/rpg/temp/mp-save-200.zip (deleted)
factorio  11597 11626   factorio   14r      REG                8,1 739463086     393892 /var/factorio/instances/rpg/temp/mp-save-214.zip (deleted)
factorio  11597 11626   factorio   15r      REG                8,1 740622323     394298 /var/factorio/instances/rpg/temp/mp-save-208.zip (deleted)
factorio  11597 11626   factorio   16r      REG                8,1 148645745     394309 /var/factorio/instances/rpg/temp/mp-save-209.zip (deleted)
factorio  11597 11626   factorio   17r      REG                8,1 739485805     394140 /var/factorio/instances/rpg/temp/mp-save-218.zip (deleted)
factorio  11597 11626   factorio   18r      REG                8,1 148248114     394327 /var/factorio/instances/rpg/temp/mp-save-219.zip (deleted)
factorio  11597 11626   factorio   19r      REG                8,1 743812915     393998 /var/factorio/instances/rpg/temp/mp-save-226.zip (deleted)
factorio  11597 11626   factorio   20r      REG                8,1 153675729     394355 /var/factorio/instances/rpg/temp/mp-save-253.zip (deleted)
factorio  11597 11626   factorio   21r      REG                8,1 784528542     392495 /var/factorio/instances/rpg/temp/mp-save-265.zip (deleted)
factorio  11597 11626   factorio   22r      REG                8,1 787961774     393041 /var/factorio/instances/rpg/temp/mp-save-274.zip (deleted)
factorio  11597 11626   factorio   23r      REG                8,1 787689839     393045 /var/factorio/instances/rpg/temp/mp-save-275.zip (deleted)
factorio  11597 11626   factorio   24r      REG                8,1 786289608     393046 /var/factorio/instances/rpg/temp/mp-save-277.zip (deleted)
factorio  11597 11626   factorio   25r      REG                8,1 794639232     394445 /var/factorio/instances/rpg/temp/mp-save-283.zip (deleted)
factorio  11597 11626   factorio   26r      REG                8,1 798340746     393054 /var/factorio/instances/rpg/temp/mp-save-286.zip (deleted)
factorio  11597 11630   factorio    9r      REG                8,1 740927424     392999 /var/factorio/instances/rpg/temp/mp-save-204.zip (deleted)
factorio  11597 11630   factorio   12r      REG                8,1 733000903     392991 /var/factorio/instances/rpg/temp/mp-save-200.zip (deleted)
factorio  11597 11630   factorio   14r      REG                8,1 739463086     393892 /var/factorio/instances/rpg/temp/mp-save-214.zip (deleted)
factorio  11597 11630   factorio   15r      REG                8,1 740622323     394298 /var/factorio/instances/rpg/temp/mp-save-208.zip (deleted)
factorio  11597 11630   factorio   16r      REG                8,1 148645745     394309 /var/factorio/instances/rpg/temp/mp-save-209.zip (deleted)
factorio  11597 11630   factorio   17r      REG                8,1 739485805     394140 /var/factorio/instances/rpg/temp/mp-save-218.zip (deleted)
factorio  11597 11630   factorio   18r      REG                8,1 148248114     394327 /var/factorio/instances/rpg/temp/mp-save-219.zip (deleted)
factorio  11597 11630   factorio   19r      REG                8,1 743812915     393998 /var/factorio/instances/rpg/temp/mp-save-226.zip (deleted)
factorio  11597 11630   factorio   20r      REG                8,1 153675729     394355 /var/factorio/instances/rpg/temp/mp-save-253.zip (deleted)
factorio  11597 11630   factorio   21r      REG                8,1 784528542     392495 /var/factorio/instances/rpg/temp/mp-save-265.zip (deleted)
factorio  11597 11630   factorio   22r      REG                8,1 787961774     393041 /var/factorio/instances/rpg/temp/mp-save-274.zip (deleted)
factorio  11597 11630   factorio   23r      REG                8,1 787689839     393045 /var/factorio/instances/rpg/temp/mp-save-275.zip (deleted)
factorio  11597 11630   factorio   24r      REG                8,1 786289608     393046 /var/factorio/instances/rpg/temp/mp-save-277.zip (deleted)
factorio  11597 11630   factorio   25r      REG                8,1 794639232     394445 /var/factorio/instances/rpg/temp/mp-save-283.zip (deleted)
factorio  11597 11630   factorio   26r      REG                8,1 798340746     393054 /var/factorio/instances/rpg/temp/mp-save-286.zip (deleted)
factorio  11597 11631   factorio    9r      REG                8,1 740927424     392999 /var/factorio/instances/rpg/temp/mp-save-204.zip (deleted)
factorio  11597 11631   factorio   12r      REG                8,1 733000903     392991 /var/factorio/instances/rpg/temp/mp-save-200.zip (deleted)
factorio  11597 11631   factorio   14r      REG                8,1 739463086     393892 /var/factorio/instances/rpg/temp/mp-save-214.zip (deleted)
factorio  11597 11631   factorio   15r      REG                8,1 740622323     394298 /var/factorio/instances/rpg/temp/mp-save-208.zip (deleted)
factorio  11597 11631   factorio   16r      REG                8,1 148645745     394309 /var/factorio/instances/rpg/temp/mp-save-209.zip (deleted)
factorio  11597 11631   factorio   17r      REG                8,1 739485805     394140 /var/factorio/instances/rpg/temp/mp-save-218.zip (deleted)
factorio  11597 11631   factorio   18r      REG                8,1 148248114     394327 /var/factorio/instances/rpg/temp/mp-save-219.zip (deleted)
factorio  11597 11631   factorio   19r      REG                8,1 743812915     393998 /var/factorio/instances/rpg/temp/mp-save-226.zip (deleted)
factorio  11597 11631   factorio   20r      REG                8,1 153675729     394355 /var/factorio/instances/rpg/temp/mp-save-253.zip (deleted)
factorio  11597 11631   factorio   21r      REG                8,1 784528542     392495 /var/factorio/instances/rpg/temp/mp-save-265.zip (deleted)
factorio  11597 11631   factorio   22r      REG                8,1 787961774     393041 /var/factorio/instances/rpg/temp/mp-save-274.zip (deleted)
factorio  11597 11631   factorio   23r      REG                8,1 787689839     393045 /var/factorio/instances/rpg/temp/mp-save-275.zip (deleted)
factorio  11597 11631   factorio   24r      REG                8,1 786289608     393046 /var/factorio/instances/rpg/temp/mp-save-277.zip (deleted)
factorio  11597 11631   factorio   25r      REG                8,1 794639232     394445 /var/factorio/instances/rpg/temp/mp-save-283.zip (deleted)
factorio  11597 11631   factorio   26r      REG                8,1 798340746     393054 /var/factorio/instances/rpg/temp/mp-save-286.zip (deleted)

That's about 7-8GB just sitting there and my monitoring tool screaming there is almost no space left

[posila]

factorio-current.log from the server, please?

[psihius]

Here's a log file, but I doubt there will be anything there.
This is the answer on the internet that lead me to the discovery of what is going on: https://serverfault.com/a/581521

[Oxyd]

Are you sure there simply aren't 15 people downloading the map at the same time? (Notice that your lsof output contains each file multiple times – once per each thread.)

When I tested this locally, Factorio correctly closes the mp-save files after its done uploading them to the connecting peers.

[psihius]

Definitely not 15. 1-2 - maybe, but 15 no. That server requires top of the line hardware to even join, so the traffic is not that big. That far I checked so I don't bother with some false-positives. It's the first thing that came to my mind - Am I deleting files still being transferred?

The main server since the time of the reporting to current message writing time has no issues, but traffic was also lower as we are in a very late game with huuuge factory.

And now I have the same issue on the second "slow" server - it's way slower, the save sizes are below 40 MB. The cleanup command for cron is:

Code: Select all

*/10 * * * * find /var/factorio/instances/*/temp -name '*.zip' -mmin +60 -delete

Code: Select all

root at factoriommo-slow in ~
$  lsof | grep "/var" | grep deleted
factorio  25758          factorio   13r      REG                8,1 113191154     652998 /var/factorio/instances/default/temp/mp-save-14.zip (deleted)
factorio  25758 25766    factorio   13r      REG                8,1 113191154     652998 /var/factorio/instances/default/temp/mp-save-14.zip (deleted)
factorio  25758 25768    factorio   13r      REG                8,1 113191154     652998 /var/factorio/instances/default/temp/mp-save-14.zip (deleted)
factorio  25758 25772    factorio   13r      REG                8,1 113191154     652998 /var/factorio/instances/default/temp/mp-save-14.zip (deleted)
factorio  25758 25773    factorio   13r      REG                8,1 113191154     652998 /var/factorio/instances/default/temp/mp-save-14.zip (deleted)
factorio  25758 25775    factorio   13r      REG                8,1 113191154     652998 /var/factorio/instances/default/temp/mp-save-14.zip (deleted)
factorio  25758 25776    factorio   13r      REG                8,1 113191154     652998 /var/factorio/instances/default/temp/mp-save-14.zip (deleted)

Weirdly it's the same file. Just let me know what info you need - I'm not going to restart that instance and keep it running as space on that server is not an issue.

[Oxyd]

Could you post the log from the slower server, then? Since it's only got one stuck file, it might be easier to plough through that log for clues. I'm assuming that nobody was joining the server at the time you ran the lsof command. (No need to restart, just grab the factorio-current.log from the running instance.)

[psihius]

Here are all 3 that I had in the server dir.

[Harkonnen]

Could it be that file is kept opened if client drops during map download or is unable to catch up?

[psihius]

Could it be that file is kept opened if client drops during map download or is unable to catch up?

Makes too much sense, and this we do get a lot of, so worth testing out.

[posila]

Thanks for the logs.
It happens when client disconnects while server is saving map for the client.
It should be fixed in 0.15.29

[psihius]

Yay, thank you and glad I helped find this for you to fix