[1.1.34] Main game hangs at the end of saving
Posted: Sun Jun 13, 2021 3:41 am
I am using a non-blocking saving feature on Linux desktop, and it usually works perfectly. Never had any crashes for last year or two, since I started using it.
I don't use any mods. I do have to loaded in the game, but not actually enabled on this map save.
I often also leave the game for hours alone playing and running, and then come back to it, to see what is broken. Often the main factorio process will have many days of wall clock time since it started with no ill effects.
However, today I found that while I was away, it did stop and hang. I had it running in the foreground, I was just standing in the middle of the factory, and had the production stats window open. This was weird, as this very same map was left alone before, with no issues.
And I see it was in the process of saving.
"Saving map _autosdave2", in the corner, with progress bar at 100%. I do have free space on file system, so that is not an issue.
Otherwise I can't interact with the game, it doesn't responds to mouse clicks, movements, keyboard, escape, Alt-Enter. The image on the screen is still, and doesn't change.
top shows 0% CPU usage.
ps aux shows:
suggesting to me that the forked saver factorio process, just finished and is now should be waited by the parent factorio process.
(167% CPU is an average CPU usage since the start of the process, which as you can see is about 2 days ago).
I am attaching a 4 second sample of strace output (captured using strace -ttt -T -f -o factorio_hang_strace.txt -p 634302 ).
I am also attaching a gdb all thread stack trace dump. And based on a look at this strace a procfs info for the process file descriptors.
I also saved a core dump of the main process. https://www.functor.xyz/factorio/factor ... ump.tar.7z (500MB compressed, 11GB uncompressed).
System is Linux 5.10.28-1, amd64, Debian testing, amdgpu, Mesa 20.3.4 on AMD Radeon R9 Fury X (FIJI). CPU ThreadRipper 2950X, no OC, 128GB (8x16GB) of DDR4 memory (ECC). System uptime 23 days.
PS. I know you might be short on staff, especially Linux, so I am willing to look essentially for free at the source code related to async saving under NDA, if you want. It looks like some sort of deadlock, maybe fork / clone issue. I know Linux, pthread and C++ very well.
I don't use any mods. I do have to loaded in the game, but not actually enabled on this map save.
I often also leave the game for hours alone playing and running, and then come back to it, to see what is broken. Often the main factorio process will have many days of wall clock time since it started with no ill effects.
However, today I found that while I was away, it did stop and hang. I had it running in the foreground, I was just standing in the middle of the factory, and had the production stats window open. This was weird, as this very same map was left alone before, with no issues.
And I see it was in the process of saving.
"Saving map _autosdave2", in the corner, with progress bar at 100%. I do have free space on file system, so that is not an issue.
Otherwise I can't interact with the game, it doesn't responds to mouse clicks, movements, keyboard, escape, Alt-Enter. The image on the screen is still, and doesn't change.
top shows 0% CPU usage.
ps aux shows:
Code: Select all
$ ps aux | grep factorio
user 634300 0.0 0.0 33100 25000 pts/2 S+ Jun11 0:00 /bin/sh -c /home/user/.steam/debian-installation/ubuntu12_32/../ubuntu12_32/reaper SteamLaunch AppId=427520 -- '/home/user/.steam/debian-installation/steamapps/common/Factorio/bin/x64/factorio'
user 634301 0.0 0.0 30480 24224 pts/2 S+ Jun11 0:00 /home/user/.steam/debian-installation/ubuntu12_32/../ubuntu12_32/reaper SteamLaunch AppId=427520 -- /home/user/.steam/debian-installation/steamapps/common/Factorio/bin/x64/factorio
user 634302 167 2.6 11004384 3539720 pts/2 Sl+ Jun11 3117:23 /home/user/.steam/debian-installation/steamapps/common/Factorio/bin/x64/factorio
user 928124 0.1 0.0 0 0 pts/2 Z+ 02:18 0:03 [factorio] <defunct>
(167% CPU is an average CPU usage since the start of the process, which as you can see is about 2 days ago).
I am attaching a 4 second sample of strace output (captured using strace -ttt -T -f -o factorio_hang_strace.txt -p 634302 ).
I am also attaching a gdb all thread stack trace dump. And based on a look at this strace a procfs info for the process file descriptors.
I also saved a core dump of the main process. https://www.functor.xyz/factorio/factor ... ump.tar.7z (500MB compressed, 11GB uncompressed).
System is Linux 5.10.28-1, amd64, Debian testing, amdgpu, Mesa 20.3.4 on AMD Radeon R9 Fury X (FIJI). CPU ThreadRipper 2950X, no OC, 128GB (8x16GB) of DDR4 memory (ECC). System uptime 23 days.
PS. I know you might be short on staff, especially Linux, so I am willing to look essentially for free at the source code related to async saving under NDA, if you want. It looks like some sort of deadlock, maybe fork / clone issue. I know Linux, pthread and C++ very well.