[2.0.21] Crash on Vulcanus: Linux amd64 Intel Integrated Graphics

[nastynafty]

System Info:

- CPU: Intel(R) Core(TM) Ultra 7 165U
- Graphics: 00:02.0 VGA compatible controller: Intel Corporation Device 7d45 (rev 08)
- RAM: 64GiB
- Storage: 2TiB NVMe
- Distribution: Ubuntu 22.04 (amd64)
- Kernel: 6.8.0-48-generic
- Running in Steam in Native Mode (Not Windows Layer)

In general, I have pretty decent performance in-game, 40-60FPS so it's entirely playable without a dedicated (non-integrated) GPU.

I've done Nauvis, Fulgora, and just landed on Vulcanis. I started running around the landing area to explore the map, and as soon as a destroyer worm enters the screen, the game stops rendering, pausing on the last frame and sometimes flickering wildly, I'm unable to do anything at all, music continues playing in the background, and the entire system more or less grinds to a halt. Even trying to switch to a TTY to log in and kill the process takes an extremely long time.

I'm not sure what specifically is related in regards to the worms, but this is reproducible. Every time a worm becomes visible on screen, everything grinds to a halt. Otherwise, the game plays extremely well. I don't know much about internals of the game but maybe this is a shader issue specific to the destroyer worms? I can't think of what else would be different specific to Vulcanis, as everything else works fine. Biters work fine, other enemies work fine.

[posila]

Thanks for the report.
Can you post your log, please? Preferably from instance when the game crashed.

Intergrated GPU on Core(TM) Ultra 7 165U is fine to run the game (as long as you don't want to run it in 4K). And demolishers work fine for me on Core i5-12450H (on Windows though)

[nastynafty]

I'm going to try to reproduce.

FYI I monitored CPU usage and GPU usage (via `intel_gpu_top`) and there was nothing significant going on when the issue occurs. I'm heading back to Vulcanus to see if I can reproduce.

[nastynafty]

Still reproducible on latest stable. When I encountered the "corpse" of dead worm, it lagged significantly but was playable, but when I encountered an actual living worm, same problem: rendering effectively stops and I can't do anything at all.

Also I couldn't find anything in the settings to change my screen resolution, not sure how I missed it/where it is.

[posila]

So I tried to visit Vulcanus on Intel Core Ultra 5 135U, on Ubuntu 22.04.05, I had the exact graphics driver version as you did. I found two demolishers close together and the game didn't crash even when I aggroed them both. I did play only in 1080p as I don't currently have bigger screen to test on. Until I aggroed the demolishers, GPU timings were miniscule, just couple of milliseconds, so I imagine it would run at or close to 60 FPS even in 4K. After I aggroed them, the framerate got really bad, but mainly due to draw call count raising to 10k and even 20k, which is a lot ... so that is something that we should look into, but the game still didn't crash.

Similar report: 120269

Let me know if you want any more information!

Update: turning off particles seems to have worked around the issue, so it is not preventing me from playing.
Update: it isn't clear that's actually true. I turned off particles and some other things, and turned down other settings, and it happens less often but still sometimes.

There is no resolution setting at the moment. I hope to add Resolution scaling setting before Christmas.
Does the crash/hang happen if you Alt+Enter into windowed mode and keep the game window in size about 1920x1080?

(No need to travel to Vulcanus in your save, you can start new map, type "/cheat vulcanus" into chat to get teleported to vulcanus, and "/cheat all" to get mecharmor so you can move fast and not die to demolishers instantly)

[nastynafty]

Since the issue may be related to 4K, I'd have to wait until resolution support is added. I can see if I can try the Alt+Enter thing.

On 2.0.23 (latest for me) at full screen 4K, issue still occurs. And when running in windowed mode at about a quarter resolution, the hang still occurs and it causes the entire display to flicker rapidly. To confirm, it's not so much of a crash as it is a hang. It's difficult to interrupt it at the system level, I need to switch to a TTY, log in, and killall factorio to make my system usable again. Though I'm a software engineer myself, I don't exactly know what could be going on here. I'd have to guess at shaders maybe, running out of GPU memory maybe, not sure. Also, I'm running X, not Wayland, as I'm on elementary OS (Ubuntu 22.04 under the hood). When I look at intel_gpu_top and regular top, the system appears to be ok. I'm definitely not running out of RAM as I have 64GiB, the CPU is not pegged, and the GPU also does not appear to be pegged.

[nastynafty]

FYI updated last reply.

[Rseding91]

Out of curiosity, if you’re able, would you be willing to test installing windows on some partition and seeing if it still locks up there? If not, then we know it’s a software issue on the Linux side. If it does, then it’s likely a hardware issue.

[nastynafty]

It would take a significant amount of effort for me to install Windows to be honest. On my other platforms, I don't have the issue but they are different hardware of course:

- Steam Deck
- Ubuntu 22.04 on an NVIDIA 1080Ti
- Windows on an NVIDIA 1070Ti

Is there anything I can do to record things like GPU/CPU utilization and memory while running it on this device? It's interesting to me that even when in a window, it completely overwhelms the display manager and starts flickering like crazy. I have no idea what it means, but I imagine that something on the GPU is overloaded, memory perhaps. It takes a while to switch to the TTY but once I'm in the TTY, no performance problems at all with logging in, executing commands, etc. so I think it's definitely GPU related. intel_gpu_top also shows no usage when I'm in a TTY so clearly in TTY mode the GPU is not doing anything at all.

[posila]

Understandable.
Since it also happens in windowed mode in smaller window, it doesn't seem to be related to 4K, so resolution option won't fix it for you.
On my machine I have tested it with Ubuntu 22.04 Live USB, and I am not sure how elementary OS is different from regular Ubuntu and if that difference could cause the issue.

Is there anything I can do to record things like GPU/CPU utilization and memory while running it on this device?

Honestly, I have no idea. Even on Windows I would have trouble diagnosing this locally, let alone remotely. I would probably end up just bisecting the code to find what exactly is triggering it, and then changing that part until it stops happening. I'll probably just add hidden option to disable demolisher shader ... it uses special shader, because the engine sorts sprite back to front (top to bottom) and right to left, but demolisher segments need to be rendered tail to head, so it uses special shader (special in sense that nothing else in the game uses it) to mask segments by their neighbours to appear rendered tail to head. It seems to me that this shader is causing the problem.

I have no idea what it means, but I imagine that something on the GPU is overloaded, memory perhaps.

Now that you mention something being overloaded, could you try to update BIOS on the device?

[nastynafty]

Finally was able to get BIOS updated (my EFI partition did not have the esp flag set in GPT so fwupdmgr was not seeing my EFI partition) and the crash/hang is not much of a thing anymore. I can witness worms and survive! However, things still kind of get bogged down a bit but it does seem to recover.

Outside of the context of worms though, performance in terms of framerate seems to be down quite a bit, was seeing 40-55 FPS before and I'm generally hovering around 25FPS now. Not sure if there is any correlation. Don't see a resolution setting in the settings menu yet so this was attempting at full screen 4K.

[posila]

Oh, cool. I am glad to hear BIOS update fixed the crashes.

Resolution scaling option is not done yet, sorry.

Outside of the context of worms though, performance in terms of framerate seems to be down quite a bit, was seeing 40-55 FPS before and I'm generally hovering around 25FPS now. Not sure if there is any correlation.

You were seeing 40-55 FPS before the BIOS update and 25 FPS after? Could you post current log, please?

[dima@stopel.org]

Same issue for me. BIOS and firmware are up to date. The game runs fine except on Vulcanus and when demolisher shows up (when becomes visible on the main screen). The games slows down significantly and then completely stops. I can still hear sounds in the background like the game is still running but the screen is frozen. Logs are attached. Tried to disable various graphics effects but it didn't help. Thanks