[1.1.59] Crash in single player without mods

[Wiwiweb]

*Describe the problem as best as you can:*
Unfortunately I wasn't doing anything special. I was in map view using an upgrade planner to upgrade modules, but I had been doing that for the last 10 minutes before the crash so it may be irrelevant.

Save file attached.

Relevant log here: (Full log attached)

Code: Select all

3885.526 Error CrashHandler.cpp:494: Exception Code: c0000005, Address: 0x00007ff72534178d
ModuleBase: 0x00007ff724fe0000, ImageSize: 01db4000, RelativeAddress: 0036178d
3885.526 Error CrashHandler.cpp:500: Access Violation: Read at address 000000CE8D77A338
3885.526 Error CrashHandler.cpp:514: Exception Context:
rax=000000ce8d77a2f0, rbx=000001cec5eaabe8, rcx=000001cec5ea9ad0,
rdx=0000000000000004, rsi=000001ce8d778df0, rdi=000001cec5eaabf8,
rip=00007ff72534178d, rsp=00000032d96ff640, rbp=000000000000000a,
 r8=000001cec5ea9820,  r9=00000000ffffffd4, r10=0000000000000000,
r11=0000000000000004, r12=000001cea2917640, r13=000001ce9bab688a,
r14=000001ce8d778df0, r15=0000000000000000
3885.526 Crashed in C:\Program Files (x86)\Steam\steamapps\common\Factorio\bin\x64\Factorio.exe (0x00007ff724fe0000 - 0x00007ff726d94000)
Factorio crashed. Generating symbolized stacktrace, please wait ...
c:\cygwin64\tmp\factorio-build-qay9dg\libraries\stackwalker\stackwalker.cpp (924): StackWalker::ShowCallstack
c:\cygwin64\tmp\factorio-build-qay9dg\src\util\logger.cpp (316): Logger::writeStacktrace
c:\cygwin64\tmp\factorio-build-qay9dg\src\util\logger.cpp (323): Logger::logStacktrace
c:\cygwin64\tmp\factorio-build-qay9dg\src\util\crashhandler.cpp (186): CrashHandler::writeStackTrace
c:\cygwin64\tmp\factorio-build-qay9dg\src\util\crashhandler.cpp (543): CrashHandler::SehHandler
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFF523EFF27)
00007FFF523EFF27 (KERNELBASE): (filename not available): UnhandledExceptionFilter
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFF549551B0)
00007FFF549551B0 (ntdll): (filename not available): memset
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFF5493C766)
00007FFF5493C766 (ntdll): (filename not available): _C_specific_handler
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFF5495209F)
00007FFF5495209F (ntdll): (filename not available): _chkstk
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFF54901454)
00007FFF54901454 (ntdll): (filename not available): RtlRaiseException
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFF54950BCE)
00007FFF54950BCE (ntdll): (filename not available): KiUserExceptionDispatcher
c:\cygwin64\tmp\factorio-build-qay9dg\src\entity\pipe.cpp (199): Pipe::shouldHaveWindow
c:\cygwin64\tmp\factorio-build-qay9dg\src\entity\pipe.cpp (233): Pipe::draw
c:\cygwin64\tmp\factorio-build-qay9dg\src\graphics\entityrenderer.cpp (325): EntityRenderer::prepareRow
c:\cygwin64\tmp\factorio-build-qay9dg\src\graphics\entityrenderer.cpp (225): EntityRenderer::threadJob
c:\cygwin64\tmp\factorio-build-qay9dg\src\util\workerthread.cpp (49): WorkerThread::loop
c:\program files (x86)\microsoft visual studio\2017\buildtools\vc\tools\msvc\14.15.26726\include\thr\xthread (230): std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl RouterBase::*)(void),ServerRouter *>,std::default_delete<std::tuple<void (__cdecl RouterBase::*)(void),ServerRouter *> > > >::_Go
c:\program files (x86)\microsoft visual studio\2017\buildtools\vc\tools\msvc\14.15.26726\include\thr\xthread (209): std::_Pad::_Call_func
minkernel\crts\ucrt\src\appcrt\startup\thread.cpp (115): thread_start<unsigned int (__cdecl*)(void * __ptr64)>
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFF539A7034)
00007FFF539A7034 (KERNEL32): (filename not available): BaseThreadInitThunk
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFF54902651)
00007FFF54902651 (ntdll): (filename not available): RtlUserThreadStart
Stack trace logging done
3889.482 Error CrashHandler.cpp:191: Map tick at moment of crash: 8522654
3889.559 Info CrashHandler.cpp:311: Executable CRC: 2138896114
3889.559 Error Util.cpp:97: Unexpected error occurred. If you're running the latest version of the game you can help us solve the problem by posting the contents of the log file on the Factorio forums.
Please also include the save file(s), any mods you may be using, and any steps you know of to reproduce the crash.
3899.132 Uploading log file
3899.563 Error CrashHandler.cpp:262: Heap validation: success.
3899.565 Creating crash dump.
3899.993 CrashDump success

[Loewchen]

Can you reproduce the crash?

[Wiwiweb]

I cannot. Unfortunately I didn't have autosave on so I lost the exact state of that base. If it happens again I will update this thread.

[Loewchen]

Did you have bluescreens on this machine at any point.

[Wiwiweb]

Before some hardware replacement, yes. But not in the last 6 months.
I also never crashed like this in 1000+ hours of Factorio.

[Rseding91]

Thanks for the report. Looking at the crash log it crashed in a seemingly random place for seemingly no reason. That combined with the fact you can't reproduce it (and I can't see any way to make it happen) I think this is either a one-off crash or possible hardware issues. Either way, if you do ever figure out a way to reproduce it let us know.

[MooseBoys]

One-off bug, segfault in multi-threaded worker... probably a race condition somewhere.

[boskid]

One-off bug, segfault in multi-threaded worker... probably a race condition somewhere.

Making such assumptions without any kind of hints is a bad habbit. Entity renderer's prepare stage is multithreaded but those worker threads are only ever reading the game state. They only write to their own thread local data structures like DrawQueue. There is no place for races here unless there would be other kind of workers mutating the game state at the same time but this is unlikely because entity renderer's prepare stage is mutually exclusive with the update and any unintended changes to the game state would cause desyncs.

[MooseBoys]

Making such assumptions without any kind of hints is a bad habbit.

But you have to start somewhere. Absent any error messages suggesting otherwise, that's where I would start looking. In any case, in any deterministic computer program, errors that only occur some of the time given the same input can only have two possible causes - hardware failure (including bit flips from cosmic rays), or race condition. So it's not really much of a leap. Maybe you're thinking I meant a bug specifically in locking/mutexes? Those can lead to one class of race condition, but it's far from the only one.

Entity renderer's prepare stage is multithreaded but those worker threads are only ever reading the game state. They only write to their own thread local data structures like DrawQueue. There is no place for races here unless there would be other kind of workers mutating the game state at the same time...

That's not accurate. There are plenty of ways for a producer-consumer system to race, even one in which producer execution is mutually exclusive with consumer execution. Contrived example:

Code: Select all

Main():
  while !quit:
    for chunk in active_chunks:
      if DistanceBetween(camera, chunk) > 1000:
        chunk.active = false
        // oops - chunk is inactive but will still be found in active_chunks for one more tick
    ParallelFor(Worker, all_chunks).join()
    UpdateChunks() // regenerates active_chunks

Worker(chunk):
  if (!chunk.active && very_uncommon_thing):
    chunk_index = inactive_chunks.find(chunk) // KEY NOT FOUND

In the above code, if `very_uncommon_thing` happens during the exact same tick as some other uncommon thing, like the chunk going outside some influence range, then the worker can crash.

An even simpler case is ordinary use-after-free. Absent any sanitization steps like zeroing memory on free, use-after-free can be asymptomatic 99.999% of the time. But if the OS decides it needs a little more memory for some unrelated process, e.g. checking for updates, at *just the right time*, it could reclaim those pages, so instead of reading garbage, the process crashes.