[0.15.6] Hard crash on iterating through chunks

[orzelek]

There seems to be a strange crash when iterating over chunks using for and LuaSurface.get_chunks().
To reproduce it you need mods and save avaialble in this thread:
viewtopic.php?p=266428#p266428

After loading the save try to run following command:

Code: Select all

/c remote.call("RSO", "regenerate")

Function called in RSO uses for loop with chunk iterator twice. First time it clears the chunks and second time it generates ores again.
I used log() statements to check that crash happens after generating a chunk when it should iterate to next one (thats during second iteration).
I'm not sure if it's something I'm doing in lua or the fact that I'm iterating twice over chunks.
Log from crash:

log

Code: Select all

   0.001 2017-05-04 23:04:00; Factorio 0.15.6 (build 29362, win64, alpha)
   0.001 Operating system: Windows 10 (build 15063) 
   0.001 Program arguments: "F:\Factorio\bin\x64\Factorio.exe" 
   0.001 Read data path: F:/Factorio/data
   0.001 Write data path: F:/Factorio
   0.001 Binaries path: F:/Factorio/bin
   0.006 System info: [CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz, 8 cores, RAM: 12231MB]
   0.006 Display options: [FullScreen: 1] [VSync: 1] [UIScale: 100%] [MultiSampling: OFF] [Screen: 255]
   0.007 Available display adapters: 1
   0.007  [0]: \\.\DISPLAY1 - Radeon (TM) RX 480 Graphics {0x05, [0,0], 1920x1080, 32bit, 60Hz}
   0.009 Create display on adapter 0. Size 1280x720 at position [310, 162].
   0.059 Initialised Direct3D:[0] Radeon (TM) RX 480 Graphics; driver: aticfx64.dll 22.19.157.3
   0.061     Video memory size (dedicated video/dedicated system/shared system/available): 8140/0/6115/4088 MB
   0.082 DSound: Starting _dsound_update thread
   0.082 DSound: Enter _dsound_update; tid=2880
   0.082 Device reset internal.
   0.084 Desktop composition is active.
   0.084 WARNING: You have vsync and desktop composition enabled on Direct3d. Consider disabling vsync to increase performance on some confirgurations.
   0.084 Graphics settings preset: very-high
   0.084 Graphics options: [Graphics quality: high] [Video memory usage: all] [Light scale: 100%] [DXT: false]
   0.154 Loading mod settings rso-mod 3.1.5 (settings.lua)
   0.155 Loading mod settings angelsinfiniteores 0.6.4 (settings.lua)
   0.159 Loading mod core 0.0.0 (data.lua)
   0.166 Loading mod base 0.15.6 (data.lua)
   0.261 Loading mod rso-mod 3.1.5 (data.lua)
   0.317 Loading mod SmartTrains 2.0.1 (data.lua)
   0.373 Loading mod TheFatController 2.0.4 (data.lua)
   0.435 Loading mod Agent-Orange 15.1.0 (data.lua)
   0.488 Loading mod angelsinfiniteores 0.6.4 (data.lua)
   0.544 Loading mod autofill 1.4.7 (data.lua)
   0.612 Loading mod Big_Brother 0.4.2 (data.lua)
   0.680 Loading mod Bottleneck 0.6.3 (data.lua)
   0.742 Loading mod EvolGUI 0.5.0 (data.lua)
   0.820 Loading mod Flow Control 3.0.1 (data.lua)
   0.884 Loading mod ModuleInserter 2.0.2 (data.lua)
   0.966 Loading mod Pipe-Manager 0.2.2 (data.lua)
   1.028 Loading mod Side Inserters 2.2.3 (data.lua)
   1.093 Loading mod GDIW 0.15.2 (data.lua)
   1.180 Loading mod base 0.15.6 (data-updates.lua)
   1.248 Loading mod rso-mod 3.1.5 (data-updates.lua)
   1.313 Loading mod Big_Brother 0.4.2 (data-updates.lua)
   1.392 Loading mod Squeak Through 1.1.7 (data-updates.lua)
   1.441 Loading mod Agent-Orange 15.1.0 (data-final-fixes.lua)
   1.518 Loading mod angelsinfiniteores 0.6.4 (data-final-fixes.lua)
   1.601 Loading mod longer-belts-redux 0.15.0 (data-final-fixes.lua)
   1.662 Loading mod ModuleInserter 2.0.2 (data-final-fixes.lua)
   1.727 Loading mod GDIW 0.15.2 (data-final-fixes.lua)
   1.804 Checksum for core: 3409166051
   1.804 Checksum of base: 2213884006
   1.804 Checksum of rso-mod: 2918425174
   1.804 Checksum of SmartTrains: 1113370295
   1.804 Checksum of TheFatController: 951285371
   1.804 Checksum of Agent-Orange: 4219057965
   1.804 Checksum of angelsinfiniteores: 230704381
   1.804 Checksum of autofill: 3086359458
   1.804 Checksum of Big_Brother: 1188679049
   1.804 Checksum of Bottleneck: 110869948
   1.804 Checksum of EvolGUI: 3542768350
   1.804 Checksum of Flow Control: 777216705
   1.804 Checksum of longer-belts-redux: 883706356
   1.804 Checksum of ModuleInserter: 1441152677
   1.804 Checksum of Pipe-Manager: 3073476668
   1.804 Checksum of Side Inserters: 3668734278
   1.804 Checksum of Squeak Through: 3430405470
   1.804 Checksum of GDIW: 2359646124
   1.932 Info PlayerData.cpp:55: Local player-data.json available, timestamp 1493931759
   1.932 Info PlayerData.cpp:62: Cloud player-data.json unavailable
   2.001 Loaded shader file F:/Factorio/data/core/graphics/shaders/game.cso
   2.002 Loaded shader file F:/Factorio/data/core/graphics/shaders/zoom-to-world.cso
   2.012 Initial atlas bitmap size is 16384
   2.028 Created atlas bitmap 16384x16378
   2.031 Created atlas bitmap 16384x9564
   2.031 Created atlas bitmap 4096x4064
   2.031 Created atlas bitmap 4096x920
   2.032 Created atlas bitmap 4096x3644
  13.723 Sprites loaded
  13.723 Convert atlas 4096x4064 to: trilinear-filtering 
  13.774 Convert atlas 4096x920 to: trilinear-filtering 
  13.789 Convert atlas 4096x3644 to: mipmap 
  14.113 Loading sounds...
  15.944 Custom inputs active: 8
  15.946 Info Updater.cpp:753: Downloading https://updater.factorio.com/get-available-versions?username=orzelek&token=<private>&apiVersion=2
  16.871 0 packages available to download (experimental updates enabled).
  16.948 Factorio initialised
  39.865 Loading map F:\Factorio\saves\sort.zip: 8052791 bytes.
  39.932 Loading Level.dat: 16277551 bytes.
  39.943 Info Scenario.cpp:133: Map version 0.15.6-0
  40.326 Info PrototypeMigrationList.cpp:51: Activating migration GDIW/GDIW_0.14.10.json
  40.326 Warning PrototypeMigrationList.cpp:191: "basic-oil-processing-GDIW-3" mentioned as FROM in prototype migration was not found in the map prototype dictionary, ignoring ...
  40.326 Warning PrototypeMigrationList.cpp:191: "advanced-oil-processing-GDIW" mentioned as FROM in prototype migration was not found in the map prototype dictionary, ignoring ...
  40.326 Warning PrototypeMigrationList.cpp:191: "advanced-oil-processing-GDIW-2" mentioned as FROM in prototype migration was not found in the map prototype dictionary, ignoring ...
  40.326 Warning PrototypeMigrationList.cpp:191: "advanced-oil-processing-GDIW-3" mentioned as FROM in prototype migration was not found in the map prototype dictionary, ignoring ...
  40.326 Warning PrototypeMigrationList.cpp:191: "heavy-oil-cracking-GDIW" mentioned as FROM in prototype migration was not found in the map prototype dictionary, ignoring ...
  40.326 Warning PrototypeMigrationList.cpp:191: "light-oil-cracking-GDIW" mentioned as FROM in prototype migration was not found in the map prototype dictionary, ignoring ...
  40.326 Warning PrototypeMigrationList.cpp:191: "sulfur-GDIW" mentioned as FROM in prototype migration was not found in the map prototype dictionary, ignoring ...
  40.326 Warning PrototypeMigrationList.cpp:191: "flame-thrower-ammo-GDIW" mentioned as FROM in prototype migration was not found in the map prototype dictionary, ignoring ...
  40.326 Warning PrototypeMigrationList.cpp:191: "bob-oil-processing-GDIW" mentioned as FROM in prototype migration was not found in the map prototype dictionary, ignoring ...
  40.328 Loading script.dat: 107880 bytes.
  40.331 Checksum for script F:/Factorio/temp/currently-playing/control.lua: 76911248
  40.352 Checksum for script __rso-mod__/control.lua: 346452855
  40.359 Checksum for script __SmartTrains__/control.lua: 655274804
  40.363 Checksum for script __TheFatController__/control.lua: 1191308127
  40.367 Checksum for script __autofill__/control.lua: 3455809202
  40.371 Checksum for script __Big_Brother__/control.lua: 549854449
  40.372 Checksum for script __Bottleneck__/control.lua: 2468296750
  40.375 Checksum for script __EvolGUI__/control.lua: 1694842198
  40.376 Checksum for script __Flow Control__/control.lua: 3027294436
  40.378 Checksum for script __ModuleInserter__/control.lua: 3511672913
  40.379 Checksum for script __Pipe-Manager__/control.lua: 2473255719
  40.380 Checksum for script __Side Inserters__/control.lua: 23258267
  40.381 Checksum for script __GDIW__/control.lua: 2648178700
  40.439 Script control.lua:356: AutoFill: Defaults Updated
  44.954 Error CrashHandler.cpp:141: Exception Code: c0000005, Address: 0x00007ff733f78bbd
ModuleBase: 0x00007ff733960000, ImageSize: 0119e000, RelativeAddress: 00618bbd
  44.954 Error CrashHandler.cpp:151: Exception Context:
rax=0000000000000000, rbx=000002cbf4c00f38, rcx=000002cbf4c00f38,
rdx=00000001d153e6c8, rsi=00007ff733fd7290, rdi=00000000000000b0,
rip=00007ff733f78bbd, rsp=000000e7bb5fe3f0, rbp=000002cbedbd12e0,
 r8=000002cb436049c0,  r9=0000000000000000, r10=000002cc11519a50,
r11=000002cc11519a50, r12=0000000000000001, r13=000002cbf49ae940,
r14=0000000000000030, r15=000002cbfac6b360
  44.954 Crashed in F:\Factorio\bin\x64\Factorio.exe (0x00007ff733960000 - 0x00007ff734afe000)
Factorio crashed. Generating symbolized stacktrace, please wait ...
c:\cygwin64\tmp\factorio-cu6znz\libraries\stackwalker\stackwalker.cpp (923): StackWalker::ShowCallstack
c:\cygwin64\tmp\factorio-cu6znz\src\util\logger.cpp (354): Logger::writeStacktrace
c:\cygwin64\tmp\factorio-cu6znz\src\util\logger.cpp (408): Logger::logStacktrace
c:\cygwin64\tmp\factorio-cu6znz\src\util\crashhandler.cpp (105): CrashHandler::writeStackTrace
c:\cygwin64\tmp\factorio-cu6znz\src\util\crashhandler.cpp (177): CrashHandler::SehHandler
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFC96B36913)
00007FFC96B36913 (KERNELBASE): (filename not available): UnhandledExceptionFilter
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFC9990DA9D)
00007FFC9990DA9D (ntdll): (filename not available): memset
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFC998F6476)
00007FFC998F6476 (ntdll): (filename not available): _C_specific_handler
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFC9990A08D)
00007FFC9990A08D (ntdll): (filename not available): _chkstk
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFC99879C58)
00007FFC99879C58 (ntdll): (filename not available): RtlLookupFunctionEntry
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFC9990910E)
00007FFC9990910E (ntdll): (filename not available): KiUserExceptionDispatcher
c:\cygwin64\tmp\factorio-cu6znz\src\util\infiniterectangle.hpp (46): InfiniteRectangle<boost::intrusive::list<LogisticCell,boost::intrusive::base_hook<boost::intrusive::list_base_hook<boost::intrusive::tag<LogisticCellsRectangleTag>,boost::intrusive::link_mode<2>,void> >,boost::intrusive::constant_time_size<0>,void,void> * __ptr64>::const_iterator::operator++
c:\cygwin64\tmp\factorio-cu6znz\src\script\luachunkiterator.cpp (107): LuaChunkIterator::luaCall
c:\cygwin64\tmp\factorio-cu6znz\src\script\luabinder.hpp (312): LuaBinder<LuaRandomGenerator>::callWrapper
c:\cygwin64\tmp\factorio-cu6znz\libraries\lua\ldo.c (320): luaD_precall
c:\cygwin64\tmp\factorio-cu6znz\libraries\lua\ldo.c (392): luaD_call
c:\cygwin64\tmp\factorio-cu6znz\libraries\lua\lvm.c (792): luaV_execute
c:\cygwin64\tmp\factorio-cu6znz\libraries\lua\lapi.c (934): f_call
c:\cygwin64\tmp\factorio-cu6znz\libraries\lua\ldo.c (131): luaD_rawrunprotected
c:\cygwin64\tmp\factorio-cu6znz\src\script\luaremote.cpp (98): LuaInterface::runFunction
c:\cygwin64\tmp\factorio-cu6znz\src\script\luaremote.cpp (191): LuaRemote::luaRemoteCall
c:\cygwin64\tmp\factorio-cu6znz\src\script\luabinder.hpp (312): LuaBinder<LuaSettings>::callWrapper
c:\cygwin64\tmp\factorio-cu6znz\libraries\lua\ldo.c (320): luaD_precall
c:\cygwin64\tmp\factorio-cu6znz\libraries\lua\lvm.c (710): luaV_execute
c:\cygwin64\tmp\factorio-cu6znz\libraries\lua\lapi.c (934): f_call
c:\cygwin64\tmp\factorio-cu6znz\libraries\lua\ldo.c (131): luaD_rawrunprotected
c:\cygwin64\tmp\factorio-cu6znz\src\script\luagamescript.cpp (997): LuaGameScript::runLuaCommand
c:\cygwin64\tmp\factorio-cu6znz\src\script\luacontext.cpp (173): LuaContext::runLuaCommand
c:\cygwin64\tmp\factorio-cu6znz\src\commandprocessor.cpp (1200): CommandProcessor::doExecuteCommand
c:\cygwin64\tmp\factorio-cu6znz\src\commandprocessor.cpp (266): CommandProcessor::executeCommand
c:\cygwin64\tmp\factorio-cu6znz\src\commandprocessor.cpp (218): CommandProcessor::process
c:\cygwin64\tmp\factorio-cu6znz\src\gameactionhandler.cpp (2877): GameActionHandler::writeToConsole
c:\cygwin64\tmp\factorio-cu6znz\src\gameactionhandler.cpp (267): GameActionHandler::actionPerformed
c:\cygwin64\tmp\factorio-cu6znz\src\input\inputhandler.cpp (55): InputHandler::flushToListeners
c:\cygwin64\tmp\factorio-cu6znz\src\input\inputhandler.cpp (47): InputHandler::flushActions
c:\cygwin64\tmp\factorio-cu6znz\src\gameactionhandler.cpp (305): GameActionHandler::update
c:\cygwin64\tmp\factorio-cu6znz\src\mainloop.cpp (819): MainLoop::gameUpdateStep
c:\cygwin64\tmp\factorio-cu6znz\src\mainloop.cpp (695): MainLoop::gameUpdateLoop
c:\cygwin64\tmp\factorio-cu6znz\src\util\workerthread.cpp (35): WorkerThread::loop
c:\program files (x86)\microsoft visual studio 14.0\vc\include\thr\xthread (247): std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(ParallelDesyncReportCreator * __ptr64),ParallelDesyncReportCreator * __ptr64>,std::default_delete<std::tuple<void (__cdecl*)(ParallelDesyncReportCreator * __ptr64),ParallelDesyncReportCreator * __ptr64> > > >::_Run
c:\program files (x86)\microsoft visual studio 14.0\vc\include\thr\xthread (210): std::_Pad::_Call_func
d:\th\minkernel\crts\ucrt\src\appcrt\startup\thread.cpp (115): thread_start<unsigned int (__cdecl*)(void * __ptr64)>
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFC98602774)
00007FFC98602774 (KERNEL32): (filename not available): BaseThreadInitThunk
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00007FFC998D0D61)
00007FFC998D0D61 (ntdll): (filename not available): RtlUserThreadStart
  46.075 Error CrashHandler.cpp:106: Map tick at moment of crash: 3115935
  46.075 Error Util.cpp:76: Unexpected error occurred. If you're running the latest version of the game you can help us solve the problem by posting the contents of the log file on the Factorio forums.
Please also include the save file(s), any mods you may be using, and any steps you know of to reproduce the crash. 

[Rseding91]

The problem is the chunk iterator becomes invalid when chunks are generated because it points at the old data the chunks were stored in.

I'll work on it and get it fixed for the next version of 0.15 so it's capable of handling when chunks are generated while iterating. You just might not get every single chunk in those instances (ones generated after the iterator moves past that chunk position).

[orzelek]

So one of ore placements causes chunk to be created?

I had an idea to calculate ore patch first, then check for chunks there, trigger their generation and place the patch after all are ready.
I'm not 100% sure how that would work since I would need to store whole ore patch somewhere in global and that would be quite a lot of coordinate pairs with one or two additional values.