Extremely long load times (Modded Savefile) [0.16.51]

Anything that prevents you play the game properly. Do you have issues with paying for the game, downloading it or properly running it on your computer? Let us know here.
Post Reply
Nihilius
Burner Inserter
Burner Inserter
Posts: 7
Joined: Fri Nov 02, 2018 4:48 pm

Extremely long load times (Modded Savefile) [0.16.51]

Post by Nihilius » Fri Nov 02, 2018 4:55 pm

Hello everyone,

this is actually my first time writing on this forum despite having the game since version 0.11.

I really enjoy it and like playing it heavily modded. Currently I am running a map in which i put almost 300 Hours. It is running with a lot of mods (Bobs, Angels and various quality of life mods). I seem to have hit a wall when it comes to loading the game. Starting up Factorio itsself is fine, takes around 1 minute which is really good given the amount of mods i am running. However loading the actual savegame of my map used to take upwards of 10 minutes, same with saving it.
As of right now it doesnt load at all, it seems to try for the first 5-6 minutes then it just completely freezes my computer.

I am happy for any help/advice provided.


What did i do ?
I built my factory, it got larger and larger, and suddenly the game wont let me load my savegame anymore.

What happened?
Upon trying to load my savefile first the RAM usage of my PC goes up to 100%, after some minutes it drops down to almost nothing, then the Disk usage shows Factorio taking upwards of 20 Mb\s. Then my PC completely locks up with the only fix beeing a hard reset.
The last File i managed to load was only around 100 KB smaller than the current one, no mod changes between the two files.

What did you expect to happen instead?
I expected Factorio to load the game, maybe take some time, but eventually load the game.
I am aware that loading a very big map (playtime approaching 300 hours, savefile size >50 MB) can take some time, but i didnt expect it to not load at all.

Attachments:
factorio-current.log
Factorio Log File
(19.43 KiB) Downloaded 9 times
https://drive.google.com/open?id=1UgV4h ... YCGgXBwpBu (savefile and mods Folder)


Nihilius
Last edited by Nihilius on Sun Nov 04, 2018 6:55 pm, edited 1 time in total.

Loewchen
Global Moderator
Global Moderator
Posts: 4651
Joined: Wed Jan 07, 2015 5:53 pm

Re: Extremely long load times (Modded Savefile)

Post by Loewchen » Fri Nov 02, 2018 5:01 pm


Nihilius
Burner Inserter
Burner Inserter
Posts: 7
Joined: Fri Nov 02, 2018 4:48 pm

Re: Extremely long load times (Modded Savefile)

Post by Nihilius » Fri Nov 02, 2018 7:00 pm

Thanks for the quick response, that was my mistake, i didnt read that section.

I will update the Post with all the necessary things once i get back home and away from the Laptop.

BR
Nihilius

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 2523
Joined: Tue Jul 12, 2016 9:03 am

Re: Extremely long load times (Modded Savefile)

Post by eradicator » Fri Nov 02, 2018 9:15 pm

Might be a mod (or several by the description) that is storing large amounts of script data. You can open your save.zip and look at script.dat, if it's very large (several megabyte) then you "just" need to find out which of your mods is causing it.

Nihilius
Burner Inserter
Burner Inserter
Posts: 7
Joined: Fri Nov 02, 2018 4:48 pm

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by Nihilius » Sun Nov 04, 2018 6:57 pm

I have updated the Initial Post with the Things required in the FAQ for making a report. I also looked at the script.dat. The size is a little over 6Mb after i unpack it. However i dont know if that is too big or too small.

Loewchen
Global Moderator
Global Moderator
Posts: 4651
Joined: Wed Jan 07, 2015 5:53 pm

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by Loewchen » Sun Nov 04, 2018 7:17 pm

The level.dat unpacks to over 2GB, I doubt you will be able to load that save again, you will simply run out of ram.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 2523
Joined: Tue Jul 12, 2016 9:03 am

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by eradicator » Sun Nov 04, 2018 9:24 pm

Well, well. What an unpleasant suprise. I tried loading the savegame and it did indeed freeze my system. But not quite as expected.

Here's what happend:
  • Fetched the save.zip and loaded it while on the map i was currently playing.
  • Minute or so later my music player suddenly stops playing. (Loading bar hadn't moved a single pixel yet.)
  • Few seconds later firefox crashes due to lack of memory.
  • Minute or two later system recovers.
  • The music starts playing again, but i have to manually restart firefox.
  • I open taskmanager and see RAM usage of 25G (i have 32G physical ram). Shortly later it drops back to 6G usage.
As my system recovered i also did get a full crash log including the memory allocation fail unlike OP.

Code: Select all

2279.173 Info Scenario.cpp:136: Map version 0.16.51-0
2371.747 Warning Map.cpp:305: Map loading failed: bad allocation
2372.358 Time travel logging:
  59.650 Player 0 ran command: a = game.player.surface.create_entity{type='flying-text',text={'item-name.iron-plate'}}
  70.316 Player 0 ran command: a = game.player.surface.create_entity{name='flying-text',text={'item-name.iron-plate'}}
  77.815 Player 0 ran command: a = game.player.surface.create_entity{name='flying-text',text={'item-name.iron-plate'},position=game.player.position}
  83.666 Player 0 ran command: a = game.player.surface.create_entity{name='flying-text',text={'item-name.iron-plate'},position=game.player.position}
2372.447 Error CrashHandler.cpp:373: Exception Code: e06d7363, Address: 0x000007fe09d5aea8
ModuleBase: 0x000007f674c30000, ImageSize: 013ee000, RelativeAddress: 9512aea8
2372.472 Error CrashHandler.cpp:393: Exception Context:
rax=0000000000000000, rbx=00000009c24de060, rcx=0000000000000000,
rdx=0000000000000000, rsi=00000009c24de690, rdi=00000009c24dcfa0,
rip=000007fe09d5aea8, rsp=00000009c24dc160, rbp=00000009c24dc7f0,
 r8=0000000000000000,  r9=0000000000000000, r10=0000000000000000,
r11=0000000000000000, r12=00000009c24dc2a0, r13=0000000000000000,
r14=00000009c24dd5b8, r15=0000000000000000
2372.586 Crashed in C:\Windows\system32\KERNELBASE.dll (0x000007fe09d10000 - 0x000007fe09e04000)
Factorio crashed. Generating symbolized stacktrace, please wait ...
c:\cygwin64\tmp\factorio-build-gumsut\libraries\stackwalker\stackwalker.cpp (924): StackWalker::ShowCallstack
c:\cygwin64\tmp\factorio-build-gumsut\src\util\logger.cpp (408): Logger::writeStacktrace
c:\cygwin64\tmp\factorio-build-gumsut\src\util\logger.cpp (521): Logger::logStacktrace
c:\cygwin64\tmp\factorio-build-gumsut\src\util\crashhandler.cpp (169): CrashHandler::writeStackTrace
c:\cygwin64\tmp\factorio-build-gumsut\src\util\crashhandler.cpp (420): CrashHandler::SehHandler
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 000007FE09D7D640)
000007FE09D7D640 (KERNELBASE): (filename not available): UnhandledExceptionFilter
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 000007FE0CBEC247)
000007FE0CBEC247 (ntdll): (filename not available): RtlNormalizeString
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 000007FE0CB9A776)
000007FE0CB9A776 (ntdll): (filename not available): _C_specific_handler
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 000007FE0CB99B2D)
000007FE0CB99B2D (ntdll): (filename not available): RtlLookupFunctionEntry
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 000007FE0CB9AE04)
000007FE0CB9AE04 (ntdll): (filename not available): DbgPrint
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 000007FE0CB9B25D)
000007FE0CB9B25D (ntdll): (filename not available): RtlRaiseException
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 000007FE09D5AEA8)
000007FE09D5AEA8 (KERNELBASE): (filename not available): RaiseException
f:\dd\vctools\crt\vcruntime\src\eh\frame.cpp (1186): __CxxCallCatchBlock
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 000007FE0CB65093)
000007FE0CB65093 (ntdll): (filename not available): RtlRestoreContext
c:\cygwin64\tmp\factorio-build-gumsut\src\map\map.cpp (282): Map::load
c:\cygwin64\tmp\factorio-build-gumsut\src\scenario\scenario.cpp (167): Scenario::loadFactory
c:\cygwin64\tmp\factorio-build-gumsut\src\scenario\parallelscenarioloader.cpp (125): ParallelScenarioLoader::doLoad
c:\program files (x86)\microsoft visual studio\2017\buildtools\vc\tools\msvc\14.12.25827\include\thr\xthread (232): std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(ParallelScenarioLoader * __ptr64,MapDeserialiser * __ptr64,enum InputType,InputHandler * __ptr64,MultiplayerManagerBase * __ptr64,bool),ParallelScenarioLoader * __ptr64,MapDeserialiser * __ptr64,enum InputType,InputHandler * __ptr64,MultiplayerManagerBase * __ptr64,bool>,std::default_delete<std::tuple<void (__cdecl*)(ParallelScenarioLoader * __ptr64,MapDeserialiser * __ptr64,enum InputType,InputHandler * __ptr64,MultiplayerManagerBase * __ptr64,bool),ParallelScenarioLoader * __ptr64,MapDeserialiser * __ptr64,enum InputType,InputHandler * __ptr64,MultiplayerManagerBase * __ptr64,bool> > > >::_Go
c:\program files (x86)\microsoft visual studio\2017\buildtools\vc\tools\msvc\14.12.25827\include\thr\xthread (211): std::_Pad::_Call_func
d:\th\minkernel\crts\ucrt\src\appcrt\startup\thread.cpp (115): thread_start<unsigned int (__cdecl*)(void * __ptr64)>
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 000007FE0C351842)
000007FE0C351842 (KERNEL32): (filename not available): BaseThreadInitThunk
ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 000007FE0CB9DF11)
000007FE0CB9DF11 (ntdll): (filename not available): RtlUserThreadStart
Stack trace logging done
2404.492 Error Util.cpp:67: Unexpected error occurred. If you're running the latest version of the game you can help us solve the problem by posting the contents of the log file on the Factorio forums.
Please also include the save file(s), any mods you may be using, and any steps you know of to reproduce the crash.
2409.928 Device reset external.
2441.159 Error CrashHandler.cpp:227: Heap validation: success.
2441.159 Creating crash dump.
2441.321 CrashDump success
2441.322 Error CrashHandler.cpp:330: Unhandled exception type: .?AVbad_alloc@std@@
2441.324 Error CrashHandler.cpp:339: Unhandled exception: bad allocation

Nihilius
Burner Inserter
Burner Inserter
Posts: 7
Joined: Fri Nov 02, 2018 4:48 pm

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by Nihilius » Mon Nov 05, 2018 12:32 am

Well than, that is not so great. I think i will try to load the game again, maybe it works after one hour of waiting or something like that, then i will deconstruct my starter base. If that does not fix it i´ll have to wait for the RAM prices to drop to a reasonable level again and upgrade from 16 GB up to 32, maybe even 64 GB.

Overall not the greatest news but at least i have confirmation.

Its just strange that it doesnt load that savefile, yet the save a couple hours back it can load (although it takes 15 minutes to do so). And for that one the savefile is just around 100 Kb smaller.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 2523
Joined: Tue Jul 12, 2016 9:03 am

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by eradicator » Mon Nov 05, 2018 8:47 am

Well, i haven't done too much with crash reports, but as the last line says "Unhandled exception: bad allocation", maybe this should be moved to bug reports?

Nihilius
Burner Inserter
Burner Inserter
Posts: 7
Joined: Fri Nov 02, 2018 4:48 pm

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by Nihilius » Mon Nov 05, 2018 12:39 pm

That is indeed strange, it crashes on your PC, and then works while writing a crash report.

I have also decided to rework the smelting layouts i have used on that map. Meaning i will start incorporating modules in my designs, not fully moduled/beaconed since that is too overpowerd for my taste, but some speed modules here and there to make the ratios fit together in a better way and to overall reduce the amount of machines in a production line. So far i have redworked about 30% of the blueprints and i expect to reduce the amount of machines by around 40%
That should help the save file size on my next map in a positive way.

Rseding91
Factorio Staff
Factorio Staff
Posts: 7946
Joined: Wed Jun 11, 2014 5:23 am

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by Rseding91 » Wed Nov 07, 2018 12:06 pm

Looking at the save it looks like one of the mods you're using made a bunch of tiny electric networks (3400~) which has a lot of memory overhead.
If you want to get ahold of me I'm almost always on IRC and Discord.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 2523
Joined: Tue Jul 12, 2016 9:03 am

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by eradicator » Wed Nov 07, 2018 12:46 pm

Rseding91 wrote:
Wed Nov 07, 2018 12:06 pm
Looking at the save it looks like one of the mods you're using made a bunch of tiny electric networks (3400~) which has a lot of memory overhead.
Can you please make it stop before it overcommits on memory and other applications start to crash or the system freezes? :D

Rseding91
Factorio Staff
Factorio Staff
Posts: 7946
Joined: Wed Jun 11, 2014 5:23 am

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by Rseding91 » Wed Nov 07, 2018 12:50 pm

eradicator wrote:
Wed Nov 07, 2018 12:46 pm
Rseding91 wrote:
Wed Nov 07, 2018 12:06 pm
Looking at the save it looks like one of the mods you're using made a bunch of tiny electric networks (3400~) which has a lot of memory overhead.
Can you please make it stop before it overcommits on memory and other applications start to crash or the system freezes? :D
No. That's outside of our control. We simply ask the system for memory and it fulfills the request we use it. If it fails the request then Factorio will exit.
If you want to get ahold of me I'm almost always on IRC and Discord.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 2523
Joined: Tue Jul 12, 2016 9:03 am

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by eradicator » Wed Nov 07, 2018 1:09 pm

Rseding91 wrote:
Wed Nov 07, 2018 12:50 pm
eradicator wrote:
Wed Nov 07, 2018 12:46 pm
Rseding91 wrote:
Wed Nov 07, 2018 12:06 pm
Looking at the save it looks like one of the mods you're using made a bunch of tiny electric networks (3400~) which has a lot of memory overhead.
Can you please make it stop before it overcommits on memory and other applications start to crash or the system freezes? :D
No. That's outside of our control. We simply ask the system for memory and it fulfills the request we use it. If it fails the request then Factorio will exit.
How about: Ask the system how much physical memory it has. If the total amount you have requested is above that then stop. Because there's no point in running factorio with half the data in swap?

Nihilius
Burner Inserter
Burner Inserter
Posts: 7
Joined: Fri Nov 02, 2018 4:48 pm

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by Nihilius » Thu Nov 08, 2018 12:11 pm

Rseding91 wrote:
Wed Nov 07, 2018 12:06 pm
Looking at the save it looks like one of the mods you're using made a bunch of tiny electric networks (3400~) which has a lot of memory overhead.
What would be the most effective way of finding out which mod did that ?

I have started a new base, i would just like to get some blueprints from the old one. All that would need to happen is me beeing able to load it just once.

Rseding91
Factorio Staff
Factorio Staff
Posts: 7946
Joined: Wed Jun 11, 2014 5:23 am

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by Rseding91 » Thu Nov 08, 2018 12:59 pm

I managed to get your save file to load on my computer after it consumed 57 GB of RAM (32 GB real + page file). The memory usage was all of the equipment grids in trains you're using. I deleted every locomotive, fluid wagon, artillery wagon, and cargo wagon and uploaded the save here: https://www.dropbox.com/s/hqq3wjegmlicj ... d.zip?dl=0

That only takes 3.5~ GB to get it to load.

Now, to look into why that was taking up so much memory.
If you want to get ahold of me I'm almost always on IRC and Discord.

Nihilius
Burner Inserter
Burner Inserter
Posts: 7
Joined: Fri Nov 02, 2018 4:48 pm

Re: Extremely long load times (Modded Savefile) [0.16.51]

Post by Nihilius » Thu Nov 08, 2018 11:29 pm

Rseding91 wrote:
Thu Nov 08, 2018 12:59 pm
I managed to get your save file to load on my computer after it consumed 57 GB of RAM (32 GB real + page file). The memory usage was all of the equipment grids in trains you're using. I deleted every locomotive, fluid wagon, artillery wagon, and cargo wagon and uploaded the save here: https://www.dropbox.com/s/hqq3wjegmlicj ... d.zip?dl=0

That only takes 3.5~ GB to get it to load.

Now, to look into why that was taking up so much memory.

Now thats why i love the community and the Devs of this game so much ! Thanks for the good news. I had a look at my mod folder as well, turns out there were two mods trying to generate vehiqle equipment grids, the one integrated in Bobs Mods and another standalone one. I am just gonna remove both of them for now.

Also i can get my Blueprints from that savefile and i wont have to spend another 5 hours on designing the make everything part of the Factory. :D :D

Post Reply

Return to “Technical Help”