[0.9.2 x64 build] Severe memory leak causing stutter?

[Thrombo]

Hi,

I'm playing on build 0.9.2 on a new map generated in this build. No Mods are used.

I have only a modest factory (just got automated smelting and an ugly setup for red/green going. When I start the game, Factorio grows to ~820MB mem util. After I load the map (attached) I get consistent memory increase along with graphical stutter every couple seconds (which seems to correspond with page fault frequency).

Video of process explorer performance graph for Factorio process after the game has been started. Game is running in the background without user input: http://screencast.com/t/u6CKXFkvi

Within about an hour or two memory utilization will grow to 5GB or so.

Edit:
Okay - I ran this through DebugDiag/DbgSvc with memory tracking for some time with immediate call stack recording turned on. Debugger was attached after the map was loaded. If you need I can send the full process dump, but it's several GB large. Otherwise I'm happy to help however else I can. One function address sort of sticks out in the report - offset 0x559027. In the call stack provided obviously ignore the LeakTrack at the top.

Further Edit: Over a 15-minute timeframe, I end up with ~765MBytes outstanding allocations from this offset.

Function details
Function Factorio+559027
Allocation type Heap allocation(s)
Heap handle 0x00000000`00000000
Allocation Count 1822541 allocation(s)
Allocation Size 249.19 MBytes
Leak Probability 95%

Top 10 allocation sizes by allocation count
48 Bytes - 1,045,219 allocation(s)
168 Bytes - 276,893 allocation(s)
16 Bytes - 112,898 allocation(s)
112 Bytes - 111,152 allocation(s)
464 Bytes - 111,151 allocation(s)
176 Bytes - 100,830 allocation(s)
40 Bytes -19,528 allocation(s)
360 Bytes - 18,422 allocation(s)
328 Bytes - 9,423 allocation(s)
8 Bytes - 8,044 allocation(s)

Top 10 allocation sizes by total size
2.08 MBytes - 64.36 MBytes
464 Bytes - 49.18 MBytes
48 Bytes - 47.85 MBytes
168 Bytes - 44.36 MBytes
176 Bytes - 16.92 MBytes
112 Bytes - 11.87 MBytes
360 Bytes - 6.32 MBytes
328 Bytes - 2.95 MBytes
16 Bytes - 1.72 MBytes
40 Bytes - 762.81 KBytes

Call stack sample 1

Address 0x00000001`4020be88
Allocation Time 00:00:05 since tracking started
Allocation Size 168 Bytes

Function Source Destination
LeakTrack+13277
Factorio+559027
Factorio+55abeb
Factorio+70cac
Factorio+21108f
Factorio+210ed6
Factorio+210ae0
Factorio+201acb
Factorio+2019d4
Factorio+201c1b
Factorio+bb06
Factorio+207bfa
Factorio+206fd2
Factorio+2092ec
Factorio+7661f
Factorio+201c78
Factorio+208573
Factorio+da995
Factorio+1939aa
Factorio+22e38a
Factorio+545223
Factorio+560073
Factorio+56021a
kernel32!BaseThreadInitThunk+d
ntdll!RtlUserThreadStart+1d

[ataaron]

Same for me
Game freezes for a few seconds then runs again, freezes for a few times more in shorter periods of time then it repeats

[Thrombo]

I have to say, for the amount of work put in and detail provided here, I'm a bit disappointed that I didn't get a prompt reply at least mentioning what the allocating function is responsible for.

[AlexPhoenix]

Win8, x86, 34hours game(started at 0.9.2) no memory leaks.
game have stable memory using.

may be this?
https://forums.factorio.com/forum/vie ... =30&t=2748

[kovarex]

I'm sorry for not replaying we are solving the bugs one by one, we will get to this one eventually, we are thankful for the report for sure.

[Calico]

At first i also thought there are no frame rate drops.... but after playing 0.9.2 for a while i noticed that the frame rate counter drops from a stable 60 to 37 for a second or two. It's not that it "feels" like the game is slowing down, without the counter i likely wouldn't have noticed. It's also very inconsistent... sometimes it happens every 10 seconds or so, sometimes it doesn't happen at all.

It doesn't seem related to map generation, i've got a fair amount explored and rarely move close to the edges of the map. I'll try to narrow the cause down, but a memory leak could likely be the cause.

[kovarex]

At first i also thought there are no frame rate drops.... but after playing 0.9.2 for a while i noticed that the frame rate counter drops from a stable 60 to 37 for a second or two. It's not that it "feels" like the game is slowing down, without the counter i likely wouldn't have noticed. It's also very inconsistent... sometimes it happens every 10 seconds or so, sometimes it doesn't happen at all.

It doesn't seem related to map generation, i've got a fair amount explored and rarely move close to the edges of the map. I'll try to narrow the cause down, but a memory leak could likely be the cause.

The engine generates one chunk per 10 seconds around the player up to the distance 20 chunks far (that is quite far), maybe it is the reason as this was added in 0.9.0.

The world is slowly generated around the player up to big distance, so exploring should not matter regarding the existance of enemies.

[Calico]

I'm fairly sure that map generation isn't the cause, i kinda have everything explored that is 40-50 chunks away. I have the habit of setting up small radar outposts along the edges of my pollution zone (which is fairly large).

The frame rate drops are still very inconsistent. Yesterday they came regurlary, today i don't notice any. Got the game running for several hours now, but still nothing.

[kovarex]

Ok, I'm going to do some leak detector tests (haven't done these some time), wish me luck to find something

[kovarex]

Do you have the savegame you used to generate the memory leaks?

[kovarex]

I have to say, for the amount of work put in and detail provided here, I'm a bit disappointed that I didn't get a prompt reply at least mentioning what the allocating function is responsible for.

I'm sorry but I'm not able to get that from the info provided, mainly because there are no debug symbols in the binary.

[Thrombo]

Okay, understood. I thought maybe there would be since development is happening rather fast. How would you like me to send the file to you? It's only about 6mb but the forums only allow 1mb attachments.

[kovarex]

Hi, I have to keep the limit, because the place on our ftp is limited as well, could you upload it using www.dropbox.com/ for example?
P.S. I have been trying to find memory leaks half a day yesterday and I found just one small memory leak, that couldn't be cause of this.

[Calico]

I can't find the source of the stutter either.... monday (?) i was sure there was some unnatural stutter. But the more i play the more i'm uncertain that there really was something. I tried everything i can think off, frum running the factory maxed out by artificially dialing up the demand, to exploring at the very far corners of the map.

I do see some very tiny drops in fps now and then, but we are talking about like 0,4 frames here. Maybe the problem (if there is one) only occurs after very very long uninterrupted sessions, like 8 hours+? Didn't have the time to do more then 2-3 hour runs the last few days.

[tiggyd]

I can confirm that I am suffering from the same FPS issues. At first I thought it was just my old laptop but the game runs perfectly 80% of the time. Every now and again, the FPS will drop to a stutter, but only if I move. If I stop moving everything seems to run at full speed again.
Keep up the good work, I'm having loads of fun regardless of the performance issues

[AlexPhoenix]

I can confirm that I am suffering from the same FPS issues. At first I thought it was just my old laptop but the game runs perfectly 80% of the time. Every now and again, the FPS will drop to a stutter, but only if I move. If I stop moving everything seems to run at full speed again.
Keep up the good work, I'm having loads of fun regardless of the performance issues

laptop you say?
which video cards it have?

[tiggyd]

I can confirm that I am suffering from the same FPS issues. At first I thought it was just my old laptop but the game runs perfectly 80% of the time. Every now and again, the FPS will drop to a stutter, but only if I move. If I stop moving everything seems to run at full speed again.
Keep up the good work, I'm having loads of fun regardless of the performance issues

laptop you say?
which video cards it have?

Haha I was surprised it ran it at all! I have an old Sony VGN-CR353 Intel Core 2 duo with a Radeon Mobility X2300 video card. The video card was made in hell to punish us as finding a working driver for use in Win 7 was a nightmare, but I managed to find a way! As I said, most of the time the game runs really smoothly, and when frame rates drop, usually a reload of my game will do the trick

[Dysoch]

i think i have found the source of this bug

i had it 2, when i had factorio 0.9.0 to 0.9.4.
i deleted my entire factorio folder, and installed factorio 0.9.5 (installer file)

the stutter was gone (YAY)

i believe the stutter was caused by files that were all over the folder.
to better explain.

i installed factorio 0.7.5 at the time, and used the updater ingame to update all the time, worked perfectly.
but after updating to factorio 0.9.0, my mod folder moved, along with the config files.
instead of being in appdata/roaming/factorio, they now were in the folder of factorio itself (at my pc, d:games/factorio)
only after installing 0.9.5 with the installer, all mods, saves and config were moved back to appdata, and it fixed the stutter i 2 was having.
@devs, any thoughts on this?

[kovarex]

I have no idea to be honest.

[AlexPhoenix]

i installed factorio 0.7.5 at the time, and used the updater ingame to update all the time, worked perfectly.
but after updating to factorio 0.9.0, my mod folder moved, along with the config files.
instead of being in appdata/roaming/factorio, they now were in the folder of factorio itself (at my pc, d:games/factorio)
only after installing 0.9.5 with the installer, all mods, saves and config were moved back to appdata, and it fixed the stutter i 2 was having.
@devs, any thoughts on this?

strange, when i made copy of my factorio(one for game, second for develop mods), then i start second copy, and all configs and saves was copied to folder with first factorio, and both of them now simple store their data in folder with it.
and no memory leaks at all(Win 8.1, x64, no mods except my own(WIP mod with even more inserters, but which must be optimized, and mod with additional event(OnTechResearchedEvent and WIP timers))