[0.10.3] Constant micro-freezes/frame drops

[Creat]

Hello,

I've had this for a while (at least since 0.10.1, hadn't played in a while before that). The game has micro-freezes for me every ~3 seconds or so (for maybe a few hundred ms). This can be a completely new game (default everything) without any mods, or one of my active games where I'm mostly through red/green/blue science (only purple left) with an appropriate factory size. I have had the game run without this stutter very occasionally, but can't pin down what (if anything) is different. The fact that it happens with games that have run for a while, where I'm just walking around my base, indicates that it's not the

I've put a small video here that illustrates the problem. It's recorded with FRAPS (half-resultion, 60 fps, no sound), but encoded to a 30 fps MP4. This is a completely new game and no mods are loaded, I'm just walking straight to make the stutter visually obvious. Ignore the minimap-flicker, that's just because of recording at half-resolution.

It's not likely that it's my system either, since I don't have this problem with any other game or program (and the PC isn't THAT slow):
Win x64, 8 GB DDR3 / AMD FX-6100 @ 3.3 GHz / Radeon HD 6850 (1 GB DDR5)

[Dark]

Could you post a video/screenshot with debug information enabled?
Just press F5 in game to show it - this will surely help the devs.

[Creat]

Sure, screenshot is attached. Please note that none of the shown numbers fluctuate noticeably when the game stutters. I've spend a good few minutes walking back and forth in the same area and observing the numbers. The one number that obviously does dip when it's stuttering is the FPS counter (no surprise there), but since it shows an average over some amount of time, it dips to either 57.2 or 56.6 (otherwise it's a rock steady 60). Also note that the dips occur just the same when I'm not walking/moving/doing anything at all.

After looking some more: one other number changes when it stutters, which is the first after "Total:". That is also nor much of a surprise and this looks suspiciously like the ms the last frame took to render or something similar. Usually it shows 16.66, 17.5-17.7 after a stutter.

[slpwnd]

In the beginning I thought this was an issue with our map generator. I even made a small optimization in that code. But that is not the case here. If you look at your F5 statistics, the problem is with the Flip. This is waiting for the VSYNC signal to flip the display. The F5 statistics show 2 numbers: first is running average and second is the maximum over some past period (I think 10 seconds or so). There is a problem with the maximum. For some reason the program waited for 106ms for the flip. While in theory it should never need to wait more than 16 (with 60HZ display). Not sure how this can happen though.

[Creat]

Your post gave me a direction to go explore, but nothing I did had any effect whatsoever. I checked my global Graphics settings, even reset them just for good measure, changed the VSync settings (as that seems to be somewhat related) and everything. Doesn't matter weather I force VSync on or off, or set it to "application defined". No change. This in itself is a bit curious to me, as it can't even force VSync to be off (not that I'd ever want that in practice as I'm quite sensitive to tearing).
In case it was related to some other program that's running (really shouldn't be on a hexa-core, but worth a shot) I tried upping the process priority of factorio, but that also didn't change anything.

Frankly, I'm out of ideas as to what I could try. If you want you can of course send me a debug/logging build that just dumps all the information about everything into a log I can upload somewhere, but other than that I can only hope you've got at least a vague idea as to what could be the problem
Or send me any other kind of build for testing purposes, like one where it doesn't wait for the buffer to flip. Might help in locating the problem?

PS: in case that is in any way helpful, I'm a also software developer (though I haven't done anything with games), in case that opens up some options on your end. I'll help any way I can

[Calico]

I've also debated for some time if i should make a bug report. But like the OP, i had some serious trouble pinpointing the problem, so i wasn't sure what to report. Basically i have the same problems, random fps drops. It very often starts when zooming in or out, but it's totally random. I can stand in the same spot, zoom out and get full 60 fps. Doing it again i might have a drop to 30-40 fps, with some small stutter. I only own a "middle-class" computer, but it's somewhat opimized for gaming so it really shouldn't happen. Neither Processor, gfx or RAM seem to be under heavyload.

I first put it it down to map generation or somewhat extensive Train networks, but this doesnt seem to be the case. While heavy fps drops early game are still somewhat rare, they still happen, even if there is nothing going on (aka not exploring). After some hours of play the random fps drops increase slightly, but looking at the debug screen i never was able to pinpoint what exactly caused it.

OFC i lack any knowledge of programing, so this will have to serve. If there are any questions i'll gladly make screens or upload a save.

[Creat]

Just to clarify it for my particular case: The stutter is extremely regular, as shown in the video I linked in the OP. It stutters in that rhythm for the entire game. It doesn't matter what I do (or don't do). New game, (very) old game, always...

[Blackence]

Maybe there's a bug with the "flip" time calculation? I remember the displayed average was more than 16ms on my cheap Intel machine running Arch Linux. I'd expect this to be less than 16ms, so I thought it's a driver bug.

If something other than swapping buffers is hidden in that "flip" timer, this might explain my observation and the OP's freezes.

[ssilk]

Hm. I would give it eventually a try and use a bootable USB stick or so to boot some blank system (windows, Linux), to see, if it is hardware or software related.

[Calico]

Just to clarify it for my particular case: The stutter is extremely regular, as shown in the video I linked in the OP. It stutters in that rhythm for the entire game. It doesn't matter what I do (or don't do). New game, (very) old game, always...

Well, if you put it that way.... yep, seems we have two unrelated problems then. Yours are very regular, looking at the video a second time. Mine are somewhat longer and totally irregular. *sigh* and i was hoping someone else had the same problem as me and could provide insight into it. Back to square one.

[slpwnd]

@Creat When I think about it, this could be related to some custom optimizations we did in the d3d backend of Allegro. This is used only for Windows users. I will ask Kovarex to look into this when he is back from vacation.

[Creat]

Sure, I'll be glad to run some test if need be, just let me know

[Drexir]

Hello,

I've had this for a while (at least since 0.10.1, hadn't played in a while before that). The game has micro-freezes for me every ~3 seconds or so (for maybe a few hundred ms). This can be a completely new game (default everything) without any mods, or one of my active games where I'm mostly through red/green/blue science (only purple left) with an appropriate factory size. I have had the game run without this stutter very occasionally, but can't pin down what (if anything) is different. The fact that it happens with games that have run for a while, where I'm just walking around my base, indicates that it's not the

I've put a small video here that illustrates the problem. It's recorded with FRAPS (half-resultion, 60 fps, no sound), but encoded to a 30 fps MP4. This is a completely new game and no mods are loaded, I'm just walking straight to make the stutter visually obvious. Ignore the minimap-flicker, that's just because of recording at half-resolution.

It's not likely that it's my system either, since I don't have this problem with any other game or program (and the PC isn't THAT slow):
Win x64, 8 GB DDR3 / AMD FX-6100 @ 3.3 GHz / Radeon HD 6850 (1 GB DDR5)

Usually micro-stutters like that are indicating either SLI or Crossfire setup is causing it. Or the processor is not keeping up with the graphics card.

Which may in fact be related to the swap of the buffering since that process is essentially tied both to the CPU and graphics card.

I've also seen micro-stutter happen related specifically to VRAM issues. I use to run 2x HD 5850's (1GB) in crossfire. Considering I was running a high res screen 2560x1600 it didn't take much to eat that video ram causing micro stutters. This game for me easily consumes over a 1GB of VRAM. Granted that may just be due to my high resolution. I recommend using a monitoring utility to see how much VRAM the game consumes.

I remember in the catalyst control center there was always a one or two things I had to turn change to stop the micro stuttering. It was the crossfire setting and I had to set it to AFR Friendly. If I remember correctly it basically means one gpu renders a frame and the second gpu renders the second frame and it alternates. But you don't have dual 6850's so just make sure crossfire is disabled. There was another option I used in some games which was A.I Texture Filtering Quality and set that to Performance. I believe this just helped because it didn't consume as much VRAM. And also changing the power saving feature on the card in the control center sadly I don't know what they call it on AMD's side. As I use nvidia now. But mine says simply "Power Management Mode" depending on the game I will either set it to Adaptive or Prefer Maximum performance (the latter option completely disables power savings).

One more thing as Factorio auto detects your display. It auto sets your resolution and limits your frame rate to the refresh rate that was detected by the monitor. The problem can come in is if you have another program that limits your frame rate. Some programs that do have that feature is monitoring programs or overclocking software. For example EVGA Precision, MSI Afterburner, etc. If you do have one of those programs installed make sure that frame limiting is not enabled. As it can cause problems as you might imagine cause the external software is conflicting with the game.

The biggest concern and i probably should have listed this first. But Windows 7 has a power saving feature. It basically will park your cores on your processor. And if the core is parked and the core needs to unpark. While this process is extremely fast. It's not fast enough for gaming purposes. As it introduces stuttering. The quickest way to find out is if your computer is idling. Go to your task manger -> resource monitor. On the side panel you should see all your cores listed along with graphs. If they are currently parked it should tell you right below the Core #. Honestly this core parking thing is really for laptops as a battery saving feature. Has no place on the desktop. You should be able to disable it in your control panel then advanced power options. In their you should find something along the lines of Processor Power Management and in their you should find a minimum and maximum setting just set both of them to 100%. If for some reason that's a bit much to follow. You can alternatively download this program: http://bitsum.com/about_cpu_core_parking.php

It essentially changes the same settings for you just with a cleaner / more intuitive interface.

On the slim chance none of that works... Well just post back here and I'll try to help you further troubleshoot it.

[Creat]

Usually micro-stutters like that are indicating either SLI or Crossfire setup is causing it. Or the processor is not keeping up with the graphics card.

Which may in fact be related to the swap of the buffering since that process is essentially tied both to the CPU and graphics card.

I've also seen micro-stutter happen related specifically to VRAM issues. I use to run 2x HD 5850's (1GB) in crossfire. Considering I was running a high res screen 2560x1600 it didn't take much to eat that video ram causing micro stutters. This game for me easily consumes over a 1GB of VRAM. Granted that may just be due to my high resolution. I recommend using a monitoring utility to see how much VRAM the game consumes.

I had checked GPU utilization and VRAM usage, but didn't post anything about it as it wasn't anything noteworthy. The GPU itself is at about 20-25%, GPU (dedicated) is at is at 450-600 MB (mostly depending on the multisampling setting in the graphics options), so about 50% is still available.

I remember in the catalyst control center there was always a one or two things I had to turn change to stop the micro stuttering. It was the crossfire setting and I had to set it to AFR Friendly. If I remember correctly it basically means one gpu renders a frame and the second gpu renders the second frame and it alternates. But you don't have dual 6850's so just make sure crossfire is disabled. There was another option I used in some games which was A.I Texture Filtering Quality and set that to Performance. I believe this just helped because it didn't consume as much VRAM. And also changing the power saving feature on the card in the control center sadly I don't know what they call it on AMD's side. As I use nvidia now. But mine says simply "Power Management Mode" depending on the game I will either set it to Adaptive or Prefer Maximum performance (the latter option completely disables power savings).

None of the Crossfire settings are available unless you actually have multiple GPUs. There are no (official?) power options for AMD cards (in the Catalyst Control Center), the only thing remotely related is the OverClock section (which is disabled). The only thing in the advanced windows power settings that could affect the GPU is the PCIe Link State Power Management (which is off for High performance by default).
I had already tested the Texture Filtering quality, since I have gone through most (if not all) of the Catalyst Control Center 3D options already (as mentioned before). And I do agree that it's likely that this helped for you is related to the reduced GPU memory requirement.

One more thing as Factorio auto detects your display. It auto sets your resolution and limits your frame rate to the refresh rate that was detected by the monitor. The problem can come in is if you have another program that limits your frame rate. Some programs that do have that feature is monitoring programs or overclocking software. For example EVGA Precision, MSI Afterburner, etc. If you do have one of those programs installed make sure that frame limiting is not enabled. As it can cause problems as you might imagine cause the external software is conflicting with the game.

No frame-limiting programs or GPU-Tweakers of any description are running (or installed). The only thing that does access the GPU is FRAPS for monitoring, to show the frame rate on my keyboard's LCD. Weather or not that is running also doesn't change anything (tested this as one of the first things). I'm usually running in windowed mode, so the resolution is set dynamically depending on the window size, but I've tried full screen with no change to the issue (as usual ).

The biggest concern and i probably should have listed this first. But Windows 7 has a power saving feature. It basically will park your cores on your processor. And if the core is parked and the core needs to unpark. While this process is extremely fast. It's not fast enough for gaming purposes. As it introduces stuttering. The quickest way to find out is if your computer is idling. Go to your task manger -> resource monitor. On the side panel you should see all your cores listed along with graphs. If they are currently parked it should tell you right below the Core #. Honestly this core parking thing is really for laptops as a battery saving feature. Has no place on the desktop. You should be able to disable it in your control panel then advanced power options. In their you should find something along the lines of Processor Power Management and in their you should find a minimum and maximum setting just set both of them to 100%. If for some reason that's a bit much to follow. You can alternatively download this program: http://bitsum.com/about_cpu_core_parking.php

It essentially changes the same settings for you just with a cleaner / more intuitive interface.

Well, that is just half true to be honest. No, core parking is not just for laptops, though it's obviously more important there. It's also for desktops, and not just because a parked core does use less power than one just running on low clock speeds (via. normal clock scaling), which you may or may not care about. I have tinkered a bit with the options (no, it didn't solve or change the severity of the problem even if I disable core parking), and core parking can be expected to be actually helpful for performance on my system. I'm using an AMD 6-core (FX-6100), which also allows for 'Turbo' mode, increasing the clock speed by 300 MHz as long as temperature and TDP allow it, but by 600 MHz if only half the cores are active. No matter how low I set the percentage in parkcontrol, there are never less than 3 cores active, which aren't even close to maxed out with factorio.
But there's more: My CPU essentially consists of 3 two-core-modules on the chip (both are real cores, this is not like hyper-threading), but both cores in one module share the level 2 cache. This is important, because when only 3 cores are awake, it's cores 1, 3 and 5 that are sleeping (0, 2, 4 are active), meaning that you're much less likely to get things like cache thrashing, as only one core puts strain on the cache and it's not really shared in that moment, reducing the chance of collisions significantly. This is most likely not a major performace boost, but might make some performance hits resulting from multiprocessing less likely.
Lastly, please check the benchmark-article that is linked on the page of the utility you linked (parkcontrol). Unfortunately, they only have a single benchmark there, which is a WinRAR benchmark. But notice how not a single AMD processor has any performance difference with/without core parking there? Also worth mentioning is that the article is rather old (2011), and it came out less than a month after the processors hit the market (the AMD line mine belongs to at least). There were multiple patches for this mechanic since (by MS as well as by AMD). This doesn't mean that this can't cause problems (as it obviously does), but it is extremely unlikely to do so with games that aren't extensively multithreaded, and then you could relatively easily notice it by seeing cores constantly park/unpark while the game is running. Basically you need the "core-requirement" (i.e. number of active threads) to change often for this to be a possible issue as a core that is busy will never go to sleep.

On the slim chance none of that works... Well just post back here and I'll try to help you further troubleshoot it.

None of this works. Sorry

While I very much appreciate the effort and time you've put into the post, I did actually know most about these things (not that you could necessarily know that). Most likely this isn't even a graphics problem in the classical sense: None of the GPU parameters are anywhere near a 'problem zone' and the problem pattern would be very unusual if it were one. The points you mention wouldn't really explain such a huge delay either (100-150 ms) in a precise interval every few seconds. Waking up a sleeping CPU core is in the order of microseconds, not 100s of milliseconds (even for C6 aka. 'parked', where it might reach the low ms, it's actually surprisingly hard to find any info on this).

As slpwnd has pointed out, the problem seems to be in the VSync-related code, which for currently unknown reasons waits way too long (or does something that isn't VSync). Let me also emphasize again that it's only factorio that has this issue.
It might also be an unexpected interaction between factorio and some other program that happens to be running (let me emphasize again though that it's only factorio that has this issue). It's really difficult for me to nail down, and believe me I've tried. As stated in my initial post, there are times when the issue doesn't appear, though it's relatively rare. I first thought it was just having Chrome running (which also uses DirectX to render web pages), but then it still happened after a reboot with nothing started. I don't have a lot of programs always running/started at boot (like tools in the system tray). But I've tried checking them as well as I could, and couldn't isolate a culprit. Note that those that I do have are usually companions to driver (like Logitech Gaming Software for mouse&keyboard, AMD CCC thingy) or things like anti-virus and Steam (which I've all exited/stopped for tests).
The problem with testing is, that sometimes the issue doesn't occur, which means I can't test anything explicitly. But it also doesn't mean that it's not one of those things that are running when it doesn't occur, as there isn't anything more running when it reoccurs. I've been checking for a total of at least 4-5 hours now, and I just can't nail it down. I couldn't find any combination where it didn't occur after (re-)testing all your suggestions for example, there was basically nothing running anymore (less things running than after a reboot) and the problem was still there. I tried rebooting and since the issue hasn't come back consistently, but it did happen for a short time when I first started Chrome again, but then stopped occurring again with nothing changed and most notably Chrome still running.

I'm just gonna wait for Kovarex to return and have a look. Maybe it'll jump out at him when he's looking at the code

[Drexir]

Yes obviously you are more tech savvy than I was aware. Admittedly I've never owned an AMD processor. So that processor / architecture might handle parking and unparking better. My intel i7-920 games that utilize the hyper threading suffer a pretty noticeable micro-stutter. In some games it will have a bigger hit where the fps will just tank as it's waiting for the CPU to respond. A huge delay can occur just from my experience and reading other's experience with at least intel hyper threading. The game is not hugely taxing on the CPU because it's well optimized to utilize multi threading. The game for me utilizes all the cores and also you never want a core to be closely to fully maxed out. Ideally you want the resources spread out as evenly as possible. So no thread is waiting on another for too long. There are many components in a computer that ultimately it's impossible to avoid a bottleneck. With that the downfall with so many components relying on each other and operating at different speeds comes when you have a bottleneck that isn't easy to spot.

For example (since this has been a common problem I've seen with others recently). The video ram may not be full or even close to full. So you find no bottleneck in regards to that. But a less common bottleneck is the actual speed of the video memory it self. I'd say a good guideline to follow is your bus width * the speed of memory clock which gives you the memory bandwidth. You take your maximum in your case 1GB * it by the refresh rate to see how much memory bandwidth is required. Granted this is max limit. You're never gonna know how a game is going to handle video memory, how the caching is going to be utilized, etc. But at least if your card's memory bandwidth is more than what is required it's easily ruled out. If not you can overclock your video memory and see if there's any difference. But that's off topic.

Not to mention it may as well just be something in the games code causing the issue. So everything I've said is void. lol

[slpwnd]

As slpwnd has pointed out, the problem seems to be in the VSync-related code, which for currently unknown reasons waits way too long (or does something that isn't VSync).

AFAIK in Allegro some of the drawing is actually postponed to when the display is flipped (vsync).

[Creat]

I'm very sorry to report that these issues are still not fixed for me in 0.11.6. It's basically the same as I've described, often I get the freezing, sometimes not. Can't figure out when exactly they happen, it's still the only game I own that does this as well.

Any news on this front? Anything I can try, turning on extra/extensive logging/profiling or something?

[kovarex]

I know what is the reason of it, it is bad synchronisation of the main loop timer and vertical synchronisation. Sometimes it is luckily synchronised, so You have 60fps, sometimes not. It bugs me a lot as well, I just didn't have time to take a look at it, as we have been solving lot of other (mostly more serious) bugs. It will definitely be solved until 0.11 gets stable.

[Creat]

Ah tat is very good news, thank you very much!

[Talon]

GoogleTranslation

Would be great.
I still play Factorio version 0.10.12.
From version 0.11+ runs unstable.
fps breaks constantly and when I activate mods in Version 0.11+, it is even worse.
Even without mods unstabiel fps of 58-60 / 73-75 ups and after a short playing time down to 14-30 fps, the ups (if I saw correctly) 45.
Option setting no matter what setting.

0.10.12 version expires after 30 hours by mod still stabiel and varies 45-60 fps in the late game.
It is a shame that I can not test 0.11+ because it runs so bad.
Because Factorio is really a fine game.

My System: intel dualcore 2 with 2.2ghz, 4gb ram (3), windows 32bit, nvidia geforce gtx 260 version 2 with 896mb