Performance optimization - post your saves

[RepairMan]

DeathMers, you implied that it was easy to fix, but you don't seem to know how optimized this game already is.
And just saying that a new version should be less resource intensive means you don't know anything about programming because developers can't make magic happen.

[DeathMers]

DeathMers, you implied that it was easy to fix, but you don't seem to know how optimized this game already is.
And just saying that a new version should be less resource intensive means you don't know anything about programming because developers can't make magic happen.

nope, i didn't say, or implied that it was easy to fix, I don't know, maybe it looks like i did for you, but i didn't. I just mentioned, this problem is known, so the proces of finding bottleneck is pretty much done, and there is only thing left to do - solve/optimize/rework/magic. And yes, I already said I know shit about programming, I presumed this from some posts, where FPS/UPS worked better in tested 0.15, than 0.14 i think, maybe i remembered it wrong.

[Koub]

Please people, stay on topic. This thread is here only for one thing : provide heavy savegames to give the devs examples of things they could optimise.
Anything else should go elsewhere.

[Zim415]

Hoping this will help you guys optimize the bots. Normally I don't have many issues. I've got 20k logistics and 20k construction bots currently. I've had all 20k of either active and normally only dip down to around 30 FPS/UPS (from a normal of 60).

In this save there are about 4k active logistic bots and all 20k construction bots are active.

I have been working on a megabase and planning out my next (massive) expansion phase. I deconstructed what my gut told me was probably too much, but I didn't expect it to get this bad. So far I've bottomed out around 4 FPS/UPS.

I'm currently on the latest main Steam release 0.14.21-0

Love the game guys, keep up the great work and let me know if you need anything else from my end to help!

-Z

[Zim415]

A big FPS slowdown I've seen can be reproduced without mods or a save:

1. Have a large parcel of land that is paved. I mean HUGE.
2. Grab a blueprint of that massive area of paved land so you get a blueprint of nothing but paved area.
3. have the blueprint in hand so you get a "ghosted" copy of the pavement.
4. Alpha blending/rendering drops my FPS to it's knees.

Specs: i7 3.2Ghz, NVIDIA® GeForce® GTX 460M 1.5GB at 1920x1080, max window (not fullscreen).

This might be a limitation of my hardware. I also see this when creating a huge blueprint of some factory set up. The blueprint ghosting kills performance.

That's just your hardware. I get some small slowdown with large blueprints but nothing like what you describe.

I also see this issue, mainly when mousing over a large blueprint. FPS drops to around 5 while UPS seems unaffected. Once I select the blueprint for placement I do not seem to have any massive FPS lag like I see with the preview, but the preview of it definitely tanks the FPS.

Observations around when this happens: My ASUS Strix GTX 1080 stays around 30% as it tends to constantly with the current megabase I'm working on; GPU-Z and ASUS monitoring tools indicate no strain on the GPU. CPU spikes slighty on the factorio process from around 25/30% to 40% when mousing over one of these large blueprints; all in all the CPU is not strained or saturated by other processes.

Considering I'm running an i7 Skylake w/32GB of 2400 DDR4 RAM and an ASUS Strix GTX 1080 @ 4k and I have zero issues with any other games at 4k resolutions under much larger loads on the GPU/CPU; this certainly does seem like some sort of issue somewhere in Factorio's rendering of the previews of large blueprints to me; at least my observations and experience as someone that has been programming for over 30 years tells me this.

Just wanted to mention this since while the OPs specs are a bit low, mine are certainly not and there is no indication of my system not able to keep up per se. Also, like with my previous savegame post, this is on 0.14.21-0, so maybe it's fixed in 0.15.x.

[Rseding91]

I fixed the performance issue with the blueprint previews for 0.15.

[Zim415]

I fixed the performance issue with the blueprint previews for 0.15.

I had a feeling, awesome! You guys rock!

[Rseding91]

Hoping this will help you guys optimize the bots. Normally I don't have many issues. I've got 20k logistics and 20k construction bots currently. I've had all 20k of either active and normally only dip down to around 30 FPS/UPS (from a normal of 60).

In this save there are about 4k active logistic bots and all 20k construction bots are active.

I have been working on a megabase and planning out my next (massive) expansion phase. I deconstructed what my gut told me was probably too much, but I didn't expect it to get this bad. So far I've bottomed out around 4 FPS/UPS.

I'm currently on the latest main Steam release 0.14.21-0

Love the game guys, keep up the great work and let me know if you need anything else from my end to help!

-Z

Fixed for 0.15

[SquarelyCircle]

I found a specific cause of performance degredation.

Ultimately, the massive drop in FPS/UPS is being caused by mods that pick items up off the ground. There was one included within Natural Evolution. I also downloaded one (https://mods.factorio.com/mods/Rseding9 ... Item Collectors) to deal with the unbelievable spam of little artifacts in Bob's Enemies (both Bob's enemies and Natural Evolution created HUNDREDS OF THOUSANDS of "new small artifacts".)

Here's my mods and saves:
Save file: https://drive.google.com/open?id=0B2dPr ... mlOQ2IwYTg
Mods zip: https://drive.google.com/open?id=0B2dPr ... GxRLVFYTXM

(Sorry about having so many mods, which I'm sure complicates the issue, but that's what I was playing with when I discovered the problem)

Load up the save, and you'll immediately experience extremely low FPS for about 30 seconds. I've set up the save to rapidly emphasize the problem, but even under less dramatic circumstances the issue is very noticeable.
Disabling small artifacts in the data.lua of Bob's enemies removes like 200,000 small artifacts and sets my FPS/UPS back up to 60.

[Rseding91]

I found a specific cause of performance degredation.

Ultimately, the massive drop in FPS/UPS is being caused by mods that pick items up off the ground. There was one included within Natural Evolution. I also downloaded one (https://mods.factorio.com/mods/Rseding9 ... Item Collectors) to deal with the unbelievable spam of little artifacts in Bob's Enemies (both Bob's enemies and Natural Evolution created HUNDREDS OF THOUSANDS of "new small artifacts".)

Here's my mods and saves:
Save file: https://drive.google.com/open?id=0B2dPr ... mlOQ2IwYTg
Mods zip: https://drive.google.com/open?id=0B2dPr ... GxRLVFYTXM

(Sorry about having so many mods, which I'm sure complicates the issue, but that's what I was playing with when I discovered the problem)

Load up the save, and you'll immediately experience extremely low FPS for about 30 seconds. I've set up the save to rapidly emphasize the problem, but even under less dramatic circumstances the issue is very noticeable.
Disabling small artifacts in the data.lua of Bob's enemies removes like 200,000 small artifacts and sets my FPS/UPS back up to 60.

There's nothing that can be done about that other than doing the logic differently in the mod.

Finding all of a specific entity in a given area is one of the more CPU intensive things you can do with a mod and that's exactly what the item-collector type mods do.

The items existing is no problem - the game will run great with hundreds of thousands (millions even) of items on the ground. Having a mod iterate over all of them every few ticks is just murder on performance.

[SquarelyCircle]

Yeah, I just checked. https://mods.factorio.com/mods/Kane.Nex ... constuctor Artifact Deconstructor is one of the lowest CPU intensive mods on my list (.002 - .05 ms) What is it doing that the other mods can't?

[Rseding91]

Yeah, I just checked. https://mods.factorio.com/mods/Kane.Nex ... constuctor Artifact Deconstructor is one of the lowest CPU intensive mods on my list (.002 - .05 ms) What is it doing that the other mods can't?

It teleports dropped items from anywhere on the map after they're created instead of picking them up slowly in a given area.

[hansinator]

There's nothing that can be done about that other than doing the logic differently in the mod.

Finding all of a specific entity in a given area is one of the more CPU intensive things you can do with a mod and that's exactly what the item-collector type mods do.

The items existing is no problem - the game will run great with hundreds of thousands (millions even) of items on the ground. Having a mod iterate over all of them every few ticks is just murder on performance.

Does finding items in a specific area iterate over a global list of all entities that exist or does the game have a list of entities per chunk or group of chunks? A quad-tree-like organization in chunk-local lists for stationary things (i.e. not on a belt, no car, etc) might speed up searches.

Another idea might be to have one list for each entity type instead of one list for all, if that's not already the case. The overall amount of list items would be the same and it is not too much of a difference to go through one big list or many smaller, except if you're looking for a specific entity type.

[Rseding91]

There's nothing that can be done about that other than doing the logic differently in the mod.

Finding all of a specific entity in a given area is one of the more CPU intensive things you can do with a mod and that's exactly what the item-collector type mods do.

The items existing is no problem - the game will run great with hundreds of thousands (millions even) of items on the ground. Having a mod iterate over all of them every few ticks is just murder on performance.

Does finding items in a specific area iterate over a global list of all entities that exist or does the game have a list of entities per chunk or group of chunks? A quad-tree-like organization in chunk-local lists for stationary things (i.e. not on a belt, no car, etc) might speed up searches.

Another idea might be to have one list for each entity type instead of one list for all, if that's not already the case. The overall amount of list items would be the same and it is not too much of a difference to go through one big list or many smaller, except if you're looking for a specific entity type.

Entities are stored on a grid inside each chunk so it's relatively fast to iterate over ones in a given area. However, when your area is massive (50x50) that doesn't really help since that's still a ton of area to check.

[hansinator]

Entities are stored on a grid inside each chunk so it's relatively fast to iterate over ones in a given area. However, when your area is massive (50x50) that doesn't really help since that's still a ton of area to check.

I see. Mhh if the search can't be optimized further, maybe the querying can. One last idea: what about an API function like find_entities_filtered only that one would register an ongoing search with a callback which the game uses to notify the script every time a matching entity has just been created. Maybe this could help a mod to reduce the amount of search operations that would be necessary? Having callbacks/triggers that notify of changes to the data model should be more efficient than querying all data of interest in regular intervals (only to see that nothing has changed for most of the time). Many SQL databases offer such a trigger concept with great success.

For many use cases it would need to be able to search in a radius around the current player coordinates without having to update the parameters, though. But that could be solved with two similar functions, one that is a member of surfaces and one that is a member of the player object.

[ssilk]

I would appreciate such filter concepts.
trigger event in this mod only, if some generalizeable special rule matches. See viewtopic.php?f=28&t=33441

A similar request is to trigger time-based events. For example trigger an event every second or trigger one event in 20 seconds.
I explained that in the same thread, search for "Timers" in viewtopic.php?f=28&t=33441&p=212495#p212481

[terror_gnom]

Hi,
heres my safe, it runs at roughly 40 UPS on a i5 6600k with 16gb ddr4 2800mhz ram.

It gets worse if the safe runs for some time (poorly I did shut down the factory in the meantime and installed ups up, so it came back to 60)

[DutchJer]

Hi Rseding,

Me, MachtigeMik3 & CountyCupid Have been working on a rocket per minute base, powered by coal. We had it running with UPS up at 17 ups, after I uninstalled UPS-up I noticed it didn't change much. still 17 ups.

I read this FFF ages ago, but the map just wasn't done. Now it is there, in multiplayer (0.14) we always have to pause the game while someone else joins. otherwise he's catching up for over 20 min. I hope it improves, or you can find some other improvements.

I ran it in single player for the last ups measurements on my 3.5GHz i5 4570K (not overclocked) with 16 GB of ram.

The map can be found here

[Rseding91]

Hi Rseding,

Me, MachtigeMik3 & CountyCupid Have been working on a rocket per minute base, powered by coal. We had it running with UPS up at 17 ups, after I uninstalled UPS-up I noticed it didn't change much. still 17 ups.

I read this FFF ages ago, but the map just wasn't done. Now it is there, in multiplayer (0.14) we always have to pause the game while someone else joins. otherwise he's catching up for over 20 min. I hope it improves, or you can find some other improvements.

I ran it in single player for the last ups measurements on my 3.5GHz i5 4570K (not overclocked) with 16 GB of ram.

The map can be found here

The save runs at 33~ UPS in 0.15 as it is now. Almost all of the time is spent on transport belts (not surprising).

[MightyMike]

that sounds awesome Rseding, that is almost the double of the UPS, so now we have to clean up some belts.