Performance optimization - post your saves

[TOGoS]

Am I missing something there?

You are not. Due to the terrain generator's nature as a massively SIMD virtual machine, it does not support lazily-evaluated function calls or if-else chains, meaning that every subexpression needs to be evaluated for each point on the map. Adding procedure delimiters probably won't help much in this case, since that only deduplicates the work of identical expressions across different named noise expressions (within one, identical subexpressions are automatically detected and only calculated once).

I can imagine solutions involving new, table-backed expression types, similar to spot noise, but that may have to wait until Factorio gets open-sourced (or someone gets me source access and a long vacation. )

[Hornwitser]

Slowdown with 200 long trains

I've built a 40 km long railway to get to those juicy 200M ore fields with default map gen options (call it a challenge if you may). To get the ore back to base there are about 200 trains moving at any given point in time consisting of 32 wagons each so that's about 6.4k wagons in motion. Unfortunately that seems to be too much for our server and it runs at about 49 UPS on this save. (For reference my underclocked Ryzen 3800x desktop does about 64 UPS on this save.) I had originally planned to build 3x 40 km fully saturated rails with 200 trains each to ship enough ore for 10kSPM, but now it looks like that's going to run at 16 UPS so that wont happen .

Save at https://www.hornwitser.no/cloud/s/ZqAsFgQWJH5Zdqo

A casual look at visual studio's profiler shows Train::update using 75% of the game update.

[Muppet9010]

Do you count continuous max pathfinder UPS for a few unit groups to get through many layers of walls as a performance issue ?
It only becomes an organic issue on very large bases, but can easily cause the pathfinder to stay at its max processing indefinitely.

Save that shows the pathfinder struggling. This map was being played with some light mods, but non affect pathing, units or the base, so save is without them.
https://www.dropbox.com/s/aqul3yldpiice ... s.zip?dl=0

I ask as in testing I found making the pathfinder focus more on going forwards helped it a lot in these situations.

Code: Select all

game.map_settings.path_finder.fwd2bwd_ratio = 20
game.map_settings.path_finder.goal_pressure_ratio = 5

If you apply the `fwd2bwd_ratio` and `goal_pressure_ratio` values above when loading the save, the current pathfinder requests all complete much faster than without. When measuring how long it takes for the 2 path requests on the east of the base (north east and south east) the ratio setting changes make the requests take 20% of the vanilla Factorio time. It does vary a bit between requests based on how straight on they are trying to meet across the wall.

Separately increasing the tiny default cache sizes (short and long) to a massive number (1000) also helps a lot 9event with default forwards ratio) as it at least avoids the need to recalculate duplicate paths for a long time afterwards, although I appreciate that will have impact on how biters approach built obstructions on the map or make use of new landfilled paths. But biters will just bump in to new obstructions and attack it, and the UPS saving of this seems worth it in our larger MP games.

Code: Select all

game.map_settings.path_finder.short_cache_size = 1000
game.map_settings.path_finder.long_cache_size = 1000

[ptx0]

https://drive.google.com/file/d/17hrs1z ... tOkhD/view

This is a rather large base that pushes the limits of the game, such that with 283 trains i'm pushing 7ms update time for the Trains category on my 5800H laptop, on Linux.

I was wondering if there's anything I can do about this, because when things really get moving (e.g. when the space science train finally fills up and goes on the move) i've got about 48FPS in the game.

that said, it's pretty spectacular, and there's a number of build style changes i can improve on my end, but i see it spending a lot of time in itemStack::sumCounts and Train::update.

[Rseding91]

https://drive.google.com/file/d/17hrs1z ... tOkhD/view

Looking at the save; it's a matter of scale and how many wagons there are. Every additional cargo wagon adds to the cost of the wait conditions "inventory empty/full" and item-counts. Since the trains you're using are so huge they take more time.

[mrvn]

Slowdown with 200 long trains

I've built a 40 km long railway to get to those juicy 200M ore fields with default map gen options (call it a challenge if you may). To get the ore back to base there are about 200 trains moving at any given point in time consisting of 32 wagons each so that's about 6.4k wagons in motion. Unfortunately that seems to be too much for our server and it runs at about 49 UPS on this save. (For reference my underclocked Ryzen 3800x desktop does about 64 UPS on this save.) I had originally planned to build 3x 40 km fully saturated rails with 200 trains each to ship enough ore for 10kSPM, but now it looks like that's going to run at 16 UPS so that wont happen .

Save at https://www.hornwitser.no/cloud/s/ZqAsFgQWJH5Zdqo

A casual look at visual studio's profiler shows Train::update using 75% of the game update.

Have you tried a few different designs?

1) First how does the time change with 400 trains of 16 wagons or 100 trains with 64 wagons?

2) Your stacker uses train stops. Have you tried a single stop at the exit of the stack and just signals inside? Or a single stop at the entry of the stacker, right before the chain signal that causes trains to recalculate what stop to choose in the stacker. That might cut down on the number or path findings and their cost.

3) I see your mines are loading 8 wagons at a time and then the train moves forward a bit. Have you considered using small trains at the mine (say 8 wagons) that then ship to a collection point where you move the ore into trains with 32 (or more) wagons.

https://drive.google.com/file/d/17hrs1z ... tOkhD/view

Looking at the save; it's a matter of scale and how many wagons there are. Every additional cargo wagon adds to the cost of the wait conditions "inventory empty/full" and item-counts. Since the trains you're using are so huge they take more time.

Having smaller trains at the mines would avoid the item-counts and make inventory empty/full cheaper. At the collection point you could avoid the empty/full condition and use a timer instead combined with only allowing a train into the station when there is enough ore to load or enough space to unload (so the timer is always sufficient).

Alternatively why not just wait for inactivity?

Note: disable the train stops when the miners signal all the ore is used up so you don't end up with empty trains going round and round.

4) You have a lot of small segments on the train track. Having signals closer than the smallest gap between trains has no benefit. The gap between trains depends on the train speed though. So it makes sense to have signals closer together at the start and end to keep the gap between trains small. But really a train line that busy just cries for longer trains.

[ra_ki@freenet.de]

Not sure if this thread is still active but in case, I'd like to submit my base savegame. It is modded with ultimate belts and factorissimo and other mods. Over the last weeks, FPS/UPS has dropped down to 13, which is not playable anymore. CPU is a Ryzen 5600X running with 64GB RAM and an Radeon RX 6800 XT.

Entities updates is extremely high. I already tried to destroy all Factorissimo buildings but that didn't help at all (3 FPS maybe?).

Any idea what casues this perfomance drop is appreciated.


https://1drv.ms/u/s!Am3_71JmLZfMu6UkBeQ ... A?e=2bbX2Y

[Rseding91]

Any idea what casues this perfomance drop is appreciated.

You have 14,444 active assembling machines, 8,280 active boilers, 14,503 active generators, 54,568 active inserters, 5,568 active radars and so on.

The "why" is: you build a LOT of stuff.

Untitled.png (115.51 KiB) Viewed 4108 times

[ra_ki@freenet.de]

LOL

Okay, I knew that the base isn't small. But what are ways to bring it up to 30 PFS at least? Start a new game?

[Rseding91]

LOL

Okay, I knew that the base isn't small. But what are ways to bring it up to 30 PFS at least? Start a new game?

Remove things/scale back.

[robot256]

LOL

Okay, I knew that the base isn't small. But what are ways to bring it up to 30 PFS at least? Start a new game?

That's an impressive map... First loading, I got 7 UPS. After I killed all the biters with the command line (there were 45000 enemy units mostly trying to pathfind to your base after artillery strikes), it went up to 12 or 13 FPS on my laptop, so nearly double. Your next biggest consumer of CPU are inserters, at 22ms. Really, the only way to deal with this is to start deleting old sections of your base, and upgrade everything to high-throughput beaconed setups.

IIRC, inserters don't go to sleep when feeding a fluid-consuming building, and maybe the same thing applies to circuit network-connected inserters. Reducing the number of inserters in those situations will have a more immediate impact, since other ones can go to sleep when the building is full/not operating.

[ra_ki@freenet.de]

Thanks guys! Now at least I know, there is no hidden mechanic I missed. I guess, it's time to start a new map then.

[TheKillerChicken]

I just put yotta-scale values into the game and run with 44m construction robots due to a 4.2b stacking and watch my memory scream in agony. Thankfully I have a mainframe motherboard, so I can configure where the bandwidth goes and prioritise the dual-QPI system and the NUMAs. Weirdly enough, my server UPS stays at 60 the whole time. Must be the bandwidth prioritising on my server also.

[wild_dog]

Got one, maybe.

This is a modded save with Space Exploration and Logistics Train Networks.
have 4ms times for trains and 10ms times for entity updates, which is fair I think.
Electric network update times is +2ms even though I'm mostly switched to solar, but the large amount of consumers that's pretty reasonable as well.
Just under 1ms in transport lines.

But also have 8ms on cirquit network, which I think is a bit much.

True, I'm using the cirquit networks on practically every train stop to automate LTN.
But I've invested quite a lot of time to optimise this: made the train stop BP as efficient as i can make them and switch old designs over to the new. Signal soruces which update (multiple signals) (nearly) every tick such as the contents of tanks or other collected 'content status' signals I'm buffering to only update once every 60/360/3600 ticks, and mostly using a centralized clock signal to not have too many individual clocks chewing up cirquit update times.

Additional insight would be appriciated.

https://wetransfer.com/downloads/c83557 ... 013/094968

[Rseding91]

Additional insight would be appriciated.

https://wetransfer.com/downloads/c83557 ... 013/094968

You have a very large amount of updatable control behaviors:

• Inserter: 64437
• Lamp: 8756
• LogisticContainer: 1640
• StorageTank: 4874
• TrainStop: 941
• TransportBelt: 6183
• Accumulator: 10
• RailSignal: 76

[wild_dog]

Would it help to put multiple inserters behind a single logic combinator?

As in, if i have 32 inserters with an everything>0 setting on the same network, instead isolating the inserters to their own network, giving them a check=1, and a everything>0 logic combinator outputing a check=1 as the only input on the network?

Depends a bit on what triggers a check update for an inserter, and if a check is only triggered if the signal in question going in changes, and every inserter checks individually, this could reduce quite a bit on my stations. If that is the case, doing this would mean only 1 logic combinator gets updated on every input signal change, and only when the condition changes, the inserters get updated. Could be a ~32 times reduction in that case, and I'd like to confirm my understanding of what triggers the updates is correct.

[Rseding91]

Every one of those control behaviors updates every tick.

[zer0t]

I got another strange one...
Background info: I'm partly rebuilding my factory. So nearly 0 activity, except from basic energy (nuclear reactor) and ~17k construction bots flying around.

Save file 1:
- Download: https://nx7119.your-storageshare.de/s/H25Qgid4w7X42DQ
- Info: Everything seems to be normal, performance ~40FPS on my PC

Save file 2:
- Download: https://nx7119.your-storageshare.de/s/Zj4QomdTZoPki69
- Info: Same game, ~5 minutes after save file 1, performance drops below 1FPS (unplayable)

As far as I understand it, construction robots are causing the problem.

1.png (145.72 KiB) Viewed 1189 times

[Rseding91]

The slowdown you're seeing is trains re-pathing or attempting to re-path as rails are added and removed. Since you have some 600~ trains they almost all try to re-path as the rails get removed or rebuilt.

[zer0t]

Ah that makes sense. Missed that one. Thx for the fast response and help