0.17.xx optimizing base for ups

Anything that prevents you from playing the game properly. Do you have issues playing for the game, downloading it or successfully running it on your computer? Let us know here.
jape3
Inserter
Inserter
Posts: 27
Joined: Tue Aug 15, 2017 10:31 pm
Contact:

0.17.xx optimizing base for ups

Post by jape3 »

Recently I've run into slight ups issues with a large-ish base. I'm considering starting a new base, and was wondering what practices should I use to maximize performance? I like to play with belts / inserters + trains for bulk and only use bots for localized deliveries. I've read through a few threads about this issue, but many seem to be quite old and I suspect they might no longer apply.

-Do idle machines contribute to ups loss? So should I demolish any unused or legacy areas of base? Does powering off stuff relieve the load on ups?
-Saw some threads that minimizing belt segments (= use as much undergrounds as possible) helps keep ups up, is this still the case or has this been optimized? Do long belt transfers eat more cpu resources than moving the same amount of product on a short belt, or is it just the amount of interactions that happen with the belt that eat CPU time?
-Do long pipe runs still load the cpu?

Also, is there a list of things that must be run on the "main" game thread? Any things that can be shifted to another cpu thread obviously won't provide a bottleneck with the availability of crazy number of cpu cores these days..

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2339
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by Jap2.0 »

Quick reply to the main points:
- If the machines aren't doing anything, getting rid of them is the best thing to do, and cutting power is the worst
- Underground belt's don't help any more
- Pipes are somewhat multi-threaded, but still aren't great
- Not much that you can use is threaded

Long unedited semi-speculative ramblings
There are 10 types of people: those who get this joke and those who don't.

jape3
Inserter
Inserter
Posts: 27
Joined: Tue Aug 15, 2017 10:31 pm
Contact:

Re: 0.17.xx optimizing base for ups

Post by jape3 »

Thanks for the pointers, especially I had forgotten about the memory latency being a possible bottleneck too. Something to keep in mind while putting together a new rig that hopefully will not choke on a large base :) Factorio not running smoothly is a perfectly legit reason to upgrade from 4-core haswell to something like ryzen 3700x, righ? :D

In the meantime I tried completely removing all biters, turn pollution off and uninstalled rampant enemies mod.. That bumped the ups back up to 60 from the 45-50 range with some room to spare, update now running in about 14.5ms.. Going to see what kind of improvement demolishing old portions of the base & defence perimeter will bring.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 5206
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by eradicator »

I'll add some ramblings too (kinda similar to @Jap2.0).

Pipes: Yes, underground pipes are (still™) better because you need fewer of them for the same distance. If "the big fluid update"™ ever comes this is likely to change.
Pipes: Different "networks" are processed in parralell. So it's (theoretically!) better to not have all your oil/gas connected to one humongous pipe sytem. I don't think you can force seperation of networks though (except maybe with a train wagon)?

Electricity: On the contraty it's best to have a single monolithic network for the whole map.

Belts: Underground belts are no longer better than straight belts (has been optimized).
Belts: Have been heavily optimized so the length of a belt doesn't matter *that* much anymore. But ofc the more belts you have the more items need to be kept in memory.
Belts: Splitter based balancers are worse than loader+large modded chests based balancers. a) because splitters are expensive (compared to straight belt), and b) because splitters also split belt segments.

Robots: Are rumored to be ever-so-slightly worse than belts after the belt optimization. (I.e. they are no longer a no-brainer-always-faster.)

Machines: If you don't need a machine it's best to block it's output (not input or power). Or demolish if you can ofc, stuff that doesn't exist doesn't cost CPU.

Trains: Try to avoid buffer chests. Every item interaction costs something, and chests means two movements for each item to be loaded.
Trains: Pathfinding is quicker the fewer segments you have. I.e. try avoiding needlessly complicated junctions. Place signals sparsely. Mostly relevant if you have a lot of trains (i.e. hundrets).

Generic: You can see belt/rail signals with the debug view (F4/F5).
Generic: You can see how much the rail pathfinder is costing in debug view too.
Author of: Belt Planner, Hand Crank Generator, Screenshot Maker, /sudo and more.
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 5206
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by eradicator »

jape3 wrote:
Sun Oct 06, 2019 8:40 am
Factorio not running smoothly is a perfectly legit reason to upgrade from 4-core haswell to something like ryzen 3700x, righ? :D
You should definetly read https://factorio.com/blog/post/fff-315 for "Ryzen vs Ram Speed". Personally i'm waiting for DDR5 to hit the consumer market as it's supposed to be like at leat 50% faster. According to wiki that'll be about 2021...
Author of: Belt Planner, Hand Crank Generator, Screenshot Maker, /sudo and more.
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

jape3
Inserter
Inserter
Posts: 27
Joined: Tue Aug 15, 2017 10:31 pm
Contact:

Re: 0.17.xx optimizing base for ups

Post by jape3 »

Good pointers, thanks for those.. Especially about RAM I already had 3200MHz modules on the shopping cart but might want to go faster.

I checked the debug list and the biggest time hog is entity update at about 6.8ms.. Transport lines are about 4ms, does that mean all belts or?

Also had a look into big chest mods.. Are there any mods with 4x4 chests? That would be the minimum size that works well for 8 belt lanes.

Zavian
Smart Inserter
Smart Inserter
Posts: 1641
Joined: Thu Mar 02, 2017 2:57 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by Zavian »

Also be aware that Rseding has recommended against using chests with large capacities (which most of the big chest mods have) since that large capacity is actually bad from a performance point of view. (I think the issue is that every time an inserter interacts with the chest, it potentially needs to examine every slot of that chest, so a chest with 200 slots is potentially 10x worse than a chest with 20 slots, from a ups point of view).

User avatar
BlueTemplar
Smart Inserter
Smart Inserter
Posts: 2420
Joined: Fri Jun 08, 2018 2:16 pm
Contact:

Re: 0.17.xx optimizing base for ups

Post by BlueTemplar »

BobDiggity (mod-scenario-pack)

quyxkh
Smart Inserter
Smart Inserter
Posts: 1027
Joined: Sun May 08, 2016 9:01 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by quyxkh »

Sure, I love a good ramble:

Inserter swings matter.

Fluid and heat junctions matter. Straight-line connections matter less, but enough that needing ~10x fewer ug pipes matters.

Travelling-locomotive count matters.

Active assembler/smelter count matters (max-boost assemblers are very good for UPS).

In-flight bot count matters, bots suck for ups over long routes. Ideally you use bots for the shortest delivery routes possible, like to hop a beacon. I've done things like circuit-control production lines so bots concentrate on one request, just loading copper buffers then just loading cable buffers then just loading chip buffers, like that. You can control your requests so the supplies are always available at the shortest hop, do the refill to half what's in the supply buffers and the circuitry's simple.

Circuit signal changes are individually too small to care about, but the cost is per entity that has to check the new value, so one wire running to hundreds of enable-if inserters/belt segments/whatever should have the results reduced to a single pulse, so all those testing entities don't wake up for nothing. (Yes, MadZuri's balanced-loading stunt is awesome early, later you just use bars and drain your buffers on pickup to maintain balance).

Belts are cheap. You can run a monster yellow-belt ammo distribution network all the way around your defenses and just keep extending it.

mrvn
Smart Inserter
Smart Inserter
Posts: 5681
Joined: Mon Sep 05, 2016 9:10 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by mrvn »

jape3 wrote:
Mon Oct 07, 2019 6:50 am
Good pointers, thanks for those.. Especially about RAM I already had 3200MHz modules on the shopping cart but might want to go faster.

I checked the debug list and the biggest time hog is entity update at about 6.8ms.. Transport lines are about 4ms, does that mean all belts or?

Also had a look into big chest mods.. Are there any mods with 4x4 chests? That would be the minimum size that works well for 8 belt lanes.
The frequency (MHz) of your ram isn't the only factor. The clock times are a huge factor too. You can get 3200MHz RAM with CL20 or CL15. The former is 1/3 slower. That's a much bigger factor than 3000MHz vs 32000MHz.

Koub
Global Moderator
Global Moderator
Posts: 7175
Joined: Fri May 30, 2014 8:54 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by Koub »

Hi @OP,

Have you seen these topics :
viewtopic.php?f=49&t=72080
viewtopic.php?f=49&t=74585

They might be of some help.
Koub - Please consider English is not my native language.

mrvn
Smart Inserter
Smart Inserter
Posts: 5681
Joined: Mon Sep 05, 2016 9:10 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by mrvn »

Trains: Avoid a uniform grid pattern.

The path finding uses the A* algorithm. In an uniform grid there are many different paths from A to B that have exactly the same cost and the A* algorithm will explore all of them in parallel until one of them reaches the destination. It can't exclude any of them from consideration since none of them have a cost advantage over others.

Trains: Intermediate stops

When a train is stopped by a chain signal it will try to find an alternate path. The longer the path the longer this takes. In the grid pattern above the number of alternate paths cost time. Even if the network isn't a grid every junction potentially has to explore both ways. A* is pretty good at only exploring paths leading to the destination but it still has to explore around the perfect path trying to see if something is a shortcut or not. And the simplest fact is exploring a long path costs more time than a short path.

If your network consists of separate clusters with long tracks between them you might consider putting train stops at the entry to each cluster and setting them as intermediate stops in the schedule. The path finding will then only search until it reaches the intermediate stop instead of continuing and branching into all the possible ways inside the next cluster.

User avatar
BlueTemplar
Smart Inserter
Smart Inserter
Posts: 2420
Joined: Fri Jun 08, 2018 2:16 pm
Contact:

Re: 0.17.xx optimizing base for ups

Post by BlueTemplar »

mrvn wrote:
Mon Oct 07, 2019 1:27 pm
Trains: Avoid a uniform grid pattern.

The path finding uses the A* algorithm. In an uniform grid there are many different paths from A to B that have exactly the same cost and the A* algorithm will explore all of them in parallel until one of them reaches the destination. It can't exclude any of them from consideration since none of them have a cost advantage over others.
Oh, wow, any benchmarks ? A lot of megabase-builders certainly seem to love uniform grids !
https://www.youtube.com/watch?v=dY2nxVNBHQs
BobDiggity (mod-scenario-pack)

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 5206
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by eradicator »

Watching that video i question my understanding of how facotrio is played. At 4:39 ("138h") he starts terraforming an area of 26*28 blocks, each block being 3*3 chunks, for a total of 6.7 million tiles. Terraforming completes after 9 hours. That's 207 tiles per second or 5 seconds per chunk going from natural forest to perfectly floored railworld. Is this a creative run where infrastructure+concrete is free? Do construction bots move faster than sound? (Serious questions, i've never played any of these these super-microcrafting modpacks).

Building that thing in just 500 hours is to unreal for my brain to comprehend.
It's a shame that there seems to be no map download.
Author of: Belt Planner, Hand Crank Generator, Screenshot Maker, /sudo and more.
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

User avatar
BlueTemplar
Smart Inserter
Smart Inserter
Posts: 2420
Joined: Fri Jun 08, 2018 2:16 pm
Contact:

Re: 0.17.xx optimizing base for ups

Post by BlueTemplar »

Py has alternate recipes for concrete (and ores) and 3 levels of additional con&logibots (BTW Py Industries that adds them is available as a standalone), and the first level of conbots, available in the science1 era (but very expensive at that point), is already 50% faster than vanilla.
BobDiggity (mod-scenario-pack)

mrvn
Smart Inserter
Smart Inserter
Posts: 5681
Joined: Mon Sep 05, 2016 9:10 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by mrvn »

BlueTemplar wrote:
Mon Oct 07, 2019 4:39 pm
mrvn wrote:
Mon Oct 07, 2019 1:27 pm
Trains: Avoid a uniform grid pattern.

The path finding uses the A* algorithm. In an uniform grid there are many different paths from A to B that have exactly the same cost and the A* algorithm will explore all of them in parallel until one of them reaches the destination. It can't exclude any of them from consideration since none of them have a cost advantage over others.
Oh, wow, any benchmarks ? A lot of megabase-builders certainly seem to love uniform grids !
https://www.youtube.com/watch?v=dY2nxVNBHQs
No benchmarks but comments from devs on it in other threads. It's kind of hard to build your megabase so you can turn on/off equal cost paths in a grid pattern to compare the two.

Note that trains repath when they get stoped at chain signals. As your network gets congested this becomes more pronounced. If your network is mostly clear then it's probably irelevant.

PunPun
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Sun Mar 27, 2016 7:08 pm
Contact:

Re: 0.17.xx optimizing base for ups

Post by PunPun »

mrvn wrote:
Mon Oct 07, 2019 1:27 pm
The path finding uses the A* algorithm. In an uniform grid there are many different paths from A to B that have exactly the same cost and the A* algorithm will explore all of them in parallel until one of them reaches the destination. It can't exclude any of them from consideration since none of them have a cost advantage over others.
If your A* implementation does this then you either have a bug or your heuristics are bad. In a uniform grid a properly implemented A* will not explore any alternate paths and goes straight to the goal. That is the whole point of the A* algorithm.

Bilka
Factorio Staff
Factorio Staff
Posts: 3123
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by Bilka »

I'll just leave this here: https://mulark.github.io/test-index.html
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.

mrvn
Smart Inserter
Smart Inserter
Posts: 5681
Joined: Mon Sep 05, 2016 9:10 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by mrvn »

PunPun wrote:
Tue Oct 08, 2019 11:58 am
mrvn wrote:
Mon Oct 07, 2019 1:27 pm
The path finding uses the A* algorithm. In an uniform grid there are many different paths from A to B that have exactly the same cost and the A* algorithm will explore all of them in parallel until one of them reaches the destination. It can't exclude any of them from consideration since none of them have a cost advantage over others.
If your A* implementation does this then you either have a bug or your heuristics are bad. In a uniform grid a properly implemented A* will not explore any alternate paths and goes straight to the goal. That is the whole point of the A* algorithm.
A* expands the path where |path| + H() is the cheapest. The heuristic H() must not overestimate the cost to reach the goal. It usually underestimates it relative to the amount of path remaining. With many paths of identical cost that means the paths that have been least explored will have the lowest estimates cost and will be expanded first. The problem is that in a grid there are many way of going straight. Going 5 north and 5 west is the same as going 5 west and 5 north.

Now in factorio trains can't turn 90° on a spot but have to take a curve. And the curve has a diagonal part. So I'm not 100% sure that in a grid all the paths between 5N+5W and 5W+5N would be the exact same cost. Going NWNWNWNWNW probably differs a bit from NNNNNWWWWW. But the presence of other trains will change those costs too, leading to many path being explored until they hit a train blocking the way and then A* will try all the other "straight" paths that don't yet have a train blocking it.

Anyway, I haven't seen any benchmarks on the effect of grid on train pathing costs but the devs have claimed they are costly. Make of that what you will or make a test map. :)

DaleStan
Filter Inserter
Filter Inserter
Posts: 368
Joined: Mon Jul 09, 2018 2:40 am
Contact:

Re: 0.17.xx optimizing base for ups

Post by DaleStan »

mrvn wrote:
Thu Oct 10, 2019 10:40 am
Anyway, I haven't seen any benchmarks on the effect of grid on train pathing costs but the devs have claimed they are costly.
Do you have a source for that, or are we just supposed to believe you?

The only mention of "grid" in Bilka's link is talking about cars and the collision grid, not trains and a grid-based rail system.

Locked

Return to “Technical Help”