Why is my UPS low?

[Kidster3001]

Hi! Love your game. So many lost hours

I don't have a specific bug to report. I can't quite figure out this issue.
I have plenty of CPU to spare, plenty of GPU to spare, lots of available Memory. Why is the UPS struggling? Not any one CPU core is pegged so I would assume even a single threaded app could go faster. I'm very computer literate, using, building, maintaining PCs since the early 80's. Currently a git/gerrit sys admin for a 500 person developer group. Don't worry about being too technical.

Trying to build a megabase with some Bob's but don't like the experience below 50 UPS. Could really use some advice. I'm not using very many heavy duty addons.
Bob's Logistics
Bob's Power. Recently destroyed all Nuclear and put up 30GW of solar to try for better UPS.
Bob's Inserters
LTN. Less than 100 trains. LOTS of train signals. 95% 2-4 trains with a signal every 3 wagons, everywhere. A few 2-2, primarily for Nuclear Fuel deliveries.
Schall Machine Scaling. Recently added to try for better UPS.
Schall Virtual Signals
Dectorio. For the concrete, waterfill and stack size of landfill
Warehouses
Loader Redux. NOT using with trains at all, just warehouses to Assemblers.

Based on all my testing, LTN seems to have the biggest impact on UPS but I still think I'm missing something considering my system is not hindering the game. I play at 4k resolution but my FPS has never been lower than my UPS.

Thanks!

[Loewchen]

Your cpu is the limiting factor, see: viewtopic.php?f=7&t=68653

[Koub]

Train pathfinding and warehouses are my best guesses for your low UPS.
Also, if you control your production by switching off parts of your base via power, consider switching to control production by starving the inputs/outputs instead.

[darkfrei]

Train pathfinding and warehouses are my best guesses for your low UPS.

Can the mod https://mods.factorio.com/mod/SmallChests fix the warehouse UPS dropping?

[Kidster3001]

@Loewchen
Yes, I assumed the CPU would be the primary cause of UPS issues. However, the cpu is only 22% utilized across my entire system, Factorio is not even maxxing out one core. I do not believe the CPU is the problem here.

@Koub
Thanks for the confirmation. I've been considering reducing signals from every half train length to 2x or 3x train length. but 7276 signals currently out there

@darkfrei
I didn't realize the number of chest slots affected UPS. I had assumed a chest was just X of these and Y of those, regardless of stack size or number of slots. I'll look into how that affects it. Unfortunately, the production lines in my base would consume SmallChests 6 slots in far less than 1 second. The bots would have it all gone before the train could even start moving after dropping it off. OMG, 10 Warehouses with million+ quantities of various types of concrete... Lot of things to test!

Thanks for the replies.

[Oktokolo]

Yes, I assumed the CPU would be the primary cause of UPS issues. However, the cpu is only 22% utilized across my entire system, Factorio is not even maxxing out one core. I do not believe the CPU is the problem here.

If nothing else is limiting, then context switching and RAM access latencies are the limiting factors.
So you might need faster RAM.

[Honktown]

I'd go with Oktokolo. The lower right CPU graph is the fourth core's hyperthreading. The very high percent implies that your ram is indeed the "problem" (the CPU is waiting often). Lua scripting wouldn't be able to run entirely in CPU, which is why turning off mods with a lot of scripting would improve the UPS. Although Lua could cache certain things, it's still a dynamic language and they tend to be very sensitive to how much memory switching is going on. Factorio itself uses a lot of memory swapping, but it's optimized so you don't see a UPS drop under normal play.

Edit: I should ask: do you have any thin landfill bridges? I've seen a few developers/people mention that the AI pathfinding does poorly with really thin land bridges, or can have issues if enemies are on islands that pollution reaches.

[Kidster3001]

@Honktown
I play with all enemies disabled. Pollution disabled. Smoke disabled. Didn't really think about the impact of LUA. My background is mostly hardware, so I was concentrating on maxxing out the utilization.

@Oktokolo
Ram speed eh? This Skylake system _is_ a few years old I suppose. CPU-Z says I have DDR4 running at 2133. I can overclock to 2400 for a quick test. Thanks.

Caching and context switching makes sense. It's basically invisible with the tools I'm using to look around.

Anyone have experience with the newer Intel i9 series? They support quad channel RAM. Could make a big difference if that is part of the problem.

[Koub]

Anyone have experience with the newer Intel i9 series? They support quad channel RAM. Could make a big difference if that is part of the problem.

Clearly, quad channel will not improve your gaming experience over dual channel (at least with Intel CPU, I don't know the impact on AMD CPU). You'd get something from it only on memory bandwidth benchmarks, or very specific server applications maybe, but no difference while gaming.

If you were to choose ram for playing, as long as you have enough of it :
- Speed matters most for performance, with diminishing returns. Speaking for DDR4, anything from 2.4 GHz and above is OK, but if you can get DDR4 3200 for roughly the same price as DDR4 2400, go for it. Example : I can find a 32 GB dual stick 137€ for DDR4 2400 and 143€ for DDR4 3000, same brand. I'd take the DDR4 3000.
- Lower timings for a given speed have very small influence.

The optimal gaming PC for Factorio would have a 2-3 GB Video card (performance doesn't matter that much), the fastest 4+ core CPU possible, with 16 or 32 GB of "as fast as you can afford" DDR4. And a M2 PCIe 3x NVMe SSD for the data as the cherry on top.

[Honktown]

1) There's a script-time debug print isn't there? If one mod is eating the most performance, there's at least one way to significantly decrease Lua's memory accesses that is not-so-obvious. I can look into it and bring it up to the author.

2) Are you playing zoomed out? (thinking about the 4k) Zooming out would need a lot more entity information and whatnot, so there'd be more memory accesses. I've had issues (very enemy related) where the more of them I could see the worse it got, and it wasn't FPS related.

You can also go over your rail system, and deconstruct chunks of it. You might find something was done slightly "non-perfect", and the trains are having problems (train navigating is pretty costly, I've heard).

[PunPun]

Anyone have experience with the newer Intel i9 series? They support quad channel RAM. Could make a big difference if that is part of the problem.

Clearly, quad channel will not improve your gaming experience over dual channel (at least with Intel CPU, I don't know the impact on AMD CPU). You'd get something from it only on memory bandwidth benchmarks, or very specific server applications maybe, but no difference while gaming.

That might have been the case 10 years ago but all the recent gaming benchmarks I have seen that were not gpu limited say quad channel gives you 20-40% more fps. That is a pretty big difference. Screenshot from a benchmark from wolfgang

dualvsquad.PNG (324.33 KiB) Viewed 9107 times

If you were to choose ram for playing, as long as you have enough of it :
- Speed matters most for performance, with diminishing returns. Speaking for DDR4, anything from 2.4 GHz and above is OK, but if you can get DDR4 3200 for roughly the same price as DDR4 2400, go for it. Example : I can find a 32 GB dual stick 137€ for DDR4 2400 and 143€ for DDR4 3000, same brand. I'd take the DDR4 3000.
- Lower timings for a given speed have very small influence.

The optimal gaming PC for Factorio would have a 2-3 GB Video card (performance doesn't matter that much), the fastest 4+ core CPU possible, with 16 or 32 GB of "as fast as you can afford" DDR4. And a M2 PCIe 3x NVMe SSD for the data as the cherry on top.

Here you are correct. To further back you up I took a screenshot of a benchmark from JayzTwoCents.

ramspeeds.PNG (370.51 KiB) Viewed 9107 times

Not any one CPU core is pegged so I would assume even a single threaded app could go faster.

Except factorio is not actually singlethreaded. it's only mostly singlethreaded. And there are times where the "main" thread needs to wait for other threads to finish their tasks for the tick before it can continue. Your screenshot shows that factorio is using 17% cpu which is almost one and a half cores worth of usage. It is very possible that you are indeed cpu bound in this case.

[Kidster3001]

Thanks everyone. my TODO list
1. OC the RAM. Buy better if it helps.
a. if RAM speed helps, more channels will also help. More bandwidth is more bandwidth.
2. OC the CPU. I can get 5GHz on this one for a day but won't run it that hard all the time.
a. _WAS_ going to buy an i9 10980XE in February. 18 Core, quad-channel. Now maybe 14 core since it clocks higher.
b. My other hobby is Video editing, don't think it's possible to have too many cores for that.
3. Optimize the rail system.
a. Never any logjams or stuck trains, but a LOT of signals. Admittedly more than I need.
b. I've always tried to keep my rail system as compressed as possible, let the trains tail-gate if they want to. They never do though, due to how signals and train acceleration work in the game. I doubt if I could make them run within two train-lengths of each other.
4. Go learn LUA, from the inside.
a. Modding WoW is obviously not enough to understand how the language works.
b. Never been a fan of non-compiled languages. Python is the only acceptable one to me. Perf is more important than ease of use.
c. Learning LUA may not help it run faster, but it's definitely something I would do just for fun.
5. Get the right computer to run Factorio well.
a. I'm only at 1719 hours played. There is obviously a LOT more to come.
6. Have my old-man brain remember to come back here someday and update this thread.

I do use Infinizoom, but only for special cases, like reaching to grab another stack of inserters from my taxi (with LongReach). I stay within normal game zoom when building and running around. It does hit FPS and UPS significantly when you zoom out beyond normal game limits.
I've tried playing with all the options in the game. The only thing that made a difference was Smoke and it wasn't much. Resolution, sprite quality, RAM residency options, etc. didn't do anything measurable. Turning off Pollution helped quite a lot.

Haven't had less than 32GB of RAM in a long time. Will have at least 64GB in Feb. but speed is what matters here it seems.
Using M.2 NVMe PCIe SSD as boot drive for Windows now. Steam is on 1TB NAND SSD. Doubt if changing that would make a difference _during_ the game. Saves are 50MB but only take a second to finish.
Currently have GTX 1070, 8GB. Might go 1080. 20xx series would be overkill/waste for what I need.

[Honktown]

If you're going to follow-up on this, I'll mention the Lua thing: since Lua's function are dynamic, they're not uh, compiled and written permanently. Every time a Lua function is called, if it's not a primitive, it has to be checked in the local function environment, and if not found, go up to the next environment, and so on. If a function repeatedly calls for a function outside of a local environment, that incurs an overhead penalty every time. The solution is to define a local variable which stores the environment's function. It might be as simple as 'local outsidefunction = outsidefunction'. I've never used that myself, so I could be wrong. Stored this way, the look-up finds it immediately. In an object/template style I think an object-specific function would only be one environment up, with a small total table (it's all the objects functions). If you're using a more general style then the whole relevant environment has to be searched.

[Koub]

Currently have GTX 1070, 8GB. Might go 1080. 20xx series would be overkill/waste for what I need.

If you're video editing - and most important video encoding in H265 a lot, you might consider getting a GTX 1660 or a RTX 2060+ : Nvidia provides the second generation of NVEnc, which crushes any CPU for H265 encoding : https://en.wikipedia.org/wiki/Nvidia_NV ... U10x/TU116. The first generation (up to GTX 1080 generation is still blazing fast, but the lack of support of HEVC-B frames is problematic.

[Kidster3001]

Thanks guys. I've made some headway.
Removing >1000 train signals.
Modified some track layout to provide better pathfinding. Forcing it's hand I suppose. Longer routes, but they make fewer decisions.
Optimized my circuit networks where possible. I use Warehouses, 4 per stop. I use circuits to control inserters between the Warehouses. Saved a few inserters x100 stops.
UPS is staying in the 48 range now.

Forgot that I upgraded the RAM on this thing a couple years ago. Mix n match some 2133 with 2400. Unfortunately the 2133 won't do any better so it's all running at that speed.

I understand localized variables are important where speed matters. CPU can waste dozens of cycles fetching globally. We used to have to occasionally optimize C++ code and/or compiler options to make sure each functions compiled machine code fit in a single page. Crossing page boundaries can be expensive also. Tweaking a custom JVM to do that automatically in another project was a nightmare.

I still use h264 for video. Haven't made the switch yet. Also encode all the audio to AAC. Plenty of work for the CPU's to keep busy.

[darkfrei]

You can try the PXL, the UPS can be also higher.

[Koub]

Optimized my circuit networks where possible. I use Warehouses, 4 per stop. I use circuits to control inserters between the Warehouses.

Do your warehouses have a lot of slots ? Have you tried replacing them with regular chests (or even with the modded 6-slots chests) ?

[Kidster3001]

Each Warehouse is exactly as wide as a train wagon, 6.
Each Warehouse has 1800 slots, vs 288 in 6 Iron Chests.
180,000 Plates or 360,000 Circuits per warehouse. Four warehouses per train stop.

I had no idea the number of slots in a container affected performance. I'll try to work around it somehow but I am afraid my entire factory would need a major overhaul if I remove the large buffers. I may send 20 trains over and over until those buffers are full, then they stop requesting until nearly empty. I fill other stops in the meantime.

@darkfrei
Cool addon. Intended primarily for systems with not much graphics memory or just for effect?

[Koub]

I had no idea the number of slots in a container affected performance.

That's the first thing we told you : that warehouses could be the source of your issues .

Train pathfinding and warehouses are my best guesses for your low UPS.

Train pathfinding and warehouses are my best guesses for your low UPS.

Can the mod https://mods.factorio.com/mod/SmallChests fix the warehouse UPS dropping?

You've even acknowledged our warnings :

@darkfrei
I didn't realize the number of chest slots affected UPS. I had assumed a chest was just X of these and Y of those, regardless of stack size or number of slots. I'll look into how that affects it. Unfortunately, the production lines in my base would consume SmallChests 6 slots in far less than 1 second. The bots would have it all gone before the train could even start moving after dropping it off. OMG, 10 Warehouses with million+ quantities of various types of concrete... Lot of things to test!

Here's more explanations from the devs :
viewtopic.php?f=23&t=20666

[darkfrei]

@darkfrei
Cool addon. Intended primarily for systems with not much graphics memory or just for effect?

Just for nice picture, then I become the info that it helps for UPS.

The stack size must not change the performance, just set to items that you store in chests the stack size to 1000 - 20 000.