Friday Facts #421 - Optimizations 2.0

[Stalinlover22]

Let´s go! Keep the hard work

[Theikkru]

I am honestly surprised you guys are iterating over everything to compute the electricity.[...]

All this.
A while ago I made a proposal about consolidating the power buffers on combinators so that they could be calculated per-network instead of per-entity, but it got tossed for unrelated reasons. Similar logic could be used to consolidate the power calculations for other machines as well, which would greatly reduce the complexity and memory burden of those operations.
Obviously you'd have to have special case handling for entities on multiple electric networks, but even then, I don't see why they can't still be thrown into common buckets.
Off the top of my head, you could try something like:
-entities in only one network are only registered to that network
-entities in multiple networks are registered to all relevant networks
-power draw is tentatively calculated with all registered entities in each network (so multi-network machines are counted multiple times)
-calculate "power rebates" based on how many times multi-network machines were over-counted
-distribute those power rebates among the networks based on how heavily they're loaded in the tentative calculations (this is the only part where you'd need an algorithm with any sophisticaion)
-finalize & apply post-rebate calculations

[Brambor]

I am honestly surprised you guys are iterating over everything to compute the electricity.[...]

All this.
A while ago I made a proposal about consolidating the power buffers on combinators so that they could be calculated per-network instead of per-entity, but it got tossed for unrelated reasons. Similar logic could be used to consolidate the power calculations for other machines as well, which would greatly reduce the complexity and memory burden of those operations.
Obviously you'd have to have special case handling for entities on multiple electric networks, but even then, I don't see why they can't still be thrown into common buckets.
Off the top of my head, you could try something like:
-entities in only one network are only registered to that network
-entities in multiple networks are registered to all relevant networks
-power draw is tentatively calculated with all registered entities in each network (so multi-network machines are counted multiple times)
-calculate "power rebates" based on how many times multi-network machines were over-counted
-distribute those power rebates among the networks based on how heavily they're loaded in the tentative calculations (this is the only part where you'd need an algorithm with any sophisticaion)
-finalize & apply post-rebate calculations

Would it work with entity that draws from 3+ electric networks?
Theoretically you can have a machine draw from unlimited amount of networks.

[picklock]

I'm always fascinated by how you developers manage to keep optimizing the game and improving its performance. You also question things that have been implemented for years and try to optimize them as well. I've only seen a few development teams do that. Keep up the good work.

[Theikkru]

Would it work with entity that draws from 3+ electric networks?

Yes; just find the least loaded network for each rebate bucket (doesn't have to be per-entity) and apply rebates to the other networks for that bucket.

Theoretically you can have a machine draw from unlimited amount of networks.

There is a geometric limitation...

[Brambor]

Would it work with entity that draws from 3+ electric networks?

Yes; just find the least loaded network for each rebate bucket (doesn't have to be per-entity) and apply rebates to the other networks for that bucket.

Theoretically you can have a machine draw from unlimited amount of networks.

There is a geometric limitation...

I don't know the limitations of mods, but here I made an inserter that is a part of... 120 networks. I wrote it wrong, it is technically limited.

You cannot simply limit it to 2, 4, or like 256, because that would break some mods. So I wrote unlimited.

Your PC memory also limits it. You cannot have more networks than the largest number your PC can store. It is ridiculously huge (not just 64 bit number, more like 4 Tera-bit number), But technically limited. If you have a 64-bit PC, you can address 2**64 (16 EB), so the largest number on that PC would theoretically be 2**(2**64), which is still limited, some might incorrectly call it unlimited.

0eNqd2u9q4kAUBfB3mc9pyfyf8VVKWbQdloBGMXHZUnz3tS1LF/aemHO/VbG/zk1PjpPou9ntL+10HsbZbN7N8HIcJ7N5ejfT8HPc7j+em99OzWzMMLeD6cy4PXw8mi67ad7Ow3E0184M42v7bTb2+tyZNs7DPLQv5fPB24/xcti18+0F0u935nSchs8fb3/txrjamTezeQjXa/cf4VYR3i4Qfh3hF4iwjogLRFxHlAUirSJCv0DkdYRbIMo6IiwQdR2RFgjbU9lyomGpcMmGo9IlG56Kl2wEKl+yEamAyUaiEiYbmYqYbBQqY7JRmYz1Yn31TMRkwjIJkwnHBEwmPJMvmQhMvGQiMumSicSESyYyky2ZKEy0xHS6ykRLJHzPREsmvtN5aK/D5fDQ9u1lPg8vD6fjvglYePyKmX2MIuhYMN4BPQvmO2BgwXQHjCxY74CJBcsdkNo3yEEp5JpCf2dNlTmRxTWFnjmRZUKdfyePFZwW9AD02hMKgUELopGj9oRCYNKCaOSs7RAEFi2IRq7aDgFg7LUgGDlabSkh0GlBNLJnWk6+ygvalkNriloQHTVq/yMPSe1/ZILa/8iDUPsfkUjU/kcm1P0f5P9PUu9/EKju/wjAoB0Zger+RyMnbbuiFWbtChH4nf9hnNp5vj2JV4XGVHc+AHOvbVQEWu0KwXHL6s5HoNf2KxpZ/Q6AVhiZqswikZiqlInMVKVMFG1vJHBkqrbZAFh6LZgBqH53QKDTNhsaWX11jFaovjpGoPrdAY2svjpGYNa2HAKLFkTHsGpHBmDtma2yWAfVaosSHLXqtCAa0jNbZXnIwGyVZYLq/yoSVP/LBNX/MlGYu9AyUZm70FX+4KhnbkMDwzLxB4Zj0gUMz8QLGIHJl7UyEpmEISQxGUNIZlKGkMLkDCGVCRpALFW0CLFM1BDimKwhxFNh8zISqLABJFJhA0iiwgaQTIUNIIUKG0AqFTYZWfmR59+wAcRSYbshz93XV1Y2/3zDpTO/2nn6fLkrNuTqcinO+pyu1z+aD4aR

Copy blueprint

[TheKillerChicken]

I do understand why this game will not utilise multicore. I have Cities Skylines II, which can use up to 64 cores and with my ryzen 9 7950x (16-cores/32-threads), it is a freaking mess trying to execute all of those threads for so little gain and also causes a lot of thrashing. I do not understand why keeping with the DRAM latencies as it makes no sense. My DRAM has a CAS of, I think 40 (DDR5-5200), not sure, but my windows 98se machine with DDR1-400 has a cas of 3. So CAS latencies are slowing down every generation, unless there is something I do not know about. Perhaps the Compute cores of the graphics cards could be used at some point in time? I don't know. It seems like this game will run the best off of 8-channel DDR4-5 whereas dual-channel simply is not suffice. I am running my server off of 6-channels of DDR3-1866 LRDIMMS and my DDR5 is having a difficult time keeping up FPS-wise after many chunks have been revealed. True SMT is is simply not there yet with games. Factorio even runs faster on my server if I enable NUMA as the server finds the path with the least resistance in that mode, but my client's FPS drops even more due to that. Perhaps abandon the lockstep-sync setup for this game, if possible? Either way, I am looking forward to the future of this game. Keep up the incredible work Wube.

[Koub]

My DRAM has a CAS of, I think 40 (DDR5-5200), not sure, but my windows 98se machine with DDR1-400 has a cas of 3. So CAS latencies are slowing down every generation, unless there is something I do not know about.

CAS is expressed in number of clock cycles, not in time. The highest the RAM frequency for a given CAS, the shortest latency.
DDR1 400 has a 200MHz frequency (200 doubled = 400). A clock cycle take 5 ns, and a CAS of 3 means there is 15 ns latency.
DDR5 5200 has a 2.6 GHz frequency (2600 doubled = 5200). A clock cycle takes ~0.385 ns, and a CAS of 40 means there is ~15.38 ns latency.

The latency in time remains in the same order of magnitude from generation to generation, so every time the RAM frequency doubles, the number of cycles latency takes doubles too, give or take.

[Twisted_Code]

So I'm wondering: did the optimization to robot movement logic also fix the scenario that led to the overcrowding? You never got back to that anecdote, though I'm inclined to assume you meant yes.
Also, to be clear, when you say someone removed a wire, you mean they removed a circuit network connection that would have switched off an inserter?

If a chunk has a counter greater than 1, it is put in an update bucket, and we loop through 1 bucket each tick.

Shouldn’t this be “ If a chunk has a counter greater than 0”?

Yes I think you're right, or else "greater than or equal to 1", but greater than 0 seems like the more likely implementation for them to be using. A fractional value could potentially be used, then, to indicate less frequent updates (and I could see them also going the other way with it)

[SHAYY]

Is it possible to make multi-cores by dividing the factory into trees\how many trees are connected?

Tree = a calculation that starts from one point and splits into several
(Every time the calculation reaches a split, the calculation moves to a different starting point until it reaches a split with all its inputs updated and continues)
which starts with belt or inserter and goes through spletrer and inserter until it reaches the structure and repeats itself


If a strip of splitters, belts, inserter runs on one tick
In every one tick, you can divide the trees each into a different core
(In any case that something goes between trees, it is only between ticks (calculation at the beginning of material that comes through an incoming inserter))

Then run on all the structures that need updating (you can add the structure to the tree on which the inserter moves the material)

Then run on an inserter that transfers between buildings or also add them to the tree

Interstellar it is possible for each planet/space to run on a separate core it is like a factory in itself
And only transfer of materials between star/space to update is needed
(every dimension building passage has a delay until the missile takes off and lands)

Or divide the calculation of cores according to separated factories like outpost

(Or do an automatic calculation of how much goes in and how much comes out, only the player looks, then there are materials in the belt or inserter according to a point in time (instead of calculating the location of each specific material) that changes when you change the electricity, input of materials