[0.18.2] Pump maximum flow is 6000/s (rated 12000/s)

[malventano]

Two notes:

• The statement that a fluidbox can only change by half per tick is incorrect.

The behavior I have observed while investigating fluid flow does not match up with what your fluid sim predicts. If the game was performing as many fractional calculations as your sim suggests, not only would performance be poorer (floating point math), but we would not experience the falloff that occurs after 197 pipe segments. Further, I was under the impression that flow momentum was not implemented, but I could be wrong there. Your fluid transfer rate calculation also does not match what is observed in-game and plotted above.

To provide another example, the same tank-pump-tank transfer to a destination tank that remains empty (contents pumped elsewhere) results in a constant 100/tick transfer, not 'somewhere between 1.2 and 1.318', it is an even 100/tick. It does not matter that the source tank is draining and that there are differences in intake pressure during the transfer, either.

[BlueTemplar]

Ah, you might not be aware of the "previous research on the topic" ?
[0.15.x] Fluid mechanics (Rseding inside)
Factorio and fluid mechanics: science, facts, myths (Math, math, graphs, and more math. But is it *actually* "experimentally backed-up" ?)

Of course the big question - how much has this changed since 0.15/0.16 ?

[TruePikachu]

My code was based 1:1 off a disassembly of one of the later 0.17 versions, but it's a simple enough thing to test results for. And, indeed, the first tick transfers 1.2 internal units of fluid.

Annotation 2020-01-29 132534.png (757.84 KiB) Viewed 6385 times

EDIT: I investigated the 197-tile-falloff problem myself; it is a result of the `min` used in the momentum part of the equation. The Common Lisp version of the flow rate calculation is:

Code: Select all

(defconstant +pressure-to-speed-ratio+ 0.4)
(defconstant +flow-to-energy-ratio+ 0.59)
[...]
    ;; FLOW is the flow amount from SRC to DST last tick
    (let* ((pressure-flow (* +pressure-to-speed-ratio+
                             (- (fluidbox-pressure src)
                                (fluidbox-pressure dst))))
           (momentum-flow (if (plusp flow)
                            (min (* +flow-to-energy-ratio+ flow)
                                 (/ (fluidbox-capacity src) 10))
                            (- (min (* +flow-to-energy-ratio+ (- flow))
                                    (/ (fluidbox-capacity dst) 10)))))
           (total-flow (+ pressure-flow momentum-flow)))

[malventano]

The desired rated 12000/s fluid flow is achieved with the following settings:

fluid_box =
{
base_area = 1,
height = 4,

(height doubled from 2 to 4).

Result:


I have also confirmed that steady-state flow through a single pump (with the above change) can once again produce 1.1GW of power, as expected per its rating.

This simple change corrects for the flow system changes that cut pump flow in half, addressing the issue that has been reported by folks in various different forms over the past year (and repeatedly shot down over what appears to be constant misunderstandings). If this change is implemented, those types of random reports should stop coming in, reducing workload on the devs .

To recap, from the code, desired pumping speed is 200/tick (12,000):

pumping_speed = 200,

...and achieving that with the current fluid system is only possible with fluid_box height = 4.

height = 1 => 3000/s maximum
height = 2 => 6000/s maximum
height = 4 => 12000/s maximum

[TruePikachu]

(All fluid counts are internal) That's a pressure differential of 4, attempting to move 1.6 units of fluid per tick. Then, the momentum factor kicks in, which will (in the second tick) attempt to add 0.944 fluid to the 1.6 differential flow, resulting in a movement of 2.544 fluid. This rate decays over time since the pump itself can only move 2 fluid/tick (and the fluidbox that is the pump output will no longer remain full since more fluid is leaving than entering), until the actual flow becomes stable at that flow rate. The stable point is 2.0 fluid units in the pump because:

• At the beginning of the tick, the pump moves 2.0 fluid from the full tank into its own fluidbox, making it contain 4.0 fluid
• The fluid logic then runs, which:
  • Computes a pressure difference of 4 (contribution of 1.6 units of fluid to move)
  • Computes a potential momentum factor of 1.18 (0.59 * 2.0 fluid/second final rate), but uses 1/10 of the source fluidbox capacity instead (0.4) since it is smaller
  • Adds these together, resulting in 2.0 units of fluid moving in the tick
• Script logic then runs, which results in the creative mod fluid destroyers emptying the output tank

---

All the internal logic is working as it is designed to, and as such, this is Not a Bug. The pump intentionally has such a high flow rate configured, even if it's unachievable in practice in the vanilla game, so that it doesn't become the bottleneck in this iteration of the fluid system.

[malventano]

...

I see, your model was based on per-tick changes, accounting for momentum, etc, while my practical experiments were using continuous flow (or at least flow over the course of a few ticks, once the momentum was established). If I'm reading that correctly, it looks like you have provided additional evidence that the current height=2 can not support the desired 200/tick / 12,000/s flow rate.

All the internal logic is working as it is designed to, and as such, this is Not a Bug. The pump intentionally has such a high flow rate configured, even if it's unachievable in practice in the vanilla game, so that it doesn't become the bottleneck in this iteration of the fluid system.

The configured flow rate is 12,000/s, and this value is presented to the player as the max flow rate. You can't base the presented maximum rating on only internal logic that can never be attained. Technically an assembler produces an item at its output on a tick, but we don't go displaying that the assembler can produce a maximum of 60 items/s. Also, this was a thing that functioned as advertised before the fluid system change, and that change caused a regression specific to the pump. Changing height from 2 to 4 fixes this and restores the behavior seen prior to the regression, so why on earth are we having an argument over if it is a bug or not when clearly it is. Devs in this thread have stated their understanding:

If you put a pump next to a storage tank it will pull 12,000 in the first second

It's not doing what the devs believe it should do. That's a bug.

[TruePikachu]

...

I see, your model was based on per-tick changes, accounting for momentum, etc, while my practical experiments were using continuous flow (or at least flow over the course of a few ticks, once the momentum was established). If I'm reading that correctly, it looks like you have provided additional evidence that the current height=2 can not support the desired 200/tick / 12,000/s flow rate.

With vanilla entities, the only way that flow can be supported is with a pump feeding directly into another pump (because of a technicality with how pumps operate; the fluid flow logic techinically would be irrelevant in this case, as it would all be entity control logic). With modded entities, of course, the vanilla pump can still achieve full throughput even if the pump itself is unmodded; the required pressure of the destination is -2.5 or less. However, since the entire fluid system is due to be replaced anyway, it doesn't make much sense to contribute time to making the current system friendler to use. It has also been acknowledged by the devs that the current pumps are overpowered, and when the new fluid system is released, their flow rate will be turned down significantly (it is only as high as it is right now because of limitations imposed by how the fluid system currently operates).

I believe the highest sustainable flow achievable in vanilla circumstances right now is 1.4/t (8400 in-game/s), with an empty destination that has base_height=-1. This is a pressure difference of 3 (contribution 1.2/t), and a momentum factor limited by the source fluidbox size (contribution 0.2/t). One way to achieve this rate is using miners configured for uranium mining, which act as a fluidbox similar to boilers except with multiple points where fluid can be extracted. (Note: this test is 100% vanilla, using map editor entities added in 0.17)

Annotation 2020-01-29 144351.png (3.13 MiB) Viewed 6356 times

(I had at one point run experiments with fluid wagon output, which bypasses the fluid flow logic as with the "line of pumps" example above, but there's an oddity in the pump entity operation there which I can't understand enough to explain; this has been considered Won't Fix pending the fluid rework).

[malventano]

Your example is going out of its way to show >6,000/s in a configuration that the player will never employ, while the real example (even provided by the devs when the pump was changed to a 1x2), was tank>pump>wagon and wagon>pump>tank. Those configurations only see 6,000/s, where they used to see 12,000/s.

However, since the entire fluid system is due to be replaced anyway, it doesn't make much sense to contribute time to making the current system friendler to use.

So instead of changing a value from 2 to 4, addressing the issues brought up repeatedly across multiple threads/months, you claim that it is more time efficient to work on the new fluid system. I direct you to this:

What about the improved fluid algorithm? The one with perfect splitting. I know you ran into some issues with waves building up, but is that still on the table?

It is no longer worked on and there are no plans to finish it afaik.

...so change one value and fix the issue, or wait for the system to be (not) replaced.

[TruePikachu]

Really, this is the most important point that I can make:

It has also been acknowledged by the devs that the current pumps are overpowered, and when the new fluid system is released, their flow rate will be turned down significantly (it is only as high as it is right now because of limitations imposed by how the fluid system currently operates).

If the fluidbox were enlarged, that would be making the pumps even more overpowered. The only viable "fix" would be to partially reduce their listed flow rate now, but I'm not sure how badly that would interefere with the fluid wagon loading logic.

Also, a rework isn't completely off the table:

I see no mention of fluid dynamics. So has that redesign been axed officially?

Right now it isn't included in our plans for 1.0.

Things in this regard could change, especially if things go smoothly and one of the team has some spare time.

[BlueTemplar]

Devs in this thread have stated their understanding:

If you put a pump next to a storage tank it will pull 12,000 in the first second

It's not doing what the devs believe it should do. That's a bug.

A digression, but AFAIK, the 12kL/s limit used to come simply from the pump ignoring fluid mechanics on input, and filling its entire fluidbox at every tick : 200 L x 60 ticks/s => 12 000 L/s ?
Now, it sounds like TruePickachu might be able to explain what exactly has changed going from 0.16 to 0.17 so that this limit got cut in ~half in practice ?

[malventano]

If the fluidbox were enlarged, that would be making the pumps even more overpowered. The only viable "fix" would be to partially reduce their listed flow rate now, but I'm not sure how badly that would interefere with the fluid wagon loading logic.

If the desire was to reduce the power of the pumps, the answer is to limit the rate to 100/tick, not to rely on an unintentional and misunderstood flow system regression to do it for you. Tank to tank transfer time is within a few ticks (out of >3 seconds) with the rate set at 100/tick.

[TruePikachu]

@BlueTemplar:
I'd need to download 0.16 again and invest the time into dissection, but my naive guess would be moving fluid simulation out of entity logic into its own system so it can run on a parallel thread. As it stands right now, all entities that have fluidboxes run their logic before fluid simulation logic runs, while it might have been the case in older versions that fluid simulation runs intertwined with entity logic (possibly as `Entity::update()` on e.g. a pipe). The actual flow math is almost certainly unchanged, since the constants 0.4 and 0.59 used to be specified on the fluid prototypes for ages until they were somewhat recently hardcoded instead to improve performance.

Pumps do indeed ignore all fluid mechanics on their input side; the `Entity::update()` for pumps not connected to rails can basically be summed up as "move an amount of fluid not exceeding our pumping speed, if we have enough energy to do so, from each of the fluidboxes all our inputs are connected to into our own internal fluidbox".

@malventano: Reducing the rate like that can severely impede the fluid wagon loading time, since pump-to-wagon transfer is not based on fluid mechanics, but still uses the pumping rate number.

[malventano]

@malventano: Reducing the rate like that can severely impede the fluid wagon loading time, since pump-to-wagon transfer is not based on fluid mechanics, but still uses the pumping rate number.

I've already tested it. Empty rate is 100/tick / 6000/s with pumping_speed = 100. Wagon fill rate is slightly lower due to an odd toggle behavior that the devs have already stated they won't fix elsewhere, but that same toggling behavior exists with the 200 setting (wagons empty slightly faster than they fill with both 100 and 200).
Setting pumping_speed = 100 is also the alternative suggestion that I've been promoting since I started this report (matching rating to reality / 6,000/s).

[malventano]

@BlueTemplar:
I'd need to download 0.16 again and invest the time into dissection, but my naive guess would be moving fluid simulation out of entity logic into its own system so it can run on a parallel thread. As it stands right now, all entities that have fluidboxes run their logic before fluid simulation logic runs, while it might have been the case in older versions that fluid simulation runs intertwined with entity logic (possibly as `Entity::update()` on e.g. a pipe). The actual flow math is almost certainly unchanged, since the constants 0.4 and 0.59 used to be specified on the fluid prototypes for ages until they were somewhat recently hardcoded instead to improve performance.

I can't find the thread, but it had to do with the older code calling the update multiple times within a tick. This was specific to the pump code.

[Bilka]

The desired rated 12000/s fluid flow is achieved with the following settings:

fluid_box =
{
base_area = 1,
height = 4,

(height doubled from 2 to 4).

Okay, done for the next version.

[malventano]

Okay, done for the next version.

Thank you! While you were testing that, did you notice the odd ‘toggle’ behavior while loading a wagon?