Friday Facts #260 - New fluid system

[McDuff]

What behaviour would you expect on a system like this?

Code: Select all

IN =======J=====J========J=======J=====OUT
          ¦¦    ¦¦       ¦¦      ¦¦
          OUT   OUT      OUT     OUT

Would you expect the first output to get fully satisfied before letting fluid pass, or for it to evenly split 50% of whatever was passing through it?

One like that should be 50%, 25%, 12.5%, 6.25%, 6.25% for each output going from left to right, not taking into account max throughput for the outputs. (Ie: If the pipe's throughput is much greater than what each output uses then the values would be much different.) If the algorithm goes on a per-junction basis then it'd just split the output evenly at each junction. So the first junction sends 50% to the output and 50% on to the next junction. The second junction then sends half of what remains to the next output (25%) and the other half to the next junction and so on.

This seems to be what I'd intuitively expect too, i.e. for each output to get *something* but for the first object on the chain to get satisfied first.

Now, with the theories about changing the pipes to be pressurised, to me, if that were the case, all outputs would get an even proportion of the input, so if demand >> supply every output gets 20%.

Would that make more or less intuitive sense from a gameplay perspective? Would treating a pipe network after a pump as a single "filled" section make more intuitive sense to the player or less?

[algogenetienne]

That's the graph theory approach. Based on full physics, I'd expect resistance and inertia to at least put them closer together, though I'm being too lazy to do the math right now.

I think the full physics model would be a pressure/flow one, but it is very far from what we have in Factorio.

If we assume the resistance of pipes to be small enough (resistance*flow²<<pressure), then you would have the same pressure at all the outputs no matter the network junction configuration. Then we should have a law machine_consumption = f(pressure) and if the machines at each output are the same, they should have the same flow going into them.

Then, one can take into consideration pressure loss along the pipes (which depend on how the graph is shaped) and it becomes very difficult to solve by head (but quite easy with a descent algorithm)

If the flow going into a network is too low, the pressure in the network drops and the machines consume less, which results in the network stabilization

[meganothing]

Assume here that all outputs are unsatisfied and that any "satisfied" output backs up and becomes "closed" according to the system. I'm talking about a situation in which all outputs/sinks are demanding fluid - say 3 boilers arranged like that.

I might assume that , but it isn't that easy for the program to find out. You are talking about the easy cases and I'm talking about what happens when it gets complicated. For example on the first junction the north OUT gets 33%. What if it only needs 15% (which you find out after following that OUT into a complicated network of pipes). Now you need to backtrack and distribute the rest, 18%, to the south and straight exits of the junction. And if you have to do this backtracking redistribution for every exit, well, you are in trouble.

If we are only talking about whether that is intuitive, I think I can easily construct a case where your model fails to be intuitive, especially if you don't redistribute the 18%

[McDuff]

I'm not sure I fully understand your objection? I'm asking what people would anticipate as expected behaviour rather than suggesting a model. Under the current system each pipe section asks the one next to it what it needs, so what happens upstream doesn't matter anyway - each pipe only cares about what the one adjacent to it wants. In the case of a junction if one output is full it "asks" for nothing, so the remaining two outputs get what would have gone to it.

What model do you suggest is more intuitive than "outputs get an evenly distributed split of inputs"?

[McDuff]

Double post: the reason I'm asking this is that, as an end user, you probably *don't* know the fundamentals of the pipe system. Indeed, the current implementation seems to be so complicated that even people who do know how it works don't really know how it works. Also, the current system works in such a way that, as has been pointed out before, it's the program, the way chunks and tiles update, that can make the difference. Not only does a theoretically balanced 1->16 branching split not deliver an equal amount to all outputs, but it does so in a way that's inconsistent and, sometimes, unreproducible.

While people here have been approaching it from the theory point of view, I thought it would be useful to consider what actual end result people would expect to see. If you don't know anything about Factorio, and you start putting down a bunch of pipes, what behaviour *intuitively makes sense*?

[meganothing]

Double post: the reason I'm asking this is that, as an end user, you probably *don't* know the fundamentals of the pipe system. Indeed, the current implementation seems to be so complicated that even people who do know how it works don't really know how it works. Also, the current system works in such a way that, as has been pointed out before, it's the program, the way chunks and tiles update, that can make the difference. Not only does a theoretically balanced 1->16 branching split not deliver an equal amount to all outputs, but it does so in a way that's inconsistent and, sometimes, unreproducible.

While people here have been approaching it from the theory point of view, I thought it would be useful to consider what actual end result people would expect to see. If you don't know anything about Factorio, and you start putting down a bunch of pipes, what behaviour *intuitively makes sense*?

Sorry, I was partly misunderstanding you and not making myself very clear. But my point was that although your simple example may bring people to say it is intuitive, but that answer doesn't help because different examples would probably make them change their mind again. In other words you posted a leading question.

The worst thing IMHO about the current implementation is the "it's the program, the way chunks and tiles update, that can make the difference". Without that an end user might still get some surprises, but that is because pipe systems can be unintuitive in real live too and (if you look closely) it simulates an open pipe/viaduct system, not a closed pipe.

As you note in a previous post and what I have been saying, the example with 50%, 25%, 12.5% for all the following OUTs may be seen as intuitive on first look, but you can easily construct examples where it gets unintuitive even for the "naive" observer. And I have listed just such examples.

[mrvn]

I have a question: what behaviour do you think users expect, and what makes sense to simulate?

For eg, I would say that it makes sense that in a 2 or 3 way junction that the liquid gets split 2 or 3 ways evenly

Code: Select all

          OUT
           ¦¦
i.e. IN === J  ===OUT
           ¦¦
           OUT

You'd expect all the outs to get 33%.

That is probably too simple. Assume the north and south "OUT"s to be dead ends (or outputs that need only minimal water, or full tanks). You would get only 33% from the right "OUT" even though nobody else needs that other water. Add another pair of north/south dead ends and you would only get 11%. Wouldn't really be intuitive

Obviously if one end doesn't consume enough it backs up and gets split to the remaining ends.


Note though that fluids have inertia. In an open pipe system at speed the east side would get more output because some fluid flows right past the north/south openings. I know that some models above include flow speed but I'm not sure if any included a direction vector.

[mrvn]

What behaviour would you expect on a system like this?

Code: Select all

IN =======J=====J========J=======J=====OUT
          ¦¦    ¦¦       ¦¦      ¦¦
          OUT   OUT      OUT     OUT

Would you expect the first output to get fully satisfied before letting fluid pass, or for it to evenly split 50% of whatever was passing through it?

One like that should be 50%, 25%, 12.5%, 6.25%, 6.25% for each output going from left to right, not taking into account max throughput for the outputs. (Ie: If the pipe's throughput is much greater than what each output uses then the values would be much different.) If the algorithm goes on a per-junction basis then it'd just split the output evenly at each junction. So the first junction sends 50% to the output and 50% on to the next junction. The second junction then sends half of what remains to the next output (25%) and the other half to the next junction and so on.

This seems to be what I'd intuitively expect too, i.e. for each output to get *something* but for the first object on the chain to get satisfied first.

Now, with the theories about changing the pipes to be pressurised, to me, if that were the case, all outputs would get an even proportion of the input, so if demand >> supply every output gets 20%.

Would that make more or less intuitive sense from a gameplay perspective? Would treating a pipe network after a pump as a single "filled" section make more intuitive sense to the player or less?

Even with pressurised pipes the first out gets more output. You might be thinking of pipes in your home where you can turn on all the sinks and they all get water. That is because each sink only allows a fraction of the water out. Even then when you turn on the cold water in the basement the shower upstairs gets hotter because the cold waters pressure drops a bit.

At each junction the pressure drops, the pressure differential to the output decreases and so the flow rate decreases. If you don't include inertia and all outputs can consume a full pipe then I would still expect that 50%, 25%, 12.5%, 6.25%, 6.25% split.

What I expect is that a pipe system like that behaves the same as a belt with splitters.

[McDuff]

(Edit: replying to megannothing)

Right but that's an analysable and solvable problem, isn't it? It's a complex result but derivable from simple rules.

Like, for any given junction, if Flow(In)<Flow(Out), for N outputs Flow(Out)=Flow(In)/N

The problems you're probably looking to solve as a game player are "why is this input not getting fluid when there's a pipe connected to it?" The problem of it getting *too much* fluid isn't really an issue, so the early player isn't actually going to care if a given 3 way junction actually delivers a 40/25/35 split because Flow(Out)<Flow(In) and the pipe just delivers what's demanded, and the advanced player could easily work it out if they wanted to.

The question for gameplay is: if you have a 3 way junction and various output complexes on either branch, should it be possible for one of them to starve the others, or should all of them be guaranteed a minimum of 33% of what comes down the pipe?

The situation you're talking about is: if one of them doesn't need the whole 33%, should the others get the excess distributed evenly between them, and that seems trivially true to me.

[meganothing]

The problems you're probably looking to solve as a game player are "why is this input not getting fluid when there's a pipe connected to it?" The problem of it getting *too much* fluid isn't really an issue, so the early player isn't actually going to care if a given 3 way junction actually delivers a 40/25/35 split because Flow(Out)<Flow(In) and the pipe just delivers what's demanded, and the advanced player could easily work it out if they wanted to.

But some other pipe getting "too much" fluid IS (usually, always, depending on the model) the reason why a pipe much later in the sequence is not getting fluid. I don't see why this should be a separate and "unimportant" issue

The question for gameplay is: if you have a 3 way junction and various output complexes on either branch, should it be possible for one of them to starve the others, or should all of them be guaranteed a minimum of 33% of what comes down the pipe?

The situation you're talking about is: if one of them doesn't need the whole 33%, should the others get the excess distributed evenly between them, and that seems trivially true to me.

Ok, thanks for the clarification. Since we are working with abstractions I did not see it as trivial

[Alice3173]

Even with pressurised pipes the first out gets more output. You might be thinking of pipes in your home where you can turn on all the sinks and they all get water. That is because each sink only allows a fraction of the water out. Even then when you turn on the cold water in the basement the shower upstairs gets hotter because the cold waters pressure drops a bit.

At each junction the pressure drops, the pressure differential to the output decreases and so the flow rate decreases. If you don't include inertia and all outputs can consume a full pipe then I would still expect that 50%, 25%, 12.5%, 6.25%, 6.25% split.

What I expect is that a pipe system like that behaves the same as a belt with splitters.

I'm fairly certain that taking into account the consumption rates of the outputs you'd probably get something close to realistic rates anyways. Ie: If each output has a consumption rate of 20% then that means for the first split it's shunting an additional 30% to the next junction and so on. So first junction would end up being 20% output/80% to next junction which would then be 20% output/60% next junction, and so on up to the fourth (or fifth, I think fourth though) junction where it'd split 20% into 10% to the output and 10% to the next junction.

[McDuff]

The problems you're probably looking to solve as a game player are "why is this input not getting fluid when there's a pipe connected to it?" The problem of it getting *too much* fluid isn't really an issue, so the early player isn't actually going to care if a given 3 way junction actually delivers a 40/25/35 split because Flow(Out)<Flow(In) and the pipe just delivers what's demanded, and the advanced player could easily work it out if they wanted to.

But some other pipe getting "too much" fluid IS (usually, always, depending on the model) the reason why a pipe much later in the sequence is not getting fluid. I don't see why this should be a separate and "unimportant" issue

What do you mean by "too much" fluid? A sink can't take more than its demand. If there is, say, a mining drill that's being starved of sulphuric acid because the ones before it are taking it all, that doesn't mean they're taking "too much," it means the system is running out before all the outputs are satisfied.

If Flow(out) >> Flow(In), then some outputs go unsatisfied. That's a given of any model. The question is "can you work that out by analysing your pipe system according to reasonably intuitive rules?" The rule "a 3 way junction delivers 33% to each output unless one output does not need the whole 33%" seems intuitive. You seem to be saying "if one of the outputs does not need the full 33% that makes the system unintuitive, but I am struggling to follow the reason. As a system designer I feel as if I would be perfectly aware of the fact that a pipe which under usual circumstances split things evenly would not do so if backed up on one output. That fluid also has to actually go somewhere, so the logical place for it to go is down the other outputs.

Your point "if output one only needs 18% then it won't get 33%" does not seem to be the kind of thing that would leave any new player scratching their head, trying to get to the bottom of the mystery, to me. Can you perhaps explain more why you think it would be confusing.

The question for gameplay is: if you have a 3 way junction and various output complexes on either branch, should it be possible for one of them to starve the others, or should all of them be guaranteed a minimum of 33% of what comes down the pipe?

The situation you're talking about is: if one of them doesn't need the whole 33%, should the others get the excess distributed evenly between them, and that seems trivially true to me.

Ok, thanks for the clarification. Since we are working with abstractions I did not see it as trivial

What rule would you apply instead that would make more sense to you?

[McDuff]

I think "behaves like a belt with splitters" is the intuitive result people think they're going to get without knowing anything about it. I guess the question is, is that the right thing they should be expecting?

[dee-]

I think "behaves like a belt with splitters" is the intuitive result people think they're going to get without knowing anything about it. I guess the question is, is that the right thing they should be expecting?

I think this is the catch... how correct is the intuitive opinion?

E.g. I would consider this

Code: Select all

          OUT
           ¦¦
i.e. IN === J  ===OUT
           ¦¦
           OUT

to split 33% / 33% / 33%

But this

Code: Select all

IN =======J=====J========J=======J=====OUT
          ¦¦    ¦¦       ¦¦      ¦¦
          OUT   OUT      OUT     OUT

to split 20% / 20% / 20% / 20% / 20% considering every output is taking some of the fluid without blocking.

If one output closes I'd expect 25 % / 25 % / 0 % / 25 % / 25 %.

In case In >> Out then the fluid level should rise over time, with all outputs getting their fair, full share.
In case In << Out then the fluid level should fall over time, with all outputs getting their fair, full share, until the pipes empty, then every Out should get their fair but reduced share, depending of (Out-In).

[meganothing]

What do you mean by "too much" fluid? A sink can't take more than its demand. If there is, say, a mining drill that's being starved of sulphuric acid because the ones before it are taking it all, that doesn't mean they're taking "too much," it means the system is running out before all the outputs are satisfied.

What do YOU mean by "too much" fluid then? I was directly quoting you with this, you said originally "The problem of it getting *too much* fluid isn't really an issue, so the early player isn't actually going to care if a given 3 way junction actually delivers ..." (with "too much fluid" a paraphrasing of what you thought I'm talking about) and I was refering to it.

I understood you saying that my question what happens with the rest of the fluid into a pipe that only takes 1% doesn't matter because a user is not interested in the full pipes but the pipes that don't get enough.

Should we stop here? Because we are obviously not getting anything resolved if we tangle ourselves in semantic constructs.

If Flow(out) >> Flow(In), then some outputs go unsatisfied. That's a given of any model. The question is "can you work that out by analysing your pipe system according to reasonably intuitive rules?" The rule "a 3 way junction delivers 33% to each output unless one output does not need the whole 33%" seems intuitive. You seem to be saying "if one of the outputs does not need the full 33% that makes the system unintuitive, but I am struggling to follow the reason. As a system designer I feel as if I would be perfectly aware of the fact that a pipe which under usual circumstances split things evenly would not do so if backed up on one output. That fluid also has to actually go somewhere, so the logical place for it to go is down the other outputs.

Your point "if output one only needs 18% then it won't get 33%" does not seem to be the kind of thing that would leave any new player scratching their head, trying to get to the bottom of the mystery, to me. Can you perhaps explain more why you think it would be confusing.

Lets just say that you left something implicit in your initial question which I thought needed to be defined explicitly. Yes, backtracking the fluid is quite more natural than just ignoring it but again, we are already talking about abstractions.

[TheYeast]

That's it for me, as I'm on vacation the coming week. I will check the thread when I return.

I improved my previous simulation to support junctions and multiple physics models.

https://jsfiddle.net/TheYeast/c9d2azjo/show/

Feel free to fiddle the source at https://jsfiddle.net/TheYeast/c9d2azjo

The simulation supports pipes, sources, sinks, and tanks.

• Blue bar = fluid content
• Orange bar = production cycle progress
• Pink bar = internal buffer
• Green bar = flow direction

Sources and sinks can be configured: Cycle duration is the number of frames (60 in a second) a source/sink needs to complete a batch.
Production and Consumption per batch can also be set.

(Not all systems/browsers can manage 60 UPS)

As Bauer correctly noted:

I'm only a little bit worried though that the "unphysical" behaviour of liquid production and consumption (delta-functions) will lead to artifacts in complex networks that will be difficult to understand by the player. Assume a ring-tube with non-symetrical producers and consumers. The flow direction will be quite random and confusing because the speed waves following the delta-peak will travel in both directions of the ring and will not cancel out due to the missing symmetry. Or am I wrong?

This is completely true. Whether the waves caused by this are aesthetically pleasing or confusing depends on the player. To compensate for this, there is a global option Smooth source output/sink input over time to gradually release/consume fluids. The logic simply tries to empty/fill the source/sink buffer in the time remaining to complete a production cycle. As a middle ground, the logic could try to do this in half the remaining time, so there would be some waves, but not as much as with the delta-peaks.

Supported pressure models:

Wave Equation: the model as I proposed in earlier posts. C-Squared and Friction can be configured.
Wave Equation, Non-Linear Pressure: same, but after filling a pipe over 60%, pressure increases more sharply.
Fixed Acceleration: a proposal by quinor to use a fixed acceleration in the direction of lower fluid level. Acceleration and Inertia can be configured. (@quinor please correct me if I made a mistake )

Non of these models suffer from update order artefacts because the flow calculation and actual fluid transfer are performed in separate passes.

Supported clamp models:

Quarter Limit: as described in previous posts, limit the in/out flow to 1/4 of the remaining space/content so even with 4 connections, the pipe cannot over-/underflow.
Connection Count Limit: as above, but instead of always using 1/4, count the actual connections. For a non-junction pipe the limit will thus be 1/2.

Both min and max clamping can be disabled (though disabling min clamping will usually cause negative fluid content).

I did not (yet) add the very clever clamp model proposed by quinor (which as I understand distributes contested available fluid/space fairly over all connections). I think this method is the best I've seen, but it is also a bit more computationally expensive. If there would be a need to maximise throughput without increasing pipe volume, this seems to be the way to do it. The reason for not adding it yet is mostly that it is less trivial to implement


Tank behaviour is different than currently in Factorio: when the fluid level in a pipe connected to a tank is above 50, the tank will take in fluid, otherwise it will output fluid as if the level in the tank were 50. This allows a tank to maintain some pressure when releasing fluid even if it is only partly filled up.

The simulation does not use the artificially lower pressure in sinks or higher pressure in sources, for the reason I explained in an earlier post.

I tried to write the simulator in such a way that new models can easily be added. Feel free to fork and add your own.

The Yeast

[Oktokolo]

I improved my previous simulation to support junctions and multiple physics models.
https://jsfiddle.net/TheYeast/c9d2azjo/show/

I fiddled around and made some observations:
- As expected, tanks need pumps. They seem to sometimes act way too greedy - blocking any fluid from reaching the consumers.
- I got a quick i/o change response with default settings + "Wave equation, nonlinear pressure" and no min/max content clamping. The resulting heavy waves of negative and positive pressure look like expected but are probably not the nicest to visualize to noob players. I like the hammer efffects, when adding/removing connections.
- The simulation works fine with multiple loops and changing producer/consumer outputs/inputs when i/o smoothing is enabled.
- If i enable min/max content clamping, the pipe system responses become too slow.

Testsetup was a pipe circuit with 11 producers connected by a single pipe to 30 consumers wich where arranged in two loops (i also had two dead end pipes attached).
I would advise against using the content (pressure) as visual indicator for the pipe windows. Flow direction also would just irritate noobs who don't expect waves in their pipe system. But absolute flow speed might be a good data source to signal whether fluid is moving in the pipes.

Would like to see this model implemented in the game.

[McDuff]

Had a quick play and it looks as if the model works how I'd expect.

One nice "intuitive" thing is that it appears from my very quick testing that a double line of adjacent pipes works to improve throughput, without any strange artifacts.

[HurkWurk]

for tanks, there are two solutions i see, for both, being able to set inputs and outputs is probably the best solution, since that way we can define logic for them.

1. if we switch to a pressure system, the tank is a large pressure vessel. once its down to the same level as a pipe, its at the pips pressure. when its above a pipe's pressure level, its pressurizing the pipe.

2. if we stay with gravity feed, then we need to cheat a little:
*tanks will be the highest entity. fluid wants to flow out of a tank all the time until its empty. its output pressure is always the full volume of a pipe until it has less than a pipe's storage remaining.
*tanks input side will be the second lowest possible entity. fluids want to go into a tank (yes you can create loops with this) in this way tanks will always want input. however, it would still be above consumers
*pumps will work the same as tanks. except they also increase the flow rates. pumps "pull" fluid, while tanks would just use the default flow rate. pumps input would still be "above" factories as well, so that they do not starve them by pumping fluids away.
*pipes would be below tanks and pumps. all at the same level. (lake level calculations are the easiest here so that junctions do not matter, the entire fluid network of pipes is simply a list of inputs and outputs and all outputs get equal shares, not realistic, but doable for cheap resources)
*factories and consumers would be below pipes IE the lowest. that way everything always drains into factories.

also multiple settings for input/output. for instance we could intentionally allow one connection more "shares" than another (IE this valve is at 1.0 0.5 0.25 flow rate) so that a tank with 1 input and 3 outputs defined could have a 1 and two 0.5 outputs so that one lane gets double share to prevent the need for pipe splitter configurations. alternatively, a pipe flow rate segment that is a separate pipe object could do this. (and might be better overall)

[andybe]

why dont you try a suction / propulsion system and a size 4 vector in a network of nodes with neighbouring relationship?

end points produce a fixed amount of s/p (end points are factories and pumps whereas a pump acts as 2 end points), pipe nodes will inherit the values of their neighbours.
the resulting underlying suction and propulsion in each node will then be calculated into a net throughput(that is not passed on to their neighbour! if we do that we get markov chains which we dont want to solve)

this has two clear advantages: when a pipe node is added to the network only that one node needs to be updated. if an end point is added only parts of the network will need to be updated and only once. We avoid complex calculations all together through a simplified model

optimisations can be done by using only one directional edges. propulsion would be updated only in the direction of edges, where as suction would only need to be updated in opposite direction of flow and only for the new nodes.

nodes with max 2 edges can still easily be found and clustered together (you called it a section) and treated as one node using the throughput of the bottleneck (lowest value)
clustering can be turned off in smaller bases in favor of realism but turned on if ups drops.

TLDR:
2 node types: pipes, end points
end points have fixed directional value
pipes inherit neighbouring values (friction values my be added during this step)
only directional values are exchanged between nodes and then translated into individual throughput.
now fluids will be passed on according to throughput


heres an example: the value in the nodes represents the total throughput. I have added a fixed friction value of 0.1
Edit: in this example at step 3 adding a new joint I made the mistake and manipulated the directional values on a junction. it would be better to do that at the throughput level. that way we wouldnt need to update the directional values of the already existing joint at all after attaching the new one