[0.15.x] Fluid mechanics

[XKnight]

Recently, I was looking for deep explanation of fluid simulation, and I found only a few posts where other people asking the same questions but don't receive answer. So I performed my little research in this area.

I hope after reading this post you will be able to predict everything with fluids.

Every fluid-containing object can be described with the following values: max capacity, max pressure and zero-pressure.
Max capacity is maximum amount of liquid which can fit into container.
Max pressure and zero pressure is a pressure of a full and empty container respectively.

Using its water level we can calculate its current pressure = zero-pressure + (max pressure - zero-pressure) * water level / max capacity.

Entities

Atomic object of the entire fluid system is a pipe. Simple pipe with 100 capacity and 0..100 pressure (first number is a zero pressure, second number is a max pressure). Later you will understand the purpose of this small picture.


Next object is storage tank. Basically storage tank can be considered as a very long pipe because it has the same pressure 0..100 but bigger capacity 25000.


Everyone knows that pipe connected with storage tank balances its pressure level, this is clearly demonstrated on this picture:


Next entities are "heart" of energy production system: boiler, heat exchanger, steam engine and steam turbine - all these entities share the same values (200 capacity and -100..100 pressure). Wait, what? negative pressure? Yep, this value is negative because it is located below the ocean level (ocean = default pipe). And here comes first interesting picture:


Last object is pump, it is very particular entity because it accepts liquid using electrical "force", and output flow obeys regular rules. Pump has 200 capacity and 0..200 pressure. Picture with disabled pump:

Powered pump increases its input flow up to pumping speed.

Also, there are several other entities that can interact with liquids:
- mining drills with sulfuric acid (200 capacity, -100..100 pressure)
- assemblies, chemical plants, oil refineries (input/output - same mechanic as with pump (speed? - untested yet))
- underground pipes have the same parameters as a regular pipe, but they "teleports" liquid, so using underground pipes allows to reduce pressure recession.
- did I forget something?

Flow

Okay, now we know and understand basic concepts of every entity and it is time to figure out how they interacts between themselves.
Each pair of adjacent entities balances their pressure every tick, this happens in the following way: entity with bigger pressure (here I means current pressure, there was a formula in the beginning) transfer its liquid to the entity with smaller pressure. This direct transfer is equal to (Pressure_A - Pressure_B)*0.4 (this coefficient is unique for every type of liquid, but as far as I know it is always 0.4). Also there are exist an inertial transfer, it can be calculated as part of previous tick flow: Previous_Flow * 0.59 (also predefined coefficient). Inertial component is limited with 10% of the max capacity of the destination entity.
So, total formula is (Pressure_A - Pressure_B) * 0.4 + Limited[Previous_Flow * 0.59, Target_Capacity * 0.1].

Manual simulation

Let's manually simulate (boiler with 200 water) connected with (pipe with 0 water)

Tick1: Boiler pressure = 100. Pipe = 0.
(100 - 0)*0.4 + 0 = 40.
Transferred 40 water units, new values 160 and 40.

Tick2: Boiler pressure = 60. Pipe = 40.
(60 - 40)*0.4 + Limited[40 * 0.59, 100 * 0.1] = 8 + 10 = 18.
Transferred 18 water units, new values 142 and 58.

Tick3: Boiler pressure = 42. Pipe = 58.
(42 - 58)*0.4 + Limited[18 * 0.59, 100 * 0.1] = -6.4 + 10 = 3.6.
Transferred 3.6 water units, new values 138.4 and 61.6.

Tick4: Boiler pressure = 38.4. Pipe = 61.6.
Back flow!!!
(61.6 - 38.4)*0.4 + Limited[-3.6 * 0.59, 200 * 0.1] = 9.28 - 2.124 = 7.156.
Transferred 7.156 water units, new values 145.556 and 54.4.

...

After few thousands ticks there will be 150 and 50 water inside.

Common mistakes

- using entities with different zero-pressure level simultaneously


- pumping from entity with low capacity
Obviously, it is not possible to pump more than we have in adjacent entity.


- pumping into entity with high zero-pressure:
using natural pressure difference you can increase flow speed:


- pressure-ignoring pumping into low zero-pressure entity
Pressure-ignoring pumping should be targeted on higher zero-pressure entity, because later natural pressure difference may be used for free.

- filling tank from low zero-pressure entity


Application

Using previous information we can calculate some interesting formulas:

Flow-to-Distance

Here we will find max transport distance depending on the desired flow value.
Because our system is stable its flow is the same for each entity in string.
"diff" = is a water level difference between two adjacent entities.
"pressure" = max pressure - zero-pressure.

Decreasing-Flow-to-Distance

This formula is applied to each entity that can consume liquid and transfer remaining to the next entity. For example heat exchanger (heating 15 C water to 500 C):


According to this calculation, 33-rd heat exchanger won't be able to keep desired cold water flow rate even with full decay.

The same calculations but for steam turbine (decay = 1), as you can see it is possible to supply 49 turbines sequentially.

Increasing-Flow-to-Distance

This is similar to decreasing flow because we can iterate not from the beginning but from the end of the stripe.


Here I performed calculation for pipe (100, 0..100) and each pipe is counted thrice. This is done to simulate standard heat exchanger build, where distance between exchanger's outputs is 3 pipe segments.

According to this calculation, 16-th exchanger won't be able to keep desired hot water flow rate.

Flow-to-Distance main chart:

X axis - flow speed in units/tick (Steam engine = 0.5, Steam turbine = 1, Offshore pump = 20, Pump = 200)
Y axis (logarithmic scale) - max pipe distance which can maintain specified flow speed.


and table


EDIT 2023-06-19: Restored images from Internet Archive. --Sanqui

[AmbulatoryCortex]

How does this compare with underground pipes?

[XKnight]

How does this compare with underground pipes?

Underground pipes have the same math as for regular pipes, the only difference is in their length.
Main advice is to use underground pipe wherever it is possible, because they are counted as 2 pipe segments but their length is 11 segments.

[ratchetfreak]

But it looks like it can help throughput to put 2 underground pipelines next to each other and interconnecting them periodically.

[XKnight]

I performed some virtual simulation and found quite interesting stuff:

Imagine you have a two storage-tanks connected via long pipe (let say 100 pipe segments), and these segments were built one by one without using robots. If we do two almost the same experiments but with opposite direction results will be different:
1 experiment: First tank has 2500 water inside, second - 0
2 experiment: First - 0, second - 2500

X axis - time in ticks, Y axis - amount of water inside empty tank

1.png (4.89 KiB) Viewed 181420 times

In average, there will be difference about 100 ticks (1 tick per pipe segment) between first and second experiment in time.
So, if you don't want to waste these ticks you should build pipe according to the flow direction.

[ssilk]

Very, very interesting. So I moved it to Gameplay Help, but left a link in General.

[golfmiketango]

I performed some virtual simulation and found quite interesting stuff:

Imagine you have a two storage-tanks connected via long pipe (let say 100 pipe segments), and these segments were built one by one without using robots. If we do two almost the same experiments but with opposite direction results will be different:
1 experiment: First tank has 2500 water inside, second - 0
2 experiment: First - 0, second - 2500

X axis - time in ticks, Y axis - amount of water inside empty tank

1.png

In average, there will be difference about 100 ticks (1 tick per pipe segment) between first and second experiment in time.
So, if you don't want to waste these ticks you should build pipe according to the flow direction.

To me, this seems like a reasonable and even intuitive result -- irl fluids don't instantaneously achieve equilibrium but seek it due to external forces and whatever fluid dynamics are applicable ... that is, water flows through pipes, it doesn't "teleport" through them. So when you build toward an empty tank the water is filling the intermediate pieces as you go but in the opposite direction you end up with empty pipes and an empty tank, at the moment of connection (in normal-human-ese: a fluid has to flow through empty pipes, before it can reach the tank; building in the full-tank -> empty-tank direction gives it time to do so while we build, but if we build the other way, the water gets no "head start" flowing toward the empty tank).

Ed: fixed to actually say something close to what I mean instead of some entirely different thing

[XKnight]

To me, this seems like a reasonable and even intuitive result -- irl fluids don't instantaneously achieve equilibrium but seek it due to external forces and whatever fluid dynamics are applicable ... that is, water flows through pipes, it doesn't "teleport" through them.

That experiment describes not the equilibrium process, but the difference between opposite flow directions.

IRL when you have two barrels of oil (yea, this is a real life) and you pour first barrel into the second barrel you will recieve the same timings as in pouring second barrel into the first one. But in Factorio... the same pipe, different directions - different results.

[golfmiketango]

Yeah, you are right, I already halfway corrected my post.. To be clear I was only commenting about the information on the last chart above, not the one in OP, which looks a fair bit more intriguing.

[Fatmice]

This is expected if not well-known. Entities in a chunk are updated north and west then south and east. Thus fluid flowing south to north or east to west will be slower and such pipe will have less throughput.

[XKnight]

This is expected if not well-known. Entities in a chunk are updated north and west then south and east. Thus fluid flowing south to north or east to west will be slower and such pipe will have less throughput.

Nope, fluid flowing depends only on build order, north/west/east/south direction is not important:

game speed = 0.1



Updated video, to show both flowing direction and both build order.

[Fatmice]

Obviously, they also get updated base on their built "order." This does not change anything that I said regarding how entities are updated when it has a fluidbox. If an entity has a fluidbox, then its fluidbox will always be updated north and west then south and east. That's the flow priority. This is straight from a dev's mouth when long ago I played with fluid and presented this information to them.

Suppose you have a string of two straight pipes

1->2->3->4->5....->100

101->102->103->104->105....->200

Suppose you attach each pipe-1 and pipe-101 to a storage tank->pump and switch them on simultaneously. The pipe-1->100 will fill then pipe-101->200. This is just as you pictured.

Now reverse the build order such that you have the 1->100 below the 101->200 but keep the source setup the same.

101->102->103->104->105....->200

1->2->3->4->5....->100

The pipe-1->100 will still fill first then pipe-101->200.

Now where the effect of fluidbox update is seen is when you attach the source to the pipe-100 and pipe-200 and pump to the left while having built the pipes in the same order to the right. The time for the latter setup to completely fill with fluid will be slower than the prior setup. In fact, without doing such lengthy setups, one can simply toggle the state of the pipe through the console. You will find that north->south and west->east flow can not be stopped but the reverse can. This entity with fluidbox update mechanic has been in around since the introduction of fluid. It was present when I first played the 0.6 version of Factorio. It might not stay though since 0.15 will likely bring some changes to how fluid updates will be handled as part of their optimization pass.

[XKnight]

Suppose you have a string of two straight pipes

1->2->3->4->5....->100

101->102->103->104->105....->200

Suppose you attach each pipe-1 and pipe-101 to a storage tank->pump and switch them on simultaneously. The pipe-1->100 will fill then pipe-101->200. This is just as you pictured.

Maybe I missed something, but if 1,2,3,...,101,...,200 is a build order then first and second strings will be filled at the same time.
Different timings will be in the next situation:
Source1->1->2->...->100
Source2->200->199->...->101

Have you made a mistake? or I didn't understand correctly?

[golfmiketango]

This is expected if not well-known. Entities in a chunk are updated north and west then south and east. Thus fluid flowing south to north or east to west will be slower and such pipe will have less throughput.

Nope, fluid flowing depends only on build order, north/west/east/south direction is not important:

game speed = 0.1



Updated video, to show both flowing direction and both build order.

Oh. Wow. I had just assumed this couldn't possibly be what people meant. That's pretty ridiculous, then

[golfmiketango]

There must be some way to exploit this for fun and profit. I can't think of any, yet, but I am now convinced this is important Kind of like, when you do your experiment and light from a moving object appears to be going at the same speed as light from a stationary object -- we must be missing something here.

[golfmiketango]

Obviously, they also get updated base on their built "order." This does not change anything that I said regarding how entities are updated when it has a fluidbox. If an entity has a fluidbox, then its fluidbox will always be updated north and west then south and east. That's the flow priority. This is straight from a dev's mouth when long ago I played with fluid and presented this information to them.

Suppose you have a string of two straight pipes

1->2->3->4->5....->100

101->102->103->104->105....->200

Suppose you attach each pipe-1 and pipe-101 to a storage tank->pump and switch them on simultaneously. The pipe-1->100 will fill then pipe-101->200. This is just as you pictured.

Now reverse the build order such that you have the 1->100 below the 101->200 but keep the source setup the same.

Here, I guess you mean, not just "build order" but to create a mirror image, including mirroring the build order used to create the original diagram?

101->102->103->104->105....->200

1->2->3->4->5....->100

The pipe-1->100 will still fill first then pipe-101->200.

Now where the effect of fluidbox update is seen is when you attach the source to the pipe-100 and pipe-200 and pump to the left while having built the pipes in the same order to the right. The time for the latter setup to completely fill with fluid will be slower than the prior setup. In fact, without doing such lengthy setups, one can simply toggle the state of the pipe through the console.

Can you be more specific about this? What state?

You will find that north->south and west->east flow can not be stopped but the reverse can.

Stopped how?

This entity with fluidbox update mechanic has been in around since the introduction of fluid. It was present when I first played the 0.6 version of Factorio. It might not stay though since 0.15 will likely bring some changes to how fluid updates will be handled as part of their optimization pass.

Sounds like you are onto something, but I don't get it. You're talking about these things I guess? Interestingly, the documentation suggests they are treated as immutables by the native game engine. This suggests, in turn, that they could be treated as singletons, iow, that what appear to be multiple instances may not actually be unique, distinct objects.

[ignatio]

This is very interesting and I have some questions on the OP:

- I'm interested in the maths there, can you show the formulas you found out?
- In your graph I take it that the pipes are full, so it's only the flow rate (2nd component) that influences it. Is that correct?
- What is the reason for the discontinuity at 1.7 units/tick? It makes no sense to me.
- I'd like to experiment a bit myself. Can you post a blueprint for your test setup?

Thanks!

[ignatio]

I found the maths in another post, so I can answer most of my questions myself:

- I'm interested in the maths there, can you show the formulas you found out?

Taking the result from said post:

Distance in pipe segments = (20 - 2 Flow) / (3 Flow - 5 * Min[Flow*0.59, 1])

Plugging in DaveMcW's values, I find this formula to be off by about 2 segments for all values (ignoring 0 pipes). At short distances that means the error is large, e.g. for 150 units/sec = 2.5 units/tick the formula gives 6 pipes, not 4. Or put the other way around, the formula says there should be 171 units/sec at 4 pipes. So it can be fitted a bit better to data.

- In your graph I take it that the pipes are full, so it's only the flow rate (2nd component) that influences it. Is that correct?

No, if the pipes are full there's no pressure difference and hence no flow. I reckon even a fairly short pipe with input capacity well above its output isn't entirely full at the output end, but it can be hard to see.

- What is the reason for the discontinuity at 1.7 units/tick? It makes no sense to me.

This would be due to that the contribution from the flow at the previous tick is simply capped at 1. That's interesting because it means the best flow/pipe ratio, iow pipe economy, might be near that spot: Put pumps at about every 195th pipe, and if you need more than 1.7 units/tick = 102 units/sec it might be better building parallel pipes instead of shortening segments further. However, the total economy for building a long pipe is also determined by the relative cost between pipes and pumps.

More measurements near this point would be nice to confirm this.

- I'd like to experiment a bit myself. Can you post a blueprint for your test setup?

[Aru]

Wow, thank you so much for figuring this out. I looked all over the forums and couldn't find this information either, then I was trying to figure out the formula myself. I was about to start searching for a way to advance the game one tick at a time so I could measure the fluid levels in all the pipes.

I was assuming "flow_to_energy_ratio = 0.59," (in data\base\prototypes\fluid\fluid.lua) had something to do with conversion of warm fluid to electrical energy, but apparently "energy" actually means the momentum of the fluid. So, it would have taken forever for me to try to integrate the 0.59 factor in any case. What a strange name! "pressure_to_speed_ratio = 0.4," makes more sense though, except that the word "pressure" is not accurate.

edit: I finally got an accurate simulation in Python. I see now why placement order matters. The silly game, instead of calculating a flow rate for each pipe joint and then going back and updating all the fluid levels, it transfers fluids immediately after every calculation of a connection. It calculates flow, then transfers, then calculates flow, then transfers, etc. for each joint in the order they were placed. It would make more sense to calculate all flows, then transfer all fluids, but it doesn't. I ran the simulation on 147 pipes for 20 game seconds, with an offshore pump on one end and no engines on the other end, both placed in order from pump to end, and also in reverse order placed from end to pump, and both calculations matched what I measured. 7.8417 for correct direction, 6.8724 for pipes placed in reverse. I could also run simulations for pipes placed in any arbitrary order, or steam engines to consume it at one end, whatever.

I deduced previously that there are only two ways to interpret factorio's pipe physics in a semi-physical way. Either A: everything that can contain fluid has a flat bottom, vertical walls, an interior lowest point of identical elevation, an interior highest point of identical elevation, a top that is freely exposed to the air (or lack of), all fluids are completely incompressible, all piped fluids sink to the bottoms of the containers. This means that pipes are actually rectangles, not cylinders, and tank walls are the same height as pipe walls, just covering a much larger ground area, though this is nowhere near proportionate to the sprites. And B: all fluids are extremely compressible, all pipes and containers (fluid boxes) are filled 100% with no air. Interpretation A, the measured values are mass or volume, thus are also proportionate to the pressure at the *bottoms* of the containers, if we pretend that atmospheric pressure is very low or nonexistent, or that the containers are sealed with empty space being vacuums which somehow never vaporize the liquids. Interpretation B, measured values are mass, thus proportionate to density and pressure in a container of a given size. As it turns out, A is the correct interpretation. In either case, the 'momentum' mechanic of flow_to_energy_ratio looks really cool visually (it's responsible for fluids slightly 'bouncing' back and forth in the pipes like gravity waves, and interpretation A is that they are indeed gravity waves of an incompressible fluid, like water waves; B is that they are compressing pressure waves). The physics are probably more accurate than if this mechanic wasn't there, but it's probably not very accurate, just a gross approximation of real fluid momentum.

Then, it is accurate to describe offshore pumps and small pumps as putting fluid out at maximum height, taking fluid in from lowest height. Consumers like engines, refineries and chemical plants also taking in from the bottom. They all produce / consume a flat amount of fluid, so long as it is there to consume. The rate is not being influenced at all by the amount of fluid or pressure in the incoming container, unless there's not enough to meet the consumer's demands, then of course it only takes what it can. Small pumps transfer 0.5 fluid per game engine tick (1/60 seconds). So, there is no form of suction at all, only consumption and 'pumping forward' of the fluid that is available at the intake.

I'm still a little hazy on pipes which connect to more than 2 things. Perhaps the game maintains a list of all connections formed, in the order that they're formed. Then the question is, when you place a pipe that connects on multiple sides simultaneously, how does it determine the order? Or, very unlikely, but perhaps it calculates the fluid changes of a group of more-than-two-way connected pipes (as I falsely assumed above for straight pipes), then applies all changes in the group at the same time. It's more plausible that the game maintains a hidden, ordered list of fluid junctions between containers, and this is the true underlying 'placement order' which influences fluid physics. Not so much the actual building placement order, though that would determine the junction order (with an unknown element determining the order of simultaneously-appearing junctions, perhaps ordered like graph quadrants as with sector updates). It is going to be very hard to determine the order of these simultaneous junctions. It might require the aid of a custom mod. Even with the knowledge of the order, it might be difficult to understand how the order was decided. But if the game calculated the changes before applying them, the order would no longer matter anyway.

The post from fatmice made just as little sense to me as it did to XKnight.

And, I don't understand what's going on in this .webm video, because I can't see the greater context. Why are all four pipes filling at different rates? It looks like the top one is the only one that's built in the right order, because it gets some water after the first tick. It also looks like, the thresholds for showing the water drop icon and showing fluid in the meter icon are different. (Then I recreated this test myself, it looks like this has been changed, they align better now, probably they use the same threshold. The water drop symbols on the top pipeline only appear at the same thresholds where the water in the meter appears, rather than beforehand. From testing, I'm fairly certain this threshold is about 0.5 in the pipes, probably exactly 0.5. Perhaps when the video was made, the threshold for water appearing in the meter for pipes was still about 0.5, but the threshold for the water drop icon was much lower, maybe 0.01.)

[Fatmice]

It has been a while since I read this topic. I'm as much interested in pipe mechanic as any of you. My mod basically depends on pipe mechanics.

I ran the simulation on 147 pipes for 20 game seconds, with an offshore pump on one end and no engines on the other end, both placed in order from pump to end, and also in reverse order placed from end to pump, and both calculations matched what I measured. 7.8417 for correct direction, 6.8724 for pipes placed in reverse. I could also run simulations for pipes placed in any arbitrary order, or steam engines to consume it at one end, whatever.

Yes, this is known and confirmed by devs a long time ago. Placing sources follow by a continuous section of pipes away from source will always have better throughput than placing pipes then source in the opposite direction of how the pipes were placed.

I deduced previously that there are only two ways to interpret factorio's pipe physics in a semi-physical way. Either A: everything that can contain fluid has a flat bottom, vertical walls, an interior lowest point of identical elevation, an interior highest point of identical elevation, a top that is freely exposed to the air (or lack of), all fluids are completely incompressible, all piped fluids sink to the bottoms of the containers. This means that pipes are actually rectangles, not cylinders, and tank walls are the same height as pipe walls, just covering a much larger ground area, though this is nowhere near proportionate to the sprites. And B: all fluids are extremely compressible, all pipes and containers (fluid boxes) are filled 100% with no air. Interpretation A, the measured values are mass or volume, thus are also proportionate to the pressure at the *bottoms* of the containers, if we pretend that atmospheric pressure is very low or nonexistent, or that the containers are sealed with empty space being vacuums which somehow never vaporize the liquids. Interpretation B, measured values are mass, thus proportionate to density and pressure in a container of a given size. As it turns out, A is the correct interpretation.

Yep. Pipes are containers with some height and a ceiling. Fluids are items stack to some height. You take some out and move them around. This is why fluid stacked to the ceiling everywhere can not move.

In either case, the 'momentum' mechanic of flow_to_energy_ratio looks really cool visually (it's responsible for fluids slightly 'bouncing' back and forth in the pipes like gravity waves, and interpretation A is that they are indeed gravity waves of an incompressible fluid, like water waves; B is that they are compressing pressure waves). The physics are probably more accurate than if this mechanic wasn't there, but it's probably not very accurate, just a gross approximation of real fluid momentum.

Don't get too deep. It's not physics.

Then, it is accurate to describe offshore pumps and small pumps as putting fluid out at maximum height, taking fluid in from lowest height. Consumers like engines, refineries and chemical plants also taking in from the bottom. They all produce / consume a flat amount of fluid, so long as it is there to consume. The rate is not being influenced at all by the amount of fluid or pressure in the incoming container, unless there's not enough to meet the consumer's demands, then of course it only takes what it can. Small pumps transfer 0.5 fluid per game engine tick (1/60 seconds). So, there is no form of suction at all, only consumption and 'pumping forward' of the fluid that is available at the intake.

There's no such thing as suction anyway, even with real physics. But entities with fluid boxes are just special pipes. They are placed higher or lower than the pipes. Like things on a hill side where things at the top roll to the bottom.

I'm still a little hazy on pipes which connect to more than 2 things. Perhaps the game maintains a list of all connections formed, in the order that they're formed. Then the question is, when you place a pipe that connects on multiple sides simultaneously, how does it determine the order? Or, very unlikely, but perhaps it calculates the fluid changes of a group of more-than-two-way connected pipes (as I falsely assumed above for straight pipes), then applies all changes in the group at the same time. It's more plausible that the game maintains a hidden, ordered list of fluid junctions between containers, and this is the true underlying 'placement order' which influences fluid physics. Not so much the actual building placement order, though that would determine the junction order (with an unknown element determining the order of simultaneously-appearing junctions, perhaps ordered like graph quadrants as with sector updates). It is going to be very hard to determine the order of these simultaneous junctions. It might require the aid of a custom mod. Even with the knowledge of the order, it might be difficult to understand how the order was decided. But if the game calculated the changes before applying them, the order would no longer matter anyway.

The order does matter. Update is done North->South and West->East direction first. This is where order really matters. If you placed a pipe, then placed another pipe directly south of it, you can not stop the flow of liquid by disabling the southern pipe with the console. The reverse can be done. Also you can never place anything simultaneously. All entities are placed individually on each tick. Thus there is always an order or indexing number. This corresponds to their id number, which you can query through lua.

The post from fatmice made just as little sense to me as it did to XKnight.

I hope that made it more clear.