Fluid Mechanics Still Broken, 6 Years Later?

[Fuddin]

I just started playing this game a couple days ago. Fluid flow seemed to be behaving in bizarre ways, so I wanted to get a greater understanding of how Factorio's fluid mechanics worked. I looked online and found various conflicting information, such as different fluids can have different viscosities, or they can't. Eventually, I came across Friday Facts # 260, written in 2018, where Dominik pointed out problems with the system, such as:

The first of three main issues is that in junctions it behaves in a very random fashion. As a result, you might get recipients that are not getting any fluid where they obviously should be.

, along with a visual:to demonstrate this, and:

fluids moving faster or slower depending on the entity update order

He then talks about some possible solutions, and says

So to sum up, we have had this on our minds for a long time now, and performance was not the only issue we have considered. The new system will hopefully address all the issues we mentioned at the start.

Then, several months later, in Friday Facts # 274, he describes a solution he came up with that fixes these problems. Under that solution, fluids can have different masses and frictions, so they behave differently in pipes. More importantly, that solution made it so that the end result would be the same regardless of the order of evaluation of the entities, which is entirely logical, and solves the problem with junctions not behaving as they should. Finally, you can now see flow rate information in the entity info of each pipe.

But, when I looked in my own game today, there was no flow rate information in the entity info of pipes. Moreover, the problems with junctions and different entity evaluation orders influencing the results clearly hadn't been fixed.

I made this setup, which started off entirely empty of any fluid, and then as my last action, placed the offshore pump.
Despite being completely symmetrical, the right storage tank got almost all the water. That was when I built the right side of the junction first. Then, I redid the experiment, but built the left side of the junction first, and almost all the water went to the left storage tank. So, the result of fluid flow is dependent on the order the pipes were built, which makes no sense - it should solely be dependent on the geometry of the system and not the arbitrary order things were created. This makes it impossible to reason about how fluids will act in junctions, especially since the order in which bots build blueprints can vary.

Then, in this forum thread from 2020, it seems that the fluid improvements were thrown out completely, and have never been touched since then?

Are there any plans to fix this core mechanic? And if not, what are people's work arounds? Just use single pipes with one producer and consumer each? Avoid pipes as much as possible and barrel everything at source and unbarrel when it's consumed? Wing it and ignore the fact that fluids move unpredictably and inconsistently?

[Koub]

Are there any plans to fix this core mechanic? And if not, what are people's work arounds? Just use single pipes with one producer and consumer each? Avoid pipes as much as possible and barrel everything at source and unbarrel when it's consumed? Wing it and ignore the fact that fluids move unpredictably and inconsistently?

Nice summary, you did your research well

Expect the worst (no change), but hope for the better (one of the devs slips on the wet toilet seat while setting up a clock, hits the toilet with the head, and when they get back to consciousness, they have the idea that will fix fluids forever, implement it as a side project, the change is accepted, and champagne !).

That's my stance on the subject at least.

[FuryoftheStars]

what are people's work arounds? Just use single pipes with one producer and consumer each? Avoid pipes as much as possible and barrel everything at source and unbarrel when it's consumed? Wing it and ignore the fact that fluids move unpredictably and inconsistently?

Avoid overly complicated pipe branching, or use a tank + pumps where needed and issues show.

Edit:
By the way, with this...

Then, in this forum thread from 2020, it seems that the fluid improvements were thrown out completely, and have never been touched since then?

...it's hard to find the relevant info you're citing with that link. Going through it, I found 1 dev comment, here: viewtopic.php?p=475106#p475106
(And they didn't directly say it was being thrown out, just that they weren't sure if it'd be in 1.0, depending on dev time.)

Without further word somewhere or parsing through everything, it's hard to know why, especially if they already had something in the works. It could have been causing a significant performance impact. It's also possible they ran into some issues they weren't able to solve at the time and did not mention. It might be coming in the 2.0 update, or it might not be at all because they were never able to find a nice, performant way that works with determinism.

[Loewchen]

Those are all distribution problems with demand exceeding supply, produce as much as you need and you have no issues.

[mmmPI]

Are there any plans to fix this core mechanic? And if not, what are people's work arounds? Just use single pipes with one producer and consumer each? Avoid pipes as much as possible and barrel everything at source and unbarrel when it's consumed? Wing it and ignore the fact that fluids move unpredictably and inconsistently?

I don't know for plans to change the core mechanic. I rarely encounter problematic situations anymore, maybe because i learned to build around the mechanic because i play for long and that is "i ignore" answer, or "i have developed workarounds". The setup you show would stabilize after around a minute because then both tanks would be filled up with water. That's an ignore to me and wing it. The moments when it would matter the most which side of the junction is updated first is when fluid quantity are quite low. In those situation to me the problem can be seen as a lack of fluid. Even if one could see it as uneven split problem, i tend to see it as : In game if you split unevenly and 1 side takes too much, it will either be full and block at some point and the other side will have its needs. Or it means you don't have enough for both sides of the junctions for a prolonged amount of time, and as such implementing a priority splitting could do the job.

In the test setup, you could have a pump after both tank and you allow both pumps to push liquid to consumer only when both tanks have 1000 fluids or more. This way if one tank runs dry, the other side's consumer is not free to take everything and starve the dried tank, it has to wait. I know it's possible to do, but i never found a situation where i needed it in vanilla factorio, only with mods where i should have used barreled but eh i was lazy. Barrels are good with robots, i had no robots. I avoid barrels when i have no robots. There used to be mandadory with trains before fluid wagons were introduced, now they are less often used to me.

There are ways to predict the level of water in junctions it's lots of heavy math. I know there are many other posts, with explanations that are often just as hard to follow for me. But for this one we were given some more explanation on the forum related to the wiki's formula, maybe you've seen it already in your research, this is not about junction, it takes the case of a straight line of pipes and goes to explain how and why the fluid will eventually settle to a level and which one. In doing so the discussion talks a little about the update order and how fluids are calculated i thought you may be interested. It's not very practical information for every game as in you don't NEED to keep all in mind to play the game sucessfully and have great fun, but if you are into the nitty gritty details of simulations then there's a lot of lecture

To me it seems like method to optimize such kind of simulations are subject of research for computer scientists that publish paper currently, it's a non trivial problem. In real life engineers when using models needs those kinds of math to predict a lot things in constructions , like air flow and temperature and they often resort to using dicrete simulations ( like in factorio) to approximate real life because it is a faster method for computers than more accurate math methods. So i doubt it's easy to make a faster model that would be as accurate. There are already little inaccuracies that are trade off for performance, a more accurate model would be slower, and a faster model less accurate. And if you ask players i guess they will want both with a 50:50 split giving priority to fixing the junctions thingy, and the other half making the system less accurate, but even faster.

It's been working for (at least) 6 years

[Tertius]

what are people's work arounds?

You can ignore the finer details of fluid distribution, if you produce as much as or slightly more than your consumers are able to consume. But not less, because in this case one consumer will get all it can and the other(s) will get the rest.

Don't design a setup where you produce 1000 fluid/s to one pipe and you split this pipe for 2 consumers who are both able to consume 900/s each, a total of 1800/s. In this case, one consumer will get 900/s and the other 100/s. But not both 500/s.

If you want both 900/s consumers get only 500/s each, don't build one producer with 1000/s. Instead, build two producers with 500/s each. Or build each consumer in a way that it isn't able to consume 900/s while you want to feed it with only 500 fluid/s.

If you want to explicitly prioritize one consumer, careful use of (perhaps even circuit controlled) pumps could help. Pumps are not only there to change the fluid level, they can also act as valves.

Don't see this behavior as bug or unwanted. Instead, it's an unexpected law of physics that only happens in the Factorio simulation but not in the real world. See it as challenge you need to handle, just as one of the many other challenges the game has to offer for the player to solve.

[Fuddin]

Re: Koub:
Haha, thanks! I spent so long scouring the web and testing various setups of mine.
And huh, darn.

Re: Loewchen:
That still doesn't solve the issue that the underlying system doesn't make sense. The order that pipes were created should not influence the behavior of the system in any situation, regardless of the amount of fluid present.

Re: mmmPI:
Re: PDF with math: I did come across that while looking online. However, it was made in 2017, before even the Factorio Friday posts I mentioned before, so I wasn't sure if that was still true or not. It also mentions parts of fluid flow they haven't figured out and assumptions they made, and says, "As a result it’s impossible to make any generic predictions about how flow numbers will be affected if you change something", as a result of entity evaluation order in the section about junctions.
Re: putting in pumps and controlling their behavior with the circuit network: That makes sense as a workaround; will try it if I need it.

Re: Tertius

Don't see this behavior as bug or unwanted. Instead, it's an unexpected law of physics that only happens in the Factorio simulation but not in the real world. See it as challenge you need to handle, just as one of the many other challenges the game has to offer for the player to solve.

Well, that's certainly a valid philosophy to have. Personally, I'm going to regard this as a bug to work around, though.

[Fuddin]

Update:
Just thought of a simple way to make junctions behave sanely that only uses 2 combinators:

The 2 combinators pictured above are a counter that counts from 0 to 1 inclusive and then repeats. The pump at left is only on when the counter is 0, and the pump at right is only on when the counter is 1. So, each one is on half the time, and only one of them is on at any given time. This isn't exactly like how the junction should actually behave, since it should in fact output half its fluid to each side at every tick, rather than all of its fluid to one side on one tick and then all fluid to the other side on the next tick, but since a tick is only 1/60th of a second, this is a close approximation to how it should actually behave.

Since the circuit network signals are 32 bit integers, you can make junction priorities any fraction a/b, where a ≤ b and b is any integer from 1 to 2^32. Though, note that the larger b is, the larger the worse the approximation is to a system that outputs a/b of the fluid to one pump and (1 - a/b) of the fluid to the other pump at every tick, since the alternation period is larger.

[aka13]

I'd go +1 with Tertius and Koub, that while I also would enjoy more "fun" pipe mechanics, in the end it's a supply-demand problem.
As long as you produce at least some amount over the required minimum, your system will balance out.
In all other cases, yeah, for maintaining flow pumps are a must.

[Stringweasel]

In the latest FFF we can see the fluidic mechanics is at least on one of the dev's minds.

I usually try not to look at the C++ code because every time I do I get compelled to start changing things… (like the fluid mechanics) when really I should be focussing on other things.

But personally I don't find the fluid mechanics that bad. If it's acting weird I just see it as my production isn't enough, which is usually the case anyway. I'm not really a megabase builder though where I can imagine it can get annoying if it's another non-intuitive contraint to design around.

[Koub]

I get into issues only when I try to max out every flow capacity possible with "megabase" level oil facilities. On setups where I don't need absolute maximal throughput, Fluid system works just fine.

[mmmPI]

Re: PDF with math: I did come across that while looking online. However, it was made in 2017, before even the Factorio Friday posts I mentioned before, so I wasn't sure if that was still true or not.

Well i may be wrong and things have changed since then, but that was not my impression and your research seem to confirm that the same system is still working. I haven't done precise measurement on junctions fluid flows to confirm personnally though, the informations on the wiki and from the discussion seem to describe in game behavior.

I found that in cases when i want to have absolute control over the fluid flow pumps and wire can do. But for big nuclear power plant though this kind of knowledge is useful as guidelines for most efficient practices.

It also mentions parts of fluid flow they haven't figured out and assumptions they made, and says, "As a result it’s impossible to make any generic predictions about how flow numbers will be affected if you change something", as a result of entity evaluation order in the section about junctions.

So again i may be wrong, from my understanding, one can't make generic predictions because one need to take into account the evaluation orders in junctions. Only specific predictions can be made, when this order of evaluation is known. Adding and removing pipes that are part of a junction can influence which pipes will be updated before which other and which one receive more fluids in case there isn't enough left when the 2nd pipes from the junction is updated.

In theory i suppose it would be possible to not update pipes individually in relation to immediate neighbour but also the neighbour of their neighbour so as to mitigate the influence of the update order. But that would come with a cost in performance, "to make the system more realistic". Since each pipes would individually require several times more computation every time it is updated, and every pipes are updated every tick. 'Not everyone's favourite' when there are also propositions to group pipes together so as to reduce amount of computation required. ( not simulating every single pipes as 1) . To me this could cause some other inaccuraries, such as allowing higher throughput inside certain section of pipes without pumps than in sections with pumps to help the flow, or fluid moving at the speed of light inside their own group of pipes in certain conditions. This is from reading and participating in such discussion but i am by no mean an expert. I thought it could be a difficult decision to make.

To me it seem related to the problem of finding numerical solution of the convection–diffusion equation which is a problem tackled by experts. It can takes some time for some of those breakthrough to be applied in domain like video games i thought, discussion around it can only help identify new methods that people develop that would be both faster and more realistic so as to please everyone if they exist. But i don't think i'm qualified for that

Just thought of a simple way to make junctions behave sanely that only uses 2 combinators:
The 2 combinators pictured above are a counter that counts from 0 to 1 inclusive and then repeats. The pump at left is only on when the counter is 0, and the pump at right is only on when the counter is 1. So, each one is on half the time, and only one of them is on at any given time.

That is more space efficient than using storage tank to read fluid quantity

This isn't exactly like how the junction should actually behave, since it should in fact output half its fluid to each side at every tick, rather than all of its fluid to one side on one tick and then all fluid to the other side on the next tick, but since a tick is only 1/60th of a second, this is a close approximation to how it should actually behave.

But if one side of the junction is backed up, then you may end up needlessly throttling the other side.

Since the circuit network signals are 32 bit integers, you can make junction priorities any fraction a/b, where a ≤ b and b is any integer from 1 to 2^32. Though, note that the larger b is, the larger the worse the approximation is to a system that outputs a/b of the fluid to one pump and (1 - a/b) of the fluid to the other pump at every tick, since the alternation period is larger.

If this is familiar to you after just a few weeks of playing, i have no doubt you will find more of such systems to deal with fluids. This one is pretty cool though

[jamiechi1]

I say throw all the current fluid code out and rewrite it. Another of my favorite games (using Unity game engine), created a system of pipes that are basically conveyor belts with different graphics. It does mean that fluids only will flow in one direction. Which does greatly simplify the code and lowers the processing required to a large degree. This will make it much easier to make Megabases. I think the processing power would be better utilized in other areas of the game.

Trying to simplify fluid dynamics just takes way too much processing power and requires too much head scratching by the developers. This is a Game and Not a physics simulation. Another game on early access, is also trying to get fluid processing to work well in the Unreal game engine. They are also failing to get it to work properly.

[NineNine]

Fluid dynamics (including turbulence) is something that actual physicists still cannot model accurately. That's hwy we still have wind tunnels. So to even be able to begin to accurately critique fluid mechanics on Factorio, I'd expect you'd need a phD.

I agree with jacmiech1. Forget the fluid mechanics and work on other parts of the game. If you don't have enough throughput, just add another pipe and some pumps.

[Fuddin]

Well i may be wrong and things have changed since then, but that was not my impression and your research seem to confirm that the same system is still working.

Yeah, I agree; it does seem the system is the same as it was then.

That is more space efficient than using storage tank to read fluid quantity

But if one side of the junction is backed up, then you may end up needlessly throttling the other side.

Since the circuit network signals are 32 bit integers, you can make junction priorities any fraction a/b, where a ≤ b and b is any integer from 1 to 2^32. Though, note that the larger b is, the larger the worse the approximation is to a system that outputs a/b of the fluid to one pump and (1 - a/b) of the fluid to the other pump at every tick, since the alternation period is larger.

If this is familiar to you after just a few weeks of playing, i have no doubt you will find more of such systems to deal with fluids. This one is pretty cool though

Thanks! And oooh, about my symmetric junction attempt unnecessarily throttling one side if the other side is backed up: that is an excellent catch, and it hadn't even occurred to me! I will have to improve my solution. Man, I wish Factorio pipes would output their current flow rate to the circuit network.

[mmmPI]

Yeah, I agree; it does seem the system is the same as it was then.

I think the temperature of fluid is not used much in vanilla factorio. The water not only propagate in pipes, but the temperature of the water doesn't have to be the same everywhere it also propagate. The same goes for steam with additionnal calculations required to mix steam from boilers or heat exchangers that have different temperature but almost no one will mix on purpose. Maybe this would be less of a change than making the fluid system less like fluid and more like hidden conveyor belts.I think it would remove some of the gameplay value in the game personnaly to remove fluid altogether and make them act like hidden item. I'd rather see the temperature control being more integrated in the game , but that's rare even in mods.

Thanks! And oooh, about my symmetric junction attempt unnecessarily throttling one side if the other side is backed up: that is an excellent catch, and it hadn't even occurred to me! I will have to improve my solution. Man, I wish Factorio pipes would output their current flow rate to the circuit network.

Thank you too x) but to be honnest i'm not sure it's a good catch, because such pump even when working half of the ticks could still be bottlenecked by the pipes on each sides. A pump is most efficient between 2 tanks, as pipes can only have 100 fluid, so best case everything is transfered every tick, and that is 60*100 fluid per second, so in your particular setup, it wouldn't matter the throttling. The limit could also happen to be between the pipe that feed into the pumps and the pipe before it. A more generic solution is difficult

When you mention the current flow rate of pipes, that's the trick, they don't have one ! It's only a quantity of fluid that changes from one tick to another, but the fluid can go one way or another. It would be negative flow ? But then pipes don't have directions they are only connected little boxes so which way is the positive and negative flow ? There are good advices on the thread as to launch rocket, you don't need to dive deep into those mechanism because you could find easier solutions. But it's also fun to do so i found I thought one can only read quantity of fluid in storage tanks, or in train wagon or in barrels but then when you're out of power you know you have a steam problem so that is also a way to get indirect information on fluid flow ^^

[XT-248]

I have dealt with Factorio Fluid Box for years in Vanilla and Modded gameplay.


I strongly recommend using a one-direct valve, a top-up valve, and an overflow valve (available in mods and indispensable for better fluid distribution). This helps mitigate the downside of unpredictable fluid flows and less-than-equal junctions.

I hope those valves become official features and will be improved with a new fluid box for the Space Age release. I hope, but I'm not expecting that both will happen for Factorio-2.0 and Space Age release.

[Tertius]

I strongly recommend using a one-direct valve, a top-up valve, and an overflow valve (available in mods and indispensable for better fluid distribution). This helps mitigate the downside of unpredictable fluid flows and less-than-equal junctions.

This sounds as the vanilla game is lacking or insufficient if it comes to fluids, but it isn't. You don't need mods for fluid management, if you follow the general rule to not produce less than you're able to consume and not split a pipe, if you require equal distribution across the split. That's really not very difficult.

There is only one situation I encountered where I was not able to produce more than I was able to consume, and I resolved this case with the use of a storage tank (to be able to measure fluid level) and a pump (acts as a valve). It's the lubricant production in my mall. It has to decide what to do with the heavy oil: either convert to lubricant or convert to light oil by cracking. So there is a storage tank for heavy oil and a storage tank for lubricant and a pump in the pipe between lubricant production and cracking. Heavy oil flows like this: storage tank → lubricant production → pump → cracking. The pump activates, if the heavy oil storage tank has more than 24000, so surplus heavy oil is cracked and removed. And lubricant is always produced at maximum, since it doesn't have any restriction, which is exactly what I want.

Another situation is water supply for a nuclear reactor plant or an oil refinery plant. You might be tempted to merge multiple offshore pumps into one pipe to have less pipes in parallel running to the power plant, then split it again for water distribution. The most easy solution is to not merge in the first place but use many offshore pumps with all their own pipes with low throughput to the plant in parallel. No split needed, and since the throughput per pipe is lower, you need less pumps (or even none at all) in between to keep up throughput as nice side effect.

[FuryoftheStars]

one-direct valve, a top-up valve, and an overflow valve (available in mods and indispensable for better fluid distribution)

For vanilla gameplay, these are achievable with pumps and tanks with circuit wires; no combinators or advanced logic required. They just take up more space and require power compared to the modded.

There certainly are some fancy setups where these make things easier or better, but I wouldn't say that they're "indispensable".

[XT-248]

I strongly recommend using a one-direct valve, a top-up valve, and an overflow valve (available in mods and indispensable for better fluid distribution). This helps mitigate the downside of unpredictable fluid flows and less-than-equal junctions.

This sounds as the vanilla game is lacking or insufficient if it comes to fluids, but it isn't. You don't need mods for fluid management, if you follow the general rule to not produce less than you're able to consume and not split a pipe, if you require equal distribution across the split. That's really not very difficult.

There is only one situation I encountered where I was not able to produce more than I was able to consume, and I resolved this case with the use of a storage tank (to be able to measure fluid level) and a pump (acts as a valve). It's the lubricant production in my mall. It has to decide what to do with the heavy oil: either convert to lubricant or convert to light oil by cracking. So there is a storage tank for heavy oil and a storage tank for lubricant and a pump in the pipe between lubricant production and cracking. Heavy oil flows like this: storage tank → lubricant production → pump → cracking. The pump activates, if the heavy oil storage tank has more than 24000, so surplus heavy oil is cracked and removed. And lubricant is always produced at maximum, since it doesn't have any restriction, which is exactly what I want.

Another situation is water supply for a nuclear reactor plant or an oil refinery plant. You might be tempted to merge multiple offshore pumps into one pipe to have less pipes in parallel running to the power plant, then split it again for water distribution. The most easy solution is to not merge in the first place but use many offshore pumps with all their own pipes with low throughput to the plant in parallel. No split needed, and since the throughput per pipe is lower, you need less pumps (or even none at all) in between to keep up throughput as nice side effect.

Problem #1) The Pump requires power to operate and requires separate isolated solar panels/accumulators to keep the pump and circuits working.

Problem #2) The Pump requires an external logic circuit to perform something slightly more complicated than a simple boolean operator on a single fluid.

Problem #3) The Fluid Box mechanics moves internal fluid into the oldest pipe first and continues to do so today, forcing the utilization of the pump with all drawbacks in #1 and #2.


The valve solves all of those and more. It goes one extra step further to enable prioritization based on the fluid level inside the pipe, similar to splitter input/output prioritization with in-built circuit logic.

Notice that the three rows of valves have different heights of green (what they have) and blue (movement between entities) bars. The top row is the same as the pump: one direct valve. The middle row is an overflow valve, allowing flow only if the input is above 80%. The bottom row is a Top-up valve, allowing flow only if the output pipe is below 80%.

Fluid_Management_Factorio.png (203.13 KiB) Viewed 4316 times

It is not that the Vanilla Factorio fluid management is lacking or insufficient. I would rather have something that doesn't have #1 and #2 drawbacks, and there is room for improvement to the Vanilla Factorio fluid management.

I am so used to having those valves that going back to the Vanilla Factorio's pumps plus circuits seems like a step backward.