Friday Facts #416 - Fluids 2.0

[Dogfood]

I was extremely excited to see that an FFF would finally address by far the most frustrating mechanic in the game: fluids.

And I was equally extremely disappointed to find out the solution that the developers went with.

I would have much preferred further effort put into refining the system from FFF-274. Teleporting fluids, single blocks... It's just so simplistic compared to FFF-274's system. And it doesn't evoke fluid flow in the same way. Now, to be clear, I am not claiming to value "realism". Realism is a red herring. I don't want something that, you know, approximates solving the Navier-Stokes equations in a pipe network with sources and sinks in a physically correct way. The problem with teleporting fluids isn't primarily that it's unrealistic, it's that it's boring.

The problems with the current fluid system are

• it's impossible to model the flow of a non-trivial pipe network
• it's impossible to diagnose the flow in a non-trivial pipe network
• it's impossible to debug a non-trivial pipe network

I have very carefully put "non-trivial" in there because if I try to make the first point, someone is going to "well ackshually, there's a table on the wiki...". First, that table only applies to the trivial pipe network consisting of pipe segments in a (topological) line -- no junctions, no tanks, no machines along the pipe. Secondly, that table doesn't even apply to that, because flow depends on update order which depends on construction order. Since in the late game you are going to build with robots construction order is out of your control, hence modelling is impossible. Over the almost six years since FFF-274 I have been repeatedly gaslit about this, but now it's in Scripture FFF so I have been vindicated.

The other core logistics mechanics of Factorio all satisfy the requirements that players can model, diagnose, and debug them, at least to a large extent. Belts can be modelled with basically complete fidelity: one side of a belt carries 7.5 items/s, splitters mix their inputs 50/50 as far as possible. How many belts to I need to feed this setup? Can I run a mixed belt with these two ingredients? Simple division. If my belt-fed block isn't running, I can diagnose the problem by following the belts backwards. Aha, past me had accidentally rotated one segment, or set the filter on a splitter backwards. Then I can try to debug it, and iterate on design-diagnose-debug.

Robots are a little harder, but with robot speed and cargo capacity you can estimate how many robots you need to bring X items/s over some distance. That doesn't account for charging time, but just observing will let you diagnose the problem. No available robots? Try adding more. Robots queueing to charge? Try more charging capacity. That didn't work? Estimate the total charging power needed. Oh, it's higher than I can fit in the available space? Ok, I will try to design for shorter robot trips... and so on. Trains have a known top speed, cargo capacity, and in the simplest case with a fixed route that gives you an estimate of the throughput. If your trains are underperforming, you can follow them to measure actual trip times. It's non-trivial to say the least to calculate how many trains can pass through a junction, but a traffic jam is obvious on the map and you can (people have) measure(d) the junction capacity. And there are all kinds of debugging measures you can take -- use more trains if that's the issue, redesign crossings, change your schedules so routes are spread out over more track if traffic is the issue. And so on.

There's nothing like this for pipes. Can I feed these machines with one pipe, or do I need two? Where is the blockage in the flow? Why is this machine not running? Fuck you for even trying to figure out, the game says.

The FFF-416 model, to its credit, fixes this. But it does so by obviating the need for the design-diagnose-debug loop that is so much fun with all the other logistics mechanics. I'll put down a pipe from the input train station to the consumer machines, done. It'll Just Work. Again, the issue isn't that it's unrealistic. The issue is I didn't have to play the game! Ideally, a good logistics mechanic

• creates interesting design challenges that
• can be (mentally) modelled by the player
• can be diagnosed by the player
• can be debugged by the player
• and evoke the real-world inspiration

The FFF-416 model passes the middle three criteria, but it fails the first and most important one, as well as the final one. Some might call that one realism, but it's not quite realism, just an attempt at representation. Van Gogh wasn't a realist painter, and a starry night doesn't really look like his painting, but we can still tell that's what it represents. Transport belts in a real-world factory don't work like Factorio belts, but the mechanics and graphics work together to make me think of a huge FedEx sorting hub when I'm routing belts for a train station. I really feel like I'm routing materials through a production line. Again, achieving this for fluids doesn't take realism in the sense of solving Navier-Stokes; that's a stupid strawman. It just needs something, like the FFF-274 model, that supported by the graphics is reminiscent enough of fluids flowing in pipes that the player feels like that's what happening. An even, instantly adjusted fill level, essentially teleporting fluid, does not achieve that.

The FFF-274 model also offers far more mechanical depth, as demonstrated in the FFF itself: different fluids can flow at different rates. You can imagine mods with several tiers of pipe with differing capacity, better and more expensive ones giving higher flow. I might splurge on the biggest pipe for bulk hard-to-pump crude, but where I only need a trace of lubricant I'll stick to something cheap and low-capacity. You could even imagine a building to heat crude specifically to make it easier to pump -- at a ginormous energy cost. (Ships have these for fuel oil, which is almost solid at room temperature.)

There have been so many fantastic additions and improvements to the game described in previous FFFs, and fluids have been a pain point for so long, that it deeply saddens me that the very promising FFF-274 model seems to have been completely discarded. We never really got to know what "issue(s with) some waves and oscillations" meant in more detail. It doesn't sound like something that couldn't be solved with stronger damping, or even left as a design challenge. (Add non-return valves to the vanilla game as a tool to deal with them?) Whatever they were, I'm sure they were not more frustrating than the unmodellable, undiagnosable, undebuggable fluid mechanics we have now, and far more interesting than FFF-416's teleporting fluids. I really hope the devs reconsider. Replacing jank with boring is a sidegrade at best.

[Malleator]

I approve of the new system, and I giddily await the construction of my (viable) intercontinental oil pipeline, but it is a shame that seemingly the flow animation is done for

[o6dukeleto]

I like the idea of "bigger" pipe segments in the new system. I think the overhead of processing every single pipe as a segment is a lot of overhead.

Maybe a solution that has a max segment length? Possibly based upon fluid viscosity?

For example water might have a max segment length of a larger number of pipes than oil?

It could be a simple as if there are too many pipes in a segment, then that segment just doesn't work for the a particular fluid - which should be easy to show in the user interface. This max segment length would also be easier to show in the wiki, as every fluid would just have a max segment length.

This would be more complex than the proposed solution, but still might be a middle ground where "enormous" pipe segments "could" become exploitable as both large storage and instantaneous delivery. It may also be more configurable as a "scenario" variable, possible as a percentage of "normal" segment length.

[Orange]

But I don't understand why throughput will be higher for longer pipelines, that doesn't make sense to me. I think that throughput in a segment should be based on the component in that segment with the lowest throughput. But if everything is simply pipes, then I think throughput should simply be the same no matter how many pipes are in that segment.

What am I missing here?

I was curious about this as well. But if true, an unintuitive corollary is that adding a pump in the middle of a pipe segment could reduce its throughput, because it gets split into 2 shorter segments.

[wretlaw120]

I don't like the new system. It reduces design complexity in the game significantly. You know, the thing I play the game for. I don't need to think when placing pipes anymore. And, unlike *some* people, I believe that's a *bad thing*. I took pride in getting my reactors to work with the constrained throughput of pipe networks in the small spaces required to keep heat pipe temps high enough to reach all the exchangers. Well, jokes on me I guess.

The aesthetics of multiple pipelines side by side all moving thousands of fluid per tick is also something that I will miss. Since all you need is one pipe, the sheer volume of throughput for a system is no longer visible, anywhere, unlike belts or trains. Ironic, considering that part of the issue with the old fluid system was readability. I guess it's pretty easy to solve readability issues if there is no longer anything to read.

Also it kinda sucks seeing people all excited for the *removal* of a feature that I enjoy, because they can't be bothered to figure it out. :/

(yes, I'm salty about it, how could you tell?)

[Svip]

I noticed that the FFF ends with this:

There are many interesting possibilities with this system that we did not predict, and we will share some of our stories in future FFFs. I may also go into detail about the architecture of this new system and how I approached the refactor at another time.

It's clear there is much more to the new fluid system than meets the eye.

Still, considering that machines, pumps and offshore pumps (I assume) are still limited by how much they produce/move, pipe segments will - as indicated - not immediately fill. This limits how much machines can draw from the pipe segment (as indicated in the FFF). So even for larger bases, you'll still need several pipes (with the same fluid) running alongside one another to achieve the throughput you are looking for.

Now the challenge becomes to think about pipe segment saturation, i.e. how full a pipe segment is, because it changes how much the machines can draw at any one time. Yes, all machines attached to the same pipe segment can draw the same amount, but the amount is dependent on how full the pipe is.

[keks244]

The new system is disappointing boring.
Not saying the old system was good, but the new one could be better.

[o6dukeleto]

From FFF #416

• Pipes, underground pipes, and storage tanks are merged into fluid "segments".
• Each segment inherits its volume from the fluid boxes that comprise it and can hold one fluid.
• Machines can push fluid into a segment at an unlimited rate, and can pull from a segment at a rate proportional to how full the segment is. In other words, if a segment is half full, then the pulling rate is half of the maximum.
• As a special case, pumps can pull at a faster rate if they are connected directly to a storage tank.

and

Fun vs Realism

Don't really care about realism, but do care about "fun".

Questions:

Should a SINGLE pipe segment be able to handle the output of 8 plants? (as shown in FFF)
Should a SINGLE pipe segment be able to handle the output of 80 plants? (seems like a natural extension, just add more fluid boxes.)

Seems to me that you would just have to add enough "fluid boxes" and you could handle the output of anything.

The only time you would need "smaller" segments is on input, you would want to keep them full enough so that each plant could pull enough fluid. But once again a SINGLE pipe "segment" could supply as many plants as you want, just make sure you have the resource and enough pumps into that segment to keep it at a minimal level that each plant can pull enough resource.

Everything becomes a single pipe segment - just appropriately sized. So the "fun" is figuring out the appropriate SEGMENT size - this is very simplistic "fun." -- almost "brain dead."

Fun in automation is having to design and new explore options as things grow bigger -- if everything just becomes a single pipe segment -- that is not much "fun."

Don't get me wrong, the current system is overly complex and has too much cpu overhead, but "dumbing it down" to this extent seems like a "one solution fits all" and removes any automation decisions that could make the game more interesting. I think some restrictions on segment size and throughput are probably needed.

[jackthesmack]

The SA expansion has so much complexity being added, and the total conversion mods currently out are brain melting. The simplifying of the fluid system will fit perfectly with the increasing scope of the game. Future total conversion mods will greatly expand the complexity, too. So don't worry about the fluid pipes being simplified, because it is necessary to achieve a balance.

I played TC mods like K2 and SE for years. Then a few months ago I decided to try Vanilla again. It was an incredible fresh experience that was so enjoyable. The game felt streamlined, technologies felt more meaningful, and advancements rewarding. This showed me that complexity for complexity's sake doesn't always make the game fun.

[boeljoet]

Valves!!!!

when i compare fluids and belts there is 1 major diference.

in one of the Friday fact there was a mention of certain rules.
one of the rules was that there had to be an inserter between every belt and assembler.

my suggestion is to do the same for fluids with a valves (new item in game)
when a valve goes into a chemical plant or oil refinery it loads
if one goes outside of the chemical plant it unloads.
you click on the valve to let certain fluids trough.
you no longer need mirror recipes

[Svip]

I think some restrictions on [...] throughput are probably needed.

Also from the FFF:

In the new system, segments are a single unit, so fluid no longer propagates across a pipeline. Longer pipelines have higher throughput, but take longer to fully empty. The exact throughput numbers are subject to change based on further playtesting.

Emphasis mine.

There are already restrictions on throughput, and the exact numbers are subject to change. You cannot simply add more machines to a pipe segment, hoping it works, because since pipe segments now behave as a storage tank, they are limited by the amount that can be in that "storage tank" (I believe each pipe piece is 100 units), and the FFF also states, that machines can only draw according to how full the segment is. So if a segment is 50% full, a machine can only draw at 50% the rate.

In order to sustain X number of machines drawing N each tick, you need Y number of machines producing at least N each tick attached to the same pipe segment. I get the notion that people are predicting, that'll just have one single heavy oil pipe segment - for instance - across the entire map, but unless you have enough crude oil to sustain all the heavy oil production, you'll still need a way to divide your priorities, and that requires pumps, and that will limit your throughput. You can of course just attach them all to the same pipe segment, but run the risks that all your machines now draw at a lower rate.

[o6dukeleto]

There are already restrictions on throughput, and the exact numbers are subject to change. You cannot simply add more machines to a pipe segment, hoping it works, because since pipe segments now behave as a storage tank, they are limited by the amount that can be in that "storage tank" (I believe each pipe piece is 100 units), and the FFF also states, that machines can only draw according to how full the segment is. So if a segment is 50% full, a machine can only draw at 50% the rate.

Actually I think you are making my point for me, just adding storage tanks will allow for more "throughput" -- then just add more storage tanks to the segment until it supports enough throughput for whatever number of machines.

In order to sustain X number of machines drawing N each tick, you need Y number of machines producing at least N each tick attached to the same pipe segment. I get the notion that people are predicting, that'll just have one single heavy oil pipe segment - for instance - across the entire map, but unless you have enough crude oil to sustain all the heavy oil production, you'll still need a way to divide your priorities, and that requires pumps, and that will limit your throughput. You can of course just attach them all to the same pipe segment, but run the risks that all your machines now draw at a lower rate.

Actually, one segment will work with "proposed" FFF rules -- as long as you are producing enough resource for the consumers. If you are not, then you didn't need that many consumers in the first place.

Once again you can just add tanks to allow for throughput. At some point you will have enough storage that all the consumers can pull from the segment. It may take a while to fill up the segment, but if you are producing enough, the segment will slowly fill up and your consumers will get enough resources. In fact it becomes a natural "balancer."

Without more information, the proposed rules seem like a single segment per resource would be the natural way to balance that resource. Of course if you are in a "deficit" situation with said resource, i.e. you have over built consumers, then things become more interesting.

That said, hopefully the devs do provide more information on any additional restrictions, and why a single segment per fluid would not be ideal.

[syuilo]

I am a little disappointed with the 2.0 approach as I liked the ingenuity of installing more piping and using pumps to address the pipe flow issue.

It already exists in the mod, but could it be solved by implementing larger pipes, etc.?

Note: I'm not criticizing the development policy, I'm just wondering if there is any way to make it work better. If the development team thinks the 2.0 approach is reasonable, I will go along with it.

[XT-248]

I am disappointed that WUBE chose to fix the black box and the unpredictability of pipe junctions in this manner. To be clear, I do agree that the fluid simulation had to be fixed somehow. I think it is clear that many of us don't feel the same way WUBE does about simplifying the fluid simulation.

Part of the fun, at least for me, is working with limited tools and adding a one-way valve, which can be found in multiple mods, whose behavior was configurable (over-flow at 80% or above from input to the outlet, Only allows flow if the outlet is at 20% or below, etc.) to allow for greater fidelity in bend the fluid simulation into something that resembles a chaotic but still predictable system.

I derived some fun and pleasure from designing a non-trivial pipe network that semi-works the way I want it to, despite the flaws with the current fluid simulation in Factorio (well-document across dozens of threads/comments/etc.; no need to resurrect or rehash all of it here).


I have some questions about the capacity of pipes and tanks.

Would the pipe fill up to 100% first before filling the storage tank above 100 fluid units? Or is the entire pipe->tank->pipe network treated as a singular 'chest box' for fluid?

Is the only way to break up the network in vanilla to use four pumps around a storage tank and treat it as a junction (pumps use the fluid level via circuit logic in a storage tank to decide which outlet takes priority)? This may be a messy solution post-2.0 fluid simulation, but one that I can learn to live with even if it means having to leave a lot of room for fluid logistic junctions.


TD; LR: I expected more fun beyond a "mindlessly placing pipes" solution. I do hope that someday we will properly see a new fluid simulation, even if it takes a decade or two for hardware to catch up.

[Blaster]

Okay, you know what? I like the idea of simplifying this... and why?

UPS

I know some of you don't care about UPS... but listen, not everyone that plays this game is running on a Ryzen X3D or an Intel i9-13k or whatever.

One of the big dissappoints I've had over time is how servers always switch to Solar endgame. Why? UPS. No nuclear reactors ever. They drag down the UPS so hard. Same with oil.

Want to talk boring? Want to talk "taking the fun out fo puzzle solving?" How about plunking down the 50,000,000th self-building solar farm square to power your megabase with a UPS-friendly design.

How about "train wagons direct to oil refinery" because trying to fix the rat's nest of pipes to flow cleanly requires literal magic as it is much more frustrating to have to redo a whole pipe section because you misclicked and to preserve build order, now you got to delete it all and start again. Yeah, not fun for the average person.

I understand the want and need to have fully realistic fluids, but while we're here, we should talk about fully realistic belts. You know how belt items stop at the end of the belt? Well that is hugely unrealistic. Shamefully unrealistic. Items should dump on the ground at the end of a belt, pile up into a pile and start causing all sorts of issues jamming up the place. It's a realistic and fun logistics challenge to make sure your belts never backstuff, and everything you produce gets used before hitting the belt's end.

Also, belts and water pumps both run on magic. They should be powered to function.

And robots. Our drones just overlap eachother magically phasing through eachother without colliding and crashing... or having infinite low battery instead of running out entirely and crashing and burning.

etc. etc. etc.

There's mods if you want ultra realism.

Keep doing what you're doing, Factorio devs. Please just work on polishing and improving this new fluid handler (and work on it with the heatpipes too, they also need a UPS bpost!) for the game. It's good already.

[tolomea]

(over-flow at 80% or above from input to the outlet, Only allows flow if the outlet is at 20% or below, etc.)

That's a tank, a pump and a wire, and you could still make a mod like that and it would still be useful.

[Svip]

Would the pipe fill up to 100% first before filling the storage tank above 100 fluid units? Or is the entire pipe->tank->pipe network treated as a singular 'chest box' for fluid?

From the FFF, it's clear that each piece of a pipe segment (pipes and tanks) are filled a percentage, so for the pipe to be filled 80%, the tanks must also be filled 80%.

Is the only way to break up the network in vanilla to use four pumps around a storage tank and treat it as a junction (pumps use the fluid level via circuit logic in a storage tank to decide which outlet takes priority)?

If by "network" you mean pipe segments, then pumps and machines separate the pipe segments and is the only way to do so.

There are already restrictions on throughput, and the exact numbers are subject to change. You cannot simply add more machines to a pipe segment, hoping it works, because since pipe segments now behave as a storage tank, they are limited by the amount that can be in that "storage tank" (I believe each pipe piece is 100 units), and the FFF also states, that machines can only draw according to how full the segment is. So if a segment is 50% full, a machine can only draw at 50% the rate.

Actually I think you are making my point for me, just adding storage tanks will allow for more "throughput" -- then just add more storage tanks to the segment until it supports enough throughput for whatever number of machines.

If your output matches your input, and other factors aren't forcing a break in the segment, then you don't need storage tanks. Storage tanks appear only to be useful when produced is more than consumed, because as - mentioned above - adding a storage tank means you lower the rate a pipe segment is filled up, since a storage tank now count as a pipe piece just with a much higher capacity. So adding storage tanks will actually lower your throughput.

[o6dukeleto]

If your output matches your input, and other factors aren't forcing a break in the segment, then you don't need storage tanks. Storage tanks appear only to be useful when produced is more than consumed, because as - mentioned above - adding a storage tank means you lower the rate a pipe segment is filled up, since a storage tank now count as a pipe piece just with a much higher capacity. So adding storage tanks will actually lower your throughput.

Actually adding storage, doesn't lower your throughput, it just adds volume to the segment, since pipes and tanks are treated the same.

Obviously only add tanks if you need as they will slow down equalization of outputs/inputs.

Basically, it will balance itself because consumers will consume less until the levels of the segment equalize. In fact the example in the FFF shows 8 plants, I would like to know what would stop 80 or even 800 plants on that same pipe? (Other than not having enough producers.)

[XT-248]

(over-flow at 80% or above from input to the outlet, Only allows flow if the outlet is at 20% or below, etc.)

That's a tank, a pump and a wire, and you could still make a mod like that and it would still be useful.

Many mods already have implemented it. I do not need to write a mod to get it.

What is curious is why WUBE chose not to implement it with a new twist or play (spoilage/scrap recycling/asteroid collecting/lava-foundry, etc.) on a classic mod staple approach.

Would the pipe fill up to 100% first before filling the storage tank above 100 fluid units? Or is the entire pipe->tank->pipe network treated as a singular 'chest box' for fluid?

From the FFF, it's clear that each piece of a pipe segment (pipes and tanks) are filled a percentage, so for the pipe to be filled 80%, the tanks must also be filled 80%.

Let me put it another way.

Fill up the 'pipe segments' first before filling up the 'storage tank segment' would let the machines pull the full 'pipe segment' content without waiting forever for the 'storage tank segment' to fill up to pull a certain amount of "XX units per second."

IE: Oil Field pump jacks push oil to 'A pipe segment' that connects to 'B Storage Tank Segment,' which is connected to 'C pipe segment' and 'D pipe segment.'

Are A, B, C, and D treated as one singular fluid box, or can A be 100% full and be distributed 50%/50% into C and D with any unused leftovers above what pipe segments can hold backfilling in B tank?

If it is the latter, I can still have some fun as it is currently.

If the former? I don't know if I will find the experience meeting my expectations.

Is the only way to break up the network in vanilla to use four pumps around a storage tank and treat it as a junction (pumps use the fluid level via circuit logic in a storage tank to decide which outlet takes priority)?

If by "network" you mean pipe segments, then pumps and machines separate the pipe segments and is the only way to do so.

I hope we have more than pumps/machines to break up the network, if not at the release date, then eventually in a future patch.

[Panzerknacker]

Okay, you know what? I like the idea of simplifying this... and why?

UPS

I know some of you don't care about UPS... but listen, not everyone that plays this game is running on a Ryzen X3D or an Intel i9-13k or whatever.

One of the big dissappoints I've had over time is how servers always switch to Solar endgame. Why? UPS. No nuclear reactors ever. They drag down the UPS so hard. Same with oil.

Want to talk boring? Want to talk "taking the fun out fo puzzle solving?" How about plunking down the 50,000,000th self-building solar farm square to power your megabase with a UPS-friendly design.

Yeah but now nuclear reactors will be just as boring as solar farms. So what did we gain? Nothing.