Friday Facts #274 - New fluid system 2

[ske]

Did anyone else notice that "iOS experience" is a plus on the MacOS job page?

iOS will probably replace OSX in the long term.

[roothorick]

I'd say that you shouldn't prevent placing things that already know what specific fluid they should be sending or receiving i.e. offshore pumps, steam engines/turbines, anything that takes a recipe. It's straightforward enough to say "this fluid system does not match this input/output so this machine won't connect" and the player already has a direct visual indication of the mismatch.

Pumps have a set input and output tile, but no predetermined liquid, so placing it in a way that connects fluid systems carrying different fluids is an obvious error.

Pipes, well if you don't want to expand on how pipes work, just blocking placement makes sense. But you're missing an opportunity to allow side-by-side fluid networks. You could, say, have a pipe connect to only one fluid system, and when first placing it the player could press R to switch which one it will connect to, visually indicated by what nearby pipes it's connecting to.

Good luck going against Nier Automata in Most fun with a Machine.

N:A has substantially fewer active Steam players than Factorio (and everything else in the category, for that matter). I'm worried about Rocket League.

[PaszaVonPomiot]

Thank you TheYeast and other contributors. Fantastic job!

[Sir3n]

The FFF says "pipes to ground won’t help you with throughput anymore"

Correct me if i'm wrong, but there is still argument for using pipe-to-ground as much as possible from a megabase performance perspective right?
I'm not sure about the exact number in vanilla anymore, but one pipe-to-ground is worth like 9 or so pipes in cpu time.

I wonder if it would be possible to apply the belt optimization to pipes so that a single series of straight pipes (no intersections) would be considered as a single fluid box.

For one long series of pipes made up of L segments, it can be considered as a single fluidbox L times the size of a regular pipe fluidbox. We can then consider the height level of the fluid in the pipe to be (total fluid / L) for the purposes of calculating how much of it goes into machines or intersections.

This would lose the wave-like behaviour of fluids in pipe though. But I think it's worth it because the wave like behaviour doesn't really matter in a long pipe; what we (as players) really care about is how much throughput we get out of it at the exit for what we put into the pipe.

Furthermore, the realistic wave algorithm wastes unnecessary resources each frame for something that isn't immediately obvious to players: the pipe animation doesn't change based on how much fluid there is in it. It either has fluid, or doesn't. Without enabling the fluid levels overlay, nothing really changes from the player's perspective.
Unlike belts where a player can actually step over things and press F to pick things up, once fluids are in a pipe, it's a locked system, we can't directly interact with the fluid by say, scooping up some of it.

In my opinion, having a less realistic simulation for fluids with instant response, instead of a more realistic simulation that uses up cpu time for something that players can't directly interact with, is worth it. Like I said, I can't press F on the pipe to pick up fluids. So as far as I'm concerned, how the game treats fluids in between input and output doesn't matter as long as it does it in a predictable way.

Don't get me wrong, I like the algorithm that TheYeast came up with, it's just that I'm concerned about performance in large bases. It is annoying from a gamer's perspective to have to think about, okay, I have to use pipe to ground constantly because that's the way to build the biggest base possible without slowing down everything to a crawl. Plus I like the look of regular pipes. I think it's a shame to have to pipe to ground everywhere.

[PunPun]

Don't get me wrong, I like the algorithm that TheYeast came up with, it's just that I'm concerned about performance in large bases. It is annoying from a gamer's perspective to have to think about, okay, I have to use pipe to ground constantly because that's the way to build the biggest base possible without slowing down everything to a crawl. Plus I like the look of regular pipes. I think it's a shame to have to pipe to ground everywhere.

As far as I understand the algorithm it is slower than the one in 0.16 but because it is an order independent algorithm it is actually possible to multithread it. That along with other optimizations you should get better performance than before even tough the algorithm is slower so there is no downside to implementing this.

[Lubricus]

The FFF says "pipes to ground won’t help you with throughput anymore"

Correct me if i'm wrong, but there is still argument for using pipe-to-ground as much as possible from a megabase performance perspective right?
I'm not sure about the exact number in vanilla anymore, but one pipe-to-ground is worth like 9 or so pipes in cpu time.

I wonder if it would be possible to apply the belt optimization to pipes so that a single series of straight pipes (no intersections) would be considered as a single fluid box.

For one long series of pipes made up of L segments, it can be considered as a single fluidbox L times the size of a regular pipe fluidbox. We can then consider the height level of the fluid in the pipe to be (total fluid / L) for the purposes of calculating how much of it goes into machines or intersections.

This would lose the wave-like behaviour of fluids in pipe though. But I think it's worth it because the wave like behaviour doesn't really matter in a long pipe; what we (as players) really care about is how much throughput we get out of it at the exit for what we put into the pipe.

Furthermore, the realistic wave algorithm wastes unnecessary resources each frame for something that isn't immediately obvious to players: the pipe animation doesn't change based on how much fluid there is in it. It either has fluid, or doesn't. Without enabling the fluid levels overlay, nothing really changes from the player's perspective.
Unlike belts where a player can actually step over things and press F to pick things up, once fluids are in a pipe, it's a locked system, we can't directly interact with the fluid by say, scooping up some of it.

In my opinion, having a less realistic simulation for fluids with instant response, instead of a more realistic simulation that uses up cpu time for something that players can't directly interact with, is worth it. Like I said, I can't press F on the pipe to pick up fluids. So as far as I'm concerned, how the game treats fluids in between input and output doesn't matter as long as it does it in a predictable way.

Don't get me wrong, I like the algorithm that TheYeast came up with, it's just that I'm concerned about performance in large bases. It is annoying from a gamer's perspective to have to think about, okay, I have to use pipe to ground constantly because that's the way to build the biggest base possible without slowing down everything to a crawl. Plus I like the look of regular pipes. I think it's a shame to have to pipe to ground everywhere.

There are some animation in the pipe windows, that is almost invisible in normal play... So they don't really matter
I think something like this proposal would be even better for UPS, gameplay and more realistic than the Yeast model.
viewtopic.php?f=38&t=63736&start=120#p390508
Real pipes is full with fluid and the pressure is traveling fast, so when they are full they work more like teleportation than factorio pipes.

It could be combined with some sort of throughput restrictions calculated from how the pipe network looks like.
Pipe to ground is also better so you can walk in the factory and so the pipes don't auto connect, so I will still use as many pipes to ground as possible even without throughput and the UPS benefits.

[Sander_Bouwhuis]

I presume this has been asked many times, but why do pipes auto-connect to each other? It's the nr 1 reason for me to hate Hate HATE pipes.

Why can't I just put 2 pipes next to each other? To connect them we could either have a popup menu to set it, or simply have T-junction pipes and cross-junction pipes (not sure about the correct names/terms for those forms/types).

[darkfrei]

I presume this has been asked many times, but why do pipes auto-connect to each other? It's the nr 1 reason for me to hate Hate HATE pipes.

Why can't I just put 2 pipes next to each other? To connect them we could either have a popup menu to set it, or simply have T-junction pipes and cross-junction pipes (not sure about the correct names/terms for those forms/types).

But you have mod for it https://mods.factorio.com/mod/PickerPipeTools

What if you place empty pipes, must they connect together or not?

[<NO_NAME>]

I don't know what you guys screwed up, but I'm noticing a distinct lack of complaints in the replies so it must be something HUGE.

Just wait until the I-want-fluids-to-be-as-simple-as-electricity proponents get here...

I want electricity to be as complex as fluids.

[Dominik]

Can you give us a bit more info on how placing pipes will work with the new anti-mixing logic? Can you confirm or deny that this will allow running parallel pipes without them accidentally intersecting? I could see this being a problem potentially (esp in blueprints) if pipe connection order is dependent on placement order though...

No, you can't put the pipes next to each other. It simply does not allow any action that would allow fluid mixing.

[Dominik]

What about the use of a warning that says the liquid cannot be drained?
Something like that
Screenshot_1.jpg

This would work too and it would make some construction easier, but also again allow flooding pipes with a wrong fluid in a setup that you have already prepared. I still think not allowing it at all is a bit better. We will se though.

[Dominik]

Can you give us a bit more info on how placing pipes will work with the new anti-mixing logic? Can you confirm or deny that this will allow running parallel pipes without them accidentally intersecting? I could see this being a problem potentially (esp in blueprints) if pipe connection order is dependent on placement order though...

We will still need o test the blueprints, I don't think we can figure out everything that can happen. AFAIK it should potentially only matter with some conflicting pipes to ground, a rather rare case that the player would still be warded about in the end.

[Dominik]

Looks like Dominik listened when I mentioned about my mods using the viscocity system and not only included the abillity to do it in the new system, but also used it in the base game.

Wonderful

I am sure I read the request a few times, probably yours too It was a matter of how difficult it would be and how slow. Since the variables are natively in the algorithm, it was easy. Adding any variable cost performance though, which was the main issue and truth is some people are still not sure it is worth it.

[Dominik]

One of the setups I had come to imagine was a workshop machine (AM2 or similar) with three fluid interfaces in different directions, and one item interface direction in the fourth direction. If the recipe is not set, the fluid interface will be visible in absolutely no manner, and it becomes frustrating guesswork to figure out how to rotate the machine before setting the recipe. I can also see similar frustration if someone plunks down a chemical plant or refinery and the pipes leading up to or from it are backwards. Not being able to set the recipe would make it far harder to troubleshoot such a problem than it currently is - at least for inputs. A new player might even mistake this behaviour for a game breaking glitch and throw Factorio out in frustration.

You would not be able to rotate it back. Any such action attempt will give you a warning about the fluid mixing, so anyone should be able to figure it out. Even if it ends up confusing for somebody, it is still less confusing then the result would be previously.

With this behaviour, the only way I see solving the problem I left in from my previous post is by breaking all the pipes adjacent to the machine to be able to rotate it or diagnose issues, which is far from ideal. I'm actually quite certain it would behave better than you describe in the current game where an AM2+ only consumes fluid, but I haven't set anything like this up in anger (so to speak) because mixing fluid networks is that much of a disaster that I'm quite paranoid about it. I think I can slap together a test setup when I have more time to play; I certainly wouldn't take the risk in any "real" games.

I understand what you mean, but how often does that happen? And if it does, mostly you already remember what the inputs are our you can have a look at another crafter. It is not ideal though, your are right, if that really proves to be an issue we would deal with it then.

[Dominik]

I am sure this new system is better than the old system as pretty much anything would be better than the old system. But i also fear it's also going to be annoying.

Here's a few concerns of mine currently, which i hope can be fixed by the end user through modding at least.

• Flow rate - There comes a point where one needs higher flow rates than what is available through a single piece of vanilla pipe. That is, no matter how many pumps one uses, even a single pipe will be a bottleneck.
  In the old system one had to pretty much destroy viscosity of the fluid for it to flow. This because the old system was only dependant on pressure, not actual cross section (volume) of pipe. And you couldn't increase pressure in a sensible manner... In real life, oil trapped underground and suddenly released can shoot out at amazingly high pressures and flow rates. Despite the "pipe" it flows through being rather small.
  
  Machines have massive limitations with pipes in the number of connections they have and where those connections are placed. Limitations that rarely exist for items.
  
  In short, i really hope we can increase pressure and volume of pipe in such a way that getting fluids around hungry machines won't be an issue with this system.
   
• Time to travel - You mention that fluid amounts < 0.05 will clear out automatically. Well, how fast will this happen? Is it faster than just removing all pipes manually and re-placing them? Again, in real life, pipes are cleaned by scrubbing and washing out the old gunk and takes forever. But factorio isn't real life, one doesn't want to wait.
  In Factorio, it usually is enough to place a pump at some end going into a single fluid box to clear a pipe and the length of pipe it clears doesn't really matter as the speed in doing this is damn high anyways. And in the off chance that fluids don't flow (< 0.05 amounts whatever) you simply rotate the pump and try again until all moves in one go due to the flow bonus.
  
  I find myself using the same pipes for different fluids currently depending on what i need at the time. Hence i empty often.
  
  But if fluids emptying is a PITA in the new system i suspect people (me included) are going to fix the issue by modding and if not possible, go back to 0.16 and stay there until it's fixed.
   
• Upgrading - You mention that using underground pipes won't help now. Well, that has been the only way to increase flow rate currently along with pumps every so often (in my case, pump->pipe->pump->pipe->pump) and this concerns me.
  Again, in a real life situation you can always use a bigger pipe. In Factorio however, pipes are one tile wide and that's it. This because machines fluid ports are one tile wide.
  
  So again, how would one be able to perform upgrades to the fluid network once they find that one single pipe is the bottleneck for the whole thing?
  It's not as simple as just using two pipes side by side. Getting the fluid in and out of machines is still a massive problem in megafactories because of how pipes connect, how they don't exist neatly side by side (different fluids) and how they are "one size fits all".
  
  I want to be able to use different pipe dimensions and different pipe materials for different applications. If i want extremely high flows i would go with a bigger cross section pipe made with an extremely strong material to be able to pressurize the system to high levels to increase flow rate even for highly viscous fluids.

But it's not all concerns of course, i am really glad that splitting flows has been dealt with now. No more intermediate storage tanks with pumps controlled to only pump when all extracting pumps have enough fluid for one tick of operation.
And i am sure the new system will allow for more options now.

I just hope it's not going to completely destroy the balance of the game just because someone wanted fluids to move "realistically" through pipes.

While finalizing it, ensure that extreme situations can still be addressed in modding and all is good.
Oh and it would be really nice if we could have some flags on fluids too. Such as "corrosive", which would require the use of a pipe material that can take corrosive fluids.
Or "gas", which requires stronger pipes to ensure gasses don't escape through the connections.
And even "max pressure" so if fluids go over that pressure in the system, the fluids would "escape" (not necessarily leak on the ground, that depends on mods that do such things) which would require the use of a stronger pipe.

EDIT: Mind you, i am really tired at the moment so i read the blog post in a dozy state and might not have understood the new system at all.

Flow rate - the issue is a part of the game. If one pipe is not enough, well, figure it out

Fluid clearing is reasonably fast. Would be faster with higher threshold but I also don't want to make fluid disappear when it should not. Anyway, I can't imagine how you use one pipe for different fluids with the new system against mixing.

Feel free to mod in as many different pipes and fluids as you like.

The other properties - unfortunately no. Any variable increases the memory footprint which is significant for performance. We eliminate all that does not make significant contribution to the game.

[Dominik]

How will this work with non-pipe fluidboxes, i.e. turbines or steam engines? For example, how will low temperature steam mix with high temperature one? Or they just won't? It's technically the same fluid and there were nuclear builds with adding additional steam engines at the end to utilize leftover steam, soooo.

This remains unchanged, temperature is an independent variable. Fact is I wanted to remove temperature as it is a lot of data for little gain but apparently some mods make a good use of it so it stays.

[Dominik]

...
Pipes, well if you don't want to expand on how pipes work, just blocking placement makes sense. But you're missing an opportunity to allow side-by-side fluid networks. You could, say, have a pipe connect to only one fluid system, and when first placing it the player could press R to switch which one it will connect to, visually indicated by what nearby pipes it's connecting to.
...

It would indeed be handy and we discuss it every couple months. Issue here is simply our limited time. The current system make the construction more difficult but that is not necessarily bad.

[Dominik]

The FFF says "pipes to ground won’t help you with throughput anymore"

Correct me if i'm wrong, but there is still argument for using pipe-to-ground as much as possible from a megabase performance perspective right?
I'm not sure about the exact number in vanilla anymore, but one pipe-to-ground is worth like 9 or so pipes in cpu time.

I wonder if it would be possible to apply the belt optimization to pipes so that a single series of straight pipes (no intersections) would be considered as a single fluid box.

For one long series of pipes made up of L segments, it can be considered as a single fluidbox L times the size of a regular pipe fluidbox. We can then consider the height level of the fluid in the pipe to be (total fluid / L) for the purposes of calculating how much of it goes into machines or intersections.

This would lose the wave-like behaviour of fluids in pipe though. But I think it's worth it because the wave like behaviour doesn't really matter in a long pipe; what we (as players) really care about is how much throughput we get out of it at the exit for what we put into the pipe.

Furthermore, the realistic wave algorithm wastes unnecessary resources each frame for something that isn't immediately obvious to players: the pipe animation doesn't change based on how much fluid there is in it. It either has fluid, or doesn't. Without enabling the fluid levels overlay, nothing really changes from the player's perspective.
Unlike belts where a player can actually step over things and press F to pick things up, once fluids are in a pipe, it's a locked system, we can't directly interact with the fluid by say, scooping up some of it.

In my opinion, having a less realistic simulation for fluids with instant response, instead of a more realistic simulation that uses up cpu time for something that players can't directly interact with, is worth it. Like I said, I can't press F on the pipe to pick up fluids. So as far as I'm concerned, how the game treats fluids in between input and output doesn't matter as long as it does it in a predictable way.

Don't get me wrong, I like the algorithm that TheYeast came up with, it's just that I'm concerned about performance in large bases. It is annoying from a gamer's perspective to have to think about, okay, I have to use pipe to ground constantly because that's the way to build the biggest base possible without slowing down everything to a crawl. Plus I like the look of regular pipes. I think it's a shame to have to pipe to ground everywhere.

Yes, the pipe to ground saves some cpu time, but I don't think it is worth any effort. We could still merge the straight pipes (it was discussed in many places before) but it would only help small portion of the pipes and it is currently just a very small portion of the update. So yes, you can still use underground pipes to squeeze out few extra cpu ticks, but I am sure you will always be able to get them elsewhere better with the same effort.

When it comes to the algo itself - sorry, we are already past that discussion, see earlier posts and discussions for the reasoning.

[Nightinggale]

I don't know what you guys screwed up, but I'm noticing a distinct lack of complaints in the replies so it must be something HUGE.

Just wait until the I-want-fluids-to-be-as-simple-as-electricity proponents get here...

I want electricity to be as complex as fluids.

Summary of long post: It doesn't make sense for Factorio to model electricity in any other way than the current model.

Real life electricity can actually be viewed like water in pipes.

Pressure is voltage. Flow is current. Resistance is narrow pipes or other flow limiting designs. Generators are pumps.

You can also view it as canals. Voltage is height of the water surface while the others remain the same.

Nobody actually use this for anything useful. The goal of this comparison (which I didn't make up. It's actually used) is to provide a model, which is easier to visualize for people, hence making electricity easier to understand. Explaining how electrons behaves in solid metal can be rather abstract. It should be noted that the comparison works best for DC as AC adds complexity, which can't be compared to flowing water.

The problem with modeling power in Factorio is speed. Not the CPU speed, but the speed of electricity. The electrons flows at 300 Mm/s* (mega meter per second). With 60 ticks/second, the speed is 300/60 = 5 Mm/tick. That's 5000 km/tick. Even if electricity doesn't flow instantly, it certainly is instant compared to the tick speed. Remember this is real time clock. The clock in Factorio is faster, making the speed per tick even higher. I do not see even megabases having even a remote chance of being large enough to make it make sense to calculate on flow of electricity.

* The actual speed is the speed of light, which is 299.792.458 m/s. Depending on the metal, it can lose a little bit of speed, but it's still very close to that number. 300.000.000 is a common approximation as it's often close enough and it makes the calculations much easier, like allowing all calculations in this post to be done without a calculator. It doesn't really matter if the answer is 4k or 5k km/tick. Both are way too fast to justify Factorio not assuming the speed to be instant.

And yes I do realize I answered seriously to what is most likely a joke, but I assume some people might actually be interested in the answer.

[Nightinggale]

what about fluid wagons? How do pumps know in advance what kind of liquid trains will have when they stop?

It will be a huge issue if fluid wagons will be locked to a certain type of liquid. Currently there are a few mods to help manage a huge fleet of trains and assign schedules as needed. We still need to be able to park unused trains and then assign schedules as needed. This means we need to be able to request something like "train with 1 engine and 3 empty liquid wagons". Being forced to make a train for each type really counters the whole idea with train automation.

Not being able to pump liquid into a wagon before the previous type of liquid is pumped out would however be a good idea. It would make it possible to have a stop, which both adds and removes liquid, effectively replacing the cargo.

Also pumps should be able to filter liquids, meaning you can have 3 pumps next to a wagon and then only the one with the right fluid type will turn on. This will be in the Factorio spirit as it gives the player more freedom to automate a train layout and/or design minimalistic designs as it prevents the need for one station for each fluid type.

The forum users produced many ideas on how the system could work. About third of them was a fluid teleportation, many where known but many were entirely new and interesting.

This gave me an idea. Say you build a pipeline from a far away oil field to your refinery. You will now need multiple pumps. This will create multiple sections of pipes divided by non-pipe entities, which is great for multi core calculations, but we can do even better.

This will create a number of pipe systems, which goes from pump to pump with no branching or anything on the pipe. It's essentially just a strait pipe even if it bends. It will fill the pipe and it will pump out of the pipe. At some point it will reach a stable situation where flow and fluid content won't change between ticks.

This got me thinking. If a pipe is really just a link between two entities, how about adding some code to detect the stable condition and if it's stable, just teleport fluid between the entities and leave the pipes with a constant level and flow. If the level at the entities goes out of the stable level (read not the same as at the start of last tick), start to calculate each pipe again. At some point it will hopefully go back to a stable condition in which case it can go back to teleport mode.

I intentionally wrote entities, not pumps because I can also imagine some other setups where this would make sense. Imagine one entity producing a fluid and it goes directly to a consumer of said fluid. The consumer also requires something solid delivered from a belt. This will work in burst mode for the pipe system and the pipe system would just stall and do nothing if the belt stops delivering for a while.

It could be worth considering how to handle forks, like one pump delivering lubrication to 3 assemblers. There is no need to spend a long time on the pipe system if the assemblers are stalled or are switched to producing something, which doesn't require any liquid input.