Friday Facts #430 - Drowning in Fluids

[protocol_1903]

[...]What if the pipe limitation was a point-to-point limit instead of a square? Each pipe can only be (for example) 500 fluid boxes away from the furthest pipe before the network becomes invalid.[...]

This would be nice, but in the first fluid FFF, the devs mentioned that one of the motivations for the implementation of "pipe segments" was that they were trying to get away from the complexities (and associated calculation costs) involved in pathing through a pipe network with an arbitrary number of junctions. Even if those calculations only had to be done during construction/deconstruction, bots exist in Factorio, so...
That's why everyone's been suggesting the slightly less realistic but far more performant "just count the tiles in the pipe segment".

There would be zero changes to the per-tick calculations compared to the ones introduced in this FFF, only checking if a network is valid by a different metric. It would be very performant, only having hits when combining networks. The arbitrary number of junctions also doesn't mean anything, as the logic doesn't need pathing.

[Theikkru]

[...]The arbitrary number of junctions also doesn't mean anything, as the logic doesn't need pathing.

It would need pathing to figure out which branch of the network has the "furthest" fluid box. It's trivial if you have an "L" shaped pipe, but if it's "H" shaped, or more complex, you'd need to keep checking each leg to make sure the longest one hasn't changed due to where pipes are being added/removed. I'm sure there are ways you could make this more performant by saving more graph data in memory, but it's never going to be comparable to just keeping a running total of how many pipe tiles there are in the whole segment.

[CrazyAmphibian]

The #1 reason the devs are reworking fluids is performance, so any idea that doesn't slash performance costs to a level comparable with what they've got in these FFFs is probably not gonna fly.

could go for a node based system where network flows, latency, and storage are precalculated between nodes (building connection points) when you construct the network to avoid constant runtime overhead.

or split pipes into separate networks (condition being where 2 pipe components never touch directly or by proxy) and add multithreadding to them.

[Koub]

I find a little disturbing that these two contraptions are equivalent

sorry for the big zoomed out picture

2024-09-27 20_28_54-Window.jpg

I'd have preferred a number of pipe+tank entities instead of an area (or maybe I missed something crucial).

I think both contraptions might be equal in terms of throughput to adjacent fluid segments at equilibrium (when fill levels are stable, meaning all input flows equal all output flows), but they are definitely not equal in capacity.

When introducing a fluid flow from one side (NOT the equilibrium case, like when a new oil train arrives at a starved factory), the segment with a higher capacity ( the right in your case ) will fill more slowly and thus also introduce a bigger temporal delay in the whole chain of segments. I think this might be what was meant by the devs with "the fluids feel[ing] much more like fluids again".

Fair enough. I do realise my post was way too laconic. What I meant was that both contraptions would be valid as "able to carry fluid without additional pumps". It's that one pump can feed a pipe network no matter how big and convoluted as long as it fits in a 250x250 square. I was wondering if making it length or "pipe number" based wouldn't make more sense.
But that's really a minor rambling of mine, the 2.0 fluid system seems vastly superior in every aspect to the 1.1 one.

[Theikkru]

[...]could go for a node based system where network flows, latency, and storage are precalculated between nodes (building connection points) when you construct the network to avoid constant runtime overhead.

or split pipes into separate networks (condition being where 2 pipe components never touch directly or by proxy) and add multithreadding to them.

Both of those are just methods of coping with the calculation costs by shunting them to other, hopefully less loaded computing components (memory in the former case and other cores in the latter). Neither actually reduce the costs like the FFFs do.

[protocol_1903]

[...]The arbitrary number of junctions also doesn't mean anything, as the logic doesn't need pathing.

It would need pathing to figure out which branch of the network has the "furthest" fluid box. It's trivial if you have an "L" shaped pipe, but if it's "H" shaped, or more complex, you'd need to keep checking each leg to make sure the longest one hasn't changed due to where pipes are being added/removed. I'm sure there are ways you could make this more performant by saving more graph data in memory, but it's never going to be comparable to just keeping a running total of how many pipe tiles there are in the whole segment.

No, it doesn't. Each pipe keeps track of a value representing the distance to the furthest pipe. Whenever a new pipe is placed, it sets its own value to the lowest value around it + 1. Then each adjacent pipe updates using the same logic if it's distance + 1 is greater than the value , starting with the pipes of the highest value. No directions needed. You just represent the network as a series of nodes. You have to do a maximum of n/2 updates, where n is the number of pipes in the network

@Koub this is basically your idea

[CrazyAmphibian]

Both of those are just methods of coping with the calculation costs by shunting them to other, hopefully less loaded computing components (memory in the former case and other cores in the latter). Neither actually reduce the costs like the FFFs do.

yeah? the FFF proposal is a heavily simplified system, of course it'll have lower resource usage.
it's not like factorio uses *that* much memory normally as-is, and i don't imagine this will actually use that much more seeing as these calculations would need to be done anyways (just done every tick), and variables are never free.
fluid simmulations (even simple ones) are very resource intensive.

[Theikkru]

[...]Each pipe keeps track of a value representing the distance to the furthest pipe. Whenever a new pipe is placed, it sets its own value to the lowest value around it + 1. Then each adjacent pipe updates using the same logic if it's distance + 1 is greater than the value , starting with the pipes of the highest value. No directions needed. You just represent the network as a series of nodes. You have to do a maximum of n/2 updates, where n is the number of pipes in the network[...]

Something about that logic doesn't seem to work quite right, but regardless, my point stands; you have to query/update n/2 pipe elements for each segment change, which is O(n) complexity, whereas keeping a running total is just O(1).

[protocol_1903]

Something about that logic doesn't seem to work quite right, but regardless, my point stands; you have to query/update n/2 pipe elements for each segment change, which is O(n) complexity, whereas keeping a running total is just O(1).

The logic is sound, assuming a wrote it correctly. Regardless, I think it's worth trying. It's certainly more performant than 1.1 fluid logistics, and if it's too much then yeah reduce to a running total. But a running total would still be better than the strange square bounding box limitation.

[jamaicancastle]

I understood that it would be rarely noticeable in real gameplay, where your production is either higher than the demand, and you won't notice the "magic" since your pipes will be always full, or lower, in which case the throughput won't be your problem, your consumers will starve anyway.

It sounds like they even managed to solve that, so that the balance of input and output will be reflected in the level that the pipe settles at. Which, hopefully, should make it easier to get an intuitive sense of "this pipe has a lot of unused throughput, I can expand no problem" vs. "this pipe is nearly tapped out, I need to build up production right away". Which was not something the old system was very helpful about in my experience.

I might be misremembering, but did the fluid barrels get removed? I know fluid trains are the way to go for bulk distance transport, but conceptually, I love the idea of using barrels and transporting those, since it feels more "industrial" if that makes sense. Not sure if it's so compatible with current fluid throughput though

Nothing's been said about them, so I assume they're still there. They're less useful for long-range, high-throughput transfer than fluid wagons (which is intentional, and always was, but was sometimes undermined by all the weirdness about fluid transport), but they have their uses:
* They can be carried by bots, which otherwise can't carry fluids at all.
* They can easily be made to piggyback in cargo cars with other materials (e.g., you can have the same cargo car carry sulfuric acid barrels to and from a uranium mine with a few slots, and carry the resulting uranium ore back to base with the rest of its inventory).
* They're easier to split and meter out than piped fluids, in case you need to manage how much of your fluid supply is going to one particular place.
* They can be transported via belt arbitrarily long distances without needing pumps, which might be useful in some cases when supplying distant flamethrower turrets or other outposts (e.g., uranium mining again).

[Theikkru]

[...]yeah? the FFF proposal is a heavily simplified system, of course it'll have lower resource usage.[...]i don't imagine this will actually use that much more seeing as these calculations would need to be done anyways (just done every tick), and variables are never free.
fluid simmulations (even simple ones) are very resource intensive.

Those statements don't seem very congruent with each other, but at the very least the last one is definitely untrue. The model as described in the FFFs is at least a class of complexity lower than the one in 1.1, so the resource savings would not only be significant, but also scale with the calculation burden.

[SpecterNadir7]

The mechanic of just having it be artificially within a region of 250x250 (or 256x256) tiles does not seem right to me. It seems very artificial, and as I saw someone else post, it is way to binary of a system. I also would prefer the idea that the longer the pipeline.

Here is what I think would feel a lot more natural, rather than it being an arbitrary area limit (which as I also saw some people say, how is that area determined?) there is a "soft" cap on the flow rate of the pipes based on the number of pipes in a pipe segment. Where each segment is determined by the number of pipes that are directly connected and not divided by pumps and other machines.

Let us pick some arbitrary value of say 50 pipes in a segment is the soft cap, each pipe above that reduces the maximum throughput of the pipe by 1% per pipe, so the base max rate is 6,000 per second, so if a pipe was 100 pipes long it's maximum throughput is instead 3,000 per second (a -50%). This feels way more natural to me.

I also like the idea that we could treat pipes like conveyor belts, rather than them automatically connecting. I can't tell you how many times I've had to use WAY more space with my pipe setup so that I can keep separate fluid lines all one tile apart. I like building factories as compact as possible.

[Theikkru]

[...]The logic is sound, assuming a wrote it correctly.[...]

Assume a T-shaped pipe with 3 along the top and 2 on the vertical.
According to instructions:

[...]Each pipe keeps track of a value representing the distance to the furthest pipe.

3 2 3
- 2
- 3
Add a pipe:

Whenever a new pipe is placed, it sets its own value to the lowest value around it + 1.

3 2 3 (4=3+1)
- 2
- 3
Check adjacent:

Then each adjacent pipe updates using the same logic if it's distance + 1 is greater than the value , starting with the pipes of the highest value.[...]

3 2 (3+1≯4) 4
- 2
- 3
Update stopped due to condition untrue, but graph is now wrong:
3 2 3 4
- 2(!)
- 3(!)

[protocol_1903]

Assume a T-shaped pipe with 3 along the top and 2 on the vertical.
According to instructions:

[...]Each pipe keeps track of a value representing the distance to the furthest pipe.

3 2 3
- 2
- 3
...
Update stopped due to condition untrue, but graph is now wrong:
3 2 3 4
- 2(!)
- 3(!)

Hm, i hadn't thought about the other end. In that case, you would need to propogate down the system until it reaches a low point (in this case, the top of the T junction) then update from there.

So it would be

base layout

Code: Select all

3 2 3
  2
  3

add a new pipe

Code: Select all

3 2 3 4*
  2
  3

update adjacent

Code: Select all

3 2 3 4
  2
  3

update nodes adjacent to low point

Code: Select all

3 2 3 4
  3*
  3

propogate updates

Code: Select all

3 2 3 4
  3
  4*

[Theikkru]

[...]you would need to propogate down the system until it reaches a low point[...]

I'm just not seeing how you could generalize this across networks of arbitrary complexity without pretty much checking every last leg, and that won't look pretty, computationally.

[protocol_1903]

I'm just not seeing how you could generalize this across networks of arbitrary complexity without pretty much checking every last leg, and that won't look pretty, computationally.

There's most likely a way to design the system to work properly. It's probably already been done before elsewhere. My point was that, if done correctly, this system would be better than the bounding box or nodes in network limits

[Theikkru]

I'm just not seeing how you could generalize this across networks of arbitrary complexity without pretty much checking every last leg, and that won't look pretty, computationally.

There's most likely a way to design the system to work properly. It's probably already been done before elsewhere. My point was that, if done correctly, this system would be better than the bounding box or nodes in network limits

Uh, the bounding box is also O(1)...

[bitofabother]

Hm.

I'm a fan of the main idea for 2.0 fluids, but... this loses me a little bit. It's probably fine, especially if you make it 256 for the chunk alignment.

But the "falloff if the area gets bigger with some flow rate multiplier" idea is absolutely preferable to just "it breaks and lets you know where" -- I agree with the idea that being able to solve the problem myself is something I'd rather see than just being told "this is where you need a new pump". Something just breaking out of nowhere because you made it one tile longer doesn't feel very Factorio, on a deep level.

1.1 Fluids had a bunch of issues that I'm glad to be rid of, just hoping this gets another pass before release compared to something this arbitrary...

[protocol_1903]

There's most likely a way to design the system to work properly. It's probably already been done before elsewhere. My point was that, if done correctly, this system would be better than the bounding box or nodes in network limits

Uh, the bounding box is also O(1)...

Better as in makes more sense, less arbitrarily restrictive, and is more intuitive to the player.

Having to fold your blueprint in half because it's "too long" is not a good constraint.
Having to split your blueprint because you have too many pipes in a network isn't as bad, but isn't amazing. (your other suggestion)

[Ohz]

I fully support those changes.
Devs you're doing an amazing job