[ANSWERED] Pipe to pipe transfers, are they hardcoded?

Place to get help with not working mods / modding interface.
Post Reply
Cadde
Fast Inserter
Fast Inserter
Posts: 149
Joined: Tue Oct 02, 2018 5:44 pm
Contact:

[ANSWERED] Pipe to pipe transfers, are they hardcoded?

Post by Cadde »

See this below post for answer: viewtopic.php?p=389013#p389013

Note: This relates to modding but is actually a game engine question...

I am running into bottlenecks with fluids as always. And so i wanted to fix it! But it seems i've run into a brick wall...

Some history first.
I altered the pump to have a base_level of 10 rather than zero, this allowed the pump to actually push 200 units of fluid/t rather than 138/t as it does vanilla. I can't say why as i don't know the source code for the games fluid simulation. Either way, it works wonders when you move fluid from storage to machine.
I also opted to make pipes carry more fluid, 1000 units in fact because of splitting. So there's enough fluid in a pipe to fill a pump in one tick to actually allow the pump to operate at max speed.

My current problem.
Whenever there's a pipe->pipe connection, the pipes will equalize their fluids at some predefined rate. At first i thought it was based on a percentage of pipe volume but it isn't.
That is, i assumed that a 100 unit pipe would move 97.5% of it's contents to a neighbor pipe, or quite simply 97.5 units of fluid.
And so i assumed that a 1,000 unit pipe would move 975 units of fluid to another pipe in the same tick. But it doesn't do that! No matter how much capacity the pipes have, they always move exactly 97.5 units of fluid between them WHEN the pipe that is being moved from is FULL. And possibly less than that if the neighboring pipe already has some fluid in it.

So my question is, is this some hard coded bit? An assumption that pipes will always have 100 units of fluid in them?
Because i am looking to increase this inter pipe transfer rate via modding and i can't do it.
Last edited by Cadde on Thu Dec 06, 2018 4:17 pm, edited 1 time in total.

Cadde
Fast Inserter
Fast Inserter
Posts: 149
Joined: Tue Oct 02, 2018 5:44 pm
Contact:

Re: [DEVS please] Pipe to pipe transfers, are they hardcoded?

Post by Cadde »

On a side note, isn't the above 97.5 related to the "infinite sloshing" that are present in pipes that have less base_area than one?
And it also explains why pipes with less than one base_area move liquids faster between them as they practically start acting like pumps at that stage?

User avatar
bobingabout
Smart Inserter
Smart Inserter
Posts: 7351
Joined: Fri May 09, 2014 1:01 pm
Contact:

Re: [DEVS please] Pipe to pipe transfers, are they hardcoded?

Post by bobingabout »

doesn't "Hardcoded" simply mean that it's part of the base game mechanics, and can't be edited by a mod?

if so, then all fluid flow is hardcoded.
I'm not a pipe expert, but I'm fairly sure there is a percentile element to it, as fluids will flow faster from 1 tank to another than they do from 1 pipe to another.
what I do know though is that higher storage capacity means it will take longer to fully transfer from one fluid box to another, so it's not just a linear percentile multiplier, which is why fluids actually flow faster with smaller pipes.
The flow mechanic is designed to be maximum when you have a base area of 1 though, so as mentioned above, having a base area less than one transfers more from one pipe to the next than it should, causing "Unstable simulation" state, where the fluid sloshes the entire length over and over again.

Just keep in mind that the entire fluid system is currently being re-written for 0.17.
Creator of Bob's mods. Expanding your gameplay since version 0.9.8.
I also have a Patreon.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 5206
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: [DEVS please] Pipe to pipe transfers, are they hardcoded?

Post by eradicator »

It's been quite a while since i experimented with this, but i remember i succeeded at making fluids flow really fast (i.e. like 100 times faster). I don't remember the details, but the fluid itself has some related values:

Code: Select all

pressure_to_speed_ratio = 0.4,
flow_to_energy_ratio = 0.59,
All base game fluids have the exact same values, but you can mod them. That said any modding you do on this now is probably going to be completely worthless when 0.17 comes with the new system ^^.
Author of: Belt Planner, Hand Crank Generator, Screenshot Maker, /sudo and more.
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

Cadde
Fast Inserter
Fast Inserter
Posts: 149
Joined: Tue Oct 02, 2018 5:44 pm
Contact:

Re: [DEVS please] Pipe to pipe transfers, are they hardcoded?

Post by Cadde »

bobingabout wrote:
Thu Dec 06, 2018 9:52 am
doesn't "Hardcoded" simply mean that it's part of the base game mechanics, and can't be edited by a mod?

if so, then all fluid flow is hardcoded.
No i meant that there would be a part of the formula that was hard coded in such a way that increasing fluid flow between two zero level pressure entities wouldn't be possible.
This is clearly not the case now that you guys made me aware of the fact the parameters are hidden inside each fluid.
bobingabout wrote:
Thu Dec 06, 2018 9:52 am
I'm not a pipe expert, but I'm fairly sure there is a percentile element to it, as fluids will flow faster from 1 tank to another than they do from 1 pipe to another.
Yes, when i learned of "pressure_to_speed_ratio" here it all fell into place for me and THIS thread all of a sudden made sense.

Neutral (not pumped, same base level) fluid flow isn't based on amount contained in neighboring entities. It's based on their pressures.

A pipe that has 100 fluid capacity and 50 fluid in it has 50% pressure.
A storage tank that has 25,000 fluid capacity and 5,000 fluid in it has 20% pressure.

For all intents and purposes, storage tanks and pipes are identical in this respect. It's the pressure differences that matter. (as in, how full are they in percent)
It shouldn't matter if the adjoining entities are tanks or pipes. The rate of transfer is the same, the difference is how fast they reach a near equal pressure level.

First, let's look at two pipes next to each other. And ignoring all inertia calculations for now.

Code: Select all

Pipe capacity = 100
pressure_to_speed_ratio = 0.4

pipe A = 100 fluid
pipe B = 0 fluid

Tick 1: (100-0)*0.4 = 40 fluid moves from A to B
Tick 2: (60-40)*0.4 = 8 fluid moves from A to B
Tick 3 (52-48)*0.4 = 1.6 fluid moves from A to B
Now, let's look at two tanks next to each other. Still ignoring inertia.

Code: Select all

Tank capacity = 25,000
pressure_to_speed_ratio = 0.4

tank A = 25,000 fluid
tank B = 0 fluid

Tick 1: (100-0)*0.4 = 40 fluid moves from A to B
Tick 2: (99.84-0.16)*0.4 = 39.872 fluid moves from A to B
Tick 3 (~99-~1)*0.4 = ~39 fluid moves from A to B
Tick 4 (~99-~1)*0.4 = ~39 fluid moves from A to B
Tick 5 (~99-~1)*0.4 = ~39 fluid moves from A to B
...
So as you can see, tank to tank will move more fluid per second than pipe to pipe hence why i assumed higher capacity = higher neutral transfer rate.
But what really happens is that the pressure differences just stay high for longer. So more fluid is moved because of that.

Note, without your answers, i would "never" have figured this out. (as in, i would have given up on it until 0.17 hit)
bobingabout wrote:
Thu Dec 06, 2018 9:52 am
Just keep in mind that the entire fluid system is currently being re-written for 0.17.
eradicator wrote:
Thu Dec 06, 2018 2:03 pm
It's been quite a while since i experimented with this, but i remember i succeeded at making fluids flow really fast (i.e. like 100 times faster). I don't remember the details, but the fluid itself has some related values:

Code: Select all

pressure_to_speed_ratio = 0.4,
flow_to_energy_ratio = 0.59,
All base game fluids have the exact same values, but you can mod them. That said any modding you do on this now is probably going to be completely worthless when 0.17 comes with the new system ^^.
I am impatient. Also, not sure the fluid mechanics themselves are changing in 0.17, only the optimizations. That's what i take away from the FFF.

Also, modding it is practically a one liner.

mod/data-final-fixes.lua:

Code: Select all

for k, v in pairs(data.raw.fluid) do
   data.raw.fluid[k].pressure_to_speed_ratio = 0.6
end
See above formulas for their effects. Again reminding everyone that inertia plays up to a part in it.
So for the FINAL calculations then... Inertia.

0.59 or 59% of the fluid transfer is down to inertia. As in, in tick 2, 59% of the last fluid movement will be added to the flow of tick 2.

40 + 40*0.59 = 63.6 fluid moves from A to B.

BUT, inertia is limited to 10% of the targets (B) capacity. 10% of 100 is 10 so the actual transfer between two pipes, assuming pipe A is still full and pipe B is empty in tick 2 would be 50.

With my 1000 capacity pipes, the math comes down to...

40 + min(40*0.59, 1000*0.1) = 40 + 23.6 = 63.6 as 63.6 is less than 100.

If one iterates that last calculation, one gets...

Code: Select all

63.6
77.524
85.73916
90.5861044
93.445801596
95.13302294164
96.1284835355676
96.7158052859849
97.0623251187311
97.2667718200513
97.3873953738303
97.4585632705599
97.5005523296303
97.5253258744819
97.5399422659443
97.5485659369072
97.5536539027752
97.5566558026374
97.5584269235561
97.5594718848981
97.5600884120899
97.560452163133
97.5606667762485
97.5607933979866
97.5608681048121
97.5609121818391
97.5609381872851
97.5609535304982
97.5609625829939
97.5609679239664
97.5609710751402
97.5609729343327
97.5609740312563
97.5609746784412
97.5609750602803
97.5609752855654
97.5609754184836
97.5609754969053
97.5609755431741
97.5609755704727
97.5609755865789
97.5609755960815
97.5609756016881
97.560975604996
97.5609756069476
97.5609756080991
97.5609756087784
97.5609756091793
97.5609756094158
97.5609756095553
97.5609756096376
97.5609756096862
97.5609756097149
97.5609756097318
97.5609756097417
97.5609756097476
97.5609756097511
97.5609756097531
97.5609756097544
97.5609756097551
97.5609756097555
97.5609756097557
97.5609756097559
97.560975609756
97.560975609756
97.560975609756
97.5609756097561
<LAST VALUE REPEATS FOREVER>
C# code for above output

Code: Select all

            double transfered = 40;
            
            for (var i = 0; i < 1000; i++) {
                transfered = 40 + Math.Min(transfered * 0.59d, 100);
                Console.WriteLine(transfered);
            }
            Console.ReadKey();
And that's where my 97.5 comes from.

In summary (AKA TL;DR)

To simplify EVERYTHING, there's two parts to the simulation.

The main part is "pressure_to_speed_ratio", it defines how much fluid can move per tick between two fluid boxes per liquid.
A value of 0.4 means at MOST 40 units can move between two fluid boxes per tick.
The more equal in level the two fluid boxes are, the less is moved between them.

The second part is "flow_to_energy_ratio", it defines how much MORE fluid is moved if fluid is flowing regularly from one fluid box to another.
A value of 0.59 means that for every tick, another 243% of the main part can move per tick. As long as 10% of the target fluid box capacity is still higher than the resulting value.

For vanilla tank to tank transfers, the maximum flow rate is 97.5 fluid/t
For vanilla pipe to pipe transfers, the maximum flow rate is 50.0 fluid/t, this because the pipes capacity is too small for the maximum inertia bonus.

If one alters "pressure_to_speed_ratio" on a particular fluid they change the base speed of said fluid.
And if one alters "flow_to_energy_ratio" they are changing the max bonus speed gained from moving said fluid continually.


THANK YOU GUYS!

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 5206
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: [DEVS please] Pipe to pipe transfers, are they hardcoded?

Post by eradicator »

Cadde wrote:
Thu Dec 06, 2018 4:16 pm
I am impatient. Also, not sure the fluid mechanics themselves are changing in 0.17, only the optimizations. That's what i take away from the FFF.
There was speculation in the FFF if the fluid mechanics do change or not. Dominik sounded vague enough about the subject that i'd guess they do.
Cadde wrote:
Thu Dec 06, 2018 4:16 pm
Also, modding it is practically a one liner.

mod/data-final-fixes.lua:

Code: Select all

for k, v in pairs(data.raw.fluid) do
   data.raw.fluid[k].pressure_to_speed_ratio = 0.6
end
That's also incompatible with any mod that adds their own "special" fluids. I.e. because someone else didn't like the idea that all fluids have the same viscosity ^^. If you do it in data-updates there might be a chance for them to fix it i guess. Bleh. Compatibility is difficult.
Author of: Belt Planner, Hand Crank Generator, Screenshot Maker, /sudo and more.
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

Cadde
Fast Inserter
Fast Inserter
Posts: 149
Joined: Tue Oct 02, 2018 5:44 pm
Contact:

Re: [DEVS please] Pipe to pipe transfers, are they hardcoded?

Post by Cadde »

bobingabout wrote:
Thu Dec 06, 2018 9:52 am
By the way bob, this explains why pumps only really pump "138.5" fluid/t.

First of all, pump outputs base level by default is 2 then.
In some way, the pressure difference between pump and target is 142. With the inertia bonus, this comes out to 138.5 fluid/t.

I put 138.5 in quotation marks because i haven't actually measured it myself, only going by what others have come to for conclusion and all i know is that the resulting pumping speed is less than 200 fluid/t on vanilla pumps.

Cadde
Fast Inserter
Fast Inserter
Posts: 149
Joined: Tue Oct 02, 2018 5:44 pm
Contact:

Re: [DEVS please] Pipe to pipe transfers, are they hardcoded?

Post by Cadde »

eradicator wrote:
Thu Dec 06, 2018 4:41 pm
Cadde wrote:
Thu Dec 06, 2018 4:16 pm
I am impatient. Also, not sure the fluid mechanics themselves are changing in 0.17, only the optimizations. That's what i take away from the FFF.
There was speculation in the FFF if the fluid mechanics do change or not. Dominik sounded vague enough about the subject that i'd guess they do.
Cadde wrote:
Thu Dec 06, 2018 4:16 pm
Also, modding it is practically a one liner.

mod/data-final-fixes.lua:

Code: Select all

for k, v in pairs(data.raw.fluid) do
   data.raw.fluid[k].pressure_to_speed_ratio = 0.6
end
That's also incompatible with any mod that adds their own "special" fluids. I.e. because someone else didn't like the idea that all fluids have the same viscosity ^^. If you do it in data-updates there might be a chance for them to fix it i guess. Bleh. Compatibility is difficult.
I know, i don't care about compatibility when i am doing it for my own game.
I could just add +0.2 to all fluids as well. But you know what happens when values get too high, right?
I don't want to watch fluids teleport back and forth through pipes. AKA, sloshing race deluxe.
All i want is faster movement between pump->underground pipe->pump as that's my bottleneck. Underground pipes count as two normal pipes.

I could rebuild my entire base to allow pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->pump->pipe->... everywhere but that really takes away the enjoyment of the game at that point.

I am eager to see the new fluid system! But in the meantime, desperate times calls for desperate measures.
None of this would be a problem if all the fluid recipes required less than 50 fluid/t. Which they do vanilla, but bob has this cool thing called speed module 8 that blows all of this out of the water and i am really dead set on having one machine that is capable of supplying basically all my needs rather than doing long rows of the same machine type to get the throughput i need.
And even then, it would still boil down to some pipe bottleneck somewhere because i simply cannot spaghettify enough pipes to get the amount of fluid i need into some location.

Hence, i look for increased flow in pipes instead.
If only the current fluid system took actual capacity (as in volume) into the equation all would be good. I could just make bigger pipes then. 2 tile wide "pipes" (read, storage tanks) even. But as shown above, fluid flow is limited by percent difference, not actual capacity.

User avatar
Optera
Smart Inserter
Smart Inserter
Posts: 2915
Joined: Sat Jun 11, 2016 6:41 am
Contact:

Re: [ANSWERED] Pipe to pipe transfers, are they hardcoded?

Post by Optera »

Thanks for this thread, it's a lot more understandable to me than xknights old one.

making sure flow doesn't get too high and start teleporting is simple enough

Code: Select all

local maximum_speed_ratio = 1
for k, v in pairs(data.raw.fluid) do
  local new_speed_ratio = data.raw.fluid[k].pressure_to_speed_ratio + 0.2
  if new_speed_ratio < maximum_speed_ratio then
    data.raw.fluid[k].pressure_to_speed_ratio = new_speed_ratio 
  else
    data.raw.fluid[k].pressure_to_speed_ratio = maximum_speed_ratio 
  end
end

Post Reply

Return to “Modding help”