Fluid physics (fix bizarreness, no extra calculation)

[ssilk]

Factorio calculates with fixed floats. That is like integer with a comma at position 4.

100 fluid is in reality calculated as 100.0000.

An amount of fluid of 0.0001 is then just "shifted" over the length of pipes; most pipes have zero, only one pipe has 0.0001.

[Aru]

... Now if we have a machine that consumes 10u of fluid per tick connected on the 3rd pipe, the machine will have an efficiency of 50% since there's lesser fluid on the pipe per tick than what the machine requires.

If we attach a machine on 1st pipe consuming 10u per tick, the other machine on pipe 3 will never work unless the first one stops consuming the fluid on pipe 1, even if we apply a pump feeding 15u to pipe 1. ...

I'm not totally sure what you're saying Mobius1, but I think in the scenario you're describing, the system behaves mostly the way I would expect it to, with or without the placement-order artifacts. Naturally if you have a machine consuming fluid "upstream" of another consumer, and it's using all the fluids, then the one downstream will not be able to get any. Also the closer the consumer is to the source, the higher the throughput can potentially be. This sounds like maybe you want pipelines to always be equalized from end to end, all the time, in which case it's a totally separate suggestion, should have it's own thread. (That said, I really am a fan of the way the fluids work right now, I think it's impressive and awesome. I just would prefer for it to work *correctly*, because nothing kills both immersion and the willingness to analyze and optimize a design-oriented game quite like finding inconsistent artifacts in it's implementations which cannot be fixed in-game.)

Factorio calculates with fixed floats. That is like integer with a comma at position 4.

100 fluid is in reality calculated as 100.0000.

An amount of fluid of 0.0001 is then just "shifted" over the length of pipes; most pipes have zero, only one pipe has 0.0001.

Totally not relevant, but I think factorio only uses floating point math underneath as is native to C++, even if the game displays them as fixed point. Also the phrase "fixed float" is self-contradictory, hah, fixed point and floating point are opposites.

[SkullTitsGaming]

I'm hardly one to be able to talk, but it sounds as though the problem lies in "we are using this method as a kludge for computational efficiency" and many folks are complaining about the kludge, correct? Maybe it would be helpful to have an example of the efficiency saved/lost for each model? I hate to be discriminatory, but depending on how much work it would be, maybe we just need to drop some lower end support for those still running on older processors?
#UnpopularOpinion

[Aru]

I think I'm the only one really complaining here. I just really like this game, and I think this aspect can be better. I don't like the fact that placement order influences results, and that the moment someone actually pays attention to how the fluid moves, they notice artifacts scattered all over. I could give a more detailed solution if I understood more about how the code is structured, but, I can't see it. So, the relatively detailed solution I had before is somewhat abstract, sadly. I could also give better calculations regarding each model, as you say, if I knew more. From what Rseding said, I am now suspecting that maybe the pipes are calculated in such a way that values have to be fetched somewhat randomly from within the fluid box array, in order to fetch volumes of adjacent boxes when simply iterating through the array. (Not really, but there's nothing enforcing correct order, being usually in order is only a convenient result of the fact that players usually place them in order.) In that case, the method I described before would actually be much faster, while also removing the artifacts.

But like I said I don't know how much sense it really makes because I don't know how it works graphically. Perhaps it is reduced to scattered access in order to update the graphics, perhaps it's scattered access times two, perhaps it's scattered but both are done at once in which case making one faster would not fix the other so it's still scattered access regardless, perhaps it is possible to fix both, perhaps the graphics aspect is not so scattered, I don't know.

[Shiandow]

From my experience the proper way to fix this kind of a problem is to just make the value of current fluidlevel an array of size 2 and then when you read the value use value[gametick mod 2] and when writing use value[gametick+1 mod 2]. Also known as the pingpong method. You use a bit more memory but its really simple to implement and the preformance impact is usually rather low. Ofcourse this may not solve all the problems and you may need to change some calculations but it's really the only way to go if you wan't the result be independent from the order in wich the pipes are accessed.

I think this is probably the best way to solve things. You will run into some minor problems because the pipes also modify each other's value, so you need to make sure that the first pipe modifying 'value[gametick + 1 mod 2]' updates it to 'value[gametick mod 2]' first, but other than that it should work.

[ssilk]

Totally not relevant,

Indeed. I thought it has to do with the rounding problems.

but I think factorio only uses floating point math underneath as is native to C++, even if the game displays them as fixed point. Also the phrase "fixed float" is self-contradictory, hah, fixed point and floating point are opposites.

https://en.wikipedia.org/wiki/Fixed_float
https://www.google.de/search?q=site:fac ... d-point%22&*
And Factorio uses it's own datatypes cause it's C++

[torne]

That's a really terrible wikipedia page written by someone who doesn't know the correct terminology for https://en.wikipedia.org/wiki/Fixed-point_arithmetic

[PunPun]

I think this is probably the best way to solve things. You will run into some minor problems because the pipes also modify each other's value, so you need to make sure that the first pipe modifying 'value[gametick + 1 mod 2]' updates it to 'value[gametick mod 2]' first, but other than that it should work.

That would break the whole thing. The point of the pingpong method is that you only read value[gametick mod 2] and only write value[gametick+1 mod 2]. The whole point here is to make the result be the same no matter what order the pipes are handled. If you start updating other pipes than the pipe you are currently handling then you are back to square one.

How you would go about this is that for each pipe you you calculate the difference in fluidlevel to all adjacent pipes then sum them up and divide by the number of adjacent pipes*2 and you get the change in this pipe. Ofcourse this has its own problems... This kind of simulation makse the fluid really viscous and makes the fluid move very slowly through the pipes. Also you run into problems with floating/fixed point accuracy and you can end up losing or generating fluid into/from nothing. If you want to avoid that then your only solution is to have a two pass approach. In the first pass you calculate how much fluid should move from each pipe to the next and in the second pass you aclually move the fluid. There's just no way around that.

[DerivePi]

I get the strange feeling that no one has bothered to look at the code snippet I linked previously so here it is specifically.

Code: Select all

struct FluidPrototype {
  double flowToEnergy; // Comes from Lua. Vanilla fluids have 0.59 here.
  double pressureToSpeed; // Comes from Lua. Vanilla fluids have 0.4 here.
};
struct Connection {
   FluidBox *other;
   Connection *facing;
   enum { INPUT, OUTPUT, BIDI} directionality; // Not sure of all the conditions here
   double flowEnergy; // This "remembers" the flow being transferred.
};
struct FluidBox {
  FluidPrototype *fluid;
  double amount; // How much fluid is in here?
  double baseArea; // Max fluid = amount * base_area * 10
  double baseLevel; // Default = 1
  std::vector<Connection> connections;

  double get_pressure() { return amount / baseArea + 10 * baseLevel; }
  double transferFluid(double transfer) {
    transfer = min(amount, baseArea * 10 - transfer);
    amount += transfer;
    return transfer;
  }
  void update() {
    for (Connection &cxn: connections) {
      if (!cxn.other) continue;
      if (cxn.other->fluid && cxn.other->fluid != fluid) {
        /* Mismatched fluids, don't know steps here yet */
      } else {
        double speed = get_pressure() - cxn.other->get_pressure() * fluid->pressureToSpeed +
            cxn.flowEnergy - cxn.facing->flowEnergy;
        if (speed > 0 && cxn.directionality != OUTPUT) {
          double amountTransferred = cxn.transferFluid(speed);
          amount -= amountTransferred;
          cxn.flowEnergy = min(baseArea, amountTransferred * fluid->flowToEnergy);
          cxn.facing->flowEnergy = 0;
        }
      }
    }
  }
};

[PunPun]

I get the strange feeling that no one has bothered to look at the code snippet I linked previously so here it is specifically.

that's because it has the same problem the current system has. Depending on wich order the pipes are added and processed it gives different results.

[DerivePi]

I was actually thinking that by looking at the code snippet, we could work on something a bit more specific.



Delaying the transfer of fluid by a tick does not solve any of the current problems. I'm sure you can be more specific now that you can see how the developers have already implemented the flow of fluids. Arguably, using the previous flow rate is using previous result to obtain a better result. How would you do it differently?

[PunPun]

I was actually thinking that by looking at the code snippet, we could work on something a bit more specific.
Delaying the transfer of fluid by a tick does not solve any of the current problems. I'm sure you can be more specific now that you can see how the developers have already implemented the flow of fluids. Arguably, using the previous flow rate is using previous result to obtain a better result. How would you do it differently?

Okay lets be very spesific. First the pingpong method does not delay the transef of fluid by any amount. you read value[gametick mod 2] and write to value[(gametick+1) mod 2] and then display the value[(gametick+1) mod 2]. Value is an array with a size of two and [gametick mod 2] is resolved to [0] on even ticks and [1] on uneven ticks. And [(gametick+1) mod 2] is resolved to [1] on even ticks and [0] on uneven ticks. Then you always display the value of [(gametick+1) mod 2] as it is the updated value. The change is as instant as any other method. The point of it is that it also preserves the value that was set the tick before so you can calculate the new value using unmodified data.

For example with the current system and your code snippet if we have pipes A, B and C in a line. Pipes A and C are empty and B is full. Now if you process A first then some fluid is transfered from B to A and then if you process C next then it gets less fluid because B is no longer full. The pingpong method avoids that. You calculate how much fluid should be transfered and save it into a different variable so all the other adjacent pipes can see the same starting value. Ofcourse the math gets complicated real fast using this method so it would propably be better to go for a two pass system. You first calculate the velocity of fluid in each pipe and after each and every pipes fluid velocity has been calculated you start applying it. It is still an O(n) problem even tough you need to go over every pipe twice. But going through every pipe twice is still better than having inconsistent behaviour.

[DerivePi]

The point of it is that it also preserves the value that was set the tick before so you can calculate the new value using unmodified data.

Using our pipe with connections A, B, and C, the developers currently have the pipe reach into each adjacent pipe and transfer that fluid explicitly in the order the pipe connections are constructed. Under the ping pong method, you transfer fluid based on the previous unmodified amount. What happens when demand exceeds supply? Say pipe connections B and C demand more than what is delivered by A and exists in the pipe? B looks at the previous amount and says I can have my demand satisfied as does C. But, B and C together can't be fully satisfied. I'd also think there is a concern for infinite fluctuations between the ping and pong states.

[PunPun]

Using our pipe with connections A, B, and C, the developers currently have the pipe reach into each adjacent pipe and transfer that fluid explicitly in the order the pipe connections are constructed.

Wich is the problem as it makes the result depend on the order of execution.

Under the ping pong method, you transfer fluid based on the previous unmodified amount. What happens when demand exceeds supply? Say pipe connections B and C demand more than what is delivered by A and exists in the pipe? B looks at the previous amount and says I can have my demand satisfied as does C. But, B and C together can't be fully satisfied.

The math gets pretty complex here but the gist of it is that each pipe distributes its content to ALL adjacent pipes evenly. Wich is the goal here. This part also gets a bit computationally heavy compared to other methods.

I'd also think there is a concern for infinite fluctuations between the ping and pong states.

There is no "state" here. With the pingpong method you on tick 1 you read value[0] and write to value[1] and display value[1] and on tick 2 you read value[1] and write to value[0] and display value[0] and on tick 3 you read value[0] and write to value[1] and display value[1] and on tick 4 you read value[1] and write to value[0] and display value[0] and so on. You are always reading the value that was calculated on the last tick and writing and displaying the value for the next tick. It generates no addinational fluctiations whatsoever.

Or you could do it the professional way and use the tried and trued method of first calculating the velocity of fluid between each pipe first and then apply fluid transfer on a second pass. The added overhead of going over each pipe twice is not that bad compared to the benefits of getting more uniform distribution of fluid.

[Aru]

I was actually thinking that by looking at the code snippet, we could work on something a bit more specific.

Delaying the transfer of fluid by a tick does not solve any of the current problems. I'm sure you can be more specific now that you can see how the developers have already implemented the flow of fluids. Arguably, using the previous flow rate is using previous result to obtain a better result. How would you do it differently?

The point of the delay (at least in my most detailed post), is that it avoids cache misses (a single pass, single iteration through fluid box junctions) when doing calculations for long, serial pipelines (which are most of the fluid boxes in the game, especially in large bases, which are the only bases where performance considerations matter) while also doing the calculations correctly (avoiding all of these implementation artifacts), while not creating any new arithmetic operations (essentially, the same physics basis the game already uses but done "right"). And my idea was not a one-tick delay, it's a one-pipe-segment delay between the flow calculations and the application of the flow. That is, updating fluid quantities after flows are calculated, rather than basing each new flow calculation on the updated quantity of the previous flow calculation, which is indeed a kludge for avoiding cache misses which produces terrible, non-physical, disruptive artifacts which no doubt no-one foresaw when the system was first decided upon.

I could give more implementation details if I knew the structure of the game code, I could even write the physics overhaul myself if I could see the code. The biggest downside of doing it as I detailed, as far as I can tell, is the added complexity. But it would still use the same numbers, still produce similar results as far as the rest of the game can tell, so at least bugs would be self-contained and therefore easily fixed (assuming there would be any).

[Koub]

[Koub] Implemented in 2.0 yay \o/ :
https://factorio.com/blog/post/fff-416