Friday Facts #312 - Fluid mixing saga & Landfill terrain

[Cadde]

Ever since the "no fluid mixing" thing entered the scene i've been a sad panda. I used to do multi use factories before the change where i could turn on a certain setup if i wanted more of a certain less frequently used fluid that was only used every once in a blue moon.
This is no longer possible under this new fancy no mixing system.

The devs should instead have worked on a player controlled fix to mixed fluids in pipes. The ONLY time fluid mixing was a problem was when there were alternating fluids in the same length of pipe. For such occasions, i wouldn't actually mind the fluids actually mixing together to form a "dirty" fluid that could then be pumped out. The same way steam temperature works, if you add 165 degree steam to 500 degree steam, the 500 degree steam almost becomes useless as it's now some 499.9999 degrees instead. (Which in of itself is pretty whack if you ask me but whatever)

And in some situations, i would even go so far as to say fluid mixing could actually be a FEATURE. If you pump 5000 units of flavored water and 400 units of alcohol into the same tank, the two fluids should actually just mix in the tank and form a new fluid aptly named "party nectar" which you use at parties.
The mix should take on a ratio of their two components and can still be used within a certain acceptable limit of ratios.

And if you mix oil and water, you get contaminated water or contaminated oil (depending on ratio) but you can actually fix this. In fact, there should be offshore oil wells where you actually get water contaminated oil and have to separate it anyways.

And the problem with the fluid simulation is simply the fact each and every pipe is simulated separately and doesn't split liquids in a logical/realistic manner but rather "who ticks first".
I've said it a few times before... Stop simulating each and every pipe section and instead simulate it as a network. You can still visually simulate fluid flow and each fluid can be a time delayed packet in the network to give the impression of fluid flow. It would work on a request basis where all active requests are gathered before the packets are sent and then divided equally based on needs, availability and the length of the network between providers and requesters.
This isn't "fluid teleportation", it's just an easier model to work with as you treat the whole connected network as a single fluidbox and only deal with packets and saturation.

[mathe172]

One suggestion that could solve all the weirdness with prevented actions etc. that I haven't seen mentioned in this form is the following:

The basic idea is that there could be special pipes for each type of fluid which are placed by the player. This way, there would be no issues with any of the fluid mixing madness, since different types of pipes simply don't interact. Of course, this would be quite painful in terms of having many recipes and items, and would require replacing all of the pipes if the player wishes to change the fluid. So I would suggest the following alternative based on filters instead:

Suggested behavior

• Pipes can have a fluid filter set
• Pipes with different filters do not connect. Pipes without a filter set do not connect with filtered pipes
• Pipes can only contain the fluid matching their filter
• Unfiltered pipes cannot contain any fluid
• Changing filters deletes any fluids contained in the pipes
• Changing filters will cause any compatible systems to be connected, and any incompatible system (e.g. assemblers) will be disconnected
• Filters should be configurable for individual pipes and whole pipe systems
• Filters should be configurable via the circuit network (again single pipes and whole systems)

Some notes

• Setting up a new pipe connection is only marginally more complicated than it is currently: The workflow is essentially: Place pipes -> Configure the filter of the pipe system
• To simplify building, it would probably be nice to be able to add configured entities to the toolbar (but this is probably a separate suggestion...)
• This gives a straightforward way for having adjacent pipes that do not connect (although some players might miss the challenge )
• Setting filters via the circuit network should allow for all the advanced craziness of having one pipe carrying multiple fluids, without distracting new players
• Actions are never prevented, removing any confusion like "Why can I not rotate this!?"
• There is no way to get fluid mixing, so it should be way less of a headache implementation-wise. Also, since disconnected pipe systems are a natural state of the design, there is no potential for weird edge-cases with crazy placement orders. Pipes will simply not connect/disconnect as necessary
• This approach is a bit more manual, but this seems like a worthwhile tradeoff, given the reduction in automagic behavior, both for users and developers (see last two points)
• A small tutorial on pipe filters when the player places the first pipe should solve any potential confusion for new-comers
• Changing filters via the circuit network potentially provides a way to efficiently void fluids, such as petroleum. If this makes handling the multiple outputs from oil-processing too trivial, a threshold for how empty pipes need to be could be added to switching via the circuit network

tl;dr
Force manually configurable filters for pipes, removing the need for any automagic weirdness and associated edge-cases and bugs.

[PhasmaNL]

I feel your pain Dominik, but the work is appreciated! With hindsight it's always easy to say if things should have been done differently but this can lead to being afraid to touch something (as a coder) which is really bad. So, glad you stuck to it. It will be better in the end!

[gravityStar]

He is actually coming to our offices next week, so follow the news for reports of developers falling out of windows.

Awaiting Wilhelm scream. (Multiples)

[Sloppyjosh]

While we are on fluid mixing... perhaps we should talk about water vs steam. This is not fluid mixing and gets in the way of nuclear power when you want to bootstrap your steam production.

[SirPrimalform]

Hmm, I like that you've given landfill its own texture, but I feel like it should gradually lose its grid shape with time. Maybe even grass over eventually?

[EstebanLB]

Ahem...

Change landfilled area texture to something more gravel-like
viewtopic.php?f=71&t=65506

Finally!

[jockeril]

I just love those stories about how you struggle with different mechanics. The fluid mixing issue is very important and I think we should all thank Dominic for the effort you put into it (I can understand the nightmares about work, as I have those too sometimes, even with issues that were already reassigned to my team mates). I think listening to Rseding, from past experience, is a good idea but only if it helps solve this issue not ignoring it

The new landfill textures look nice and this puts me in a position of conflict - I don't like the current texture so I found the mod that restores the textures that would have been there if water wasn't. I've gotten very used to the original texture of an area returning when you refill it with landfill - it seems very natural,

Now that you've introduced this texture, I'm conflicted as it looks nice and I don't know if I like it or the mod's solution better, though I must admit that having the natural texture of an area return when land filling still feels right to me. Maybe it's the look of that new texture your showing that is still weird to me

In conclusion - we should all thank Wube for making this amazing game and working with you're community on it - that's what makes it so popular - you're all amazing ! Also or modding community is amazing !

[Light]

It seems to me that what some people here are looking for is platforms and not landfill. The platform can cover the water but also be removed to restore the water underneath. After using the mod for a while, I must agree that being able to remove a misplaced tile or create a watering hole down the road is very useful.

Given we're building a lot of industry and paving over the planet, these platforms seem to be more fitting than landfill for that industrial feel. It also restores water if needed without adding any digging mechanic that could be abused to create impassable moats, as the platformed area might need a few well placed holes for water pumps that you might not have planned for at the time you filled the lake.

[eradicator]

It seems to me that what some people here are looking for is platforms and not landfill. The platform can cover the water but also be removed to restore the water underneath. After using the mod for a while, I must agree that being able to remove a misplaced tile or create a watering hole down the road is very useful.

Given we're building a lot of industry and paving over the planet, these platforms seem to be more fitting than landfill for that industrial feel. It also restores water if needed without adding any digging mechanic that could be abused to create impassable moats, as the platformed area might need a few well placed holes for water pumps that you might not have planned for at the time you filled the lake.

+1

For nuclear reactors on maps with low water even one misplaced landfill can mean that the "pond" is no longer usable and another one has to be found. And with the way water-border graphics works it's sometimes difficult to tell where exactly one has to fill in to get the desired result.

[planetmaker]

Surely I'm missing something pretty obvious or a big part and the devil certainly is in the detail:

Do the single pipe elements (including pumps, storage tanks, pipe-to-ground, straight pipe) have all a separate storage to indicate how much fluid they contain?

If so, wouldn't a possible fix to the fluid-mixing problem be such that
* devices are only connected when adjacent entities have identical fluids
* placement of fluid entities could be allowed anywhere. If it is adjacent to different fluids: don't connect to either. This condition could also hold when it would join different fluids by removal of an entity (like a pipe-to-ground). Remove fluid from an entity being rotated and apply this rule.
* a warning sign (like turret action, missing logistics supply,...) could flash on the map at locations where connection of fluid entities is refused due to different fluids being connected otherwise.

If not: why don't they? On the map, when I check items their fluid contents is already reported to me. I see this as step towards the (much earlier) discussed 'realistic fluid treatment' as well, should that is still be a thing (yes, differential treatment of pressures is less accurate than the analytic integral... alas...). I understand that it's more difficult than treating them as belts as pipes basically are a two-way belt, so there's usually not a single way from source(s) to drain(s).

[Dominik]

What if the underground part of a underground pipes really existed, just didn't connect sideways? It would not be possible for undergrounds to cross each other, one would have to go over the surface. On the other hand, no weird reconnect behavior.

I have a proposal for underground pipes:

How about making their connections unbreakable by placing another underground pipe in between them?
Let any given underground pipe connect to all underground pipes that are in the direction it's facing, have the opposite orientation, and are within it's range. (and of the same type of course)
This allows more than one connection per underground pipe and it simplifies a lot of logic because removing an underground pipe would never create a new connection, and adding an underground pipe would never remove an already existing connection. This even removes the need for blocked connections because this logic would be equivalent to over-the-ground pipes where it already happens.
Simply, you shouldn't be able to make a new connection by mining a pipe nor remove one by building a pipe. Let this apply to underground pipes as well.

This would mean that one fluid box can have an arbitrary number of connections and would have problems both with design and performance. How would they be organized/ordered? How would the flow split when going from one pipe into many pipes in various distances? I don't think it would be player friendly either.

[Dominik]

Please stop messing with established mechanics. Was fluid mixing annoying? Yes. Is this better? Not necessarily, your gif shows a quite awful example. Does this change add to the game? Absolutely not.

Feels like so much cool stuff could have been done with all that time.

I feel like my favorite game is just getting dumbed down with every update. I dont want an easy game, that's not why *anyone* plays factorio. Think about what minecraft would be like if they went back and changed all the weird unintentional quirks that popped up in early development that are now just rock solid mechanics.

Something being established does not mean it is good. We are going to mess with mechanics any way we find appropriate to make the game even better than it is. The fact we do is one of the reason that the game is good in the first place. The fact that we often hold our hand - because of players who are used to something - is a reason why some things are not going to get better even though they could.

Those gifs show rather entertaining examples. They are not something that a player would actually encounter in their game (unless they try to). If you mean the pvp that is already fixed.

This is not dumbing down. It is preventing something that is just frustrating and clearly not following players intention. Difficulty is good - but it should be about challenges, not traps.

[Dominik]

About fluid mixing. Inputs on assembler and chem plants have never mixed fluids so they should not set fluid filter or hinder building pipes around them.

For the new fluid algoritm I must say I am not a big fan. Real pipes is usually filled with fluids and is not half empty as in factorio. The factorio simulation is more similar to an open channel flow than a pipe flow. Using the factorio simulation get the odd side effect that full pipes has more or less no flow.
For pipe flow, pressure is moving fast and can be approximated to be instant so then you can base the flow on the pressure difference between outputs and inputs instead of simulating every segment. In that way we would get an more realistic and UPS friendly pipe flow simulation. Because of the discreet nature of Factorio recipes I think the fluid boxes in the machines should stay similar as is.

new fluid algo is not there yet.

[Dominik]

Maybe i just dont understand something, but why not ONLY allow placing fluid boxes when it doesnt mix fluids? And dont allow to place them in ANY other case?

that is the whole point

[Dominik]

Just wanna throw my vote in for reconsidering the ban on fluid mixing.

I think the current mechanic is more annoying than the occasional accidental mixing. And honestly, a lot of that is just because "well that was stupid" is far less frustrating to me than "WHY won't it let me build this here?"

But mostly I've always wanted fluid mixing to be a useful mechanic, and it's been SO CLOSE, so many times. When you introduced nuclear and we could mix different steam types I played around with that quite a bit, trying to find a useful way to mix the steam and water to upgrade power production in stages or something like that. And I've made numerous, somewhat successful universal train stops and fluid filters, and was super hopeful that the "fluid squashing" discussed a while back in FFF would finally make that more than just an experiment... But instead we got a system where that kind of experimentation isn't even conceivable. Personally, I find that to be more frustrating than fun. It just feels like an arbitrary, unnecessary restriction...

Well it's a close call. But there simply is no place in our game for unexpected traps with big frustrating results.

[Dominik]

I don't care much about having fluid mixing prevention. But i would like to be able to design flexible systems, which change recipes dynamically and use different fluids delivered using a pipe network wich is connected to multiple different fluid sources by pumps of wich only one is enabled at a time.
So please consider a less restrictive fluid mixing prevention.
Also, please adopt Picker Pipe Tools into vanilla.

We all hate refactoring don't we?

I really like refactoring code to be easier to understand / better maintainable, less bug-prone, or more performant.

Splitting systems by pumps was considered but it is bad for performance. I know that setups could be build that use same pipes for different fluids but honestly it is a very miner use case that is not actually very useful and very few people appreciate. The simplicity and performance is more important.

Picker pipe tools - yes, we would like to. But probably won't have capacity for it before 1.0.

I like the result of refactoring. But I don't like to have to do it in the first place. It is a lot of work with zero impact on the outside.

[Dominik]

One suggestion: Make a "disconnected underground pipe" graphic. Like a capped pipe going toward the ground or something. Used for any case where you've placed one end but not the other. Would be useful for detecting these pipe network changed because a pipe in the middle was destroyed cases. May also be useful for belts.

Noted, that sounds useful.

[Dominik]

Why is this something you can't fix? Why is that happening at all?



Lets describe what I see going on:

1) The crude oil pipe is empty but locked by the assembler.
2) Placing a ghost disconnects part of the pipe making it unlocked.
3) Replacing the ghost does two things (in this order):
- connect the empty pipe to sulfuric acid (allowed)
- re-connect the empty pipe to crude oil because the ghost was emoved (mixing fluids)

It looks to me like you only check if placing a pipe is allowed but not if any side effect, like ghosts getting removed due to it, breaks things.

I know any action involving multiple steps (remove ghost, place pipe) can have many corner cases. But there would be two simple solutions I can see here:

1) reverse the order. First remove the ghost. This makes the empty pipe a crude oil pipe again. Then placing the new pipe becomes illegal and fails. If you can't do better the ghost will be gone but no fluids mixed.

But would it be impossible to undo the ghost removal? The game already has undo functionality which records how to reverse an action. At first glance I would think an undo should never fail if no fluid has flown because both the old new state of each performed sub-action was valid. So "all" you have to do if any action causes a fluid mixing is to call undo. If undo does fail leave the user in the state with no mixed fluids.

The drawback there is that there might be corner cases where removing the ghost makes placing the new pipe impossible. But with the new pipe placed the result would have been fine (no mixed fluids). Well, to bad. Deconstruct something temporarily to work around it. Should be rather rare and not too annoying.

2) Handle this as a transaction. If the end state does not validate then you reset to the start.

This would mean you do several actions while allowing fluid mixing and at the end you do an extra check if the result still has any mixed fluids. If any mixing is going on you undo all actions and recalculate the fluid locks. This would involve tons of code because every entity with fluids would have to be changed to allow mixing fluids temporarily and have a is_mixed() method. So I won't hold my breath.


Overall I love the fluid mixing prevention despite the corner cases where it doesn't allow things. Like changing a recipe in a row of chemical plants without first removing the old recipe in all of them first. There is only one thing missing now. Pipes with no lock from fluid box filters but minimal fluid (<0.1) should be considered empty. Setting a new fluid filter or pumping in new fluid should be allowed and destroy the remaining drops of wrong fluid.

The use case I have is for LTN depots where I want to remove left over fluids from returning trains. So I have 6 depot stops with pumps all connected to one provider stop. Normaly the pipes are empty. But if a train with fluid comes in the fluid is pumped to the provider stop and some other train will pick it up there and return it to the system. So far so good. Problem now is that the pumps don't manage to pump the pipe system 100% empty. All pipes show 0.0 fluids but some drops must remain somewhere. So the pipe system remains locked to the fluid and no other fluid can be remove from returning trains anymore. Result: It only woks once.

Of course it is theoretically fixable. But practically it is not worth it. The fluid mechanics and checks are contained in some fluid related parts of code and the entities and this would require it to overflow into more generic parts of code outside it. On the other side, this is a case that would not happen accidentally in a real game and if it did, it's not a big deal.

When building, the checks are done first, while affecting as little as possible of the game state. Because changing something, like placing an entity or removing a ghost, has many side effects spanning quite far into different places in the game. Undoing it is complicated and prone to errors so it is avoided as much as possible.

So issue in this case is that when building the pipe there is no mixing, but removing the ghost creates it and batching these two operations into one action makes it hard to detect before it is too late. It could be solved, yes, but it is just not worth the clutter and new issues.

[Dominik]

For the pipes I think several people have offered better suggestions than just preventing mixing. The entire problem with pipes is that they are this special thing that users can't interact with, and they have all of these partial item and weird properties going on. If we could start over on pipes I think it would have been beneficial to treat them like fluid belts. My suggestion for probably another game would be something like this:

1. Put a static speed on the flow rate, that is well known. Belts all have static "flow" rates and it allows everyone to easily understand how much is moving down the belt.

2. Numbers should go back down to something manageable. Everyone knows you can fit 8 items on one belt square, a pipe should be able to hold say 8 or 10 items per pipe tile.

3. Allow fluids to just mix in the pipe. hover over it and you can see a section contains 5 water, 3 crude oil, 2 lubricant

4. Fluids can only move if there is room in an adjacent pipe. If there is no room then a fluid can't move.

6. Pipes can flow in any direction, but you can have T junction splitters were you can filter one of the input / outputs to only allow certain things to flow. (Since it can flow in either direction, you could say block a junction so that one side only allows crude oil to flow. This means it can pass either way through this, and would allow you to use that as a mechanism to prevent unwanted things from flowing. It would prevent certain fluids from both going into, and going out of a section of pipe.)

7. Items ability to move is going to be governed by consumption, not by production. Put water and crude oil into the same pipe, and have a factory only consuming water. If you have 9 water and 1 crude in that pipe, then 9 units of water will be able to flow provided you aren't pushing additional crude into the pipe. Then once you get more crude, the flow will start to fall off. If there is 5 crude units sitting in the pipe in front of a factory consuming water, then only 5 water units can make it into that factory at a time. Once you fill that pipe with other fluids then it will starve the factory and it will stop producing since no water can flow.

8. If nothing is in a pipe, then a consumption item (Factory, pump) is treated as a vacuum and can move up to 1 pipes worth of void per movement. If there are 10 items of "air" in the pipe, a factory will keep consuming 10 items per movement rate until it pulls a fluid to it. Once it pulls the desired fluid to itself it can start consuming it, or if it pulls another fluid then it starts cutting down how much of a vacuum it can create. (If we wanted tiers we could potentially cap low level things to say only consume 4 items / s, where larger ones can consume say 6, and the largest can move 10)

9. Allow the player to handle fluids, and store them in their inventory. They should be able to hand barrel fluids, so they can fix any pipe at any time.

Using the t junction would allow you to fix busted lines. simply drop a t junction down on a line, put a filter on it for a certain item. Then you need a pump so it can "consume" items, and finally put them into storage. (Basically the equivalent of inserters putting items into storage chests) We could also have "filtered pumps" so if you wanted to you could toss one of those down right next to a pipe, and have it pull certain items directly off of the pipe. It would be a similar mechanic to the t junction, but pumps are one way where a t junction is two way.

Advantages:

No need to worry about fluid mixing
Provide ways to actually fix when a fluid mixes with another fluid
Allow pipe reuse so low consuming entities can share a pipe
Provide a new mechanic that is similar to an existing one, but still works in a unique way
Provides a way to actually calculate item movement
Retains two way flow, which makes it unique to belts

disadvantages:

More complex to manage
Can cause input problems until players learn how to deal with them
May be CPU intensive since you can't just rely on items only traveling in one direction at a set speed like belts do

I'm sure there are a number of holes you can poke into my idea, but I think overall it would fit into a Factorio like game. It provides something for people to optimize and obsess over, but it doesn't have all of the unknowns the current pipe system has. Everything should be something that can be calculated out, and would provide unique challenges while retaining some consistency with current game items.

I don't understand why you would want to ignore the one thing that makes items and fluid different, that is that fluid can be split into any amount. Fluid does not come in boxes so why treat it that way? People can use barrels if they want that.
It would not be intuitive and look silly in some cases. A "box" of fluid arrives to a junction. What does it do?

From the code perspective there is this small word with big implications - in no. 8 - a factory "pulls" the fluid. How would you model the vacuum/pulling? I think that you would end up with the same thing we have now.