Towards Sophisticated Train Loading

[loganb]

Hello,

So I've been trying to design a multi-resource train loading/unloading stations for quite some time. After many design iterations, I've realized it's almost but not quite possible in the current game even including any published mods. I've tried to outline below the challenges and possible fixes/changes/enhancements to make such a contraption actually possible. Would love to see what folks think and/or how they've tackled this problem.

Goal: Create a train stop that can load an arbitrary number of different resources onto trains at full speed.

Here's a layout that almost works:

A Smart Train Station.png (4.48 MiB) Viewed 7238 times

How it works:

• Multiple resources come in on the belt from the upper left, in this example Pipe & plastic bottles (Bob's mods)
• The resources are evenly distributed across 24 crates using belt splitters & the circuit network (for maximum loading speed)
• Combinators control stack filter inserters (or filter inserters) to load appropriate quantities of resources on the train
• The "Smart Trains" mod is used so that the combinators can tell what is already loaded onto the train and the constant combinator in the lower right sets the desired total quantities to be loaded
• The arithmetic combinator in the upper right outputs the contents of all storage chests, so that the upstream production facilities can know when to stop producing (e.g. to load the buffer chests with up to 1k of each resource)

Why this doesn't quite work:

• The buffer chests can become unbalanced due to uneven loading of the train. If the station is only stocked to, e.g., 1k of each resource, it's possible to have all 1k in the chests for car 2 while car 1 is completely devoid of resources and never loads
• Trains with filtered stacks cannot load properly. Suppose a train wants to pick up only pipes, so it's cargo stacks are all filtered to pipe. The circuit network can't know this, and stack filter inserters can only load one resource at a time. The circuit network may set them to plastic bottles and the train never fills
• Inserters can become deadlocked if they pick up a stack but the train fills up before the stack is placed. Bad cases can make it impossible to fill the train
• SmartTrains only exposes total contents of the train, but there's no way to determine which wagon needs which resources. Only one resource can be loaded at a time for the whole train even if all but one car is already full of that resource.

Thoughts on solutions:

So first I'll assert that I think this is a reasonable problem that should be possible, though difficult, rather than the current situation where it's impossible. Hopefully we can agree on that. Also, I know inserter deadlock has been discussed ad naseum in many contexts, so I'll try not to rehash elements of that discussion and only discuss things I haven't seen before (but there are a few inserter-related things that matter).

• Knowing available space: I think it's necessary to know the available space in a cargo wagon for smart loading, rather than know its contents. This matters particularly for handling filtered stacks. The circuit network must know what can be loaded to properly program the stack filter inserters that load the wagons. It's not a panacea, though, because it's nearly impossible to avoid deadlock on resources with low stack sizes. For example, if there's one open stack, 12 stack inserters of Oil Barrels would more than fill the open slot, so some Stack inserters would become deadlocked. Available space must also be knowable on a per wagon basis rather than a per-train basis to similarly avoid deadlock
• Backing out deadlock: Even knowing available space, it's not possible to avoid deadlock at all times since the train can leave mid-insertion, or inserters can race with each other to fill only a small amount of remaining space.
• Rebalancing Chests: It's possible to have some buffer chests run dry while others are full. It is therefore necessary to have some kind of solution to rebalance the buffer chests until resources are redistributed.

Minimum Viable Changes:

In order to address the three problems above, I think this is the minimum set of game changes necessary to make a smart loading station possible:

• Connect Rails to Circuit Network: Rails should be connectable to the circuit network just like transport belts. They should sense the available space in any wagon stopped above them. They should output available space in filtered stacks, and also a special signal that indicates the number of completely open stacks
• Reversible Inserters: Reversible inserters would make it possible to both rebalance chests AND back out inserter deadlock when it inevitably happens. The suggested behavior is that Filter Inserters & Stack Filter Inserters forward load positive values on the circuit network (existing behavior), and reverse load negative values on the circuit network (new behavior). They'd have a special "Everything" signal that means "forward load (or reverse load)" any resource, e.g., a wildcard load/unload. This would make it possible to break deadlocks on the train-loading Stack Filter Inserters as they could be programmed to reverse load themselves after the train leaves the station. It would also make it possible to rebalance the chests since the filter inserters that pull off the belts can unload excessive quantities of resources back onto the input belts.

These two changes represent minimal cognitive burden for players, but open up a huge range of interesting design possibilities. What do ya'll think?

[ssilk]

I think this suggestion is well thought and includes a list of good insights.

But the idea of the suggested target goes beyond possibilities of informatics, mathematics and physics. It would fix a lot of transportation problems, if this game finds a solution of the problem you're trying to solve.

In short words: IMHO impossible to solve all problems.

Maybe one problem or two can be fixed (inserters grabbing, calculation of free stacks, for example).

I try to explain, think to calculation of free space left in wagon (just one of the problems):
- to calculate the amount of free space in a wagon you need to multiply the free stacks with the item-stack-size of an item-type. Ore has currently 50 (will be extended to 100 in 0.15) but plates have 100. Circuits have 200. The problem is: you cannot really know, which item will be loaded, cause that depends on the amount of free space left. Which means: you need to calculate that for all items (or a list of items). And for each wagon. This is CPU heavy.
- this is still not really correct: you need to take in account the stacks with the same item-type (what you are currently be loading or what has been loaded from before) and subtract the content of this stack from stack-size (depending on item-type).
- this is still not correct, cause you need to take into account the items, that has already been grabbed by the inserters and WILL be loaded.
- and then take into account, that all this calculations can only be made once per tick. Which means: no matter what you are doing, you're doing it one tick too late.

IMHO: You need to make a loading plan BEFORE you begin to load. Which inserters should move. How many times. Which item-type. But that is just another aspect of the rucksack-problem. You tap into the area of unsolved mathematical problems.

My advice: ask this question: is this target really useful? Does it bring the game really forward or is it just a problem in the form of "well, I thought it's a good idea..."
If useful: is it possible to split this problem into several smaller problems? That can also discussed here.

[Ragnaman]

What if a single cargo train would be split into 7 segments (or was it 6?) ?
Each segment would have its own cargo space of 3 squares or something.

Theoretically a single cargo train could carry 7 (6?) different items without complex logistic issues.

[loganb]

I try to explain, think to calculation of free space left in wagon (just one of the problems):
- to calculate the amount of free space in a wagon you need to multiply the free stacks with the item-stack-size of an item-type. Ore has currently 50 (will be extended to 100 in 0.15) but plates have 100. Circuits have 200. The problem is: you cannot really know, which item will be loaded, cause that depends on the amount of free space left. Which means: you need to calculate that for all items (or a list of items). And for each wagon. This is CPU heavy.
- this is still not really correct: you need to take in account the stacks with the same item-type (what you are currently be loading or what has been loaded from before) and subtract the content of this stack from stack-size (depending on item-type).
- this is still not correct, cause you need to take into account the items, that has already been grabbed by the inserters and WILL be loaded.
- and then take into account, that all this calculations can only be made once per tick. Which means: no matter what you are doing, you're doing it one tick too late.

Yes, it gets complicated to handle, and it's not 100% (hence the need for reversible inserters), you can get pretty darn close through a challenging Circuit Network problem. Your point about tick delay is really good, since a circuit network design to control the loading would end up 3-4 ticks behind. That notwithstanding, here's some examples of outputs that could then be reasonably used by the circuit and would work in almost all circumstances:

Imagine an empty cargo wagon with 20 slots, 5 of which are locked to iron ore, 5 of which are locked to iron plate. The circuit network would output:
- 5 * 50 = 250 iron ore
- 5 * 100 = 500 iron plate
- 10 open slots = 10 "S" signal (slots available)

Now suppose the cargo wagon is the same as above, but the cargo wagon has exactly one iron ore in an open slot that can hold any resource. The circuit network would output:
- 5 * 50 + 1 * 49 = 299 iron ore
- 5 * 100 = 500 iron plate
- 10 - 1 open slots = 9 "S" signal

The loading strategy would be to load the "locked" resources first. Often, they would spill over into open stacks which effectively causes them to become locked as well. Of there are free open stacks ("S" signal > 0), then an arbitrary resource can be loaded. Oil barrels and other resources with low* stack sizes could still lock inserters under this logic.

Another option that would just make everything trivially work is if inserters "reserved" the destination space. Just as trains turn a rail block "yellow" if they're destined to enter block but haven't arrived yet, an inserter could mark a section of the destination space "yellow" and not allow other inserters to attempt to fill it. I'm guessing devs don't want to do this given the CPU overhead. Synchronization is a bitch.


* "Low" means less than (stack size bonus) * 12 stack inserters

[loganb]

My advice: ask this question: is this target really useful? Does it bring the game really forward or is it just a problem in the form of "well, I thought it's a good idea..."
If useful: is it possible to split this problem into several smaller problems? That can also discussed here.

Oh, also really good comment here. So without the ability to make multi-resource train stops, late game train outputs, especially with mods that make things harder, become incredibly unwieldy because there are so many different output products.

Consider a smelting outpost in a game w/Bob's Mods. We're talking 10+ stations to output all the different plates. Yes, granted the problem isn't as acute in the base game, but a few minor changes could open up a range of interesting design problems and add a lot of depth.

[Optera]

Even with knowing the exact number of items in the train there's another problem I witnessed.
When using multiple inserters they randomly start 2 stacks at once, making stack calculation impossible.
You have to synchronize inserters to run with at least 1 tick delay between them so there can only ever be one stack started. I shudder at using one otherwise pointless nop gate for every inserter.

A clean solution would be to give wagons an option to sort their inventory like a players inventory. That would automatically compress all partial stacks into full stacks.

[Mendel]

instead of buffer chests, use a buffer wagon. (on piece of track that isnt connected anywhere, no locomotive obviously)
It balances perfectly. If you need more buffer, use multiple wagons on a row above your main train track.

warehouse mod might also give you the same effect.

[Deadly-Bagel]

See my topic viewtopic.php?f=194&t=39636

Multi ore unloading I've been using for my Bob's Mods game, seems pretty reliable. Got every type of ore unloading and it's not breaking a sweat. In fact the biggest problem is finding the room to smelt everything =/

[Engimage]

First of all the total train load speed when using stack inserters is like 10 seconds while using 1 side of inserters and buffer chests are loaded. Implementing a smart loading system which contains belts will make this system too slow and unresponsitive no matter what logic are you going to implement. Inserter deadlocks are 100% guaranteed.

The best way to make a multi-cargo stations is locking wagon slots to a selected product and dedicating certain buffer chests+inserters to those goods. It does not make sense to make it more than 4 different good types per station. And it makes even less sense to feed the station via mixed belts. You just have to sort out items beforehand and make a steady flow of them towards buffer chests. The loss of loading speed cause of low number of inserters per item type will be mostly the same as if you implement a working smart system if it was even possible.

One thing that might be really useful for smart inserters would be an ability to read target container demand (or free space).

[loganb]

The best way to make a multi-cargo stations is locking wagon slots to a selected product and dedicating certain buffer chests+inserters to those goods. It does not make sense to make it more than 4 different good types per station. And it makes even less sense to feed the station via mixed belts. You just have to sort out items beforehand and make a steady flow of them towards buffer chests. The loss of loading speed cause of low number of inserters per item type will be mostly the same as if you implement a working smart system if it was even possible.

I'm not sure I agree with your assertion. The design I posted at the top is fed by four belts. If made out of blue, that's 160 items/sec, which fills a cargo wagon with ore in 12.5 seconds. For two car trains, that's 25 seconds per train to fill the buffer chests. If you account for time for trains to enter and leave the station, you're within 50% of max throughput. The filter inserters will bottleneck a bit lower, but not a huge amount lower.

So it's possible, and it's challenging. This is why the circuit network is great "optional content"; it allows for interesting, complex structures—like mixed-resource belts and smart loading. It allows me as the player to do less belt micro-management (which, personally, I've played enough that it's now boring), and more control-systems problems.

One thing that might be really useful for smart inserters would be an ability to read target container demand (or free space).

OMG YES PLEASE.

[Engimage]

The best way to make a multi-cargo stations is locking wagon slots to a selected product and dedicating certain buffer chests+inserters to those goods. It does not make sense to make it more than 4 different good types per station. And it makes even less sense to feed the station via mixed belts. You just have to sort out items beforehand and make a steady flow of them towards buffer chests. The loss of loading speed cause of low number of inserters per item type will be mostly the same as if you implement a working smart system if it was even possible.

I'm not sure I agree with your assertion. The design I posted at the top is fed by four belts. If made out of blue, that's 160 items/sec, which fills a cargo wagon with ore in 12.5 seconds. For two car trains, that's 25 seconds per train to fill the buffer chests. If you account for time for trains to enter and leave the station, you're within 50% of max throughput. The filter inserters will bottleneck a bit lower, but not a huge amount lower.

While in a simple number (160/sec) you are right you do not account item travel time. You can't keep belts full all time cause you need to start feeding a particular resource on demand and you should let belts go empty after demand is satisfied. So when demand is triggered (train has stopped on your station) you will need to start the flow. To feed 4 blue belts with required resources you will need a pretty darn complex mechanism looking at the number of different resources you gonna feed it. And you have to account the travel time for those resources.

Even if you have every loading buffer chest bufferred with 4 different types of resources using slot locking, you will meet a belt deadlocks as you can't split resources evenly enough for belt ends not to deadlock. Such deadlocks can be avoided by either looping loading belts or just avoiding belts at all and going for requester chests.
With all that stuff managed you come to a loader inserter deadlocks which is the only technical issue in the game which might be solved at some point.

I see 2 possible solutions for this. One easy but inaccurate and one complex but accurate.

First is implement the ability for inserters to read target container free space so you can disable inserter prematurely to avoid it filling its hand. But for multiple items and wagons with locked slots this is kinda issue as well as you should specify which particular item do you want to put in the container. And if you want different items I can't find a possible logic for this feature.
Second one would be introducing a "reserved" item count in the container so that inserters while filling its hand would reserve a space inside container to prevent other inserters from requesting extra items for the same space. This would require serious game code changes and might be implemented "at some point in the future". Have to be aware that train moving from the stop prematurely will still cause deadlocks in this case.