High-Throughput Exact-Items Train Loader (w/o Bots)

Smart setups of railway stations, intelligent routing, solutions to complex train-routing problems.
Please provide - only if it makes sense of course - a blueprint of your creation.
Post Reply
Builder_K
Burner Inserter
Burner Inserter
Posts: 7
Joined: Sat Dec 07, 2019 8:10 pm
Contact:

High-Throughput Exact-Items Train Loader (w/o Bots)

Post by Builder_K »

factorio-train-station-2.png
factorio-train-station-2.png (5.47 MiB) Viewed 647 times


Goals:
  • Load exact amount of multiple item types, no matter which chest holds which items.
  • Use all 12 possible inserter locations (single cargo wagon assumed for now).
  • Have each inserter running at max speed, moving the maximum possible stack size based on contents of its source chest.
  • Queue up items for next train while the current train is loading (or at least ensure this can be done in future).
  • Do *not* supply the train via bots; that would be too easy.
Caveats:
  • When there are only a few items left to transfer, inserters may skip their turn (due to having precalculated the wrong stack size); so the throughput won't quite reach the theoretical maximum.
> Walkthrough Video <
Describing how it works in detail (maybe too much detail)
https://youtu.be/d90H9KUAMUg


Logic of Each Section
factorio-train-station-4.png
factorio-train-station-4.png (5.02 MiB) Viewed 647 times
  1. Memory cell containing number of requested items to be loaded, decreased during the loading process.
  2. Left box limits values from A to the range [1-12], passes this onto the F wire. Right box simply passes values from A onto the G wire.
  3. Train arrival starts a 26-tick clock. Each inserter has a unique circuit signal, and these are triggered sequentially two ticks apart. (`A`, None, `B`, None, `C`, None, `D`, None, etc.) 12 inserters * 2 ticks each = 24 ticks, so not every timeslot is filled. These signals are sent with the value -2^30-1, or -1073741825. (one more than half-way to `min_i32`)
  4. Check what resources are available in this chest. Resources present are all output as the same value, 2^30.
  5. (This logic is repeated for each chest-inserter pair.)
    1. (F1) - Increment/decrement logic used to determine what stack size should be used for this inserter. Uses resource counts from the F wire (written by B logic), and only considers resource counts where the resources are in the chest as read by B. Increments/decrements a signal that's specific to this chest-inserter pair. Can only increment/decrement by +1/-1 per tick, so this value on it's own might be outdated by the time it's read (this is handled by later logic).
    2. (F2) - Stores increment/decrement values for each chest-inserter pair, and injects some constants needed by F1. Also carefully ignores values from B as they are passed along the F wire to F1.
  6. Combines clock signals from C with stack size signals from F2 and resources-required signals from B; and passes these on to H for each chest-insterter pair. Resources-required signals here may also be decreased by just-moved signals coming from I.
  7. Outputs a very-positive value (close to MAX_i32) for each resource signal when:
    1. Clock value indicates it's this inserter's turn to fire. (`-2^30-1`)
    2. Resource type is needed, as read from G. (`-count`)
    3. Resource type is present in the chest as indicated by D. (`-2^30`)
    4. Computed stack size is equal or less than the number of this resource required. (`+stack_size`)
    The output value for each resource will be MAX_i32 if the computed stack size exactly equals the requested resource count, any greater requested resource count will decrease the output value away from MAX_i32.
  8. The signal from H indicates which resources can be picked up. However, we don't want to do this if the previous inserter just grabbed a resource, and this inserter also grabbing the same resource would cause too many to be added to the train.
    In the same tick that signals arrive from H, the previously-activated inserter will pulse what it's just picked up. The resource values from H are close to MAX_i32, so the previous inserter picking up a particular resource can cause this value to overflow, disabling this filter, and causing the insterter to try and grab something else instead.
    Also note that the computed stack size doesn't come via H, but instead comes to from F2 via I.
  9. Items picked up by each inserter will need to reduce subsequent insertions. Insertion hand pulses are propagated back out to A, while making sure every step of the computation involving the resource-needed count is updated along the way.
Thoughts?

P.S. - I deliberately avoided checking whether someone's solved this problem already; way more fun to have a go at solving it myself. I think it's probably time to go see what exists out there already...

Tertius
Filter Inserter
Filter Inserter
Posts: 683
Joined: Fri Mar 19, 2021 5:58 pm
Contact:

Re: High-Throughput Exact-Items Train Loader (w/o Bots)

Post by Tertius »

It's not clear for me how you fill and refill the chests. In your video, you just insert a few items manually and completely ignore this aspect. But that's vital for practical and automated use. It seems you intend shuffling items with some splitter contraption, then insert them into the chests by uncontrolled mechanical means until they are full, so every chest will contain a bit of everything. However, if you always pull the same items over some time, freed slots tend to get filled with other items, so after some time you will end up with some item that has no slot any more in any chest. So after some time you have some item that cannot be filled into the train, because it's in no chest any more.

Exactly this very hard to solve issue is solved very easily and very elegantly by using the logistics network to fill circuit controlled requester chests. It roughly fills the chests with about the amounts intended to transfer to the train and will not grossly overfill the chests. Then circuit controlled filter inserters will transfer exactly the intended amounts. However, you cannot read the content of a requester chest and at the same time set a filter, so the whole setup will not work by just replacing the chests with requester chests.

What's the use case for this loader? It seems you intend to use this loader for resources like steel and copper. As far as I see, the demand for such items is so high, you will rather build dedicated and bigger trains and stations for each of these materials. Just one station+loader and train for copper, one for steel and so on, next to the respective smelter area.

Instead, such a flexible station is useful for supplying outposts with a variety of items you create in your mall to build new outposts and dependent infrastructure such as belts, rails, signals, power poles, miners, inserters, walls, probably turrets and ammo. And the mall is where your items are usually in the logistics network anyway. At least that's the way I have it.

Qon
Smart Inserter
Smart Inserter
Posts: 2120
Joined: Thu Mar 17, 2016 6:27 am
Contact:

Re: High-Throughput Exact-Items Train Loader (w/o Bots)

Post by Qon »

I agree with Tertius, and would like to add some:

Why?

Is this supposed to be a solution to a problem? Because what kind of train has only a single wagon (and no room to expand solution) and high throughput requirements and also carries multiple items in arbitrary configurations? I can't find a use for 2 of those criterion at once, and you try to combine all three. And they seem completely conflicting.

If you need to carry multiple items just have multiple wagons and load each type into each wagon by belt until it fills up.
If you want multiple types in 1 wagon because you want to carry a lot of low volume items then use wagon item slot filters and just load them in as well.
If you need high throughput then you also need multiple wagons and just a single item per train.
If you don't have a lot of space so you prefer small single wagon trains, then look at the picture you uploaded and consider if maybe you used an order of magnitude more space than necessary here or there.
Get rid of all the combinators, they aren't helping you.

However, if the goal is a useless do-nothing machine, then sure in that category it's a bit funky.

mmmPI
Smart Inserter
Smart Inserter
Posts: 2774
Joined: Mon Jun 20, 2016 6:10 pm
Contact:

Re: High-Throughput Exact-Items Train Loader (w/o Bots)

Post by mmmPI »

Builder_K wrote:
Tue Apr 09, 2024 12:19 am
Thoughts?
1) That has to be one of the most convoluted way to solve this problem i've ever seen

2) i think for long videos of explanations like those, people sometimes make several cuts and/or bullet points on a list to look at to avoid skipping something or saying another thing twice.

3) It's cool, it's original and inspiring. It's wayyy overkill for train design x) But i feel like you've learned a lot through this, maybe i'm wrong, i felt i could relate , when you explain things you start at the level of understanding you were not so long ago before you started this project, and it's a lot of things to explain because you also explain everything you've learned for it to be understood by other people that would start from the same level of understanding.

4) If instead of an inserter it was a speaker, and if instead of the stacksize you were giving a random number, it would make a robot that play music :) I've spent quite some time making some and it's not so different, staggering the inserters 2 tick apart from each other with a clock, is similar to controlling when the speaker will play its sound, more ticks in between make the tempo slower of faster. This is unique/original to me , as other blueprint loading trains typically do not combine staggering accross time the inserter activation and having individual logic for inserter, that's where it is similar to music box to me, unlike other train loading station.

The bitwise logic operation i'm not too sure i understand why it is necessary rather than a subtraction , i think i understand it's used to keep track of the counts of items to change the filter if a certain item is no longer needed and adapt the stacksize. Maybe it's easier to explain in 2 different parts, first the general idea " excluding certain amount of item" and after in a second part, "to achieve that we use signal [D], and then the explanation on which signification is attached to each bit. At this point i think it doesn't matter, it just cool it adds to the grandiose and i'm not sure i'm qualified to give advice on how to explain things without too much details :) I don't think it's the kind of blueprint people will use to quickly deploy, but i can see how it would be studied for its mechanic by players attempting something similar.

I have to say, it'd be cool to see it in automated action even in a editor setup with makeshift requester and provider and you are making this harder for yourself by having this difficult belt feeding problem still pending. You're not getting the reward for part 1 yet, when you can just watch. But hey if you make a part 2 you wouldn't have to re-explain many of the things, you could just say " see my previous video at time xx:xx " or something if you write it on a list :).

Builder_K
Burner Inserter
Burner Inserter
Posts: 7
Joined: Sat Dec 07, 2019 8:10 pm
Contact:

Re: High-Throughput Exact-Items Train Loader (w/o Bots)

Post by Builder_K »

Tertius wrote:
Tue Apr 09, 2024 1:14 pm
It's not clear for me how you fill and refill the chests. In your video, you just insert a few items manually and completely ignore this aspect. But that's vital for practical and automated use. It seems you intend shuffling items with some splitter contraption, then insert them into the chests by uncontrolled mechanical means until they are full, so every chest will contain a bit of everything. However, if you always pull the same items over some time, freed slots tend to get filled with other items, so after some time you will end up with some item that has no slot any more in any chest. So after some time you have some item that cannot be filled into the train, because it's in no chest any more.
The idea is that resources should be pulled out of storage with the same exact-quantities that this station needs to load onto the train. So only resources for the next few trains would be pulled out of the larger storage. The storage for the trains would also be limited to train stations.
Tertius wrote:
Tue Apr 09, 2024 1:14 pm
What's the use case for this loader? It seems you intend to use this loader for resources like steel and copper. As far as I see, the demand for such items is so high, you will rather build dedicated and bigger trains and stations for each of these materials.
I like to separate my factory into logical sub-factories, with trains running resources between them. However, I quickly end up needing a complicated train-unloading setup for each sub-factory, since each requires a different ratio of different resources.
I initially just wanted to be able to throw down a simple blueprint, and set a constant combinator to "make sure I always have enough of x, y, z..."

For a sufficiently large factory this design is impractical; simply directing base resources directly somewhere else makes sense.
However, for me this game has never been about building bigger or being most optimal, it's about making something interesting.
Qon wrote:
Tue Apr 09, 2024 3:07 pm
If you need to carry multiple items just have multiple wagons and load each type into each wagon by belt until it fills up.
If you want multiple types in 1 wagon because you want to carry a lot of low volume items then use wagon item slot filters and just load them in as well.
If you need high throughput then you also need multiple wagons and just a single item per train.
If you don't have a lot of space so you prefer small single wagon trains, then look at the picture you uploaded and consider if maybe you used an order of magnitude more space than necessary here or there.
I want multi- wagon trains, each wagon having exact quantities of multiple items, loaded quickly to achieve high throughput, and with reduced footprint on the destination/unloading side since every needed resource arrives on one train.

Again, this isn't the solution for an "optimal" megabase; but I think that having everything go through one super over-engineered central station would be cool.
mmmPI wrote:
Tue Apr 09, 2024 8:44 pm
2) i think for long videos of explanations like those, people sometimes make several cuts and/or bullet points on a list to look at to avoid skipping something or saying another thing twice.
Yeah, TBH I haven't worked on this in a little while, but thought I should document what I'd put together.
I created the labeled image (see original post) only after recording the video, it would have been better to make these in the opposite order. Hindsight is 20/20 ¯\_(ツ)_/¯.

mmmPI wrote:
Tue Apr 09, 2024 8:44 pm
3)... ...when you explain things you start at the level of understanding you were not so long ago before you started this project, and it's a lot of things to explain because you also explain everything you've learned for it to be understood by other people that would start from the same level of understanding
I try to explain things so anyone can understand, even if it means re-explaining "obvious" things.
Even technical people often benefit from this, since it's a constant reminder of why things are done the way they are. We nerdy people often get lost in the weeds (as an example: this whole project), and it's easier to see where a design might be improved, or what parts could be re-used elsewhere, when you known not just how it works but why.
mmmPI wrote:
Tue Apr 09, 2024 8:44 pm
4) If instead of an inserter it was a speaker...
...that's where it is similar to music box to me, unlike other train loading station.
This is just done strictly out of of necessity. I agree it's a similar sort of timing problem, where everything needs to be synced up just so.
mmmPI wrote:
Tue Apr 09, 2024 8:44 pm
The bitwise logic operation i'm not too sure i understand why it is necessary rather than a subtraction...
Bitwise operations are often easier in programming for dealing with power-of-2 numbers. In this case 1<<30 is the same as 2^30, and is exactly 1/4th the range of the 32-bit integer (the full range is exactly 2^32). The particular bits being set here don't actually matter, addition or subtraction would work just as well.
mmmPI wrote:
Tue Apr 09, 2024 8:44 pm
...it'd be cool to see it in automated action even in a editor setup with makeshift requester and provider and you are making this harder for yourself by having this difficult belt feeding problem still pending. You're not getting the reward for part 1 yet, when you can just watch.
Tell me about it.

There was a moment where it dawned on me how much other infrastructure needs to be built out to actually use this.

I was starting to loose interest actually.... but making this post, recording the video, and getting some feedback has been very motivating. We'll see if there's a part 2, no promises. ;)

mmmPI
Smart Inserter
Smart Inserter
Posts: 2774
Joined: Mon Jun 20, 2016 6:10 pm
Contact:

Re: High-Throughput Exact-Items Train Loader (w/o Bots)

Post by mmmPI »

Builder_K wrote:
Fri Apr 12, 2024 11:59 pm
Bitwise operations are often easier in programming for dealing with power-of-2 numbers. In this case 1<<30 is the same as 2^30, and is exactly 1/4th the range of the 32-bit integer (the full range is exactly 2^32). The particular bits being set here don't actually matter, addition or subtraction would work just as well.
Thanks for the additionnal explanations, i only play with binary numbers in factorio, it isn't obvious to me what you just said. To me "1/4th of the range" would be the different groups of 8 bits in the number as in you can write 4 different 8 bits number in a 32 bit number. I use the shift operation to "read" the position of bits , not do arithmetic operation.

Builder_K wrote:
Fri Apr 12, 2024 11:59 pm
There was a moment where it dawned on me how much other infrastructure needs to be built out to actually use this.

I was starting to loose interest actually.... but making this post, recording the video, and getting some feedback has been very motivating. We'll see if there's a part 2, no promises. ;)
Ah there's this saying "if you want to go fast, or if you want to far" , i'm not sure in english, something like if you took a lot of time to explain your design already, you are more likely to find someone who understand it and can help if need be :)

Builder_K
Burner Inserter
Burner Inserter
Posts: 7
Joined: Sat Dec 07, 2019 8:10 pm
Contact:

Re: High-Throughput Exact-Items Train Loader (w/o Bots)

Post by Builder_K »

mmmPI wrote:
Sat Apr 13, 2024 1:34 am
Builder_K wrote:
Fri Apr 12, 2024 11:59 pm
Bitwise operations are often easier in programming for dealing with power-of-2 numbers. In this case 1<<30 is the same as 2^30, and is exactly 1/4th the range of the 32-bit integer (the full range is exactly 2^32). The particular bits being set here don't actually matter, addition or subtraction would work just as well.
Thanks for the additionnal explanations, i only play with binary numbers in factorio, it isn't obvious to me what you just said. To me "1/4th of the range" would be the different groups of 8 bits in the number as in you can write 4 different 8 bits number in a 32 bit number. I use the shift operation to "read" the position of bits , not do arithmetic operation.
In this case I mean the "range" of numbers that a signal could contain.

1<<30 is basically saying "set just the 30th bit to 1". In decimal this number is 1,073,741,824 or 2^30.

The highest number you can have is 2,147,483,647 or (2^31)-1 (ignore the -1 for now, it's due to zero being considered positive)
So 1<<30 is halfway to the largest possible number.

The lowest possible number is -2^31, or 2,147,483,648.
So 1<<30 is exactly 1/4th the way through the range of every possible number. (Remember that adding 1 to the largest possible number "wraps around", and gives you the smallest possible number, so starting from 0 you have to go through all the positive numbers and then all the negative numbers to get back to where you started. 4,294,967,296 or 2^32 numbers total.)

Hopefully this makes a little more sense...

Tertius
Filter Inserter
Filter Inserter
Posts: 683
Joined: Fri Mar 19, 2021 5:58 pm
Contact:

Re: High-Throughput Exact-Items Train Loader (w/o Bots)

Post by Tertius »

Builder_K wrote:
Fri Apr 12, 2024 11:59 pm
There was a moment where it dawned on me how much other infrastructure needs to be built out to actually use this.
Keep this in mind. That's a vital consideration.
From my own experience, it's fun to create a solution for some real problem in a controlled lab environment. However, when I start to put some "finished" development into my real game maps, I realized there is always something secondary missing. And when I started to add these secondary things, I realized these are not always so secondary as I thought. To be able to actually use my development, I often had to change some design detail, which resulted in a whole new development process in the end.

So keep the whole thing in mind. Don't develop an isolated component. Always develop in the context of a whole map, of a whole base, and how you will actually connect your development with the real world on a real map.

There is an example for your loader. You mention you want multiple wagons for your trains in the end. But your current solution doesn't scale to multiple wagons. I don't mean the size and amount of combinators. I mean the fact you're not able to read the content of single wagons. Factorio only gives you the content of the whole train. You might not realize the difference, but you will if you start to scale your loader for multiple wagons. You will realize you need a completely different circuit setup, and there are aspects within a loader that have no solution, at least not a solution that runs fast.

I recommend you downscale your loader to 1 inserter per wagon and use dedicated trains. One material, one train. If you don't like the huge capacity of a whole train, use trains with 1 locomotive and only 1 wagon. And don't try to reproduce production ratios of materials within a train. The strength of the train system is you can uncouple the ratio of producers and consumers. If you need double the copper, build 2 copper stations with the same train. Don't load double the amount of copper in a train. Instead, run double the copper trains. This approach vastly reduces complexity. Although fun, micromanagement by sophisticated circuits will break very soon, if some production part of your base needs a change. But just building another station is very simple.

mmmPI
Smart Inserter
Smart Inserter
Posts: 2774
Joined: Mon Jun 20, 2016 6:10 pm
Contact:

Re: High-Throughput Exact-Items Train Loader (w/o Bots)

Post by mmmPI »

Builder_K wrote:
Sat Apr 13, 2024 3:22 am
Hopefully this makes a little more sense...
It does make sense, but it's doesn't make it more intuitive for me x)

Tertius wrote:
Sat Apr 13, 2024 11:58 am
There is an example for your loader. You mention you want multiple wagons for your trains in the end. But your current solution doesn't scale to multiple wagons. I don't mean the size and amount of combinators. I mean the fact you're not able to read the content of single wagons
I thought it would be possible to create a train with a wagon in front, then some loco, and then another wagon, to have room to build the loader on each wagons for the size and combinators.

Why would this not apply for multiples wagons ? it would mean doing them 1 after the other ? or not possible at all ? i'm not sure what you mean.

Tertius
Filter Inserter
Filter Inserter
Posts: 683
Joined: Fri Mar 19, 2021 5:58 pm
Contact:

Re: High-Throughput Exact-Items Train Loader (w/o Bots)

Post by Tertius »

mmmPI wrote:
Sat Apr 13, 2024 12:48 pm
Why would this not apply for multiples wagons ? it would mean doing them 1 after the other ? or not possible at all ? i'm not sure what you mean.
Ok, I looked more thoroughly. This implementation just loads what has been defined externally. It doesn't even look if there is already something of that in the train. So it's a completely static loader and the whole setup is just about feeding 12 inserters at the same time. To make it work right, you need to unload everything from the train before you start loading the train again.
My own expectations are that such a loader looks what's already in the train and just fills what's missing, and I overlooked this loader doesn't do this. So my comment above doesn't apply. However, this makes this loader even more not practical for real use. Or not a complete design. At least what I expect from such a loader.
It's a proof of concept how to make it so that you can use multiple filtering inserters adding all different things into a train but never overfill and grab the same things at the same time, which leads to overfill.
However, it's not ready for real use. It's not shown how to deliver items to that station. It can only work, if you provide items in a distributed manner to be able to feed all the 12 inserters in parallel. It doesn't handle items already on the train. It has to be set up differently to support multiple wagons, then the item delivery from outside becomes even more important. How can it fit into some base - it has an enormous space footprint. Usually, you have many small insignificantly sized stations where you just fill chests. With such stations, half of your base is filled with them.

I expect loading stations fit the base in an integrated and productive manner. This is an example from my current mall. It is a station with 4 individual stations for 4 different support train types and a waiting area above. The bottom 3 stations are the same layout/blueprint, just different sets of items to load. It loads what is defined in constant combinators per wagon, 1 filter inserter per wagon and only 2 combinators per wagon and additionally 2 per train. Essentially, these do the same thing as in the OP of this thread, just only 1 filter inserter instead of 12 per wagon.
The 4th station is for a specialized artillery train, only 1 wagon reserved for items, but supports dynamic trash unloading for that wagon at the same time, so it has a few more combinators for that wagon (the others support trash disposal via one dedicated wagon without circuit control).
In my opinion, such stations should not occupy more space than what is shown here.
Screenshot 2024-04-13 153852.png
Screenshot 2024-04-13 153852.png (2.49 MiB) Viewed 333 times

mmmPI
Smart Inserter
Smart Inserter
Posts: 2774
Joined: Mon Jun 20, 2016 6:10 pm
Contact:

Re: High-Throughput Exact-Items Train Loader (w/o Bots)

Post by mmmPI »

Tertius wrote:
Sat Apr 13, 2024 1:50 pm
It's a proof of concept how to make it so that you can use multiple filtering inserters adding all different things into a train but never overfill and grab the same things at the same time, which leads to overfill.
I was not sure if i had missed a flaw or not because the design is complex, to me it could work if finished but it would be (very) difficult to finish. Had it not been the case, i'd try to explain what i see as problem, maybe OP has a plan and i would learn things, maybe not and maybe it's a fun challenge to try. Such task could require some help from players that understand well circuits, like you ! I proposed mine but it's not much :)

Tertius wrote:
Sat Apr 13, 2024 1:50 pm
However, it's not ready for real use. It's not shown how to deliver items to that station. It can only work, if you provide items in a distributed manner to be able to feed all the 12 inserters in parallel. It doesn't handle items already on the train. It has to be set up differently to support multiple wagons, then the item delivery from outside becomes even more important. How can it fit into some base - it has an enormous space footprint. Usually, you have many small insignificantly sized stations where you just fill chests. With such stations, half of your base is filled with them.
My understanding is that train would come empty. To keep things simple at first, also stick with 1 wagon conceptually to finish the thing. For the fitting into a base, i think this system could go in city blocks, even if i don't like them all that much anymore, the future train change may change some of my reserve. The cityblock idea is not super efficient in many things, but here it would shine as it would allow integrating 1 such station per block and dimension such block to not exceed the capacity of 1 station in ressources input/output. Half the base filled with them seems a bit severe but that's part of the deal with city blocks.

I know of other ways to solve the problem, found some of my own, i would not have attempted such a thing myself i'd use something simpler. I had not think all the "missing" thing through as you listed them but i don't see anything wrong. I'm not sure it has to be done the way you say, like "there is no other way", more like "what you say seem consistent with my understanding of the blueprint and the mentionned goal".
Tertius wrote:
Sat Apr 13, 2024 1:50 pm
In my opinion, such stations should not occupy more space than what is shown here.
That is one (very) efficient way to do, but it uses robots so that wouldn't correspond to the definition of train loader (w/o Bots). "High" throughput is "subjective", OP could have staggered inserters 4 or 10 ticks aparts but that would have been against the design goal /self-imposed challenge. Same as copying/taking inspiration from other players. That's a tough thing to do and i found it interesting to understand the process of creation, to me it allows to reflect on the way i do things, identify things that help/don't to emulate.

A finished creation is different in my mind it's inspiring differently.

Post Reply

Return to “Railway Setups”