Nonblocking, expandable buffer

This board is to show, discuss and archive useful combinator- and logic-creations.
Smart triggering, counters and sensors, useful circuitry, switching as an art :), computers.
Please provide if possible always a blueprint of your creation.
Post Reply
Copy link
canidae
Long Handed Inserter
Long Handed Inserter
Posts: 72
Joined: Thu Feb 18, 2016 6:24 pm
Contact:
Contact canidae

Nonblocking, expandable buffer

Post by canidae »

This buffer keeps the input belt always running at full speed (as long as buffer isn't [almost] full) and the output saturated (assuming buffer isn't nearly empty). It's easy to expand and it works on any item types without configuration.
nonblocking_buffer.gif
nonblocking_buffer.gif (13.58 MiB) Viewed 9110 times
Blueprint
Here's an example on using the buffer for multiple belts (i.e. a "bus"):
4-belts-buffer.png
4-belts-buffer.png (1.97 MiB) Viewed 9110 times
Blueprint
I'm reusing an old post, here's the previous entry (with a flawed design) to make sense of some of the comments below
Old, flawed design
Last edited by canidae on Wed Sep 28, 2016 6:21 am, edited 4 times in total.
Top
sabriath
Inserter
Inserter
Posts: 26
Joined: Fri Jan 10, 2014 6:49 pm
Contact:
Contact sabriath

Re: 4 Lane buffer (full throughput, expandable, fairly compact)

Post by sabriath »

Slight improvement to your setup:

Image

Your current picture shows a feedback system, where the materials are pulled off the belt to buffer AFTER the placement to put back onto the belts....this can cause a ripple effect. The picture I'm showing is a feed forward system. When the split line has enough material (overflow of reading more than '11' or so combined), it starts to buffer that material and immediately try to put it back onto the line in priority insertion which keeps the output flowing constantly.

The buffer itself contains MadZuri circuitry for equal inline box splitting. The boxes are connected with red to arithmetic combinator that does "Each" divided by "4" (or the number of boxes in your buffer) and outputs that value on 'A'. The constant combinator provides 1 'A' as a booster to the network to start with (since factorio truncates). Each inserter has a single green wire to its corresponding box and a red wire from the 'A' output line with a condition 'Item' < 'A'.

Obviously, this design can be duplicated for all 4 lanes, and there's enough space to put underground belts to avoid each other.
Top
canidae
Long Handed Inserter
Long Handed Inserter
Posts: 72
Joined: Thu Feb 18, 2016 6:24 pm
Contact:
Contact canidae

Re: 4 Lane buffer (full throughput, expandable, fairly compact)

Post by canidae »

Thanks for your feedback, I'll take a look at it.
Good idea to feed forward, although I'm not certain if you can make as compact design with this setup with multiple lanes (and with as much storage capacity per tile used). Also, as I wrote in my post I don't want the input to ever be slowed down, loading the chest evenly cause the input to be slowed unless you have some few inserters at the end to pick up what passed the evenly loading inserters. I did do this at first, thought it added too much complexity to an already complex design, and realized that fast inserters spreads out the loading fairly well when the buffer input is fully saturated. Although, it'll fail when the buffer input isn't fully saturated, so I'm thinking of going back to evenly loading again, with some solution to prevent input from being delayed.
I did discover some mistakes in my design (not to mention that I completely forgot about power poles! I need to stop using that super power pole supplied by Creative Mode mod), namely where I use circuitry to join them back to saturate the output. I didn't want to build above the top belt, so I cheesed it with circuitry instead of side loading. The circuitry will not work properly when the input belt is nearly saturated.
I intend to revise my design, see if I can improve some of the shortcomings. The difficult part really is to make it as compact as possible when using multiple lanes.
Top
canidae
Long Handed Inserter
Long Handed Inserter
Posts: 72
Joined: Thu Feb 18, 2016 6:24 pm
Contact:
Contact canidae

Re: 4 Lane buffer (full throughput, expandable, fairly compact)

Post by canidae »

sabriath wrote:Slight improvement to your setup:
[...]
I've updated the design to use feed forward and generally use a smaller footprint. It did lose some storage/tile with only one chest, but since you can expand that's arguably not the most important aspect. I also added partial even loading, but let the last two inserters (for each segment of six inserters) load as much as they can. This is to prevent the input from getting clogged and start stuttering. I also improved the buffer unloading, so the output belt is about as saturated as you can get it.
(A minor detail is that I'm using only an arithmetic combinator for the even loading, the constant combinator is not necessary).
Last edited by canidae on Tue Sep 27, 2016 9:04 pm, edited 1 time in total.
Top
Linosaurus
Long Handed Inserter
Long Handed Inserter
Posts: 89
Joined: Thu Jun 11, 2015 5:50 pm
Contact:
Contact Linosaurus

Re: 4 Lane buffer (full throughput, expandable, fairly compact)

Post by Linosaurus »

canidae wrote:(A minor detail is that I'm using only an arithmetic combinator for the even loading, the constant combinator is not necessary).
Wait really? I'd expect the inserter condition to become '0 < 0' when the chests are empty. I'm probably misunderstanding something.

I've used a similar even-loading setup, but I set the constant combinator to a higher value like 50. It *feels* like that should increase throughput, to let the inserters keep working even when things are slightly uneven. But I haven't actually measured anything.
Top
canidae
Long Handed Inserter
Long Handed Inserter
Posts: 72
Joined: Thu Feb 18, 2016 6:24 pm
Contact:
Contact canidae

Re: 4 Lane buffer (full throughput, expandable, fairly compact)

Post by canidae »

Linosaurus wrote:
canidae wrote:(A minor detail is that I'm using only an arithmetic combinator for the even loading, the constant combinator is not necessary).
Wait really? I'd expect the inserter condition to become '0 < 0' when the chests are empty. I'm probably misunderstanding something.
Yeah, really :)
I can try to explain (I didn't figure this out, I don't know who did, but I read about it here):
You connect all the chests (that are to be used for even loading) to the arithmetic combinator input. The combinator you set input to be "Each", output to "Each" and to divide by 0 - chest_count, so -4 in this case.
Then you connect each inserter to their respective chest and the arithmetic combinator output. The inserter check if Everything < 1.
The way this work is that the signals will be summed together, so the arithmetic combinator produce the average amount of items as a negative number, while the inserter sees the amount of items in its respective chest as a positive number. So if the arithmetic combinator produce "-50" and the chest contains 40 elements, the sum will be -10 which is less than 1, and the inserter will put items into the chest.

Oh, another thing I discovered while doing this:
Everything and Anything behaves opposite when there are no signals sent. An inserter with condition Everything will be enabled when there are no signals sent, an inserter with condition Anything will be disabled when there are no signals sent. Kinda peculiar, but sometimes useful.
Top
canidae
Long Handed Inserter
Long Handed Inserter
Posts: 72
Joined: Thu Feb 18, 2016 6:24 pm
Contact:
Contact canidae

Re: Nonblocking, expandable buffer

Post by canidae »

Updated blueprints of buffers in first post. As I wrote about Everything and Anything in previous post I realized a minor improvement to the buffer design (using Anything instead of Everything on input to buffer to prevent items on nearly empty input belts to take a trip into the chests before returning).
Designs. Must. Be. Perfect.
Must. Optimize.
Top
Linosaurus
Long Handed Inserter
Long Handed Inserter
Posts: 89
Joined: Thu Jun 11, 2015 5:50 pm
Contact:
Contact Linosaurus

Re: 4 Lane buffer (full throughput, expandable, fairly compact)

Post by Linosaurus »

canidae wrote:Then you connect each inserter to their respective chest and the arithmetic combinator output. The inserter check if Everything < 1.
Great explanation, thanks. I forgot about the divide by -4 trick. Saw it a few months ago, probably the same post you linked, but not the method I used myself.
canidae wrote:Oh, another thing I discovered while doing this:
Everything and Anything behaves opposite when there are no signals sent. An inserter with condition Everything will be enabled when there are no signals sent, an inserter with condition Anything will be disabled when there are no signals sent. Kinda peculiar, but sometimes useful.
Yeah that is surprising, but the difference makes a certain amount of sense if you think of them as the base value of a recursive AND/OR operation.
Top
User avatar
hansinator
Fast Inserter
Fast Inserter
Posts: 160
Joined: Sat Sep 10, 2016 10:42 pm
Contact:
Contact hansinator

Re: Nonblocking, expandable buffer

Post by hansinator »

Which mod are you using to fill and clear belts?
Top
canidae
Long Handed Inserter
Long Handed Inserter
Posts: 72
Joined: Thu Feb 18, 2016 6:24 pm
Contact:
Contact canidae

Re: Nonblocking, expandable buffer

Post by canidae »

hansinator wrote:Which mod are you using to fill and clear belts?
"Creative Mode" is the mod called. Great mod for testing stuff like this.
Top
User avatar
hansinator
Fast Inserter
Fast Inserter
Posts: 160
Joined: Sat Sep 10, 2016 10:42 pm
Contact:
Contact hansinator

Re: Nonblocking, expandable buffer

Post by hansinator »

This mod makes testing a lot easier.

I've designed a compact inline buffer for blue belts a couple of days ago and wanted to add control to it like you did. It should be in-line to save space.

Here is the basic version:
Image

It is supposed to buffer and fill 2 blue belts. However, to fully buffer the blue belts you need to cascade two of them.
I wanted to use an overflow sensor to enable the inserters to control loading and unloading. As it turns out, it needs two belt jam sensors, one for loading and one for unloading.

Like this:
Image

The first two belt jam sensors enable the unloading belts if the outputs are not stalled. The two sensors at the end enable the loading inserters if the belt is stalled. The two intermediate belts connected with green wire are just hops so I don't need poles.
The problem when using just one belt sensor per side is, that the unloading causes it to overflow which in turn will activate the loading inserters when they should be inactive.

Any ideas how to get this more compact?
Top
User avatar
hansinator
Fast Inserter
Fast Inserter
Posts: 160
Joined: Sat Sep 10, 2016 10:42 pm
Contact:
Contact hansinator

Re: Nonblocking, expandable buffer

Post by hansinator »

Here is the blueprint string for the basic version

Code: Select all

H4sIAAAAAAAA/61X0W6DMAz8lYpnMpFAu4eJb6kYWF20NEFJkFoh/n0UbVM7CDH1nipVvotj313TxuyUqSu1O5Y9aC+9BFf2va7OUCZwaS04x7yttGuN9ewd
lE/S1rix0Oiyv5TskF5Llg9pIy3U07diSLcQ7KkEBZVAUAlyKgEnEmREPPV86gSpA6RKgKzB1xsBvycofgmcr+pPJrUD68Eu2Ycvn+w8gGL1B7hFz/ABUVnM
KleamYoFsnj/p/gMjezODNR4Dytr1hoFSz65g+DdKaj2xo4gn+3jMDut0w3YkzXjZ9DMjw2n/tqOBFK3nU9QjWSzVQQ2nG25HV9RW3x/YoM8OLZ/vqF/gVhO
NCYC10fjxbLLceLIg9ownX8QBzo2A6nzg3etkn42ycm82ct+fRYRod+yK0PoHP2OIKxmSuCM+AyhnF+E8dEYzZ744SjCOOT6RGh9z0hxCk7K/EQcj8hdjBxj
sRuZauwdRhkCf05Es5tv63rlVOzjjxiqEePGM/XfdLxiY2wiE+ALITSksjb69qfMyZOuVNl/i9rDOUlXqUfoOLhxv8MwvFnwndW74xvo5gsVSKlz9g0AAA==
And the controlled version:

Code: Select all

H4sIAAAAAAAA/+1ZYW+bMBD9KxWfYQoQCFHH/sg0IQq3xBoxyDZVI5T/Pic02gyG2LlUS7T2S5qYO/C7d3d+R1k/VXWRV09Z2gEVRBDgadfRfAepA28NA849
wXLKm5oJ7wUq4bhNzeWFNU27t9TzfXefesHBLQmD4vSz/GLlYTHjgYu8+OURyoEJYAPL5GjoH9yipoLVVfYC2/yV1CztCsKKlohMrpTv1/8kjIuMkw3Nq7QT
+0Z6fyVMtHnluO83Oy16Od2LLaEb5+RZPgEVaSL/3TU5y4V073xzDqc12j+vhOy74zs/0m7DAOS9eiz3GSnT1eH4ZwVIgkV0PeNgByVpdx5UcoWRwmvqCgbm
cQ+rWQxWDxCDUG7lzzd/aR+SGBuSFdbBEusguokDf95BS0tgG1bLz8lNqC7cngWENq1wzCgXPgDlVirlIoVys7sLHmB3MoX+3l4Q2GdUgKVjiCty/qDImbUq
5EPPdTqzFFpMZlDdCiWFTJ5HE4QZ3gHNXyrISsKPn+nPvOLgnhcZ5GW2zWl5pKeQ3OCDC84/91fu6hLShTHfIoVvYWjNN2zk/BHuSyv7ObYa2/umkbqqQsAr
sHGNWCs14utxSU8HwdoPYQODUuFCkGiDz5uKiFE17RvOl+h64DVd74YpooCmwcQ3xCRW82NhAVF8G4g+rIrcCiI9beb2pTmz/Sc5pzlC6w/NekolaEqt7zHr
xqgEgZp39spCI9XucaOhKqFC34INvb5H0eFGQ4bF9f27dzChfrgAqLxiC1wn5ReGB//18AktpiDqyeSutcMy0nLngnJcj9CfEo5ToVhZhCIehiI2H4Y8Uih8
JaujwFylLqfJajxMWFizYGnIAkN30ZS7oZaaZlU4ZNXUhYEF/cIHo5GqzqLQnEbBY+00Uvt9om+DlxW8IelmgPOnK5TpIOJCN9Q38xONUa0cVzj86S5sKqM1
ZeeSyafwlmyP9VOXaeGNFpX3JgBGiKj9M7EBKDYBaFpeodXVvWE7LrbJAF115pzoj5EzEgQFWf+eE61AMA4SpP0aab9C2sdI+yXSPkLaB0j7EEsgnD2Sfsi7
j8G7yxcNsTpIjexfNCCjPCbpvy/NhgeBS6PUz0nqmWLXDAnvjRQ32ZiuqR5cUpxu1KlhJwJ255hP+JamUnpJt/IxnhmIltGn7Blo+RvW/Oho5yQAAA==
Edit: I updated the controlled version with the proper blueprint that works.
Top
User avatar
hansinator
Fast Inserter
Fast Inserter
Posts: 160
Joined: Sat Sep 10, 2016 10:42 pm
Contact:
Contact hansinator

Re: Nonblocking, expandable buffer

Post by hansinator »

I have improved upon my last design and built a universally throughput unlimited 2 blue belt in-line buffer. Picture below.

Image

Key features
°Only buffers excess items
°Universally throughput unlimited (it never ever stalls or fails to fully compress the output as long as buffer is not completely full/empty)
°Always works, never fails, no setup required
°Loads and unloads from/to two blue belts
°Compact size (51 x 7 tiles), can be used in-line
°Stores 38400 items (8 minutes if both belts are full)
°Visual indication of what it is doing for each input/output belt
°Chest loading and unloading is balanced
°As symmetrical as I could get it

Drawbacks
° Only blue belts
° Only two belts
° Blue is expensive

Lamp display colors
°Red: buffer full/empty
°Yellow: buffer is loading, output belt is stalled
°Green: output belt is moving (either unloading buffer or pass-through input) and buffer is not full/empty

Description
The basic principle of this buffer is that it is in-line and only buffers when the output belts (right side) overflow. If the output belts do not overflow it doesn't do anything but pass the inputs to the outputs. It further guarantees that the output belts are always fully compressed unless the chests are empty. If an input belt is only partially saturated it will unload just the right amount from the chests to fill up the outputs. When an output belt is overflowing (or moving slowly and blocking intermittently) it will switch from pass-through or unloading to loading the chests. It guarantees that you can always feed in two blue belts at max speed and full compression.

Even if it is switching between loading/unloading rapidly there are never any glitches like uncompressed output or stalling input belts. It doesn't matter if inputs or outputs are balanced. However ill-arranged the items are on the input belt, they will be passed through and all free spots are filled seamlessly with material from the buffer. Passing through all input makes sure that the inserters are only active when it is necessary.

Technical explanation
It basically consists of 4 subsystems which are:
1. Buffer unit
2. Belt compresser and unloading flow control
3. Output stall detector and low-latency buffer
4. Lamp color control

Image

The buffer unit has two identical subunits with their inputs cascaded in series and their outputs in parallel. They consist of 4 loading inserters plus chests per side and 4 unloading inserters. The chests are shared for two belts to make the whole thing more compact. Unloading inserter arrangement makes sure both lanes of a belt are evenly filled. Loading the chests is balanced via combinators to make sure the left inserters don't starve the ones to the right from them. Unloading chests is also balanced via combinators and additionally by splitters because they are more reliable under certain circumstances.
If it would not be balanced some chests may fill up or empty sooner than others and thus don't contribute to the throughput anymore because their inserters become inactive. I am using the balancing circuit from this thread viewtopic.php?f=193&t=23125. It is the two combinator solution with a few modifications. First mod is that I am adding (chest_count * chest_count) - chest_count instead of just the chest_count in the adder combinator of the loading curcuit. This is because the original balancing circuit made those stack inserters work so slow that they weren't able to catch up with the belts under all circumstances.
The effect is that it produces a slight but unnoticeable imbalance that encourages the inserters to work harder ;) . If I would have used the 1 combinator solution from MadZuri (also in the linked thread) I would need a constant combinator with resource specific programming to create the aforementioned imbalance. The balancing circuit has additional control signals such that the inserters are only turned on when their matching output belt is stalled.
Between both buffer subunits there is a lane rebalancer for each belt that always takes equally from its input lanes. It is necessary to prevent the inputs from stalling. The first 4 inserters on each belt will only take from the lane close to them. Without the rebalancer the last 4 inserters would also draw exclusively from the closer lane which may lead to a backlog on the far lane. Eventually the far lane would stall a tiny bit in regular intervals. With the rebalancer the problem is sufficiently solved. I used this type of rebalancer because I like its design. I found it here https://imgur.com/a/sgAsj. I also believe its more stable in always taking equally from both input lanes.

Following the buffer section there is the output flow control and belt compressor. This is basically a simple belt-jam detector like you have seen it from other people. However, it has two inputs and two outputs where the outputs merge to one belt via sideloading. The pass-through input input together with the buffer output go into the first splitter. The belts inside the splitter are wired together and set to read contents hold. The buffer output that goes into this splitter has a wired belt that is only enabled when the amount of items between both splitters is less than 10. A value of 9 would suffice for a belt jam detector circuit. 10 is used here to make sure the buffer output fills the belts between the blue splitters always a little in excess of whats necessary to fill one blue belt. This way the construction doubles as a low-latency buffer. Because the exit splitter also has both of its outputs connected to belts it can actually pass through the excess items from between the splitters on to the sideloading circuit with a capacity of two blue belts. The result is a little overpressure on the sideloading of the output belt which leads to full compression in any case. Hence the name output belt compressor.

The output stall detector gets the one compressed output belt from the compressor as its input to the first splitter. If the output lane is stalled, the space between the splitters fill up with more items than one belt can carry and a stall is deteced by combinators. The loading inserters for this belt will turn on when there are more than 8 items between the splitters. But there's more to it. If the output is flowing again this jam detector will not be able to push out its excess items unless you stop the input belt. It does this such that the input is stopped until the count goes below 9 such that the inserters get deactivated when the output flows again. Now if we only take slowly from the output belt and it is stalled and moving just a little (imagine a fast inserter taking from the output into a chest for example) then the loading inserters oscillate between on and off very quickly and as a result will work slower. Eventually the inputs would stall. This is where a hysteresis comes in (thanks XKnight, viewtopic.php?f=6&t=28899&start=50#p189322). When the output is jammed the space between the splitters will actually fill up to 16 items before the its input belt is deactivated. But it will only activate again when the item count goes below 9. So this doubles as a low-latency buffer too. It builds up a little overpressure when the output is stalled and if the output is only drained slowly it first releases its excess items and then activate the input again to re-fill. This still oscillates but at a much slower speed.

Finally the visual indication. Each belt has a lamp that can light up red green or yellow. I am using color signal precedence here. There are two combinators which compare against buffer full or empty conditions with fixed constants and will output a red signal if such an condition is reached. Red will override all other colors so if the buffer is at its limits it will always show red, even if the input is in pass through mode.
The circuit that enables the loading inserters actually internally uses signal yellow which is tapped to control the lamp. Yellow has the lowest priority. The green color signal is controlled by the hysteresis function of the output belt stall detector. If the belt is not stalled the lamp will show green. Green overrides yellow so if the buffer is unloading and loading at the same time (happens when just one output belt-side is blocked) it will still show green. Actually the hysteresis adds a little bonus to this. When loading and unloading at the same time the lamp will alternate between yellow and green. It blinks faster if the output belt is moving fast and slower otherwise.

One last thing. The inputs are balanced to avoid situations where the upper belt overflows and loads and the lower belt is starved and unloads. It works but not flawlessly in rare cases, so this is completely avoided by balancing the inputs. If you already have balanced inputs, you can delete the very first splitter on the left. The outputs are belt and lane balanced to compensate for unequal loads on the output.

I designed this for my new factory where I want to use a lot of these to buffer between train arrivals.
Blueprint
Top
Post Reply

Return to “Combinator Creations”