Logistics Network/Robots needs new priorities

[Sharidan]

This can not be easily confined within the "Read or be ignored" post guidelines, sorry.

Game version: 0.10.6

Problem:
Overproduction caused by logistics network / logistics / construction robots choosing to pickup from production provider chests, rather than picking up from storage chests.

Scenario:
I've got an assembling machine producing fast inserters, which are dropped in a passive provider chest for my robots to pickup and keep my toolbelt filled to a certain amount as well as for construction.
At times I use an active provider chest to clean up my inventory, by dumping everything in it. Robots pick up the items and drop them in storage chests.
Next time I need my toolbelt restocked, the robots go straight to the passive provider chest by the assembling machine to get more fast inserters, rather than going the shorter distance to the storage chest that already holds some fast inserters.
Parts of this scenario also involved constructing using blueprints and deconstructing things, forcing the Construction Robots to drop everything they deconstructed into storage chests.

Result:
I now have some 1.3k fast inserters sitting in a storage chest.

The analysis:
Or the TL;DR; if you so please - skip further down to solution suggestion, if you don't care about the analysis.
It took a while for me to discover this particular problem, so once I discovered it I decided to do a proper analysis of this. I've been play testing this scenario and possible work-arounds for quite some time now. This is not just another one of those "oh I only got to <this> an hour ago". I've put in days of play testing on this particular setup, just to make sure I've got the problem figured out right.
As described above the robots choose to go to my production line passive provider chests to pick up the fast inserters, which holds true for the most part. At random they do choose to go the shorter distance to fetch the fast inserters from the storage chest, but I beleive that has more to do with lack of power for the bot, than an actual choice of going to the storage chest rather than the provider chest.
I have an area set up for stuff storage, where I also have that active provider chest I can drop my inventory contents into and let my factory and robots do the sorting and storing. At first I only had a bunch of storage chests there, but at some point I discovered that the robots hardly ever went to the storage chest with the fast inserters, but rather choose to fly the long distance to the passive provider chest at the other end of my factory. That told me that the current code design of the logistics network chests has a higher priority on the provider chests than on the storage chests. The work-around I came up with is to have a double line of chests; storage chests on one side of a belt for the robots to drop off into and passive provider chests on the other side of the belt. I've setup smart inserters to filter things dropped in the storage chests so only certain items go into the passive provider chests I've got setup in a circle. "This should solve that problem!" was my initial thought. I soon realized that it didnt solve the problem at all.
Logistics (and Construction) robots go by place-order. Their first priority when picking up items, appears to always be passive provider chests first and which ever passive provider chest was placed first, will be the one chosen. If none of the passive provider chests holds the item needed, they will then search through the storage chest again in place-order. On a few of my production output passive provider chests, I've removed them and replaced them with new ones just to test if this really holds true and it seems that it does. For certain items in my storage circle they now choose the passive provider chest at the circle, rather than the one by the assembling machine. The only real work-around to this particular problem, would be to place storage chests as output chests on the production line, but that would contaminate those output chests when ever Construction Robots are ordered to deconstruct something. Yes, I have test this too, however it does work slightly better.

I'm guessing that the robot choices are done by looping through some sort of array that holds a list of the different logistics network chests and which ever chest comes up first with the items needed, will be the destination of the robot in question. Their behaviour seems to indicate that this is the case. Usually if I deconstruct something (robots drop off in the nearest storage chest or a storage chest that already holds "that" item) and then immediately after order construction of something by placing down a blueprint, the robots will pick up from the same chests they have just dropped stuff into. After a while though this "short term memory" array gets cleared (probably some form of garbage collection?) but on more than one occation they have still chosen to go to my production line (which is far away), rather than picking up from the storage chest only 10 tiles away where they minutes before dropped the same materials off.

Solution suggestion:
Swap the priority of the storage chests with the provider chests, setting the storage chests the first to be searched, then the provider chests. This would effectively force the robots to pick stuff up that they had previously stored in storage chests, rather than forcing production of new items by grabbing them from the production line.
At some point each robot has to calculate it's line of flight to the destination chest. Rather than checking the place-order of chests (provider / requester / storage) it would make more sense to have the robots go to the chest with the shortest distance, once the chest has been chosen (still holding the same chest type priority: storage first, provider second).

For robot pick ups (either to replenish the player's inventory or for construction):

• First priority: Storage Chest
  Second priority: Provider Chest
  Pick the chest with the shortest distance
  Possibly add a filter option (or research upgrade?) to storage chests

[ssilk]

You're right with your analysis.
But I think the scenario is wrong. I repeat: You put all your stuff into an active provider and let it automatically restock. Well I make that too, but only with stuff I don't need. I've several active provider all around for that case, cause it is extremly bothersome to handle a full inventory. In most cases I let 2-3 stacks free, exactly for that case - which is a waste.

This was the last change of the priorities: https://forums.factorio.com/forum/vie ... 824#p31824
The reason, why the storage chest is not the highest priority is, because it is thought for keeping the stored items in the storage - as long as possible.
Now I deleted about 40 lines in this post, because I found out, that I need to think a lot more about it.

Essentials of my thoughts now:
- It is not clear for the player, which chests or combinations of chests should or can be used for which case.
- This is complex, cause there is not only the types of chests. It is also
--- the distance of them and between them
--- the amount of content in them
--- the problems with leftovers, cargo stack sizes,
--- unusual behavior of the player, see above
--- needs for distribution near (see for example https://forums.factorio.com/forum/vie ... f=6&t=5007) and far to distant outposts
- Not everything, which can be done with logistic bots makes sense. The game should pull the player a bit back to the right direction.
- On the other hand: Not everything, what can be made with logistics has been tried yet. I think it takes a lot more time to find that out.
- I have this vision of "central places". Something like a castle. There haven been made some suggestions about such a place. And logically the storage is inside, cause that it is a main target of the natives.

[Sharidan]

Forgive me for lacking the ability to fully explain the scenario and for posting lengthy like this (I'm a bit of a TL;DR; poster, sorry). I'll try to correct a few things I missed in the initial post. You are absolutely correct ssilk, it does take some wrapping around the brain to get this right.

Backtracking through what I did exactly, this "problem" started before I set up my drop-off area.

I started moving around big chunks of my factory to utilize the much needed space I had conquered from the biters. So I used a bunch of blueprints to create the puzzle pieces I needed for the relocation and then started deconstructing everything section by section and moving it to it's new location. As far as I can tell, the Construction Robots will only drop off the parts they deconstruct into a storage chest. Correct me if I'm wrong, but if so please enlighten me how to make them drop off in different chests (unless requester chests have been setup to request the materials the robots are deconstructing). As far as I know they will not drop off deconstructed materials in provider chests.

This is where this whole scenario actually took off. After deconstructing sections of my factory and re-establishing parts of my production line (mainly: fast inserts, fast transport belts) the Construction Robots started leeching off the production line, rather than grabbing the parts they had previously stored in these storage chests. That's how this entire 1.3k fast inserter problem started building up. By the time I realized what was going on, I changed my behaviour towards the logistics network and the robots to try to combat their faulty selection of parts from the production line vs. the parts they had previously dumped in the storage chests.

Further down the line in this scenario the problem has basically just escallated. I can stand inside my storage circle not 10 tiles away from two passive provider chests that contain some 1k fast transport belts and maybe 15 tiles from one passive provider chest that holds the 1.3k fast inserters, yet when the robots replenish my inventory, they still go to the production line far away, rather than grabbing those closest to me. This behaviour the robots are exhibiting seems odd to me. Now, this particular thing was set up after I realized that the robots will only grab from storage chests, when no provider chests holds the items in question. Mind you, that I did spend days play testing this before I posted this thread and I have in fact tried over 100 different setups trying to adjust my behaviour and the behaviour of the robots, yet have come up empty on a solution.

The current layout of the storage circle looks like this (after the changes I made with misc. inserters, belts and filtering etc.):


In the following screenshot of the map, I've marked the two spots of interest. The green dot lower-left is where the storage circle is as depicted in the above screenshot. The green dot to the upper-right is where they pick up inserts and transport belts and where they above all else choose to go pickup these items.


As you can see in the first screenshot, there are at the making of the screenshots 3 different types of inserts available at the storage circle, aswell as plenty of fast transport belts. In spite of this fact, even when standing right there as indicated in the first screenshot, the robots persitently choose to fly off to the far reaches of my factory where the production line for these items are.

I'm fully aware that there are tons of different ways things can be combined, but I do feel that I have a firm grasp of how these chests should be utilized after days of strictly play testing these aspects of the game only. The only way I can trick this system into changing it's behaviour is by (in this order):

• Tear down the production line of these items. Sometimes the assembling machine too.
• Tear down all the storage chests and the passive provider chests in the storage circle.
• Place down the passive provider chests, one by one, setting the filter options for them for these items.
• Go back to where the production line was and one by one rebuild the production line.
• Go back to the storage circle and place down the storage chests again.

The result of this process is that the storage chests end up getting placed last, but that also appears to change which of the storage chests get selected first when robots (both Construction and Logistics Robots) pick a chest to drop stuff into. Since the passive provider chests in the production line will be the last ones placed for this item, those placed inside the storage circle will now take priority over those in the production line, thus forcing the robots to go to those chests first and then the ones in the production line. I have already been through this process quite a few times (20+ times) to test that my thesis was correct and it appears that I'm getting the same results every time I do this. I have lost count of how many times I have stopped and gone back a few savegames to retry the different aspects of this.

The first post you linked me describes a change to the priority of the provider / storage chests. The new priority listing is supposed to be:

• Active Provider Chest
• Storage Chest
• Passive Provider Chest

This priority listing is the way it should be, however in reality it appears that it's not, hence the priority problems I'm trying to describe / explain here. If you take a look at my map screenshot again and go further south-west from the storage circle. In that area down there I requested my construction robots to build parts of my factory down there by placing down the blueprints as one would normally do. I know for a fact that there were plenty of materials available at the storage circle at that time (I've repeated the several times to properly test it). Remember that the production line for all the belts and inserts are where the upper-right green dot is. At the time the passive provider chests at the production line could only deliver 50 fast transport belts and 50 fast inserters for the construction. The blueprint demanded some 300+ belts and roughly 200 fast inserters for the construction. For what ever reason that escapes me, the construction robots persistently went to the passive provider chests in the production line and waited for the production line to finish producing the needed materials, rather than going to the passive provider chests in the storage circle to pick up the remaining needed items of which there were more than plenty available at the storage circle. Once they finished construction I went to the storage circle to check those passive provider chests and the robots had not picked up a single item from either of them. If the amount of items in a chest has some kind of influence on which provider chest is chosen I have yet to see proof of this. The two provider chests holding fast transport belts in my storage circle, each have over 1000 belts in them, where as the provider chest at the production line only holds 200 max. I have re-tested the idea that item amounts in chests has something to do with the choices, by removing most of the belts and inserters from the storage circle passive provider chests, yet I still saw exactly the same behaviour of the robots: "Yup, let's fly all the way to the production line even though I can pick up these items right next to where I am". As I've mentioned, initially I only had storage chests at my storage circle - something that definately did not work. That's why I afterwards chose to "upgrade" my storage circle to the layout you see in the screenshot. This does not seem to work either, unless I go tear down every single provider chest in my entire factory, including those in the storage circle and then rebuild all of them starting with the storage circle and finally placing down those at my production lines.

Please note, that I'm not new to logistical systems and the one in Factorio felt right at home from the get-go. I've designed quite a few logistical systems both in real life and in other games. For the most part the logistical system in Factorio runs very smooth as long as the player doesnt inject items into the system through any of the logistical chests. As soon as blueprinting, deconstruction and therefore also storage chests get involved in this, things start to get overproduced, because the robots appear to rather want to go to the production line provider chests to pick up new items, rather than re-using those they dropped off in storage chests not that far away. Since a production line is a rather involved "entity" it usually catches up to the grabbed materials and replenishes them fairly quick. Moving these items from storage chests to provider chests makes little to no difference, since the production line provider chests were placed first. Injecting items into the system through storage chests while deconstructing starts messing things up. The same holds true when injecting items into the system by dropping them off in provider chests. I've tried using passive provider chests to drop off stuff, aswell as active. The result is pretty much the same. The only work-around I have been able to find for this, is the above described process of tearing chests down and rebuilding them in that specific order.

[micomico]

• Coloured logistics networks - define colours in roboports AND logistic chests;
• robots assigned to a coloured roboport can only satisfy that logistic colour(s) network(s);
• filters in logistics storage chests, with increased priority for the robots when filters are set, both in storing and getting items;
• robots respecting previous priorities.

This will solve most people legit problems and also the ones they come up with. All the latest suggestions for logistic robot behaviour would be "fixed"/"satisfied" by these features being added.

[ssilk]

Pfff... hard work to read through all of that.
Thoughts:
- Very nice idea, this storage circle.
- I doubt, that Factorio makes a difference between chests near at a production site or that are inserts via player. What counts are only the contents.
- My idea for that is, that the logistic network is thought for equal distribution. That means in my logic, that there are sometimes situations, which doesn't make sense, cause of that. Maybe this is one.
- But maybe it's a bug? Who knows?
- I promise to reproduce it, cause I think in this current form this can't be understood. Maybe it takes a while, you should meanwhile try to break this down to the most simple example: Built a own logistic network (not connected to the other) and make an example with one type of item, one chest and explain in (one ) word. Sorry, but I think you see yourself, that this post is hard to read - and understand.

[Khyron]

Problem:
Overproduction caused by logistics network / logistics / construction robots choosing to pickup from production provider chests, rather than picking up from storage chests.

Solution: Use smart inserter to throttle the production of items entering the logistics network.

Result:
I now have some 1.3k fast inserters sitting in a storage chest.

If you use a smart inserter to feed the provider chest beside the assembly machine you can set it to stop moving items once there are, say, 20 inserters in the logistics network. Then it doesn't matter where the inserters are stored. Once the provider chest is empty, the bots will be forced to collect other inserters from the storage chest.

In the screenshot below, the smart inserter is set to move any items it can reach so long as there are less than 20 inserters in the logistics network.



Edit: you can also minimise this problem from happening in the first place by using a passive provider chest by the assembly machine. That will prevent logistics drones moving inserters to the storage chest. Then the only way that some inserters should arrive in the storage chest is if drones attempt to deliver some inserters to you but you move out of range of the roboport. Obviously this will change the behavior and it's up to you whether you want to involve the storage chest in the flow of inserters by design for whatever reason.

[Sharidan]

I'm aware of how the red/green wires are used to control the production of things. Using the red/green wires with chests and inserts to control production only slows down this over-production, but it doesnt solve the fact that the robots fly off to my production line first, leaving the items at my storage circle behind. The red/green wires can only lessen the surplus items. I have already tried setting up some 40+ different such control mechanisms, still without huge success. The major problem with the robots / chests in their current design is how they choose which chest to pickup items from. I've replayed this savegame from the point where I realized this over-production in almost 300 variations to try to determine a way to "fix" this problem. The red/green wires work fine as long as you just need to control the chained production of items and how ever many of each you need in provider storage, but throw in deconstruction and reconstruction using the robots and miscellaneous chests plus doing and redoing the same blueprint a few times (place it down, realize it has to be moved so deconstructing it and then placing it down once more - repeat 5-10 times) and that's when things become weird and over-production starts happening.
The only way I've managed to get rid of this problem, is by not having any other robotports anywhere near where I'm constructing/deconstructing using red/blue-prints. But that then presents another problem: getting supplies for the construction robots to actually build the blueprint.

I'm sorry if I wasted your time guys. Thanks for reading and dropping some feedback though! I'll drop my brain back in it's jar and see if I can somehow break this further down. Behaviour analysis and explanation is rather complex.

[Khyron]

@Sharidan
I'm not sure if that was for me, but my suggestion does not include the use of red/green wires.

Edit:

The major problem with the robots / chests in their current design is how they choose which chest to pickup items from.

I think I disagree at a philosophical level. I would prefer drones collect items from provider chests as part of normal operations. If something breaks the flow of items, then drones can continue to supply part of the factory by drawing from the storage stockpile. I don't see storage chests as part of the normal flow.

Long ago I had a casual job working in a factory that produced magazines. In the bindary, each magazine was assembled by a two part process: one part to glue together folded sections and another part to cut the edges. Either part could operate without the other, and more importantly, each part had different speed limits depending on the circumstances of the magazine. Sometimes we would have to take untrimmed magazines off the belt connecting the two machines and store them while the trimmer was being repaired or adjusted. Later, once the trimmer was running again we could slowly reduce our stockpile of untrimmed magazines by re-injecting stored, untrimmed magazines back on to the belt, and then running the trimmer faster than the binder. Alternatively, the crew manning the binder could go home while the hopper and stacker continued operating. The parallel here is storage chests can store the untrimmed magazines.

I think my perspective is only viable if you use smart inserters as I described above though, so if you haven't tried that I guess my way might not make a lot of sense.

[Sharidan]

@Khyron
Well, I've tried God knows how many different combinations, both with the red/green wires and without them, but thank you kindly for the suggestion anyways. I have already tried setting values in many different ways

[Khyron]

@Sharidan
It would be strange if you took all that time doing testing and then all that time writing out those massive forum posts to quickly dismiss any answers given to you.

[Sharidan]

@Khyron
Argh. I put my foot in my mouth again. No my friend, I'm definately not dismissing anything. In fact I've taken all of it very much to heart and started another replay of this weird acting savegame to test out ways of implementing all these great suggestions with what I've already tried. This feedback is worth a lot more to me than I let on: I've got new ideas on how to try to stop this weirdness from happening, so definately something I'm taking very much to heart for the next 50 replays
I'm sorry that it came out as being dismissive. That was just me being socially stupid as usual.
One of the key changes I'll make to my behaviour towards the logistics network / robots, is implementing the network wide limitations at a much earlier stage. This is an idea based off your suggestion Khyron, one that I have not yet tested stand-alone. So a huge thank you Khyron for throwing that idea my way

[micomico]

If you're trying to do what's in the first picture, to have the inserters available to the next machine in the assembly line and to you via logistics, you'll end up in two situations:

A - You limited the production by setting a filter in the chest and let the smart inserter move freely. This leads to your situation. Robots will pickup stuff from the chest and the machine will produce more. Lots of excess pile up in storage chests.

B - You limited the production in the smart inserter with the logistics condition (if you do it with circuit logic to just check the chest, you end up in A). This way you risk this chest being empty, due to items existing in other parts of the logistic network, stopping the assembly line.




Try using the following design. You can limit the storage available for the assembly line by setting a limit on the wooden chest and you can set a limit to have available in the logistic network in the smart inserter.

With this, even if the robots empty the provider chest, the smart inserter won't pull anymore from the assembler while items are available in other locations.




Keep in mind that some overproduction will always happen, no matter what method you come up with. When you send you construction robots do their work, the items they carry are not in the network and production will eventually start according to the limits you set.

[ssilk]

moved from suggestion to here cause it seems to be the be placefor it -- ssilk

[Sharidan]

So, I've had a chance to replay the scenario a few more times with some additional changes to the network limits based off Khyron's suggestions. I implemented the limitations at a much earlier stage which seems to reduce the over-production a lot more.
However the new limitations have not changed the order of pick up points the robots choose. They still go to the production line first, because I established the production line provider chests before I established the provider chests at the storage circle. Robots still appear to go by place-order of the provider chests and they do pick up from provider chests before they pick up from storage chests. Of course once they run out of material at the production line, they start heading to the storage circle to pick up items. Once the amounts drop below the limits, they appear to start toggling between the production line and the storage circle to pick up items. The design for the storage circle I've posted a screenshot of, which includes the provider chest filtering, still appears to the best working solution. I beleive ssilk said that the robots prefer picking up from provider chests before storage chests to conform with the idea of using the storage chests for long term storage.

Behaviour of the robots still appears to be place-order of the provider chests if two or more provider chests provide the same item, meaning they empty out the production line before they empty out the items stored through deconstruction. The only solution I have seen working, is the tear-down / build-up in reverse order that I described earlier. They choose the longer fly distance to the production line over the provider chests 10 tiles away. If the robot behaviour was altered to pick the closer chest rather than the first-placed chest, the over production would be reduced further.

Final results of all this is, that I'm now left with only 10-30 items over produced overall (depending how many speed upgrades I've got installed) and by tearing down the provider chests at the production line and the storage circle, then placing down the provider chests at the storage circle first and the production line last, I'm able to influence which provider chests the robots pick up from first; thus forcing them to pick up more from the storage circle than from the production line.

@Khyron
Thank you for your suggestion. It pushed me to initiate the limitations network wide at a much earlier stage which does reduce the over production the best, compared to all the other combinations I had tried.

@micomico
Yup, that was the initial model I was running with and I let play out after I realized what was going on, resulting in the 1.3k fast inserter overproduction. As you correctly mention, the items are not registered in the network while robots are carrying them, however at times they show up as negative values while robots are carrying them. This can be seen if you mouse-over a chest in the network. Every now and then the items they are carrying will show up with a negative value. This is especially visible if you blueprint order construction of something and you place down build orders that exceed the current limits of your production limits. For example: my production line is set to a limit of 100 solar panels. I've got a blueprint that places down a substation with 24 solar panels inside the substation's range. Place down 5 of these blueprints for construction. Once the 100 solar panels already in the network are spent, they will start showing up with negative values in the network contents list shown when you mouse-over a provider / requester / storage chest.
While robots are carrying the items and they are not registered as being in the network, there will be some over production yes, due to that fact. This over production is not really worth worrying about as it's not something that will grow out of control.
Thanks for adding these insights to the thread - you saved me the trouble of doing the same. I was about to do the same today, now that I've had the chance to replay my scenario with Khyron's network wide limitation ideas.

@ssilk
The storage circle concept works rather well. Even with the active provider chest to drop off inventory junk. With the latest model which includes the network wide limitations set right at the beginning and the tear-down / build-up of the provider chests (always setting the provider chests at the storage circle the first ones placed!), the over production really is very small and the robots will in most cases refill my inventory from the storage circle while items are available there. Having to rebuild the passive provider chests at the circle and production line is a little bit annoying, but it does seem to alter the behaviour of the robots enough to have them go pick up at the circle before picking up at the production line, which is parts of what this thread is about. If you choose to adopt the storage circle setup, be sure to add the smart inserters and the passive provider chests as in my screenshot. It will not work properly if you only use storage chests.

Thanks again guys. I now know how to work around these limit settings and robot behaviour problems.