Smart Train Deliveries with Combinator Magick [0.13/0.14]

[siggboy]

SmartTrains is missing from the list and the Radar Mk II is a part of SmallFixes but it needs to be enabled in the config.

[doxsroxs]

New update, I have somehow resolved the issue with the train stops getting replaced by the migration script.

This is what I did:
Checked migrations scrips in mods, they where irrelevant since they all basically said the same
Checked all mod files for relevant phrases in relation to the replaced items. Found none that matched...
Removed all mods and put them back a few at a time to check who was the offender

The funny part is I didnt get the same migration error when loading the game.
Basically I still got all my mods and the demo works.

Either the game somehow keeps track of the order of the mods and that screwed me over before and now fixed itself when I added them in a different order.
The other case is of course that Im more drunk then I think I am... totally possible!

[doxsroxs]

SmartTrains is missing from the list and the Radar Mk II is a part of SmallFixes but it needs to be enabled in the config.

Thanks! Thats a very interesting observation there. I posted my post above without seeing your post...
Empirical evidence thus point toward the "Im drunk explanation". Ohh well, time for another beer, and thanks again for the help, Im sure Ill run into more issues once I start picking this apart.

Best part is I actually compared my mod list in excel with the one I posted before and apparantly managed to paste and compare the wrong lists, or I would have spotted it as well

[doxsroxs]

A new update. Apparantly I was cheering to early

One of my mods seem to cause something to break, have not identified it yet, but with all mods in some trains will get stuck loading until completely full and it somehow breaks the logic built into the map.
It works with your supplied mods though, so Im just going to go through the list again until I can identify the suspect

[doxsroxs]

Ok, I have pinned the problem down to Bobs mods. I assume there is something in the mod that makes the counters go wrong seing as it adds a lot of new materials.

The trains will go where they are supposed to, but will not stop loading at 8k, the problem does not affect the barrels, only coal and iron. Once they are full they depart and then it just seems to fall apart more after that.

Edit
I have removed Iron and coal from the combinators as per the instructions, but the trains still seem to get stuck loading to much instead of the requested amount...

[siggboy]

Ok, I have pinned the problem down to Bobs mods. I assume there is something in the mod that makes the counters go wrong seing as it adds a lot of new materials.

The trains will go where they are supposed to, but will not stop loading at 8k, the problem does not affect the barrels, only coal and iron. Once they are full they depart and then it just seems to fall apart more after that.

Bob's mods increases the stack sizes. That throws the system off balance, because the stack sizes are in a constant combinator.

I think I need to make a version for Bob's Mods, with all the ores and stuff in the combinators. I'll add it to the instructions anyway.

Here's how you can adapt it for Bob's Mods (for the demo map you only need to change the stack sizes as described in step 3, but if you want to use it in a game with all the new materials from the mod then you have to do all steps):

1. There are two groups of constant combinators. The left group has all materials numbered in increasing order, starting with Coal, Iron, Copper, ...
Add another constant combinator or two, and put the Bob's materials in there, continuing the numbering. Simply connect the new constant combinators to the existing ones. I think the last resource is Batteries, with number 22, so you continue with 23, e.g. "Galena 23; Nickel 24; ..." and so on. When you run out of space in the combinator, add another one, connect it with red wire to the existing ones. You get my drift.
2. In the counter directly below the constant combinator, change the number so it will count through all your resources. If you have 40 resources total listed, the number needs to be 41.
3. Go to the other group of constant combinators. Change all the numbers in there so they will list the number of items that fit into 1 train wagon. So it would be "stack_size * 40". Be aware that Bob's Mods also adds higher tiers of wagons that fit even more materials. So you have to change the numbers again if you use those wagons.
4. Add all the missing resources to the same group of combinators from step 3, always using the correct stack size for 1 wagon.

Yeah. Bob's Mods makes this a bit more maintenance heavy, but that's the price you have to pay I guess .

In any case, if you run out of space in the existing combinators you can simply add more and chain them together, it makes no difference how many you have.

[doxsroxs]

Ok, I have pinned the problem down to Bobs mods. I assume there is something in the mod that makes the counters go wrong seing as it adds a lot of new materials.

The trains will go where they are supposed to, but will not stop loading at 8k, the problem does not affect the barrels, only coal and iron. Once they are full they depart and then it just seems to fall apart more after that.

Bob's mods increases the stack sizes. That throws the system off balance, because the stack sizes are in a constant combinator.

I think I need to make a version for Bob's Mods, with all the ores and stuff in the combinators. I'll add it to the instructions anyway.

Here's how you can adapt it for Bob's Mods (for the demo map you only need to change the stack sizes as described in step 3, but if you want to use it in a game with all the new materials from the mod then you have to do all steps):

1. There are two groups of constant combinators. The left group has all materials numbered in increasing order, starting with Coal, Iron, Copper, ...
Add another constant combinator or two, and put the Bob's materials in there, continuing the numbering. Simply connect the new constant combinators to the existing ones. I think the last resource is Batteries, with number 22, so you continue with 23, e.g. "Galena 23; Nickel 24; ..." and so on. When you run out of space in the combinator, add another one, connect it with red wire to the existing ones. You get my drift.
2. In the counter directly below the constant combinator, change the number so it will count through all your resources. If you have 40 resources total listed, the number needs to be 41.
3. Go to the other group of constant combinators. Change all the numbers in there so they will list the number of items that fit into 1 train wagon. So it would be "stack_size * 40". Be aware that Bob's Mods also adds higher tiers of wagons that fit even more materials. So you have to change the numbers again if you use those wagons.
4. Add all the missing resources to the same group of combinators from step 3, always using the correct stack size for 1 wagon.

Yeah. Bob's Mods makes this a bit more maintenance heavy, but that's the price you have to pay I guess .

In any case, if you run out of space in the existing combinators you can simply add more and chain them together, it makes no difference how many you have.

Awsome instructions! Im pretty sure this should be no problem, Ill test it as soon as I can

I must ask though, would it not be easier to just read the max stack size for each item directly from the game and eliminate the need for the boxes?

[siggboy]

I must ask though, would it not be easier to just read the max stack size for each item directly from the game and eliminate the need for the boxes?

There's no way to directly read them. So I would need a mod that somehow makes them available on the circuit network. The system should not require any mods other than SmartTrains, so this is the only way to do it.

Lots of other ways exist to make the thing work without knowing the stack sizes -- this approach is taken because it does make setting up the outposts a lot easier; and you do not have to change the stack sizes after they've been configured.

Bob's Mods is a bit problematic because of the added tiers of train wagon, that add more cargo slots -- but in a Vanilla game there's no reason to even change the combinator settings at all.

[doxsroxs]

I must ask though, would it not be easier to just read the max stack size for each item directly from the game and eliminate the need for the boxes?

There's no way to directly read them. So I would need a mod that somehow makes them available on the circuit network. The system should not require any mods other than SmartTrains, so this is the only way to do it.

Lots of other ways exist to make the thing work without knowing the stack sizes -- this approach is taken because it does make setting up the outposts a lot easier; and you do not have to change the stack sizes after they've been configured.

Bob's Mods is a bit problematic because of the added tiers of train wagon, that add more cargo slots -- but in a Vanilla game there's no reason to even change the combinator settings at all.

Aha, I totally forgot that you didnt actually write any code to do this. Awesome work by the way, I actually think coding would have been easier, but that kind of makes this more awesome!

[siggboy]

Awesome work by the way, I actually think coding would have been easier, but that kind of makes this more awesome!

Of course coding would have been easier -- a lot. That's not Factorio then, that's computer programming, and I guess there are more interesting mods to write. Nobody wants a mod to solve their problems for them. This train scheduler uses only unmodded combinators, and the only required feature which is not in the base game is the one from SmartTrains that allows you to dynamically pick a destination station for the train. So it's still 99% pure Factorio .

[doxsroxs]

It seems I have run into some issues again.
When copying the blueprints from your demo map they somehow got distorted.

The smart stop signal output and the smart stop signal lamp where both included as separate entities in the blueprint while the station was not.
On top of this they where placed under the arithmetic combinators sitting just to the right of the station. I first encountered this with the depot station.

I also noticed all my blueprints since are a bit distorted, even new ones look off, with panels shadowed outside the blueprint, belts not aligning like they should etc.
I dont know if this is at all related to any mods I have installed or not, but trying to check through that now.
I do know the issue cropped up first when copying the stations from your test map.

[doxsroxs]

Noticed a button in Foreman blueprint manager that says "Fix positions". It breaks the blueprints in a similar manner. Starting to think that how it could have happened...

[doxsroxs]

Hi again, just made my first automated run.
Was a pain to set this up since a some of it involved me manually checking how and what everything was set to since blueprints got corrupted in the beginning.

I have my own buffers that I use to transmit the request for material, however, Im not quite sure how it works on the requester station since it looks like the original constant output box was connected to the output of another unit.
Made no sense, or maybe I was just tired. I have not gotten all the signs working either, and I replaced your counting system with manuals to get the run to work.

With some tinkering maybe I can get this to work properly, Ill get back with more questions later, now I need some sleep

[siggboy]

I have my own buffers that I use to transmit the request for material, however, Im not quite sure how it works on the requester station since it looks like the original constant output box was connected to the output of another unit.
Made no sense, or maybe I was just tired. I have not gotten all the signs working either, and I replaced your counting system with manuals to get the run to work.

First of all, I recommend you capture the blueprints again from the demo map, and then export them with the mod "Blueprint String". In your main game save you can then import them. You can also do it with Foreman or KBM, but I don't use these mods for that purpose right now so I cannot pledge for them working.

Don't use the "Fix Positions" button at all, it is only for importing old blueprint strings from other people. You will not need it, all it does is going to break your blueprints.

At the requester station, the constant combinator will need to output all the resources that you want to request, but the numbers need to be negative. So you can output "Coal -1" if you want to request coal when there's no more coal in the buffer. If you output "Coal -1000" instead, then you will request coal when there's less than 1000 coal in the buffer; so you can request early, before you completely run out. The buffer chests need to be connected to the same combinator as that constant combinator.

[doxsroxs]

I have my own buffers that I use to transmit the request for material, however, Im not quite sure how it works on the requester station since it looks like the original constant output box was connected to the output of another unit.
Made no sense, or maybe I was just tired. I have not gotten all the signs working either, and I replaced your counting system with manuals to get the run to work.

First of all, I recommend you capture the blueprints again from the demo map, and then export them with the mod "Blueprint String". In your main game save you can then import them. You can also do it with Foreman or KBM, but I don't use these mods for that purpose right now so I cannot pledge for them working.

Don't use the "Fix Positions" button at all, it is only for importing old blueprint strings from other people. You will not need it, all it does is going to break your blueprints.

At the requester station, the constant combinator will need to output all the resources that you want to request, but the numbers need to be negative. So you can output "Coal -1" if you want to request coal when there's no more coal in the buffer. If you output "Coal -1000" instead, then you will request coal when there's less than 1000 coal in the buffer; so you can request early, before you completely run out. The buffer chests need to be connected to the same combinator as that constant combinator.

Aha, that explains a lot, I was signaling how much to request, not at what storage level to do it
I read the entire thread from the beginning but last night I was tired and probably made tons of wierd stuff that I should not have done.

Thanks again for the tips, Ill try and review the mess I made. I also went back to copy the blueprints again, I think they work now so no excuses.
I only wish there was a logic mode/view where I could clearly see what is connected to what, the damn wires are impossible to tell apart sometimes!

[siggboy]

Aha, that explains a lot, I was signaling how much to request, not at what storage level to do it :lol:

You cannot request specific amounts. How much is delivered depends on how much the train loads at the provider. So if you want to limit the amounts that are shipped, you have to do that at the provider station. Let's say you're shipping something valuable, such as Processing Units. That's something that you're usually not going to fill an entire train with. Then, at your provider for the PU, you will simply load only 1,000 units, or some other low amount. That's then going to be delivered to the requester.

There's one detail you need to keep track of, however. The amounts that are delivered are advertised to the requester. This happens when the train leaves the depot, and the number that is reported is how much the provider currently has in stock. So, let's say you trigger the provider when it has AT LEAST 1,000 Processing Units in stock. The delivery then gets made when it does have 1,500 units. This is the number that is going to be promised to the requester. So you will have to load at least that much (which is not a problem since it's already in the buffers). Don't cap the deliveries at 1,000 units, because then the requester will keep waiting for the missing 500, and it will throw the accounting off balance.

In other words: if you set the provider to report back when it has "> X units", then you still need to load everything that it has buffered into the train. Don't cap it at "X units", because more might have been promised to the requester. A good way to deal with that is to create a circuit condition on the train stop, and have it trigger when the buffer chests are almost empty. Then you always load as much as possible and don't have to set specific amounts in train conditions.

That's also the reason why you have to set the stack sizes and train wagon count at the depot. Because from there the depot can calculate the "maximum load" that is going to be delivered by a single train, and it's not going to promise more than that to the requester for a single delivery.

Sounds complicated (because it kind of is :), but it's a lot more simple if you always ship full train loads; then you don't really have to worry about all this. It's more for advanced usage when you build a fully distributed base (which works really well, I have done that in 0.12).

I read the entire thread from the beginning but last night I was tired and probably made tons of wierd stuff that I should not have done.

Wow, yeah, this thread has grown quite large over time. Also some of the things are slightly obsolete, especially the resource accounting and the multi-lane depot is something that I've never published for 0.12, so it's not discussed in the thread at all (except for the few things I've written about it).

I only wish there was a logic mode/view where I could clearly see what is connected to what, the damn wires are impossible to tell apart sometimes!

I also wish there were better tools to debug these complicated setups. It helps to draw a diagram of everything, like I've done for the very early version. The diagrams are still in the original post, you can look at them -- but it's nothing near what the current version does.

If you actually want to reverse engineer the entire thing, then I strongly recommend you draw such a diagram for the depot and the requester and provider circuits. Then you don't have to worry about the wiring.

I've already cleaned up the wiring for the outpost circuits, but not really for the depot. I also think the depot can probably be optimized a little, maybe made a little smaller and have less messy wiring. But it's never going to be straightforward to understand, no matter how "elegant" you could make it.

[doxsroxs]

Aha, that explains a lot, I was signaling how much to request, not at what storage level to do it

You cannot request specific amounts.

Are you working on a solution for it?
Would be awesome to be able to have the requester ask for a specific amount instead of just a full train. Especially since I already have a buffer solution and sorting system in place that can report the figures.

How much is delivered depends on how much the train loads at the provider. So if you want to limit the amounts that are shipped, you have to do that at the provider station. Let's say you're shipping something valuable, such as Processing Units. That's something that you're usually not going to fill an entire train with. Then, at your provider for the PU, you will simply load only 1,000 units, or some other low amount. That's then going to be delivered to the requester.

There's one detail you need to keep track of, however. The amounts that are delivered are advertised to the requester. This happens when the train leaves the depot, and the number that is reported is how much the provider currently has in stock. So, let's say you trigger the provider when it has AT LEAST 1,000 Processing Units in stock. The delivery then gets made when it does have 1,500 units. This is the number that is going to be promised to the requester. So you will have to load at least that much (which is not a problem since it's already in the buffers). Don't cap the deliveries at 1,000 units, because then the requester will keep waiting for the missing 500, and it will throw the accounting off balance.

In other words: if you set the provider to report back when it has "> X units", then you still need to load everything that it has buffered into the train. Don't cap it at "X units", because more might have been promised to the requester. A good way to deal with that is to create a circuit condition on the train stop, and have it trigger when the buffer chests are almost empty. Then you always load as much as possible and don't have to set specific amounts in train conditions.

That's also the reason why you have to set the stack sizes and train wagon count at the depot. Because from there the depot can calculate the "maximum load" that is going to be delivered by a single train, and it's not going to promise more than that to the requester for a single delivery.

Sounds complicated (because it kind of is , but it's a lot more simple if you always ship full train loads; then you don't really have to worry about all this. It's more for advanced usage when you build a fully distributed base (which works really well, I have done that in 0.12).

I read the entire thread from the beginning but last night I was tired and probably made tons of wierd stuff that I should not have done.

Wow, yeah, this thread has grown quite large over time. Also some of the things are slightly obsolete, especially the resource accounting and the multi-lane depot is something that I've never published for 0.12, so it's not discussed in the thread at all (except for the few things I've written about it).

I only wish there was a logic mode/view where I could clearly see what is connected to what, the damn wires are impossible to tell apart sometimes!

I also wish there were better tools to debug these complicated setups. It helps to draw a diagram of everything, like I've done for the very early version. The diagrams are still in the original post, you can look at them -- but it's nothing near what the current version does.

If you actually want to reverse engineer the entire thing, then I strongly recommend you draw such a diagram for the depot and the requester and provider circuits. Then you don't have to worry about the wiring.

I've already cleaned up the wiring for the outpost circuits, but not really for the depot. I also think the depot can probably be optimized a little, maybe made a little smaller and have less messy wiring. But it's never going to be straightforward to understand, no matter how "elegant" you could make it.

Thanks for all the help! I got it going now with a few things left that I dont quite understand.
A picture overlay with current setup and its input/output for the stations would be awesome since the old ones no longer match and I want to learn how to work these better.
My current problem is I dont quite know how to handle the filter inserters for loading and unloading. It would be really nice if you can help me there a bit since I do want to run filtered stations later on.

[siggboy]

You cannot request specific amounts.

Are you working on a solution for it? ;)
Would be awesome to be able to have the requester ask for a specific amount instead of just a full train. Especially since I already have a buffer solution and sorting system in place that can report the figures.

I'm not planning to implement that because I don't think that it solves an actual problem. The providers can only deliver what they have. If the requester is ready to take more supplies, and a provider is ready to ship supplies, then the delivery is made; if there's no provider then you don't have the supply -- it wouldn't really change the big picture if the requester could ask for less in that instance.

At the end of the day, you have an aggregate demand of things that you need to fulfill, and the most efficient way to do that is to ship entire trains. If you allow for "partial orders" then you'll only waste train capacity.

The only exception that I would make is for certain materials that you usually don't ship in bulk, but you still want to ship -- for example it makes sense to ship oil barrels in lots of 400 instead of 1600, or to ship speed modules in smaller amounts. That will reduce latency in your system because the outposts don't have to buffer a full train load before a shipment is ready.

These scenarios are nicely handled by the current system, so I don't see a reason to change that (i.e. overengineer it :).

What do you have in mind when you want to be able to request specific amounts?

A picture overlay with current setup and its input/output for the stations would be awesome since the old ones no longer match and I want to learn how to work these better.

Yes, I will make a diagram, but it's a lot of work so I wasn't really motivated to update this for the current version. I also kept making a lot of changes and so diagramming it wouldn't have made sense yet.

My current problem is I dont quite know how to handle the filter inserters for loading and unloading. It would be really nice if you can help me there a bit since I do want to run filtered stations later on.

For unloading you don't need filters. You simply unload everything, then you sort the stuff with robots. Or if you have a station that only requests two different ores, say copper and iron, you can just unload each onto one side of the station, no robots needed.

Loading is a lot more difficult. That's a multi-product provider. The problem here is not related to the train stuff at all. It is that you must avoid that the inserter finishes loading (wagon is full) and still has items in the inserter hand. Those items would be dropped into the next train, which is very bad because it will lead to mixed wagons.

There's a nice trick with limiting the filter so this is not going to happen, but it requires to set up the correct numbers in the circuit. So there's no generic circuit that handles all of this and can be used everywhere (well, there is but it would be rather complex).

Another problem with the multi-product provider is that the system only allows one train at a time to a provider. So if you combine two high volume outposts into one provider, this may lead to a bottleneck situation.

I think the multi-product provider is best for shipping intermediate products, an advanced scenario, or if you use Bob's Mods and have lots of ores that you don't need much of. Then it's easier to have one provider handle several and not being required to make a lot of outposts for all the trace metals.

I have made a circuit that avoids the inserter problem. Other than that it's rather easy, because the register at the provider tells you what item needs to be loaded into the next train, so you can use that to set the filter.

[doxsroxs]

Hi again

Our multiplayer game got desynced so we lost something like 13 hours
This also meant I had to start all over on this, thought it would be a cake since I now have done this several times.
However, I cannot seem to get it to work this time around. For example, setting up the Depot, I cannot get a light indication from the first train when I send it to the depot as per your instruction post.

Initial setup, depot (in some detail):

1. Put a train on the rail and send it to the Depot (the lane selector should now pick that lane; it's indicated with a lamp). Set the number of wagons per train in the appropriate combinator (arithmetic combinator in the top row of the depot circuit); the number has to be set as a negative number ("-4" => "4 wagons per train).

Looking at your demo map, the lights dont seem to match the occupied Depot tracks either. Am I missing something here?
I do recall it didnt work when I got everything else working earlier a well, so I kindof didnt care, but now that things are not working Im starting to wonder what I missed.

Im sure I have the blueprint copying down to an art by now, I also use screenshots from the demo map to ensure any wires I put down are right where they need to be.
Currently I only have the main combinator cluster and the Depot tracks connnected, nothing else, and I was careful to ensure no bogus signals could enter via wire.
The main cluster is attached to each track via red and green wire from the right hand pole just like on the demo track.

[doxsroxs]

You cannot request specific amounts.

Are you working on a solution for it?
Would be awesome to be able to have the requester ask for a specific amount instead of just a full train. Especially since I already have a buffer solution and sorting system in place that can report the figures.

I'm not planning to implement that because I don't think that it solves an actual problem. The providers can only deliver what they have. If the requester is ready to take more supplies, and a provider is ready to ship supplies, then the delivery is made; if there's no provider then you don't have the supply -- it wouldn't really change the big picture if the requester could ask for less in that instance.

At the end of the day, you have an aggregate demand of things that you need to fulfill, and the most efficient way to do that is to ship entire trains. If you allow for "partial orders" then you'll only waste train capacity.

The only exception that I would make is for certain materials that you usually don't ship in bulk, but you still want to ship -- for example it makes sense to ship oil barrels in lots of 400 instead of 1600, or to ship speed modules in smaller amounts. That will reduce latency in your system because the outposts don't have to buffer a full train load before a shipment is ready.

These scenarios are nicely handled by the current system, so I don't see a reason to change that (i.e. overengineer it .

What do you have in mind when you want to be able to request specific amounts?

I was only looking to handle less than train load amounts for valuable items not shipped so often, but you are right, I could probably handle them within the current system.

I plan on using a buffer for each material for any double item stations and then only unload one car load of items by counting to each car, should avoid the issue with items ending up on the wrong train.