old LTN discussion thread

[Optera]

You use a bad lane balancer design, it hurts my ocd.
here's a much better design that will work on clogged belts.

It's funny, the version you linked to hurts my OCD - I can't stand to see a balancer that strands materials so they can never be used, even if I know it's more efficient.

My OCD hurts when a lane balancer only swaps lanes instead of truly balancing.

This mod is awesome. I've been using it heavily the past few days and have two suggestions. First, I'd like to have a priority signal I could send to a station. There are a few recipes where an item is produced from multiple recipes. In that case when a secondary item is backed up causing the primary to not be produced, say copper plates from cobalt, then I'd want that secondary product to be picked up before another factory where the secondary is the only thing being produced.

The priority signal would override the normal order of determining jobs and assign the next available train to pick up from that station. In cases where multiple stations have a signal, assign the highest signal takes priority. A config option for allowing priority signals to override a station's minimum delivery setting, however no job should be created if there are no stations requesting the item.

Are priorities really needed? No matter what sort of transport you use, belts, bots, normal trains, logistic train network it always boils down to providing enough input, production at providing stations, and having enough throughput,enough trains and a well thought out network, station design.
If you don't have enough mines they will starve your furnaces, if you lack furnaces your EC production starves, and so on. Giving priority to shipping ore will not help if you simply lack production capacity or throughput.

I might have understood you wrong however and you want certain stations to be preferred as provider, overriding the current way of picking the station with the most items provided. Like a near depleted mine you want to finally drain out. That's something I would add, as it's something available in logistic network (active provider) that can't be achieved with the current system.

The second suggestion would be to have a max trains at one time signal to pass to a station. Suppose I have a station with 32,000 items and 4 trains, each with 1 wagon with a capacity of 8,000. Currently the system will send all 4 trains to collect that 32,000 cargo. If my station has one loading bay and two waiting bays, I'd set the max trains to three, and then once three trains have been sent to a station no more jobs will be created until one of the trains has finished the loading process.

That should happen only once when a station requesting items is empty. After filling the station up it will use only one train at a time, unless the station burns through items faster than an 8k delivery takes. In that case increase throughput by either using longer or more trains.

[Kormer]

Are priorities really needed? No matter what sort of transport you use, belts, bots, normal trains, logistic train network it always boils down to providing enough input, production at providing stations, and having enough throughput,enough trains and a well thought out network, station design.
If you don't have enough mines they will starve your furnaces, if you lack furnaces your EC production starves, and so on. Giving priority to shipping ore will not help if you simply lack production capacity or throughput.

I might have understood you wrong however and you want certain stations to be preferred as provider, overriding the current way of picking the station with the most items provided. Like a near depleted mine you want to finally drain out. That's something I would add, as it's something available in logistic network (active provider) that can't be achieved with the current system.

I'm thinking of the case where I have both a dedicated copper smeltery as well as a second smaller smeltery for cobalt where copper happens to be a secondary output. If the copper at the cobalt smeltery is full, that stops cobalt production so I'd want to prioritize picking up that copper first before I grab copper from the copper smeltery.

Also if it helps, I'm working on a 100% rail map because of this mod which is both awesome and complicated. In a vanilla setup you'd just run the copper and copper/cobalt smelteries next to each other and merge the copper outputs before sending them on their way. You'd probably also add in circuitry so that copper coming off the cobalt/copper line is prioritized onto the main bus so cobalt is never backed up. This is the functionality that I'd like to duplicate and it's really only a problem when an item is in more than one recipe and at least one of them includes multiple outputs.

[Optera]

Are priorities really needed? No matter what sort of transport you use, belts, bots, normal trains, logistic train network it always boils down to providing enough input, production at providing stations, and having enough throughput,enough trains and a well thought out network, station design.
If you don't have enough mines they will starve your furnaces, if you lack furnaces your EC production starves, and so on. Giving priority to shipping ore will not help if you simply lack production capacity or throughput.

I might have understood you wrong however and you want certain stations to be preferred as provider, overriding the current way of picking the station with the most items provided. Like a near depleted mine you want to finally drain out. That's something I would add, as it's something available in logistic network (active provider) that can't be achieved with the current system.

I'm thinking of the case where I have both a dedicated copper smeltery as well as a second smaller smeltery for cobalt where copper happens to be a secondary output. If the copper at the cobalt smeltery is full, that stops cobalt production so I'd want to prioritize picking up that copper first before I grab copper from the copper smeltery.

Also if it helps, I'm working on a 100% rail map because of this mod which is both awesome and complicated. In a vanilla setup you'd just run the copper and copper/cobalt smelteries next to each other and merge the copper outputs before sending them on their way. You'd probably also add in circuitry so that copper coming off the cobalt/copper line is prioritized onto the main bus so cobalt is never backed up. This is the functionality that I'd like to duplicate and it's really only a problem when an item is in more than one recipe and at least one of them includes multiple outputs.

Thanks that example makes it clear what you want. I'll add a priority signal for provider stations in 0.4.4.
2^32 priority stages should be enough I guess.

[ransburger]

I've had a few instances with Depots setting up schedules that look like the following

Depot
Iron Drop 3
Iron Drop 1

Both Iron Drop 3 and Iron Drop 1 are requestor stations, and the vast majority of the time the Depot schedules them as such. However, it occasionally hiccups and tries to make a train pickup supplies at a requestor. In all instances when I have noticed this the not-a-provider and requestor stations have matching items.

I've included a save file where the consist I am standing in front of (patiently waiting in the holding area for iron dropoff) is exhibiting this schedule snafu.

[Exasperation]

You use a bad lane balancer design, it hurts my ocd.
here's a much better design that will work on clogged belts.

It's funny, the version you linked to hurts my OCD - I can't stand to see a balancer that strands materials so they can never be used, even if I know it's more efficient.

My OCD hurts when a lane balancer only swaps lanes instead of truly balancing.

That's a separate issue; here's the version you linked to next to a version that does the same thing minus the stranded items:

Balancers

[Kormer]

Small bug to report. When a station changes names, LTN doesn't always see the name change before creating a new job and instead sends a train to an invalid station. Most of the time it works fine and I haven't figured a particular pattern to make it repeatable.

[Optera]

I've had a few instances with Depots setting up schedules that look like the following

Depot
Iron Drop 3
Iron Drop 1

Both Iron Drop 3 and Iron Drop 1 are requestor stations, and the vast majority of the time the Depot schedules them as such. However, it occasionally hiccups and tries to make a train pickup supplies at a requestor. In all instances when I have noticed this the not-a-provider and requestor stations have matching items.

I've included a save file where the consist I am standing in front of (patiently waiting in the holding area for iron dropoff) is exhibiting this schedule snafu.

That's simply configuration errors.

• #cargo wagons is a read only signal, if you want to limit the size of trains use Maximum Train Length
• some your Drop Stations have positive counts. Drop4 for example has +6.1k, for LTN that means those are providers

Small bug to report. When a station changes names, LTN doesn't always see the name change before creating a new job and instead sends a train to an invalid station. Most of the time it works fine and I haven't figured a particular pattern to make it repeatable.

Yes, that's possible if you happen to change the name in the time frame between station update and generating a schedule.
With 0.15 I'll get an event when names change so it will be less likely to happen. viewtopic.php?f=65&t=37595
I'm also gradually moving towards only using unit_numbers internally, so it should be solved in future versions. Until then you can disconnect stops before renaming to be safe.

[ransburger]

That's simply configuration errors.
#cargo wagons is a read only signal, if you want to limit the size of trains use Maximum Train Length
some your Drop Stations have positive counts. Drop4 for example has +6.1k, for LTN that means those are providers

Clearly I didn't understand how to set things up quite as well as I thought.

My updated understanding is that the drop station has a positive count because the -32k that is stored in the combinator + the 38k in the chests results in a +6k. That positive value means that the system reads the drop station as a provider. Since a drop station does not have inserters facing the other way to be a properly functioning provider, I should prevent the value from going positive so that the system doesn't try to use it as a provider. To do so I could limit the amount in each chest so that the sum total is never above the -32k. I could also put the requested amount above the chest capacity.

2 semi-related questions...

1) Is there an easy way for me to read what value the system is outputting at a station, aside from adding up the totals in chests.
[EDIT : I figured this one out, I just plugged in an arithmatic combinator. Mousing over gives the net difference in the station.]

2) Probably another configuration issue. I will occasionally have a train attempt to fulfill a delivery for items that are not yet available in chests. This happens most often for full crude barrels. This results in the train sitting idle waiting for the requisite number of barrels to be completed and loaded. What is the most likely culprit here? Its probably due to a faulty understanding of the interplay between minimum delivery size and the value I assign to the crude barrels in the combinator.

[EDIT]

I have been setting crude barrels to 50, assuming that meant that I wanted it to pick up 50 at the minimum. I think what is happening is that the system is reading the 100 barrels in the box, adding the +50 from the constant combinator, and outputting a 150 barrel pickup request. So the train arrives to pickup the actual + 40-50 non-existant barrels and waits until those are manufactured, leading to increased down time. Is that about right?

[Optera]

Clearly I didn't understand how to set things up quite as well as I thought.

My updated understanding is that the drop station has a positive count because the -32k that is stored in the combinator + the 38k in the chests results in a +6k. That positive value means that the system reads the drop station as a provider. Since a drop station does not have inserters facing the other way to be a properly functioning provider, I should prevent the value from going positive so that the system doesn't try to use it as a provider. To do so I could limit the amount in each chest so that the sum total is never above the -32k. I could also put the requested amount above the chest capacity.

You simply hook a constant combinator with the negative amount you want inside the boxes to the boxes and stop lamp.
From what I recall you had 4 chests/wagon with 4 wagon trains = 38400 ore max if you request more than that you will get trains stuck in those stations trying to unload ore into full chests.
Just manually clean out some ore until you are at 32k.

Hint: don't forget to set minimum delivery size to 6k so trains ship full loads

2 semi-related questions...

1) Is there an easy way for me to read what value the system is outputting at a station, aside from adding up the totals in chests.
[EDIT : I figured this one out, I just plugged in an arithmatic combinator. Mousing over gives the net difference in the station.]

I use either Nixie Tubes or Smart Display for permanent displays or mouse over a power pole for a quick glance.
Hooking up combinators takes runtime. (only noticable when you hook up several whole logistic network inventories via roboport)

2) Probably another configuration issue. I will occasionally have a train attempt to fulfill a delivery for items that are not yet available in chests. This happens most often for full crude barrels. This results in the train sitting idle waiting for the requisite number of barrels to be completed and loaded. What is the most likely culprit here? Its probably due to a faulty understanding of the interplay between minimum delivery size and the value I assign to the crude barrels in the combinator.

[EDIT]
I have been setting crude barrels to 50, assuming that meant that I wanted it to pick up 50 at the minimum. I think what is happening is that the system is reading the 100 barrels in the box, adding the +50 from the constant combinator, and outputting a 150 barrel pickup request. So the train arrives to pickup the actual + 40-50 non-existant barrels and waits until those are manufactured, leading to increased down time. Is that about right?

If you add 50 from a constant combinator to 100 stored in a chest you tell LTN here's 150 barrels ready to pick up.
Forcing pickup of 50 is done at the requesting station by setting minimum delivery size to 50.

[ransburger]

I didn't know the powerpole, that's nice.

After going around and modifying the amount in the constant combinators, things are much smoother. No issues so far for the last 90 minutes yet. The biggest issue was my lack of understanding of circuit networks, so I've been forced to learn them.

Thanks! I really like this mod. I like not needing to alter train schedules by hand, and I like having a yard where all the idle trains are sitting waiting for work. The fact that I can run my base with fewer less specialized trains is gravy.

[Optera]

I didn't know the powerpole, that's nice.

After going around and modifying the amount in the constant combinators, things are much smoother. No issues so far for the last 90 minutes yet. The biggest issue was my lack of understanding of circuit networks, so I've been forced to learn them.

Thanks! I really like this mod. I like not needing to alter train schedules by hand, and I like having a yard where all the idle trains are sitting waiting for work. The fact that I can run my base with fewer less specialized trains is gravy.

That's nice to hear.
I was a bit surprised myself of how many useless trains i had around. My Testbase went down from 4 Rail Tanker, 10 ore trains, 2 engineering trains to 2 tanker and 4 freight trains. It probably would even work with only 1 tanker and 2 freight trains.

[Kormer]

I found a rather minor bug. Occasionally LTN will send a train on an order, and if there is capacity in the order, another train will also be sent on the same order. This is fine, but the log will not display a message for subsequent trains. I've confirmed that LTN is in fact changing the schedules and that the stations had the proper capacity on both ends to warrant multiple trains, so it's only the log message being affected.

[Optera]

I found a rather minor bug. Occasionally LTN will send a train on an order, and if there is capacity in the order, another train will also be sent on the same order. This is fine, but the log will not display a message for subsequent trains. I've confirmed that LTN is in fact changing the schedules and that the stations had the proper capacity on both ends to warrant multiple trains, so it's only the log message being affected.

Please set log output to log file and log level to 4 and send me the log file so i can see what's going on.

Code: Select all

/c remote.call('LTN', 'log_output', 'console,log')
/c remote.call('LTN', 'log_level', 4) 

[Kormer]

I found a rather minor bug. Occasionally LTN will send a train on an order, and if there is capacity in the order, another train will also be sent on the same order. This is fine, but the log will not display a message for subsequent trains. I've confirmed that LTN is in fact changing the schedules and that the stations had the proper capacity on both ends to warrant multiple trains, so it's only the log message being affected.

Please set log output to log file and log level to 4 and send me the log file so i can see what's going on.

Code: Select all

/c remote.call('LTN', 'log_output', 'console,log')
/c remote.call('LTN', 'log_level', 4) 

Should the log output go to the script-output directory? I'm not seeing an LTN file show up in there.

Nevermind, found it in the main factorio log when I noticed it doubling every few minutes.

[Kormer]

Bad news is I didn't get very far in getting a test case for the minor bug reported earlier. The good news is I just spent the past few hours reading logs and getting an understanding of your code and have some feedback for you.

First, I'm working on a game where the only hard rule is that the output of every recipe must travel via rail before being consumed in another recipe. This includes no hand-crafting, but I am allowed creative mode items to be spawned until a factory station can be constructed for that item. Also all belts/inserters/assemblers are kept at lowest levels until a higher tier factory is constructed. On top of all that, I've limited myself to 10x L-C-L trains on the network until a locomotive factory can be built.

The modlist here includes Bob's + Marathon, along with most helper mods and a few others for fun. Given the above rules, I'm just about at the point of having a self-replicating network with around 100 station stops total. The current network is horribly resource starved and has nowhere near the level of trains necessary to fulfill all orders. I suspect your test worlds do not have such a resource starved network, and I'm seeing some odd scheduling patterns that you probably haven't seen yet.

The first one is if you have two stations requesting the same item, but on has a lower minimum delivery than the other and the provider station's capacity is not enough to supply both requesters, then only the station with the lowest min delivery will ever have an order fulfilled. This will be solved either when the smaller station eventually reaches capacity, or the provider station can have a circuit to only report it's availability when the amount is greater than the network's largest min delivery for that item. I don't think anything necessarily needs to change in LTN for this issue, but players should however be aware of it and might be something to include in a FAQ.

The second more serious thing I found what that the BuildOrders() function appears to favor some stations over others. This had a devastating cascading effect in my game where a critical factory never got resources because an earlier station consumed everything fairly fast. In my case it was tinned copper wire and basic electronic circuits fighting over the scarce copper cables.

I could be off on this here, but it looks like you are tracking age for requests. If I'm reading this right, then if you had a requester that never had it's request fulfilled 100%, it would permanently stay at the front of the line for new orders. If we're talking about something like copper cables, then the answer is always going to be more, so this scenario is quite likely. My recommendation would be to setup a round-robin approach to dispatching orders. Instead of tracking the age of the request, track the age of the last order fulfilled from each station. When processing orders, sort your stations by priority and last order tick so stations that have gone the longest without an order get processed first assuming same priority.

Once an order has been created for a particular stopID, skip to the next station even if we could create another order for that station. We're already processing orders once a minute by default, and the reality is we're not going to need two trains going to the same station less than a minute apart. By waiting for the next update to process the any remainder quantities for a new order, another requesting station will now have an older LastOrderTick value, so that station will get what is left. If nothing is left, that's fine because the next station by LastOrderTick will be first up when supplies are available.

Right now you're dispatching 3 trains to Station A, but in this system you'll dispatch one each to A, B, and C before looping back to A again. Stations could still get locked out due to one consuming large quantities with a lower min delivery setting, but at least that's manageable on the player side so I'm a little more OK with it. This also incidentally partially solves my earlier reported issue of multiple trains being dispatched to the same station at once. It's still possible, just a little less likely.

I'd still like to get you a test world that's resource and transportation starved similar to my current modded game, but closer to a vanilla experience. Maybe tomorrow.

[Optera]

The current network is horribly resource starved and has nowhere near the level of trains necessary to fulfill all orders. I suspect your test worlds do not have such a resource starved network, and I'm seeing some odd scheduling patterns that you probably haven't seen yet.

You caught me there. I only tested just in time and over saturated networks.
I don't think running a starved network makes any sense to begin with.

The first one is if you have two stations requesting the same item, but on has a lower minimum delivery than the other and the provider station's capacity is not enough to supply both requesters, then only the station with the lowest min delivery will ever have an order fulfilled. This will be solved either when the smaller station eventually reaches capacity, or the provider station can have a circuit to only report it's availability when the amount is greater than the network's largest min delivery for that item. I don't think anything necessarily needs to change in LTN for this issue, but players should however be aware of it and might be something to include in a FAQ.

It should ship to both, aiming to evenly fill them similar to how bots do. Basically it's just another effect stemming from the same oversight as the second bug.

The second more serious thing I found what that the BuildOrders() function appears to favor some stations over others. This had a devastating cascading effect in my game where a critical factory never got resources because an earlier station consumed everything fairly fast. In my case it was tinned copper wire and basic electronic circuits fighting over the scarce copper cables.

I could be off on this here, but it looks like you are tracking age for requests. If I'm reading this right, then if you had a requester that never had it's request fulfilled 100%, it would permanently stay at the front of the line for new orders. If we're talking about something like copper cables, then the answer is always going to be more, so this scenario is quite likely. My recommendation would be to setup a round-robin approach to dispatching orders. Instead of tracking the age of the request, track the age of the last order fulfilled from each station. When processing orders, sort your stations by priority and last order tick so stations that have gone the longest without an order get processed first assuming same priority.

Once an order has been created for a particular stopID, skip to the next station even if we could create another order for that station. We're already processing orders once a minute by default, and the reality is we're not going to need two trains going to the same station less than a minute apart. By waiting for the next update to process the any remainder quantities for a new order, another requesting station will now have an older LastOrderTick value, so that station will get what is left. If nothing is left, that's fine because the next station by LastOrderTick will be first up when supplies are available.

Right now you're dispatching 3 trains to Station A, but in this system you'll dispatch one each to A, B, and C before looping back to A again. Stations could still get locked out due to one consuming large quantities with a lower min delivery setting, but at least that's manageable on the player side so I'm a little more OK with it. This also incidentally partially solves my earlier reported issue of multiple trains being dispatched to the same station at once. It's still possible, just a little less likely.

BuildOrders is creating merged orders and also prioritizes providers that way.
Skipping an item already already added to an order for the requester by itself only moves the problem. What will happen is station A receives a best effort shipment of 100 cables from a high priority provider, while station B gets a full 8k from a low priority provider.

What I would have to do is sum up all provided items and evenly distribute them to all requests.
Considering I won't have nowhere as much time as the last weeks It'll take a while until I can add such a big change to the logic.

I'd still like to get you a test world that's resource and transportation starved similar to my current modded game, but closer to a vanilla experience. Maybe tomorrow.

Thanks, seeing it in action will help.

[Kormer]

I've attached a demo world I put together for you. The world was built according to my arbitrary rule that the output of all recipes must travel via rail before being used in another recipe. It also intentionally uses low level tech and limits the number of trains to simulate the lack of a main base and the user going straight for a large rail network instead. I believe all circuits should be wired correctly, but I may have a mistake or two someplace. I'm also aware there are much better ways of doing a lot of what's on the map so try not to criticize the build too much.

With all that being said, play around with it and re-read my earlier comments. I think a lot of my questions boil down to, Why are some stations receiving supplies while others are not, and what can I as a player do to control that?

[Optera]

I've attached a demo world I put together for you. The world was built according to my arbitrary rule that the output of all recipes must travel via rail before being used in another recipe. It also intentionally uses low level tech and limits the number of trains to simulate the lack of a main base and the user going straight for a large rail network instead. I believe all circuits should be wired correctly, but I may have a mistake or two someplace. I'm also aware there are much better ways of doing a lot of what's on the map so try not to criticize the build too much.

With all that being said, play around with it and re-read my earlier comments. I think a lot of my questions boil down to, Why are some stations receiving supplies while others are not, and what can I as a player do to control that?

I only did a quick analyse of the map. Apart from some entities no being built here's what I'd start to fix.

• All loading stations only use 8 out of 12 possible inserter and only the slow normal inserter.
  You get fast inserter very early on, use them.
• Initial shipping of 32k ore to every smelter at once with only 3 L-C-L trains does take forever. Combined with the slow loading speeds (see 1) that alone kills any throughput.
  Use more and longer trains, at least L-CC-L.
  Set initial requests lower, 8k should be enough to get it running.
• Did you try this with best effort on or off?
  For systems that can't keep up with incoming orders it's better to set use_best_effort = false in config.

I got it somewhat running by doing all those.

[Kormer]

I've attached a demo world I put together for you. The world was built according to my arbitrary rule that the output of all recipes must travel via rail before being used in another recipe. It also intentionally uses low level tech and limits the number of trains to simulate the lack of a main base and the user going straight for a large rail network instead. I believe all circuits should be wired correctly, but I may have a mistake or two someplace. I'm also aware there are much better ways of doing a lot of what's on the map so try not to criticize the build too much.

With all that being said, play around with it and re-read my earlier comments. I think a lot of my questions boil down to, Why are some stations receiving supplies while others are not, and what can I as a player do to control that?

I only did a quick analyse of the map. Apart from some entities no being built here's what I'd start to fix.

• All loading stations only use 8 out of 12 possible inserter and only the slow normal inserter.
  You get fast inserter very early on, use them.
• Initial shipping of 32k ore to every smelter at once with only 3 L-C-L trains does take forever. Combined with the slow loading speeds (see 1) that alone kills any throughput.
  Use more and longer trains, at least L-CC-L.
  Set initial requests lower, 8k should be enough to get it running.
• Did you try this with best effort on or off?
  For systems that can't keep up with incoming orders it's better to set use_best_effort = false in config.

I got it somewhat running by doing all those.

Thanks, all great feedback. Most of those design decisions were made deliberately. I think you've made a great mod that works in perfect conditions, while I'm trying to see if an algorithm can make up for a player's mistakes.

I made this world in a non-modded environment since I've heavily edited some other common mods in my existing world and you'd never get compatibility with it. With the recipes in my modded world, fast inserters are a bit further down the line to get so that's why I hadn't included them. The 32k number should have been cut to 8k for your test world due to different stack sizes. I didn't include this here, but I have a combinator blueprint that makes the requested material at a station equal to the inverse of the output materials plus some for a buffer.

The use of fewer inserters at the stations was intentional, and I may go back on that. The theory there is that the stations you see can be expanded to 1/2/4/8 wagon stops all with compatible blueprints. By making the number of inserters on a side 4 per wagon, now all of your wagons, chests, inserters, and belts will be powers of two. The reason that is important is then you can modularly expand your factory as resources become available and expanding/compressing lines from 2^X to 2^Y is fantastically easier to blueprint for than other combinations. Having one set of blueprints that can easily be mixed, matched, and expanded across all recipes in the game is more important for building a wide base than having the most efficient throughput. Besides, the throughput will be solved once upgraded express stack inserters get researched.

I did use this with best effort turned off. I've tried best effort and to be honest, it needs some work to be useful. The problem I ran into is that with more goods to move than trains, LTN just assigns every single new train that enters a depot to move the same order, but at a fraction of the trains capacity. I'd end up moving 100 gears 4 times in a row when I've got 30k copper plates sitting and a copper cables factory that's idle. I do like the idea of a best effort algorithm and think it's needed, I just don't see the current form as being useful. My idea to fulfill orders in a round-robin fashion to the requesters is an attempt to solve the same problem. I've actually had to write an algorithm for a very similar problem professionally so I may end up sending you a pull request if I can find the time to implement it.

[exofact]

I just installed 0.4.5 and am getting this error on loading my save:

"Error while running the on_init: __LogisticTrainNetwork__/control.lua:32: attempt to concatenate field '?' (a nil value)"

I am running Factorio 0.14.21.

Is there any easy way to get an export of what mods are installed?