Add simple train stop priorities on top of 1.1 train limit system.

Moderator: ickputzdirwech

Kaktusbot
Burner Inserter
Burner Inserter
Posts: 15
Joined: Sun Jun 04, 2017 6:54 pm
Contact:

Add simple train stop priorities on top of 1.1 train limit system.

Post by Kaktusbot »

TL;DR
Please, add to train stops new setting 'priority' and change the train pathfinder slightly to choose first by highest priority and then by distance.

What ?
1.1 added nice system of train limits, but it lacks one small detail - priority. For now trains choose closest train stop with "empty slots".

At first I tried to solve this by circuit networks, I had some clever system which sets train limit to '0' instead of desired 'limit'. And only if all train stops with higher priority are full ( their 'count'=='limit' ), then it will change '0' to 'limit'. It even allowed changing priorities on the fly by circuit network. All worked nice in a testing environment with only one train in the network.
But when I added this to real environment it stopped work, because trains always choose by distance and they are frequently repathing when they see red signal.

Lets narrow this to simple case.
Image
If lower station have train going to it(it's full), it opens upper station. So it works like higher priority to lower station. And it works if nothing triggers a repath. But for example some train is going through crossroad, so our train needs to brake which triggers repath. Then our train sees open closer station and repaths to it. Of course lower station train count becomes 0 and upper station limit becomes 0 too, but no repath now can repath to lower station, because train already reserved slot in upper and will see it as open.

Possible hacky solutions to this:
1. Disable train stops. But disabling is evil and train will swap between stations every repath.
2. Adding circuit controlled signal to add distance penalty. Works better, but penalty is not infinite and more then 1 kilometer distant stations will not work anyway. That's even not talking about some weird situations when train passed through controlled signal and station becomes closed in front of it.
3. Force temporary stops by some script on_train_state_changed. But when train goes to temporary stop it does not reserve slot in next station, so it will not work too...

The only real solution is to change pathfinder algorithm, so it will sort stations by priority and then sort same priority subgroups by distance.
It is not complex solution, it does not require reworking everything with pathfinder logic, just simple change in sorting and adding a couple of events when priority of station is changed.
Now if you want train stop priorities you need to use some overly complicated mods like LTN or TSM, they are heavy, slow, have their own train limit and counting systems which is prone to errors.
Why ?
It would be great to add this simple but powerful feature of train stop priorities. It will allow players who do not need many features of TSM and LTN to use just vanilla trains.
User avatar
NotRexButCaesar
Smart Inserter
Smart Inserter
Posts: 1133
Joined: Sun Feb 16, 2020 12:47 am
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by NotRexButCaesar »

Instead of measuring train “count,” you could look at the number of red signals on the stacker to find the number of trains that won’t repath away.

I know this doesn’t do as much as the suggestion, but I thought it might be helpful.

I believe Boskid said on the technical factorio discord that this would never be implemented. (but don’t take my word for it Ick)
Last edited by NotRexButCaesar on Sat May 08, 2021 7:11 pm, edited 1 time in total.
—Crevez, chiens, si vous n'étes pas contents!
Kaktusbot
Burner Inserter
Burner Inserter
Posts: 15
Joined: Sun Jun 04, 2017 6:54 pm
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by Kaktusbot »

Sorry, I do not know what Roath means and google did not helped me to understand it. And if it's a typo
why else it starts with capital letter? looks like android/ios error correction to me
my non-native English skill can't decipher it.

From my understanding If i count red signals in the stacker it won't work, because if trains go to station but did not yet counted in the stacker the lower priority station won't open, and excess trains that should go to lower priority will fail with "Destination Full". Maybe I did not understand something in your suggestion.
I believe Boskid said on the technical factorio discord that this would never be implemented.
Was it before 1.1 or after?
If it was before there is a chance. If it was after... sadly I think I need to write my own TSM with nice GUI.
I were praising the devs in the past, they seemed cool guys to me, but after I reported a few bugs, started modding, saw a reddit post with a guy who recreated whole factorio engine fixing all their stubbornness
luajit, multithreading
, I'm slowly growing hate to them.
technical factorio discord
Friday facts migrated to discord from us mortals
boomers without discord
, I see.
User avatar
NotRexButCaesar
Smart Inserter
Smart Inserter
Posts: 1133
Joined: Sun Feb 16, 2020 12:47 am
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by NotRexButCaesar »

Kaktusbot wrote: Sat May 08, 2021 3:43 pm Sorry, I do not know what Roath means and google did not helped me to understand it. And if it's a typo
why else it starts with capital letter? looks like android/ios error correction to me
my non-native English skill can't decipher it.
I am not sure what I meant. It was “repath.” I often use my cell phone on the forums and it autocorrected something into a name, so your guess was correct.

Kaktusbot wrote: Sat May 08, 2021 3:43 pm From my understanding If i count red signals in the stacker it won't work, because if trains go to station but did not yet counted in the stacker the lower priority station won't open, and excess trains that should go to lower priority will fail with "Destination Full". Maybe I did not understand something in your suggestion.
It would work, but with delay. It may be that I will try to implement it when I get home.

Here is what Boskid said:
38BD421E-C5A1-4A80-A2D9-57AE10316CB8.png
38BD421E-C5A1-4A80-A2D9-57AE10316CB8.png (376.82 KiB) Viewed 5846 times
The thing that will “never be implemented” is just above that and is a reverse pathfinder to pick the closest train to a station.

The thing that was implemented is a straight line distance closest train selector.
—Crevez, chiens, si vous n'étes pas contents!
User avatar
boskid
Factorio Staff
Factorio Staff
Posts: 3017
Joined: Thu Dec 14, 2017 6:56 pm
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by boskid »

If possible please discuss about topic itself, not about one particular dev opinions.

I was already thinking about adding priorities to train stops and this idea seems to be easier to implement than reverse trains pathfinder. There are simply not many cases where byproduct handling (where i think it would be primarily used for) would be useful in the vanilla itself, so it would be strictly a feature for mods to be used. The most amount of the issues to solve would come from the player expectation for the train stop priority itself: would it be a source priority, destination priority or both. When a train leaves train stop with high priority it should be considered first when searching for a path to another train stop, and the destination train stop could also have a priority so it would have to find a path to any available train stop with the highest possible priority.

For now this idea was stopped because of external reasons, independent of me. Even if it would be implemented, it is super unlikely it would ever go to 1.1.
Kaktusbot
Burner Inserter
Burner Inserter
Posts: 15
Joined: Sun Jun 04, 2017 6:54 pm
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by Kaktusbot »

NotRexButCaesar wrote:It would work, but with delay.
And delay will be big for distant stations. Well... it can work and something is better than nothing, but I can't accept that delay, I'd better use TSM-like mod, maybe I'll stick to Cargo Train Manager, looks like TSM but with nice GUI.

I tried googling "technical factorio discord" and found only one public server but there is no channel named "#train"
Is this "technical factorio discord" closed private server for chosen ones, like the ones who have access to the source code?
Or is it me who just can't understand how to use all this discord bells and whistles?
boskid wrote:would it be a source priority, destination priority or both
From my point of view source priorities are not needed. And how they will even work? If two trains depart in the exact same tick then they will fight who will go to closest station? Or are they will reserving slots while waiting at load/unload station? This all seems so complicated and I know you guys love simple solutions like the train limit system of 1.1 itself.
Maybe I miss something about sourcing priorities but in my mind destination priority should be much simpler to implement than sourcing priority. Sourcing priorities are a thing in mods like TSM where trains are usually waiting at provider stackers.
boskid wrote:Even if it would be implemented, it is super unlikely it would ever go to 1.1
I'm not asking to implement it in 1.1, I'm just asking to implement it at all.
User avatar
boskid
Factorio Staff
Factorio Staff
Posts: 3017
Joined: Thu Dec 14, 2017 6:56 pm
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by boskid »

Kaktusbot wrote: Sat May 08, 2021 8:44 pmFrom my point of view source priorities are not needed. And how they will even work? If two trains depart in the exact same tick then they will fight who will go to closest station?
Using train stop limits this case happens quite often when you see the "destination full" notification. When a train leaves train stop it decreases counter of reservations (in game called "trains count" as its easier to understand it that way) which makes a train stop to become not full (assuming C=L before). In that case if there were 2 trains they will fight with each other because first one to repath will get the reservation and the other one will not.

Destination priority makes sense if there are more goals than there are trains available at that moment. Train can choose to which goal it wants to go and the priority would have to be able to influence that.
For example an iron ore train could go to 2 different places: iron plates production and concrete production. If the concrete production has higher priority the train would have to choose that goal even if it has higher distance.

Source priority makes sense if there are multiple trains waiting to depart but all the goals are currently full. If only one becomes non-full, multiple trains will want to go to that goal in the same tick. Without source priority in the case from above it could happen that the train that already arrived to the concrete production dropped all of the iron ore and is now waiting to leave but it has no place to go because all the iron mines are currently full. If the train is stuck far away of possible goals it would keep being stuck here because other empty trains would be delegated first to get more iron ore from the iron mines which are closer to them. Basically not having a source priority would defeat the priority of a train stop. Train could go with high priority but would not be able to leave, which is dumb.

Should the destination priority be always equal to the source priority or should they be provided as a separate values?
Kaktusbot
Burner Inserter
Burner Inserter
Posts: 15
Joined: Sun Jun 04, 2017 6:54 pm
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by Kaktusbot »

boskid wrote: Sat May 08, 2021 11:11 pm Using train stop limits this case happens quite often when you see the "destination full" notification.
Ok, this is my mistake, I generally think that seeing "destination full" is evil and try not to overflow my train network with trains, sum of train limits in stops with the same name >= #trains having this station in schedule.
But if players do not follow this rule things can go wrong... or wait, what if this one train in the concrete production stuck there now, in 1.1? For example it got there because iron mine from which it departed was closest to concrete production and now this mine is empty, or simply it got in concrete production because distance penalties somehow made it closest for this train in past moment of time. As I understand 1.1 sourcing priority chooses based on distance too, just like destination priority. So this stuck situation can happen even now...
My point is - do you really need to fix this situation, maybe it's a player mistake to overflow network?
boskid wrote: Sat May 08, 2021 11:11 pm Should the destination priority be always equal to the source priority or should they be provided as a separate values?
If you want to add source priorities anyway both proposed solutions seems bad to me and I understand that there is no third.
Why bad? Because it make things complicated, it will be hard to understand how this all system works for a unprepared player.
But... you are right that this priority system is not required in vanilla and should be activated by mods only, and if player install mods with complicated byproducts system he can spend some time learning how complicated priority system works.
Ah, the joy of programming, finding compromises.
Well, if player wants to understand complicated system then the game should probably provide both destination and source priorities as separate values.
I just need to check if there is use case for separate values at all.

So, lets summarize this all:
1. If you add priority system to vanilla, I still think you do not need to fix problems caused by player mistakes. Destination priorities should be enough.
2. If you add priority system activated only by mods, lets throw all complicated stuff in the player's face.
User avatar
NotRexButCaesar
Smart Inserter
Smart Inserter
Posts: 1133
Joined: Sun Feb 16, 2020 12:47 am
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by NotRexButCaesar »

Kaktusbot wrote: Sat May 08, 2021 8:44 pm I tried googling "technical factorio discord" and found only one public server but there is no channel named "#train"
Is this "technical factorio discord" closed private server for chosen ones, like the ones who have access to the source code?
Or is it me who just can't understand how to use all this discord bells and whistles?
No, it is the server that accompanies the technical factorio reddit. Here is an invite link: https://discord.gg/yYapYe6g.
—Crevez, chiens, si vous n'étes pas contents!
Trific
Fast Inserter
Fast Inserter
Posts: 155
Joined: Thu Dec 31, 2020 7:57 pm
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by Trific »

Kaktusbot wrote: Sun May 09, 2021 1:29 am Ok, this is my mistake, I generally think that seeing "destination full" is evil and try not to overflow my train network with trains, sum of train limits in stops with the same name >= #trains having this station in schedule.
I don't think it is evil at all. I know I have the correct balance of trains when they occasionally briefly flash "destination full" upon picking up a load of material. Otherwise it is likely that I don't have enough trains to transport the needed amount of material demanded. "Destination full" when they are empty at the delivery depot is a major sin, but one of not enough supply, not too many trains.
But if players do not follow this rule things can go wrong... or wait, what if this one train in the concrete production stuck there now, in 1.1? For example it got there because iron mine from which it departed was closest to concrete production and now this mine is empty, or simply it got in concrete production because distance penalties somehow made it closest for this train in past moment of time. As I understand 1.1 sourcing priority chooses based on distance too, just like destination priority. So this stuck situation can happen even now...
My point is - do you really need to fix this situation, maybe it's a player mistake to overflow network?
If a bunch of trains are flashing "destination full" and a station raises its train limit such that they can now path to it, the first available train that the game looks at during that tick will get the reservation, and distance to the station is not considered.

Priorities are a fix for demand exceeding supply, and in the long run the solution is to fix the supply issue. My own rule of thumb is never more trains than available pickup slots, but sometimes bursty demand can cause a supply shortfall. For instance, my 900 SPM base requires only one train to deliver sulfuric acid over the long run, and the supply depot will buffer up to five loads. Sometimes the train will run through all of that buffer when all of the demanders (uranium mining, blue circuits, batteries) want some all at the same time. For short term imbalances, big buffers can cover, but I've been experimenting with a variety of methods to basically ensure that vital production doesn't run short due to bursty demand. They are all pretty fragile and not scalable.

I think a lot of this issue would go away if trains would repath to farther stations if the train limit at their current station has dropped below the number of trains that have reservations. I often have multiple delivery stops servicing a production line, and use a system similar to that described by the OP to only open the stops that are "emptiest". This is often the one that is the farthest away from the main line because it has the greatest path distance. A train takes that reservation, and then the circuits open other stops for other trains, but the original train hits a repath point and repaths to one of the slightly closer stops. The circuits then drop the limit at the closer stops in order to give priority to the farther one again, but dropping the limit doesn't cause a repath, and if the train does hit a repath point, it won't repath back to the slightly farther stop. I'm not sure I would want dropping the limit to cause a repath, but I think a lot of this problem could be fixed if there were some way to let the train repath to a farther station if the train limit dropped (give a penalty if the limit has dropped, for example).

Priorities as a whole could be implemented within the "path penalty" system. Just give train stops a way of reading a signal that adds to the penalty when pathing to the stop (negative values make the stop more desirable, or call it a "priority" rating, and subtract the "priority" signal from the penalty).
User avatar
NotRexButCaesar
Smart Inserter
Smart Inserter
Posts: 1133
Joined: Sun Feb 16, 2020 12:47 am
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by NotRexButCaesar »

Trific wrote: Thu May 13, 2021 9:14 am If a bunch of trains are flashing "destination full" and a station raises its train limit such that they can now path to it, the first available train that the game looks at during that tick will get the reservation, and distance to the station is not considered.
I believe it is the closest straight line distance train.
—Crevez, chiens, si vous n'étes pas contents!
Trific
Fast Inserter
Fast Inserter
Posts: 155
Joined: Thu Dec 31, 2020 7:57 pm
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by Trific »

NotRexButCaesar wrote: Thu May 13, 2021 7:36 pm
Trific wrote: Thu May 13, 2021 9:14 am If a bunch of trains are flashing "destination full" and a station raises its train limit such that they can now path to it, the first available train that the game looks at during that tick will get the reservation, and distance to the station is not considered.
I believe it is the closest straight line distance train.
That's... possible. My grenade production tends to call for a bunch of coal all at once, and I know the waiting trains don't leave the rack of stops in "pathfinding" order. It always appeared to me to be fairly random, although throw in the stops being lined up fairly perpendicular to the straight line path to the grenade production, and maybe a plastic factory throwing in a call too, and it ends up looking random. Definitely not closest end to the mainline first.

I have seen the issue of trains being stuck in delivery stops due to not enough supply when I miscalculated, but I don't think the proper solution is source priorities, but to either fix the issue of not enough supply, or at least supply something like a stacker the trains can go to. The major issue is when delivery stops get starved of material because trains keep repathing to closer stops that have opened while they are en route.

As for byproducts, my vanilla build BPs feature stops that material is taken from preferentially (i.e the belt from the regular train is side joined onto the belt from the priority recycling stop). I use them for recycling when I do a teardown, and was planning to use them on my IR2 playthrough to handle byproducts in a similar fashion. But it would need source priorities to replace the recycling runs with regular trains. Destination priorities would get them to pick up the priority material, but they wouldn't be able to make them leave to take it somewhere. I think in a situation like that, you would want to push both source and destination priority high in order to get trains to pick up and take the material out pronto, but in a "distant station getting starved" situation, you would only use destination priority, although I can't think of a situation where it would be undesirable to only have one priority at a stop that would be applied to both source and destination. If starvation is happening due to insufficient supply, you would want the empty train leaving ASAP to clear the spot for more supply coming in and want to give it priority over other trains. Although you might want to drop the destination priority as the starved stop is starting to get material, but still keep the source priority high to get the trains on a distant station out of there.

Don't mind the braindump, sigh.
User avatar
NotRexButCaesar
Smart Inserter
Smart Inserter
Posts: 1133
Joined: Sun Feb 16, 2020 12:47 am
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by NotRexButCaesar »

Trific wrote: Fri May 14, 2021 2:08 am I have seen the issue of trains being stuck in delivery stops due to not enough supply when I miscalculated, but I don't think the proper solution is source priorities, but to either fix the issue of not enough supply, or at least supply something like a stacker the trains can go to. The major issue is when delivery stops get starved of material because trains keep repathing to closer stops that have opened while they are en route.
Do you perhaps not have enough trains? Trains will only repath to a closer station if it doesn't already have a full limit. If you have enough supply and enough trains to carry that supply, closer stations should eventually back up and allow farther stations to be serviced -- even if you have fewer trains than you have spaces for unloading.

If you have trains waiting at unloading stations for more than a second or two: you need either more loading stations/throughput, or you have too many trains.
—Crevez, chiens, si vous n'étes pas contents!
Trific
Fast Inserter
Fast Inserter
Posts: 155
Joined: Thu Dec 31, 2020 7:57 pm
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by Trific »

NotRexButCaesar wrote: Fri May 14, 2021 2:46 pm
Trific wrote: Fri May 14, 2021 2:08 am I have seen the issue of trains being stuck in delivery stops due to not enough supply when I miscalculated, but I don't think the proper solution is source priorities, but to either fix the issue of not enough supply, or at least supply something like a stacker the trains can go to. The major issue is when delivery stops get starved of material because trains keep repathing to closer stops that have opened while they are en route.
Do you perhaps not have enough trains? Trains will only repath to a closer station if it doesn't already have a full limit. If you have enough supply and enough trains to carry that supply, closer stations should eventually back up and allow farther stations to be serviced -- even if you have fewer trains than you have spaces for unloading.
There are enough trains over the long run. The problem is when demand is bursty, and temporarily exceeds supply. The overall system has enough buffer to keep going until supply catches up, it's just that the closer in stations get all of the supply at first, and the farther out stations start to starve. The easy solution is to overbuild supply (including trains), the hard solution is to have starving stations communicate that to the other stations so they throttle down their demand temporarily, chewing on their buffer for a bit instead of instantly calling for a load when they have space. Priorities would let starving stations communicate that to the trains themselves without any need for crosstalk between stations.
If you have trains waiting at unloading stations for more than a second or two: you need either more loading stations/throughput, or you have too many trains.
Agreed. That is a "shouldn't happen" thing, and "standing at" a deliver site is a major red flag. Like I said, I've only had that problem when I screwed up somewhere, once when I miscalculated production, the other when I forgot to set the station names on a new mining patch.
evg-zhabotinsky
Inserter
Inserter
Posts: 22
Joined: Wed Dec 09, 2020 3:53 pm
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by evg-zhabotinsky »

My thoughts on the subject:

1. Destination priorities
I suspect that, in most cases, prioritizing train stops with largest "train shortage" (limit minus reservations) is sufficient. Unlimited stations have lowest priority.
"Time till buffer empty" might be better, which should be equivalent to prioritizing by train shortage divided by consumption speed, and that would require a separate circuit input for destination priority.
Static priorities might also be useful, depending on how you organize things, to override the default distance-based priorities.
However, all of above already can be implemented.
One option is using a chain of circuit-controlled rail signals before a station. The more are closed, the lower is priority (+1000m penalty per signal), and when a train comes close (an extra open signal turns yellow) they are automatically opened to let it through.
Another option is to use "global" circuit network connecting all stations to be prioritized. The major pain here is calculating a maximum signal over indeterminate amount of stations connected to a wire, but it is possible. Once done, all but the most starving station, as calculated by adding remaining buffer and incoming train count times cargo, set their train limit to zero, and the next train to path chooses the most starving one.
So, overall, adding explicit destination priorities would be just a convenience thing, maybe even "Making trains too easy!"

2. Source priorities
First, a train stuck in the middle of nowhere has the highest source priority, for obvious reasons. Repathing trains go next. And lastly, all trains stopped at stations use their station's source priority, either static or dynamic, to avoid byproduct buffer overflowing (in vanilla mostly oil and uranium).
Signals won't help here, but the circuit solution still works: Find the most "starving for free space" station, possibly limiting to those with a full train, and give only that train a signal to leave.
So again, mostly just a convenience thing, with a chance of "too easy trains", except for prioritizing stuck trains which is a user error anyway.
burninghey
Fast Inserter
Fast Inserter
Posts: 127
Joined: Fri Sep 14, 2018 2:06 am
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by burninghey »

How about adding a penalty based on how many slots are reserved?
FuryoftheStars
Smart Inserter
Smart Inserter
Posts: 2768
Joined: Tue Apr 25, 2017 2:01 pm
Contact:

Re: Add simple train stop priorities on top of 1.1 train limit system.

Post by FuryoftheStars »

Implemented for 2.0, unless I'm misunderstanding something.:

https://factorio.com/blog/post/fff-395
My Mods: Classic Factorio Basic Oil Processing | Sulfur Production from Oils | Wood to Oil Processing | Infinite Resources - Normal Yield | Tree Saplings (Redux) | Alien Biomes Tweaked | Restrictions on Artificial Tiles | New Gear Girl & HR Graphics
Post Reply

Return to “Implemented in 2.0”