Round robin scheduling for trains / Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post your ideas and suggestions how to improve the game.

Moderator: ickputzdirwech

GPFault
Burner Inserter
Burner Inserter
Posts: 11
Joined: Mon May 01, 2017 6:10 am
Contact:

Round robin scheduling for trains / Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post by GPFault »

Hello,

>> What did you do?

I have several unload and load stations. I found that some trains are never moving. I created a test setup to highlight the problem.
I placed three "A1" stations and one "B" station with a limit of 1 train. I placed three trains with the same schedule: move to "A1", wait 5 seconds, move "B", wait 20 seconds (time enough to move from B to A1).

>> What happened?

The train above never moves.

>> What did you expect to happen instead? It might be obvious to you, but do it anyway!

I expect that all trains will visit "B" station one after one.
Perhaps it needs to make that train early found that "Destination full" will move early than a train that found the same later. It will balance the load evenly.
Screenshot_6.png
Screenshot_6.png (1.46 MiB) Viewed 1901 times
Save file:
nevermove.zip
(1.74 MiB) Downloaded 127 times
Import Blueprint:
0eNqdnO1OGzsQhu9lfwfk77FzK0cVSiFqVwoBJaE6COXem5BsWpLZ7vvOPyjwMOvxrMePXT6676u35eumX++6+UfXP76st938v49u2/9YL1bHf9u9vy67edfvls/drFsvno+fbRb9qtvPun79tPy/m/v9t1m3XO/6Xb88/fznJ+8P67fn78vN4RsuP/n4tvm1fLr7BMy615ft4Wde1sdfdOCE6tyse+/mRcKB/tRvlo+nL5f97AYaYGjVoVGBxgt0uzvwfvzcjWGlyRmbv2JFwSYYW73XsV7BZnAIpKUztE5Dy5c0352ngjoA93mI9f4q2qyABU2Y/xOtgqlEiuIJVKMGakRS8r9A3hlCyirJG2LSScEQk6ikaIhJJyVDTE0lZXCSVh+HSVrbvT5SxfB8elTCP1/zKqnyMY2QmiEmdZYHZ4hJJ3lDTGruQjDEpJMi/C4NnxxxbvqVF5IhPrV2QjaQ1Fka+PkuTp1bAX2jSy1nTvk6ZkGjEi/42s7cOL20hwYvbP5+eO6ILGzRGUZUrYzoDSR1Pke0PZLkzpw63XNFoj9KQ3wynfNIrAxJCG4muI3g4jUk2RNcYgXJkeASNZWZvBGrSybyloiOKhN5S8TKU4i8JaLbKkTeElFvhchbIuqtMHkj6q0weSPqTZi8EfUmTN6IehMmb0S9CZG3TNSbEHnLRL1VIm850P3/offQ+/8cDWuu2qNloqriOTveT9uDjO52JNXL09brzkVbz/OfujoGvL7b7l5e/5n0dpWc2eFRF6ePu077DWIYXbVvzRXtaGI6j20CFApvAY4507yJgzuuoMenZah4vjPycbqCSuA7I4gb+c4I4ia+M4K4me+MIG7hOyOIK3xnBHEr3xlB3MZ3RghXHN8ZQVzPd0YQN/CdEcSNfGcEcRPfGUHczHdGELfwnRHEFb4zgriV74wgLlFvlchbJeqtEnmrTIdI5K0G3iJBXOaEiMhb5d04xiXqrTF5K/RJGcZlHDqTN8KoOyZvRP/oiLw1Rx5v3nSTmgNtBr/o9cMzor7ikCeZ3uu0CO912rDX8QnZ67RE7nV8Jvc6Dd6ntUrGXugD5JvR1nY/zbA78+ouulXD3rcB86EZuDJdCd458jhagtMPfy01pZ+POqao4mUQVVRkd8shTM8X75LhYUcizPzDBv2M0xmOtMZQYohq5FZAZY9gAiAEvGv8GUwAVhzPXGMYVAMG9rxrwMCBlw0YOPK2AQMnXjdg4Mz7BgxceOGAgYU3Dhi48soBAzfeOUDg4HjpgIE9bx0wcOC1AwaOvHfAwIkXDxg48+YBAxdePWBg4d0DBq68fMDAjbcPEDg6Xj9gYM/7BwwceAGBgSNvIDBw4hUEBs68g8DAhZcQGFh4C4GBK68hMHBjPUQAPIRPhitTI202c6fj0rHrN0GZaxzDZjMAUsMng9UIkBnwf13lmECLZ9GZNCaBNSY+4bfN6YER3pkEwJn4VA1zd2TCNXaXHh0QIXOd41ISupLIntUkUdckORhGbSSmaHi8kXvkBp8xhjL4jKhLiGzwGWMog8+I+os2G+b9GKoZohq5vW9YScZQhpUk6lO0GC4oRRn7DwqGG0pjcSXDI+rzvcC2O6bLI7brV3dS0Zb5PxKlYf4nvZSKYf6Poej3firAuAnttK+xWcV6vi1KDQHTl9RzALotiWzPeo1VW2FJcE27YcKnm2t46oItme9VrkdY7bMF7q/ypTs8foQMh4Dd4aAQ8tVVx/KlO5x1z4v122L18AnbPqz65343NlwV7hod+1QGu56RjVQ12HUMbLDrGNhg1zGwwa5jYINdx8AGu46BDXYdAxvsOgY22HUMbLDrELgZ7DoGNth1DGyw6xjYYNcxsMGuY2CDXcfABruOgQ12HQMb7DoGNth1BBycwa5jYINdx8AGu46BDXYdAxvsOgY22HUMbLDrGNhg1zGwwa7fgr/NTn9GZf7XX12Zdb+Wm+3pGw6bMmmHHq20UFzd738D1A9+nA==
Last edited by ssilk on Wed Jan 19, 2022 7:10 am, edited 1 time in total.
Reason: Extended title to merged thread

mrvn
Smart Inserter
Smart Inserter
Posts: 5112
Joined: Mon Sep 05, 2016 9:10 am
Contact:

Re: [1.1.46] Wrong scheduling for trains (no round robin)

Post by mrvn »

I'm split on this.

On the one hand round-robin of trains is what other cases use.

On the other hand activating the closest train is also good.

Is ore from one mine any better than ore from another? Or iron plates from a smelter? What situation do you have where round-robin is better?

User avatar
Klonan
Factorio Staff
Factorio Staff
Posts: 5030
Joined: Sun Jan 11, 2015 2:09 pm
Contact:

Re: [1.1.46] Wrong scheduling for trains (no round robin)

Post by Klonan »

I am moving this to ideas and suggestions, nothing here is to suggest this is a bug

GPFault
Burner Inserter
Burner Inserter
Posts: 11
Joined: Mon May 01, 2017 6:10 am
Contact:

Re: [1.1.46] Wrong scheduling for trains (no round robin)

Post by GPFault »

mrvn wrote:
Wed Nov 10, 2021 10:01 am
Is ore from one mine any better than ore from another? Or iron plates from a smelter? What situation do you have where round-robin is better?
Please consider if you have two Red Circuits block and one Blue Circuits block as "A1" stations (Blue is on top). And station "B" is Green Circuits supplying block. Blue Circuits will never get a chance to start production until Red Circuits will not be filled with trains. It is pretty difficult to manage. With round-robin production of Blue and Red Circuits will be more balanced.

mrvn
Smart Inserter
Smart Inserter
Posts: 5112
Joined: Mon Sep 05, 2016 9:10 am
Contact:

Re: [1.1.46] Wrong scheduling for trains (no round robin)

Post by mrvn »

GPFault wrote:
Wed Nov 10, 2021 3:04 pm
mrvn wrote:
Wed Nov 10, 2021 10:01 am
Is ore from one mine any better than ore from another? Or iron plates from a smelter? What situation do you have where round-robin is better?
Please consider if you have two Red Circuits block and one Blue Circuits block as "A1" stations (Blue is on top). And station "B" is Green Circuits supplying block. Blue Circuits will never get a chance to start production until Red Circuits will not be filled with trains. It is pretty difficult to manage. With round-robin production of Blue and Red Circuits will be more balanced.
True. If you look at this from a supply point of view with trains stationed at the consumers.

I tend to have the trains at the producer except for ore. Because:

1) generally there are more consumers than producers so it saves trains
2) trains as the producer will be filled already when a consumer needs goods and only need to travel 1 way instead of a round trip. So delivering items is faster.
3) factorio is more tuned for over-producing than over-consuming, balancing happens when one consumer backs up and stops consuming

But it's a valid example to have it the other way around. Not sure how to fix this unless you manually balance your stops with circuit logic.

In the long run the red circuits will back up at some point unless you constantly drain them with something else and then blue circuits and red circuits both get trains. It just takes forever till that happens with the amounts of green circuits consumed. And every time the blue circuits eat a red circuit train they get exclusive deliveries till they are backed up again.

The simplest solution is: Produce more green circuits.

User avatar
Khagan
Fast Inserter
Fast Inserter
Posts: 141
Joined: Mon Mar 25, 2019 9:40 pm
Contact:

Re: [1.1.46] Wrong scheduling for trains (no round robin)

Post by Khagan »

mrvn wrote:
Thu Nov 11, 2021 1:13 pm
The simplest solution is: Produce more green circuits.
In fact, just produce more of everything. The factory grows ...

GPFault
Burner Inserter
Burner Inserter
Posts: 11
Joined: Mon May 01, 2017 6:10 am
Contact:

Re: [1.1.46] Wrong scheduling for trains (no round robin)

Post by GPFault »

mrvn wrote:
Thu Nov 11, 2021 1:13 pm
3) factorio is more tuned for over-producing than over-consuming, balancing happens when one consumer backs up and stops consuming
It is always over-consuming of something :)

DarkShadow44
Fast Inserter
Fast Inserter
Posts: 168
Joined: Thu Jun 01, 2017 12:05 pm
Contact:

Re: Round robin scheduling for trains

Post by DarkShadow44 »

How do you propose this should be implemented? Because round robin is not always desired either...

Nidan
Long Handed Inserter
Long Handed Inserter
Posts: 96
Joined: Sat Nov 21, 2015 1:40 am
Contact:

Re: Round robin scheduling for trains

Post by Nidan »

You can force round robin behavior using the circuit network. Implement a counter or pass around some token to control which station releases the next train.

User avatar
MEOWMI
Fast Inserter
Fast Inserter
Posts: 229
Joined: Wed May 22, 2019 12:21 pm
Contact:

Re: Round robin scheduling for trains

Post by MEOWMI »

It is nice for figuring out that you aren't producing enough, because it's easier to notice one station struggling to keep up with production than ALL stations struggling to. As others have pointed out, you almost always want to have an excess of supply trains.

Hard to say exactly which way is better, though I did sort of expect it to be round robin since inserters work this way and IIRC trains trying to reserve the same block works like that too.

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 12719
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Round robin scheduling for trains

Post by ssilk »

GPFault wrote:
Wed Nov 10, 2021 9:53 am
I have several unload and load stations. I found that some trains are never moving. I created a test setup to highlight the problem.
I placed three "A1" stations and one "B" station with a limit of 1 train. I placed three trains with the same schedule: move to "A1", wait 5 seconds, move "B", wait 20 seconds (time enough to move from B to A1).
I just would add a station “Z” which is in front of B and is able to hold at minimum 3 trains (nearly as much as are on all “A” stations).

The schedule for all trains is then A1, Z, B.
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

Tricorius
Fast Inserter
Fast Inserter
Posts: 224
Joined: Fri Jul 01, 2016 9:04 pm
Contact:

Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post by Tricorius »

TL;DR
Trains (particularly with dynamic train limits) can soft-deadlock when both “ends” of a loop are “full” (in regard to Train Limit).
What ?
It’s not my solution, so I’ll point to Nilaus’ recent video: https://www.youtube.com/watch?v=i2DyxgQxxmA&t=245s.

Scenario: fairly standard logistics train “loop” consisting of load (e.g. iron ore mine) and unload (iron ore for smelting).

Assume two stations, load and unload; two trains; and both stations set with Train Limit: 1.

Trains that have fulfilled their “leave” criteria (fully unloaded / fully loaded / satisfied circuit condition) are still counted toward the train limit. In the stated situation, both stations are at Train Limit: 1/1 and soft-deadlock (neither station accepts either train). They both wait with “Destination Full”.

Removing “pending” trains, or whatever the condition is called, would set both limits to 0/1 and allow both trains to proceed to their next station, freeing the deadlock.
Why ?
It’s frustrating debugging clashing states and I think this soft-lock could pretty easily happen to someone without in-depth knowledge of the train system; therefore being nearly impossible to debug. I can’t think of a time when this would be an undesired behavior; so I don’t see any downsides.
Last edited by Tricorius on Wed Dec 22, 2021 11:01 am, edited 1 time in total.

Kyralessa
Filter Inserter
Filter Inserter
Posts: 373
Joined: Thu Sep 29, 2016 5:58 pm
Contact:

Re: Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post by Kyralessa »

If you only have two stations (in this schedule) and two trains, why are you using train limits in the first place?

Tricorius
Fast Inserter
Fast Inserter
Posts: 224
Joined: Fri Jul 01, 2016 9:04 pm
Contact:

Re: Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post by Tricorius »

Kyralessa wrote:
Wed Dec 22, 2021 10:38 am
If you only have two stations (in this schedule) and two trains, why are you using train limits in the first place?
I don’t. This is a simplified example. I have blocks of sets of stations. But the math is the same for 2 or 2000. If you have the same number of trains as you have stations with train limits they soft-deadlock. (Ten loaded trains, 2 each in five blocks of loaders; ten unloaded trains, 1 or 2 of each block of unloaders. It’s not fun to debug. I, for one, wrongly assumed that only an active “at the station” train or one in transit had a reserved spot in the limit queue.)

Also, the train limits get set dynamically based on the number of loads available in the chests at the station.

If it has enough to load (or unload) one train, then the limit is set to 1. If it’s storing 4 train-loads it is set to 4. (Although there is a max override in case there are only two local waiting branches.

For the curious, here’s a megabase (Nilaus) showing the problem at a much grander scale: https://youtu.be/uyenBf4g9Zk

There could also be reasons you’ve set Limit: 1 on a simple “loop”. If you didn’t build a stacker, you don’t want a second train blocking the main railway (again, intermediate rail knowledge wouldn’t necessarily know they need waiting branches or off-network load / unload so a second train may clog their entire network). This solution would resolve these soft-deadlocks.
Last edited by Tricorius on Wed Dec 22, 2021 11:05 am, edited 1 time in total.

User avatar
disentius
Filter Inserter
Filter Inserter
Posts: 659
Joined: Fri May 12, 2017 3:17 pm
Contact:

Re: Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post by disentius »

To clarify the "bug":
1. train wants to leave
2. there is no free station available. All destinations report x/x as trainlimit/train reservation.
3.The train has left the station (no "T" signal) yet the station counts the departed train toward the reserved trains in x/x

Tricorius
Fast Inserter
Fast Inserter
Posts: 224
Joined: Fri Jul 01, 2016 9:04 pm
Contact:

Re: Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post by Tricorius »

disentius wrote:
Wed Dec 22, 2021 10:57 am
To clarify the "bug":
1. train wants to leave
2. there is no free station available. All destinations report x/x as trainlimit/train reservation.
3.The train has left the station (no "T" signal) yet the station counts the departed train toward the reserved trains in x/x
Exactly. The proposed solution: if a train has “left the station”, it should free it’s reservation from the limits queue / array.

mmmPI
Smart Inserter
Smart Inserter
Posts: 1600
Joined: Mon Jun 20, 2016 6:10 pm
Contact:

Re: Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post by mmmPI »

Tricorius wrote:
Wed Dec 22, 2021 9:48 am
Removing “pending” trains, or whatever the condition is called, would set both limits to 0/1 and allow both trains to proceed to their next station, freeing the deadlock.
Yes it would allow both train to move in the example.

But it is not something you can generalize (imo).

All the trains with "destination full" located at the top of example in the video would be removed from all their stations since they all have the condition for that particular station clear.

This means the game would see 4 or 5 station "open". But in this case sending 4 or 5 trains could cause problems.

In the example it's a particular case, the one where the soft-deadlock occur in a visible way but with only 2 different station name and also 1 train only in one of the 2 areas containing train stops.


If you where to have train schedule with A B C for example, and more trains, then some trains could be considered "pending" for C being stationnary at station B preventing some trains leaving A. The Soft deadlock could still occur if those trains waiting at A prevent train at C from completing the 3step loop.

But in this case, it could be that train at B are waiting for materials. trains at A are thus logically waiting for room in the next train stop. and preventing trains at C from piling up behind them despite train condition for train stop A being fulfilled. This is the acual behavior. Here the proposed change would allows all trains at C to complete the loop and you'd have more trains behind each A station than you set the limit to. ( potentially causing train waiting outside stackers or crash if you don't use signals :) ).

In other word, the proposed change wouldn't function in all soft-deadlock situations but would cause some existing system to break. ( at least from my understanding maybe i've missed someting).


On a side note the same thing can happen if you only have 2 block in your train network, and 2 trains. Then each train is on its own block and there is a soft deadlock. Or let say 5 block and 5 trains. The same logic apply if you have 20 trains and the sum of the limit allowed in your station is also exactly 20.

Tricorius
Fast Inserter
Fast Inserter
Posts: 224
Joined: Fri Jul 01, 2016 9:04 pm
Contact:

Re: Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post by Tricorius »

mmmPI wrote:
Wed Dec 22, 2021 11:03 am
Tricorius wrote:
Wed Dec 22, 2021 9:48 am
Removing “pending” trains, or whatever the condition is called, would set both limits to 0/1 and allow both trains to proceed to their next station, freeing the deadlock.
Yes it would allow both train to move in the example.

But it is not something you can generalize (imo).

All the trains with "destination full" located at the top of example in the video would be removed from all their stations since they all have the condition for that particular station clear.

This means the game would see 4 or 5 station "open". But in this case sending 4 or 5 trains could cause problems.
This is a good potential point and one reason I posted this in Ideas and Suggestions instead of bugs. There could be unforeseen side-effects.

And if I recall the whole Train Limits feature was added to help reduce the “18 trains race to a single station that just opened” issue.

However, in the video those stations are closed. When they were open the trains were running smoothly since there was a free station available. However, adding another train in the empty branch and enabling all the stations would likely trigger this same condition. And if all five of those trains were empty there could be a rush of trains.

Perhaps a better solution might be for the limits system to check if a train pair are both “waiting to leave” and therefore have valid paths to each other (or basically is there a receiving station for this train that wants to leave which *also* has a train wanting to leave) then, in very certain circumstances schedule an overflow limit (or whatever has to happen behind the scenes).

asheiduk
Long Handed Inserter
Long Handed Inserter
Posts: 64
Joined: Sat Dec 05, 2020 9:46 am
Contact:

Re: Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post by asheiduk »

disentius wrote:
Wed Dec 22, 2021 10:57 am
To clarify the "bug":
1. train wants to leave
2. there is no free station available. All destinations report x/x as trainlimit/train reservation.
3.The train has left the station (no "T" signal) yet the station counts the departed train toward the reserved trains in x/x
I'd like to extand 3) with: ("C" signal is still greater than zero)

The tooltip for the "C" signal is: "Number of trains going to this train stop. Includes the stopped train" So even the tooltip suggests that "Read stopped train" and "Read train count" should correlate -- but don't actually.

Tricorius
Fast Inserter
Fast Inserter
Posts: 224
Joined: Fri Jul 01, 2016 9:04 pm
Contact:

Re: Train Limits can Soft-Deadlock when Incoming and Outgoing are Both Full

Post by Tricorius »

asheiduk wrote:
Wed Dec 22, 2021 11:31 am
disentius wrote:
Wed Dec 22, 2021 10:57 am
To clarify the "bug":
1. train wants to leave
2. there is no free station available. All destinations report x/x as trainlimit/train reservation.
3.The train has left the station (no "T" signal) yet the station counts the departed train toward the reserved trains in x/x
I'd like to extand 3) with: ("C" signal is still greater than zero)

The tooltip for the "C" signal is: "Number of trains going to this train stop. Includes the stopped train" So even the tooltip suggests that "Read stopped train" and "Read train count" should correlate -- but don't actually.
Excellent point. My understanding is that “C” is basically the first number (x) in the Train Limit: x/y label.

Post Reply

Return to “Ideas and Suggestions”