Friday Facts #374 - Smarter robots

[RedViper]

I don't see any reason why this would be noticably more UPS intensive. The operations involved are:
1. like 4 multiply operations to find the location the robot would recharge at
2. "find roboport to recharge at" (notably this operation would have had to be done anyway)
3. literally nothing else
I'm not clear why it's believed this problem exists, and I'd like you to explain what you think would be so UPS intensive

2. Why would they "have" to be done anyway? This operation is only done if the robot needs to charge, it's not done at all if it manages to go from point A to point B. So you're already adding redundant operations.

If you have a base optimized around robots most of the time they will be going straight from point A to point B.

Adding operations to check if they *can* make this trip is clearly more UPS intensive than not doing so.

Well, no, the implementation shown here is objectively better than the original, performing at least as well. Note that in the original, flying on low energy to the destination was not the behaviour the bots showed. They just got stuck. The new behaviour is an improvement.

I agree.

The solution we are proposing can *literally never* result in a longer path than the new behaviour.

I said it takes longer than going in a straight line, I didn't dispute that your implementation reduces travel time, just that it is more computational intensive.

No, the last part does not address it. If there is no closer roboport to the destination than the drone currently is, the drone will still turn around, away from the destination to recharge, at least, that's my interpretation.

I don't think this is the expected behavior, the last video doesn't seem to support it, either way it would be trivial to make it so it doesn't path to the same roboport it just came from. I'd also add that your solution doesn't fix it, it would still be possible to make bots get stuck, but I'd call that a base design issue and it's up to the players to fix it.

Also same as above, it doesn't seem to add any non-trivial performance cost to robot journeys.

You're adding redundant operations to bots behavior. I guess it would be better for poorly designed bases and worse for well designed ones.

The "solution" for bots running out of fuel and turning back, seems like a good idea when you have it set up as in the example. But if their start point was the nearest roboport to their destination, it wouldn't help.

Why not treat the bots like ants. If x number of bots have to turn around then the next x don't. Instead they sacrifice themselves for the greater good and drop down transforming into temporary mini charge points. The next tranche of bots can then get further and maybe they too will have to sacrifice themselves at 2/3rds of the trip. But eventually they will get there, with their temporary charge point brothers and sisters charging and cheering them on.

You can make it so if the nearest roboport is the one they came from they just continue towards their destination.

[MrGuffels]

I love this update. The construction changes and traversal loop fixes are huge.

One questions, Are we waiting until the full expansion is ready for these QoL updates or will they come out in smaller patches over time?

[RedViper]

I love this update. The construction changes and traversal loop fixes are huge.

One questions, Are we waiting until the full expansion is ready for these QoL updates or will they come out in smaller patches over time?

I think later acording to Kovarex https://www.reddit.com/r/factorio/comme ... s/jyng4bh/

[BicycleEater]

The "solution" for bots running out of fuel and turning back, seems like a good idea when you have it set up as in the example. But if their start point was the nearest roboport to their destination, it wouldn't help.

The construction area should prevent this under all conditions.
If a destination is within the construction area of a roboport, it must be close enough to a roboport that it can get there without running out of charge. Mods can break this, but that's kinda their own fault. It's not considered a supported case.
If the game could handle it, that would be cool, but I don't think the devs consider it at all a priority.

[BicycleEater]

With the addition of both queue's, have deadlocks been resolved? I saw no mention of this in the FF, and it was the first thing that crossed my mind as an interesting topic to discus.

Imagine a task to cut down a tree, and another to build a building in that location. What if both tasks are assigned to the same bot, but the tree task is second? This would cause an deadlock.

Another more complicated scenario, bot 1 tries to build building A and then deconstruct tree B, and bot 2 tries to building building C and then deconstruct tree D, but tree D blocks building A and tree B blocks building C. etc.

Ooooh, that's a really interesting point. I imagine they cope by adding the deconstruct to the orders first, but if you removed the drone that was going to deconstruct the tree initially, then the game could end up in a situation where it assigned deconstructing tree B after building building A onto the same drone, even when there were other drones in the network.
That's a really nasty one actually, as it's kinda arbitrary complexity to resolve.
Maybe ban having deconstruct orders after construct orders on the same drone? That would certainly be sufficient, but maybe inelegant.

[jgilmore42]

TL;DR: For personal roboports only, we really, really, really need jobs sorted by distance from the player.

This is awesome. Stupidity of construction and logistics robots has been a problem for a long time. But it didn't address my personal worst case scenario, which happens ALL THE TIME. So this is great and all, and it's certainly much much better, but you missed a trick. In particular, the "lake effect" where robots go over water and stall forever because they can't make it, is a problem that can be worked around with design. Though I'll readily admit that it's much more of a problem for new players who naively design something that won't work at all because of this problem.

My beef is this: Construction of railways (and other such things) with a personal roboport is extremely slow and inefficient, especially if you're walking! This problem is SO BAD, that at least one mod has added the only available workaround: modular personal roboports. By deliberately having ONLY ONE area expander, but lots of robot control and robot charging, you can... Well, not stop the problem, but at least make it not be nearly as bad. I notice in the example GIF you gave that this problem still exists - the robots prioritize the trees on the bottom of the area, and then go up in layers. I don't know if they're going in the order jobs were added to the queue, or geographically, or what. But they AREN'T hitting the closest jobs first.

You see, the more roboports you have, the larger the construction area is. And the larger the construction area is, the more likely the robots take construction jobs that are a LONG LONG WAY AWAY from the player. These jobs take such a long time that the whole process stalls. This leads to the robots failing to keep up with even a moderate walking pace, and I have to stop and wait a LONG TIME for them to catch up.

To see an example of this, build a four-track railway with maxed out (nothing else but batteries) MK2 suit, and you'll be cursing the designers of the robot AI in no time. It's even worse with conveyor belts though, because there's more of them. And with a wider rail or set of conveyors.

Honestly, this problem is really strongly evident any time a player with a large-area personal roboport setup tries to build anything large.

Conclusion: Smaller personal roboport construction areas are so much more efficient, that in some cases it's faster to build things with only one lower-tier roboport, and walk around a little.

Note: This is a really a noticable problem only for the players personal roboport, as the player moving means the time cost of jobs is subject to change. So whatever solution is implemented, it only needs to effect the player's robots, not the base as a whole. I mention this because sorting the whole damn job queue by distance, (distance to what?) and then repeating that every second or so, isn't really feasible for 10,000 robots. But for 200-500 it'd be a snap.

[BicycleEater]

...

I agree with @BicycleEater.
Literally every single statement he said is true.

Also, addressing the UPS performance. There are some ways to optimize robot pathfinding, like pre-calculating maps from tile to tile, etc.
However, even if the new pathfinding would be slower, it would be a breakthrough because robots are already UPS-intense entities so you should not rely on them when your base is so big that the performance drop would be noticeable. I.e., I am building a 2k eco-friendly megabase (which generates mathematically the lowest amount of pollution possible), and this means huge, enormous fields of 12-beaconed assemblers and 16-beaconed oil refineries. And you know what? I didn't have that many UPS drops while tens of thousands of construction robots flew away. Yeah, there were some drops, but there were tens of thousands of them. As long as you do not passthrough the entire factory bus via the bots, the UX should be fine.

She, but thanks!
Yeah, I ran some brief calculations (in a post the forum seemed to ignore, arg), and it roughly works out that even in short back-and-forth point-to-point hops, the savings from not travelling part way along the route before recharging are waaay more than the costs, even if you assume the calculations I'm proposing cost a substantial portion of the per-frame cost of a drone's operation. Like, at 1/3 the per-frame cost, across 10 tiles, it takes speed research level 15 to not break even.
And like, 4 arithmetic operations on data already in the CPU cache is not gonna cost anything like that much.

[BicycleEater]

TL;DR: For personal roboports only, we really, really, really need jobs sorted by distance from the player.
...

This could honestly be done as a tweak to the deconstruction planner/blueprint deployer, at least I'd guess... Just adding tasks in a different order onto the queue, so the close ones get assigned first.

[Hares]

Finally, I was wondering if it would be possible to use the GPU for robot planning. After all, it's one program running on many data (robots), which is a well parallelizable task.

At first, you could use multiple CPU cores to do so. Factorio runs mostly on a single thread, and parallelling tasks could improve the performance drastically. I.e, Circuit Network could definitely be split between cores.

[Hares]

The "solution" for bots running out of fuel and turning back, seems like a good idea when you have it set up as in the example. But if their start point was the nearest roboport to their destination, it wouldn't help.

The construction area should prevent this under all conditions.
If a destination is within the construction area of a roboport, it must be close enough to a roboport that it can get there without running out of charge. Mods can break this, but that's kinda their own fault. It's not considered a supported case.
If the game could handle it, that would be cool, but I don't think the devs consider it at all a priority.

Unfortunately, this can happen even in vanilla, if the robot flies by the edge roboport on medium charge.

[SquarelyCircle]

The apologetic nature of the beginning is unnecessary. The game is wonderful and complex, already. I am most excited by these sorts of QoL improvements. Things that make the game run better are what make we want to play more.
More flashy content is still cool. It's not unwelcome. But the QoL changes make me love the team working on it.

[BicycleEater]

I don't see any reason why this would be noticably more UPS intensive. The operations involved are:
1. like 4 multiply operations to find the location the robot would recharge at
2. "find roboport to recharge at" (notably this operation would have had to be done anyway)
3. literally nothing else
I'm not clear why it's believed this problem exists, and I'd like you to explain what you think would be so UPS intensive

2. Why would they "have" to be done anyway? This operation is only done if the robot needs to charge, it's not done at all if it manages to go from point A to point B. So you're already adding redundant operations.

If you have a base optimized around robots most of the time they will be going straight from point A to point B.

Adding operations to check if they *can* make this trip is clearly more UPS intensive than not doing so.

They would have to be done anyway because they would only be added if the system detected that they would have to be done anyway. I'm sorry I didn't make this clear, but part of step 1 is "work out if the drones can reach their destination without recharging", and if they can, do nothing. This is a very small cost, as it's no more than: squaring two ints, and comparing the result with the multiplication of two doubles (maybe three). Since all these numbers would be in L1 cache (if not registers) anyway, these operations would be absurdly fast. Like so so fast. Anyway, even if they weren't that fast, the drone would still save the cost of travelling half the distance every time it needed to recharge. Since this is fairly uniform across drone speed and distance, this is around 1/60th of the time cost of the drone's back and forth travel. So, provided these checks would take less than 1/60th of the time required to operate the drone (that's a full frame of drone processing every second, or for a fast short range drone (10 tiles distance, level 15 speed), 1/3 a frame of drone processing per journey)

I said it takes longer than going in a straight line, I didn't dispute that your implementation reduces travel time, just that it is more computational intensive.

My apologies, I read into it as a criticism, but that's fine then.

No, the last part does not address it. If there is no closer roboport to the destination than the drone currently is, the drone will still turn around, away from the destination to recharge, at least, that's my interpretation.

I don't think this is the expected behavior, the last video doesn't seem to support it, either way it would be trivial to make it so it doesn't path to the same roboport it just came from. I'd also add that your solution doesn't fix it, it would still be possible to make bots get stuck, but I'd call that a base design issue and it's up to the players to fix it.

The issue I'm referencing is that a drone goes past a roboport (not starts at), towards a destination. Then it gets low on power, and goes back to recharge at the roboport it went past. My solution would resolve this, as the drone would forsee running out of power, then recharge preemptively.
There are still issues, of course - the drone would just as happily pick a roboport the far side of the destination, and so pass the destination before going back, but again, I'm only working to improve, not perfect.

[jgilmore42]

[...]
Regarding the pathing over lakes: it seems that now you are using the selection of a roboport closer to the destination basically as a failure mode for when the robot needs to charge. Wouldn't it be possible to check immediately after a job is assigned to a robot if the distance is actually reachable without charging (either assuming a full charge or if possible considering the actual charge level of that robot) and if not immediately path to a roboport closer to the destination instead, then try again.

That would mean robots altogether avoid situations where they are out nowhere running out of charge and instead intelligently move along roboport lines. [...]

I had the same idea when reading the last section of the article. If the destination is to far away, to get there with one charge, the robot should plan find path over one or multiple roboborts that are within the maximum distance.

Robots don't do pathfinding, because it's too expensive. And it will almost certainly always be too expensive. Did you know that pathfinding lag is what kills most games of Dwarf Fortress? It is. Pathfinding doesn't have simple, efficient, solutions. Particularly in scenarios where the map can change, and the entity will need to update the path as it goes along. Ain't nobody got time (cpu cycles) for that!

Well, a solution COULD be coded, but it would involve lazy updates, storing paths (more memory!) and such. Not at all simple, a major time sink for the devs for little to no benefit. So unless there's MAJOR problems that can only be solved by proper pathfinding, we're not going to see it.

[Hares]

You've already made so many improvements in the cleverness of the behavior of various entities that it might be worth considering adding a "better software" item to the research (Seriously, it would feel great to suddenly unlock such improvements after using dumb robots for a while.)

Actually, I like that idea.
I don't know if this is technically possible in Factorio to keep two distinct implementations of the interface and choose one based on the game state, but the idea overall is goddamn brilliant.

[nuhll]

I like.

Do we get those now finally again every week???

[jgilmore42]

TL;DR: For personal roboports only, we really, really, really need jobs sorted by distance from the player.
...

This could honestly be done as a tweak to the deconstruction planner/blueprint deployer, at least I'd guess... Just adding tasks in a different order onto the queue, so the close ones get assigned first.

Nope. The player MOVES, and thus the order would need to be re-sorted, at least occasionally, for the tasks nearest to the player to stay at the top of the queue.

Also, I'm not sure what order the queue is in, it may already be sorted in some manner. There's no requirement that tasks only be added to the end of the queue, after all.

[jgilmore42]

You've already made so many improvements in the cleverness of the behavior of various entities that it might be worth considering adding a "better software" item to the research (Seriously, it would feel great to suddenly unlock such improvements after using dumb robots for a while.)

Actually, I like that idea.
I don't know if this is technically possible in Factorio to keep two distinct implementations of the interface and choose one based on the game state, but the idea overall is goddamn brilliant.

I admit that I have often fantasized about a "smarter robots" research being available. The trick would be to add it into the research tree in a way that new(ish) players are annoyed by the stupidity of the robots slightly before the research becomes available.

[Hares]

Robots don't do pathfinding, because it's too expensive. And it will almost certainly always be too expensive. Did you know that pathfinding lag is what kills most games of Dwarf Fortress? It is. Pathfinding doesn't have simple, efficient, solutions. Particularly in scenarios where the map can change, and the entity will need to update the path as it goes along. Ain't nobody got time (cpu cycles) for that!

Well, when people speak of pathfinding, they usually refer to dynamic obstacles and collision detection. Since bots are flying, none of them is applicable. There's just a number of static waypoints. Unlike the true pathfinding, this operation is quite cheap.

[jgilmore42]

Robots don't do pathfinding, because it's too expensive. And it will almost certainly always be too expensive. Did you know that pathfinding lag is what kills most games of Dwarf Fortress? It is. Pathfinding doesn't have simple, efficient, solutions. Particularly in scenarios where the map can change, and the entity will need to update the path as it goes along. Ain't nobody got time (cpu cycles) for that!

Well, when people speak of pathfinding, they usually refer to dynamic obstacles and collision detection. Since bots are flying, none of them is applicable. There's just a number of static waypoints. Unlike the true pathfinding, this operation is quite cheap.

Waypoints is pathfinding. It was popular with quake bots, for instance. That way they didn't have to worry about clipping through walls and such. There's no fundamental difference between finding paths through a network of roboports and finding paths through a grid of tiles - minor features of the network may differ, but the algorithms are the same. It's still pathfinding, and I'm fairly certain that's what the FFF author meant - pathfinding among a network of roboports is still a variety of pathfinding. And still expensive both in CPU and complexity of code. And the lighter you tried to make it on the CPU, the more complex the code would be, and the more difficult to debug. Which is my point, really.

[MrGrim]

This is fantastic! I love QoL features. I'd be happy if the expansion was nothing but them I have a couple of question if you have the time and inclination to answer.

First, is there any chance some of these can be backported to the current release over time? If there's no time, then so be it, but it would be very cool to get a small trickle of these QoL features to keep the appetite sated.

Second, re pathfinding, would a slow path/fast path kind of setup possibly work to get the best of both worlds? Something like:

Code: Select all

if (line between the source and destination intersects a logistics network border)
    // Slow path
    if (path cached)
        assign_path()
    else
        calculate_path() // Could maybe do off thread?
        cache_path()
        assign_path()
    end
else
    // Fast path
end

The path could just be a sequence of chunk ID's where the bot flies from chunk center to chunk center. Each leg of the journey could even be an item in the queue using the new queuing system. This way all of the extra work is hoisted into the initial job assignment and the bot tick function should remain as simple as it is now.

That initial check could be made pretty fuzzy too. The line could be at chunk granularity, and "intersect" defined as "this logistics network isn't present anywhere in this chunk". Maybe a cache could be useful here too?

Anyway, thanks for humoring me. Looking forward to what's to come!