CODE: Optimize Robot Orders outside reach of Roboports

[Taoh]

TL;DR

Make a third buffer for orders outside roboport reach for robot tasks.

What ?

I am making a fairly large base and was expanding via laser grids, and other large blueprints. I noticed a slowdown where it looked like the game did nothing. Based on that I believe that your current robotorder system works with 2 buffers, a 600 (in my case) active orders that tries to be filled that slowly time out, with a max of new orders in the buffer, and a larger buffer with everything else.

Now if you place a blueprint outside roboport reach it still goes in the larger buffer, but only X orders are checked per cycle, so if you have huge orders outside reach (landfill, pavement, or large laser grids/solar panel groups without roboports) a lot of those tasks can not be filled, but the buffer still uses its precious checks if they are available. This in term means that you get huge slowdowns where nothing is done as all that is checked is inaccesible stuff.

My suggestion is that everything works as now, but inaccessable things are moved to a third buffer when found (still cost 1 check, but no rechecks and initial check can be done on placement), and whenever a new roboport is put online its area is checked against this larger buffer, and if anything is found they are moved back into the secondary buffer.

A possible downside would be bases which turns roboports off and on a lot (power issues), but beside that the large scale constructions outside the roboports should be more achieveable.

[JohnyDL]

My understanding from what mods said in this thread viewtopic.php?f=6&t=49429 is that there's only one list for everything and it does one check of one item per tick.

The 600 items is actually a side effect of the last 600 (ten seconds worth) items that were attempted to build and failed.

I really like this idea but it won't actually work because of player roboports, if there were only infrastructure roboports this would be easy as you only need to surface scan when a roboport is plopped down a and ghosts could detect their roboport like buildings can connect to a power grid. But determining whether a ghost is inside or outside of the personal roboport range is none trivial.

If this were inexact ghosts in the 'inside' list that have no roboport could swap lists and those 'outside' that find they do move to 'inside' and its 2 checks per tick rather than 1 a 'almost guaranteed to be workable' and a 'hope the player has moved over here' ghost, that might be workable but the benefit might be minute

[Taoh]

I had understood that a players roboport had it's own scan for its smaller area, and if it has work slots available it assigns them to unassigned ghosts. So no need to change this logic (and if the ghost object has a reference to the "passive" stack, low effort lookup to remove it)

As for the only one list, that makes sense with the 10 second, 60 UPS. Still a list to move inaccessable builds out could speed up the construction as mentioned. I did not know how it worked and had assigned a lot of inaccessable builds and it looked like the game was doing nothing for the most time in regards to building. Now when I am careful it is working well, keeping the builds pretty much capped, but it is more a fight against the game engine instead of focusing on what I want done.

[JohnyDL]

This is a common misconception due to how it appears being completely separate from how it's coded, scanning the personal roboport area is not only inefficient it'd cause lag cause the surface scanning algorithm isn't designed or optimized for repeated running on large areas.

I'll repeat from the first page of the linked thread:

A thing to work on finds a robot not the other way around.

And the third page:

No we don't. We take a ghost from the list of all ghosts on the surface and check which logsitic network(s) it's in and try to dispatch a robot to handle it once per tick.

So once per tick the job at the top of the queue looks if it's in a roboport? Yes, is there a bot? Yes, does the associated logistics network have the item? Yes, do job. Otherwise move to back of the queue.

Which is why there can be noticeable delay between some ghost(s) entering your roboport range and it getting done. It doesn't matter if that roboport is a player or infrastructure the job/ghost looks for the bot and item rather than the other way round.

The inexact method would definitely improve things for placing ghosts so large they need to include roboports for the construction process but for 99% of players it would have detrimental effect at worst or be unnoticed at best.

[Taoh]

For arguments sake, lets say it is not a bad idea, instead it is a matter of implementation. We want a way that has no/low impact on current setup, but help in the edge case (no orders being dispatched for long periods because they are outside read.

Proposed solution is a secondary buffer for those tasks.

3 problems to handle

Adding Roboport
Removing Roboport
Keep player roboport relevant

---
Adding a roboport, check its area and add any ghost in extra list to main list ... that should be doable, is a onetime hit on placement, and if nothing in the list it is done quick. Lot's of optimization methodes for such checks.
Removing a roboport ... nothing done, but if the ghost is checked and outside reach (standard check) it gets moved to the secondary buffer instead. So it is either dispatch bot, or remove object from available list ... should have minimal impact, and gain if another cycle pass it again.
---
Personal Roboports.

You said there can be a delay from when you move into an area until it the order is picked up if the order is not part of the scan ... so add large chunck values too the secondary buffer. Add a 1 per second or even slower checks for chunks at and around the player, move anything there to the top of main buffer.

This accomplish two things. The players orders are put first, so game is MORE interactive now, and it ensure that chunks gets checked at all. They get removed at a failed check from above logic anyway. Additionally it handles the recheck of entities outside range that was the problem I would like to get solved.

[Impatient]

I get the feeling, that it is very tempting to jump into coding and design questions. But what is the actual problem and feature request from the perspective of a user who has no knowledge about and does not make any assumptions about the implementation details?

My try to distill it from your feature request (which goes far beyond that of an ignorant or indifferent user):

... well, i tried and there is no feature request from the view of an ignorant user in here. A bug report would be the max: "When a lot of ghosts are outside roboport reach, construction robot activity seems to come to a halt."

In my opinion such a bug report is much more motivating to those responsible, than pondering over your suggestions. Presuming, that your assumptions about how the game engine works, are correct in the first place. If the devs then respond with implementation details to explain or justify why and why not, then it is the right place and time to bring in and lay out your ideas - in my opinion.

This isn't an attempt to turn down your ideas. Maybe they are great ideas and very helpful. My reply is an attempt to structure the communication from a user to the devs.

There is a good reason why there is an agnostic approach between the different layers of software development and between software testing and software development. The reason is everyone is worrying about the things she/he can influence and does not carry any more stakes than necessary. The tester (you) says "Construction robots don't do anything when xyz is done. Fix it!" The dev thinks "Crap, the engine works this way, we need to change it." Maybe the dev comes up with a solution and at the next test the problem is solved. But maybe the problem is not solved or just altered in an unsatisfying way. An ignorant tester would just say: "The problem is still there. Fix it!" A non-ignorant tester, who brought in his own ideas, on the other side might be inclined to say: "This idea did not work, so I file it as unsolvable." Or he gets into an argument with the dev, about whether the implementation was done right or tries to adapt it or something else ... . And he might also not see other problematic areas when having a stake in the implementation. Point is, and I hope the example shows this, the tester is way out of his zone of influence suddenly has a stake more where his only stake should be to file correct and complete bug reports. It is similar with feature request. The only worry of a requesting user should be, if the feature request was done in a clear way.

I don't say never discuss implementation details or ways of coding something. It is fun to discuss it and exchange with the devs and users on the forum. I just say, don't mix everything together. Be clear about what you want. Express it in a clear manner. Imo a cool and heplful way to add own assumptions and ideas would be the appendix-style. "This is what I want. Period. And if you are interested in all my ideas on that topic, continue reading down there." Ever noticed, that the devs often respond to the initial topic of a thread, but rarely are seen taking part in any of the following discussions, arguments and sometimes fights? Imo this is because the already got what they needed: The problem or wish and the motivation for asking something.

[Taoh]

Perhaps this should have been a bug report and not a feature request. If it is posted wrong, well I made a post of my findings, past that not my circus, not my monkeys.

You brought up valid concerns of why my understanding was flawed, and I read those as a "Due to this it cannot be done", my arguments was meant to be along the line of "Then something else can be done, and should be doable efficient", and due to this, this feature request should not be dismissed.

As for the concerns, should it be done just because we can, is it priority, should it be a bug instead of feature, is it worth addressing, should a larger priority rework be made instead when looking at it, etc., well that is all stuff I will refrain from making any opinion on, that is decisions outside my reach, same as any opinion on actual implementation.

[JohnyDL]

Yeah I didn't raise the details of the other thread because this idea is bad but to point out that the way in which you think it works doesn't match reality

Code: Select all

Inside[0].dispatch()
Flag = Inside[0].flag //say 0 for dispatched, 1 for waiting on item and/or bot, 2 for not in roboport
temp=Inside[0].pop()
Select Flag
Case 0 Break
Case 1 Inside[0].push(temp) Break
Case 2 Outside[0].push(temp) Break
Outside...........

I'm assuming the flag is currently a Boolean, and the select case is just an if statement but this could work just by duplicating the current implementation and some tweaking rather than writing thousands of lines of code

This would rapidly speed up construction projects inside the range as they get checked more often and speed up those outside the range when a player walks over as they get checked more often too.

I wouldn't even scan when adding roboports, just place them and the lists will fix themselves eventually this means if you've placed a huge blueprint outside with roboports to expand, the items inside the area and thus closer will be focused on first rather than adding lots of ghosts with each new roboport and potentially slowing overall progress.

[posila]

I haven't read whole thread, just first three posts. There are lists for construction, deconstruction and repair and out of network variants of these lists. If there is more than 270 unassigned jobs total (which should take 1.5 second to scan trough normally), personal roboport starts searching its area regadless of the lists. It doesn't search whole are all at once, but divides it to 7x7 grid, so it takes 49 ticks to search whole area it covers.

Code: Select all

  ToRebuildList toRebuild;
  ToRebuildList toRebuildOutOfNetwork;
  ToRepairQueue toRepair;
  ToRepairQueue toRepairOutOfNetwork;
  ToDeconstructQueue toDeconstruct;
  ToDeconstructQueue toDeconstructOutOfNetwork;

[JohnyDL]

I haven't read whole thread, just first three posts. There are lists for construction, deconstruction and repair and out of network variants of these lists. If there is more than 270 unassigned jobs total (which should take 1.5 second to scan trough normally), personal roboport starts searching its area regadless of the lists. It doesn't search whole are all at once, but divides it to 7x7 grid, so it takes 49 ticks to search whole area it covers.

Code: Select all

  ToRebuildList toRebuild;
  ToRebuildList toRebuildOutOfNetwork;
  ToRepairQueue toRepair;
  ToRepairQueue toRepairOutOfNetwork;
  ToDeconstructQueue toDeconstruct;
  ToDeconstructQueue toDeconstructOutOfNetwork;

Now that is interesting so I got very lucky with getting this screenshot and personal bots doing nothing then?

[posila]

Now that is interesting so I got very lucky with getting this screenshot and personal bots doing nothing then?

I can't diagnose the problem from just screenshot, so feel free to post your save with this problem.

It's possible there is a bug somewhere ... for example, there used to be a bug - if a personal roboport found a ghost it couldn't build (because character didn't have required item), it would stop search.

[JohnyDL]

like I say I got super lucky, my bots handled it a few seconds after the screenshot, I'm sure it's not a bug