Why are containers UPS problems?

[blazespinnaker]

I was reading somewhere the following. Not sure if it's accurate:

In: every slot checked with the first to see if there's room, stopping at the limit if there is one.

Out: Every slot (limit doesn't matter, you can manually put stuff in the red/limited slots) checked to see if there's something that can be taken out.

This seems very weird to me, as a programmer. I'd assume that pointers/indicies are used to cache bottom most slots for removing, and first non full slot for filling. Frequently (not always) containers contain the same thing in each slot so that should be sufficient for most cases. Likely the pointers will point to the same slot in these cases and only update when a slot empties or fills, generally just to the previous/next slot.

Chests which have different things in it might suffer from sort of cache miss, though some kind of hashtable could help theoretically. Also, if you disturb a container by manually removing something, then the in pointers may become less optimal and some searching will be required.

I can understand with filter inserters there might be an issue, but otherwise it seems to me chests shouldn't suffer from the above.

[Stringweasel]

As a programmer myself, I think it's very easy to look from the outside and think "it should just work, right?". And I've stepped it that trap a lot, because code is more complicated and not as simple as it seems. Yeah, sometimes it's possible, but is it a good idea? I always try to guage myself when I think that, because it's very easy to oversimplify the actual problem, and get closer to the thinking like:

Code: Select all

if (goingToCrash()){
    dont(); // should work, right?
}

But to your question, if I had to guess, the answer is they need to handle every scenario. Just like you mentioned filter inserters. Then there's robots, the multiple players, different item-types, chest limiting, not to even mention mods, all interacting with chests in weird ways. And this might be the best solution to handle the endless edge cases in a robust, maintainable, en deteministic way. Just a guess though

[blazespinnaker]

As a programmer myself, I think it's very easy to look from the outside and think "it should just work, right?". And I've stepped it that trap a lot, because code is more complicated and not as simple as it seems. Yeah, sometimes it's possible, but is it a good idea? I always try to guage myself when I think that, because it's very easy to oversimplify the actual problem, and get closer to the thinking like:

Code: Select all

if (goingToCrash()){
    dont(); // should work, right?
}

But to your question, if I had to guess, the answer is they need to handle every scenario. Just like you mentioned filter inserters. Then there's robots, the multiple players, different item-types, chest limiting, not to even mention mods, all interacting with chests in weird ways. And this might be the best solution to handle the endless edge cases in a robust, maintainable, en deteministic way. Just a guess though

Well, the quote above didn't come from a developer, so no clue if it's accurate or just an urban myth.

[FuryoftheStars]

All I know is that this is not the first time I've seen this come up on these forums and I have seen dev responses saying that containers are optimized and that so far all of the instances of "but look at this, this case is slow!" have been cases of modified containers with hundreds of slots being interacted with a slew of inserters replicated many, many times over. To which the devs have replied "umm, yeah, it will be slow like that no matter what we do."

[quyxkh]

Talking out of turn here, but hey.

Optimize for the common case. There are a *lot* of inventories in this game. A *lot*. Almost none of them ever use more than a handful of slots.

Optimize for the bottleneck. When Factorio bogs down, it's basically always on memory access. Bulking up the memory it has to touch to search/update inventories, to save a few cycles once you've loaded them, would be a bad trade overall, even if it completely eliminated the hot spots the game would still run slower.

[blazespinnaker]

All I know is that this is not the first time I've seen this come up on these forums and I have seen dev responses saying that containers are optimized and that so far all of the instances of "but look at this, this case is slow!" have been cases of modified containers with hundreds of slots being interacted with a slew of inserters replicated many, many times over. To which the devs have replied "umm, yeah, it will be slow like that no matter what we do."

Yeah, thanks. That's what I expected.

[MEOWMI]

I'm kind of speculating, but it feels like the short answer is "decent idea but not worth the effort (right now)". Like mentioned, not being needed in vanilla makes it a lot less desirable of an idea.

I don't think even building a hashtable of any sort would be even remotely trivial. At first glance, it seems like a good idea to use something like "use pointers/indices to point to the empty slot if it's a simple/well behaved case" (something like only 1 item type and no empty gaps), but just checking for that condition in the first place is going to be performance intensive. You could further store data for whether the container is "well behaved", cache the item type and make separate specialized functions for input/output depending on whether they break or sustain that state (players vs inserter using it etc.)... and if you kept on building that, you could probably make a system that has significantly better performance in a significant number of cases.

However this is all functionality that is never needed in vanilla. I don't see it being a thing just because of that. There are quite a few features that would be genuine feature improvements to the code, and this seems like one of those, but unfortunately a lot of them have to be triaged in favor of more important features. Despite how good as the codebase is, there's almost always more important work to do nearly everywhere.

It's of course always worth throwing around ideas.

[ssilk]

It’s not the CPU, which is the bottleneck with containers - as the original post correctly stated - it’s the memory bandwidth.

As it always is in context of Factorio.