Chests/inventories could probably be optimized

[jodokus31]

As far as i understand, the engine works quite ok, as long the chest sizes are not too big.
By adding an index to each chest, the memory usage would increase disproportionately to the benefit.

I further can imagine, that "something" could be optimized, if there would be any interest from the view point of the devs.

In the meantime, the devs might be more motivated to participate, if they would be treated with appropriate respect. Or are we already that far, that we have to provoke them hard to get any reaction?

[ptx0]

As far as i understand, the engine works quite ok, as long the chest sizes are not too big.
By adding an index to each chest, the memory usage would increase disproportionately to the benefit.

I further can imagine, that "something" could be optimized, if there would be any interest from the view point of the devs.

In the meantime, the devs might be more motivated to participate, if they would be treated with appropriate respect. Or are we already that far, that we have to provoke them hard to get any reaction?

i guess we could ask the scientist that reimplemented major portions of the factorio engine from scratch

[Architector_4]

Sounds to me like you're only considering CPU overhead and forgetting about other things like RAM, savetime, code maintainability etc.

"Just storing last non-empty cell index" introduces 4 bytes (an int) per chest (or 2 bytes if it's a shortint). With 5000 chests, that's less than 20KB of memory overhead. The value is very easy to figure if you have all chests in RAM, so even if storing 20KB in a savefile is a huge burden, I'd imagine it could very well just be recalculated for each chest on startup.

Even "just storing non-empty cell index" means that each inserter on every swing has to try to update that index. I don't know how much that costs, but it's not "free".

It's a trade-off. What is more performance demanding on one such swing - an inserter checking additionally 10 empty cells it doesn't need to, or an inserter reading that index once and then updating it? We may not know exactly, but at the very least it's something worth considering.

It doesn't improve the much more common case of full chests.

Well, it was only one suggestion. For all I know, it may be that the memory overhead associated with this optimization may be negligible even in full chest situations. With chests being an integral part of Factorio, as much as belts are, I feel like it's worth making at least some suggestions on making them faster, even if only in some cases, and even only for modded playthroughs.

Heck - considering that a much more common case of full chests has a much more common case of there being just one singular type of an item in a chest (i.e. only iron plates), a little bit different method of optimization may equally apply to empty and full chests! Maybe using some methods already in use with belt sides could help a lot - after all, we most often have rows of identical item stacks, rows of empty slots, and occasional non-identical one or two slots with less items. Anyway, brainstorming on another way to optimize it here lol

If an optimization only helping with a situation involving mods is considered not worth implementing, I don't see why other features and additions and stuff that is only helpful in the context of mods is even implemented in Factorio. If devs don't recommend people making "ridiculous" chest sizes and refuse optimizing that, why introduce the feature in the first place?

[eradicator]

If an optimization only helping with a situation involving mods is considered not worth implementing,

There's a different between "not worth it" and not wanting to. Look at anything related to hand crafting - devs want you to automate so they refuse to make hand crafting more comfortable. Only the devs know what the devs think about chests.

If an optimization only helping with a situation involving mods is considered not worth implementing, I don't see why other features and additions and stuff that is only helpful in the context of mods is even implemented in Factorio.

Flawed logic. Optimizing one thing does not imply anything about having to optimize everything else.

If devs don't recommend people making "ridiculous" chest sizes and refuse optimizing that, why introduce the feature in the first place?

You're thinking too binary. Allowing something does not imply recomending it. And not recommending it doesn't mean they have to waste time trying to prevent people from shooting themselfs in the foot. Someone exposed the raw data type without additional limits, I doubt they had a meeting about the pros and cons before they did it. Anecdote: The last time people pestered the devs too much about wanting to use performance-crippling numbers they hard-limited the maximum range for electric poles. So don't be suprised if the "optimization" ends up limiting max size down to 1000 or something.

Btw, I'm certainly not arguing against optimization. But I've been here for a while, and I've seen that there isn't time to implement everything that "makes sense". And maybe chests are just very low priority.

[Architector_4]

There's a different between "not worth it" and not wanting to. Look at anything related to hand crafting - devs want you to automate so they refuse to make hand crafting more comfortable. Only the devs know what the devs think about chests.

I understand that. Well, at least I can make a suggestion, and, if the thunk the devs did was originally not in favor of my suggestion, probably sway it the other way, as that would be convenient for me personally and for a bunch of other players who end up using gigachests.

Though, I'd argue that altering hand crafting as a gameplay element i.e. altering game design, and an internal optimization that does not (in theory) alter absolutely anything about gameplay design except potentially bring more UPS or save CPU cycles, are two different things, and "not wanting to" do the latter - assuming it's possible and doable and the devs just don't want to for no valid reason - doesn't sound right.

But I don't know if there is a valid reason or not, nor how reasonably was such a decision assessed or reassessed the last time, nor when that last time was, nor if the potential various methods of optimization were considered or not, nor whether it makes sense or not to optimize it at all (it sounds like the kind of a thing that will bring UPS benefits even in vanilla steel chests due to them having whole 48 slots, most often filled with 1 type of item and empty slots, but I can't know).

I am providing my own opinion, since the devs provided a way for me to, in hopes of having the game improved for many other players beside myself (if such an optimization applies to vanilla chests), or at least a few like myself, because I love the game. At least it'd be heard and considered, which in itself is neat.

So don't be suprised if the "optimization" ends up limiting max size down to 1000 or something.

Honestly, that'd be neat. It kind of feels weird to have a system that is made and optimized for one workflow, yet allows another workflow, but that another workflow is not recommended even though it's implemented and allowed... You are making a fair point of it being not that binary, but honestly, I personally would prefer it to be that way! A definite "yes, we support and optimize for it" or "no, we don't allow it".

[jodokus31]

So don't be suprised if the "optimization" ends up limiting max size down to 1000 or something.

Honestly, that'd be neat. It kind of feels weird to have a system that is made and optimized for one workflow, yet allows another workflow, but that another workflow is not recommended even though it's implemented and allowed... You are making a fair point of it being not that binary, but honestly, I personally would prefer it to be that way! A definite "yes, we support and optimize for it" or "no, we don't allow it".

Be careful with those kind of demands.
There is f.e.: this linux feature of non-blocking save, which is not safe and supported, but many people appreciate it.
By your logic, it should be removed, but many would miss it deeply

Also, it sometimes better to don't over regulate things, to keep it simple

[Architector_4]

By your logic, it should be removed, but many would miss it deeply

I understand that and as a Linux user I appreciate that one in particular too. With such a feature however, I feel like it's obvious (or at least I expect so) that they are basically leaving that particular feature as a public beta test, not recommended for general usage if you 100% want to keep your world safe, but nonetheless available because the developers are ready and willing to fully fix any bugs and issues that may arise with it, to make it more safe and general use ready. Here it only makes sense that they would keep the feature in one way or another, for players to test it and report back when something is wrong. They want the willing players to use it.

With gigachests however, it seems like the kind of a feature that isn't supported or recommended specifically because it isn't optimized well, and it isn't optimized well because the developers just don't want to, because it isn't a part of vanilla gameplay. There's no reason or need for the developers to have anyone test that feature, and it doesn't seem like they would even want anyone to use it since any time anyone points out issues with it, they are simply recommended to just not use that feature and stray away from it. At that point, why even have it if you don't want the willing players to use it?

[eradicator]

I personally would prefer it to be that way! A definite "yes, we support and optimize for it" or "no, we don't allow it".

I guess we can agree to disagree then :p. Development costs time and money. If they had to make binding promises on every tiny thing they implement they wouldn't get very far.

[Architector_4]

If they had to make binding promises on every tiny thing they implement they wouldn't get very far.

Makes sense, but there's a difference between implementing a thing and not making binding promises, and implementing a thing and refusing to maintain or improve it in any way specifically, i.e. making a promise not to. Like, if you are going to implement a subpar feature, why set it in stone to be in a state of being there and not deprecated, yet being there in the subpar form and never improved lol

Though at this point I feel like we are trying to play mind games and 6D chess on what Factorio developers think and don't think, and arguing over the possible ways this question can go with specific ideas and stuff in mind, in pure hypotheticals that may be absurdly far from the truth. So yeah, at this point, it may just be worth it to agree to disagree. :D



Anyhow, here's a suggestion on one random thing that may probably be improved, and may probably benefit a subset of Factorio players, and suggestions on how to improve it, aswell as various additional discussion below that regarding the subject matter. At this point, my biggest wish in this thread is to at least get some kind of a response from a Factorio dev or anyone related saying something conclusive, so that I can either get my hopes up and hyped for 1.1.33, or get over it and make my next playthroughs with plans of never using big chests, with Merging Chest mod's limit set to like 50 so that the only use it would bring is with chest forms lol

Still love the game, after all! And even though I might ended up being an ass in this discussion, I nonetheless still love the devs. Thanks for all the hard work, gigachest related or not. <3

[eradicator]

a Factorio dev or anyone related saying something conclusive

Wouldn't get my hopes up on it getting any more substantial than in the threads that @disentius linked. Imho any optimization would either (suddenly be announced in a changelog or FFF) or (never happen).