Optimization, megastructures and logistic bots

[Chromatix]

Hi,

I didn't really know, where to post this, since I am going to tackle several different topics and try to merge them together into single suggestion for discussion.

I am aware of current "belts vs. bots" hot issue and I suggested a simple idea on that topic. This idea brought me to the broader issue such as possibility of adding new type of building and even new type of entity / object.

Basically, at certain point very late in the game, when players try to push their factories bigger, they start to break the "intended" behavior of the game. Reason for this is obvious, for such late game there is no actual goal other than making more infinite research faster and there is not other tool for it, than making larger, faster updating factories. This behavior natural, but it results into excessive usage of basic building blocks, such as basic buildings, belts, bots. Note that we are not talking here about hundreds or thousands of entities, but rather about tens or hundreds of thousands of them, each updating very rapidly. For instance, recently posted 1TW nuclear reactor. Here, the game simply breaks. The good thing is that such massive factory doesn't break the game engine, it rather breaks due to the hardware problems or the way the game communicates with the hardware (perhaps not the best formulation here, but I am quite aware of multithreading issues, its practical impossibility / difficulty-to-do at the moment, memory throughput etc.). Sure, you can try to optimize the shit out of the game, and you definitely should. But at some point, you probably realize, that polishing your core game data handling won't solve the problem that there are 750k entities requesting update several times a second. You also should try to multithread the code on CPU, or rather in my opinion on GPU, because simply - doing it for GPU is not that more difficult (you just need to approach data structures differently, perhaps think of them as sub-matrix updates), but with significantly higher yield - typical GPUs has up to 100x more computational power than typical CPUs.

You should consider both of these things first and I well know you do that. But still, both these measures move the problem only slightly further away. I am quite skeptical that even multithreading can make this 1TW reactor go 60 UPS (it is 3 UPS running atm.). With multithreading there is always huge performance loss, but i believe it is possible to do 30 UPS with it. Still, what happens when adding second 1TW reactor, or third, or tenth?

Of course, you can say that for such massive builds the game simply won't work, but this is the case we have now, so why bother with optimization or multithreading? I feel you want to make the game go faster without dramatically restrict the players. For this purpose I suggest dusting off the idea vaguely visible in Factorissimo mod.

The idea is simple - wrap a part of your factory into neat package and leave it laying around until you need to manage it again. Simple. The problem is, the actual mod Factorissimo, as I understand it doesn't solve the performance issue, since the inside of the factory is still "alive" in sense of updating, thus demanding full computational power from CPU. I must say I am not familiar to huge extent with it from the design point of view, I just know it as a player, but as far as I know, the mod generates ad hoc surface, which is then processed in real time along the main surface[1]. Wrapping a some part of your main surface factory into Factorissimo factory therefore doesn't solve anything from the performance perspective, on the contrary, it may even harm. But (and here finally comes my suggestion), what if you "approximate" the internal behavior of the wrapped factory and take it simply as a single entity. By approximation I don't mean some rough guess. It may be very well a totally deterministic function based on the number of input resources and power, internal speed of machines (or generally entities) and output bottleneck.

This may seem little bit odd, but I am thinking about it as a system with delay. In math, and continuous domain, you would use delayed differential equations for its description, i.e. its output based on its input. In discrete domain, you can perhaps use recurrence relations, or even simple algebra. In fact, if we know, what exactly is in the system and what exact inputs the system has, you can exactly calculate its outputs. This trivial statement has far-reaching consequences. You can use this approach for system of any internal complexity, built its function approximation and use it as a single entity. Single entity with custom internal logic will produce anything is internally programmed to and will consume corresponding resources. The difference is, its computational requirements from the outer perspective are way lower than inside.

Such wrapped factory would be still accessible the same way Factorissimo factory is. The difference would be, that the surface generated in the factory (containing all the machinery) would be inactive when no players are inside. So, if the players are inside - it works as normal factorissimo factory, is there are no players inside, the surface would stop/even load of the memory and the factory would work as a single object transforming inputs into outputs at given calculated rate, very similar as normal assembly machine. I find this approach "infinitely" (you know how it goes in computer science) scalable, definitely more scalable than multithreading.

Of course, here come all the NOOOO shouting. All this "megabase killer", "epicness killer" and "Factorissimo mod is bullshit, i like the complexity" etc. ... I understand. But before you start, stop for a while and think. When you make your megabase, how often do you need to manage 8754th electric furnace in your 2 sq. km smeltery region? Or how often do you even visually see whole your factory on the screen? I mean, when you start mass-producing lines of assembly machines, furnaces, batteries, reactors, anything, you kinda lose interest in particularly each individual object. You abstract it as a "furnace region", "power generation region" etc., or even "my furnaces" (all 5000 of them), my reactors (my 100 of them) etc. It certainly is nice to have the ability to change easily every single aspect of the factory (manually tweak an inserter in one massive furnace array), but how often do you do that from the global perspective? Don't you like lost of epicness when observing those arrays? You don't have to! Think about this wrapped part of your factory as a sector. This don't have to be a massive box. It can have the same visual style as the original entity array. Only you cannot directly manage the sector, you have to "open it" and "close it" the same way you enter and leave the Factorissimo factory.

Briefly about the megastructures and bots in the title.

When considering the above, you might think about new endgame gameplay options, such as massive smeltery object, massive nuclear plant - array of X single reactors. You can say that playfields like Space of underground cave system are suddenly more achievable, since they would be virtually inactive when the player is not present and would work only as input-output functions. Note that such function can work as output only function - for instance when I build space elevator to my platform (again, it is a surface), but which sends me down the resources mined from asteroids. The amount of resources-to-mine would be X when I leave the space platform surface, then it turns on "sending-resources-at-given-rate", and when I enter the platform again, it would be X-sent resources left.

Logistic bots are related to this only vaguely - if you are creating a new sector/megastructure, you need some time to process it and it must not be free, or even cheap thing to do. You can use logistic bots for material delivery in oppose to construction bots, which are for building purposes.

The only issue I see with all of this is that I am quite sure, that this must be supported in the core game. This "wrapping" cannot be modded. I may be wrong, but you are here to fix me.

Thanks for patience.
Say what you think.