[Oxyd] [0.15.12] Crash when killing biters and performance issues

[Qon]

So I had some performance issues. I turned my base off completely (Production stats don't even exist atm, machines still have power from creative mode mod though). No impact.
So I checked my pollution cloud. Fairly big, shown in the attached picture. Map view at max zoom. It's concrete everywhere and no trees (Flatland mod) so natives are the only ones really eating my pollution. I think I've removed my pollution by /c ommand earlier (to improve performance) but my base outputs quite a bit (Sent 6k rockets so far). The base was bigger earlier, but it's a bit less than 6 x 8 chunks big on the map atm, for scale.

So I tried using creative mode mod "kill all biters" functionality to improve the UPS performance. Factorio ate all my RAM and crashed, together with my browser. (Tested again later, same effect) Think it went close to 7GB. Hard to check when the computer stops responding to input.
Tried the "remove all biters" functionality instead and it worked. Twice. Factorio stayed at about 3GB RAM.

I guessed it would be better removing them since that should be slightly simpler than killing them. But the difference is more dramatic than expected. Killing them no longer generates any loot even though it leaves corpses. I expected CPU time improvements and not that great RAM usage improvements. Maybe it's not really fixable. Probably not a high priority issue to performance optimize mass killing of all biters on the map. But it crashes the game, other programs and almost windows too...

Also, the biters are on peaceful and my pollution/natives still take all my precious UPS away from my factory. I mean my pollution is at 200k/chunk now that the factory has been off for a while. Was at 700k before I think. And I've lost (bad fps efter removing the entire factory) another older factory to the smog death also (normal world with water, grass and trees), before I knew how to remove pollution and biters with commands. Would really like if it was possible to build big without these features being ticking time bomb that ends your ability to continue playing that save.

[Harkonnen]

43 ups is not so few for big bases or big pollution cloud, many are still somewhat playable with 25-30 ups. What's terribly low on this screenshot is fps - but that's due to all that additional information overlayed on map via F4 side-menu.

[Oxyd]

Please post the save and your mod folder.

[Qon]

http://www33.zippyshare.com/v/oMxHxCTW/file.html
http://www33.zippyshare.com/v/C9SHAp9p/file.html

Removing power cheat entity to shut down base gives about 7 UPS, 45 to 52.
Removing (instead of powering it off) base gives me about 30 UPS, 45 to 75.
Removing natives (while keeping the 1RPM base going at 100%) gives 50 UPS, from 45 to 95.
Get about 350-450 UPS if I remove the base and natives.

So the natives are using quite a bit more CPU cycles than my base.

I'm a bit surprised a base without power uses CPU cycles comparable to a base that is 100% active. It seems like something that should be possible to optimize to the point where it almost doesn't really use any cycles at all. That would be really useful when I'm scaling it up if I could build at full UPS without slowdowns by just powering my base off when I scale up it massivly later. Without biters I could scale up my base to any size I can build, at 300+ UPS, if only unpowered (electric grid not connected to any power source) entities didn't use CPU cycles.

[Harkonnen]

Most times entities still get their update cycles when powered down because this situation is considered temporary and unnatural that a lot of entities would be powered down on purpose.

[Qon]

Most times entities still get their update cycles when powered down because this situation is considered temporary and unnatural that a lot of entities would be powered down on purpose.

Yes, I do realise that. But detecting that a power grid isn't connected at all and toggling update cycles off until the network is reconnected seems pretty straightforward (algorithmically). I'm talking about the cases where power sources are non-existant on the network (which is permanent until a power switch flips or a power pole/source is built, and binary if it's on or off), not when power shuts down because you run out of fuel. So when a network is disconnected from a power source you could loop through all electric entities on the network that are not any any other network also and just remove them from the "update queue" completely.

Of course I don't how flexible the engine is with changing entities to permanent sleep based on such conditions. The simple case where entities are only powered by a single power network is also the most common and most important to optimize for I would say. It seems like the simplest optimization that would enable builds orders of magnitude larger than what is currently possible (until you turn them on, but for builds this size it's ok to only run them over night to test capacity anyways). At the moment, once you reach 20 RPM it's difficult to keep going because what you have built in the past uses all your CPU cycles when they could be used for doing new extensions in new power networks.

Yes, I do realise that people who build 1 RPS gigafactories are an even smaller subset of your playerbase than the people who build 1 RPM megafactories, which is already a small group. But it could be useful for people on lower end machines as well to build and plan big factories instead of stopping at the first satellite launched.

[Oxyd]

When you click the “kill all enemies” button in the Creative Mode mod, it loops through all units on the map and calls die() on each of them. die() goes through all the normal motions as if the biter were killed in-game – it plays the dying animation, creates the blood splatters and a corpse.

You have 190 406 units on your map. Each blood splatter particle takes 224 bytes of memory and a dying biter will create 20 to 150 blood particles when it dies (depending on whether it's a small or big biter). So if we just assume 100 particles per biter, that leads us to a grand total of 4 265 094 400 B or 4.26 GB required to do all that. Which is consistent with your observation that the game took about 7 GB of RAM while it normally takes about 3 GB.

The “remove all enemies” button instead calls destroy() which simply pops the biter off the map without any ceremonies, and indeed it works just fine.

Point is, this is not a bug. You simply don't have enough RAM to do all that – and it isn't really expected that anyone would ever want to create this amount of blood splatters. A good question would be why a mod even has a button that will do all that for each biter on the map, but that's a question for the mod author.

The active chunks issue is another story, though. Normally, spawners are active when they're polluted and they also remove pollution from air by absorbing it. The reason they are active is because they give the signal to start an attack when they've gathered enough pollution. In peaceful mode, however, spawners don't absorb any pollution and biters don't attack on their own anyway, so they have no reason to be active at all, but they were still kept activated when polluted.

The spawners are an easy fix, of course – so, fixed in 0.15.20.

[Qon]

Point is, this is not a bug. You simply don't have enough RAM to do all that – and it isn't really expected that anyone would ever want to create this amount of blood splatters.

Ok, I understand. Didn't have the numbers. 4GB of corpses and exploding guts, that's a lot of gore

The active chunks issue is another story, though. Normally, spawners are active when they're polluted and they also remove pollution from air by absorbing it. The reason they are active is because they give the signal to start an attack when they've gathered enough pollution. In peaceful mode, however, spawners don't absorb any pollution and biters don't attack on their own anyway, so they have no reason to be active at all, but they were still kept activated when polluted.

The spawners are an easy fix, of course – so, fixed in 0.15.20.

Nice! Does that mean natives are effectivly "free" in peaceful mode and I don't have to turn them off any more?

But how will it work in non-peaceful? Would performance improve because pollution was absorbed and less area and units activated, instead of 190k active non-absorbing native units that just keeps on growing? Or would it be worse because spawners would spawn more enemies by absorbing pollution; units that are more active since they want to pick a fight?

Most times entities still get their update cycles when powered down because this situation is considered temporary and unnatural that a lot of entities would be powered down on purpose.

Any news on this? Are optimizations for factories are disconnected from power sources something that is being considered? That would really help me in the vanilla game.

[Rseding91]

Most times entities still get their update cycles when powered down because this situation is considered temporary and unnatural that a lot of entities would be powered down on purpose.

Any news on this? Are optimizations for factories are disconnected from power sources something that is being considered? That would really help me in the vanilla game.

No, it's a non-normal state that entities are unpowered and would add a massive drain to has-power entities performance (the 99% use-case).

[Qon]

Most times entities still get their update cycles when powered down because this situation is considered temporary and unnatural that a lot of entities would be powered down on purpose.

Any news on this? Are optimizations for factories are disconnected from power sources something that is being considered? That would really help me in the vanilla game.

No, it's a non-normal state that entities are unpowered and would add a massive drain to has-power entities performance (the 99% use-case).

Ok. thanks for the response.

Honstly though it seems too me to be doable without adding any drain, if you try to optimize it with a bit of creativity. But I don't know how the game engine for factorio works and what can be done in it so I'll trust you on that.

And yes it's only really common with unpowered bases when you go for 1RPM.