Debugging poor performance on gigantic map

[husnikadam]

Hi, I am exploring the limits of Factorio engine and I am doing an experiment of generating milions of chunks via pollution. I know that pollution has a performance impact, especially on this extreme scale, but I am wondering, how do I read that from the debug menu? It says it takes about 75ms on Entity Update each frame, but entity time usage shows basically zero time as only entities on the map are rocks (and removing the rocks doesn't help). Am I reading the stats wrong, or is there a missing debug property, something like a pollution calculation time usage?

factorio_QuefHIfkCW.png (4.44 MiB) Viewed 1515 times

factorio_syNhzu5T7n.png (383.31 KiB) Viewed 1515 times

[disentius]

Assuming biters are off: fish could be responsible for the entity count.
interesting:)

[husnikadam]

Map generation is such that there is no water, no trees, no cliffs, no ores, no biters, basically nothing excepts rocks. The poor performance might also have something to do with requested chunks count, not sure what that means though

[FuryoftheStars]

Using /editor, there is a button under the map manipulation tools that can remove all entities. It can also be used to force generating lab tiles and thus guarantee no more entities will be created.

[Bilka]

If you're looking for more granular performance profiling than the debug options provide then you can use a profiler like Very Sleepy. Some instructions for how to run it can be found here: viewtopic.php?p=501968#p501968

[husnikadam]

I have tried that and it had no effect on the performance. I have used a command to force generate all requested chunks and after the unfroze after 20 minutes, the performance jumped from 16 ups to about 70 ups (requested game speed is 100x). The requested chunks had dramatic impact on performance, just don't know why. After removing the pollustion, the game runs on about 5500 UPS, takes 24GB of RAM and has about 3.7M chunks.

My remaining question is, why was the slowdown reported on entity-update usage even though there were no entities and entity-time-usage info show no use either? (check previous images)

Command used to force generate chunks: /c game.player.surface.force_generate_chunk_requests()

[FuryoftheStars]

force_generate_chunk_requests() does exactly as the name implies... it forces the generation of chunks that have been requested to be generated. It will use all CPU cores to their max to accomplish this. This is the only thing I have ever had peg my CPU at 100% and cause thermal throttling.

[husnikadam]

Well it used only about 1 of 8 of my CPU cores and it didn't actually generate new chunks, didn't help with my confusion, what the requested chunks mean

[FuryoftheStars]

On mine all cores get used. But also, generating a chunk =/= making a chunk visible. You need to use one of the chart commands for that.

https://lua-api.factorio.com/latest/cla ... ate_chunks
https://lua-api.factorio.com/latest/cla ... k_requests
https://lua-api.factorio.com/latest/cla ... html#chart
https://lua-api.factorio.com/latest/cla ... #chart_all

[...]what the requested chunks mean

As some things near the edge of what's already been generated, such as pollution, the game will request more chunks to be generated so there's someplace for it to go rather than a hard stop to it. However, this is queued up in a way to minimize impact on the running game, so in your case where you have a super large cloud, it's possible for it to be requesting chunks faster than it can actually generate them.

[Rseding91]

If you can post the save file I can tell you where the time is spent.

[husnikadam]

Here is the save https://drive.google.com/file/d/1a1Rx1h ... sp=sharing

[Rseding91]

The time is spent trying to spread pollution. Specifically it's requesting more chunks be generated as pollution spreads but due to the speed map generation happens at, the list of chunks to generate is growing faster than they generate. Pollution keeps checking to add chunks to the list and that check. There are 2.5 million chunks requested to be generated so every time pollution spreads to another chunk it has to do Log(2'500'000) checks (roughly 7). There are roughly 40'000 chunks spreading pollution per tick and each one spreads theoretically in 1 direction meaning: 40'000 * 7 checks to add a chunk to the to-generate list each tick.

[husnikadam]

Very interesting insight into the map generation. Thank you very much!

[vjbone]

TLDR pollution bad

[Rseding91]

TLDR pollution bad

No. The levels of pollution here are I.N.S.A.N.E. to the point where they have overwhelmed the map generator. The actual pollution spread logic is measuring almost nothing compared to overwhelming the map generator.