[0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Anything that prevents you from playing the game properly. Do you have issues playing for the game, downloading it or successfully running it on your computer? Let us know here.
Post Reply
Eximius
Burner Inserter
Burner Inserter
Posts: 10
Joined: Tue Jun 11, 2019 8:47 am
Contact:

[0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Eximius »

The Minor update of 0.17.53 -> 0.17.54 seems to have caused chunk-scanning to slow down 100 times.

This is seen in a save with Alien Loot Economy ("alien-module" mod). Before the update, it was only taking < 0.25 ms per tick. Now it is taking 25ms.

I am attaching two different saves. An older (8) that had a smaller active map, and now has 4ms / tick of time spent in alien-module mod, and the latest game (11), that now has 25ms / tick in alien-module mod.

The reason I am attributing this to filtered chunk-scanning, is this part of the "alien-module" mod:

Code: Select all

	for index, surface_iterator in pairs(global.surface_iterators) do
		for i=1, batch_size do
			--***Want to get all the chunk logic in here so we can scan a surface more quickly per cycle.

			local chunk = surface_iterator()
			if chunk == nil then
				-- Disable the printing if not in debug mode
				if debug_mode then
					game.print('Rescanning chunks on surface # ' .. tostring(index))
				end
				global.surface_iterators[index] = game.surfaces[index].get_chunks()
			else
				--game.print("x: " .. tostring(chunk.x).. "y: " .. tostring(chunk.y))
				--include logic here to scan each surface @ chunk.
				local surface = game.surfaces[index]
				local chunk_position = { x = chunk.x * 32, y = chunk.y*32 }
				update_modules_on_surface(surface,chunk_position)
				--game.print(serpent.block(chunk_position))

				local assemblers = surface.find_entities_filtered { type = "assembling-machine", position = chunk_position, radius = 16}
				local miners = surface.find_entities_filtered { type = "mining-drill" , position = chunk_position, radius = 16}
				local labs = surface.find_entities_filtered { type = "lab" , position = chunk_position, radius = 16}
				local furnaces = surface.find_entities_filtered { type = "furnace" , position = chunk_position, radius = 16}
				local rocketSilos = surface.find_entities_filtered { name = "rocket-silo" , position = chunk_position, radius = 16}
				local chests = surface.find_entities_filtered { type = "container" , position = chunk_position, radius = 16}
				local logisticChests = surface.find_entities_filtered { type = "logistic-container" , position = chunk_position, radius = 16}
				local beacons = surface.find_entities_filtered { type = "beacon" , position = chunk_position, radius = 16}

				if debug_mode then
					rendering.draw_circle{color = {r = 1, g = 0, b = 0, a = 0.5}, radius = 16, target = chunk_position, filled = true, surface = game.surfaces[index], time_to_live = 30}
				end

				update_modules(assemblers, "machine")
				update_modules(miners, "machine")
				update_modules(labs, "machine")
				update_modules(furnaces, "machine")
				update_modules(rocketSilos, "machine")
				update_modules(chests, "chest")
				update_modules(logisticChests, "chest")
				update_modules(beacons, "machine")

				update_recipes(assemblers)

			end
		end
Which is run every 2 secs.

I am not the author of the mod, just peered into it.
Attachments
Rekrastificate 11 - Rocket.zip
(33.33 MiB) Downloaded 93 times
Rekrastificate 8.zip
(18.46 MiB) Downloaded 62 times

Rseding91
Factorio Staff
Factorio Staff
Posts: 13172
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Rseding91 »

Thanks for the report however nothing about that logic changed in 0.17.54.

Based off the code you've provided that mod is just doing an incredibly slow operation: scanning entire chunks using entity search multiple times is an incredibly slow operation.

Unless you can give me a save file which takes 0.25 MS/tick in 0.17.53 and 25 MS/tick in 0.17.54 I'm just going to say: don't use that mod because it's written terribly.
If you want to get ahold of me I'm almost always on Discord.

User avatar
darkfrei
Smart Inserter
Smart Inserter
Posts: 2903
Joined: Thu Nov 20, 2014 11:11 pm
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by darkfrei »

Is the using of chnk middle position better than the corner one?

Code: Select all

local chunk_top_left_position = { x = chunk.x*32, y = chunk.y*32 }
local entities = surface.find_entities_filtered { type = types, 
  position = chunk_top_left_position, 
  radius = 16}
vs

Code: Select all

local chunk_middle_position = { x = chunk.x*32+16, y = chunk.y*32+16}
local entities = surface.find_entities_filtered { type = types, 
  position = chunk_middle_position, 
  radius = 16}
A lot of lua rendering needs too much UPS time too.

Eximius
Burner Inserter
Burner Inserter
Posts: 10
Joined: Tue Jun 11, 2019 8:47 am
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Eximius »

I agree that the mod is written terribly, however, please find that both of the save files are 0.17.53.

For further proof, please find the attached screenshot of savefile (8) on my newly downloaded version of 0.17.53:
Screenshot_20190705_194544.png
Screenshot_20190705_194544.png (1.01 MiB) Viewed 2083 times
With alien-modules taking <0.01 ms

Rseding91
Factorio Staff
Factorio Staff
Posts: 13172
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Rseding91 »

I loaded your (11) save file and it's not taking any measurable amount of time.

You must have changed something on your end when you updated the game - added/removed mods, changed mod settings, or been running the computer in low power mode?
show
If you want to get ahold of me I'm almost always on Discord.

Eximius
Burner Inserter
Burner Inserter
Posts: 10
Joined: Tue Jun 11, 2019 8:47 am
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Eximius »

Loading up (11) with synchronization still leaves my 0.17.54 game having alien-modules take 29ms / tick. Also, seems the game is unable to differentiate mod versions under sync, because now I see that the game actually loaded a debug version I added (which I had set a different mod version number for) :| . I removed this folder from mods.

As a note, low-power laptop mode would not disproportionately increase 100-fold the time usage of a single mod, I do not think :)

I tried turning off all mods and then synchronizing, loading (11) :
Screenshot_20190705_205811.png
Screenshot_20190705_205811.png (267.13 KiB) Viewed 2062 times
I definitely updated a few mods that had new versions, none of them were alien-module. Also, none of them had any major changes (I actually check the changelogs out of morbid curiosity).

My go to guess would be paging, considering how big the world (with space exploration), but I am running steadily on 50% usage of 16GB of ram, with no swap enabled.

Rseding91
Factorio Staff
Factorio Staff
Posts: 13172
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Rseding91 »

Since I can't reproduce any slowdown and I know for a fact we haven't changed anything that would cause the mod to take more time than before this version i'm going to move this to technical help.

If you do manage to track down what mod is causing the issue and it ends up being a base game thing let me know.
If you want to get ahold of me I'm almost always on Discord.

Eximius
Burner Inserter
Burner Inserter
Posts: 10
Joined: Tue Jun 11, 2019 8:47 am
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Eximius »

Seems the only mods that have multiple mod zips under mods/ are:
laser_fix_0.17.9
laser_fix_0.17.10

space-exploration_0.1.96.zip
space-exploration_0.1.98.zip
space-exploration_0.1.103.zip
space-exploration_0.1.106.zip
space-exploration_0.1.111.zip
space-exploration_0.1.112.zip
space-exploration_0.1.113.zip
space-exploration_0.1.114.zip

stdlib_1.1.0.zip
stdlib_1.1.1.zip
stdlib_1.2.0.zip

Does mod sinchronization failure count as base-game? :) Otherwise, I understand.

Bilka
Factorio Staff
Factorio Staff
Posts: 3123
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Bilka »

My guess would be that the "reset technology effects" migration that we shipped with 0.17.54 triggered the chunk scan code of the mod which upgrades the modules. That might not be intended to happen from the mods side, so it would be best if you could report that as a bug to the mod author. However, I also cannot reproduce the slowdown.
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.

Eximius
Burner Inserter
Burner Inserter
Posts: 10
Joined: Tue Jun 11, 2019 8:47 am
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Eximius »

I tried synchronizing from empty mods to the load (11), but still no dice. 30ms / tick.

Reset technology effects? I did not see that in 0.17.54 changelog.

The mod does not really have a trigger, it is coded that it just runs a batch of chunks to scan for modules... Which is the so-called poorman's code part :P

Eximius
Burner Inserter
Burner Inserter
Posts: 10
Joined: Tue Jun 11, 2019 8:47 am
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Eximius »

Um... Having _only_ the alien-module mod enabled, I still get 20ms / tick:
Attachments
Screenshot_20190705_220121.png
Screenshot_20190705_220121.png (1.06 MiB) Viewed 2039 times

Eximius
Burner Inserter
Burner Inserter
Posts: 10
Joined: Tue Jun 11, 2019 8:47 am
Contact:

Re: [0.17.54] Filtered Chunk Scanning slowed down by a factor of 100

Post by Eximius »

Yep, this was a false alarm. Sorry. Apparently, I was using version 0.6.5 of the mod, but that got updated to 0.6.6 which was the bad one, and now the mod is updated to 0.6.6, 0.6.7, 0.6.8 literally today.

Post Reply

Return to “Technical Help”