Optimization - tile/surface management

[Nebelwolfi]

Hey everyone!

Since code says more than 100 words - I'm trying to achieve something like this:

Code: Select all

local pos = player.position
local startingSurface = player.surface
local tiles = {}
for i = -1000, 1000 do
	for j = -1000, 1000 do
		tiles[#tiles + 1] = {
			name = ((i < -10 or i > 10) or (j < -10 or j > 10)) and "out-of-map" or "grass",
			position = {pos.x + i, pos.y + j}
		}
	end
end
startingSurface.set_tiles(tiles, false)

For big i/j ranges (2000+) this uses up a massive amount of RAM (2000 * 2000 * allocation size...) and CPU cycles.
Now the real question: Is there a better API or approach for this?

Greetings

[Yoyobuae]

Maybe something like:

Code: Select all

local pos = player.position
local startingSurface = player.surface
local tiles = {}
for i = -1000, 1000 do
	for j = -1000, 1000 do
		tiles[#tiles + 1] = {
			name = ((i < -10 or i > 10) or (j < -10 or j > 10)) and "out-of-map" or "grass",
			position = {pos.x + i, pos.y + j}
		}
		if #tiles > 1999 then
			startingSurface.set_tiles(tiles, false)
			tiles = {}
		end
	end
end
if #tiles > 0 then
	startingSurface.set_tiles(tiles, false)
end

Does only 2000 tiles at a time, so hopefully has lower memory requirements. CPU use would still be bad, though.

[cpeosphoros]

Hey everyone!

Since code says more than 100 words - I'm trying to achieve something like this:

Code: Select all

local pos = player.position
local startingSurface = player.surface
local tiles = {}
for i = -1000, 1000 do
	for j = -1000, 1000 do
		tiles[#tiles + 1] = {
			name = ((i < -10 or i > 10) or (j < -10 or j > 10)) and "out-of-map" or "grass",
			position = {pos.x + i, pos.y + j}
		}
	end
end
startingSurface.set_tiles(tiles, false)

For big i/j ranges (2000+) this uses up a massive amount of RAM (2000 * 2000 * allocation size...) and CPU cycles.
Now the real question: Is there a better API or approach for this?

Greetings

Code: Select all

local limit = 100 --put something sensible here, possibly delegate it to a config.lua variable
local count = 0
local stopped = false
global.posX = global.posX or player.position.x
global.posY = global.posY or player.position.y
local posX = global.posX
local posY = global.posY 
global.startingSurface = global.startingSurface  or player.surface
local startingSurface = global.startingSurface
local tiles = {}
global.lasti = global.lasti or -1000
global.lastj = global.lastj or -1000
local lasti
local lastj 

for i = global.lasti, 1000 do
	lasti = i
	for j = lastj, 1000 do
		lastj = j
		tiles[#tiles + 1] = {
			name = ((i < -10 or i > 10) or (j < -10 or j > 10)) and "out-of-map" or "grass",
			position = {posX + i, posY + j}
		}
		count = count + 1
		if count >= limit then
			startingSurface.set_tiles(tiles, false)
			tiles = {}
			stopped = true
			break
		end
	end
	if stopped then break end
	lastj = -1000
end
if stopped then
	global.lasti = lasti 
	global.lastj = lastj
else
	global.posX = nil
	global.posY = nil
	global.startingSurface = nil
	global.lasti = nil
	global.lastj = nil
end

Not tested, some of the loop controlling variables (lasti, lastj, both local and global) settings may be out of place, but you get the general gist. This all assumes, of course, you can share the load between ticks. If you can't, this approach will not work. Since the player will not stand put between all those ticks, you also have to save posX and posY in global until the work is finished.

I have some huge loops on my Concreteer mod (nothing on the 2k*2k range, but with a range about 200*200, multiplied by dozens of roboports, performance is already a issue). So I've found that locally caching, outside the loop, even simple things like "position.x" shaves off really large gains in performance.

As the customized Lua interpreter stands, any table indexing costs performance. So, avoid anything like t.y, or t[y], or even t["y"] inside a loop, whatever possible.

Your "tiles[#tiles + 1] = {...}" approach to feeding the temp table is the fastest I've found by time profiling. You can read the entire analysis here: viewtopic.php?f=25&t=39025#p233173

Also, as a side note, it doesn't show in the sample code you posted, but beware of calling subroutines inside large loops. Callings themselves are not a problem, function calling is very cheap, performance-wise. The problem is that it can mask caching needs. So it's also a good practice having no table indexing inside functions called from loops - just pass the cached versions by parameters.