Treefarms and Co-Routines

[Fatmice]

What about storing a growing table to be checked on the tick? When planting the seed, compute the next tick that it will be updated. Say planted on tick A, next update is tick A+B, where B is computed on some algorithm. Add the seed to the growth-table

Code: Select all

table.insert(growth-table, A+B, seed-info)

Then on the tick, check

Code: Select all

if growth.table[tick] then
    actions/function calls
done

So at most 1 seed is updated per tick and not a bunch of seeds causing hick-ups. If you have 12000 seeds, it would take at least 12000 ticks to update all of them, one at a time on each tick. I know there is an effect of the soil and a random number added to each seed's growth time such that it is almost impossible to get two seed to grow at the same time. Am I understanding this problem right?

Edit: alternative to table.insert is simply direct assignment.

Code: Select all

growth-table={}
growth-table[A+B]=seed-info

Either is going to result in a sparse table so do not use #growth-table to count.

[Zeblote]

How is it the game can have thousands of things running on belts all over your base, but a few treefarms absolutely kill the game update stats?

Because we don't have luajit yet. Normal lua is quite slow when compared to c++.

[kiba]

Random growth and variation causes unnecessary complexity that does not increase gameplay value.

All it does is force players to build more trees to handle the inconsistent output.

[Zeblote]

Random growth and variation causes unnecessary complexity that does not increase gameplay value.

All it does is force players to build more trees to handle the inconsistent output.

Wait, the trees don't grow with a constant speed ...?

[kiba]

Random growth and variation causes unnecessary complexity that does not increase gameplay value.

All it does is force players to build more trees to handle the inconsistent output.

Wait, the trees don't grow with a constant speed ...?

From my understanding of what was discussed, yes.

[Fatmice]

Random growth and variation causes unnecessary complexity that does not increase gameplay value.

All it does is force players to build more trees to handle the inconsistent output.

Wait, the trees don't grow with a constant speed ...?

No, it's been like that since forever. This is to give the illusion that the trees are alive.
I'm sad to hear drs9999 has stop working on it.

[Rseding91]

If you're looking for ways to optimize TreeFarm you can look at the re-write I did of it for my own game:

Treefarm-Mod_1.2.0.zip

(2.42 MiB) Downloaded 283 times

Note: this was written when there was a bug with saving sparse tables so I wrote it by converting index values to strings - don't use the code in that attached file directly - just use it as a reference and then write the correct code Also it went through *many* iterations of different attempts so the code in that zip is a giant mess.

It's an older version and I didn't put any effort into making it backwards compatible with old saves but the logic can still be extracted and applied to the current version of TreeFarm.

The primary reasons TreeFarm (now discontinued) is slow:

• The trees list to grow is acting like a deque: the first item in the trees-to-grow table is the next tree to tick. When it's ticked it is removed and inserted into the trees-to-grow at the next tick it should grow
• The farms scan the entire area for the "next spot to plant" each time they tick

Lua tables do not work well when you table.insert into the middle of them or table.remove from not-the-end of them. I can't stress this enough: every time you table.insert() into the middle of an indexed lua table it has to push all elements after the insertion point out by 1. Every time you table.remove() from not-the-end all elements after the one you remove are shifted up by one. This happens every time a tree is grown and so it eats a *ton* of CPU time.

To fix this: you can use (like I did) a sparse table of trees to grow. The "trees to grow" contains a table of tables. You index the table with the current tick to see if that index contains a table of trees to grow and if it does you grow all of those trees and then re-insert them at the next tick they should grow.

It looks like this:

Code: Select all

script.on_event(defines.events.on_tick, function(event)
  if global.treesToGrow[event.tick] ~= nil then
    growTrees(global.treesToGrow[event.tick])
    global.treesToGrow[event.tick] = nil
  end
end)

When inserting seeds into the table to grow:

Code: Select all

function insertSeed(seedTable, nextGrowthTick)
  if global.treesToGrow[nextGrowthTick] == nil then
    global.treesToGrow[nextGrowthTick] = {}
  end
  
  table.insert(globa.treesToGrow[nextGrowthTick], seedTable)
end

That way you only ever tick the trees you want to grow. Checking if you have trees to grow is a single index operation (fast), inserting is at max 2 index operations (fast) and moving seeds around between growth stages is also fast.


As for the planting of trees on farms: store the x/y of the last plant attempt and increment y each attempt. If y > a column reset it to 0 and increment x. If x > then a row reset both to 0 and restart. That way it scans the entire area 1 tile per growth attempt (or how ever many you decide per attempt) and it doesn't need to re-scan the entire area each time.


The above changes when I tested TreeFarm showed a constant 1/10th tick time as before the changes. That means that before it would take 2 MS/tick to run X tree farms and after it would take 0.2 MS/tick to run the same tree farms.

[Fatmice]

Yes, sparse table is the key to not compute needlessly.

[Buggi]

This has turned into a very helpful thread. Thanks everyone!

[roy7]

Speaking of which, is there an updated TreeFarm anywhere with Rseding's performance improvements applied?

[roy7]

Blu3wolf has started a project to fold in Rseding91's suggestions, info is here:

viewtopic.php?f=44&t=19124&p=133153#p133153

However it isn't working at the moment, there's a bug he doesn't have time to try and locate. It is on github if anyone would like to contribute and help try to bring about a high performance TreeFarm. I don't know LUA so I'm kinda useless in that regard.

[Blu3wolf]

As for the planting of trees on farms: store the x/y of the last plant attempt and increment y each attempt. If y > a column reset it to 0 and increment x. If x > then a row reset both to 0 and restart. That way it scans the entire area 1 tile per growth attempt (or how ever many you decide per attempt) and it doesn't need to re-scan the entire area each time.


The above changes when I tested TreeFarm showed a constant 1/10th tick time as before the changes. That means that before it would take 2 MS/tick to run X tree farms and after it would take 0.2 MS/tick to run the same tree farms.

For the second suggestion you made, about how to selectively scan the treefarm for the next location to plant - Does this really improve performance all that much? Compared to scanning the whole treefarm on each attempt - which only happens once every 60 ticks anyway?

Thanks for the suggestions!

Blu3wolf has started a project to fold in Rseding91's suggestions, info is here:

viewtopic.php?f=44&t=19124&p=133153#p133153

However it isn't working at the moment, there's a bug he doesn't have time to try and locate. It is on github if anyone would like to contribute and help try to bring about a high performance TreeFarm. I don't know LUA so I'm kinda useless in that regard.

The bug in question is fixed, but thanks anyway. With that said I welcome PRs, I am very new to Lua and scripting in general. Cheers!

[orzelek]

As for the planting of trees on farms: store the x/y of the last plant attempt and increment y each attempt. If y > a column reset it to 0 and increment x. If x > then a row reset both to 0 and restart. That way it scans the entire area 1 tile per growth attempt (or how ever many you decide per attempt) and it doesn't need to re-scan the entire area each time.


The above changes when I tested TreeFarm showed a constant 1/10th tick time as before the changes. That means that before it would take 2 MS/tick to run X tree farms and after it would take 0.2 MS/tick to run the same tree farms.

For the second suggestion you made, about how to selectively scan the treefarm for the next location to plant - Does this really improve performance all that much? Compared to scanning the whole treefarm on each attempt - which only happens once every 60 ticks anyway?

Main reason behind this idea is to not cause game to stutter. You need to spread out the work among game ticks to keep it happening with lower speed but constantly. Scanning 60 locations at once causes "spike" of CPU time usage on mod that might be visible as stuttering.

[Zeblote]

Those hitches will most likely disappear with luajit in 0.13

[orzelek]

Those hitches will most likely disappear with luajit in 0.13

They would be reduced significantly for sure

[Fatmice]

Who told you that? luajit hasn't been updated to lua 5.3 and the difference between 5.1 and 5.3 is quite a lot. Luajit 2.1-beta1 is compatible with lua 5.1 code only.

[orzelek]

Who told you that? luajit hasn't been updated to 5.3 and the difference between 5.1 and 5.3 is quite a lot. Luajit 2.1-beta1 is compatible with lua 5.1 code only.

It's about the premise of luajit. Not sure what version difference means in practical use here.
With jit lua is no longer interpreted but compiled - should net significant speed increase. That would mean that any spikes are significantly shorter = less stuttering.

[Zeblote]

Who told you that? luajit hasn't been updated to lua 5.3 and the difference between 5.1 and 5.3 is quite a lot. Luajit 2.1-beta1 is compatible with lua 5.1 code only.

Factorio doesn't use 5.3 anyways, and the difference between 5.1 and 5.2 is not that much..

Would prefer performance over new features.

[Fatmice]

That's not what I meant. =) Who told you that luajit will be replacing the lua interpreter? I thought the current lua interpreter is lua 5.3 with os module removed. Code written currently might break with luajit in its place since 2.1-beta1 is compatible with lua 5.1 and not lua 5.3. There are many differences between the subversion of lua.

Regarding spikes, they are only stuttering if the execution takes longer than 1/60th of a second, in which case, optimization on the code can be done right now to remove/reduce that even without luajit.

[Zeblote]

Well, /c game.local_player.print(_VERSIONI) says 5.2 so that's where I got that it doesn't use 5.3.

For luajit, look at the to do list here: https://www.factorio.com/blog/post/fff-111