Do an operation each n tick not submitted with on_tick event

[Linver]

Hi guy!
I'm trying to write a code in control for do a think like this:

1. Game call an event different from on_tick, let's call this event A
2. A call is associated to a function that put some values in a global table, like global.B
3. Let's call on_tick each 60 tick and take one item from global.B do something and wait.

I have some problem with this, in global.B I put some value from an entity (taken from differents call of A), what I expect are there is inside is some like

Code: Select all

{
    ...
    {entity.surface, entity.position}
    ...
}

but when I call on_tick I'm not sure about that if take one element and after I remove it, the table will be iterated one item a time. For example:

Code: Select all

function doThings(event)
	game.print(#global.B)
	if event.tick % 60 == 0 and #global.B> 0 then
		local i, entry = next(global.B)	
		if entry then				
			doSomething(entry)
			table.remove(global.B, i)
		end
	end
end
...
script.event(...on_tick, doThings)

In this case game.print(#global.B) will print very huge number (if I expect 10 entities can be 100) and not all will be called.
When the game call 2 event in control of one mod, is a sequential execution? Can be some call ignored if too much? There's a limit on the space taken in global by one mod?
Any suggestion?

[DaveMcW]

Lua has two kinds of tables.

Array
Must be indexed 1, 2, 3, 4...
Uses # for table size
Must use table.insert()
Must use table.remove()

Associative
Index can be any value
Uses table_size() for table size
Must insert using brackets, data[key] = value
Must delete using brackets, data[key] = nil

If you do something wrong to an array-style table, it will become associative, and array functions will not work correctly.

[PyroFire]

If you do something wrong to an array-style table, it will become associative, and array functions will not work correctly.

This has nothing to do with the issue.

it's because you're calling it on tick, and the doThings() only iterates over one item
so doThings() once in on_tick does one entity
if you wanted 10 then its for i=1,10,1 do doThings() end for example
note when using next() for delayed iteration (1 at a time), you need to store the key last used, in this case its i
in the local i,val = next(table)
store i then do local next_i,next_val=next(table,previous_i) for e.g.
make sure the key exists etc
basically you're doing next() wrong. you need to store previous_i in global.

Currently each step is only acting on the first element in your tables because of this not being setup correctly.

[Linver]

After some trying I understand that the problem is different, this call:

Code: Select all

script.on_event(defines.events.on_chunk_generated, functionA)

In function A i do something like:

Code: Select all

function functionA(event)
	local entities = event.surface.find_entities_filtered{...}
	...
	for to iter on entities
	 put some position in the famous global.B (in the first message of the post)
	...
	end
end

The functionA will be called more than one time for the same chuck, how is this possible?

[Linver]

Solved, thanks again to PyroFire

[Honktown]

Don't iterate over a table you're editing. It's undefined normally, don't know if Factorio's version of Lua changes it.

You should make a temp table of keys, where each key in pairs(first_table) is stored, using table.insert(temp_table, key). Then do for i in ipairs(temp_table) do... myfunction( first_table[temp_table[ i ]] )... first_table[temp_table[ i ]] = nil (spaces around i to avoid the italics tag)

[quyxkh]

If you're not going to use `script.on_nth_tick`, the easiest way to scan an entire table while potentially adding and removing elements is to not insist on any particular ordering:

Code: Select all

-- this is how you efficiently remove items from a numbered but not particularly ordered table named `zlist`:
n=1 while n <= #zlist
    if shouldremove(zlist[n])
        then zlist[n]=zlist[#zlist] zlist[#zlist]=nil 
        else n=n+1
        end
    end
-- adding to unordered tables like that is just table.insert however you want to spell it:
zlist[#zlist+1]=newvalue

[Linver]

Honktown,
in my implementation I don't iterate on a table that I modify constantly, I say to the on nth tick callback to take the first sure element and remove it if no one have compute it before. I test it with a constant input and unstable one. No errors.
My initial misunderstand was in the filter declaration on "on chuck generated" event (noticed by Pyrofire).
Thank u anyway for the suggestion.

quyxkh,
I'm using on_nth_tick, and that function take the first sure item, the other callbacks will do only some insert operation, and isn't important with what index they insert the item, so there's no reason to order the table, like is now already is processed like expected.
My initial misunderstand was in the filter declaration on "on chuck generated" event (noticed by Pyrofire).
Thank u too for the suggestion.

[Honktown]

Honktown,
in my implementation I don't iterate on a table that I modify constantly, I say to the on nth tick callback to take the first sure element and remove it if no one have compute it before. I test it with a constant input and unstable one. No errors.
My initial misunderstand was in the filter declaration on "on chuck generated" event (noticed by Pyrofire).
Thank u anyway for the suggestion.

A random performance optimization to consider after the mod is working: depending on how big the table is, removing from the end is more efficient than removing from the front. If you remove from the front, it has to re-index the table from 2-n and it can do not-great things with memory (fragmentation). If you remove from the end, there's no re-indexing, and all memory is available from the end. Would matter if you're following tons of entities and expect to do the removal operation every time. It changes the behavior, because you're operating on the last thing inserted, instead of the first thing.

Edit: I had to check, although Lua is dynamic with keys and things, it does index table objects. A for loop is faster than pairs/ipairs, obviously only works for indexed, hole-less tables

[Optera]

My preferred way to spread entity iteration over multiple ticks would be to use a global associative array and use next.

Code: Select all

function OnTick(event)
  if global.Entity_Index and not global.MyEntities[global.Entity_Index] then
    -- the index is no longer valid, probably because the associated entity was removed
    global.Entity_Index = nil
    return
  end
  
  for i=1, i<updates_per_tick, 1 do
    local unit_number, entity= next(global.MyEntities, global.Entity_Index)
    if entity then
      global.Entity_Index = unit_number
      -- do stuff with your entity
    else
      -- we reached the end of global.MyEntities
      global.Entity_Index = nil
      return
  end  
end

Advantages:
+ the same code will work for 1 entity every n ticks, m entities every tick or m entities every n ticks
+ performance is much better than using modulo or some other form of calculation on the entire array to find the current pointer
+ adding and removing entities can be done at any time and is (somewhat) gracefully handled
+ adding, removing entities doesn't require loops

Disadvantages:
- If done wrong it will produce weird errors

[Honktown]

My preferred way to spread entity iteration over multiple ticks would be to use a global associative array and use next.

Code: Select all

function OnTick(event)
  if global.Entity_Index and not global.MyEntities[global.Entity_Index] then
    -- the index is no longer valid, probably because the associated entity was removed
    global.Entity_Index = nil
    return
  end
  
  for i=1, i<updates_per_tick, 1 do
    local unit_number, entity= next(global.MyEntities, global.Entity_Index)
    if entity then
      global.Entity_Index = unit_number
      -- do stuff with your entity
    else
      -- we reached the end of global.MyEntities
      global.Entity_Index = nil
      return
  end  
end

Advantages:
+ the same code will work for 1 entity every n ticks, m entities every tick or m entities every n ticks
+ performance is much better than using modulo or some other form of calculation on the entire array to find the current pointer
+ adding and removing entities can be done at any time and is (somewhat) gracefully handled
+ adding, removing entities doesn't require loops

Disadvantages:
- If done wrong it will produce weird errors

Using normal key'ing gives a weird advantage: you can set the key to nil, even if it already didn't exist. A convenient shortcut I found useful (I can remove something I don't want to track, even if I wasn't already tracking it).

[Klonan]

+ performance is much better than using modulo or some other form of calculation on the entire array to find the current pointer

With modulo, you can modulo by the update rate to find the 'bucket' that you want to update, instead of checking the modulo for each key value:

Code: Select all

local on_tick = function(event)

  local bucket = script_data.depots[event.tick % depot_update_rate]
  if bucket then
    for unit_number, depot in pairs (bucket) do
      if not (depot.entity.valid) then
        bucket[unit_number] = nil
      else
        depot:update()
      end
    end
  end

end

[Honktown]

+ performance is much better than using modulo or some other form of calculation on the entire array to find the current pointer

With modulo, you can modulo by the update rate to find the 'bucket' that you want to update, instead of checking the modulo for each key value:

Code: Select all

local on_tick = function(event)

  local bucket = script_data.depots[event.tick % depot_update_rate]
  if bucket then
    for unit_number, depot in pairs (bucket) do
      if not (depot.entity.valid) then
        bucket[unit_number] = nil
      else
        depot:update()
      end
    end
  end

end

Just because that's literally how you do it

A side note, if someone could update the API to be a little clearer on on_nth_tick: it can have multiple functions registered for different nth ticks, even though it's in bootstrap, like on_init and on_load which can only be used once. It makes sense different nth ticks can have different functions, but it's not like they're a different bootstrappy command, you know?

Page in question~~ https://lua-api.factorio.com/latest/LuaBootstrap.html

[Optera]

+ performance is much better than using modulo or some other form of calculation on the entire array to find the current pointer

With modulo, you can modulo by the update rate to find the 'bucket' that you want to update, instead of checking the modulo for each key value:

Code: Select all

local on_tick = function(event)

  local bucket = script_data.depots[event.tick % depot_update_rate]
  if bucket then
    for unit_number, depot in pairs (bucket) do
      if not (depot.entity.valid) then
        bucket[unit_number] = nil
      else
        depot:update()
      end
    end
  end

end

Not bad, but still more expensive than index, data = next(dataset, index)
Beside binding the modulo to tick means updates of a given entity are directly joined to a tick making it unreliable for updating m entities every n ticks where both m and n can fluctuate heavily.