[semi solved] restore event handler on load

[ZlovreD]

I had searched everywhere but i still don't get how to make it work.

Let's assume i have in control.lua something like this:

Code: Select all

arrayOfTickFunctions = {}

function reactionOnSomeEvent(event)
	local _on_tick = function(event) if game.tick % 10 < 1 then game.print("Hello World!") end end
	global.store_my_function_dump = serpent.dump(_on_tick)
	script.on_event(0, _on_tick)
end

function globalOnTick(event)
	for _,func in pairs(arrayOfTickFunctions) do
		func()
	end
end

function myOnLoad()
	arrayOfTickFunctions[#arrayOfTickFunctions] = loadstring(global.store_my_function_dump)
	script.on_event(0, globalOnTick)
end

script.on_load(myOnLoad)

When i start new game i get "Hello World!" message and all is fine. But when i save and loading back - nothing happens.
What i'm doing wrong?

[eradicator]

Any changes made to global before on_init are ignored. Don't access global outside of event handlers.

I'm not sure if storing serialized functions works at all. (Storing non-serialized functions works though i think.)

But more importantly...why do you even need to store serialized functions? What is wrong with a plain:

Code: Select all

script.on_event(defines.events.on_tick,function(e)
  for _,f in pairs(arrayOfTickFunctions) do
    f(e.tick)
    end
  end) 

The only reason i see to use serialized functions is if you were doing dynamic code generation. And even then you should just store the parameters used to generate the code in global and generate them freshly in on_load to reduce loading times. And to avoid future bugs when you want to change any of the functions.

[ZlovreD]

Any changes made to global before on_init are ignored. Don't access global outside of event handlers.

It's just example of code to understand what happening

I'm not sure if storing serialized functions works at all. (Storing non-serialized functions works though i think.)

I'm transfer serialized functions from data to control and it's work. Non-serialized functions not saving in save file. Also: viewtopic.php?p=232964#p232964

But more importantly...why do you even need to store serialized functions? What is wrong with a plain:

Yep, i need a way to re-enable event handlers after load with initially unknown code which is compiled in data and transfered to control and then may be changed again in control.

If im change func() to func()() inside globalOnTick to lower abstraction - i'm getting error witn game==nil

[eradicator]

Did you actually test if saving serialized functions to global works? I remember remote.call will simply replace any string containing bytecode by nil.

If im change func() to func()() inside globalOnTick tol lower abstraction - i'm getting error witn game==nil

What?

Transfering anything from data to control is not good design. Let alone transferring serialized functions. If you want any futher help you'll have to thoroughly explain wtf you're trying to do (and use a spell checker). I find it hard to believe that your current method is the best way.

[ZlovreD]

Ok, as temporal solution.

preparing some information in data-final-fixes.lua

Code: Select all

local data = { }
data.generateMessage = function()
	return "Hello world!"
end

local _dump = serpent.dump(data)
local chunks = math.floor(#_dump / 199) + 1

-- remember number of chunks
data:extend({
	{
		type = "flying-text",
		name = "DATA_C",
		time_to_live = 0,
		speed = 1,
		order = "".. chunks
	}
})

-- write data chunks
for i=0, chunks do
	local name = "ZADV_DATA_"..i
	data:extend({
	{
		type = "flying-text",
		name = name,
		time_to_live = 0,
		speed = 1,
		order = "".. dump:sub(i*199, (i+1)*199-1)
	}})
end

prepare stored information and use it in control.lua

Code: Select all

local Events = {}

local function AssingOnEventHandler()
	local toSerialize = {}
	toSerialize.data = global.Data.generateMessage()
	toSerialize.func = function(event,game,data)
		if game.tick % 60 == 0 then
			game.print(data)
		end
	end
	global.Data.serdata = serpent.dump(toSerialize)
	ReEvent()
end

function On_Ticker(event)
	
	if #Events then
		for name,data in pairs(Events) do
			local done, err = pcall(data.func, event, game, data.data)
			if not done then game.print("\n[%s].on_tick error:\n%s", name, err) end
		end
	end
	
end

local function ReEvent()
	local _data = global.Data or {}
	if _data.serdata then
		local unser = loadstring(data.serdata)() or {}
		Events['test'] = {data=unser.data, func=unser.func}
	end
end

local function Init()

	--parse raw data
	local dump
	local chunks = game.entity_prototypes["DATA_C"].order

	for i=0, chunks do
		local name = "DATA_"..i
		dump = dump .. game.entity_prototypes[name].order
	end

	-- apply parsed data
	global.Data = loadstring(dump)() or {}
end

script.on_init(Init)
script.on_load(ReEvent)
script.on_event(defines.events.on_tick, On_Ticker)
script.on_event(defines.events.on_player_crafted_item, AssingOnEventHandler) -- as trigger to create on_tick handler

[Bilka]

OP, can you please explain what you are transferring from data to control, and why you are doing it.

[ZlovreD]

OP, can you please explain what you are transferring from data to control, and why you are doing it.

I have core mod and unknown (in perspective) amount of expansion mods.
Each expansion provide their own data and functions (including event handlers).

Core data generated in "data", expansions inject their data in "data-update" and all this finalized (some heavy-load actions) in "data-final-update".
When all data is gathered and finalized - i need to make it accessable in "control". That why i'm serialize it.

Then in "control", if appropriate event is accure i need to take data for specific event/situation, update it for the current state of the game and assign prepared handler with actual data to the event.

And, of course, i need to make this continue to work after loading.

For more details, you can check mod itself at https://github.com/iZlovreD/ZAdventure

[Bilka]

When all data is gathered and finalized - i need to make it accessable in "control". That why i'm serialize it.

Then in "control", if appropriate event is accure i need to take data for specific event/situation, update it for the current state of the game and assign prepared handler with actual data to the event.

Sorry to repeat myself, but what data and why?

You don't seem to be making prototypes, so why do you generate the data in the data stage? You can also generate data in control and share it between mods there. And that way would not be as hacky as what you are doing now.

[ZlovreD]

When all data is gathered and finalized - i need to make it accessable in "control". That why i'm serialize it.

Then in "control", if appropriate event is accure i need to take data for specific event/situation, update it for the current state of the game and assign prepared handler with actual data to the event.

Sorry to repeat myself, but what data and why?

You don't seem to be making prototypes, so why do you generate the data in the data stage? You can also generate data in control and share it between mods there. And that way would not be as hacky as what you are doing now.

I need to get data from loaded expansion mods,manipulate with blueprints (over 250k of blueprint text atm) - parse, operate and prepare data. I can do it in "control" but it may cause some additional delays, lags or desyncs. If been more precisely - all content of areas.lua plus additional informations must be available in control.
I'm prefer to do all data manipulations on "data" stage, pack it and then just unpack ready-to-use data in "control". Because additional time on game launch not so mentionable than on map start.

[eradicator]

Looked at the github because you're descriptions are too vague...

So, from what i can tell you're trying to create some blueprints and each blueprint has a function attached to it to fill chests in that blueprint when it is placed?

Question 1: As per the opening post this transfer of data to control stage is actually working, right?

If that is working then in control stage you should have access to a table of functions.

Something like

Code: Select all

local data = {
  ['blueprint_name'] = {
    bpstring = 'abdc',
    bpfunction = local function() end
    }
  }

Question 2: So why do you need to store those functions in global, and why do you need to load them in on_load?
As far as i can tell they're just static functions (i.e. they never change during a running game) and can be called like i described in my first answer.

[ZlovreD]

Question 1: As per the opening post this transfer of data to control stage is actually working, right?

Yes, working well.

Question 2: So why do you need to store those functions in global, and why do you need to load them in on_load?
As far as i can tell they're just static functions (i.e. they never change during a running game) and can be called like i described in my first answer.

I have a list of area prototypes. Each time when new chunk generated one of prototypes may be placed on it (random probability).
There is two functions in each prototypes: first one works after each entity created; second after all tiles and entities created.
Some prototypes must be served by events. Some not.
I don't know how many prototypes placed and which types, how many event handlers is on. And i need to re-assign them to events every time after load.
Yes. All of functions is static. But they must know at least where is their area, because something may happens with one of them. So i need transfer additional data into the event functions.
And how many copies of them actually working at the same moment? And what if i need to cancel one of them?

And i came to storing serialized functions.

Main goal is lower any operations per second as much as possible and manipulate with already prepared data.
That's why i'm trying to find out the best solution with less operations.

Imagine... You have a blueprint with single wooden chest. You place it several times in random places. Each chest filled with, let's say, coal by default. And you need to refill it when it's empty.
How do you solve this task?

[eradicator]

And i need to re-assign them to events every time after load.

I'm still not sure if you're too unfamilar with the API, or if i simply don't understand why you insist to use on_load.

So let's try some simple examples:

Code: Select all

local funcs= {
     ['handler1-name'] = {
         event = defines.events.on_built_entity,
         handler = function(e) end
    },
  }

for handler_name, handler_data in pairs(funcs) do --example to activate everything, you can customize.
    global.active_handlers[handler_name] = true
    script.on_event(funcs[handler_name].event, funcs[handler_name].handler)
    end

This will make every handler handle whatever events it needs all the time. No need for on_load.

In my own mods if i want handlers that are not always active i simply store a flag in global, but not the handler itself:

Code: Select all

local funcs= {
     ['handler1-name'] = {
         event = defines.events.on_built_entity,
         handler = function(e) end
    },
  }

for handler_name, handler_data in pairs(funcs) do --example to activate everything, you can customize.
    global.active_handlers[handler_name] = true
    activate_handlers()
    end

local function activate_handlers()
   for handler_name, enable in pairs(global.active_handlers) do
    if enable then
       script.on_event(funcs[handler_name].event, funcs[handler_name].handler)
  end
  
script.on_load(activate_handlers)
  

You can now deactivate handlers as you want, and don't need to store serialized functions anywhere. (Storing less data in global -> faster loading.)

And if you don't want to deserialize all functions at once because you think it's too slow you can attach a metatable to funcs to do the construction on demand:

Code: Select all

setmetatable(funcs,{
  __index = function (self,key)
    rawset(self,key,deserialize_handler_function(key))
    return rawget(self,key)
  })

This will deserialize a function when it is first accessed, and store it in the table. The calling function does not see this, i.e.

Code: Select all

local x = funcs['handler1-name']

Will deserialize if it hasn't been done yet, or return the already deserialized function if it's stored.

[ZlovreD]

I'm still not sure if you're too unfamilar with the API, or if i simply don't understand why you insist to use on_load.

Because events not working after load and there is no way to store functions in global (event handlers including). And i don't know how many area prototypes may be on next load and which events they use (3rd side expansions).

So let's try some simple examples:
This will make every handler handle whatever events it needs all the time. No need for on_load.

It's works. But what if you need to pass additional and individual data for different handlers ?

In my own mods if i want handlers that are not always active i simply store a flag in global, but not the handler itself

But you still storing this functions.
In my case - originals is deleted after serialization

Code: Select all

ZADV.adata[newarea.name][center.x][center.y].serdata = serpd(ZADV.adata[newarea.name][center.x][center.y].events)
ZADV.adata[newarea.name][center.x][center.y].events = nil

And when script find any changes in data after load (comparing hash) new handlers added into the table.

You can now deactivate handlers as you want, and don't need to store serialized functions anywhere. (Storing less data in global -> faster loading.)

I'm using similar trigger inside handler itself at the start. And if i'm right, disabling an event - releasing just few bites, except cases if you assign handlers inline (script.on_event(defines, function() end))
And what is difference between storing data in control or in globals?

And if you don't want to deserialize all functions at once because you think it's too slow you can attach a metatable to funcs to do the construction on demand:
This will deserialize a function when it is first accessed, and store it in the table. The calling function does not see this, i.e.

Code: Select all

local x = funcs['handler1-name']

Will deserialize if it hasn't been done yet, or return the already deserialized function if it's stored.

Hm.. Good idea. But need to don't forget to recreate metatables in on_load.

[eradicator]

Q: It's works. But what if you need to pass additional and individual data for different handlers ?
A: Then you store the set or parameters each handler needs in yet another table. So if your problem is that you're prebaking the variables into the functions, you can do:

Code: Select all

global.params = {
  ['handler1-name'] = {
    {center.x1,center.y1,whatever1},
    {center.x2,center.y2,whatever2},
    }
  }

local handler1(x,y,w)
  --dostuff
  end

local function handler1-wrapper()
  for _,param in pairs(params['handler1-name']) do
    funcs['handler1-name].handler(unpack(params))
    end
  end

script.on_event(funcs['handler1-name].event,handler1-wrapper)

This also removes the need to hash functions to tell if they changed, because they're always the latest version.

Q: And what is difference between storing data in control or in globals?
A: All data in globals needs to be serialized by the game engine on saving, and deserialized on loading, as that is a relatively slow process it's usually faster to recreate the things you need in control. Could make saving faster too.

Q: Hm.. Good idea. But need to don't forget to recreate metatables in on_load.
A: If funcs is a local table of functions then the metatable is created in the body of control.lua, not in an event, and thus on_load is not required.