glob-table and load/save [Fixed in 0.5.3]

[drs9999]

hello,
I just noticed that my variables that are stored in the glob-table are cleared after quitting a game. So probably I made something wrong.

Here is my code of the onLoad-event:

Code: Select all

game.onload(function()
	if glob.treefarm == nil then
		glob.treefarm = {}
			glob.treefarm.field = {}
			glob.treefarm.fieldcounter = 0
			glob.treefarm.tick = 0

	end
end)

I did the same in the onInit to initialize the variables/table if this matters.
What I found out so far:
If I save a game and resume to it, everything is still fine, but if I close factorio or return to main-menu and load a savegame the tables are empty.

EDIT:
I just noticed that this behaviour just appears in savegames from one specific game, so now I am very confused...

[ficolas]

can you share the rest of the code?

I see no problem there...
And my mod works fine in this way

[drs9999]

IMO the rest of the code is not relevant, because the same happens to my blueprints-data and the only thing they have in common is the code I posted.

And my mod works fine in this way

That is what confused me the most. My mods works fine this way, too, except all savegames from this game.
I just started a new game with approx 3 hours playtime now. I saved, loaded, quit several times and everything is just fine. It just happens in all savegames from one game.

I think I started the "broken" game in 0.4.x maybe that is a reason? But that is just guessing, I have no idea...

[drs9999]

Well, strange things happened again...
I loaded a savegame from the game I started yesterday and again the glob-table is completly messed up.
Just in case here is the code:

Code: Select all

require "defines"

game.oninit(function()


	if glob.treefarm == nil then
		glob.treefarm = {}													-- Namespace
			glob.treefarm.field = {}											-- stores fieldentity position etc
			glob.treefarm.fieldcounter = 0
			glob.treefarm.tick = 0												-- general mod-tickcounter
	end
end)


game.onload(function()
	if glob.treefarm == nil then
		glob.treefarm = {}													-- Namespace
			glob.treefarm.field = {}
			glob.treefarm.fieldcounter = 0
			glob.treefarm.tick = 0

	end
end)


game.onevent(defines.events.onentitydied, function(event)
	for k, field in ipairs(glob.treefarm.field) do
		if not field.isvalid() then
			table.remove(glob.treefarm.field, k)
			glob.treefarm.fieldcounter = glob.treefarm.fieldcounter - 1
		end
	end
end)


game.onevent(defines.events.onplayermineditem, function(event)
	for k, field in ipairs(glob.treefarm.field) do
		if not field.isvalid() then
			table.remove(glob.treefarm.field, k)
			glob.treefarm.fieldcounter = glob.treefarm.fieldcounter - 1
		end
	end
end)


game.onevent(defines.events.onbuiltentity, function(event)

	if event.createdentity.name == "field" then
		glob.treefarm.fieldcounter = glob.treefarm.fieldcounter + 1
		glob.treefarm.field[glob.treefarm.fieldcounter] = event.createdentity
	end
end)


game.onevent(defines.events.ontick, function(event)

	glob.treefarm.tick = glob.treefarm.tick + 1

	if (glob.treefarm.tick % 60) == 0 then	
		for _, field in ipairs(glob.treefarm.field) do
			local growchance = math.ceil(math.random()*100)
			if field.getitemcount("fertilizer") > 0 then growchance = growchance + 25 end			
			local growntrees = game.findentitiesfiltered{area = {field.position, {field.position.x + 7, field.position.y + 7}}, type="tree"}
			if (growchance > 95) and (#growntrees < 40) then
				local treeplaced = false
				while (treeplaced~= true) do
					local growntree = {}					
					local treeposition ={}
					treeposition.x = math.floor(math.random()*7) + field.position.x + 1
					treeposition.y = math.floor(math.random()*7) + field.position.y + 1
					growntree = game.findentitiesfiltered{area = {treeposition, treeposition}, type="tree"}
					if growntree[1] == nil then
						game.createentity{name = "big-tree", position = treeposition}
						treeplaced = true
					end
				end -- (treeplaced~= true)
			end -- (growchance > 99) and (#growntrees < 40)


		end -- _, field in ipairs(glob.treefarm.field)
	end -- (glob.treefarm.tick % 60) == 0

	if ((glob.treefarm.tick + 30) % 60) == 0 then		
		for _, field in ipairs(glob.treefarm.field) do
			local diechance = math.ceil(math.random()*100)			
			local growntrees = game.findentitiesfiltered{area = {field.position, {field.position.x + 7, field.position.y + 7}}, type="tree"}
			if (#growntrees > 10) and (diechance > 98) then
				growntrees[math.ceil(math.random()*#growntrees)].destroy()
			end -- (#growntrees > 10) and (diechance > 80)
		end -- _, field in ipairs(glob.treefarm.field)
	end -- ((glob.treefarm.tick + 30) % 60) == 0

	if (glob.treefarm.tick % 1200) == 0 then		
		for _, field in ipairs(glob.treefarm.field) do
			if field.getitemcount("fertilizer") > 0 then
				field.getinventory(1).remove{name = "fertilizer", count = 1}
			end
		end -- _, field in ipairs(glob.treefarm.field)
	end -- ((glob.treefarm.tick + 30) % 60) == 0

	if (glob.treefarm.tick % (300 + math.ceil(math.random()*300))) == 0 then		
		for _, field in ipairs(glob.treefarm.field) do
			local growntrees = game.findentitiesfiltered{area = {field.position, {field.position.x + 7, field.position.y + 7}}, type="tree"}
			if #growntrees > 20 then
				local rnd_out = math.floor(math.random()*2)
				if rnd_out > 0 then
					if (field.getitemcount("fertilizer") > 0) and (field.getitemcount("raw-wood") <= 116) then
						field.getinventory(1).insert{name = "raw-wood", count = (4 * rnd_out)}
					elseif (field.getitemcount("fertilizer") == 0) and (field.getitemcount("raw-wood") <= 125) then
						field.getinventory(1).insert{name = "raw-wood", count = rnd_out}
					end
				end
			end
		end -- _, field in ipairs(glob.treefarm.field)
	end -- ((glob.treefarm.tick + 30) % 60) == 0
end)

So like you can see I store the pointers to the field-entities in glob.treefarm.field When I loaded the game now, the pointer is not assign to a field-entity anymore instead it points to an inserter. And again the same happens in all savegames that I made in this game.

[rk84]

table.remove() inside for loop might make some odd stuff.

http://stackoverflow.com/questions/1239 ... -iterating

[drs9999]

What do you mean with odd stuff?!

So yes if the remove is executed in the loop the next element is skipped, but it does not matter, because it is in the onEntitydied/mined - event, so there is just one invalid element at time.

Anyway I am pretty sure that I did not mined/destroyed any fields in the last game so there were no elements that had to be removed. And finally it will not explain why it occurs in every save I made in this game.
To make that sure, I loaded the last savegame I made and saw that is was messed up, so I loaded a save that I made 2h before the other one and exactly the same happened there as well.

[rk84]

Yea you are right. I was not sure about how ipairs would work near end of table.
btw why you use fieldcounter with array table?
also are there any changes between this script and the one used with 0.4.x?

[drs9999]

btw why you use fieldcounter with array table?

good question
Actually there is no need for it anymore. I replace it with

Code: Select all

glob.treefarm.field[#glob.treefarm.field + 1] = event.createdentity

.

All in all I rewrote it completly. E.g tree-generation looks much better now.

But the problem I have with the glob-data still exists and again it is not limited to the treefarm mod. The same happens to my blueprints-mod data at the same time. E.g saved blueprints are lost and I have no "delete blueprint"-function in that script, so I exclude that the player/or script removed them accidently.

[slpwnd]

Could you please provide a link to that save you are talking about?

[drs9999]

Of course.

You have approx 10 sec until the game crashes. If you are running north you can see that trees are spawning next to some chests, but the only fields I ever placed are in the east.

[slpwnd]

Allright so I tried to reproduce your error on that map. However without success. No crashes. It all works fine for me. Even saving / loading the game couple of times.

I have used following mods:
Blueprints 0.5.1
Treefarm-Mod 0.6.1
testingmode 0.5.1

The only relevant difference with your setup (what mods were active when the map was last saved) is the version of the Treefarm-Mod - mine (publicly available) is 0.6.1 while yours is 0.7.0. Is that like a development version?

[drs9999]

Dammit... I forgot that I did not upload the actual version ( because of this problem)

Here is the actual mod-version:

Treefarm-Mod.zip

(254.18 KiB) Downloaded 181 times

Overwriting the old mod-version with the new one put the error back into.

EDIT: And keep in mind that this behaviour appears in the bp-mod as well, I had stored several blueprints, but just one is left. But I guess you can not trace that back.

[slpwnd]

Ok so I dived in and found quite a surprising bug regarding the script.dat which holds the serialized state of the script. The problem was that after the first save the script.dat has never been saved again. Just a stupid error in order of operations (first it saved the new script state and then it copied over the current directory - which overwrote the script.dat).

I think this was the source of the problems. The "lua pointers" to fields would get completely messed up. In the game I tried the fields were growing around the inserters above the furnace area. Later on the game crashed because the script wanted to insert something into the inserter (the field was supposed to be harvested). So in this case some of the pointers for the fields were pointing to those inserters.

So the good news is that this bug is fixed now and the bugfix will be included in the 0.5.3. Hopefully last bugfix release for 0.5.x. This should be released today. The bad news is that existing script.dat files are probably of no use since their state is invalid. You can keep the save and delete the script.dat manually however then the fields that you built already will be "abandoned" by the script.

Thanks again for the report. This was a serious issue (not only for your mod) and would go unnoticed otherwise.

[drs9999]

Ah finally great news!

I just have 2 questions.
Just to clarify removing the script.dat means a complete gamestate-reset? So actually it behaves like a fresh started game from the script-point of view?

And second more important, you said that the script.dat is just saved once and that is/was the reason, but I played this game you tried for 6+ hours, had autosaves every 2 mins, made several saves with different names and quit factorio several times. So why does it happen at exact this random point. And why are different savegames that were made earlier also "infected"? Sadly I overwrite or delete the other savegames I made in this game, but I had a save that was made approx 30 mins earlier and it showed exactly the same behaviour ( spawning trees at the storage area + the gamecrash after few seconds) after loading.
So if this weird behaviour started when the earlier save was made I were not able to continue for more than 30 mins, right?
I do not understand it because every save has its own scipt.dat

[slpwnd]

Yes removing the script.dat "resets" the state of the script.

Can't quite follow your flow, however here is an attempt for the longer explanation:
1) When there is the first save of the game (autosave or manual) the script.dat is written correctly.
2) If you don't load the game "in-between" then the further saves also work properly. That is because how the internals of game spawning work. There is a temp directory which contains the data from when the game was loaded. I think it is in [write-data]/temp/currently-playing or so. You can check that after you load a game. So as long you are only saving the script.dat is correct. This is because there is no script.dat in that temp directory and therefore nothing is overwritten.
3) Once you load the game, the following saves should have the same script.dat as that one you loaded. Maybe I am missing something subtle here though, because your script.dat actually had multiple states from the Treemod in it. That was weird.
4) Even if you are using the old script.dat it might still work very well. The crashes would start to appear when you are serializing entities (they are stored as "pointers" internally). In your case the fields. The crash appeared when the tree was harvested and the script wanted to put it into the inserter.
5) Actually even like this it can somehow work if there were not "substantial changes" in the game state. This is because how the internals of serializing "pointers" work. If the ordering of entities that are pointed to up to the last entity your script is pointig to stays the same, then it will work (because the serialized pointers - numbers, stay the same).

Hope this explains it better.

[drs9999]

Thanks for the detailed explanation, some points are much clearer now.

BTW

Maybe I am missing something subtle here though, because your script.dat actually had multiple states from the Treemod in it. That was weird.

I guess that is because you installed mod-version 0.6.1 first. I decided to change variable/table names between 0.6.1 and 0.7

[kovarex]

So these problems should be fixed in 0.5.3 (as in the edited caption).
For games that are already broken, ... these are just broken, nothing to do about it.
If you think it is not fixed for any reason, let us know here.

[drs9999]

So I let run the game in background for a few hours and everything looks fine.
Thx for this and the other bugfixes. Railroadbridges are awesome

For games that are already broken, ... these are just broken, nothing to do about it.

So that is not 100% true. At least you can "rescue" it if you delete the script.dat

[slpwnd]

So I let run the game in background for a few hours and everything looks fine.
Thx for this and the other bugfixes. Railroadbridges are awesome

Good to hear that:)

So that is not 100% true. At least you can "rescue" it if you delete the script.dat

True.