[MOD 1.1] Will-o'-the-Wisps updated

Topics and discussion about specific mods
mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Save and mod list would do the same btw, but it's a bit of work installing all that, and I hope that maybe running such command on your end might be simplier.

Bloodymight
Inserter
Inserter
Posts: 20
Joined: Sun Nov 04, 2018 4:21 pm
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by Bloodymight »

Hi,

I replaced the mod with your "debug" one

game didn't start, got an error at start up: https://imgur.com/a/8FWBsRL
...so I removed "-debug"(essentially same name as the replaced one) from it's name and it worked

but using your "/wisp zone export" command gave me this error:

"Error while running command "wisp": __Will-o-the-Wisps_updated__/libs/zones.lua:349: bad argument #1 to 'format' (not a number in proper range)"

feels like I "installed" it wrong but not sure

kinda off-topic... currently running 88 mods on my biggest map yet, and the game hasn't crashed after 12 hours. but eats like 16GB RAM :shock:

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Forgot that zip files can be used in mod dir as-is, so didn't think that its name would matter, my bad.

You did rename it correctly though, but quick and dirty serialization code for that export command couldn't handle chunk offsets used in your game.
(used %d serialization for json keys, which can't handle 31bit+ integers, but didn't cause a problem in a relatively early save test)

Should be fixed it now, please try same stuff with the attached version (except zip should be named correctly now).
Attachments
Will-o-the-Wisps_updated_0.0.52.zip
(1003.32 KiB) Downloaded 97 times

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Bloodymight wrote:
Thu Nov 08, 2018 11:44 pm
kinda off-topic... currently running 88 mods on my biggest map yet, and the game hasn't crashed after 12 hours. but eats like 16GB RAM :shock:
I think that's likely to be a memory leak in one of these mods, as I doubt anything in lua code can actually use 16G of data and don't lag horribly while doing that :)
("memory leak" is when code stores values somewhere but never cleans them up later, so memory usage grows indefinitely)

One quick test for that might be loading specific save before you'll go off somewhere, leaving it running for a few hours (unpaused) while doing absolutely nothing.
After these few hours, make a save and check memory usage.
If memory usage grows significantly (e.g. 2x, 3x or such) - almost certainly a leak in one of the mods.
Comparing disk space used by pre- and post-idle-time savegames, you can also check if this data ends up stored by some mod there, which might be even more troubling.

There are more sophisticated tools in factorio to debug this issue though, like F-something key to show mod resource usage, which should probably show which mod it is exactly, but don't quite remember how to read or use these off the top of my head, you might want to check more dedicated forum threads or wiki pages on that.

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Also, to be fair, loading an 8-hour game here with a more modest collection of 37 mostly quality-of-life mods, factorio takes 1.7G actual and 3.2G virtual memory (allocated, but not necessarily ever used) here, so guess 4-8G might be normal for more extensive and more content-heavy mod collection.

Bloodymight
Inserter
Inserter
Posts: 20
Joined: Sun Nov 04, 2018 4:21 pm
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by Bloodymight »

game eats 13GB in main menu and 16GB after loading savegame... so yeah

btw I tried it again and got the exact same error 8-)
didn't cause a problem in a relatively early save test
I'm currently at 170 hours on my map

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Bloodymight wrote:
Fri Nov 09, 2018 1:00 am
btw I tried it again and got the exact same error 8-)
Ah, damn, guess these integers might be too big for any printf() format, sorry :(
Guess I'll test it better locally this time and come up with something more likely to work in your game.
Bloodymight wrote:
Fri Nov 09, 2018 1:00 am
game eats 13GB in main menu and 16GB after loading savegame... so yeah
I'm currently at 170 hours on my map
Ah, guess 16G might indeed be normal for that kind of game :)
Mine never got anywhere near that for sure, but I tend to play with reduced research costs and generally less grindy stuff, so probably not a good judge of factorio game length either.

Bloodymight
Inserter
Inserter
Posts: 20
Joined: Sun Nov 04, 2018 4:21 pm
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by Bloodymight »

interesting thing is, I still didnt have any crashes since I reinstalled your mod, so not sure what to make out of that

I packed all mods and 2 saves together(latest main save(test 2) we use and one where it all started (test))
Save file where it all started uses less mods but both crashed before I reinstalled wisps

reduced research costs and generally less grindy stuff
wanted to start with bobs+angels but that was a bit too much

bob is fine atm :D


Edit: apparently I cant upload a 374MB file here :roll:

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Bloodymight wrote:
Fri Nov 09, 2018 1:53 am
interesting thing is, I still didnt have any crashes since I reinstalled your mod, so not sure what to make out of that

I packed all mods and 2 saves together(latest main save(test 2) we use and one where it all started (test))
Save file where it all started uses less mods but both crashed before I reinstalled wisps
With my earlier theory, I just thought you might have some mod that does something with the map.
Can be any map-generation mod, though not entirely sure how they'd unload previously-generated map chunks, but didn't look into it much either.

Did just try to reproduce the issue with serialization, and couldn't do it with any large integer, so guess it might be some other bug after all - maybe some integer keys overflowing into floats randomly.
So currently was fixing serialization to properly represent either in json as strings, to see what ended up happening in your game with them.
Bloodymight wrote:
Fri Nov 09, 2018 1:53 am
Edit: apparently I cant upload a 374MB file here :roll:
You can maybe cheat the system and upload a bunch of smaller files, if limit is anywhere close to that.
Otherwise I'd probably try something like dropbox, onedrive, google drive, etc - "cloud" things, whichever one you already have an account with.

Will probably end up running exactly same routine on that save though, just need to get it working for weirder values which I didn't expect to be there at all (and they probably shouldn't be there).

Bloodymight
Inserter
Inserter
Posts: 20
Joined: Sun Nov 04, 2018 4:21 pm
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by Bloodymight »

I used a mod to remove unused chunks so map adds newly enabled resources maybe that did something?


well nvm you said there were other people with the same error

currently uploading on onedrive, my inet is pretty slow should be done in 15 minutes ._.

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Bloodymight wrote:
Fri Nov 09, 2018 2:11 am
I used a mod to remove unused chunks so map adds newly enabled resources maybe that did something?


well nvm you said there were other people with the same error
No no, that'd probably do it, if it does this by re-generating map chunks, which it probably does.

Other people had error in same function due to unrelated cache invalidation issue, which doesn't seem to be the case here at all, I just thought at first it might be, due to how easy it is to mess up with caches.

There is even a rather common programming proverb/mantra for that :)
There are only two hard things in Computer Science: cache invalidation and naming things.

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

I did fix that "/wisp zone export" command once again, if you want to give it another try, should definitely work now.
Though if you'll upload the save anyway, maybe not much need to bother.
Attachments
Will-o-the-Wisps_updated_0.0.52.zip
(1003.39 KiB) Downloaded 87 times

Bloodymight
Inserter
Inserter
Posts: 20
Joined: Sun Nov 04, 2018 4:21 pm
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by Bloodymight »

https://1drv.ms/u/s!AniVyk3veOIicVZw3pv8nMsRYcM

uploaded save and modlist

looks like it worked, this time I got another error

"Error while running event Will-o-the-Wisps_updated::on_tick (ID 0)
__Will-o-the-Wisps_updated__/libs/zones.lua:75: attempt to call field 'get_pollution' (a nil value)"

I attached the files, hopefully

Edit: I made a .rar instead of .zip because I didnt really pay attention to that, I hope thats not a problem :?
Attachments
wisps.rar
testtest
(77.07 KiB) Downloaded 78 times
Last edited by Bloodymight on Fri Nov 09, 2018 2:49 am, edited 1 time in total.

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Bloodymight wrote:
Fri Nov 09, 2018 2:43 am
uploaded save and modlist

looks like it worked, this time I got another error

"Error while running event Will-o-the-Wisps_updated::on_tick (ID 0)
__Will-o-the-Wisps_updated__/libs/zones.lua:75: attempt to call field 'get_pollution' (a nil value)"
Yeah, this is totally normal after the command, as this quick-n-dirty data-mangling for export is destructive.
Thanks a lot for sticking with the issue, would almost be a shame if ends up being just the thing I suspected in the second reply, which it seem to be so far.
Will check/confirm, report back and upload fixed version soon.

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Bloodymight wrote:
Fri Nov 09, 2018 2:43 am
Edit: I made a .rar instead of .zip because I didnt really pay attention to that, I hope thats not a problem :?
Definitely not, besides who doesn't have 7zip these days which opens any archive format ayway :)

Bloodymight
Inserter
Inserter
Posts: 20
Joined: Sun Nov 04, 2018 4:21 pm
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by Bloodymight »

I did the command on both save games and got another error this time

"Error while running event Will-o-the-Wisps_updated::on_tick (ID 0)
__Will-o-the-Wisps_updated__/libs/zones.lua:150: attempt to index local 'chunk' (a nil value)"

pretty close to the error which started this all, second one is something something pollution for some reason, and this one(the third) is about chunks again with another lua(attached the files again)
Attachments
wisps2.rar
(67.66 KiB) Downloaded 72 times

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Bloodymight wrote:
Fri Nov 09, 2018 2:57 am
I did the command on both save games and got another error this time

"Error while running event Will-o-the-Wisps_updated::on_tick (ID 0)
__Will-o-the-Wisps_updated__/libs/zones.lua:150: attempt to index local 'chunk' (a nil value)"

pretty close to the error which started this all, second one is something something pollution for some reason, and this one(the third) is about chunks again with another lua(attached the files again)
It's also due to export, same as above - mod will inevitably crash after that console command.
(as that export overrides all big-int keys with json-friendly strings, any lookup later will fail, and pollution thing above is due to it replacing chunk.surface userdata reference with "s.1" string representation)

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

Oh dear, I just realized that I don't actually have 16G of RAM on my 6-year old machine while trying to load your set of mods and save... will see how it goes :)

Bloodymight
Inserter
Inserter
Posts: 20
Joined: Sun Nov 04, 2018 4:21 pm
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by Bloodymight »

I think the biggest RAM eater is Alien Biomes

didn't really pay attention to that ._.

you could remove "lightorio", "minime", "Bot-Servicing", "actual craft time" and "Alien Biomes" since they're all later additions

mk-fg
Long Handed Inserter
Long Handed Inserter
Posts: 69
Joined: Wed May 30, 2018 11:01 am
Contact:

Re: [MOD 0.16.x] Will-o'-the-Wisps updated

Post by mk-fg »

It's definitely something with a ton of sprites - when loading game, it passes lua init with something like 3-4G iirc, but when it gets to loading sprites, quickly goes over 11G and segfaults due to lack of memory.
No worries though, wisps.rar is all I need from there, and aside from definitely having that mismatch issue I mentioned earlier, seem to have two other minor bugs (non-critical - some stuff gets duplicated needlessly, probably due to those chunk replacements), which I'll also look into.

Post Reply

Return to “Mods”