[MOD 0.14.x] Water maze

[orzelek]

@aaargha
It's a good hack since it works I might add it to RSO if there are more people interested in out of the box compatibilty.

@swni
Chunk generation gets a bit more complex because it's async.
Steps I planned were:
1. Generate the ore patch and store it's layout (quite a big table sadly but required due to determinism and stuff
2. Check all the chunks under it plus 1 chunk radius (to move the patch if there is to much water)
3. If all of the chunks are up then place the patch, if not then call gen request for them and add patch to queue.
4. In on_tick check chunks periodically and when all are ready then place the patch.

One of the issues with this is that RSO has a bit of spaghetti in it and is not fully prepared for this. I'm planning a cleanup with new rng's in 0.15 (and hoping I can store them in lua table in global) so with that I'd split properly the gen code from spawning code and have more freedom to add queue.

[aaargha]

@orzelek
The thing is, the more I read up on what survives save/load the less sure I am the my settings survive that. Will the on_chunk_generated listener in RSO be reapplied after a load? Will the custom callback survive a load? At the moment I have no idea. Some more testing will probably be needed if you want to include it in the mod. Unless you, unlike me, know how those thing work that is:)

@swni
I'm not too sure if a general Memoize is that useful, the patterns are rarely accessed many times in the same spot. I think adding it to the patterns/transformations that do makes more sense. I was thinking that tile could, in a way, "compile" the given patterns over an area and then just access the resulting matrix. Would probably be pretty powerful if tile was one of the last operations used as that would allow caching basically the whole pattern.
It's not much of an issue during standard play (I think, might be with driving) just noticed that it was alot easier to overrun the generator when exploring in sandbox mode, and some lag when exploring new areas. However as this runs in the main thread (it's not on the map gen thread right?) it does cause some lag instead of just making the reveal slower.

Some benchmarks, on a i7 350 at 3GHz, by timing the following:

Code: Select all

/c game.speed = 30
/c game.forces.player.chart(game.player.surface, {lefttop = {x = -1024, y = -1024}, rightbottom = {x = 1024, y = 1024}})

Running RSO and water maze with my ribbon completes the chart in about 3:16 while maintaining about 80 UPS.
Running only RSO completes in 1:03 and maintains just short of 300 UPS.
Idle is about 1500 UPS. At normal game speed charting with the maze dropps the UPS to about 50, so it's noticable during play but not too bad.

While looking at the code I noticed the overlap in make_chunk.

Code: Select all

	local k = 5

	tiles = {}

	for x = x1 - k, x2 + k do
		for y = y1 - k, y2 + k do

Is it really needed? It increases the number of iterations in the loop by about 70% and removing it makes the chart take about 2:23 at 110-120 UPS.

I think I'll add a Rectangle(x1, y1, x2, y2) method and maybe let Circle take an optional center argument and let Union and Intersection have any number of arguments... that should help a bit with this sort of stuff. Then you can union together three rectangles, tile it, and union with a strip and starting area.

Thats sounds much better than the voodoo I'm currently using.

To do a 180 rotation you should be able to do Affine(pattern, -1, 0, 0, -1). It should keep everything as integers and therefore avoid rounding. Let me know if that doesn't work well. To do a mirror do Affine(pattern, -1, 0, 0, 1) or Affine(pattern, 1, 0, 0, -1) for x and y respectively. (Also, I'd be curious to know which pattern you tried that caused Rotate to round poorly so that I can fix that one.)

Huh, that will teach me (probably not) to properly look at what's available My first apporach was basically to make one side and then rotate and translate it to make the other. Something like: "side2 = Translate(Rotate(side1, 180), 64, 96)".This gave some uneveness on the nodes and some noise on the edge of the trunk a few nodes from the start iirc.

[swni]

From what I saw of your code I would think it survives save/load fine. You are just storing strings (and tables of strings, etc.) in the global table which is persistent.

If I provided Memoize then you can just do Tile(Memoize(super complicated pattern)) to get what you want.

I believe the overlap is needed because the game refuses to put land except grass next to water, so you can occasionally have artifacts caused by boundaries between chunks that have been processed vs not processed. The overlap makes sure these artifacts are dealt with before the user sees them. Of course there's no documentation about these details of Factorio's terrain generation and this is rather unpredictable so it was a bit of a pain. I'm pretty sure you can safely reduce k to 2, and I'll do that on next release.

By the way, thanks for all the feedback / bug hunting / example code!

[aaargha]

Never thought I'd say this but, playing Factorio might have to wait. It's time to start making mods (maybe)

What I'm planning is to turn this custom call-back system and make some kind of code package/library/interface and an optional mod to manage it. The plan is to allow for a more fine grained map generation by replacing the on_chunk_generated listener with a custom listener. This would allow more events representing different parts of map generation, ex: new_chunk, terrain_done, ores_done, enemies_done, trees_done, or whatever. This would allow mods like RSO that depend on the terrain being generated to register for the terrain_done callback. Mods like Water maze would register for the new_chunk callback and it would be guaranteed that they are done before the ore is generated (as long as people adhere to the interface).

It would also remove the need for different world gen mods to know of each other, as long as they use this instead of on_chunk_generated they can happily coexist.

What I'd like to do is make it more robust, new mods/removed mods/fallback if manager mod is removed/not present, ensure determinism (might be an issue, not sure), and add some example code that those who want to use it can more or less copy/paste or include in some way, to make it easily addable.

So, as my potential user-base (both of you ) yay, nay, thoughts?

@swni
Memoize like that would work just fine with me. I could also do Memoize(Union(Spiral(), Cross())) and just waste some memory, but that's on the user

Ah, that explains things. Didn't take a closer look at the result, just ran the benchmark. However won't this almost always end up processing tiles in chunks that haven't been generated yet? I mean, it obviously works, it just feels weird.

Cheers, it's a really neat mod, glad to be of help.

[swni]

Do you know other mods besides these two that would benefit from a more fully featured world generation system? I did consider branching out into resource generation, but in the end all I did was write the Diffuse Resources mod, which works out of the box with Water Maze and is totally incompatible with RSO.

It does go over into chunks not ready yet, yes. Those chunks have terrain generated but don't have resources placed, which led to artifacts in resource generation in early versions of this mod (see https://imgur.com/a/bxKRP). And it weirds me out too hahaha

[orzelek]

Making fully blown event handling as separate mod might be tricky - mods would need to be able work solo most likely. And there are not that many mods that change world gen.

Making RSO and Water maze work together might be worth it - on the other hand it might come with 0.15 for free

[aaargha]

While I'm not too worried about getting it to work, it's like you say, there is basically no demand. Looking around a bit I couldn't find any other mod that would use this. That leaves RSO, Water maze and Straight world. Seeing as Straight world is already bundled with RSO for compatibility it makes more sense to focus on making Water maze and RSO play nice, even if it soon won't be needed, don't really want to wait to .15 to play

I'll do a bit more testing, mostly for the RSO side, and try to make sure that it's possible to slot these changes into an ongoing save running any/both of these mods. Hopefully I'll be back later today/tomorrow with the completed code files for anyone else who wants to run this, or if any of you want to include it in your mods.

@orzelek
Not sure if this is a feature but the remote calls for "clear" and "regenerate" don't touch the spawn area as they seem to rely on init() to do that (regenerate_everything() excludes any starting area), and init() is made to only run once.

[aaargha]

Right, here are the modified control.lua files. The pieces I've messed with are tagged with --new or --changed/new, and --/new . For water maze it should be easy to find, for RSO there is a chunk starting at line 1585 and a little bit more at 1747.

From my (limited) testing it survives save/load, toggling any of the mods on or off (see the beautiful attachment) and both normal and sandbox play. If there are any issues, please let me know and I'll take a look at them.

Also, how do I make attached images smaller, this is retarded.

@swni
It seems like water maze does not properly handle maps of a limited size (as seen in the other oves-ized attachment), seems like there's a boundary check missing somewhere.

[orzelek]

I see I should really fix the delayed init trick at some point
Code is good there and it should work without problems with one assumption. Water maze needs to make sure that it passes unmodified event object to not cause crashes
I could try to add some guards but I'm hoping they are not needed - it would die on first chunk gen so should be obvious.

Regens in RSO ignoring starting areas is a feature.
Your code survives save/load since it's in the on_load event handler. And delayed init is my way of not caring about event order.

Actually looking at this code it might be even more complex then it needs to be.
There is no need to have the registration in water maze at all. I'll simplify this a bit.

There will be two external interfaces - one to disable built in generation and other to generate the chunk.
So from water maze perspective you'd need to call the first one in on_ini and on_load and then call ore generation after spawning terrain.

This way means that in theory any mod will be able to do this and in case of two like those present I think that generation is currently protected from double calls

Edit:
Attached modified version here.
Please call:

Code: Select all

disableChunkHandler

from both on_init and on_load and then use:

Code: Select all

generateChunk(event)

from chunk event handler after generating terrain.

It's double protected on handler disabling - thats by design in case the order of calls goes the wrong way (first RSO, then water maze).

[aaargha]

I see I should really fix the delayed init trick at some point

Yeah, that had me confused for a while. My first reaction was basically 'man they sure are trying hard to avoid putting all this init code in on_init' it took a while to figure out how badly that would work out. That said it is possible to crash RSO by using either the regenerate or clear interfaces from another mod on_init.

Regens in RSO ignoring starting areas is a feature.

Is there currently any way to re-roll a starting area? The one I'll be using is pretty small, and it'd be much faster than making a new map each time. Alternatively is it possible to make it cram the resources closer together for the starting area?

Actually looking at this code it might be even more complex then it needs to be.
There is no need to have the registration in water maze at all. I'll simplify this a bit.

There will be two external interfaces - one to disable built in generation and other to generate the chunk.
So from water maze perspective you'd need to call the first one in on_ini and on_load and then call ore generation after spawning terrain.

This way means that in theory any mod will be able to do this and in case of two like those present I think that generation is currently protected from double calls

That is entirely fair, it's about the same amount of code anyway, just in Water maze instead of RSO. It was mostly just a question of where to start and the ~70 line file was a bit less intimidating than the 1.7k one

Edit:
Attached modified version here.
Please call:

Code: Select all

disableChunkHandler

from both on_init and on_load and then use:

Code: Select all

generateChunk(event)

from chunk event handler after generating terrain.

It's double protected on handler disabling - thats by design in case the order of calls goes the wrong way (first RSO, then water maze).

I'll take a peek, shouldn't be too hard... Let me just jinx my self a little more; I'll have it done in no time!

[aaargha]

Didn't manage to jinx myself too bad. I left the call back system in place as that seemed easiest. On init and load, we simply check if the RSO interface exists, disable the chunk handler, and then pretend that RSO signed up for call-backs.

Does not crash on map spawn/save/load, and seems to otherwise function as intended. Not the most thorough testing, but hopefully it's fine

@swni
If you want to include any part/all of this into your mod, feel free, maybe this feature will even get more than one user

[orzelek]

Please test it a bit more and let me know if it works and I'll publish that RSO on mod portal.
I'll do few tries here too.

[aaargha]

I'll get on that tomorrow, when I'll have some more time on my hands.

[orzelek]

I've attached simplier version. You are still writing your planned mod in one you attached aaargha

I tested it a bit on default maze and I can see resource generation will be a problem - some of patches end up a bit cut due to generation order.
It'll need to remain a known issue for now - can't fix this one without RSO rework.

[aaargha]

I've attached simplier version. You are still writing your planned mod in one you attached aaargha

Well, kind of? It seemed like a working feature that might (probably not) be used by some mod, but meh. This also seems stable as far as I can tell.

I tested it a bit on default maze and I can see resource generation will be a problem - some of patches end up a bit cut due to generation order.
It'll need to remain a known issue for now - can't fix this one without RSO rework.

It mostly works fine outside the starting area. I think the issue is that the starting area can be spawned before water maze is finished. My guess is that right now it is spawned when the first chunk is generated, which might even cause problems without Water maze. Moving starting area spawn to tick 1 seems to help the issue, not as many cut-off resources, but there is still no guarantee that all the different starting resources will spawn.

Part of the issue is that can_place_entity is a filthy liar

Code: Select all

/c game.player.print(game.surfaces['nauvis'].can_place_entity{name = "angels-ore1", position = {x = 0,y = 1000}})

Returns true in a game where the map height is set to 500. It seems to only work for chunks that have already been generated/tiles already set.

Anyway, sorry swni for derailing the thread a bit most of this should probably be in the RSO thread/forum w/e.

[system]

Latest version available through game doesn't have islandify (not even that file)

[system]

Actually some guide with pictures would be nice

[sintri]

local pattern =AllWater()

what do i change that intoo to get everything water?

Code: Select all

function AllLand()
    local function get(x, y)
        return true
    end
    return {create = noop, reload = noop, get = get, lua = 'AllLand()'}
end

function AllWater()
    local function get(x, y)
        return false
    end
    return {create = noop, reload = noop, get = get, lua = 'AllLand()'}
end

Gets all land?

Fixed by changing

Code: Select all

function AllWater()
    local function get(x, y)
        return false
    end
    return {create = noop, reload = noop, get = get, lua = 'AllLand()'}
end
}

too

Code: Select all

function AllWater()
    local function get(x, y)
        return false
    end
    return {create = noop, reload = noop, get = get, lua = 'AllWater()'}
end

[swni]

I stopped getting thread notifications again, so sorry I didn't respond to comments.

sintri, thanks for pointing it out, that will be fixed in next release.

system, not sure what you mean. Is the mod not working correctly for you? Some of the patterns got renamed recently so you might need to update a config written for an older version.

orzelek, I lost track of the back-and-forth discussion there... did you end up changing RSO in some way that requires a matching change from Water Maze?

[orzelek]

I stopped getting thread notifications again, so sorry I didn't respond to comments.

...

orzelek, I lost track of the back-and-forth discussion there... did you end up changing RSO in some way that requires a matching change from Water Maze?

I think that control.lua that I attached above is the version that should work with latest RSO. No extensive testing done on it by me and it has 4 downloads so might need more testing.