Resource Spawner Overhaul - Old discussion thread

[yeganer]

With 0.12 it is possible to read the mapsettings so i think the best would be to use that feature and use the config for more advanced stuff.

That way the settings at the game creation screen are meaningful again.
In the config one could control how big a size setting of very large actually is etc.

[orzelek]

With 0.12 it is possible to read the mapsettings so i think the best would be to use that feature and use the config for more advanced stuff.

That way the settings at the game creation screen are meaningful again.
In the config one could control how big a size setting of very large actually is etc.

I'd like to do that - can you tell me where do you have this info from?
I didn't see it in modding changes list.

[yeganer]

With 0.12 it is possible to read the mapsettings so i think the best would be to use that feature and use the config for more advanced stuff.

That way the settings at the game creation screen are meaningful again.
In the config one could control how big a size setting of very large actually is etc.

I'd like to do that - can you tell me where do you have this info from?
I didn't see it in modding changes list.

Hey, i saw it here: http://pastebin.com/P7MX7X06 line 68.

Code: Select all

    - Added LuaSurface::map_gen_settings - the current map gen settings for the surface.

I think with that change it could be a great opportunity to refactor some of the codebase.
With 0.12 there are no absolute coordinates and thus RSO would fail on surface switching.
My idea on the fix would be to move the config to global.rso and include the surface as a entry so one can store different settings for each surface. If there is a mod in the future that creates new surfaces it won't break RSO. The next benefit of storing the config in global would be that the config is basically independent of your local rso settings because they are saved into the savegame. that way one could play different worlds with different settings without modifying the config.lua.

If one would want to change the config midgame, i'd provide interface methods to do so and a migration.lua file that is empty by default but one could fill it to force changing the config of already started maps.

Another benefit of using a global setting and introducing an interface to manipulate the current config would be that mods could call the RSO interface directly and add it's own resources.

Well these were my ideas how to refactor RSO when 0.12 is released. I already tried to do some of the stuff on a local copy, but stopped because i wanted to do it proper and for that i needed the map_gen_settings.
I'd be glad to help refactoring if you think that's a good idea.
Feel free to PM me any questions you have regarding this topic.

[orzelek]

I'm not sure I understand all of this - I don't have any idea what LuaSurface represents in logical terms.
And you are writing that absolute coordinates will go away ? That sounds like very unfun for how RSO is working currently.
One thing I can't find is how to get LuaSurface for arbitrary location - assuming there are different LuaSurfaces objects used to compose the whole map. Without this capability I'm not sure how to place ore at given coordinates.

Until I can experiment with it there is not much I can plan to do.

As for config - it's already stored in global data structure. This is one of the reasons why it won't notice new resources if you add new mods. Regenerating it or allowing other mods to modify it might get tricky - from the moment it is changes whole resource generation changes to different probabilities. Allowing mods to modify it - that I'm not sure about. It opens a lot of potential problems since there is no longer any control on whats in the config actually.

I'm all after reading map gen settings - that will make RSO work like normal generation in regards to resource configuration.

From what I see updating RSO to 0.12 might take much longer then I anticipated - and it will require new wiki update on logical changes probably.

Edit:
And ofc I forgot the most important part - I'd prefer to refactor after integrating for 0.12. It will be much easier to test it and keep stuff under control if changes are separated.

[yeganer]

I'm not sure I understand all of this - I don't have any idea what LuaSurface represents in logical terms.
And you are writing that absolute coordinates will go away ? That sounds like very unfun for how RSO is working currently.
One thing I can't find is how to get LuaSurface for arbitrary location - assuming there are different LuaSurfaces objects used to compose the whole map. Without this capability I'm not sure how to place ore at given coordinates.

From the modding change list it looks like LuaSurface replaces the part of LuaGame that controls interaction with the world. If i remember correctly stuff like create_entity will be moved to LuaSurface. I think there will be some way of optaining the currenct surface. And i hope there is a way to create new surfaces because that would allow mods to create new dimensions. Like for example introduce a new endgame material that is only optainable in a new surface where biters are super strong.

I think the way it works is, that instead of one world there will be different surfaces, one for the space platform for example, one for the normal ground. It would make sense to have one set of coordinates for each surface. that means, RSO has to check which surface the new chunk is on, and then use the appropriate settings. I think it shouldn't be that hard to port RSO to 0.12. It should be just the renaming of some functions (use surface instead of game, createentity -> create_entity) and modifying the global config to store settings on a per surface basis. With that RSO should be working.

As for config - it's already stored in global data structure. This is one of the reasons why it won't notice new resources if you add new mods. Regenerating it or allowing other mods to modify it might get tricky - from the moment it is changes whole resource generation changes to different probabilities. Allowing mods to modify it - that I'm not sure about. It opens a lot of potential problems since there is no longer any control on whats in the config actually.

Sorry for that, the RSO version i looked at and modified was 1.0.6 i think and the 1st thing i experimentet with were the global settings.

Factorio provides two events to help with modifyable config.

game.oninit is ONLY called when the map is created. This would be the place for other mods that want to access the interface of factorio to do that.

game.onload on the other side is called every time the save is loaded. This would be the place for other mods to check if the resources are already loaded or if the mod just got installed recently after the game was started to set the config up.
This would allow the mod author to have some control over the settings RSO uses.

I'm really looking forward to the map gen settings. there is so much potential.

[n9103]

Regarding your last point, yeaganer, take note that *any* change in the map generation process results in highly noticeable terrain and resource discontinuities.
I would not be surprised in the least if any modification to RSO after map start resulted in that.

The discontinuity isn't as big a problem as it used to be, however, since the map is much better at generating transition zones.

[orzelek]

Any random generation based on order of events will give very different results if any condition changes.

Adding new ore to config after game started is easy - it will have it's consequences.
Main one being regeneration failure since it will create completely different layout. Multiplayer might be damaged unless all the adding happens in exactly same way on both sides.

Also ore you add might not appear quickly due to way that RSO works - regions are pregenerated. This means that new ore has chance to appear only on regions that were never touched till now.

Regeneration of config data is really easy to do currently - adding new config by external mods is technically also quite simple. In practice it creates host of logical problems that can lead to potential crashes. Some kind of validation of what is added would need to be done and thats a bit tricky to ode - lots of bugs/crashes potential. I'd really like to avoid that one.

Config data is currently preprocessed once and then persisted - I think that actual config might be referenced somewhere still - I'd need to make sure thats not the case since it can cause problems. Removing mod that adds ore from game with RSO is not really supported currently (by which I mean it might crash on you some time later ).

[SirRichie]

Having looked at the RSO code extensively in the past, and also created a test mod, I think the only sensible way to get new resources to work is to regenerate everything.
Thereby, it is unlikely that a desync will occurr, as everyone generates everything afresh.

Thus, you would basically remove all resources from the map, then delete the region tracking information and call the onchunk method handling for every already known chunk. The drawback with this of course would be that resources may end up somewhere completely different, forcing players to rearrange their map.

[orzelek]

Thats why I'm staying away from it

Also I'm not 100% sure that full regen would be deterministic. One of players would need to do that and then allow others to join with new resource layout.

This still means that there are few bugs if you remove a mod that adds ore - it's protected now I think so it won't crash but it will still try to add that ore.

[yeganer]

I think all of the problems could be solved with some tricks and proper initialization at save loading.

I'll see what i manage to do but i have exams the next weeks so don't expect anything before early august.

[orzelek]

I don't think they can't be solved - I still don't see enough reasons to risk this in first place.
It will create a lot of tricky logic trying to handle all the cases that can happen. And a lot of protections in places to make sure nothing will crash if something is removed.
All of that would allow adding or removing mods that add/remove ores on the fly. While I can see it being helpful in some rare cases overall it's not worth the risks in my book. Allowing mods to add/remove ores would need a lot of support on mod authors side and give potentially a lot more balancing problems due to way few things work (like allotment mechanics).

It might be worth it to add rescan remote call with big disclaimer along the lines of "Use at your own risk - might break the game".

[yeganer]

My main intention is not to allow adding of ores in an existing game but to allow mod authors to use RSO and not the other way round.
At the moment RSO has to add every ore config by itself and mod authors have to request that their config is added to RSO. My goal is to establish an API that mod authors can call.
This would consist of several things.

Mod authors need to be able to check in their data-updates.lua wheter RSO is loaded ( solved by creating a noise-layer called rso and then checking for it's existance). If RSO is loaded, skip adding the autoplace stuff. (no need to loop over all resources and potentionally disable ones without RSO config)

The next step is to call RSO's API and tell it what ores to add, how much of that ore is needed etc.
Internally this will be translated to size, richness, spawn_chance etc.
Optional parameters for the API are the factors for the map setting stuff. Like how much the map settings will affect the spawn properties. Or they could set yield based mining for specific ores etc.

If this is in place. One could write a simple and fast guide and any mod could use RSO within 5 minutes.

The last thing one could add would be an 'overwrite mode' where at game init RSO scans all available resources and then uses default settings for their spawn.

But you are right, implementing such an API has the drawback that one has to validate the input and take care of conflicts. what if one resource is added by two different mods? Use the 1st or the 2nd or both or mix them?

I think it's worth trying and if it works, RSO could easily become one of the most used mods.

[orzelek]

Other then using noise layer to mark that rso is present it's a sound plan. For mod detection you simply check for it's remote interface. Thats done then from control.lua.
Also any mod author would need to add RSO as optional dependency to load after it to see that it's there.

Translating some yet to be defined parameters to size and other settings is where it might fall through. I tried to make those work semi like normal config but it's not possible. So mod authors would need to provide those specific settings and from that point onward RSO generation would need to be fixed or all mods would need to be updated to account for new or changed settings. Also trying to actually describe those settings to say what they exactly do is not that simple due to complicated generation algorithm.
This still doesn't account for mod interdependencies and potential resource conflicts - and those can be solved only if the configs are written by one person in RSO. I'll rest my case that remote interface will cause trouble

If you want to add it then we can try to see how it will work in practice.

[yeganer]

The problem is that data.lua is read at game launch(when you execute the .exe). I wanted other mods to check in their data-updates.lua if RSO is loaded or not. That's why i need a noise layer. In the control.lua i can check for the Interface, that's right but i can't change any data inside control.lua, that needs to be done in data.lua etc.

[orzelek]

You don't need to change any data for RSO ore generation. It's all handled in control files.

[yeganer]

I have to somehow disable natural spawning. To achieve that there are 4 options:

1. Disable all resource spawns (resources not handeled by RSO will be missing)

2. Disable resource spawns by list (dosn't work when RSO is loaded 1st and no interface is present to check which mods are there)

3. Disable resource spawn only for vanilla and give mod authors the option to disable their own spawns if RSO is present

4. Remove specific ores from the chunk in onchunkgenerated and then spawn RSO ores. This wouldn't need any change in data but i feel it's not an elegant solution.

currently option 1 is chosen

[n9103]

Personally I don't see the point in trying to support mid-game mod adding/removal.

[orzelek]

I have to somehow disable natural spawning. To achieve that there are 4 options:

1. Disable all resource spawns (resources not handeled by RSO will be missing)

2. Disable resource spawns by list (dosn't work when RSO is loaded 1st and no interface is present to check which mods are there)

3. Disable resource spawn only for vanilla and give mod authors the option to disable their own spawns if RSO is present

4. Remove specific ores from the chunk in onchunkgenerated and then spawn RSO ores. This wouldn't need any change in data but i feel it's not an elegant solution.

currently option 1 is chosen

Mixed resource spawns look.. strange. But thats my personal opinion. So option 1 works for me.

Variant 3 is most friendly in theory. It does require each mod author to add and maintain some additional code thats not handled by remote interface call.
Even when using the layer we are still dependent on mod load order - when mod is loading it needs to see the noise layer already. So it needs to be loaded after RSO.

I don't see a clean way to do this really. Will need to be very careful about mod load order to make it work properly.
I was hoping that new map gen settings access would give us some way to handle this - we'll need to see what it brings in practice.

[yeganer]

I don't see a mod load order problem. When the feature is 1st implemented, RSO will still have all the supported mods as optional dependencies. Mods that implement RSO map gen can add RSO as optional dependency. In that state the order is not determined but the fallback of RSO will have a check if the ore is already added and the interface will check if the fallback is loaded and if so, then it will override it with the new settings, so if the interface is called, these settings will be used. As soon, as mods add RSO support, RSO can drop the optional dependency. This will result in RSO being loaded 1st.

For mod authors the support would be as simple as downloding a file and loading it in control.lua and data-updates.lua

[orzelek]

Please continue discussions in 0.12 thread.