Bring back resource_prototype.autoplace_specification.coverage

[Mylon]

tl;dr I made heavy use of the coverage attribute for determining the relative frequency of ores and scaling my map generation based on this information. The API for ore prototypes needs its own interface to be usable.

So I went down the rabbit hole of trying to figure out how ore generation works. For the sake of curiosity, I thought to peruse the autoplace specification object and here's what it looks like in runtime: https://pastebin.com/LCHRfJmG

Now that's some crazy layout. To access a property inside of that mess, I need to do something like:

Code: Select all

/c game.print(serpent.line(game.entity_prototypes["uranium-ore"].autoplace_specification.probability_expression.arguments[1].expression.arguments[1].arguments.basement_value.arguments[1].arguments[2].arguments[1].literal_value))

and this returns "0.125". Which going by the path to get there doesn't really tell me anything as there are no sane names to suggest what this number means. When attempting to run the same code but using "stone", it fails (basement_value is nil).

I believe the simple solution would be to preserve the information available at the data stage (the arguments passed to resource_autoplace_settings()) and make that accessible at runtime in a sane manner.

The alternative is that I may need to measure this empirically via generating a large area of a dummy surface and taking measurements.

[Arch666Angel]

Do it old school style: Read it in data, store it in some dummy entity.

[Mylon]

Turns out coverage isn't even a data property, but an argument fed into the autoplace_specification generation function so that doesn't work either. Most of the data fed into the autoplace function is still there, just extremely difficult to parse since it turns it until a 3000 line table via multiple recursions.

[BlueTemplar]

Oh, so this is what they meant when they talked about "the autoplace generator being a literal portal to hell" ?

[orzelek]

Oh, so this is what they meant when they talked about "the autoplace generator being a literal portal to hell" ?

Yep....
If you feel brave dump autoplace of any resource at the end of data stage.
It's a "compiled" expression - huge mess of tables in tables with various values and relations inside.

[Arch666Angel]

I had no 666 in the name prior to seeing it...

[darkfrei]

Here must be option to write autoplace specification directly in the game and see how it's looking like. I cannot imagine the result of the ore generation function.

[PyroFire]

A simpler method of tweaking the autoplacement (without significantly altering it) might be to look at and re-use the existing autoplacement generation functions for the ores.

These functions can be found in steamapps\common\Factorio\data\base\prototypes\entity\demo-resources.lua and steamapps\common\Factorio\data\core\lualib\resource-autoplace.lua


You're still looking at a headache of work though.

[Honktown]

Coverage is defined as the proportion of the map covered in that resource.

Since "coverage" was removed and replaced with a probability_expression, it is better stated as the odds of finding that resource however location is indicated, presumably chunks. If some threshold is met, then a patch is spawned, according to the richness function and other bits here and there. Mathematically, if you're having a chance at finding a resource in a given spot, you can run into an impossible situation where you don't have enough spots left to fulfill the coverage requirement. "close enough" could work, but Factorio devs opted to use a generator expression going forward, apparently.

There's remnants of code here and there.

In base\prototypes\entity\demo-resources.lua coverage is defined in the coverage key, the third argument passed to resource_autoplace_settings. However, resource_autoplace.resource_autoplace_settings are assigned now.

In the same function, coverage is taken from global, and if it is nil, then it's assigned 0.02. Since it's unused it doesn't matter.

core\lualib\resource_autoplace.lua or 'require("autoplace")' is where the 'new' autoplace function is. The text generation is from core\lualib\noise.lua.

At this point, we're interested in determining a) can we use the data available, b) can we use it automatically to get ideas and if not, c) what is the effect of the variable components on the probability Jesus Christ I did not think I'd be recommending Dimensional Analysis for a video game.

The noise expression generator is filled with a bunch of locals and returns a table of functions, the functions compose tables, keys, and values.

Personally, I'm choosing to see the resulting calculation before I decide if it's worth it to dig around and screw with the programming to get results from it. We can also turn the string back into functions/calculations with heavy regular expression use. A big complication is a lot of the constants will be calculated from other values. As long as our variables are not hidden in constant calculation results elsewhere, we can do predictions for a single map. Then we can work on expanding the calculations to include other components if we so choose (the map seed is one variable for example). Since resource distribution should be semi-predictable, one should be able to make decisions on how to scale/distribute other things.

Writing a parser for this is simple enough. If we find a string variable, we can leave it as is or format it a little better (colons to periods). If a number, convert to strings. If we're at a table with subtables, go down, evaluate arguments, return string up. Keep going down/returning up until we're left with a single string. Functions can have something like six or seven values, which is really annoying, but just means we need to have more string variables before we return up.

Functions describe themselves. Since we're not evaluating, all numbers are strings without conversion. I'll be using polish notation (lisp-like) because syntax is unambiguous. I'll worry about that later, during simplification. Thankfully, Factorio devs separated expressions and arguments. Arguments don't need parentheses for unambiguity (they can be processed in order). Expressions do (it'd be impossible to determine where one ends and another ends - like a dangling else)

Parsing note: be careful if using pairs. "type" ~= type.
...many bugs later
Okay, wrote a simple parser.

For uranium's probability I get at start-up for my game, I get:

Code: Select all

Result of analyzing uranium's probability expression: (clamp (+ (spot-noise x y map_seed 100 1024 21 45.254833995939 regular-resource-patch-set-count 5 (* (* (* 0.9 (* control-setting:uranium-ore:frequency:multiplier control-setting:uranium-ore:size:multiplier ) ) (+ 1 (/ (clamp (- distance 300 ) 0 1300 ) 1300 ) ) ) (clamp (/ (- distance 120 ) 300 ) 0 1 ) ) (* (random-penalty x y 4 2 ) (/ (* (* (* (* 0.9 (* control-setting:uranium-ore:frequency:multiplier control-setting:uranium-ore:size:multiplier ) ) (+ 1 (/ (clamp (- distance 300 ) 0 1300 ) 1300 ) ) ) (clamp (/ (- distance 120 ) 300 ) 0 1 ) ) 1000000 ) (* 1.25 control-setting:uranium-ore:frequency:multiplier ) ) ) false (clamp 32 -inf (* 0.1 (pow (* (random-penalty x y 4 2 ) (/ (* (* (* (* 0.9 (* control-setting:uranium-ore:frequency:multiplier control-setting:uranium-ore:size:multiplier ) ) (+ 1 (/ (clamp (- distance 300 ) 0 1300 ) 1300 ) ) ) (clamp (/ (- distance 120 ) 300 ) 0 1 ) ) 1000000 ) (* 1.25 control-setting:uranium-ore:frequency:multiplier ) ) ) (/ 1 3 ) ) ) ) 1 (clamp (* -6 (* 0.125 (clamp (/ (pow (* 3 (/ (* (* (* (* 0.9 (* control-setting:uranium-ore:frequency:multiplier control-setting:uranium-ore:size:multiplier ) ) (+ 1 (/ (clamp 1300 0 1300 ) 1300 ) ) ) (clamp 4.9333333333333 0 1 ) ) 1000000 ) (* 1.25 control-setting:uranium-ore:frequency:multiplier ) ) ) (/ 1 3 ) ) 0.010471975511966 ) -inf (/ (pow (* 3 (/ (* (* (* (* 0.9 (* control-setting:uranium-ore:frequency:multiplier control-setting:uranium-ore:size:multiplier ) ) (+ 1 (/ (clamp 1300 0 1300 ) 1300 ) ) ) (clamp 4.9333333333333 0 1 ) ) 1000000 ) (* 1.25 control-setting:uranium-ore:frequency:multiplier ) ) ) (/ 1 3 ) ) 0.010471975511966 ) ) ) ) -inf (* -6 (* 0.125 (/ (pow (/ (/ (* (* 36000 (+ (* (- control-setting:uranium-ore:frequency:multiplier 1 ) 0.5 ) 1 ) ) control-setting:uranium-ore:size:multiplier ) 0.5 ) control-setting:uranium-ore:frequency:multiplier ) 0.33333333333333 ) 0.021371378595849 ) ) ) ) 128 ) (* (- (+ (+ (factorio-basis-noise x y map_seed 100 0.125 1 ) (factorio-basis-noise x y map_seed 100 0.041666666666667 1 ) ) (factorio-basis-noise x y map_seed 100 0.015625 1.5 ) ) (/ 1 3 ) ) (* 0.125 (clamp (/ (pow (* 3 (/ (* (* (* (* 0.9 (* control-setting:uranium-ore:frequency:multiplier control-setting:uranium-ore:size:multiplier ) ) (+ 1 (/ (clamp 1300 0 1300 ) 1300 ) ) ) (clamp 4.9333333333333 0 1 ) ) 1000000 ) (* 1.25 control-setting:uranium-ore:frequency:multiplier ) ) ) (/ 1 3 ) ) 0.010471975511966 ) -inf (/ (pow (* 3 (/ (* (* (* (* 0.9 (* control-setting:uranium-ore:frequency:multiplier control-setting:uranium-ore:size:multiplier ) ) (+ 1 (/ (clamp (- distance 300 ) 0 1300 ) 1300 ) ) ) (clamp (/ (- distance 120 ) 300 ) 0 1 ) ) 1000000 ) (* 1.25 control-setting:uranium-ore:frequency:multiplier ) ) ) (/ 1 3 ) ) 0.010471975511966 ) ) ) ) ) 0 1 )

Yikes.

If there's an internal translator for the built-in functions (clamp, etc) we can just s/clamp/actualfunction/g. We can assign variables however we want. Obviously we'd like to use ones that represent normal values.

Unfortunately a lot of the other values are calculate from other things, but by building a reverse calculator, we can break them back into starting variables.

Getting somewhere!

[Honktown]

I'll just say now, it'd probably be easier to write a mod that adds up resources in an area, and then if the areas are small enough, you can get a rough idea of how resources scale with distance (maybe you could load the x/y/amount in a 3D plotter and look at areas made by concentric circles or how distance change is). Really, distance and richness should be the only values that matter. Unless they changed frequency, it made the patches smaller, but there were more of them. I don't know if the frequency * richness remained constant.

[BlueTemplar]

Jesus Christ I did not think I'd be recommending Dimensional Analysis for a video game

Welcome to Factorio.

I'll just say now, it'd probably be easier to write a mod that adds up resources in an area, and then if the areas are small enough, you can get a rough idea of how resources scale with distance (maybe you could load the x/y/amount in a 3D plotter and look at areas made by concentric circles or how distance change is).

Yup, this is pretty much what I plan to do with RSO after this discussion (except I would have just compressed the circles of constant radius to a single point)... when I get around to it... someday

[orzelek]

Jesus Christ I did not think I'd be recommending Dimensional Analysis for a video game

Welcome to Factorio.

I'll just say now, it'd probably be easier to write a mod that adds up resources in an area, and then if the areas are small enough, you can get a rough idea of how resources scale with distance (maybe you could load the x/y/amount in a 3D plotter and look at areas made by concentric circles or how distance change is).

Yup, this is pretty much what I plan to do with RSO after this discussion (except I would have just compressed the circles of constant radius to a single point)... when I get around to it... someday

Tbh I don't think you need to do that for RSO.
RSO has a formula it kind of follows and it can be used to figure out at least approximate amount of resources in an area by distance.
It's all a bit vague since there is a lot of randomness applied to it (chance to spawn resource, multiple spawns with lots of ores, random factor applied to resource amount, naughty terrain eating up parts of ore patch) but there is a general formula in RSO and I think it's much simplier that what you have above in vanilla
Here is an approximate formula (with lots of randomness added on top of it in reality and assuming I did not forget anything while checking code):

Code: Select all

distanceInRegions^richnessExponent*distanceInRegions^sizeExponent^0.6

Exponents are available in RSO settings.

[BlueTemplar]

That's great (and thanks for clarifying the formula), but since the goal is to compare with vanilla, I would have to code it anyway, at which point it's just a matter of running the same program on RSO...

[orzelek]

That's great (and thanks for clarifying the formula), but since the goal is to compare with vanilla, I would have to code it anyway, at which point it's just a matter of running the same program on RSO...

I'd be interested in results of something like that.
Base goal for RSO was that it would have more sparse resources that increase faster then vanilla. I have no idea how this works after 0.17 generation update.