Request: Move recycling recipe generation to data-final-fixes

[0n0w1c]

I agree, however you can just put:

Code: Select all

if mods["quality"] then require("__quality__.data-updates") end

In your own mods data-final-fixes (data, data-updates, or anywhere else after you've added or changed recipes)

As of the date of this reply, it is not a good idea to implement the above... at least not without more context.
Generally calling it, results in the recycling recipe for nutrients to be changed to biter eggs.

I switched to this to re-generate only specific recipes:

Code: Select all

    local recipe = data.raw.recipe[recipe_name]
    if mods["quality"] then
        local recycling = require("__quality__/prototypes/recycling")

        recycling.generate_recycling_recipe(recipe)
        recipe.auto_recycle = nil
    end

[IsaacOscar]

As of the date of this reply, it is not a good idea to implement the above... at least not without more context.
Generally calling it, results in the recycling recipe for nutrients to be changed to biter eggs.

Oh god, why?
(I guess the order of iteration of the "for name, recipe in pairs(data.raw.recipe) do" loop changes when data.raw.recipe is modified?
Maybe someone could ask the dev's to modify the code to sort data.raw.recipe first?)

I switched to this to re-generate only specific recipes:

Code: Select all

    local recipe = data.raw.recipe[recipe_name]
    if mods["quality"] then
        local recycling = require("__quality__/prototypes/recycling")

        recycling.generate_recycling_recipe(recipe)
        recipe.auto_recycle = nil
    end

I think you mean auto_recycle = false.

Also the above isn't great either as:

1. There may already be a recipe with the same output
2. The above may fail to generate any recycling recipes (in particular, it won't generate a self-recycling recipe if the given recipe isn't recyclable)
3. You have to actually run the above code for each recipe

Unfortunately, without reimplementing all the quality code I can't fix all three issues, but this may still be a better compromise?

So I think a better option is this:

Code: Select all

local recycling = mods["quality"] and require("__quality__/prototypes/recycling")
function maybe_generate_recycling(recipes, item) -- item can be nil
	if recycling then 
		if item ~= nil then
			data.raw.recipe[item.name .. "-recycling] = nil -- delete any existing recipe
		end
		for recipe in pairs(recipes) do
			recycling.generate_recycling_recipe(recipe) 
		end
		-- Generate a self recycling recipe if the above did nothing
		if item ~= nil and not data.raw.recipe[item.name .. "-recycling"] and not string.find(item.name, "-barrel") then
			recycling.generate_self_recycling_recipe(data.raw.)
		end
	end
end

Sadly the above isn't very good as it won't deduce the item from the given recipes. Also, if there are multiple recipes that can be recycled, then the last one wins? Also if the recipes recycle to something other than the given item, it won't work properly.

[0n0w1c]

I think you mean auto_recycle = false.

Actually, I think auto_recycle should not be changed, if (re)generating a recycling recipe. I believe the intent of the field is to not generate a recycling recipe.
I took that line from the quality mod's data-updates.lua script, but I think for my purposes, it should be left alone.

[curiosity]

Actually, I think auto_recycle should not be changed, if (re)generating a recycling recipe. I believe the intent of the field is to not generate a recycling recipe.
I took that line from the quality mod's data-updates.lua script, but I think for my purposes, it should be left alone.

Hmm. Actually, because of that line you can not simply rerun the same code. And because of that line the quality mod's changes are final and should be in data-final-fixes. So either they need to remove that or move the generation.

[IsaacOscar]

I believe the intent of the field is to not generate a recycling recipe.

Yup

I took that line from the quality mod's data-updates.lua script, but I think for my purposes, it should be left alone.

Ahh, my guess is it was simply deleting it as the field is no longer necessary. Also that line may then be what's causing the bitter egg problem from before.

[IsaacOscar]

I tested it, and it is indeed that line that's causing the problem. The order of iteration is irrelevant (as Vanilla doesn't have any ambiguous recycling recipes).
So I've made a new modding interface request (125567) to delete the auto_recycle = nil.

Note that the order of iteration will become relevant if there are multiple recyclable recipes that output the same item. But this is the mod's fault, and can be resolved by ensuring all but one of them have auto_recycle = false.

[0n0w1c]

In with testing on a custom entity without a defined recycling recipe, the recycling recipe is still auto generated when auto_recycle = false. Not what I expected,
the results become of 25% of the item rather than 25% of the ingredients.

[IsaacOscar]

In with testing on a custom entity without a defined recycling recipe, the recycling recipe is still auto generated when auto_recycle = false. Not what I expected,
the results become of 25% of the item rather than 25% of the ingredients.

Yup!
auto_recycle means the recipe is ignored when generating the recycling recipe.
If it can't find any recipes for a given item to recycle, it always give you a 25% of the item.
(That way absolutely every item can be put in a recycler, even items you can't normally obtain like vehicle weapons).
If for some reason you don't want that, you can do data.raw[item.name .. "-recycling"] = nil in data-final-fixes.lua, but that won't stop it getting added if another mod tries to generate the recycling recipes.

[brevven]

Actually, I think auto_recycle should not be changed, if (re)generating a recycling recipe. I believe the intent of the field is to not generate a recycling recipe.
I took that line from the quality mod's data-updates.lua script, but I think for my purposes, it should be left alone.

Hmm. Actually, because of that line you can not simply rerun the same code. And because of that line the quality mod's changes are final and should be in data-final-fixes. So either they need to remove that or move the generation.

Yeah we just ran into this issue. Recycling recipe generation currently can't work out of the box because it removes the "data.raw.recipe[recipe_name].auto_recycle" metadata at the data-updates stage, and many mods will make their updates after quality due to mod load order.

At the very least, the developers will need to remove the line that sets "recipe.auto_recycle = nil" otherwise there's no possible universal workaround. Incidentally, I tend to agree that it should be in data final fixes, because it's cross-cutting and should inherently respect the updates mods make to one another's prototypes, rather than ignoring updates by mods that load after it.

In the meantime, I'm going to do the best I can in my mods to keep the recycling fidelity by copying data over to a "recipe.auto_recycle_helper" parameter as noted in the mod portal thread I linked, but it won't be perfect.

[IsaacOscar]

Yeah we just ran into this issue. Recycling recipe generation currently can't work out of the box because it removes the "data.raw.recipe[recipe_name].auto_recycle" metadata at the data-updates stage, and many mods will make their updates after quality due to mod load order.

At the very least, the developers will need to remove the line that sets "recipe.auto_recycle = nil" otherwise there's no possible universal workaround. Incidentally, I tend to agree that it should be in data final fixes, because it's cross-cutting and should inherently respect the updates mods make to one another's prototypes, rather than ignoring updates by mods that load after it.

In the meantime, I'm going to do the best I can in my mods to keep the recycling fidelity by copying data over to a "recipe.auto_recycle_helper" parameter as noted in the mod portal thread I linked, but it won't be perfect.

It might be possible to recover that field:

1. Make a copy of __quality__/prototypes/recycling.lua, and include it in your mod
2. At the end of the generate_recycling_recipe function, where it does:

   Code: Select all

   recipes[result.name] = result 
   

   Change it to

   Code: Select all

   if recipes[result.name] and not table.compare(recipes[result.name], result) then
       recipe.auto_recycle_helper =  false
       recipe.auto_recycle = false
   end 
   

3. Replace the line at the end of the file, return lib
   with

   Code: Select all

   for name, recipe in pairs(data.raw.recipe) do
     lib.generate_recycling_recipe(recipe)
   end
   

4. Now just require the above file at the start of your data-updates, and the auto recycle field should be recovered

This has two major problems though:

• It requires you to update your recycling.lua file whenever Factorio releases a new version
• It won't work if another mod has regenerated the recycling recipes before your data-updates, but has not restored the auto_recycle field

So basically only useful if you can ensure your data-updates.lua is the first non-Wube one to run. (perhaps give the containing mod a name like "000000" or something else that is likely to sort first).

[brevven]

I do not believe we will be doing this. Next someone will ask for a final final fixes because they want to add recipes in final fixes and then have recycling recipes made.

And then a request to move it to final final final fixes.

(Sorry for the double post, just wanted to reply directly to this.)

Rseding, I know your development team are (rightfully) wary of requests to add more phases, but that's not what this request is about.

Rather, the issue is about the fact that recycling recipes as currently implemented in the quality mod do not play well with other mods' recipe prototype updates. For many years, the data-updates phase has been the place that we modders add updates to other mods prototypes (including recipes). Now, a huge number of mods that modify other mods' recipe prototype updates are going to load after quality due to a (possibly optional) dependency on space-age mod, which itself depends on quality.

Also, any mod that doesn't depend on quality cannot be sure whether their updates will happen before or after quality generates recycling recipes.

Because no mod can guarantee their modifications done during data-update phase are seen by the quality mod, every mod compatible with quality that modifies recipe prototypes from other mods will need to regenerate recycling recipes. That's an extra level of boilerplate needed for a large number of mods.

(And just to note, this is a related but separate issue from it being currently impossible to reliably know whether a recycling recipe should be regenerated or not, due to quality deleting the "auto_recycle" metadata from the prototype in the data-update phase)

[safthelamb@gmail.com]

+1 as the other dev in the thread linked by brevven. Since all mods effectively depend on the quality module, mods are effectively limited to only modifying recipes in the data stage unless they manually regenerate recycling recipes, which is quite restrictive.

I do genuinely like modifying recipes to all be within the one data stage, following the same paradigm space-age uses. It's enforced a rough hierarchy for my mods in a way that's mostly helpful and only sometimes convoluting. But I think having a community consensus amongst mod developers on best practice for modifying recipes is mandatory, and the fact that auto_recycle is set to nil causes mods that re-run generate_recycling_recipe() on all recipes to be very destructive to other mods.

EDIT: +2 to brevven's follow-up. I absolutely sympathize with the fear of requests for data-final-final-v2 stages. I specifically don't want that, as I like the paradigm mods have used for a while of:

define new prototypes in data
modify prototypes in data-updates
make prototypes based on the "final version" of other prototypes (e.g. https://mods.factorio.com/mod/the-one-m ... rom=search) in data-final fixes

It's somewhat arbitrary, as like I mentioned above, most recipe updates can absolutely be solved in a singular stage, but it's very intuitive to update recipes in the updates stage. And so the fact many of us have already agreed upon it is quite valuable.

[IsaacOscar]

Also, any mod that doesn't depend on quality cannot be sure whether their updates will happen before or after quality generates recycling recipes.

I recall being told that the Wube mods always get loaded first, so depending on Quality won't make a difference here.

[Khazul]

+1 as the other dev in the thread linked by brevven. Since all mods effectively depend on the quality module, mods are effectively limited to only modifying recipes in the data stage unless they manually regenerate recycling recipes, which is quite restrictive.

Best to treat SA as base game once the dependencies are declared. If your dependencies or mandatory, then you can definitely treat them as base game as far as I can tell. Given that much of SA seems to be implemented in C++ anyway, then I guess this make sense. If not, then run-time check and then treat a base game.

In my SA mods that need to make data changes before automated generators get to run, then this is exactly what I do - like you suggest, do it in data.lua just as if you were modding base game data, which in the case of an SA mod, you essentially are. Initially I was trying to treat SA as a mod and ran into all sorts of issues, but since just treating it as base game, then it all gets a lot simpler and things seem to work as expected.

Perhaps the correct response from Wube on this in not to say 'wont implement', but say 'wont implement because' we should treat it as base game after declaring the dependencies - ie provide the clarity around it if they havn't already somewhere.

[safthelamb@gmail.com]

For those looking, an alternative solution has been approved: viewtopic.php?f=65&t=125567

The Quality module will, afaik, still load in data-updates.lua, but mod developers are free to re-run the generate_recycling_recipes loop.

[Stringweasel]

I want to +1 this Won't Implement so that it might be reconsidered for 2.1 whenever it releases.

Ideally the recipe generation should occur where it's most useful for the developers and for most modders. I think for the the developers it doesn't make much of a difference, but for most modders it will be a big improvement.

There are many mods that changes prototypes, it's basically the main thing to mod. Big mod packs have thousands of lines of code handling this, and many more small mods too, and they all need to remain compatible somehow. Luckily we have three stages and dependencies for that, but that still results that many mods having to make large amounts of changes in `data-updates`. For comparison, probably orders of magnitude more than changes done in `data-final-fixes`.

And the part I want to address is that with the recycling generation also happening in `data-updates` means that every change to a recipe needs to update it's corresponding recycling recipe. Because it's not possible to force your mod to run before recycling generation using dependencies. That means a lot of work for big mods to handle all these edge cases. Many new modders will not even know about this side-effect. There will almost allways be recipes that will slip through the cracks, meaning there will almost always be mismatching recycling recipes. And moving the recycling generation will likely remove 95% of these instances with almost no side effects. For AAI Industries we've already gotten a few bug reports on this exact issue, and will likely receive more in the future.

There are no new limitations created by moving it to `data-final-fixes`. All it would mean is that now if you want to modify recycling recipes you have to do it in `data-final-fixes` and depend on `quality`. But there's not many mods that do that, there's not even many mods that do anything in `data-final-fixes`. For all the other recipe changes in the first two stages, which is likely 90% of all recipe changes, they won't even need to think recycling generation.

I'm not asking to add more stages. I just think it will make more sense in data-final-fixes and save many-many hours to most modders.

[KiwiHawk]

I'm not asking to add more stages. I just think it will make more sense in data-final-fixes and save many-many hours to most modders.

Well put Stringweasel! I fully support this proposal! I've released Bob's mods with support for Quality mod and it added a huge amount of additional work. I'm currently updating Angel's mods and that is going to be even more as it updates all of Bob's recipes as well as recipes from the other Angel's mods. Sea Block is going to be another layer on top.

Technically it's not just recipe changes that currently require recycling recipes to be updated. When updating item icons, the recycling recipe icon needs to be updated too.

Moving the recycling recipe generation to data-final-fixes would fix almost every issue "for free". Is there any downside to it that I'm missing?