Page 1 of 1

Data stage prototype filters.

Posted: Sat Nov 02, 2019 11:34 pm
by eradicator
If filters are getting more offical support now @FFF319 it would be nice ot have data stage support too.

Examples:
All technologies that unlock a specific recipe.
All recipes with a specific ingredient.
All ammo for a particular gun.
All recipes for a particular assembler.
...

The return value should ideally be a table of references to the actual subtables of data.raw.

Re: Data stage prototype filters.

Posted: Sun Nov 03, 2019 2:16 am
by Honktown
Check out my mod, in data-final-fixes, or modules\intermediate.lua (ores are in there, as is the smelting category recipe search)

https://mods.factorio.com/mod/DeepPockets

I added n-deep searches for properties today. In the changelog I mention how to search for specific recipes.

I didn't note in the changelog that the filter works with recipe items, technically, so you can search for recipes having certain properties, AND filter by ingredients that are named iron plates, or search by ingredients with the raw-material subgroup***...

I don't have mixed pattern matching, so you can't do items that end in -plate for example

Haven't tried with technologies, but you can do arbitrary categories and arbitrary properties. The mod is designed around changing stack sizes, so you'll have to write your own changing functions.

Note: SearchByCategory returns two tables. The first table is the found entries, and the second table is found subitems (under the "and_subitems" value). You should be able to do something like and_subitem = this.that. category is for the "item", search_category is where to search, so I think you retrieve things that... aren't items. It works for capsules and stuff, should work for recipes or technologies if you get the subitem path right

Re: Data stage prototype filters.

Posted: Sun Nov 03, 2019 2:21 am
by Honktown
Er, the recipe change is designed around stack sizes, so I can change the function to have a second argument, which only returns the ingredients. Currently they're of the form { "iron-plate" = 1, "copper-plate" = 1}.

With a small tweak I can turn them back into item paths, and return them from the function.

Re: Data stage prototype filters.

Posted: Sun Nov 03, 2019 3:25 am
by Honktown
I'm having some weird bug :/ I'm going to need a bit

Re: Data stage prototype filters.

Posted: Sun Nov 03, 2019 4:39 am
by Honktown
Got it working :) Just for you <3

https://mods.factorio.com/mod/DeepPockets/changelog

Edit: forgot I use core_table all over the place instead of data.raw. Moved the definition to lib

Re: Data stage prototype filters.

Posted: Sun Nov 03, 2019 5:17 am
by Boodals
The main benefit of the prototype filters is to save the amount of LuaObjects being created. That doesn't happen in the data stage since everything is already in lua. C++ objects for the prototypes aren't even loaded until after the data stage finishes.
Maybe some functions could be written in lualib for convenience, but in terms of performance, the best solution is to do the searching yourself.

Re: Data stage prototype filters.

Posted: Sun Nov 03, 2019 5:58 am
by Honktown
Boodals wrote:
Sun Nov 03, 2019 5:17 am
The main benefit of the prototype filters is to save the amount of LuaObjects being created. That doesn't happen in the data stage since everything is already in lua. C++ objects for the prototypes aren't even loaded until after the data stage finishes.
Maybe some functions could be written in lualib for convenience, but in terms of performance, the best solution is to do the searching yourself.
Not OP. Ouch. I was assuming I didn't know enough about the data stage and something like that could happen. Is there a guide on what each load phase does?

Re: Data stage prototype filters.

Posted: Sun Nov 03, 2019 6:23 am
by Pi-C
Honktown wrote:
Sun Nov 03, 2019 5:58 am
Is there a guide on what each load phase does?
Is this what you mean?

Re: Data stage prototype filters.

Posted: Sun Nov 03, 2019 6:30 am
by Honktown
Pi-C wrote:
Sun Nov 03, 2019 6:23 am
Honktown wrote:
Sun Nov 03, 2019 5:58 am
Is there a guide on what each load phase does?
Is this what you mean?
Heh. I'm running functions and stuff during the data-final-fixes phase when I should only be doing small tweaks. Fun stuff. Thanks.

Good to know I can't modify settings. I had been thinking of seeing if I could populate a list dynamically.

Re: Data stage prototype filters.

Posted: Sun Nov 03, 2019 7:43 pm
by eradicator
Boodals wrote:
Sun Nov 03, 2019 5:17 am
The main benefit of the prototype filters is to save the amount of LuaObjects being created. That doesn't happen in the data stage since everything is already in lua. C++ objects for the prototypes aren't even loaded until after the data stage finishes.
Maybe some functions could be written in lualib for convenience, but in terms of performance, the best solution is to do the searching yourself.
Yes, this is purely a "convenience" request since it's a really common thing that many mods do and i think that reducing the "reinvention of the wheel" is a good idea. And an official implementation would also likely be faster simply because the average modder isn't as good at performance optimization as you guys are ;).

Re: Data stage prototype filters.

Posted: Mon Nov 04, 2019 12:40 am
by Honktown
eradicator wrote:
Sun Nov 03, 2019 7:43 pm
Boodals wrote:
Sun Nov 03, 2019 5:17 am
The main benefit of the prototype filters is to save the amount of LuaObjects being created. That doesn't happen in the data stage since everything is already in lua. C++ objects for the prototypes aren't even loaded until after the data stage finishes.
Maybe some functions could be written in lualib for convenience, but in terms of performance, the best solution is to do the searching yourself.
Yes, this is purely a "convenience" request since it's a really common thing that many mods do and i think that reducing the "reinvention of the wheel" is a good idea. And an official implementation would also likely be faster simply because the average modder isn't as good at performance optimization as you guys are ;).
What are you doing with searching that actually has a performance impact? I search through data.raw many times and it has very little performance difference. I'm usually only creating a handful of locals and re-using the same data object. I never have to copy an item itself.

Re: Data stage prototype filters.

Posted: Mon Nov 04, 2019 1:00 am
by Honktown
Actually, I'm going to have to do a bunch of shit to get "a recipe with a specific ingredient" working. I will do it, because I'm an asshole, but recipes have up to 3 difficulties, and, I had to check, thankfully they seem to have only two representations. The items can be named if the recipe has a catalyst, vs a string and a number that are unlabeled. There's no order to items in the ingredient list, so a specific ingredient has to be checked among any of them. I knew having search algorithms that returned true/false if something was present in a table would come in handy.

Re: Data stage prototype filters.

Posted: Mon Nov 04, 2019 5:21 am
by Honktown
https://mods.factorio.com/mod/DeepPockets/changelog

:lol:

See 0.0.18. Now I can retrieve results that have one ingredient among them, and change it to something else. Even includes detecting if the ingredient you're changing to already exists.

Edit: doesn't do catalyst checking, but I can completely change iron-plates to copper-plates without the game throwing a fit. Also uh, I left the code for paths to the items themselves, but it can point to nil tables or tables that do nothing, I think? The table of returned ingredients is not safe to use after changing ingredients.