Custom Research Effects For Entities

[twinkle_toes]

Firstly, I apologize for the length of this - nobody likes walls of text, but as Mark Twain once said: “I didn't have time to write a short letter, so I wrote a long one instead.”

The Problem

I'd like to be able to define custom research/technology effects (in particular for entities, but I can imagine this being useful outside of entities alone). This is particularly relevant for entities (which might not necessarily be part of the base game) that a modder might want to give research upgrades (especially infinite upgrades) to.

I feel that the best way to convey what I would like to be able to do is by contrasting something that can do this with something that can't.

Already Implemented Case
This effect can already be achieved (and indeed has already been done and put into a mod) for logistic robot carrying capacity research, and is done by adding a new infinite research with:

Code: Select all

effects =
    {
      {
        type = "worker-robot-storage",
        modifier = 1
      }
    }

Code extracted from this mod.

This code only works if the type "worker-robot-storage" is defined. It is also relevant to note that this effect seems to override the value for max_payload_size that is defined in the entity (on the off chance someone is reading this and doesn't know: see C:\Program Files (x86)\Steam\steamapps\common\Factorio\data\base\prototypes\entity\entities.lua line 6233; Windows installation).

Not Implemented Case
However, if the research effect you want isn't already defined (e.g. assembling machine craft speed, which is an attribute of the assembly machine entity - just as cargo capacity is for logistic robots), there is no way to define it yourself (to my knowledge - please correct me if I am wrong). Instead, you need to do something like the following process (using the example of increasing assembly machine crafting speed):
- Define a new assembly machine entity for each level of research which is exactly the same but with a higher crafting speed
- Create a new technology entry for each level (I don't believe there is a way to work it with being an infinite technology - someone correct me if I am wrong on this)
- Add to on_research_finished:
- - For all possible places that an assembling machine could be (on the ground, in player inventory, hotbar, on a belt, in chests, in a logistic robot, in a blueprint (very problematic!), etc):

• Take a copy of all the assembly machine details (progress, recipe, loaded item counts, location, etc)
• Delete the old assembly machine
• Place the new assembly machine
• Restore all of the old assembly machine details in the new assembly machine

A note: this process can potentially be simpler by having all assembly machines be the base version until placed, where they magically transform into the upgraded version, but this comes with its own problems (primarily: incorrect tooltip). Admittedly, the replacing on place method does resolve a lot of issues with how blueprints function.

Clearly, the second case is much less fun than in the first case. Notably, the second case also requires you to generate exponentially more entities relative to number of attributes you want to be able to infinitely research. It also notably doesn't play nice with other mods that use the entity in question. I also suspect that this is much worse for performance than if the research effect was supported by the base game, but its hard to say for sure, and depends how research effects work behind the scenes.

My Proposal

Carrying on the example of assembly machine crafting speed, we should be able to define our own research effect. I'd expect it to look something like the following (of course, this could take any form - the functionality is more important than the syntax/format/style/etc I use here):

Code: Select all

type = "technology-effect",
name = "assembly-machine-crafting-speed",
modifier_target = "assembly-machine-3"
modification = {
    entity.crafting_speed = entity.crafting_speed + modifier
}

Then, in the technology file, you can use:

Code: Select all

effects =
    {
      {
        type = "assembly-machine-crafting-speed",
        modifier = 0.1
      }
    }

and would work as though "assembly-machine-crafting-speed" was something that was defined in the base game (similar to the first example in "the problem").

My Proposal v2

Another solution to this problem is that instead of allowing us to define the research effect, research effects for all the attributes of all entities are added to the base game. I expect this will not be favored because it doesn't address items not part of the base game, and most of them would be unused.

Closing Thoughts

I've used terms like "all", "any", or "custom" here a lot. I'd be quite happy if the research effects supported in modding was just expanded in some way, even if it is not extensive and allow us to go crazy with infinite things.
I understand that the ability to have everything infinitely upgrade-able may not be in the spirit of Factorio, but that is why I think it is apt for a modding/scripting capability - not as part of the base game by default.
There may be some nice simple solution for the second case I describe in "the problem" that I'm just not aware of. If there is, let me know. If there isn't, feel free to let me know all the same; I'd be interested to know if modders have run into this situation themselves.
I think it would be interesting to have a segment of a Factorio Friday Facts on how the current built-in research effects work 'behind the scenes' (unless it has already been done, in which case, I wouldn't mind a link to it).
I'm aware of a request for range research which was put into the "Won't implement" category (which, if I am honest, is where I expect this request will go to as well - but I like to think that it won't). However, I think this suggestion is quite different in what it asks for. I am aware that there may be a lot of technical issues with this (as noted in the response to the linked request), but I think that this could be reasonably achieved for a fair amount of cases without it breaking things.

[Rseding91]

The main limiting factors in adding any new research effect are:

• 1. Any new effect needs to be stored in the map somewhere - which means map version logic
  2. Any new effect needs to be accounted for in every place that should be using it - slowing each area down very slightly
  3. Any new effect needs someone to write the code for it

The example you gave with assembling machine speed mostly impacts point #2 where it would be measurably slower if each assembling machine/furnace had to go also check the speed modifier. If it was per-machine that would mean it has to be stored on the machine itself which increases save file size and memory footprint of the machine.

That means that lua-defined modifiers simply aren't going to happen and C++ support for new modifiers need good arguments to be added.

[mrvn]

The main limiting factors in adding any new research effect are:

• 1. Any new effect needs to be stored in the map somewhere - which means map version logic
  2. Any new effect needs to be accounted for in every place that should be using it - slowing each area down very slightly
  3. Any new effect needs someone to write the code for it

The example you gave with assembling machine speed mostly impacts point #2 where it would be measurably slower if each assembling machine/furnace had to go also check the speed modifier. If it was per-machine that would mean it has to be stored on the machine itself which increases save file size and memory footprint of the machine.

That means that lua-defined modifiers simply aren't going to happen and C++ support for new modifiers need good arguments to be added.

But don't you already have that with beacons? The speed of every assembler can be different. And afaik you compute the speed only when the beacons change and not every tick.

[Rseding91]

But don't you already have that with beacons? The speed of every assembler can be different. And afaik you compute the speed only when the beacons change and not every tick.

Yes, it's stored as a product of the beacon and the modules in it + the modules in the machine. Those modules don't change runtime - just the count of them does - which is still saved in the map.

[mrvn]

So how about having a hidden beacon in the map (on every surface) spanning all of the map and the research effects change the modules in that beacon.

Or would a beacon with reach 2.000.000.000 kill the game engine?

[Rseding91]

So how about having a hidden beacon in the map (on every surface) spanning all of the map and the research effects change the modules in that beacon.

Or would a beacon with reach 2.000.000.000 kill the game engine?

It would murder game performance.

[bobingabout]

So how about having a hidden beacon in the map (on every surface) spanning all of the map and the research effects change the modules in that beacon.

Or would a beacon with reach 2.000.000.000 kill the game engine?

It would murder game performance.

Did you try it, or use your matrix voodoo powers to read the code to come up with that answer?

[Bilka]

Did you try it, or use your matrix voodoo powers to read the code to come up with that answer? :P

Whenever an effectreceiver is setup or has a module transferred into it, the list of effect sources (beacons + modules) has to be rebuilt. This is done by searching for beacons around the entity which have the entity in their supply area. So, the search box increases if beacons have a bigger supply area, because they could be farther away from the entity and still affect it. If a beacon had a supply area that supplies the entire surface, all entities would then have to search the entire surface for beacons when they have to rebuild their effect sources. An entity search is noticeably slow when you search the entire surface (you can see the game freeze for seconds depending on the size of the world).

[bobingabout]

Did you try it, or use your matrix voodoo powers to read the code to come up with that answer?

Whenever an effectreceiver is setup or has a module transferred into it, the list of effect sources (beacons + modules) has to be rebuilt. This is done by searching for beacons around the entity which have the entity in their supply area. So, the search box increases if beacons have a bigger supply area, because they could be farther away from the entity and still affect it. If a beacon had a supply area that supplies the entire surface, all entities would then have to search the entire surface for beacons when they have to rebuild their effect sources. An entity search is noticeably slow when you search the entire surface (you can see the game freeze for seconds depending on the size of the world).

of course.

[mrudat]

Perhaps a generalish solution that might not kill performance; allow modifying numeric values on entity prototypes for an entire force?

Probably simplest to cache the updated prototype in a per-force clone EntityPrototype (where different from base).

The game would probably pause as every effected entity for a force has its stats upgraded, but probably wouldn't otherwise greatly impact on-going performance.

This is obviously non-trivial to implement, but should remove lots of existing per-force special cases; carry capacity; stack size; damage/range; ...

[Rseding91]

Perhaps a generalish solution that might not kill performance; allow modifying numeric values on entity prototypes for an entire force?

Probably simplest to cache the updated prototype in a per-force clone EntityPrototype (where different from base).

The game would probably pause as every effected entity for a force has its stats upgraded, but probably wouldn't otherwise greatly impact on-going performance.

This is obviously non-trivial to implement, but should remove lots of existing per-force special cases; carry capacity; stack size; damage/range; ...

Mutating prototypes runtime is not allowed and would break virtually everything about how the game works.

[mrvn]

Did you try it, or use your matrix voodoo powers to read the code to come up with that answer?

Whenever an effectreceiver is setup or has a module transferred into it, the list of effect sources (beacons + modules) has to be rebuilt. This is done by searching for beacons around the entity which have the entity in their supply area. So, the search box increases if beacons have a bigger supply area, because they could be farther away from the entity and still affect it. If a beacon had a supply area that supplies the entire surface, all entities would then have to search the entire surface for beacons when they have to rebuild their effect sources. An entity search is noticeably slow when you search the entire surface (you can see the game freeze for seconds depending on the size of the world).

Shouldn't be too hard to add the magic "Custom Research Effects" beacon to be always found when an entity updates its effect sources. So on top of effect sources searched it simply adds one more. That would only cost O(1) and only when an effectreceiver is setup or has a module transferred into it.

I would suggest making this 2 magic beacons. One force wide beacon and one per surface beacon. That way mods could make custom research effects or alter the effects for different surfaces.

[Reika]

Just putting this forward: Something like this would have made my Turret Range Boost research much simpler to implement as well.

[demongo]

yeah im trying to make inf research techs to improve on my deepmine mod for mega late game but im hitting this same issue...
https://mods.factorio.com/mod/DeepMineALT/discussion
trying to add a inf tech to improve my deepmines productivity... but due to the way mine is done the beacon method the other deepmine author uses does not work that or im messing up somewhere...