Lua prototype definitions schema

Place to post guides, observations, things related to modding that are not mods themselves.
jcranmer
Long Handed Inserter
Long Handed Inserter
Posts: 90
Joined: Wed Jun 29, 2016 9:59 pm
Contact:

Lua prototype definitions schema

Post by jcranmer »

I'm not entirely sure if this is the best forum for this, but it does seem to be relevant to people here, so I'll post here instead of the Tools forum.

I was working on a Python library to help me build Factorio tools (a Factorio tool meta-tool if you will :D ). Loading the information from the Lua files directly, including supporting mods properly, took some time, but I did get it working [1] (including mods downloaded as zip files or specified as symlinks to another directory!). However, one challenge is that the definitions of the prototypes in the raw data table aren't well-specified to a public schema. Since trying to track down where the files might be located to look at examples quickly becomes a pain, I decided to first dump the entire Lua data table out in a JSON format. After staring at that for a bit, I realized what I needed to do.

I'm in the midst of writing a library that can load a static schema file, defined in JSON (the WIP is provided below). This schema file most importantly defines the inheritance order of the various types (e.g., an accumulator is an EntityWithHealth is an Entity). A Python module loads this schema and turns it into Python classes that you can instantiate [2] with the Lua objects. You can even get useful results if you run help on the module, e.g.:
example help output
The WIP I have for the schema is below. I copied it mostly from the existing wiki pages, but there's a lot missing. The latter two sections are entirely guesswork on my part as to the hierarchy, and so I have 0 confidence in them. I'm also missing several tables entirely (beam, blueprint, combat-robot-count, curved-rail, custom-input, deconstructible-tile-proxy, deconstruction-item, electric-energy-interface, entity-ghost, fire, fluid, item-request-proxy, leaf-particle, module-category, particle-source, rail-planner, rocket-silo-rocket, rocket-silo-rocket-shadow, simple-entity, smoke-with-trigger, straight-rail, stream, tile-ghost, utility-sprites, virtual-signal). The optional/mandatory details according to the wiki seem wrong, given that several items seem to lack them at times.I do hope to validate the types and default values, but that is somewhat less important to me than getting the entire hierarchy accurate first.
Prototype schema definition
If any developers are reading this thread, can you validate my facts? I could give you the Python help tree if it helps (or even the module code, though you probably have to be on Linux to get it to work). It would be even more wonderful if this kind of information could be produced automatically (like the LUA documentation), but I'm guessing that the code isn't so nicely structured as to give this information with automated documentation tools.

Thoughts/questions/comments/concerns?

[1] Turns out that Lua's custom module loader requires that the data in question be a function, not merely something callable. So I had to write a Lua function that returns a lua function that calls a function passed into it and call the function from Python to get the right data type. Aren't proxies fun?
[2] Yes, this is dynamically defining classes in a module. It's a lot easier to do this in Python than when I had to write out the constant pools for Java interfaces...
User avatar
Adil
Filter Inserter
Filter Inserter
Posts: 945
Joined: Fri Aug 15, 2014 8:36 pm
Contact:

Re: Lua prototype definitions schema

Post by Adil »

jcranmer wrote:Since trying to track down where the files might be located to look at examples quickly becomes a pain
Wait a minute, you've made a data loader without finding out 'prototypes' folders in "data/base" \ "data/core" ?

This documentation has been for basically since the modding became a thing, but guess it takes time.
If I'd need one, I'd only hoped programmatically to build it myself from the contents of data.raw.
I do mods. Modding wiki is friend, it teaches how to mod. Api docs is friend too...
I also update mods, some of them even work.
Recently I did a mod tutorial.
jcranmer
Long Handed Inserter
Long Handed Inserter
Posts: 90
Joined: Wed Jun 29, 2016 9:59 pm
Contact:

Re: Lua prototype definitions schema

Post by jcranmer »

Adil wrote:
jcranmer wrote:Since trying to track down where the files might be located to look at examples quickly becomes a pain
Wait a minute, you've made a data loader without finding out 'prototypes' folders in "data/base" \ "data/core" ?
The problem is when you have a datatype failure, like expecting recipe to be well-behaved, but it turns out some specific prototype for the recipe is different in class. (The fluid stuff comes to mind, that's inconsistent in handling).

Code: Select all

class FactorioRecipe(object):
    def __init__(self, data, lua_recipe):
        self.data = data
        self.lua = lua_recipe

    def _map_array(self, arr):
        return dict(map(lambda e: (e[1], e[2])
                if e.keys().next() == 1 else (e.name, e.amount),
                arr.values()))

    def get_results(self):
        '''Return a dictionary of result name -> result count.'''
        if self.lua.result:
            count = self.lua.count if self.lua.count else 1
            results = dict()
            results[self.lua.result] = count
        else:
            results = self._map_array(self.lua.results)
        return results

    def get_ingredients(self):
        '''Return a dictionary of ingredient name -> ingredient count.'''
        return self._map_array(self.lua.ingredients)

    def get_crafttime(self):
        '''Return crafting time in seconds.'''
        return self.lua.energy_required if self.lua.energy_required else 0.5
(That's my Python code pre-auto-schema generation). Yeah, it's somewhere in __base__/prototypes/recipe/*.lua, but which one? It doesn't help that the repr for LuaTable is a singularly unhelpful <Lua table at 0x2d3b4b0>.
User avatar
aubergine18
Smart Inserter
Smart Inserter
Posts: 1264
Joined: Fri Jul 22, 2016 8:51 pm
Contact:

Re: Lua prototype definitions schema

Post by aubergine18 »

The wiki is very outdated, the most up-to-date reference is the online API docs: http://lua-api.factorio.com/latest/index.html
Better forum search for modders: Enclose your search term in quotes, eg. "font_color" or "custom-input" - it prevents the forum search from splitting on hypens and underscores, resulting in much more accurate results.
User avatar
Adil
Filter Inserter
Filter Inserter
Posts: 945
Joined: Fri Aug 15, 2014 8:36 pm
Contact:

Re: Lua prototype definitions schema

Post by Adil »

Those are API, not the prototype reference, you know.
While concepts do hold a pair of info bits related to prototypes, it is focused on scripting, not data. Thus wiki, as obsolete as it is, is most up to date source on the matter.
I do mods. Modding wiki is friend, it teaches how to mod. Api docs is friend too...
I also update mods, some of them even work.
Recently I did a mod tutorial.
User avatar
aubergine18
Smart Inserter
Smart Inserter
Posts: 1264
Joined: Fri Jul 22, 2016 8:51 pm
Contact:

Re: Lua prototype definitions schema

Post by aubergine18 »

@jcranmer did you make any progress on this? I'm currently wading through the prototypes to try and identify all the properties and what they do.
Better forum search for modders: Enclose your search term in quotes, eg. "font_color" or "custom-input" - it prevents the forum search from splitting on hypens and underscores, resulting in much more accurate results.
jcranmer
Long Handed Inserter
Long Handed Inserter
Posts: 90
Joined: Wed Jun 29, 2016 9:59 pm
Contact:

Re: Lua prototype definitions schema

Post by jcranmer »

aubergine18 wrote:@jcranmer did you make any progress on this? I'm currently wading through the prototypes to try and identify all the properties and what they do.
https://github.com/jcranmer/factorio-to ... chema.json is the up-to-date version of the schema, as often as I update it, although a few complex stuff requires more python logic to parse (the recipe ingredients/results come to mind here). There's some more data I've added locally that's not in the public repo (mostly entity pictures, which I was using to build a blueprint string viewer)
User avatar
aubergine18
Smart Inserter
Smart Inserter
Posts: 1264
Joined: Fri Jul 22, 2016 8:51 pm
Contact:

Re: Lua prototype definitions schema

Post by aubergine18 »

Looking good, already helped me work out several bits I was stumped on!
Better forum search for modders: Enclose your search term in quotes, eg. "font_color" or "custom-input" - it prevents the forum search from splitting on hypens and underscores, resulting in much more accurate results.
Post Reply

Return to “Modding discussion”