[1.1.85] Icon dump truncates names with `:`

[ilbJanissary]

1. Attempting to use the --dump-icon-sprites command with the mod "Exotic Industries: Official Modpack" (exotic-industries-modpack) and its required dependencies, to get icons from this mod for use in FactorioLab.
2. The icons for many recipes appear to be generated but use names that are truncated compared to the internal names. It seems that the icon names truncate all characters including & after the first colon (:) character in the name. So, the recipe "ei_landfill:sand" is generated as /recipe/ei_landfill.png, and "ei_solid-fuel:residual-oil" is generated as /recipe/ei_solid-fuel.png. If this name conflicts with another entity's name (e.g. if ei_solid-fuel also exists on its own), then only one icon gets created.
3. I would expect the icons to get created using the same names as the internal names generated from the raw data dump. Since ":" is not a prohibited character as far as I know, I would not expect any issues with these.
4. This appears to happen on every dump for all recipes with names including ":" characters. I would guess it also occurs with other prototype types, but this largely seems to occur with recipes in Exotic Industries.
5. To reproduce, install and enable exotic-industries-modpack, and run factorio.exe --dump-icon-sprites, then check the output.

Example raw recipe data from the data raw dump is below. Generated icon is attached. No recipe named "ei_solid-fuel" exists, so this appears to be the right icon, just under a different name.

Code: Select all

    "ei_solid-fuel:residual-oil": 
    {
      "name": "ei_solid-fuel:residual-oil",
      "type": "recipe",
      "category": "chemistry",
      "energy_required": 2,
      "ingredients": 
      [
        
        {
          "type": "fluid",
          "name": "ei_residual-oil",
          "amount": 30
        }
      ],
      "results": 
      [
        
        {
          "type": "item",
          "name": "solid-fuel",
          "amount": 1
        }
      ],
      "always_show_made_in": true,
      "enabled": false,
      "icon": "__exotic-industries-graphics__/graphics/other/residual-oil_solid-fuel.png",
      "icon_size": 64,
      "subgroup": "fluid-recipes",
      "order": "b[fluid-chemistry]-c[solid-fuel:residual-oil]"
    },

[Rseding91]

Thanks for the report. There are a few options here but the easiest one is for mods to not use that character in prototype names if they want to work with icon dumping. Windows does not allow you to create a file with the ":" character in the filename. The underlying filesystem might 'accept' it; but the general world is going to complain *real* hard about a file with that name.

1. Do nothing
2. The game generates a 'best-guess' file name that won't 1-1 match prototype names if they use special characters
3. The game errors if it encounters invalid file names letting you know it can't do the operation

[ilbJanissary]

That makes sense. I obviously didn't author the mod, so I could suggest to the author not to use that character but in general that seems like it might not be the best approach.

Could the dump swap any disallowed characters for "_" underscore characters, or something like that? Any tool reading the data could also scan the prototype names and swap the list of disallowed characters for "_" characters, so that it would at least know what name to expect the image export to use. This wouldn't work if the mod has separate entities using the name "this:name" AND "this_name", but at that point, it feels like self-inflicted sabotage.

If you really want to over-engineer it, you could also output a JSON file with a map of prototype names : file names, and then you'd have free reign to use whatever mechanism you want to generate valid unique file names - even GUIDs. That would probably be the *right* way to do it, but it also feels like overkill.

[atomizer]

If you really want to over-engineer it, you could also output a JSON file with a map of prototype names : file names, and then you'd have free reign to use whatever mechanism you want to generate valid unique file names - even GUIDs. That would probably be the *right* way to do it, but it also feels like overkill.

Using a hash of prototype name would avoid having to output the mapping, and should be trivial to implement both in Factorio and in tools.

[Rseding91]

If you really want to over-engineer it, you could also output a JSON file with a map of prototype names : file names, and then you'd have free reign to use whatever mechanism you want to generate valid unique file names - even GUIDs. That would probably be the *right* way to do it, but it also feels like overkill.

Using a hash of prototype name would avoid having to output the mapping, and should be trivial to implement both in Factorio and in tools.

Unless I misunderstand; using a hash would require all external tools to know the names ahead of time so they can figure out what hash-name-file is the one they are looking for.

At that point it's still a mapping just done through every external tool. It would also still (eventually) run into name collisions due to the nature of hashing.

[atomizer]

Unless I misunderstand; using a hash would require all external tools to know the names ahead of time so they can figure out what hash-name-file is the one they are looking for.

If the tool doesn't know what it wants, does it really matter what the file is called? It already doesn't match the exact prototype name, see the first post.

It would also still (eventually) run into name collisions due to the nature of hashing.

A collision on short strings of text would be a big discovery, I would love to see it.

[SoShootMe]

A collision on short strings of text would be a big discovery, I would love to see it.

Hardly . You mean a collision using an algorithm such as one of the SHA family - which would be a good choice to meet the "trivial" requirement and reliable in practice.

Another reliable alternative would be to escape characters of a prototype name outside a defined "safe" set (similar to URL encoding). Like hash-based names, this would avoid the need for a file defining the mapping. Unlike hashing, it would also allow mapping from filename to prototype name, but I can't think of any need for that.

If (as is the case) a direct mapping is impossible in general, one must choose between not working in all cases, or implementing something in both Factorio and tools. Hash-based names using eg SHA-256 are probably the simplest both in terms of defining the interface (ie Factorio/tool requirements) and to implement.

[ilbJanissary]

Is there any further information I can provide to get traction on this issue? The author of Exotic Industries is holding off on making any changes pending any possible fix to the dump command itself, since their mod is unlikely to be the only one affected by this issue. I think if we had a mapping of internal id -> file name as part of the icon dump so that we could use safe file names for each sprite, it would be ideal, although I recognize that will require some effort to build.

(https://mods.factorio.com/mod/exotic-in ... e325bb0949)

[ixu]

Another suggestion would be to simply mention this in the documentation. It's a bit unsatisfactory, but then at least mod authors know about it and can avoid/fix the problem.

[Qon]

There are a few options here but the easiest one is for mods to not use that character in prototype names if they want to work with icon dumping. Windows does not allow you to create a file with the ":" character in the filename. The underlying filesystem might 'accept' it; but the general world is going to complain *real* hard about a file with that name.

I use : and some other special characters in prototype names. I wasn't made aware through documentation that "icon dumping" was a tool that my names may or may not be compatible with, I didn't choose to be incompatible. I wasn't made aware that prototype names would end up being file names. And the documentation doesn't tell me which characters are not available on every operating system that Factorio runs on.

And prototype names need a migration to change to not corrupt (remove items) savefiles, which is a lot of work if you are asking that from every mod that doesn't follow a new stricter prototype standard added after the mod was written, by many mod makers that don't even know how to do it. And the mod writers are following spec, unlike the the dump tool.

The documentation just tells me that dot "." is not allowed. Yes, that means everything else in unicode is allowed, including newlines, control sequences, noncharacters and private use characters.

It's the dumping tool that doesn't follow the the prototype spec. Something trivial like URL encoding could have 1 to 1 mapping and be easy for external tools to handle.

[ilbJanissary]

URL encoding also seems like a great way to address this issue.

[bits-orio]

Thanks for the report. There are a few options here but the easiest one is for mods to not use that character in prototype names if they want to work with icon dumping. Windows does not allow you to create a file with the ":" character in the filename. The underlying filesystem might 'accept' it; but the general world is going to complain *real* hard about a file with that name.

1. Do nothing
2. The game generates a 'best-guess' file name that won't 1-1 match prototype names if they use special characters
3. The game errors if it encounters invalid file names letting you know it can't do the operation

Has there been a resolution to this issue yet?
Reminds me of that spider-man meme, where everyone is pointing fingers at everyone.

[Rseding91]

Has there been a resolution to this issue yet?
Reminds me of that spider-man meme, where everyone is pointing fingers at everyone.

There has not. So far I've been rolling with option #1 because all other options seem equally bad.

[Bilka]

In 2.0, prototype names will be limited to alphanumeric characters, dashes and underscores: 114525

That will fix this issue.

[ilbJanissary]

Awesome, thank you for the update!

[plexpt]

In 2.0, prototype names will be limited to alphanumeric characters, dashes and underscores: 114525

That will fix this issue.

new issue

viewtopic.php?f=28&t=120904