[1.1.85] Icon dump truncates names with `:`

This subforum contains all the issues which we already resolved.
ilbJanissary
Inserter
Inserter
Posts: 40
Joined: Tue May 28, 2019 12:34 pm
Contact:

[1.1.85] Icon dump truncates names with `:`

Post by ilbJanissary »

1. Attempting to use the --dump-icon-sprites command with the mod "Exotic Industries: Official Modpack" (exotic-industries-modpack) and its required dependencies, to get icons from this mod for use in FactorioLab.
2. The icons for many recipes appear to be generated but use names that are truncated compared to the internal names. It seems that the icon names truncate all characters including & after the first colon (:) character in the name. So, the recipe "ei_landfill:sand" is generated as /recipe/ei_landfill.png, and "ei_solid-fuel:residual-oil" is generated as /recipe/ei_solid-fuel.png. If this name conflicts with another entity's name (e.g. if ei_solid-fuel also exists on its own), then only one icon gets created.
3. I would expect the icons to get created using the same names as the internal names generated from the raw data dump. Since ":" is not a prohibited character as far as I know, I would not expect any issues with these.
4. This appears to happen on every dump for all recipes with names including ":" characters. I would guess it also occurs with other prototype types, but this largely seems to occur with recipes in Exotic Industries.
5. To reproduce, install and enable exotic-industries-modpack, and run factorio.exe --dump-icon-sprites, then check the output.

Example raw recipe data from the data raw dump is below. Generated icon is attached. No recipe named "ei_solid-fuel" exists, so this appears to be the right icon, just under a different name.

Code: Select all

    "ei_solid-fuel:residual-oil": 
    {
      "name": "ei_solid-fuel:residual-oil",
      "type": "recipe",
      "category": "chemistry",
      "energy_required": 2,
      "ingredients": 
      [
        
        {
          "type": "fluid",
          "name": "ei_residual-oil",
          "amount": 30
        }
      ],
      "results": 
      [
        
        {
          "type": "item",
          "name": "solid-fuel",
          "amount": 1
        }
      ],
      "always_show_made_in": true,
      "enabled": false,
      "icon": "__exotic-industries-graphics__/graphics/other/residual-oil_solid-fuel.png",
      "icon_size": 64,
      "subgroup": "fluid-recipes",
      "order": "b[fluid-chemistry]-c[solid-fuel:residual-oil]"
    },
Attachments
ei_solid-fuel.png
ei_solid-fuel.png (5.39 KiB) Viewed 3358 times
Rseding91
Factorio Staff
Factorio Staff
Posts: 14657
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by Rseding91 »

Thanks for the report. There are a few options here but the easiest one is for mods to not use that character in prototype names if they want to work with icon dumping. Windows does not allow you to create a file with the ":" character in the filename. The underlying filesystem might 'accept' it; but the general world is going to complain *real* hard about a file with that name.

1. Do nothing
2. The game generates a 'best-guess' file name that won't 1-1 match prototype names if they use special characters
3. The game errors if it encounters invalid file names letting you know it can't do the operation
If you want to get ahold of me I'm almost always on Discord.
ilbJanissary
Inserter
Inserter
Posts: 40
Joined: Tue May 28, 2019 12:34 pm
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by ilbJanissary »

That makes sense. I obviously didn't author the mod, so I could suggest to the author not to use that character but in general that seems like it might not be the best approach.

Could the dump swap any disallowed characters for "_" underscore characters, or something like that? Any tool reading the data could also scan the prototype names and swap the list of disallowed characters for "_" characters, so that it would at least know what name to expect the image export to use. This wouldn't work if the mod has separate entities using the name "this:name" AND "this_name", but at that point, it feels like self-inflicted sabotage.

If you really want to over-engineer it, you could also output a JSON file with a map of prototype names : file names, and then you'd have free reign to use whatever mechanism you want to generate valid unique file names - even GUIDs. That would probably be the *right* way to do it, but it also feels like overkill.
User avatar
atomizer
Fast Inserter
Fast Inserter
Posts: 119
Joined: Sat Sep 22, 2018 3:18 pm
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by atomizer »

ilbJanissary wrote: Thu Jul 06, 2023 2:32 pmIf you really want to over-engineer it, you could also output a JSON file with a map of prototype names : file names, and then you'd have free reign to use whatever mechanism you want to generate valid unique file names - even GUIDs. That would probably be the *right* way to do it, but it also feels like overkill.
Using a hash of prototype name would avoid having to output the mapping, and should be trivial to implement both in Factorio and in tools.
Rseding91
Factorio Staff
Factorio Staff
Posts: 14657
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by Rseding91 »

atomizer wrote: Thu Jul 06, 2023 7:17 pm
ilbJanissary wrote: Thu Jul 06, 2023 2:32 pmIf you really want to over-engineer it, you could also output a JSON file with a map of prototype names : file names, and then you'd have free reign to use whatever mechanism you want to generate valid unique file names - even GUIDs. That would probably be the *right* way to do it, but it also feels like overkill.
Using a hash of prototype name would avoid having to output the mapping, and should be trivial to implement both in Factorio and in tools.
Unless I misunderstand; using a hash would require all external tools to know the names ahead of time so they can figure out what hash-name-file is the one they are looking for.

At that point it's still a mapping just done through every external tool. It would also still (eventually) run into name collisions due to the nature of hashing.
If you want to get ahold of me I'm almost always on Discord.
User avatar
atomizer
Fast Inserter
Fast Inserter
Posts: 119
Joined: Sat Sep 22, 2018 3:18 pm
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by atomizer »

Rseding91 wrote: Thu Jul 06, 2023 8:46 pmUnless I misunderstand; using a hash would require all external tools to know the names ahead of time so they can figure out what hash-name-file is the one they are looking for.
If the tool doesn't know what it wants, does it really matter what the file is called? It already doesn't match the exact prototype name, see the first post.
Rseding91 wrote: Thu Jul 06, 2023 8:46 pm It would also still (eventually) run into name collisions due to the nature of hashing.
A collision on short strings of text would be a big discovery, I would love to see it.
SoShootMe
Filter Inserter
Filter Inserter
Posts: 517
Joined: Mon Aug 03, 2020 4:16 pm
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by SoShootMe »

atomizer wrote: Fri Jul 07, 2023 6:25 am A collision on short strings of text would be a big discovery, I would love to see it.
Hardly ;). You mean a collision using an algorithm such as one of the SHA family - which would be a good choice to meet the "trivial" requirement and reliable in practice.

Another reliable alternative would be to escape characters of a prototype name outside a defined "safe" set (similar to URL encoding). Like hash-based names, this would avoid the need for a file defining the mapping. Unlike hashing, it would also allow mapping from filename to prototype name, but I can't think of any need for that.

If (as is the case) a direct mapping is impossible in general, one must choose between not working in all cases, or implementing something in both Factorio and tools. Hash-based names using eg SHA-256 are probably the simplest both in terms of defining the interface (ie Factorio/tool requirements) and to implement.
ilbJanissary
Inserter
Inserter
Posts: 40
Joined: Tue May 28, 2019 12:34 pm
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by ilbJanissary »

Is there any further information I can provide to get traction on this issue? The author of Exotic Industries is holding off on making any changes pending any possible fix to the dump command itself, since their mod is unlikely to be the only one affected by this issue. I think if we had a mapping of internal id -> file name as part of the icon dump so that we could use safe file names for each sprite, it would be ideal, although I recognize that will require some effort to build.

(https://mods.factorio.com/mod/exotic-in ... e325bb0949)
ixu
Fast Inserter
Fast Inserter
Posts: 107
Joined: Sun May 29, 2016 6:18 pm
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by ixu »

Another suggestion would be to simply mention this in the documentation. It's a bit unsatisfactory, but then at least mod authors know about it and can avoid/fix the problem.
Mods: ingteb, ixuClock, ixuAutoSave, NoCheating, hardCrafting (Maintainer) and more
Qon
Smart Inserter
Smart Inserter
Posts: 2164
Joined: Thu Mar 17, 2016 6:27 am
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by Qon »

Rseding91 wrote: Mon Jun 19, 2023 9:10 pm There are a few options here but the easiest one is for mods to not use that character in prototype names if they want to work with icon dumping. Windows does not allow you to create a file with the ":" character in the filename. The underlying filesystem might 'accept' it; but the general world is going to complain *real* hard about a file with that name.
I use : and some other special characters in prototype names. I wasn't made aware through documentation that "icon dumping" was a tool that my names may or may not be compatible with, I didn't choose to be incompatible. I wasn't made aware that prototype names would end up being file names. And the documentation doesn't tell me which characters are not available on every operating system that Factorio runs on.

And prototype names need a migration to change to not corrupt (remove items) savefiles, which is a lot of work if you are asking that from every mod that doesn't follow a new stricter prototype standard added after the mod was written, by many mod makers that don't even know how to do it. And the mod writers are following spec, unlike the the dump tool.

The documentation just tells me that dot "." is not allowed. Yes, that means everything else in unicode is allowed, including newlines, control sequences, noncharacters and private use characters.

It's the dumping tool that doesn't follow the the prototype spec. Something trivial like URL encoding could have 1 to 1 mapping and be easy for external tools to handle.
My mods: Capsule Ammo | HandyHands - Automatic handcrafting | ChunkyChunks - Configurable Gridlines
Some other creations: Combinassembly Language GitHub w instructions and link to run it in your browser | 0~drain Laser
ilbJanissary
Inserter
Inserter
Posts: 40
Joined: Tue May 28, 2019 12:34 pm
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by ilbJanissary »

URL encoding also seems like a great way to address this issue.
bits-orio
Burner Inserter
Burner Inserter
Posts: 11
Joined: Wed Feb 28, 2024 6:08 pm
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by bits-orio »

Rseding91 wrote: Mon Jun 19, 2023 9:10 pm Thanks for the report. There are a few options here but the easiest one is for mods to not use that character in prototype names if they want to work with icon dumping. Windows does not allow you to create a file with the ":" character in the filename. The underlying filesystem might 'accept' it; but the general world is going to complain *real* hard about a file with that name.

1. Do nothing
2. The game generates a 'best-guess' file name that won't 1-1 match prototype names if they use special characters
3. The game errors if it encounters invalid file names letting you know it can't do the operation
Has there been a resolution to this issue yet?
Reminds me of that spider-man meme, where everyone is pointing fingers at everyone.
Rseding91
Factorio Staff
Factorio Staff
Posts: 14657
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by Rseding91 »

bits-orio wrote: Mon Jul 29, 2024 4:41 pm Has there been a resolution to this issue yet?
Reminds me of that spider-man meme, where everyone is pointing fingers at everyone.
There has not. So far I've been rolling with option #1 because all other options seem equally bad.
If you want to get ahold of me I'm almost always on Discord.
Bilka
Factorio Staff
Factorio Staff
Posts: 3310
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by Bilka »

In 2.0, prototype names will be limited to alphanumeric characters, dashes and underscores: 114525

That will fix this issue.
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.
ilbJanissary
Inserter
Inserter
Posts: 40
Joined: Tue May 28, 2019 12:34 pm
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by ilbJanissary »

Awesome, thank you for the update!
plexpt
Long Handed Inserter
Long Handed Inserter
Posts: 94
Joined: Mon Feb 07, 2022 2:32 am
Contact:

Re: [1.1.85] Icon dump truncates names with `:`

Post by plexpt »

Bilka wrote: Wed Sep 11, 2024 2:52 pm In 2.0, prototype names will be limited to alphanumeric characters, dashes and underscores: 114525

That will fix this issue.
new issue

viewtopic.php?f=28&t=120904
Post Reply

Return to “Resolved Problems and Bugs”