[1.1.92] data.raw JSON dump serializes empty arrays wrong

[BrainGamer_]

In JSON array types are expected to be something like this

Code: Select all

[ value1, value2, ... ]

and an empty array should be

Code: Select all

[]

but when getting the data.raw JSON dump by using the --dump-data flag fields that should be arrays and are empty are serialized as

Code: Select all

{}

which implies the field to be a map type of sorts and not an array.

[Rseding91]

Thanks for the report. Factorio doesn't actually support array types in data.raw. All types in data.raw are dictionaries with anything looking like an array being numeric-dictionaries.

{ value1, value2, ... } is actually short-hand for { 1=value1, 2=value2, 3=... }

[BrainGamer_]

All types in data.raw are dictionaries with anything looking like an array being numeric-dictionaries

Interesting to know! The prototype docs just specified some types to be array[union] which is how I ended up at the assumption that they are arrays and that tripped when trying to deserialize "{}" as an array.

[Rseding91]

Interesting to know! The prototype docs just specified some types to be array[union] which is how I ended up at the assumption that they are arrays and that tripped when trying to deserialize "{}" as an array.

We use the term "array" to mean "you don't need to care about the key values" but Lua itself only supports dictionaries (actually hash-maps under the hood).

[BrainGamer_]

We use the term "array" to mean "you don't need to care about the key values"

Makes sense. I still need to care about it in some way tho since the JSON changes "type" from being an array to a map when the "array" (or internal hash-map) is empty

I found a workaround but it is pretty hacky..

[moon69]

Why is it inconsistent though?

Code: Select all

      "flags": 
      [
        "placeable-neutral",
        "player-creation"
      ],

vs

Code: Select all

      "flags": 
      {},

Please can we have empty flags in JSON dump as follows?

Code: Select all

 "flags": 
      [],

[Rseding91]

I went and looked into the "flags" case and found something interesting. Because Lua has no such concept as "dictionary" vs "array" when we parse it from Lua into C++ we have no idea what the type is when it's empty. To Lua, an empty table is an empty table - it's not a dictionary or array. So we have to pick one and when the Lua table is empty it defaults to "empty dictionary" on the C++ side. Along comes the "convert to Json" and it sees an empty dictionary and writes out {}.

To summarize: I don't see this ever changing. Unless Lua gets built in support for JSON (which i'm going to go out on a limb and say it never will).

[Rseding91]

Perhaps a more realistic solution would be writing everything as dictionaries when converting to JSON since that's actually closer to what Lua uses.

[BrainGamer_]

Code: Select all

"flags": 
      [
        "placeable-neutral",
        "player-creation"
      ],

The fun thing is that this is not the only valid representation for this! Since Lua basically does everything with HashMaps like Rseding said you can have the following:

Code: Select all

"flags": 
      {
        1: "placeable-neutral",
        2: "player-creation"
      }

or even better:

Code: Select all

"flags": 
      {
        "1": "placeable-neutral",
        "2": "player-creation"
      }

My solution to all of this in the end was to write a completely custom serde deserializer for this special type: https://github.com/fgardt/factorio-scan ... #L107-L154