entity.id, a unique, persistent ID

ljdp
Long Handed Inserter
Long Handed Inserter
Posts: 96
Joined: Wed Jul 01, 2015 12:33 am
Contact:

entity.id, a unique, persistent ID

Post by ljdp »

Add a unique and persistent identifier for entities.

Wyrm
Long Handed Inserter
Long Handed Inserter
Posts: 55
Joined: Fri Jan 30, 2015 3:56 am
Contact:

Re: entity.id, a unique, persistent ID

Post by Wyrm »

This may not be necessary if entity references in the implementation do not change from load to load — in which case, entity.id already exists: this_entity.id == this_entity. Otherwise, it would be a great addition.

Choumiko
Smart Inserter
Smart Inserter
Posts: 1352
Joined: Fri Mar 21, 2014 10:51 pm
Contact:

Re: entity.id, a unique, persistent ID

Post by Choumiko »

As long as you don't use it as a key , entity == entity 2 works.
Still, I'd like it at least for LuaPlayer, since player_index and player.name might change in MP games.

Rseding91
Factorio Staff
Factorio Staff
Posts: 11923
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: entity.id, a unique, persistent ID

Post by Rseding91 »

Choumiko wrote:As long as you don't use it as a key , entity == entity 2 works.
Still, I'd like it at least for LuaPlayer, since player_index and player.name might change in MP games.
You can do player == player2 to check if they're the same player.

You can do anything == anything - inventory references, entity references, GUI references - they all work with the == operator.
If you want to get ahold of me I'm almost always on Discord.

Wyrm
Long Handed Inserter
Long Handed Inserter
Posts: 55
Joined: Fri Jan 30, 2015 3:56 am
Contact:

Re: entity.id, a unique, persistent ID

Post by Wyrm »

Rseding91 wrote:You can do player == player2 to check if they're the same player.

You can do anything == anything - inventory references, entity references, GUI references - they all work with the == operator.
Across play sessions?

Also, having entities be usable as keys is very, very handy. That way, I don't have to search through the table of paired values, seeking a particular entity. I can just say some_table[entity_as_key] and be done with it.

Rseding91
Factorio Staff
Factorio Staff
Posts: 11923
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: entity.id, a unique, persistent ID

Post by Rseding91 »

Wyrm wrote:
Rseding91 wrote:You can do player == player2 to check if they're the same player.

You can do anything == anything - inventory references, entity references, GUI references - they all work with the == operator.
Across play sessions?

Also, having entities be usable as keys is very, very handy. That way, I don't have to search through the table of paired values, seeking a particular entity. I can just say some_table[entity_as_key] and be done with it.
Yes across play sessions. When you compare 2 Factorio LuaObjects using == it calls an internal compare (__eq meta method if you know Lua).

I do however agree a unique ID per entity would be useful. Entities with an owner (entities with a force) already have such an ID and it would be simple to expose it. It wouldn't work for *everything* - trees wouldn't have it, items on the ground wouldn't have it but most entities that you'd want to store would have it.
If you want to get ahold of me I'm almost always on Discord.

User avatar
ArderBlackard
Long Handed Inserter
Long Handed Inserter
Posts: 74
Joined: Thu May 05, 2016 12:41 pm
Contact:

Re: entity.id, a unique, persistent ID

Post by ArderBlackard »

Sorry for resurrecting the topic, but this request seems to be still quite relevant.
While the equality comparison between entity references works fine, it's very difficult to use entities as table keys. Two entities which are equal are not necessary the same and thus are considered different as table keys.

It also seems that the API functions always return different wrapper tables so even the code like this (assuming we point mouse at some entity):

Code: Select all

/c map = {}; 
map[game.player.selected] = "dummy"; 
game.player.print( tostring( map[game.player.selected] ) )
will print "nil" instead of "dummy"

while, of course, this:

Code: Select all

/c game.player.print( tostring( game.player.selected == game.player.selected ) )
will print "true"

As a result the only workaround I see so far is to store entities in arrays and find them using equality comparison, while using keys should be a more performant approach I guess.
Gib dich hin bis du Glück bist

Rseding91
Factorio Staff
Factorio Staff
Posts: 11923
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: entity.id, a unique, persistent ID

Post by Rseding91 »

As I've already said: there's no property or ID to expose that would solve this. Most entities in a given map don't have any unique identifier about them other than the memory address they have.

If an ID was added to every single entity we'd definitely need to make it an 8 byte unsigned int due to the amount of entities created and destroyed over the course of a game.

The map I have going has roughly 5.5 million entities on it. At 8 bytes per entity that would add 8 * 5,500,000 bytes of space to store the ID in RAM and in the save file. That would convert to roughly 42 megabytes of space required in the save file just to save IDs if each entity had one.
If you want to get ahold of me I'm almost always on Discord.

User avatar
DaveMcW
Smart Inserter
Smart Inserter
Posts: 3457
Joined: Tue May 13, 2014 11:06 am
Contact:

Re: entity.id, a unique, persistent ID

Post by DaveMcW »

If id was writable but defaulted to nil, how much space would that take on an unmodded save?

I'm thinking something like:

Code: Select all

function entity.getId()
  if not this.id then
    this.id = get_unique_id()
  end
  return this.id
end
It would only assign an id when a modder specifically asked for it.

User avatar
ArderBlackard
Long Handed Inserter
Long Handed Inserter
Posts: 74
Joined: Thu May 05, 2016 12:41 pm
Contact:

Re: entity.id, a unique, persistent ID

Post by ArderBlackard »

Rseding91, thanks for the answer! Hardly ever we will need an ID of every entity in the game including all the doodads :D
But you have said earlier that
Rseding91 wrote:Entities with an owner (entities with a force) already have such an ID and it would be simple to expose it.
If implemented, this would be quite enough, as tracking "large" long-living objects is the most common usage I can imagine for these IDs.

And aside from the idea above just another crazy one:
Rseding91 wrote:As I've already said: there's no property or ID to expose that would solve this. Most entities in a given map don't have any unique identifier about them other than the memory address they have.
Still there is some way to detect whether two entities are equal or not. I mean the __eq metamethod is implemented somehow and it compares something (presumably the related c++ structures pointers/addresses?) Then these addresses seem to be good candidates for entity IDs (if they are not being moved in memory over time of course), dont' they? :) They may even be obfuscated in some way to prevent from exposing raw addresses (like xor-ing them with some value). Of course they would not be preserved across game sessions, but it's still something. And as DaveMcW suggested, the actual entity ID value can be lazy-computed and stored only per-request.
But as I've stated already, it's just a crazy idea :D
Gib dich hin bis du Glück bist

Rseding91
Factorio Staff
Factorio Staff
Posts: 11923
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: entity.id, a unique, persistent ID

Post by Rseding91 »

Memory addresses aren't save load or MP stable so they won't work for key values. They work fine for equality comparison because that's done as a one-off thing and not something you can "save" - you can only save the result of which will be identical every time it's run on any system.
If you want to get ahold of me I'm almost always on Discord.

User avatar
ArderBlackard
Long Handed Inserter
Long Handed Inserter
Posts: 74
Joined: Thu May 05, 2016 12:41 pm
Contact:

Re: entity.id, a unique, persistent ID

Post by ArderBlackard »

Thanks for the explanation. Quite reasonable, I see :)
Anyway it would be nice to have access to the IDs of the object that actualy have them. Hope we will get this functionality sooner or later.
Gib dich hin bis du Glück bist

User avatar
Afforess
Filter Inserter
Filter Inserter
Posts: 422
Joined: Tue May 05, 2015 6:07 pm
Contact:

Re: entity.id, a unique, persistent ID

Post by Afforess »

I have a compromise proposal. Have a field in the data prototypes which turns on persistent entity ids for an entity (like persistent_id = true or something). By default, no entities have it. Then, if a modder wants a particular entity type to have persistent ids, they can set the data prototype field and enable it for individual entities, otherwise, regular users don't experience overhead.

User avatar
ArderBlackard
Long Handed Inserter
Long Handed Inserter
Posts: 74
Joined: Thu May 05, 2016 12:41 pm
Contact:

Re: entity.id, a unique, persistent ID

Post by ArderBlackard »

From what I have read on the forum I guess that entities in C++ code have a rigid format without ability to change their number of fields. Not like in Lua, where you can add and remove fields in runtime. Even having a field for "per-request-computed" ID will still require memory for at least pointer-size variable in C++ entity representation.
So each entity either have or have not an ID and it's only a matter of exposing to Lua those IDs that exist.
Of course, it's just a speculation, I don't know the internal structures for sure.
Gib dich hin bis du Glück bist

ljdp
Long Handed Inserter
Long Handed Inserter
Posts: 96
Joined: Wed Jul 01, 2015 12:33 am
Contact:

Re: entity.id, a unique, persistent ID

Post by ljdp »

Alternatively there could be some hashing method that generates a hash depending on all it's unique properties (i.e position, name, etc).

User avatar
DaveMcW
Smart Inserter
Smart Inserter
Posts: 3457
Joined: Tue May 13, 2014 11:06 am
Contact:

Re: entity.id, a unique, persistent ID

Post by DaveMcW »

ljdp wrote:Alternatively there could be some hashing method that generates a hash depending on all it's unique properties (i.e position, name, etc).
That's the hack modders use now. But it doesn't work for anything that moves.

seronis
Fast Inserter
Fast Inserter
Posts: 225
Joined: Fri Mar 04, 2016 8:04 pm
Contact:

Re: entity.id, a unique, persistent ID

Post by seronis »

DaveMcW wrote:
ljdp wrote:Alternatively there could be some hashing method that generates a hash depending on all it's unique properties (i.e position, name, etc).
That's the hack modders use now. But it doesn't work for anything that moves.
Well position isnt something that should be used in hashes anyways. No non static data should be used in an id hash. In a different game i've modded that uses lua I just had my get_id(entity) function check for the presence of my id variable. If it was nil I would simply assign an unused id. I dont know if Factorio will serialize random mod added variables on default classes.

User avatar
ArderBlackard
Long Handed Inserter
Long Handed Inserter
Posts: 74
Joined: Thu May 05, 2016 12:41 pm
Contact:

Re: entity.id, a unique, persistent ID

Post by ArderBlackard »

Factorio prevents setting any unknown fields to default entities, not to mention serialization.
Gib dich hin bis du Glück bist

Rseding91
Factorio Staff
Factorio Staff
Posts: 11923
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: entity.id, a unique, persistent ID

Post by Rseding91 »

seronis wrote:
DaveMcW wrote:
ljdp wrote:Alternatively there could be some hashing method that generates a hash depending on all it's unique properties (i.e position, name, etc).
That's the hack modders use now. But it doesn't work for anything that moves.
Well position isnt something that should be used in hashes anyways. No non static data should be used in an id hash. In a different game i've modded that uses lua I just had my get_id(entity) function check for the presence of my id variable. If it was nil I would simply assign an unused id. I dont know if Factorio will serialize random mod added variables on default classes.
All properties available in the Factorio Lua API are backed by some C++ class that holds and maintains the data so runtime properties are not supported. The only way to add new properties in is to change the source code and recompile the program on our end.
If you want to get ahold of me I'm almost always on Discord.

Zeblote
Filter Inserter
Filter Inserter
Posts: 972
Joined: Fri Oct 31, 2014 11:55 am
Contact:

Re: entity.id, a unique, persistent ID

Post by Zeblote »

Rseding91 wrote:As I've already said: there's no property or ID to expose that would solve this. Most entities in a given map don't have any unique identifier about them other than the memory address they have.

If an ID was added to every single entity we'd definitely need to make it an 8 byte unsigned int due to the amount of entities created and destroyed over the course of a game.

The map I have going has roughly 5.5 million entities on it. At 8 bytes per entity that would add 8 * 5,500,000 bytes of space to store the ID in RAM and in the save file. That would convert to roughly 42 megabytes of space required in the save file just to save IDs if each entity had one.
It doesn't need to have a unique id for everything. For example, there's no reason why iron plates on a belt should have one.

Surely you can add a unique id to specific classes like trains?

Post Reply

Return to “Implemented mod requests”