64-bit integer limit

[Epb7304]

TL;DR

expand integer limit to 64-bit

What ?

Currently signals work on a 32-bit system, which means that each signal can have 2^32 possible values (-2.1G < x < 2.1G)

I propose this be expanded to a 64-bit number, which would mean that each signal would have a possible 2^64 value or (-9.2E < x < 9.2E)

Why?

I know I am not alone when I say that I constantly will hit the 32-bit limit when designing circuits, this is typically when I want to compress a few smaller signals into 1 larger one, or when I want to track a large number of items. Expanding the integer limit to 64-bit would allow values in the Quintillions and I for one would believe the Can to be sufficiently kicked.

This also would not require the game to be run in 64-bit (if its not already) since you should just be able to assign 2 number slots per signal

[Illiander42]

This also would not require the game to be run in 64-bit (if its not already) since you should just be able to assign 2 number slots per signal

As someone who's had to do that, it's a mess.

Anything you're doing in particular that needs more than 32 bits?

[FuryoftheStars]

This also would not require the game to be run in 64-bit (if its not already)

https://factorio.com/blog/post/fff-158

[Epb7304]

This also would not require the game to be run in 64-bit (if its not already)

https://factorio.com/blog/post/fff-158

Even better, that would make this change nearly seamless and not overly complicated either

[Epb7304]

This also would not require the game to be run in 64-bit (if its not already) since you should just be able to assign 2 number slots per signal

As someone who's had to do that, it's a mess.

Anything you're doing in particular that needs more than 32 bits?

The biggest thing is actually to do with FFF 402 and compressing my data streams to only 1 signal so it will not interfere with other information in the data stream, this issue also arises however when wanting to store a particularly large value. Say, the amount of power your base is producing/consuming, or how many seconds your base has been working for, or how many iron plates you have made (yes those are in statistics, but this would be for a display board or a smart base system).

[mmmPI]

this issue also arises however when wanting to store a particularly large value. Say, the amount of power your base is producing/consuming,

One can't read that on circuit though, it's only the accumulator charge.

or how many seconds your base has been working for, or how many iron plates you have made (yes those are in statistics, but this would be for a display board or a smart base system).

2 billion second is 60 years or so which makes it unlikely to me that it would be achieved in a game of factorio , but even if going x64 speed during a full year, it would be possible to store the number of hour, or days, similar to storing the number of MW and not Watt to reduce by a factor 1 000 000.

This also would not require the game to be run in 64-bit (if its not already) since you should just be able to assign 2 number slots per signal

The same apply in the game, it's possible if you want to store 3 billions second worth of time to use the channel "C" and "D" instead of just 1 that would see an overflow.

I'm not against the suggestion but there are other topics with better arguments

[Illiander42]

This also would not require the game to be run in 64-bit (if its not already) since you should just be able to assign 2 number slots per signal

As someone who's had to do that, it's a mess.

Anything you're doing in particular that needs more than 32 bits?

The biggest thing is actually to do with FFF 402 and compressing my data streams to only 1 signal so it will not interfere with other information in the data stream, this issue also arises however when wanting to store a particularly large value. Say, the amount of power your base is producing/consuming, or how many seconds your base has been working for, or how many iron plates you have made (yes those are in statistics, but this would be for a display board or a smart base system).

Timeslotting and negotiating for channel ownership works fine for putting mutiple data streams over radar connections. As soon as we get access to new combinators there'll probably be a few people putting out blueprint books for that. Happy to run you through the broad strokes of how to do those if you want (I'm waiting for new combinators to do my blueprints).

And as the other poster said, you can do large value counting yourself if you really need it.

The only reason I can think of for this would be if we run out of bitflag space in the non-item signals, and that can be solved by adding more virtual signals to the pile instead of making the numbers bigger.

[boskid]

This is unlikely to happen unless we would update Lua to something newer than 5.2. If circuit network signals would have values based on 64 bit integers then scripts would have a hard time setting some values since all values in lua 5.2 are based on a `double` which cannot represent all 64 bit integers precisely.

[Qon]

This is unlikely to happen unless we would update Lua to something newer than 5.2. If circuit network signals would have values based on 64 bit integers then scripts would have a hard time setting some values since all values in lua 5.2 are based on a `double` which cannot represent all 64 bit integers precisely.

How about 53-bit integers then? It's more than 32 so we get some benefit. We can't store 2 32-bit signals in one, but we can store 2 26-bit signals or 3 17-bit signals, which is both a benefit over being limited to e.g. 2 16-bit numbers when packing multiple values into a single signal type.

[boskid]

How about 53-bit integers then? It's more than 32 so we get some benefit. We can't store 2 32-bit signals in one, but we can store 2 26-bit signals or 3 17-bit signals, which is both a benefit over being limited to e.g. 2 16-bit numbers when packing multiple values into a single signal type.

How about 32-bit integers? Its possible to store 3 10-bit signals or 2 16-bit signals.

[Illiander42]

This is unlikely to happen unless we would update Lua to something newer than 5.2. If circuit network signals would have values based on 64 bit integers then scripts would have a hard time setting some values since all values in lua 5.2 are based on a `double` which cannot represent all 64 bit integers precisely.

Would the UTF support (which is always good for localisation) be worth the upgrade pain?

[pleegwat]

for a display board

A limited bignum implementation in combinators should not be too difficult, especially since you're only adding in scalars. If the accumulator is larger than 1 billion decimal, subtract 1 billion from it and increment the next accumulator up. I guess that would be 3 or 4 combinators for every additional factor of a billion? And I'm sure it vectorizes.

[Illiander42]

for a display board

A limited bignum implementation in combinators should not be too difficult, especially since you're only adding in scalars. If the accumulator is larger than 1 billion decimal, subtract 1 billion from it and increment the next accumulator up. I guess that would be 3 or 4 combinators for every additional factor of a billion? And I'm sure it vectorizes.

2 combinators if deciders could output an arbitrary number, unless it needs a tick delay to not double-count? Possibly a single combinator in 2.0.

[TheKillerChicken]

No thanks, I do not want to invest in a server that will require 4 terabytes of memory due to the integers flooding the memory pool. 32-bit is just fine for most circumstances in this game. I have all my stacks at 2 billion, and that is suffice for me as I play this game with stupidly huge values.

[DarkShadow44]

The biggest thing is actually to do with FFF 402 and compressing my data streams to only 1 signal so it will not interfere with other information in the data stream, this issue also arises however when wanting to store a particularly large value.

I'd recommend using a mod that adds more virtual signals, AFAIK you can add an arbitrary number of them - if you feel like having 2 signals for each item, it should be doable.

This is unlikely to happen unless we would update Lua to something newer than 5.2.

Is there anything else new in recent lua release that would be useful for factorio? Because I didn't think so. And upgrading sounds like a massive pain...

No thanks, I do not want to invest in a server that will require 4 terabytes of memory due to the integers flooding the memory pool. 32-bit is just fine for most circumstances in this game. I have all my stacks at 2 billion, and that is suffice for me as I play this game with stupidly huge values.

Curious though, what happens at those sizes? Do the circuit values overflow or does it cap at the maximum positive number?
Not sure what you mean by "flooding the memory pool".

[TheKillerChicken]

The biggest thing is actually to do with FFF 402 and compressing my data streams to only 1 signal so it will not interfere with other information in the data stream, this issue also arises however when wanting to store a particularly large value.

I'd recommend using a mod that adds more virtual signals, AFAIK you can add an arbitrary number of them - if you feel like having 2 signals for each item, it should be doable.

This is unlikely to happen unless we would update Lua to something newer than 5.2.

Is there anything else new in recent lua release that would be useful for factorio? Because I didn't think so. And upgrading sounds like a massive pain...

No thanks, I do not want to invest in a server that will require 4 terabytes of memory due to the integers flooding the memory pool. 32-bit is just fine for most circumstances in this game. I have all my stacks at 2 billion, and that is suffice for me as I play this game with stupidly huge values.

Curious though, what happens at those sizes? Do the circuit values overflow or does it cap at the maximum positive number?
Not sure what you mean by "flooding the memory pool".

If the entire 64-bit pool was utilised, it would require super-computer level memory space. All my stacks at 2 billion eat up about 10 GB of memory space, but 64-bit integers are 1.8 duodecillion. No possible home machine can even begin to have that kind of memory capacity (~1-4 exabytes).

[DarkShadow44]

If the entire 64-bit pool was utilised, it would require super-computer level memory space. All my stacks at 2 billion eat up about 10 GB of memory space, but 64-bit integers are 1.8 duodecillion. No possible home machine can even begin to have that kind of memory capacity (~1-4 exabytes).

That's not how any of this works though? Just because you have 64bit integers doesn't mean you use exabytes of storage.
I don't understand what "stacks" eat 10GB of memory for you, you really have 2 billion chests? Anyways, doubling 32bit to 64bit only doubles the storage requirements.

[TheKillerChicken]

If the entire 64-bit pool was utilised, it would require super-computer level memory space. All my stacks at 2 billion eat up about 10 GB of memory space, but 64-bit integers are 1.8 duodecillion. No possible home machine can even begin to have that kind of memory capacity (~1-4 exabytes).

That's not how any of this works though? Just because you have 64bit integers doesn't mean you use exabytes of storage.
I don't understand what "stacks" eat 10GB of memory for you, you really have 2 billion chests? Anyways, doubling 32bit to 64bit only doubles the storage requirements.

What I mean is that if the value keyed into the game logic was a full-blown duodecillion, it will hit ~2 exabytes of ram. Now, will someone even be able to achieve that? I do not think so. Now with all of this being stated, perhaps with this case, setting a duodecillion may NOT eat up much DRAM at all. And what I mean with the stacks is that every item that I create can stack up to 2 billion. It is slowly eating up my memory as I get into the 100 billion+ stacks of items. I treat this game more like a DRAM/Processor benchmark with huge values and ultra-overpowered machines/beacons than an actual game to see how much abuse my server/client can take before they choke and crash or cause a permanent de-syncornisation. Thankfully, this game has a very optimised memory pool code, so I also feel that it COULD be possible in Factorio Space Age to increase the stacks, logics, and arithmatic values to the full 64-bit integers. Once again, it all depends on how the user treats this game and I would hit an exabyte really quickly with those values due to how crazy I have my values. I would use the 4.2 billion value, but the machines break due to the recipes creating some kind of memory-feedback or memory-kludge, thus causing the machines to not acknowledge the ingredients in them. Hand-crafting is not effected. I think a perfect balance would be 48-bit integers, but I do not develop games, I just play with the data and scripts that were created for the game(s). The maximum storage chest size is 65536, which is a 16-bit integer, and due to my said values, those chests fill up within a period of 1 week. I see a memory hit with those, but not significantly. I think the item stacks will hit the memory pool the hardest as it has to calculate the code of each item. I DO wish the beacon radius could be as large as the logistic and construction bot port can go (mine is set to 2 million by 2 million). The beacon is limited to 64x64 radius. Also, apparently the module slots can go up to 65536 as of 1.0, a definite increase from its' previous 6. I modified the big lab to have 250 module slots in it. I use the god modules (250% buffs) with a beacon of 64x64 radius with 8 module slots in it. Yes, my goal is simply: How much can a 6-channel DDR3-1866 LRDIMM server and a 2-channel DDR5-4800 gaming client can take in this game.

[DarkShadow44]

What I mean is that if the value keyed into the game logic was a full-blown duodecillion, it will hit ~2 exabytes of ram.

Again, that's now how it works. Going from 32bit to 64bit simply doubles the memory requirements. For those integers.