Friday Facts #108 - Unexpected Features

[kovarex]

I hasn't broken any of the mods i have, (except 1, the clock from bobs mods), but something really annoying is that all the names of stuff say entity:name:name of thing instead of the normal name, can u fix it soon, because its really annoying, and waiting an entire week seems like soooooooo long, also before friday facts came out on friday like at 1 pm, now the come out at like 1 am.... of the next day, Please Fix, thumbs up for the smoke graphics

I just fixed that, so yes, we will probably do the release sooner than on friday.

[bobucles]

Mine is, that floats are really practical, if you want to measure something and keep the relative precision small.

You can keep relative precision exactly the same way by using integers. Instead of storing whole number values, you simply shift the decimal point. Now 17 becomes 17.000 and adding one is adding 1.000. Big whoop. It's all one CPU operation after the initial conversion, and translating it for human display is trivial. There is a big advantage in always knowing what precision is in use- I'm sure you've seen plenty of patch notes because the game does something stupid with excess floating point decimals.

The vast majority of a float's bit space is utter waste. You do not need to know how many angstroms a player is from origin, and that value is only meaningful when the player is standing in proximity to 0,0 anyway. Gaming interactions become meaningless when you attempt to use the full exponential range; no one cares about adding 5E02 to 7E908 or comparing it to 1 E-500. If your numbers are reaching these values the game has either exploded in some terrible terrible way or is in fact an incremental game.

Integers are direct. Integers are exact. You know exactly the ranges they will deal with, and they give EVERY significant digit at ALL times. Floats only matter for certain types of math. In every other gaming case where you are presenting some kind of value to the player, xx-bit ints can do everything exactly as designed. There is no human situation where you will exceed 64-bit values and be able to grasp just how ridiculous of a range it really is.

[ratchetfreak]

Mine is, that floats are really practical, if you want to measure something and keep the relative precision small.

You can keep relative precision exactly the same way by using integers. Instead of storing whole number values, you simply shift the decimal point. Now 17 becomes 17.000 and adding one is adding 1.000. Big whoop. It's all one CPU operation after the initial conversion, and translating it for human display is trivial. There is a big advantage in always knowing what precision is in use- I'm sure you've seen plenty of patch notes because the game does something stupid with excess floating point decimals.

The vast majority of a float's bit space is utter waste. You do not need to know how many angstroms a player is from origin, and that value is only meaningful when the player is standing in proximity to 0,0 anyway. Gaming interactions become meaningless when you attempt to use the full exponential range; no one cares about adding 5E02 to 7E908 or comparing it to 1 E-500. If your numbers are reaching these values the game has either exploded in some terrible terrible way or is in fact an incremental game.

Integers are direct. Integers are exact. You know exactly the ranges they will deal with, and they give EVERY significant digit at ALL times. Floats only matter for certain types of math. In every other gaming case where you are presenting some kind of value to the player, xx-bit ints can do everything exactly as designed. There is no human situation where you will exceed 64-bit values and be able to grasp just how ridiculous of a range it really is.

But if you multiply 17 with 1 you end up with 17000*1000 = 17000000 -> 17000 which you have to correct for. If you multiply 3000 with 1000 you end up with 3000 000*1000 000 = 3000 000 000 000 which exceeds the precision of a 32 bit signed int even though the result fits within the range

[kovarex]

We use fixed point for few things in the game, but for many things it would be very impractical.

[ske]

We use fixed point for few things in the game, but for many things it would be very impractical.

If I may ask:
Are you only using single precision?
Which compiler do you use? (Assuming gcc would be most likely, but llvm on osx and vc on windows could be possible.)
And which compiler flags do you use regarding floats?

[kovarex]

We use fixed point for few things in the game, but for many things it would be very impractical.

If I may ask:
Are you only using single precision?
Which compiler do you use? (Assuming gcc would be most likely, but llvm on osx and vc on windows could be possible.)
And which compiler flags do you use regarding floats?

It is gcc/vs we use strict for floats. We use both double precision and precision, it depends on usage.

[ske]

It is gcc/vs we use strict for floats. We use both double precision and precision, it depends on usage.

Thanks for the info

One thing that might be peripherally relevant for factorio are denormalized floats. As far as I see, denormalized numbers are enabled when using strict float handling. In the past they used to be very slow. They can occur when calculating differential equations (e.g. a slow exponential decay) which is often used in audio signal processing. Things like fog calculations might use some similar equations but I think it is unlikely (=it can and will happen when we least expect it) that many denormalized floats occur at one time. Luckily Factorio is no hard realtime simulation

Two pages I've found for those who like to know more about float:
A good description on how they work: http://steve.hollasch.net/cgindex/coding/ieeefloat.html
A list of things that could go wrong when compiler optimization is turned on for floats: https://gcc.gnu.org/wiki/FloatingPointMath

Since factorio is only on x86/AMD64 architectures, there shouldn't be problems with buggy implementations, but I assume that other processor architectures could have some bugs which makes them handle some of the edge cases a little differently. Maybe someone knows about ARM?

[nulbloat]

To be honest, I still don't understand why the display should be synchronized/deterministic - it should not concern the game engine if one player sees 1e-05 and another sees 1e-005 - that's just the view layer - the data model should not be affected by this.
What am I missing here?

It is not just a display. The console history is contained in the save, so suddenly two saves are not identical even when they should be.
Yes, in this case, it would probably not cause any problem, but our checking mechanisms just blindly compare all data to check that the game is deterministic, so we have to fix even small issues, so the mechanism is still usable to find other issues that might be more important.

And also, you can write something like this:

Code: Select all

if tostring(0.00001) == "1e-005" then
  game.players[1].printf("You are on windows")
  game.players[1].insert{name="power-armor-mk2", count=1}
end

This will obviously make a desync.

If these issues will break the sync how will you handle different localisations?

Code: Select all

if game.get_localised_item_name("pistol") == "Pistol" then
  ...
end

I would imagine the console log could look different when printing out based on localised string also.

[kovarex]

To be honest, I still don't understand why the display should be synchronized/deterministic - it should not concern the game engine if one player sees 1e-05 and another sees 1e-005 - that's just the view layer - the data model should not be affected by this.
What am I missing here?

It is not just a display. The console history is contained in the save, so suddenly two saves are not identical even when they should be.
Yes, in this case, it would probably not cause any problem, but our checking mechanisms just blindly compare all data to check that the game is deterministic, so we have to fix even small issues, so the mechanism is still usable to find other issues that might be more important.

And also, you can write something like this:

Code: Select all

if tostring(0.00001) == "1e-005" then
  game.players[1].printf("You are on windows")
  game.players[1].insert{name="power-armor-mk2", count=1}
end

This will obviously make a desync.

If these issues will break the sync how will you handle different localisations?

Code: Select all

if game.get_localised_item_name("pistol") == "Pistol" then
  ...
end

I would imagine the console log could look different when printing out based on localised string also.

This is already sorted out. the get_localised_item_name("pistol") actually returns table {"item.pistol"}, not "Pistol". This table is same for everyone as it is localisation key. It is only eveluated when displayed (WHen it is in the console, or custom gui, flying text etc.) https://www.factorio.com/blog/post/fff-53

[kovarex]

I just wanted to leave this here:

http://www.netlib.org/fp/dtoa.c

If you're going to write your own print formatting (printf replacement), you'll need a way to output floating point values. It probably won't surprise you that this part is harder than everything else put together! I haven't experimented to much with that link, but it is a liberally licensed method for converting floating point values to strings. It is claimed to be (1) fast and (2) accurate. If you do decide to go that way, I hope that this might help you . I have also had to rewrite output code (in my case, I had to rewrite the C++ streams so that they would work across DLL boundaries for incompatible compiler settings), and if I hadn't been able to get sprintf working acceptably, this would have been my next resort. Good luck, those floats are a beast!

I spent some time trying to use this dtoa just to find out, that it translates 0.458 to "458", while the sign and the "dot" position is given in other parameters, and it is up to you to figure it out. That is really lame.

[Twisted_Code]

Yay, my idea is implemented ... thanks Rseding91!

[Ashnoom]

The hardest part with fixed point math is choosing your range and avoiding over/underflow in the temporary values. Floats will auto shift to keep as many bits as possible

Just use python. No under/overflowing (well, ok, underflowing) of integers. Infinite number of bits available for your mathematics. Want to add a 64bit and a 128bit value together? No problem. Python basically allows infite(*) large numbers. Sure things might start to slow down if you try to calulcate the power of 128bits value with another 128bits value.. But you can


(*) Infinite depends on the amount of storage space (RAM/page file) available to your system.

[ratchetfreak]

The hardest part with fixed point math is choosing your range and avoiding over/underflow in the temporary values. Floats will auto shift to keep as many bits as possible

Just use python. No under/overflowing (well, ok, underflowing) of integers. Infinite number of bits available for your mathematics. Want to add a 64bit and a 128bit value together? No problem. Python basically allows infite(*) large numbers. Sure things might start to slow down if you try to calulcate the power of 128bits value with another 128bits value.. But you can


(*) Infinite depends on the amount of storage space (RAM/page file) available to your system.

Have you ever tried working with BigNums in a performance critical environment? doing heavy calculations of 128+ bit numbers will lead to a lot of allocations as the digit buffer gets resized to fit the result.

Not something you'll want to do in a already CPU bound game.

[enmoku]

custom trigonometric functions

Didn't GCC and the others have a compiler option to cause various math functions to produce consistent results? I think they were meant for scientific apps for the most part, but seems like they'd be useful for this as well. Looking up, I found IEEE 754 compliance mentioned (https://gcc.gnu.org/wiki/FloatingPointMath). MSVC seems to have similar option (https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx). Not sure if this affects the print function tho.

[kovarex]

custom trigonometric functions

Didn't GCC and the others have a compiler option to cause various math functions to produce consistent results? I think they were meant for scientific apps for the most part, but seems like they'd be useful for this as well. Looking up, I found IEEE 754 compliance mentioned (https://gcc.gnu.org/wiki/FloatingPointMath). MSVC seems to have similar option (https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx). Not sure if this affects the print function tho.

Yes, and we use the strict floating point model, so all the calculations are consistent. The problem is, that these trigionometric methods are not part of the compiler, but are written in the standard template library. The problem is, that C++ standards nor IEEE 754 specify the precision or results of these methods.