Unexpected multiplication result

[Pi-C]

Code: Select all

local p = 0.025
local ret = {}

for x, name in ipairs({"A", "B", "C"}) do
    ret[x] = {
        item = name,
        probability = x * p,
    }
end
log("ret: " .. serpent.block(ret))

This results in:

Code: Select all

ret: {
  {
    item = "A",
    probability = 0.025
  },
  {
    item = "B",
    probability = 0.05
  },
  {
    item = "C",
    probability = 0.075000000000000009
  }
}

The last value for probability seems to be a bit off. This happened in 0.18.6, by the way …

[Klonan]

This is just how floating point numbers work

[darkfrei]

This is just how floating point numbers work

How to fix it?
x = math.floor (x * 10^4) / 10^4

[Klonan]

This is just how floating point numbers work

How to fix it?
x = math.floor (x * 10^4) / 10^4

Why do you need to fix it?

[darkfrei]

This is just how floating point numbers work

How to fix it?
x = math.floor (x * 10^4) / 10^4

Why do you need to fix it?

I want to reed the
color = {r = 0.5, g = 0.25, b = 0.75}
but not the
color = {r = 0.5000000000012, g = 0.2500000000006, b = 0.75000000000018}

It's easier to read the short form.

[Pi-C]

Why do you need to fix it?

I want to reed the
color = {r = 0.5, g = 0.25, b = 0.75}
but not the
color = {r = 0.5000000000012, g = 0.2500000000006, b = 0.75000000000018}

It's easier to read the short form.

Correct. It doesn't really matter in the game if the calculated probability is a miniscule fraction bigger than expected, but it doesn't look nice. I've worked around it by using integers and dividing the result by 1000 -- basically the same thing darkfrei did.

[movax20h]

The reason you are seeing these numbers is because "0.025" is not a representable floating point number. 0.25 is. But 0.025 isn't. Closest value is "0.025000000000000001387778780781".

Agreed with devs. Not a bug. If you want to format values to be "nicer" you need to do it manually when printing. Or by pre-multiplicating things by some reasonable constant (1000 for example) maybe.