Page 1 of 1
Unexpected multiplication result
Posted: Fri Feb 14, 2020 9:58 am
by Pi-C
Code: Select all
local p = 0.025
local ret = {}
for x, name in ipairs({"A", "B", "C"}) do
ret[x] = {
item = name,
probability = x * p,
}
end
log("ret: " .. serpent.block(ret))
This results in:
Code: Select all
ret: {
{
item = "A",
probability = 0.025
},
{
item = "B",
probability = 0.05
},
{
item = "C",
probability = 0.075000000000000009
}
}
The last value for probability seems to be a bit off. This happened in 0.18.6, by the way …
Re: Unexpected multiplication result
Posted: Fri Feb 14, 2020 9:58 am
by Klonan
This is just how floating point numbers work
Re: Unexpected multiplication result
Posted: Fri Feb 14, 2020 11:18 am
by darkfrei
Klonan wrote: ↑Fri Feb 14, 2020 9:58 am
This is just how floating point numbers work
How to fix it?
x = math.floor (x * 10^4) / 10^4
Re: Unexpected multiplication result
Posted: Fri Feb 14, 2020 11:21 am
by Klonan
darkfrei wrote: ↑Fri Feb 14, 2020 11:18 am
Klonan wrote: ↑Fri Feb 14, 2020 9:58 am
This is just how floating point numbers work
How to fix it?
x = math.floor (x * 10^4) / 10^4
Why do you need to fix it?
Re: Unexpected multiplication result
Posted: Fri Feb 14, 2020 11:25 am
by darkfrei
Klonan wrote: ↑Fri Feb 14, 2020 11:21 am
darkfrei wrote: ↑Fri Feb 14, 2020 11:18 am
Klonan wrote: ↑Fri Feb 14, 2020 9:58 am
This is just how floating point numbers work
How to fix it?
x = math.floor (x * 10^4) / 10^4
Why do you need to fix it?
I want to reed the
color = {r = 0.5, g = 0.25, b = 0.75}
but not the
color = {r = 0.5000000000012, g = 0.2500000000006, b = 0.75000000000018}
It's easier to read the short form.
Re: Unexpected multiplication result
Posted: Fri Feb 14, 2020 11:48 am
by Pi-C
darkfrei wrote: ↑Fri Feb 14, 2020 11:25 am
Klonan wrote: ↑Fri Feb 14, 2020 11:21 am
Why do you need to fix it?
I want to reed the
color = {r = 0.5, g = 0.25, b = 0.75}
but not the
color = {r = 0.5000000000012, g = 0.2500000000006, b = 0.75000000000018}
It's easier to read the short form.
Correct. It doesn't really matter in the game if the calculated probability is a miniscule fraction bigger than expected, but it doesn't look nice. I've worked around it by using integers and dividing the result by 1000 -- basically the same thing darkfrei did.
Re: Unexpected multiplication result
Posted: Fri Feb 14, 2020 5:39 pm
by movax20h
The reason you are seeing these numbers is because "0.025" is not a representable floating point number. 0.25 is. But 0.025 isn't. Closest value is "0.025000000000000001387778780781".
Agreed with devs. Not a bug. If you want to format values to be "nicer" you need to do it manually when printing. Or by pre-multiplicating things by some reasonable constant (1000 for example) maybe.