Roll20 uses cookies to improve your experience on our site. Cookies enable you to enjoy certain features, social sharing functionality, and tailor message and display ads to your interests on our site and others. They also help us understand how our site is being used. By continuing to use our site, you consent to our use of cookies. Update your cookie preferences .
×
Create a free account

Odd rounding error.

This code: [[1d20+1.01*(6) &{tracker}]] should return numbers in the format of 1d20+6+0.06, so from 7.06 to 20.06 in increments of 1.00. Works fine with any integer substituted for the '6' which I tried, except for the 6. That introduces bizarre rounding errors. Initiative Result: 13.059999999999999 Dean Nerum: Initiative Result: 15.06 Initiative Result: 16.060000000000002 Initiative Result: 25.060000000000002 Dean Nerum: Initiative Result: 22.06 Initiative Result: 9.059999999999999 Initiative Result: 15.059999999999999 I fixed this by changing to [[1d20+6+0.06 &{tracker}]] which should be mathematically equivalent. Occurred for everyone in my group who tried to use this macro, regardless of browser or OS.
1407468218
Gauss
Forum Champion
This is a known problem with how Javascript handles floating point numbers. <a href="https://app.roll20.net/forum/post/363762/round-counter-trick-macro#post-364294" rel="nofollow">https://app.roll20.net/forum/post/363762/round-counter-trick-macro#post-364294</a> and
1407478677
Lithl
Pro
Sheet Author
API Scripter
More specifically, it's an artifact of how computers handle floating-point numbers, not just JavaScript.
Then why does the second way I posted it work? It's very confusing to me.
1407519235
The Aaron
Roll20 Production Team
API Scripter
Because it stores the numbers as a mantissa and exponent. When you do the multiplication, it makes various adjustments based on the relative exponents, which moves the decimal place and introduces small errors. Afterwards, it may have the same exponent again, but the error has already been introduced and is thus propagated to the answer. With addition, it uses a different set of rules that do not introduce the same errors because the size of the result is smaller so it doesn't need the same amount of headroom. Here's a page that's much better at explaining it than me, if you want to go deep: <a href="http://www.cs.umd.edu/class/sum2003/cmsc311/Notes/" rel="nofollow">http://www.cs.umd.edu/class/sum2003/cmsc311/Notes/</a>... Cheers!
1407520852
The Aaron
Roll20 Production Team
API Scripter
If you want to look at the binary representations of the numbers involved, try this site: <a href="http://babbage.cs.qc.cuny.edu/IEEE-754.old/Decimal" rel="nofollow">http://babbage.cs.qc.cuny.edu/IEEE-754.old/Decimal</a>... Here are the numbers from your example: # exponent mantissa 1.01: 01111111111 1.0000001010001111010111000010100011110101110000101001 6: 10000000001 1.1000000000000000000000000000000000000000000000000000 Results of multiplication: 6.059..: 10000000001 1.100000111101011100001010001111010111000010100011110 0 6.06: 10000000001 1.100000111101011100001010001111010111000010100011110 1 You can see that the number which results from the multiplication has lost the least significant bit of the mantissa when compared to the desired result. When the mantissae are multiplied, the result must then be shifted by one bit to bring a 1 back in the most significant bit. When this happens, a 0 is shifted on (I believe because the least significant bit at that time is a 0, so it is extended, but I don't remember precisely). When you add, it's much easier to explain. Add numbers: 0.06: 01111111010 1.111010111000010100011110101110000101000111101 0111000 6: 10000000001 1.100000 0000000000000000000000000000000000000000000000 Results 6.06: 10000000001 1.100000 1111010111000010100011110101110000101000111101 To add, the smaller number is shifted to the right by the difference in the exponents (7 in this case). I've underlined the parts of the added numbers that make up the result. The underlined part of the result is what was contributed by the right shifted 0.06. Hope that gives you deeper understanding. Let me know if you want to know more and I'll see what I can do. =D
1407539185

Edited 1407539296
I see. Thanks for the explanation! Would it be possible to add a truncate/truncate+round command in the future to deal with this?
1407546015

Edited 1407546066
The Aaron
Roll20 Production Team
API Scripter
It's a native feature of the way the language handles math, creating a rounding function might not be a perfect fix for all cases. I have no idea if they will do that. In the interim, you can do: [[1d20+(1* (6) )+(.01* (6) ) &{tracker}]] and avoid the floating point rounding error. This will still allow you to use an attribute to represent whatever the (6) really is.