Decimal rounding is off for (276/304)*304?

276/304 = 69/76 is a recurring "decimal" in both base 10 and base 2.

276/304 = 69/76 is a recurring "decimal" in both base 10 and base 2. Decimal: 0.90(789473684210526315) binary: 0.11(101000011010111100) So the result gets rounded off, and multiplying by the denominator may not result in the orginal numerator. A more commonly-cited example of this situation is 1/3*3 = 0.33333333*3 = 0.99999999. That the double version gives the exact answer is just a coincidence.

The rounding error in the multiplication just happens to cancel out the rounding error in the division. If this result is confusing, it may be because you've heard that "double has rounding errors and decimal is exact". But decimal is only exact at representing decimal fractions like 0.1 (which is 0.0 0011 0011... in binary).

When you have a factor of 19 in the denominator, it doesn't help you.

Thanks for giving me a clear explanation! So the solution is when comparing numeric values you should always round to the amount of decimals you're calculating with and you should be fine? In my case I need to verify if 275.999999999999999 == 276 so I'll just round up by 6 decimals and the result should be true.

– Peter Feb 17 '11 at 14:36 1 @Peter No this doesn't work in every case either. For example imagine you're comparing 1.000000499 with 1.000000501 and round both to 6 decimals then you get 1.000000! = 1.000001 even so their difference is very small.

– CodeInChaos Feb 17 '11 at 14:50 But the floating point rounding error in decimals could never be 1.000000499 right? It would always be the 27th decimal that's off? – Peter Feb 17 '11 at 14:59.

Well, floating point precision isn't 100%. See for example: effbot.org/pyfaq/why-are-floating-point-....

Also see this for Decimal vs Double: social.msdn.microsoft.com/forums/en-US/c...… – Filip Ekberg Feb 17 '11 at 14:08 @Filip Pretty much every post in that thread contains big mistakes. – CodeInChaos Feb 17 '11 at 14:15 so why is there a different result between the two? If both are floating point variables?

– Peter Feb 17 '11 at 14:17 3 One is in base 10 and one in base 2. There are numbers you can exactly represent in base 10 and not base 2. For example 0.1 can be represented exactly in Decimal but not Double.

And Decimal throws exceptions on overflows instead of silently using infinities or NaNs. But neither can represent 1/3 exactly. – CodeInChaos Feb 17 '11 at 14:20.

Well, mathematically 0.99999... == 1. Have a look at en.wikipedia.org/wiki/0.999... I know that programtically it poses some problems, but it's not totally a floating-point issue.

2 0.9999999... == 1, but 0.99999999999999999999999 == 1-1e-23. – dan04 Feb 18 '11 at 5:52.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.