Skip to content
Advertisement

Odd Rounding Logic

— Short Solution —

Thank you for all the informative replies… @PetrJaneček @Matt pointed me in the right direction. I agree I had to brush up on my understanding of floating point arithmetic; but the difference in calling the constructor vs double’s canonical string representation by using valueOf solved my problem.

JavaScript

As provided by @PetrJaneček a useful video:

youtu.be/wbp-3BJWsU8?t=246

— Original Question —

I’ve come across a really odd one regarding rounding and just want to understand where I’m going wrong.

The function is:

JavaScript

If I pass the value “1942.945”, I would expect it to output “1942.95”, but instead I get “1942.94”. However if I pass “1942.9451” I get “1942.95”. So ok fair then let’s assume I carry over this logic to 3 decimal places instead. So if I pass “1942.9445” I expect “1942.944”, but instead I get “1942.945”, how and why? The logic seems broken to me?

sampleRound(1942.945, 2) -> 1942.94

[Does NOT make sense, should be 1942.95]

sampleRound(1942.9450000000001, 2)-> 1942.95

*Edit: I’ve also passed this (1 additional zero), again I understand data types and it’s constraints, I guess the point I’m making is the fraction .005 isn’t being seen as a half up >= 5: in accordance with the documentation of java “discarded fraction is ≥ 0.5;” it’s essentially seeing it as > 0.5 not >= 0.5 *

sampleRound(1942.94500000000001, 2) -> 1942.94

[Then this should be equivalent to the top]

Ok but then fine I can deal with the above logic, but as soon as I do 3 decimal places:

sampleRound(1942.9445, 3) -> 1942.945

[Does NOT make sense, in accordance with top logic should be 19423.944]. Then just like I mentioned in the above logic where franaction .005 isn’t being seen as a >= 0.5 then .0005 should be treated the same way?

I hope by description makes sense, but I’m stumped?

Regards,

Advertisement

Answer

As suggested in a comment by @PetrJaneček You can get around this issue by using BigDecimal.valueOf that will create a BigDecimal “using the double’s canonical string representation provided by the Double.toString(double) method.”.

The BigDecimal constructor, new BigDecimal(double) creates a BigDecimal based on the exact value of the the double provided.

JavaScript

1942.945
1942.944999999999936335370875895023345947265625
1942.945

What we see, is first the “canonical string” of v which is what you expect, but the exact value is not “1942.945” it is slightly less. We can see that when we use the BigDecimal constructor and get the exact value back.

For the rest of your examples, we just have to look at the exact representation.

JavaScript

1942.94500000000016370904631912708282470703125

JavaScript

1942.944999999999936335370875895023345947265625

JavaScript

1942.94450000000006184563972055912017822265625

From that, I think it is clear why your rounding behavior is the way it is.

User contributions licensed under: CC BY-SA
4 People found this is helpful
Advertisement