Joel
06/25/2020, 10:47 PMassertEquals(
BigDecimal("0.08333333333333333"),
(1.toBigDecimal() / 12.toBigDecimal())
)
expected: <0.08333333333333333> but was: <0>
The bottom evaluates to 0
which is clearly incorrect. What is the rule here and how can I avoid this pitfall?Robert Jaros
06/25/2020, 11:25 PM1.toBigDecimal().setScale(17) / 12.toBigDecimal()
Joel
06/25/2020, 11:27 PMRobert Jaros
06/25/2020, 11:34 PM1.toBigDecimal().divide(12.toBigDecimal(), 17, java.math.BigDecimal.ROUND_HALF_UP)
Robert Jaros
06/25/2020, 11:36 PMJoel
06/26/2020, 12:09 PMJoel
06/26/2020, 12:11 PM@kotlin.internal.InlineOnly
public inline operator fun BigDecimal.div(other: BigDecimal): BigDecimal = this.divide(other, RoundingMode.HALF_EVEN)
The default divide operator uses half even rounding and determines scale from the operating decimals. If neither have a scale, it rounds to the nearest integer. You can override the division operator using:
private operator fun BigDecimal.div(other: BigDecimal) = divide(other, MathContext.DECIMAL128)
which changes the behavior to compute the decimal positions.