https://kotlinlang.org logo
Join Slack
Powered by
alright friends I cant think of a better place to ...
# compiler
g
alright friends I cant think of a better place to put this so im plopping it in here. I need to understand IEEE 754 and error with respect to writing values. For my problem there are no math operations to induce error, only attempts to represent a double value. the code
Copy code
val firstDbl: Double = 0.1
val firstStr: String = "0.1"
val firstBigStr = BigDecimal("0.1")
val firstBigDbl = BigDecimal(0.1)
has some funny equality implications:
Copy code
firstDbl == firstStr.toDouble() //true
firstBigStr == firstBigDbl //false
firstBigStr.toDouble() == firstBigDbl.toDouble() // false, 0.1 != 0.0...555

firstBigDb.toString() // 0.1000000000000000055511151231257827021181583404541015625
Can somebody help me understand where this error came from? Should i be calling a different BigDecimal constructor?
e
you've already lost precision by writing the literal 
0.1
in code
g
but 
Double.toString()
knew how to get it back?
what else does it know how ot get back?
e
.toString()
finds the shortest string representation that produces the same value, not the exact representation
0.1000000000000000055511151231257827021181583404541015625 == 0.1
g
right
0x3FB999999999999A
thanks i hate it
XD
e
toString()
produces the latter, 
BigDecimal
captures all of the former
g
so I have some legacy model code that is using 
double
fields to store some input from a user. Its pretty straight forward. I was aware that the first time you do an operation on that value you've incurred a loss, but I had thought that any value that was denotable within the precision limits of IEEE 754 was, how should I say, recoverable as its denotation within those bounds. IE, if i wrote 
0.1
in a text field (or indeed in intelliJ as a constant), the I would get the bits 
0x3FB999999999999A
, and, like an ASCII coding of 
A
to 
0x41
, i could convert back and forth between them as I saw fit.
e
the floating-point value that you get when you write 
0.1
is not actually 
0.1
as you think of it, as that cannot be exactly represented
g
well if i was in closure it would be, but i take your point
i really really wish i could get a 
ratio
primative in kotlin, i wonder if thats already an issue
e
you'd have to write 1/10 to get a ratio in clojure, 0.1 is still a IEEE floating point
or suffix M I think?
g
i thought you could write like 
0.1r
, but im digressing
yeah
given that I write 
val myDouble = doubleParsedFromDenotableUserText(text)
how safe is it to write 
val bigDecimal = BigDecimal(myDouble.toString())
, I guess my question is how much of a perfect dual operation is 
text.toDouble()
and 
double.toString()
? when will the two not produce a result that agrees?
e
double.toString().toDouble()
is supposed to round-trip (for finite values, on JVM - not JS), but 
string.toDouble().toString()
is not, as there are many possible string representations for the same floating-point value
g
I mean i understand the concept that with the nature of finite storage and truncation we're trying to encode a range on the real number line to a single bit pattern
are there other bitpatterns that correspond to the same value 
0.1
?
e
your wording still indicates some real confusion to me
g
so i understand that 
0.1 == 0.100000000000000005  && 0.1 == 0.0.100000000000000005999
is true
e
"0.099999999999999999".toDouble() == "0.1".toDouble()
g
right
e
are you asking if there are any other values of 
Double
which will 
.toString()
to 
"0.1"
?
g
yes i suppose i am
a value of double other than `0x3FB999999999999A``
e
on JVM, no there are not, if the JVM is not buggy. this is specified by https://docs.oracle.com/javase/8/docs/api/java/lang/Double.html#toString-double-
there have been bugs, but generally in the other direction: https://bugs.openjdk.org/browse/JDK-4511638
g
ok so lets say "the Denotable numbers" is "the set of decimal numbers that can be written down". This is a strict subset of rational numbers. (1/3 is a ratio but is not part of my set). can all members of the set of denotable numbers with less than 14 sig figs be converted from its denotation, to a 
double
, and back again, losslessly?
my understanding is that there is an exact sparse mapping, every member of the <14-sig-fig Denotables would map to exactly one representation under IEEE754 double, and that given such a 
double
value (IE, a double value we knew was parsed from a Denotable), we could get that denotable representation back
e
under what I believe is the common definition of sig fig: no
1.1E-323 == 1.2E-323
g
I didnt think double representations could be that small
e
1.8E308 == 1.9E308
g
i think 1.9E308 is also bigger than Double.MaxValue but
i take your point
with E307
well wait
e
the example in the bug I linked before applies to Java 17 (although it's in violation of 
Double.toString
docs): 
1.0E23.toString() == "9.999999999999999E22"
g
yeah i saw
the tl;dr is that I wish I had used a 
BigDecimal
on my model istead of a 
double
, but im trying to understand the scope of the failures I can see with the use of 
double
. It seems like, perhalps a reasonable intermediate fix is to do a 
double.toString().toDouble
as a kind of validation on user input, and then... do something if 
input != input.toDouble().toString()
given that I cant simply replace the stored type with BigDecimal
i dont understand why so much of the mentissa would be ignored when the exponent is really large
but im willing to write this off as "FP being wierd"
e
Copy code
0.0 == -0.0
(0.0).toString() != (-0.0).toString()
g
yeah and of course negative zero
good ol' sign bit problems
but im not actually sure thats against my question
-0.0
is denotable differently than 
0.0
, so the fact that it gives you different representation doesnt violate the rule
as long as 
-0.0.toString() == "-0.0"
its fine
thanks for your help
e
require(input == input.toDouble().toString())
may force you to enter certain values differently on different JVMs - aside from the bug above, the representation is supposed to be minimal but not necessarily unique
and it would prevent you from entering 10000000.0 without scientific notation
g
i probably wouldnt do it as a requirement i would just make it a filter and replace the user entered value with the one that i get from 
toDouble.toString()
where appropriate
i suppose what might be nicer is a 
DoubleConverter
or 
DecimalFormat
thats a bit more strict about this
so ill look into that
e
DecimalFormat still gets 1e23 wrong since it's built on the same operation
if you just want to keep 14 sigfigs, `.toBigDecimal().round(MathContext(14))`will do that
34 Views
Open in Slack
PreviousNext
Powered by