https://kotlinlang.org logo
Join Slack
Powered by
The kotlin duration is killing me: ```(200.days + ...
# announcements
p
The kotlin duration is killing me:
Copy code
(200.days + 5.nanoseconds - 200.days).toLongNanoseconds()
This evaluates to 
4
, not 
5
.
c
4 nanoseconds?
Hahaha. I was right! I was going to say 5, but then I was like... that's too obvious.
e
What was the problem you were solving that you ended up doing such a computation on durations?
p
Adding duration to a 
LocalDateTime
Copy code
@Test
fun plus() {
  val localDateTime = localDateTime(
    date = localDate(year = 2020, month = 1, dayOfMonth = 2),
    time = localTime(hour = 1, minute = 2, second = 3, nanoOfSecond = 4)
  )
  localDateTime.plus(365.days + 2.days + 2.hours + 5.nanoseconds)
    .shouldBe(
      localDateTime(
        date = localDate(2021, 1, 4),
        time = localTime(1, 4, 3, 9)
      )
    )
}
Here it's an 
expect class LocalDateTime
that has an 
actual typalias
resolving 
java.time.LocalDateTime
e
What kind of business problem is that when you need to add such year-scale durations to a local date time?
(That's what calendar periods are designed to be used for, so that you can find, for example, the same day next year to schedule a repeating annual event)
r
Floating-point arithmetic strikes again... Duration stores its value internally as a 
Double
. So you're basically just running into a FP rounding error when adding the double that represents 200 days with the double that represents 5 nanoseconds.
☝️ 1
👍 1
n
Yeah... That's why no other mainstream language uses double to back its duration type. Blaming the use case here is just incorrect.
p
Yeah the 310 implementation is acutally quite nice regarding this. We now have to always think carefully when we use the nanos of a duration
n
A lot of people brought this point up on the github for this keep (myself included), pointing out other language implementations, etc
e
Pointers to other languages do not count. We need real-life use-cases.
So far, a real-live use-case to precisely represent a duration of 365 days and 5 nanoseconds had eluded us.
n
Pointers to other languages don't count? You don't think maybe those other languages have done research perhaps that backs their decision to use integers?
e
We'd be extremely glad if you'd give us a pointers to such research that itemizes the use-cases they have discovered.
n
I just don't understand why failure to perform a basic date time operation requires a specific business use case...
e
I don't claim we've made a correct or even somehow better choice. Every design choice is always a compromise and we are aware of downsides of the choices we take (but also of downsides of choices we did not take). I'm just saying that so far we've failed to discover real-life use-cases to rule out the choice of double as a carrier type for duration. If we had those, the tradeoff balance would have been different.
n
Something as simple as "loop over every second for the next year and pull some database records", depending how it's written, could run into this issue
Is the benefit basically that by using a single type to back duration, you'll be able to use it with inline class?
e
Our design rule is that the shortest and simplest code must be correct. In every API we work with a longer, more contrived code would run into all sorts of problems.
➕ 2
The other rule is that our design decisions are pragmatic. We don't design for abstract or theoretical reasons. We design for practical, real-life use-case. If we can also imbue our designs with some nice theoretical/algebraic properties without undermining the other aspects that's great, but it is not a goal in itself.
👍 2
n
I don't understand how what I'm suggesting is contrived
e
I don't understand how what you're suggesting is pragmatic. None of the types we use in programming enjoy all of the algebraic properties we've learned in school. They all have their tradeoffs and limits, they all cater to specific use-cases. My misunderstanding is easy to correct by explaining the real-life use-case behind your suggestion.
For every type (especially for every date-time and numeric type) I can say "but this and this does not produce the correct result from my PoV". Which gives us? Nothing. Does it mean the type is broken or useless? No. It means nothing, unless there's a real use-case behind the example.
Also, claiming that 
Int
type is bad because is cannot compute 2 billion + 2 billions without overflowing does not gives much data even with a real-life use-case. An 
Int
type, just like everything in software engineering, is an example a pragmatic tradeoff.
➕ 3
n
Yes, Int is just usually not a good trade-off on a modern machine because there's in most cases no performance penalty for a 64 bit integer
👍 2
❌ 2
The JVM is another story it appears
We know in real life use cases integer overflows happen and they happen more often with 32 bit integers, that's not really debatable
e
Int
still occupies 4 bytes of memory and 
Long
8 byte. Replace every 
Int
with 
Long
inside a modern business app without thinking about and suddenly it consumes more memory for no gain. You also replace all of them with 
BigInteger
and it consumes way more CPU and memory. You still gained nothing of a substance. Do overflows happens and bugs occur because of them? They do! How often do they happen and does getting rid of them balances out? That's a tradeoff.
n
Anyway, you never told me why something as simple as "loop over every second for the next year and pull some database records", depending how it's written, could run into this issue. I don't find it very artificial to multiple one second by a number first, and then add it, and continually multiply it by larger numbers
p
For us the business logic is quite complex. We have a multi platform fasting feature where the difference between dates is calculated and for overlapping fasting periods a nano is substracted. Then we decide to display stuff differently in the ui based on if that nano difference is there. Currently the bug is more of hypothetical nature and will only happen if the user is fasting for more than 200 days. But I still would love my program to be correct.
We could work around it differently but now every Dev needs to carefully think about durations imprecision which wasn't there as Java stores seconds and nanos separately
➕ 3
n
Especially circumstantially e.g. you may be processing data where ever row is a different second, now you can easily loop over the rows with withIndex and multiply index by seconds and add it to the start date time
I think there are third party libraries in Kotlin that do datetime with integer duration, or potentially just use the Java library
p
For us that nano minus represents the ending of the previous LocalDateTime.
n
Write a small handful of utility and extension functions and call it a day (if you're on the JVM)
i
@Paul Woitaschek Have you considered using ranges with the excluded end to represent these periods?
p
My issue isn't that I can't find a workaround
My issue is that there are impressions which affect us. And we're not in the millions of years, it starts at day 200
The fact that there are extensions for nanos on duration implies that it can handle that precision
e
@Paul Woitaschek The way I see is that you have used to having nanosecond precision in Java Duration, but not having it in Kotlin Duration is a surprise. For example, have Java Duration had ms precision, you would not expect it to support adding a few nanos; if you didn't have prior experience with Java Duration you would not have any prior expectations on Kotlin Duration precision and would have designed your code not to rely on it.
➕ 3
p
Then duration should remove the nano functions
i
I don't find it very artificial to multiple one second by a number first, and then add it, and continually multiply it by larger numbers
@Nir If you operate with whole seconds, Duration provides the exact representation for +-146 years range
n
Who said I was operating with whole seconds?
Every second, for a year, with nanosecond precision
e
@Paul Woitaschek Actually, one of the design goals was ability to represent a sub-nano durations, so that you can run your benchmark, measure its time, divide duration by the number of iterations and get a result that is not rounded to nanos, but more precise -- something Java Duration cannot do.
n
@elizarov are there are any business examples of people that need picosecond accuracy but are willing to trust the vagaries of floating point?
I'd like to meet that person
e
@Nir I've just explained it. We look at sub-nano benchmark results all the time.
n
you're just talking about printing at the end of a benchmark.... literally just 
prinln(picoseconds: duration.nanoseconds.toDouble() / 1000)
e
@Nir We use JMH for benchmarking. Natuarally, it does not use Java Duration under the hood to present its results, but does it in floating point.
n
this isn't a "real" use case to be blunt
😶 1
e
It is as real as it can be, because it is a part of the actual software.
n
it's a one liner at the end of the software
e
Just like @Paul Woitaschek’s case is real.
n
paul's case is a systematic problem that can cause hard to find bugs throughout the entire codebase
e
With good API all use-cases are one-liners
K 1
n
come on, are you really comparing here
You aren't doing any duration operations with picoseconds in your example. It's just display. This is not the same thing at all.
I can tell you that in multiple of the domains where high accuracy time keeping is essential, through multipel of these domains and multiple of these languages, companies for instance that have written their own datetime (in cases where it predated standardization) choose integers again, and again, and again.
i
@Nir I'm not sure I get what you meant by "Every second, for a year, with nanosecond precision". You said you multiplied an integer index by seconds. This implies that you're operating with whole seconds.
n
The original question was:
What kind of business problem is that when you need to add such year-scale durations to a local date time?
this is an example where you could easily end up adding a year scale duration to a local date time
i
There's no problem if that duration is a multiple of whole seconds.
n
I'm actually not sure if the problem shows up in addition; Roman said "add" but in the original problem it's subtraction that causes the issue.
sure, but it's easy enough now to start adding modifications to the problem where they are not
for example you may have data within one second bins that a vendor stored with offsets from the edge of the second bin, in nanoseconds (to save space as fewer bits suffices, for instance). Now if you want to calculate datetimes of those offsets, you still have the issue.
l
Wild idea to solve this fight/misunderstanding regarding the expected precision abilities of 
Duration
of kotlin.time: Introduce 
BigDuration
(following naming of 
BigInteger
) that uses only integers and supports arbitrary precision, at the expense of memory consumption. Then it's up to the programmers to pick the right for the use case they have at hand, hoping they never need to hold zillions of 
BigDuration
instances in memory. I think a 
BigDuration
would make a lot of sense since the loss of precision of 
Duration
arises as the duration increases (e.g. you lose seconds precision after over 146 years long duration, ~200 days for nanoseconds, and probably sooner for picoseconds or tinier). This, plus proper documentation in 
Duration
doc about its limits and in 
BigDuration
about its memory and CPU impact would probably make a lot of sense and make Kotlin cover all the use cases about durations. Then, we could get IDE warnings when mixing small precision with big durations when it can be checked, as is the case for the original snippet in this thread where you have 5 nanos and 200 days mixed. Actually, the possible loss of precision could be checked statically if only constants are used (though adding parameters would make things a little more complex and require to extract sub-expressions that can be analyzed with actual values).
➕ 2
6 Views
Open in Slack
PreviousNext
Powered by