https://kotlinlang.org logo
m

Mike Conley

03/17/2020, 6:09 AM
depending on what
foo
actually does
a

Alessandro Tagliapietra

03/17/2020, 6:14 AM
yeah and also I've just checked, is still doesn't work (it makes you do
is Iterable<*>
) which makes sense
m

Mike Conley

03/17/2020, 6:15 AM
yeah, you can't do type checks on generic type arguments like that for the same reason
the runtime doesn't see the
A
in
Iterable<A>
and
is
checks happen at runtime
a

Alessandro Tagliapietra

03/17/2020, 6:16 AM
Yup, mmhh, should I like, get the first element of the iterable and check there?
m

Mike Conley

03/17/2020, 6:16 AM
let's talk about what foo actually does first
specifically, why do you need to know the types of the things in your iterable?
this is probably something you can design your code around pretty easily.
a

Alessandro Tagliapietra

03/17/2020, 6:17 AM
• depending on T, uses a different variable (kafka producer in my case) to start a transaction • iterates over the iterable and sends each element to kafka • using the variable in point 1, commits the transaction
you mean, calling other functions that has non erased types?
m

Mike Conley

03/17/2020, 6:18 AM
what are the possible values for
T
here?
a

Alessandro Tagliapietra

03/17/2020, 6:19 AM
three different classes
Metric, State, Event
m

Mike Conley

03/17/2020, 6:19 AM
yours or someone elses?
a

Alessandro Tagliapietra

03/17/2020, 6:19 AM
ours
(avro generated so with limited amount of customization)
m

Mike Conley

03/17/2020, 6:19 AM
ok, so
do they implement some common interface or have some common subclass so you can constrain you generics?
(I don't know anything about avro, and very little about kafka, so apologies in advance there)
a

Alessandro Tagliapietra

03/17/2020, 6:20 AM
they implement
org.apache.avro.specific.SpecificRecord
m

Mike Conley

03/17/2020, 6:20 AM
ok
a

Alessandro Tagliapietra

03/17/2020, 6:20 AM
avro generates java classes from a json like structure, like google's protobuf basically
m

Mike Conley

03/17/2020, 6:20 AM
ahh, ok
and you need different transaction types based on the type of the thing, or something along those lines
a

Alessandro Tagliapietra

03/17/2020, 6:22 AM
yeah I have 3 producers (variables basically), and on Metric I have to do metricProducer.startTransaction(), on Event I have to do eventProducer.startTransaction() and so on
m

Mike Conley

03/17/2020, 6:22 AM
ok.
my first thought is: why are all three of these getting dumped into one
Iterable
-- separate into three?
a

Alessandro Tagliapietra

03/17/2020, 6:23 AM
no it's just that the code inside is the same
only the variable used changes
I could call it fooMetric, fooState, fooEvent
I was just trying to make it nicer as the code is exactly the same, it just changes the variable used
m

Mike Conley

03/17/2020, 6:23 AM
well, you wouldn't need to if you also passed that second variable in, right?
foo(stuff: Iterable<Metric>, metricProducer)
a

Alessandro Tagliapietra

03/17/2020, 6:24 AM
right, but this is the public API, outside classes dont' know about the producer
m

Mike Conley

03/17/2020, 6:24 AM
ahh, ok
a

Alessandro Tagliapietra

03/17/2020, 6:24 AM
it's more like "take these records, ingest them"
m

Mike Conley

03/17/2020, 6:24 AM
gotcha. ok, then you're kinda stuck with it 🙂
a

Alessandro Tagliapietra

03/17/2020, 6:25 AM
well, I can make three functions
I was just trying to find a cleaner solution
m

Mike Conley

03/17/2020, 6:25 AM
this is a method on a class that holds the three producers, right?
a

Alessandro Tagliapietra

03/17/2020, 6:26 AM
holds as, has those as private variables? yes
m

Mike Conley

03/17/2020, 6:26 AM
probably your best bet is a private function to get the producer for each object as you iterate over it
and probably that function will check the runtime type of each object
something like
a

Alessandro Tagliapietra

03/17/2020, 6:27 AM
it's like
Copy code
private var stateProducer = KafkaProducer<String, State>(Settings.getProducer("api-states"))

and then

stateProducer.send(ProducerRecord("states", machineId, stateRecord))
so KafkaProducer basically accepts any kind of value, and will throw a runtime exception if it's not serializable
m

Mike Conley

03/17/2020, 6:28 AM
Copy code
private fun producerFor(thing: SpecificRecord): ProducerType { 
    val klass = thing::class
    when (klass) {
        is Metric -> metricProducer,
        is Event -> eventProducer,
        ...
}
a

Alessandro Tagliapietra

03/17/2020, 6:28 AM
yeah I could do that
as that will always return a KafkaProducer
however, to get the "thing" I'll need to get the first element of the iterable right?
m

Mike Conley

03/17/2020, 6:29 AM
seems reasonable enough.
well, you'd do that for each element
foo looks like
Copy code
fun foo(stuff: Iterable<SpecificRecord>) {
stuff.forEach {
   val producer = producerFor(it)
   // do stuff with producer and it
}
}
sorry about the formatting.
a

Alessandro Tagliapietra

03/17/2020, 6:29 AM
problem is, that'll result like
Copy code
...each(
  producer = producerFot(it)
  producer.startTransaction()
  producer.send(it)
  producer.commitTransaction()
)
m

Mike Conley

03/17/2020, 6:30 AM
yup
a

Alessandro Tagliapietra

03/17/2020, 6:30 AM
however, I already have such a function
m

Mike Conley

03/17/2020, 6:30 AM
what's the code look like now?
a

Alessandro Tagliapietra

03/17/2020, 6:30 AM
and I want
Copy code
producer.startTransaction()
list.each(
  producer.send(it)
)
producer.commitTransaction()
otherwise the overhead of the transaction start/commit makes it useless to be able to process lists
m

Mike Conley

03/17/2020, 6:31 AM
I thought you had multiple producers based on type?
a

Alessandro Tagliapietra

03/17/2020, 6:31 AM
and I could just make the foreach in the caller
m

Mike Conley

03/17/2020, 6:31 AM
sure, sure
a

Alessandro Tagliapietra

03/17/2020, 6:31 AM
yes but the type within a single iterator is always the same
m

Mike Conley

03/17/2020, 6:31 AM
even when there are multiple types in the iterator?
or do you always have iterators containing exactly one type
?
a

Alessandro Tagliapietra

03/17/2020, 6:32 AM
there aren't, I have multiple endpoints, one for each type in the API so they won't mix
m

Mike Conley

03/17/2020, 6:32 AM
ok, ok, I misunderstood
a

Alessandro Tagliapietra

03/17/2020, 6:32 AM
yes but the type within a single iterator is always the same
^ that's what I meant by this sorry
m

Mike Conley

03/17/2020, 6:32 AM
yeah, gotcha.
I would probably go with 3 functions, one for each of the types you're dealing with (just because you get better safety that way -- you can't accidentally mix them, because the compiler will catch it)
a

Alessandro Tagliapietra

03/17/2020, 6:35 AM
I see, probably the easiest way too
m

Mike Conley

03/17/2020, 6:35 AM
they can call a common private function passing the list and appropriate producer. a bit of boilerplate, but not too bad.
a

Alessandro Tagliapietra

03/17/2020, 6:36 AM
I'm also trying to find a way to simplify these functions
right now I have in 6 functions the same code
Copy code
try {
            stateProducer.beginTransaction()
                     stateProducer.send(ProducerRecord("states", machineId, state))
            stateProducer.commitTransaction()
        } catch (e: ProducerFencedException) {
            stateProducer.close()
            recreateStateProducer()
        } catch (e: OutOfOrderSequenceException) {
            stateProducer.close()
            recreateStateProducer()
        } catch (e: AuthorizationException) {
            stateProducer.close()
            recreateStateProducer()
        } catch (e: KafkaException) {
            stateProducer.abortTransaction()
        }
so I was thinking to make a function where I can pass a block
that wraps the block in a transaction
m

Mike Conley

03/17/2020, 6:37 AM
sounds like a reasonable plan
a

Alessandro Tagliapietra

03/17/2020, 6:37 AM
anyway, thanks a lot for helping!
m

Mike Conley

03/17/2020, 6:37 AM
sure thing. gonna go to bed now. it's late here!
a

Alessandro Tagliapietra

03/17/2020, 6:37 AM
learned a new thing today as well
👍 1
oh yeah I see, I've still a few hours
👋
m

Mike Conley

03/17/2020, 6:38 AM
good luck -- I'll be back around tomorrow if you're still working on it. 👋
a

Alessandro Tagliapietra

03/17/2020, 6:38 AM
It has to be done by tomorrow 😄 so tonight whatever it takes I have to deliver it
m

Mike Conley

03/17/2020, 6:41 AM
oof 😞
c

cedric

03/17/2020, 8:06 PM
@Alessandro Tagliapietra, with a little magic, you could cut this down to:
Copy code
producer.transaction {
    list.each {
        producer.send(this)
    }
}
and that would automatically start and end the transaction.
a

Alessandro Tagliapietra

03/17/2020, 8:06 PM
@cedric can you share the magic? 🙂 What's producer in this case?
right now I've created a function to wrap the actual code in a transaction
m

Mike Conley

03/17/2020, 8:08 PM
I'm also curious about the magic
c

cedric

03/17/2020, 8:09 PM
Something like
Copy code
class Producer {
    fun transaction(lambda: () -> Boolean) {
        startTransaction()
        lambda()
        commitTransaction()
    }
}
Add
try/catch
, etc... but you get the idea
a

Alessandro Tagliapietra

03/17/2020, 8:15 PM
oh yeah but Producer isn't my class
c

cedric

03/17/2020, 8:15 PM
Write an extension function.
m

Mike Conley

03/17/2020, 8:15 PM
extension functions are a thing
c

cedric

03/17/2020, 8:15 PM
Copy code
fun Producer.transaction(...) {
a

Alessandro Tagliapietra

03/17/2020, 8:16 PM
Copy code
private fun <K, V> wrapInTransaction(producer: KafkaProducer<K, V>, lambda: (producer: KafkaProducer<K, V>) -> Unit) {
    try {
        producer.beginTransaction()
        lambda(producer)
        producer.commitTransaction()
    } catch (e: ProducerFencedException) {
        eventProducer.close()
    } catch (e: OutOfOrderSequenceException) {
        eventProducer.close()
    } catch (e: AuthorizationException) {
        eventProducer.close()
    } catch (e: KafkaException) {
       eventProducer.abortTransaction()
    }
}
oh that's nice
I'll give it a try, thanks! 😄
m

Mike Conley

03/17/2020, 8:16 PM
in truth, it really just moves the argument list around, but it can be pretty handy regardless 🙂
c

cedric

03/17/2020, 8:17 PM
Personally, if I see at least two
Copy code
foo.x()
foo.y()
I introduce an
apply
I only want to read
foo
once. DRY!
m

Mike Conley

03/17/2020, 8:18 PM
I like the rule of threes, but it's circumstantial.
c

cedric

03/17/2020, 8:18 PM
Fine, I can compromise on that 🙂
m

Mike Conley

03/17/2020, 8:18 PM
heh
a

Alessandro Tagliapietra

03/17/2020, 8:21 PM
I was just going to ask you what you meant with
empty
m

Mike Conley

03/17/2020, 8:22 PM
did it say
empty
? because I read
apply
(before the edit)
might be losing my mind -- it's been a long day 🙂
a

Alessandro Tagliapietra

03/17/2020, 8:22 PM
now you make me doubt what I saw 😕
c

cedric

03/17/2020, 8:22 PM
Copy code
foo.x()
foo.y()
// becomes
foo.apply {
  x()
  y()
}
a

Alessandro Tagliapietra

03/17/2020, 8:22 PM
not sure, however from extension functions I can't call private methods do I?
m

Mike Conley

03/17/2020, 8:23 PM
I usually do
Copy code
with(foo) {
  x()
  y()
}
probably another matter of taste
you can't call private methods, no
c

cedric

03/17/2020, 8:23 PM
Oops you're right, should have been
with
m

Mike Conley

03/17/2020, 8:23 PM
extension functions don't break encapsulation
a

Alessandro Tagliapietra

03/17/2020, 8:24 PM
ok
4 Views