https://kotlinlang.org logo
Join Slack
Powered by
depending on what `foo` actually does
# announcements
m
depending on what 
foo
actually does
a
yeah and also I've just checked, is still doesn't work (it makes you do 
is Iterable<*>
) which makes sense
m
yeah, you can't do type checks on generic type arguments like that for the same reason
the runtime doesn't see the 
A
in 
Iterable<A>
and 
is
checks happen at runtime
a
Yup, mmhh, should I like, get the first element of the iterable and check there?
m
let's talk about what foo actually does first
specifically, why do you need to know the types of the things in your iterable?
this is probably something you can design your code around pretty easily.
a
• depending on T, uses a different variable (kafka producer in my case) to start a transaction • iterates over the iterable and sends each element to kafka • using the variable in point 1, commits the transaction
you mean, calling other functions that has non erased types?
m
what are the possible values for 
T
here?
a
three different classes
Metric, State, Event
m
yours or someone elses?
a
ours
m
ok, so
a
(avro generated so with limited amount of customization)
m
do they implement some common interface or have some common subclass so you can constrain you generics?
(I don't know anything about avro, and very little about kafka, so apologies in advance there)
a
they implement 
org.apache.avro.specific.SpecificRecord
m
ok
a
avro generates java classes from a json like structure, like google's protobuf basically
m
ahh, ok
and you need different transaction types based on the type of the thing, or something along those lines
a
yeah I have 3 producers (variables basically), and on Metric I have to do metricProducer.startTransaction(), on Event I have to do eventProducer.startTransaction() and so on
m
ok.
my first thought is: why are all three of these getting dumped into one 
Iterable
-- separate into three?
a
no it's just that the code inside is the same
only the variable used changes
I could call it fooMetric, fooState, fooEvent
I was just trying to make it nicer as the code is exactly the same, it just changes the variable used
m
well, you wouldn't need to if you also passed that second variable in, right?
foo(stuff: Iterable<Metric>, metricProducer)
a
right, but this is the public API, outside classes dont' know about the producer
m
ahh, ok
a
it's more like "take these records, ingest them"
m
gotcha. ok, then you're kinda stuck with it 🙂
a
well, I can make three functions
I was just trying to find a cleaner solution
m
this is a method on a class that holds the three producers, right?
a
holds as, has those as private variables? yes
m
probably your best bet is a private function to get the producer for each object as you iterate over it
and probably that function will check the runtime type of each object
something like
a
it's like
Copy code
private var stateProducer = KafkaProducer<String, State>(Settings.getProducer("api-states"))

and then

stateProducer.send(ProducerRecord("states", machineId, stateRecord))
so KafkaProducer basically accepts any kind of value, and will throw a runtime exception if it's not serializable
m
Copy code
private fun producerFor(thing: SpecificRecord): ProducerType { 
    val klass = thing::class
    when (klass) {
        is Metric -> metricProducer,
        is Event -> eventProducer,
        ...
}
a
yeah I could do that
as that will always return a KafkaProducer
however, to get the "thing" I'll need to get the first element of the iterable right?
m
seems reasonable enough.
well, you'd do that for each element
foo looks like
Copy code
fun foo(stuff: Iterable<SpecificRecord>) {
stuff.forEach {
   val producer = producerFor(it)
   // do stuff with producer and it
}
}
sorry about the formatting.
a
problem is, that'll result like
Copy code
...each(
  producer = producerFot(it)
  producer.startTransaction()
  producer.send(it)
  producer.commitTransaction()
)
m
yup
a
however, I already have such a function
m
what's the code look like now?
a
and I want
Copy code
producer.startTransaction()
list.each(
  producer.send(it)
)
producer.commitTransaction()
otherwise the overhead of the transaction start/commit makes it useless to be able to process lists
m
I thought you had multiple producers based on type?
a
and I could just make the foreach in the caller
m
sure, sure
a
yes but the type within a single iterator is always the same
m
even when there are multiple types in the iterator?
or do you always have iterators containing exactly one type
?
a
there aren't, I have multiple endpoints, one for each type in the API so they won't mix
m
ok, ok, I misunderstood
a
yes but the type within a single iterator is always the same
^ that's what I meant by this sorry
m
yeah, gotcha.
I would probably go with 3 functions, one for each of the types you're dealing with (just because you get better safety that way -- you can't accidentally mix them, because the compiler will catch it)
a
I see, probably the easiest way too
m
they can call a common private function passing the list and appropriate producer. a bit of boilerplate, but not too bad.
a
I'm also trying to find a way to simplify these functions
right now I have in 6 functions the same code
Copy code
try {
            stateProducer.beginTransaction()
                     stateProducer.send(ProducerRecord("states", machineId, state))
            stateProducer.commitTransaction()
        } catch (e: ProducerFencedException) {
            stateProducer.close()
            recreateStateProducer()
        } catch (e: OutOfOrderSequenceException) {
            stateProducer.close()
            recreateStateProducer()
        } catch (e: AuthorizationException) {
            stateProducer.close()
            recreateStateProducer()
        } catch (e: KafkaException) {
            stateProducer.abortTransaction()
        }
so I was thinking to make a function where I can pass a block
that wraps the block in a transaction
m
sounds like a reasonable plan
a
anyway, thanks a lot for helping!
m
sure thing. gonna go to bed now. it's late here!
a
learned a new thing today as well
👍 1
oh yeah I see, I've still a few hours
👋
m
good luck -- I'll be back around tomorrow if you're still working on it. 👋
a
It has to be done by tomorrow 😄 so tonight whatever it takes I have to deliver it
m
oof 😞
c
@Alessandro Tagliapietra, with a little magic, you could cut this down to:
Copy code
producer.transaction {
    list.each {
        producer.send(this)
    }
}
and that would automatically start and end the transaction.
a
@cedric can you share the magic? 🙂 What's producer in this case?
right now I've created a function to wrap the actual code in a transaction
m
I'm also curious about the magic
c
Something like
Copy code
class Producer {
    fun transaction(lambda: () -> Boolean) {
        startTransaction()
        lambda()
        commitTransaction()
    }
}
Add 
try/catch
, etc... but you get the idea
a
oh yeah but Producer isn't my class
c
Write an extension function.
m
extension functions are a thing
c
Copy code
fun Producer.transaction(...) {
a
Copy code
private fun <K, V> wrapInTransaction(producer: KafkaProducer<K, V>, lambda: (producer: KafkaProducer<K, V>) -> Unit) {
    try {
        producer.beginTransaction()
        lambda(producer)
        producer.commitTransaction()
    } catch (e: ProducerFencedException) {
        eventProducer.close()
    } catch (e: OutOfOrderSequenceException) {
        eventProducer.close()
    } catch (e: AuthorizationException) {
        eventProducer.close()
    } catch (e: KafkaException) {
       eventProducer.abortTransaction()
    }
}
oh that's nice
I'll give it a try, thanks! 😄
m
in truth, it really just moves the argument list around, but it can be pretty handy regardless 🙂
c
Personally, if I see at least two
Copy code
foo.x()
foo.y()
I introduce an 
apply
I only want to read 
foo
once. DRY!
m
I like the rule of threes, but it's circumstantial.
c
Fine, I can compromise on that 🙂
m
heh
a
I was just going to ask you what you meant with 
empty
m
did it say 
empty
? because I read 
apply
(before the edit)
might be losing my mind -- it's been a long day 🙂
a
now you make me doubt what I saw 😕
c
Copy code
foo.x()
foo.y()
// becomes
foo.apply {
  x()
  y()
}
a
not sure, however from extension functions I can't call private methods do I?
m
I usually do
Copy code
with(foo) {
  x()
  y()
}
probably another matter of taste
you can't call private methods, no
c
Oops you're right, should have been 
with
m
extension functions don't break encapsulation
a
ok
10 Views
Open in Slack
PreviousNext
Powered by