Are you a KotlinJS enthusiastic and would like to ...

# javascript

[Danilo Reinert]

Are you a KotlinJS enthusiastic and would like to integrate a multiplatform open source project to make the KotlinJS ecosystem even better?

I'm sharing the new v1.1 release of the requestor library, an HTTP Client API that unlocks the development of communication-centric frontend apps - a sustainable software design for micro clients. The lib’s core is written in vanilla Java with functional programming as a first class citizen, and it's designed to be compatible with any underlying platform, like jvm or the browser. Currently, there are two requestor impls: requestor-javanet for Jvm/Android and requestor-gwt for gwt2.

Now we're looking to provide a new impl for KotlinJS: requestor-kotlinjs. The path to get there is really simple and already planned. If you're a kotlinjs developer and would like to learn and contribute to the ecosystem with a modern library for HTTP communication, check out the project at https://github.com/reinert/requestor and get in touch with me here or at the project’s chat room to know how to contribute.

[Oliver.O]

Could you explain where the Kotlin ecosystem would benefit from requestor? Comparing this to Ktor client, for example, it seems like we'd lose coroutines and structured concurrency. We'd also lose the simple API style provided by DSLs and lambdas. What did I miss?

[Danilo Reinert]

Hi @Oliver.O, thanks for replying! Any ecosystem will certainly benefit from alternative solutions to common problems with different emphasis or perspectives. I'm not a kotlin expert (that's why I'm making this call) but I'll try to address your points with some examples.

Coroutines - requestor-core abstracts the async task scheduling which must be implemented by requestor impls. The requestor-javanet impl - that targets jvm/android - implements it using a scheduler executor interface (which we can use either a thread pool impl or a virtual thread impl); requestor-gwt - that targets the js runtime - implements the task scheduler using setTimeout. This is is an option for kotlinjs. Alternatively, we could implement the task scheduler in kotlin using a coroutine builder like launch. This last option may be better since it would support both kotlinjs and kotlin native.

Simple API style - that's indeed one of the goals of the requestor project: providing a simple and fluent API for requesting. In this regard, functional programming is treated as a first class citizen. Thus, almost every feature in requestor is exposed through functional interfaces so the user can leverage the most of lambdas. For example, check how we can make a simple GET request using requestor and receive the response with optional args (no-args, response payload (deserialized body), response, and request respectively)

// Recommended to use as a singleton
val session = Requestor.newSession()

// Make a simple GET request deserializing the payload as String
session.get("<https://httpbin.org/ip>", String::class.java)
    .onSuccess { -> println("request was successful") } // no-arg callback
    .onSuccess { ip -> println(ip) } // set a callback accessing the response payload
    .onSuccess { ip, res -> println(res.status) } // access the payload and the response
    .onSuccess { ip, res, req -> println(req.uri) } // access payload, response, and request

Also notice we can attach as many lambda callbacks as we need, separating them according to the specific outcome we want to handle, what makes our code highly readable.

<http://session.post|session.post>("/endpoint")
    .onSuccess { -> println("response was 2xx") }
    .onFail { res -> println("response was ${res.statusCode}") }
    .onStatus(429) { res -> println(res.statusCode == 429) }
    .onTimeout { e -> println("request timed out in ${e.timeoutMillis}ms") }
    .onCancel { e -> println("request was interrupted due to ${e.message}") }

Benefits of using requestor - there are many since requestor-core provides a lot of features related to communication (both network and between app components), but I'll try to list a few of them.

Fundamentally, requestor can be used as a lean integration point for your whole application, allowing you to build uncoupled interconnected middleware components and end user interfaces (graphical or apis). On top of this precept, requestor was designed to support common requirements related to http communication that are often hard or overwhelming to implement.

For example, enabling a connection polling while requesting is done with a simple command like below:

// Set a Long Polling request each 5s
session.req("<https://httpbin.org/ip>")
    .poll(PollingStrategy.LONG, 5_000) // Set the polling option
    .get(String::class.java) // Call GET deserializing the payload as String
    .onSuccess { ip -> println(ip) } // Executed on each successful response
    .onSuccess { _, _, req -> if (req.pollingCount == 5) req.stopPolling() } // Stop polling after 5 requests

// Set a Short Polling each 5s up to 10 requests
session.req("<https://httpbin.org/ip>")
    .poll(PollingStrategy.SHORT, 5_000, 10)
    .get(String::class.java)
    .onSuccess { ip -> println(ip) }

Another common but complex feature is HTTP Streaming. This is easily done like below:

val os = FileOutputStream(Files.createTempFile("file", "tmp").toFile())

session.req("/api/download")
    .save(Requestor.READ_CHUNKING_ENABLED, true) // Enable read chunking (a.k.a. streaming) on the request
    .get()
    .onRead { progress -> os.write(progress.chunk.asBytes()) } // Write each chunk of bytes as soon they're received into the OS
    .onLoad(os::close) // Close the OS when the request finishes (response completely received)
    .onError(os::close) // Close the OS when the request crashes (got no response)

Other good example of a complex feature made simple is the *_*retry*_* option.

// Set the request to retry up to three times on 'timeout' or '429' responses
// The first retry is done in 1s, the second in 2s, and the third in 4s
session.req("/api/may-fail")
    .retry( DelaySequence.fixed(1, 2, 4), RequestEvent.TIMEOUT, Status.TOO_MANY_REQUESTS )
    .post()
    .onTimeout { _ -> println("executed only after three retries") }
    .onStatus(429) { _ -> println("executed only after three retries") }

Besides these three examples of complex features made simple by design, requestor provides a fine-grained request-response processing cycle in which we can make pre and post request processing at different milestones. All processors are async, so we can perform IO bound operations before submitting a request or after receiving a response but before returning to the user.

You can check more about this in the Processors section.

Other features requestor-core provides are authentication flows, multi content-type serialization, compression, caching, header and links interaction, futures and await (sync requesting flow), etc.

There are other advanced topics that deserves technical writing like how to use requestor Sessions and Services to structure a frontend app.

Finally, requestor-core is implemented in vanilla java (types and collections only) thinking in interoperability. So it's multiplatform and we can use the same API for different target runtimes like jvm, android, gwt, j2cl, kotlin and so forth.

Again, thank you for replying and feel free to ask me more. I'd love to answer other questions you might have.

[Oliver.O]

Thanks for the detailed explanations. In the Kotlin ecosystem, Ktor provides multiplatform HTTP client and server infrastructure. On the client side, things would look like this, for example when retrying with dynamic timeouts:

val baseTimeout: Duration = ... 

val httpClient = HttpClient(CIO) {
    install(HttpTimeout)
}

val response = httpClient.get("<https://somewhere.example.com/this/api/>...") {
    headers {
        append(HttpHeaders.Accept, "application/json")
    }
    timeout {
        socketTimeoutMillis = baseTimeout.inWholeMilliseconds * (retryCount + 1)
        connectTimeoutMillis = socketTimeoutMillis
    }
}

Also, we have multiplatform reflection-less serialization and deserialization via the kotlinx.serialization library and a compiler plugin.

Beyond async scheduling, coroutines offer structured concurrency, which means you write asynchronous code almost like you write sequential code, which includes exception handling.

It seems that you are about to duplicate much of the infrastructure that already exists (for multiple platforms) in the Kotlin ecosystem. Maybe worth to look more closely at what's already there.

[Danilo Reinert]

Hey @Oliver.O, thanks again for your response. A few more examples...

Setting an exponential backoff retry policy with random jitter up to 3 retries:

session.req("/endpoint")
    .header(AcceptHeader("application/json"))
    .timeout(baseTimeout)
    .retry { attempt -> if (attempt.retryCount < 3) (attempt.retryCount * baseTimeout) + (1..1000).random() else -1 }
    .get()

Requesting in a sync flow is also possible with the await() function:

try {
    // holds until the response is completely received
    val res = session.get("/endpoint").await()
    handleResponse(res)
} catch (e: Exception) {
    handleError(e)
}

You can even get early access to the response headers as soon they are received, before the body is available, with the response future:

try {
    // holds until the response *header* is received
    val response = session.get("/endpoint").response.get()
    handleHeaders(response.headers)

    // holds until the response *body* is received
    val payload = response.serializedPayload.get()
    handleBody(payload)
} catch (e: Exception) {
    handleError(e)
}

As for serialization, the same way you use kotlinx.serialization with ktor, you can use it with requestor to have reflection-less serialization. Requestor was designed to support any serialization mechanism under the hood (there are three integrations currently).

(I'm curious how could we do long/short polling and http streaming using ktor)

Anyway, my point here is not to compare existing options, in an effort to invalidate alternatives and publicly elect the "#1 swiss knife solution for everything". People can do it particularly based on their own bias.

As I said, any ecosystem will benefit from alternative solutions for common problems. There's react, angular, or vue for web app development. They all solve common problems with different approaches, having a broad intersection of similar features and a few exclusive ones. Selecting one of them is a matter of analyzing which best suits your requirements (even taste).

I'm here to share knowledge and information in a democrat and (hopefully) welcoming and respectful space (as any dev community should be) and also to invite good people that would like to get involved in open source to join the project and contribute. The doors are open to anyone interested.

Having questions about the library itself, please let me know and I'll try my best to contribute.

[Oliver.O]

@Danilo Reinert That's absolutely fine with me if you are aware of existing stuff and decide to offer an alternative. I was just trying to point out the overlap. 🙂

[spierce7]

So if you reproduce the same logic across multiple platforms for a library, there is always the chance that things aren’t perfectly aligned between platforms.

Ktor does a good job of mitigating this problem by sharing the majority of its code between platforms, so you don’t have the above problem, and you also one person can write the entire library. Platform specific logic lies in the Engine. This is a better approach.

I think if you want to make this library multiplatform, the first step would be to re-write the Java client library in Kotlin, and move as much of this as possible to the Kotlin common layer so it can be compiled to all platforms.