05/14/2023, 10:54 PM
River-kt 0.0.1-alpha04 was just released! Glad to announce this fresh release, with several enhancements and new features that hopefully will improve your development experience: - HTTP Connector Release: The HTTP connector is now available! You can construct and send HTTP requests using HTTP request extensions. Here's a quick example:
val pageNumberFlow = unboundedLongFlow(startsAt = 1)

val responseFlow: HttpResponse<String> =
	    .map { page ->
	        get("<>") {
	            query("page", "$page")
	    .sendAndHandle(bodyHandler = ofString, parallelism = 10)
	    .takeWhile { it.body() == "[]" }
With this, you can create a pagination flow with a parallelism factor of 10. All responses will be emitted sequentially, stopping the pagination once the first empty list arrives. You may refer to the kdocs for more information. - New Connector in Development #1 - Google Drive: The early version of the Google Drive connector is now available, probably with potential bugs (I have warned!). Initial implementation includes two APIs:
val serviceAccount: JsonNode = // provide your credentials here  

val googleDriveApi = GoogleDriveApi(serviceAccount)

val files: Flow<GoogleDriveFile> = 
		.listFiles {
		    driveId = "x"

val fileContent: Flow<ByteArray> =
- New Connector in Development #2 - GitHub API: I have kick-started the development of a GitHub API connector. The goal here is to enable the manipulation of an entire codebase using Kotlin flows. - New Connector in Development #3 - OpenAI API: All the kids are trying it nowadays, so, why not? While still in its early stages and likely to have bugs, I imagine multiple possibilities such as streaming large amounts of data for model training and building documentation using data from other connectors. - JSON Connector Fixes: Some issues were fixed in the JSON connector where non-trimmed string data was parsed incorrectly. Also, we've refined the function names for better readability and conciseness. Not necessarily related to the version release, but: - Added the first examples: Practical examples showcasing the capabilities of River-kt were added. Currently, we have three examples: JSON streaming (using Ktor as the HTTP server), downloading an CSV file from S3, parsing it, and saving each line to a PostgreSQL database via JDBC, and listening to an SQS queue and forwarding the requests to an HTTP API. More examples are in the oven. - Issues for future development: I have filled the repository with issues to map the future development of other connectors. This will not only help us gather community feedback on the most requested connectors but also provide visibility into ongoing development. Stay tuned for more updates, and thank you for your support!