https://kotlinlang.org logo
Channels
100daysofcode
100daysofkotlin
100daysofkotlin-2021
advent-of-code
aem
ai
alexa
algeria
algolialibraries
amsterdam
android
android-architecture
android-databinding
android-studio
androidgithubprojects
androidthings
androidx
androidx-xprocessing
anime
anko
announcements
apollo-kotlin
appintro
arabic
argentina
arkenv
arksemdevteam
armenia
arrow
arrow-contributors
arrow-meta
ass
atlanta
atm17
atrium
austin
australia
austria
awesome-kotlin
ballast
bangladesh
barcelona
bayarea
bazel
beepiz-libraries
belgium
benchmarks
berlin
big-data
books
boston
brazil
brikk
budapest
build
build-tools
bulgaria
bydgoszcz
cambodia
canada
carrat
carrat-dev
carrat-feed
chicago
chile
china
chucker
cincinnati-user-group
cli
clikt
cloudfoundry
cn
cobalt
code-coverage
codeforces
codemash-precompiler
codereview
codingame
codingconventions
coimbatore
collaborations
colombia
colorado
communities
competitive-programming
competitivecoding
compiler
compose
compose-android
compose-desktop
compose-hiring
compose-ios
compose-mp
compose-ui-showcase
compose-wear
compose-web
confetti
connect-audit-events
corda
cork
coroutines
couchbase
coursera
croatia
cryptography
cscenter-course-2016
cucumber-bdd
cyprus
czech
dagger
data2viz
databinding
datascience
dckotlin
debugging
decompose
decouple
denmark
deprecated
detekt
detekt-hint
dev-core
dfw
docs-revamped
dokka
domain-driven-design
doodle
dsl
dublin
dutch
eap
eclipse
ecuador
edinburgh
education
effective-kotlin
effectivekotlin
emacs
embedded-kotlin
estatik
event21-community-content
events
exposed
failgood
fb-internal-demo
feed
firebase
flow
fluid-libraries
forkhandles
forum
fosdem
fp-in-kotlin
framework-elide
freenode
french
fritz2
fuchsia
functional
funktionale
gamedev
ge-kotlin
general-advice
georgia
geospatial
german-lang
getting-started
github-workflows-kt
glance
godot-kotlin
google-io
gradle
graphic
graphkool
graphql
graphql-kotlin
graviton-browser
greece
grpc
gsoc
gui
hackathons
hacktoberfest
hamburg
hamkrest
helios
helsinki
hexagon
hibernate
hikari-cp
hire-me
hiring
hongkong
hoplite
http4k
hungary
hyderabad
image-processing
india
indonesia
inkremental
intellij
intellij-plugins
intellij-tricks
internships
introduce-yourself
io
ios
iran
israel
istanbulcoders
italian
jackson-kotlin
jadx
japanese
jasync-sql
java-to-kotlin-refactoring
javadevelopers
javafx
javalin
javascript
jdbi
jhipster-kotlin
jobsworldwide
jpa
jshdq
juul-libraries
jvm-ir-backend-feedback
jxadapter
k2-early-adopters
kaal
kafka
kakao
kalasim
kapt
karachi
karg
karlsruhe
kash_shell
kaskade
kbuild
kdbc
kgen-doc-tools
kgraphql
kinta
klaxon
klock
kloudformation
kmdc
kmm-español
kmongo
knbt
knote
koalaql
koans
kobalt
kobweb
kodein
kodex
kohesive
koin
koin-dev
komapper
kondor-json
kong
kontent
kontributors
korau
korean
korge
korim
korio
korlibs
korte
kotest
kotest-contributors
kotless
kotlick
kotlin-asia
kotlin-beam
kotlin-by-example
kotlin-csv
kotlin-data-storage
kotlin-foundation
kotlin-fuel
kotlin-in-action
kotlin-inject
kotlin-latam
kotlin-logging
kotlin-multiplatform-contest
kotlin-mumbai
kotlin-native
kotlin-pakistan
kotlin-plugin
kotlin-pune
kotlin-roadmap
kotlin-samples
kotlin-sap
kotlin-serbia
kotlin-spark
kotlin-szeged
kotlin-website
kotlinacademy
kotlinbot
kotlinconf
kotlindl
kotlinforbeginners
kotlingforbeginners
kotlinlondon
kotlinmad
kotlinprogrammers
kotlinsu
kotlintest
kotlintest-devs
kotlintlv
kotlinultimatechallenge
kotlinx-datetime
kotlinx-files
kotlinx-html
kotrix
kotson
kovenant
kprompt
kraph
krawler
kroto-plus
ksp
ktcc
ktfmt
ktlint
ktor
ktp
kubed
kug-leads
kug-torino
kvision
kweb
lambdaworld_cadiz
lanark
language-evolution
language-proposals
latvia
leakcanary
leedskotlinusergroup
lets-have-fun
libgdx
libkgd
library-development
lincheck
linkeddata
lithuania
london
losangeles
lottie
love
lychee
macedonia
machinelearningbawas
madrid
malaysia
mathematics
meetkotlin
memes
meta
metro-detroit
mexico
miami
micronaut
minnesota
minutest
mirror
mockk
moko
moldova
monsterpuzzle
montreal
moonbean
morocco
motionlayout
mpapt
mu
multiplatform
mumbai
munich
mvikotlin
mvrx
myndocs-oauth2-server
naming
navigation-architecture-component
nepal
new-mexico
new-zealand
newname
nigeria
nodejs
norway
npm-publish
nyc
oceania
ohio-kotlin-users
oldenburg
oolong
opensource
orbit-mvi
osgi
otpisani
package-search
pakistan
panamá
pattern-matching
pbandk
pdx
peru
philippines
phoenix
pinoy
pocketgitclient
polish
popkorn
portugal
practical-functional-programming
proguard
prozis-android-backup
pyhsikal
python
python-contributors
quasar
random
re
react
reaktive
realm
realworldkotlin
reductor
reduks
redux
redux-kotlin
refactoring-to-kotlin
reflect
refreshversions
reports
result
rethink
revolver
rhein-main
rocksdb
romania
room
rpi-pico
rsocket
russian
russian_feed
russian-kotlinasfirst
rx
rxjava
san-diego
science
scotland
scrcast
scrimage
script
scripting
seattle
serialization
server
sg-user-group
singapore
skia-wasm-interop-temp
skrape-it
slovak
snake
sofl-user-group
southafrica
spacemacs
spain
spanish
speaking
spek
spin
splitties
spotify-mobius
spring
spring-security
squarelibraries
stackoverflow
stacks
stayhungrystayfoolish
stdlib
stlouis
strife-discord-lib
strikt
students
stuttgart
sudan
swagger-gradle-codegen
swarm
sweden
swing
swiss-user-group
switzerland
talking-kotlin
tallinn
tampa
teamcity
tegal
tempe
tensorflow
terminal
test
testing
testtestest
texas
tgbotapi
thailand
tornadofx
touchlab-tools
training
tricity-kotlin-user-group
trójmiasto
truth
tunisia
turkey
turkiye
twitter-feed
uae
udacityindia
uk
ukrainian
uniflow
unkonf
uruguay
utah
uuid
vancouver
vankotlin
vertx
videos
vienna
vietnam
vim
vkug
vuejs
web-mpp
webassembly
webrtc
wimix_sentry
wwdc
zircon
Powered by
Title
b

bitkid

01/21/2020, 10:16 AM
i have a question regarding uploading files using the Apache http client. i do a simple submitFormWithBinaryData (that's the code https://gist.github.com/bitkid/e49c12c63cc4277a6139d3992a16ecb8) in multiple upload coroutines and the weird thing is .. it never gets to the log message. on the server side i see that the server returns status code 409 (conflict) which is correct but it never makes it to that log line and it just blocks in that call. any ideas?
e

e5l

01/21/2020, 10:20 AM
Hi @bitkid, could you install logging feature and show the output?
b

bitkid

01/21/2020, 10:20 AM
i init the client like this
private val httpClient = HttpClient(Apache) {
        install(Auth) {
            basic {
                username = "bla"
                password = "blubb"
                sendWithoutRequest = true
            }
        }
        engine {
            customizeClient {
                connectTimeout = 0
                socketTimeout = 0
            }
        }
    }
e

e5l

01/21/2020, 10:20 AM
b

bitkid

01/21/2020, 10:21 AM
sure .. cool. gimme 2 minutes
@e5l so one funny thing that happens .. when i init the logging with
level = LogLevel.ALL
my integration tests immediately start to fail, but they work with LogLever.HEADERS
👀 1
LogLevel.INFO also works
e

e5l

01/21/2020, 10:47 AM
It looks like a bug. Could you provide where it fails?
b

bitkid

01/21/2020, 10:55 AM
this is with log level=HEADERS which works
this is with log level=ALL
which failes
changing the log level is the only difference between these 2
ah sorry .. there are also the ktor server logs in there
let me remove them
e

e5l

01/21/2020, 10:59 AM
What
body
do you try to pass?
b

bitkid

01/21/2020, 11:00 AM
`
2020-01-21 11:58:45.317 [DefaultDispatcher-worker-5 @coroutine#5] INFO  io.ktor.client.HttpClient - 

---7484af384037e9be-506970d9-67a2ebac-70592eff12f476f731ac94c4-5cf322841
Content-Disposition: form-data; name=file; file; name=scientific1; filename=scientific1.csv

"","counts","calc_data","calc_data_dim"
"ENSG00000000003",1e+05,6.44626,"log2tpm"
"ENSG00000000005",9,0.163499e+02,"log2tpm"
"ENSG00000000419",2931,5.86146,"log2tpm"

---7484af384037e9be-506970d9-67a2ebac-70592eff12f476f731ac94c4-5cf322841--


2020-01-21 11:58:45.317 [DefaultDispatcher-worker-2 @coroutine#6] INFO  io.ktor.client.HttpClient - BODY END
that's how the body is created https://gist.github.com/bitkid/e49c12c63cc4277a6139d3992a16ecb8 (the file is a .csv file)
the broken delimiter error log message from the server confuses me (it only happens when i use loglevel ALL)
e

e5l

01/21/2020, 11:04 AM
It looks like form body doesn’t work well with logging, consumes the source twice and fail. Could you create an issue about?
b

bitkid

01/21/2020, 11:05 AM
ok!
does the consuming side look correct to you?
e

e5l

01/21/2020, 11:07 AM
Yep
b

bitkid

01/21/2020, 11:11 AM
but back to my original question .. how can i find out why that call is not returning submitFormWithBinaryData without logging? 🙂
i can try with level=HEADER .. if that gives any information
e

e5l

01/21/2020, 11:11 AM
Yep, could you show it headers?
b

bitkid

01/21/2020, 11:38 AM
ok that's weird .. that's all the output i get
2020-01-21 12:37:33.731 [DefaultDispatcher-worker-1] INFO  io.ktor.client.HttpClient - REQUEST: <http://url/objectname>
2020-01-21 12:37:33.731 [DefaultDispatcher-worker-1] INFO  io.ktor.client.HttpClient - METHOD: HttpMethod(value=POST)
2020-01-21 12:37:33.732 [DefaultDispatcher-worker-1] INFO  io.ktor.client.HttpClient - COMMON HEADERS
2020-01-21 12:37:33.732 [DefaultDispatcher-worker-1] INFO  io.ktor.client.HttpClient - -> Authorization: Basic YWRtaW46YWRtaW4xMjM=
2020-01-21 12:37:33.732 [DefaultDispatcher-worker-1] INFO  io.ktor.client.HttpClient - -> Accept-Charset: UTF-8
2020-01-21 12:37:33.732 [DefaultDispatcher-worker-1] INFO  io.ktor.client.HttpClient - -> Accept: */*
2020-01-21 12:37:33.732 [DefaultDispatcher-worker-1] INFO  io.ktor.client.HttpClient - CONTENT HEADERS
and then it just stays there ... nothing more
does httpClient.submitFormWithBinaryData need an execute call or something like that?
e

e5l

01/21/2020, 11:42 AM
Nope, it looks like they fail to send body
b

bitkid

01/21/2020, 11:44 AM
i see the request in the server logs
file.inputStream().buffered().use
and then it.asInput() is correct?
:yes: 1
e

e5l

01/21/2020, 11:47 AM
Could you file a github issue with reproducer?
b

bitkid

01/21/2020, 11:48 AM
i can't reproduce it in a unit/integration test .. that was my first try .. only when i let the whole thing run on our prod server
e

e5l

01/21/2020, 11:49 AM
So next step is try to watch network request dump(from tool like wireshark) and clarify where is the problem in details
b

bitkid

01/21/2020, 1:32 PM
it's getting weirder and weirder ... i tried replacing the apache client with the jetty client (same code) and i see this weird exception .. which imho comes from the server, not the client 😮
2020-01-21 14:31:32.774 [nioEventLoopGroup-4-3 @call-handler#34] ERROR Application - refCnt: 0, decrement: 1
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
	at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74)
	at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138)
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100)
	at io.netty.handler.codec.http.DefaultFullHttpRequest.release(DefaultFullHttpRequest.java:102)
	at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88)
	at io.ktor.server.netty.NettyApplicationCall.finishComplete(NettyApplicationCall.kt:51)
	at io.ktor.server.netty.NettyApplicationCall.finishSuspend(NettyApplicationCall.kt:44)
	at io.ktor.server.netty.NettyApplicationCall$finishSuspend$1.invokeSuspend(NettyApplicationCall.kt)
	(Coroutine boundary)
	at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invokeSuspend(NettyApplicationCallHandler.kt:40)
Caused by: io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
	at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74)
	at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138)
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100)
	at io.netty.handler.codec.http.DefaultFullHttpRequest.release(DefaultFullHttpRequest.java:102)
	at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:88)
	at io.ktor.server.netty.NettyApplicationCall.finishComplete(NettyApplicationCall.kt:51)
	at io.ktor.server.netty.NettyApplicationCall.finishSuspend(NettyApplicationCall.kt:44)
	at io.ktor.server.netty.NettyApplicationCall$finishSuspend$1.invokeSuspend(NettyApplicationCall.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:510)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:518)
	at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:835)
@e5l i have no idea what that kind of error that is
i think i will switch back to 1.2.6 for the time being
ok downgrading to 1.2.6 works
but i see that as a personal defeat 😉
what i had to change was:
httpClient.submitFormWithBinaryData<HttpResponse>
HttpResponse is in a different package .. and for the response i can use .use{} instead of .also{} (in 1.3.0) ... inputstream.asInput() seems to also be a different extension function in 1.3.0
this seems to match the 1.3.0 release notes