https://kotlinlang.org logo
Join Slack
Powered by
I'm starting to use Ktor to connect to a new endpo...
# ktor
j
I'm starting to use Ktor to connect to a new endpoint and am getting the following. I don't have any control over headers returned....anyone know if there's way to work around this?
Copy code
io.ktor.http.IllegalHeaderValueException: Header value '___utmvazauvysSB=TIAZsaY; path=/; Max-Age=900' contains illegal character '' (code 1)

	at io.ktor.http.HttpHeaders.checkHeaderValue(HttpHeaders.kt:153)
	at io.ktor.http.HeadersBuilder.validateValue(Headers.kt:43)
	at io.ktor.util.StringValuesBuilder.appendAll(StringValues.kt:208)
	at io.ktor.client.engine.android.AndroidClientEngine.execute(AndroidClientEngine.kt:92)
I can open url in browser for example and do see valid json returned.
h
Do you have a case to repro?
j
here's an example endpoint that I'm just doing normal get against...I can try and put sample code together as well if needed
Copy code
<https://data.smartdublin.ie/cgi-bin/rtpi/routeinformation?operator=be&routeid=401>
h
Apologies for delay. I just tried this with 1.3.72 and it seems to work fine. What Accept header are you passing in?
j
Thanks for checking....won't be able to confirm until later but is there particular one we should be passing in....don't think I had anything other than default (default for that endpoint should be json)
hmm, just tried it now and it worked! Perhaps owners of that endpoint made some changes to what was in response headers (so it no longer has that illegal character)
It's back happening again.....is there any way for example to stop ktor performing that check?
Copy code
main @coroutine#1] INFO io.ktor.client.HttpClient - REQUEST: <https://data.smartdublin.ie/cgi-bin/rtpi/realtimebusinformation?operator=be&maxresults=10&stopid=522301>
[main @coroutine#1] INFO io.ktor.client.HttpClient - METHOD: HttpMethod(value=GET)
[main @coroutine#1] INFO io.ktor.client.HttpClient - COMMON HEADERS
[main @coroutine#1] INFO io.ktor.client.HttpClient - -> Accept: application/json
[main @coroutine#1] INFO io.ktor.client.HttpClient - -> Accept-Charset: UTF-8
[main @coroutine#1] INFO io.ktor.client.HttpClient - CONTENT HEADERS
[main @coroutine#1] INFO io.ktor.client.HttpClient - BODY Content-Type: null
[main @coroutine#2] INFO io.ktor.client.HttpClient - BODY START
[main @coroutine#2] INFO io.ktor.client.HttpClient - 
[main @coroutine#2] INFO io.ktor.client.HttpClient - BODY END
[main @coroutine#1] INFO io.ktor.client.HttpClient - REQUEST <https://data.smartdublin.ie/cgi-bin/rtpi/realtimebusinformation?operator=be&maxresults=10&stopid=522301> failed with exception: io.ktor.http.IllegalHeaderValueException: Header value '___utmvazauvysSB=mNVidUA; path=/; Max-Age=900' contains illegal character '' (code 1)

io.ktor.http.IllegalHeaderValueException: Header value '___utmvazauvysSB=mNVidUA; path=/; Max-Age=900' contains illegal character '' (code 1)

	at io.ktor.http.HttpHeaders.checkHeaderValue(HttpHeaders.kt:153)
	at io.ktor.http.HeadersBuilder.validateValue(Headers.kt:43)
	at io.ktor.util.StringValuesBuilder.appendAll(StringValues.kt:208)
	at io.ktor.client.engine.android.AndroidClientEngine.execute(AndroidClientEngine.kt:92)
	at io.ktor.client.engine.HttpClientEngine$executeWithinCallContext$2.invokeSuspend(HttpClientEngine.kt:83)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
@hhariri do you know of any workarounds for this? Unfortunately, as mentioned, this is endpoint I have no control over so can't affect what's in the response headers. I'm using ktor successfully in a few other projects but obviously this is blocker for this particular one.
h
Can you reproduce for us in any way?
j
I'll try to put small sample together ....hopefully it will still be failing at time you run it though 🙂....I guess there's some level of randomness to what gets included in those header values that triggers this
Following should hopefully be pretty self contained code you should be able to run there
Copy code
import io.ktor.client.HttpClient
import io.ktor.client.engine.cio.CIO
import io.ktor.client.features.json.JsonFeature
import io.ktor.client.features.json.serializer.KotlinxSerializer
import io.ktor.client.request.get
import io.ktor.client.request.parameter
import io.ktor.client.request.url
import kotlinx.coroutines.runBlocking
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonConfiguration


@Serializable
data class Stop(val stopid: String, val shortname: String, val fullname: String, val latitude: String, val longitude: String)


@Serializable
data class BusStopInformationResult(val results: List<Stop>)


class RTPIApi {
    private val baseUrl = "<https://data.smartdublin.ie/cgi-bin/rtpi>"
    private val nonStrictJson = Json(JsonConfiguration(isLenient = true, ignoreUnknownKeys = true))

    private val client by lazy {
        HttpClient(CIO) {
            install(JsonFeature) {
                serializer = KotlinxSerializer(nonStrictJson)
            }
        }
    }


    suspend fun getBusStopInformation(): BusStopInformationResult {
        return client.get {
            url("$baseUrl/busstopinformation")
            parameter("operator", "be")
        }
    }
}





fun main() {
    val rtpiApi = RTPIApi()

    runBlocking {
        rtpiApi.getBusStopInformation()
    }
}
using these dependencies
Copy code
implementation "io.ktor:ktor-client-core-jvm:1.3.2"
    implementation "io.ktor:ktor-client-json-jvm:1.3.2"
    implementation "io.ktor:ktor-client-serialization-jvm:1.3.2"
    implementation "io.ktor:ktor-client-cio:1.3.2"
    implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.20.0"
Running that in IntelliJ here and getting
Copy code
Exception in thread "main" io.ktor.http.cio.ParserException: Character with code 1 is not allowed in header names, 
HTTP/1.1 200 OKServer: nginx/1.17.8Date: Fri, 17 Jul 2020 17:02:42 GMTContent-Type: application/json; charset=utf-8Content-Length: 1596054Connection: keep-aliveX-Powered-By: ASP.NETSet-Cookie: visid_incap_1822411=6/TavgoATn2fIJqXLil94rLZEV8AAAAAQUIPAAAAAAA2pXqBkrH+X0bxcOBu7tLA; expires=Sat, 17 Jul 2021 08:40:34 GMT; HttpOnly; path=/; Domain=.nationaltransport.ieSet-Cookie: incap_ses_535_1822411=2uPhR+2fyjexa0dYuLNsB7LZEV8AAAAAYXloqvhgqt+OzLpFszbb8Q==; path=/; Domain=.nationaltransport.ieSet-Cookie: ___utmvmzauvysSB=vRnGmVILTRo; path=/; Max-Age=900Set-Cookie: ___utmvazauvysSB=nqVRlec; path=/; Max-Age=900
	at io.ktor.http.cio.HttpParserKt.characterIsNotAllowed(HttpParser.kt:279)
	at io.ktor.http.cio.HttpParserKt.parseHeaderValue(HttpParser.kt:261)
	at io.ktor.http.cio.HttpParserKt.parseHeaders(HttpParser.kt:119)
	at io.ktor.http.cio.HttpParserKt.parseResponse(HttpParser.kt:73)
	at io.ktor.http.cio.HttpParserKt$parseResponse$1.invokeSuspend(HttpParser.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
h
There’s definitely a strange character in there.
j
It's likely there probably is something being done incorrectly by that endpoint but unfortunately it's one I don't have any control over
h
It's ASP.NET. For sure there's some nonstandard behaviour. Typical of MS tech :)
89 Views
Open in Slack
PreviousNext
Powered by