I am facing an issue indicated by the error messag...

# ktor

[원석]

I am facing an issue indicated by the error message: "<http://io.ktor.utils.io|io.ktor.utils.io>.charsets.MalformedInputException: Failed to convert Bytes to String using UTF-8".
This problem arises during the process of loading images in a Kotlin Multiplatform project, specifically on iOS Darwin. Has anyone encountered a similar issue or successfully resolved it? I could really use some help.

[Sam]

Client or server? How is the client/server configured, and what does the invocation where the image is actually loaded look like? It sounds like something is incorrectly trying to interpret the image data as text data.

[원석]

It seems to be occurring on the client side – I am using the Ktor client in a project where I am developing for both iOS and Android through Kotlin Multiplatform. I suspect the issue might be arising because the image URL contains a file format extension that is duplicated, causing some kind of UTF-8 related error during the data interpretation process. The error seems to be occurring with data like this: <https://sample-url.com/20200419_34/1587264166705H6VSV_JPEG/sampleimage.jpeg.jpg>

During the image loading process, I am utilizing the Kamel library, which is known for facilitating image loading in Kotlin Multiplatform projects. You can find more about it here: Kamel Library on GitHub.
Additionally, here is a snippet of my configuration code:

defaultRequest {
    url {
        protocol = URLProtocol.HTTPS
        host = "SAMPLE_SERVER_HOST"
        port = if (isDebug) 8443 else DEFAULT_PORT
    }
    header(
        HttpHeaders.ContentType,
        ContentType.Application.Json
    )
}

install(ContentNegotiation) {
    json(
        Json {
            prettyPrint = true
            isLenient = true
            encodeDefaults = true
        }
    )
}

ContentEncoding {
    gzip()
    deflate()
}

Charsets {
    register(UTF_8)
}

install(Auth) {
    bearer {
        loadTokens {
            BearerTokens(
                accessToken = localPreferenceLocalDataSource.getAccessToken(),
                refreshToken = String.Companion.Empty
            )
        }
    }
}

install(Logging) {
    level = LogLevel.ALL
    logger = object : Logger {
        override fun log(message: String) {
            Napier.i(message)
        }
    }
}

install(HttpTimeout) {
    requestTimeoutMillis = 50_000
}

[Sam]

By adding the Charsets configuration block I think you are instructing the HTTP client to try and treat everything it receives as text data. Does it work better if you just remove that block?

[Sam]

Images are not text and do not have charsets 😄

[Sam]

I’m not sure why there is an attempt to convert the data to a string in the first place, though 🤔 if that’s being done by Kamel, something else is wrong

[원석]

You’re absolutely right that images are not text and do not have charsets. I appreciate your thoughts on this. Regarding your concern about the data conversion to a string, I would like to mention that I am using the Kamel library to facilitate image loading. It seems that the library can handle various types of input data formats to create an image, as illustrated in these usage examples:

// String
asyncPainterResource(data = "<https://www.example.com/image.jpg>")

// Ktor Url
asyncPainterResource(data = Url("<https://www.example.com/image.jpg>"))

// URI
asyncPainterResource(data = URI("<https://www.example.com/image.png>"))

// File (JVM, Native)
asyncPainterResource(data = File("/path/to/image.png"))

// File (JS)
asyncPainterResource(data = File(org.w3c.files.File(arrayOf(blob), "/path/to/image.png")))

// URL
asyncPainterResource(data = URL("<https://www.example.com/image.jpg>"))

[Sam]

👍 makes sense — which one of those options is the one used in your code? Can you share the snippet where the image is loaded?

[원석]

Absolutely, I’m using the first option from the list which involves passing a string URL directly to the asyncPainterResource method. Here is the snippet from my code where the image is loaded:

restaurant.images.firstOrNull()?.let { imageUrl ->
    KamelImage(
        resource = asyncPainterResource(
            data = imageUrl
        ),
        contentDescription = null,
        modifier = Modifier
            .background(
                brush = shimmerBrush(
                    targetValue = 1300f,
                    showShimmer = showShimmer.value
                )
            ),
        onLoading = { progress ->
            if (progress == 1.0f) {
                showShimmer.value = false
            }
        },
        onFailure = {
            showShimmer.value = false
        },
        contentScale = ContentScale.Crop
    )
} ?: Image(
    painter = painterResource(SharedRes.images.img_restaurant_placeholder),
    contentDescription = null)

In this snippet, asyncPainterResource is passed a URL as a string (retrieved as imageUrl from the first image in the restaurant’s image list) and it’s used to load the image with the Kamel library. The code also includes handling for loading progress and failure, utilizing a shimmer effect during the loading process.

I hope this helps to clarify! Let me know if there’s any other information you need😀

[Sam]

When you see the error, does it come with a stacktrace that indicates the line in your code (or in Kamel) where the error originates? Although I’m not familiar with Kamel, I can’t see any obvious place where it would be trying to convert the data to a String based on what you’ve shared so far.

[원석]

Absolutely, the error I encountered comes with a stacktrace. It seems to originate from the Kamel library trying to convert bytes to a String using UTF-8 encoding. Here is the stacktrace to provide more details:

Uncaught Kotlin exception: io.ktor.utils.io.charsets.MalformedInputException: Failed to convert Bytes to String using UTF-8
    kfun:kotlin.Throwable#<init>(kotlin.String?){} + 123 
    kfun:io.ktor.utils.io.charsets.MalformedInputException#<init>(kotlin.String){} + 119 
    kfun:io.ktor.utils.io.charsets#decode__at__io.ktor.utils.io.charsets.CharsetDecoder(io.ktor.utils.io.core.Input;kotlin.text.Appendable;kotlin.Int){}<http://kotlin.Int|kotlin.Int> + 1507 
    kfun:io.ktor.utils.io.charsets#decode__at__io.ktor.utils.io.charsets.CharsetDecoder(io.ktor.utils.io.core.Input;kotlin.Int){}kotlin.String + 403 
    kfun:io.ktor.utils.io.core#String(kotlin.ByteArray;kotlin.Int;kotlin.Int;io.ktor.utils.io.charsets.Charset){}kotlin.String + 979 
    kfun:io.ktor.http.decodeImpl#internal + 3663 
    kfun:io.ktor.http.decodeScan#internal + 475 
    kfun:io.ktor.http#decodeURLPart__at__kotlin.String(<http://kotlin.Int;kotlin.Int;io.ktor.utils.io.charsets.Charset|kotlin.Int;kotlin.Int;io.ktor.utils.io.charsets.Charset>){}kotlin.String + 215 
    kfun:io.ktor.http#decodeURLPart$default__at__kotlin.String(<http://kotlin.Int;kotlin.Int;io.ktor.utils.io.charsets.Charset?;kotlin.Int|kotlin.Int;kotlin.Int;io.ktor.utils.io.charsets.Charset?;kotlin.Int>){}kotlin.String + 491 
    kfun:io.ktor.http.URLBuilder#<get-pathSegments>(){}kotlin.collections.List<kotlin.String> + 687 
    kfun:io.ktor.http.URLBuilder#build(){}io.ktor.http.Url + 375 
    kfun:io.ktor.http#Url(kotlin.String){}io.ktor.http.Url + 171 
    kfun:io.kamel.core.mapper.object-1.map#internal + 111 
    kfun:io.kamel.core.utils#mapInput__at__io.kamel.core.config.KamelConfig(kotlin.Any;kotlin.reflect.KClass<*>){}kotlin.Any + 547 
    kfun:io.kamel.core#loadCachedResourceOrNull__at__io.kamel.core.config.KamelConfig(kotlin.Any;io.kamel.core.cache.Cache<kotlin.Any,0:0>;kotlin.reflect.KClass<*>){0§<kotlin.Any>}io.kamel.core.Resource<0:0>? + 295 
    kfun:io.kamel.core#loadCachedResourceOrNull$default__at__io.kamel.core.config.KamelConfig(kotlin.Any;io.kamel.core.cache.Cache<kotlin.Any,0:0>;kotlin.reflect.KClass<*>?;kotlin.Int){0§<kotlin.Any>}io.kamel.core.Resource<0:0>? + 375

[Sam]

Interesting! I’m not familiar with Kamel so I don’t fully understand the stacktrace, but the sequence of function calls does seem to back up your theory that something is going wrong with the URL itself, not with the actual image data

[Sam]

You mentioned the duplicated file extension, but I don’t think that would be sufficient to cause a text encoding error, it would have to be something more than that

[Sam]

Does the actual URL contain any %-encoded characters?

[Sam]

Or just any % symbols in general

[원석]

I’ve checked the URL and yes, it does contain %-encoded characters.

[Sam]

👍 based on the error, it appears that at least one of the %-encoded characters in the URL is not valid (or was encoded using something other than UTF-8)

[원석]

Thank you for pointing that out 👍 I’ll investigate this further to pinpoint the exact issue. It seems like decoding the %-encoded characters to their original form before processing them might be a solution. I’ll give it a try and update you on the findings. 😀

[원석]

@Sam Thank you for your insight and patience as we navigated this issue. I followed your suggestion and investigated the URL more deeply. You were spot on about the %-encoded characters being the root cause.

I experimented with decoding the URL using different character sets and found that decoding it with ISO-8859-1 actually solved the problem. Here is the updated snippet of the code that addressed the issue:

restaurant.images.firstOrNull()?.let { imageUrl ->
    val decodedUrl = imageUrl.decodeURLPart(charset = Charsets.ISO_8859_1)
    KamelImage(
        resource = asyncPainterResource(
            data = Url(decodedUrl)
        ),
        contentDescription = null,
        modifier = Modifier
            .background(
                brush = shimmerBrush(
                    targetValue = 1300f,
                    showShimmer = showShimmer.value
                )
            ),
        onLoading = { progress ->
            if (progress == 1.0f) {
                showShimmer.value = false
            }
        },
        onFailure = {
            showShimmer.value = false
        },
        contentScale = ContentScale.Crop
    )
} ?: Image(
    painter = painterResource(SharedRes.images.img_restaurant_placeholder),
    contentDescription = null)

I am really grateful for your assistance and cooperation throughout this process. Thank you once again 👍

[Sam]

That’s great news, I’m glad I could help!