Some more details on the issue I posted a few days...

# ktor

[Jilles van Gurp]

Some more details on the issue I posted a few days ago.

I'm trying to use the curl engine for ktor client in a multi platform project (kt-search) and I'm having issues with running this on multi platform builds on my m1 mac and kotlin 1.9.20-RC. Curl seems to be broken for Linux builds.

Several issues that I'm working around:

1. with home brew curl installed, I get a linking error during linkDebugTestMacosX64. Solution: brew remove curl. The mac build works fine without this.
1. The linux targets don't build on mac with some linking error on x86 and curl can't be found at all for arm64 I work around that with an if on the operating system:

kotlin
jvm {
}
js(IR) {
    browser {
        testTask(Action {
            useMocha {
                // javascript is a lot slower than Java, we hit the default timeout of 2000
                timeout = "30s"
            }
        })
    }
    nodejs {
        testTask(Action {
            useMocha {
                // javascript is a lot slower than Java, we hit the default timeout of 2000
                timeout = "30s"
            }
        })
    }
}
// only enable the linux targets when building on linux
// some issues with linux lib curl availability on other platforms
if(DefaultNativePlatform.getCurrentOperatingSystem().isLinux) {
    // some weird linking error
    linuxX64()
    // lib curl is not found for this one :-(
    linuxArm64()
}
mingwX64()
macosX64()
macosArm64()

1. On github actions the linux build fails with a different error:

> Task :search-client:linkDebugTestLinuxX64 FAILED
e: /home/runner/.konan/dependencies/x86_64-unknown-linux-gnu-gcc-8.3.0-glibc-2.19-kernel-4.9-2/x86_64-unknown-linux-gnu/bin/ld.gold invocation reported errors

The /home/runner/.konan/dependencies/x86_64-unknown-linux-gnu-gcc-8.3.0-glibc-2.19-kernel-4.9-2/x86_64-unknown-linux-gnu/bin/ld.gold command returned non-zero exit code: 1.
output:
/home/runner/.konan/dependencies/x86_64-unknown-linux-gnu-gcc-8.3.0-glibc-2.19-kernel-4.9-2/x86_64-unknown-linux-gnu/bin/ld.gold: error: cannot find -lcurl
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_global_init_wrapper23: error: undefined reference to 'curl_global_init'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_slist_append_wrapper27: error: undefined reference to 'curl_slist_append'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_slist_free_all_wrapper28: error: undefined reference to 'curl_slist_free_all'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_easy_strerror_wrapper33: error: undefined reference to 'curl_easy_strerror'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_easy_pause_wrapper35: error: undefined reference to 'curl_easy_pause'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_easy_init_wrapper36: error: undefined reference to 'curl_easy_init'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_easy_cleanup_wrapper38: error: undefined reference to 'curl_easy_cleanup'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_multi_init_wrapper44: error: undefined reference to 'curl_multi_init'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_multi_add_handle_wrapper45: error: undefined reference to 'curl_multi_add_handle'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_multi_remove_handle_wrapper46: error: undefined reference to 'curl_multi_remove_handle'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_multi_poll_wrapper49: error: undefined reference to 'curl_multi_poll'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_multi_wakeup_wrapper50: error: undefined reference to 'curl_multi_wakeup'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_multi_perform_wrapper51: error: undefined reference to 'curl_multi_perform'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_multi_cleanup_wrapper52: error: undefined reference to 'curl_multi_cleanup'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_multi_info_read_wrapper53: error: undefined reference to 'curl_multi_info_read'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:function libcurl_curl_multi_strerror_wrapper54: error: undefined reference to 'curl_multi_strerror'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:knifunptr_libcurl39_curl_easy_setopt: error: undefined reference to 'curl_easy_setopt'
/tmp/konan_temp6861980240265917513/test.kexe.o:out:knifunptr_libcurl42_curl_easy_getinfo: error: undefined reference to 'curl_easy_getinfo'

So, my conclusion so far is that the Curl engine simply doesn't work on Linux. Is this correct or is there something else I need to do to get this working? My impression is that curl in the konan instal is just broken for linux.

If you want to have a go at this, please use the more-multi-platform-targets branch on kt-search. The search-client module is the one that I'm having issues with.

[Aleksei Tirman [JB]]

Can you please file an issue?

[Artyom Gornostayev]

I guess I have same problem...

[Artyom Gornostayev]

@Jilles van Gurp, did you find a solution for this?

[Jilles van Gurp]

no not really, I have a hunch I have to mess with library configuration for this somehow. But the documentation is a bit hand wavy on this.

I'd love to get this working as I have a bunch of multiplatform stuff that uses ktor-client. Most platforms seem to work out of the box. Except linux.

[Artyom Gornostayev]

I've been playing with this problem for a while and I have found a workaround with Docker...
This Dockerfile works on MacOS (amd64):

FROM --platform=linux/amd64 gradle:8.5.0-jdk17 as builder

ENV KONAN_DATA_DIR=/home/gradle/.gradle

WORKDIR /build

COPY . .

RUN apt update && \
    apt install -y build-essential liburing-dev libcurl4-gnutls-dev curl
RUN gradle --stacktrace nativeBinaries

FROM ubuntu:latest

WORKDIR /

COPY --from=builder /build/build/bin/native/debugExecutable/app.kexe app.bin

ENTRYPOINT ["/app.bin"]

Than you can copy binaries from the Docker Container... If you need it...

[Artyom Gornostayev]

If someone is interested in building with Docker, this article might be helpful.

[Jilles van Gurp]

Interesting, will check that out.