Hi, I'm struggling using http4k-connect. I try to ...

# http4k

[DanielZ]

Hi,
I'm struggling using http4k-connect. I try to use S3 with an ARN role based authentication but failing with a lens error of a missing AWS_ACCESS_KEY_ID. Wondering where I miss something?

const val USE_REAL_CLIENT = false
fun main() {
    val env = Environment.defaults(
        AWS_REGION of Region.EU_WEST_1,
        AWS_ROLE_ARN of ARN.of("arn:aws:sts:us-east-1:000000000001:role:myrole")
    )
    val http: HttpHandler = if (USE_REAL_CLIENT) JavaHttpClient() else FakeS3().debug()
    val credentialsProvider = CredentialsProvider.STS(env, FakeSTS().debug())
    val s3 = S3.Http(credentialsProvider = credentialsProvider, http)

    val sourceName = BucketName.of("source-bucket")
    val targetName = BucketName.of("target-bucket")
    s3.createBucket(sourceName, env[AWS_REGION]).recover(RemoteFailure::throwIt) // <<< fails here: env 'AWS_ACCESS_KEY_ID' is required
    s3.createBucket(targetName, env[AWS_REGION]).recover(RemoteFailure::throwIt)

    val source = S3Bucket.Http(sourceName, env[AWS_REGION], credentialsProvider, http)
    source.putObject(BucketKey.of("hello"), "hello ".byteInputStream()).recover(RemoteFailure::throwIt)

    val target = S3Bucket.Http(targetName, env[AWS_REGION], credentialsProvider, http)
    target.copyObject(source.bucketName, BucketKey.of("hello"), BucketKey.of("copy"))
        .recover(RemoteFailure::throwIt)
}

[dave]

The problem is happening because you're trying to connect to STS and assume a role that will then be used to access the S3 service. The STS connection itself will require some type of credentials to sign the STS request

[dave]

If you trace it through you can see that it's using the CredentialsProvider.Environment - which expects the access key etc. if you want to use another form of auth then you should swap that out

[Andrew O'Hara]

I haven't dug too deeply, but I'm not seeing anywhere where FakeSts requires source credentials. The fake assumeRole endpoint just uses hardcoded creds.

[dave]

It's not the fake that is barfing... it's the credentials provider which is used to sign the request to assume role 🙂

[Andrew O'Hara]

Ah, I see it. Even though you're injecting FakeSts, the credentials provider just treats it as an HttpHandler and wraps it in a real HTTP STS client; that's where it requires credentials to sign the assumeRole request.

So, what you need to do is either:
1. Add the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to the env ; if you're using FakeSts, then a dummy key and secret should work
2. Build your own STS.Http implementation using a custom credentials provider, such as CredentialsProvider.Profile.
Personally, my preference is to use CredentialsProvider.StsProfile , inject AWS_PROFILE=prod and have my ~/.aws/credentials something like

[dev]
aws_access_key_id = <key>
aws_secret_access_key = <secret>
aws_region = us-east-1
aws_default_region = us-east-1

[prod]
role_arn = <prod_role_arn>
source_profile = dev
aws_region = us-east-1
aws_default_region = us-east-1

That will be suitable for local development. But when you deploy, you'll need a way to authenticate as the instance or container. You have a few options there:
1. Multiple main methods or entrypoints to your application; each of which uses a different CredentialsProvider
2. Build a CredentialsChain akin to the offical SDK
For example, I often use a chain like this:

val credentialsProvider = CredentialsChain.Environment(env)
    .orElse(CredentialsChain.StsProfile(env))
    .orElse(CredentialsChain.ContainerCredentials(env))
    .provider()

Just swap ContainerCredentials with Ec2InstanceProfile for Ec2. You'll need the corresponding connect SDKs for those.

[DanielZ]

sorry for late answer.
so far we used AWS_ACCESS_KEY_ID and AWS_ECRET_ACCESS_KEY_ID for authentication, now we are starting to introduce ARN roles - so we are still noobs at that topic 😉
thanks for your awesome input and inspiration 🙏.