Hello, any of you already deployed a ktor service ...

# ktor

[Jamy]

Hello, any of you already deployed a ktor service with environment variables on Google App Engine from Google Cloud Platform? If possible without saving the variables in a file on git (security issue)

[Johannes Zick]

I have not done it yet, but I was planning to deploy to App Engine with Terraform. There I know you can supply variables at deploy time from somewhere, like some form of CI variables.

[Jamy]

I’ll take a look, thanks. This could be a good opportunity to test Terraform.
But I think it’s strange that I can’t add an environment variable via GCP’s backoffice or have another simple way.

[Johannes Zick]

True, maybe there is a way. I am also just starting with GCP. But I've had good experience with Terraform. It has a very nice model to work with different environments. Managing secrets can still be annoying though. Be careful to not accidentally print the secret values to the CI log when you deploy,

[Johannes Zick]

@Jamy Guess I stand corrected. Because of their weird flow, Terraform looks to be the wrong tool for the job here. But the only other thing I can see is templating the app.yaml file myself. If you find anything better, can you share back? I also still need to get my service going somewhere.

[Jamy]

What’s wrong with Terraform? Should I avoid wasting time testing it? Of course if I find a solution I will share it with you here!

[Johannes Zick]

Feel free to test it. But looking at the documentation, I'm not sure how you are supposed to deploy it. It only accepts some path to a GCP bucket.

[Johannes Zick]

Which makes me wonder how to integrate it into a CI flow. What should I even upload there? They have a Gradle plugin, but that one is apparently meant to deploy directly?

[Jamy]

Yes that’s right, the goal is to run the command ./gradlew appengineDeploy and it deploys in GCloud depending on the app.yaml

[Johannes Zick]

Yeah, but that one I can also find no way to provide env vars except per app.yaml file. Terraform has a section for those, so you can get the values from anywhere. There I cannot see how to specify the artifact to deploy. Maybe you can point it at the executable jar file. I cannot find any examples though, and I'm still at work. I'd probably try uploading the jar into the bucket, then using Terraform to deploy. If that does not work, you might have to resort to the official Gradle plugin. But then we are back to square one for env variables.

[Jamy]

I haven’t tried it yet but in the app.yaml file we can add environment variables. It’s not very secure if the file is on git.
But I can declare them in another .yaml file and include it in the app.yaml file.
Finally the new .yaml file I add it in the .gitignore
But it’s not very “DevOps”.

[Johannes Zick]

No, not at all. That is why I prefer Terraform to these hacky cloud provider specific CLIs. The documentation is just lacking here. And Google also does not help. So, yeah. My approach, when I finally find the time to work on my project again, would be: build an executable jar (Spring Boot FTW). Upload that jar into some private bucket. Use Terraform to deploy, pointing at the jar and injecting secrets securely in the CI. Not sure if that will work.

[Jamy]

I think I’ll try and explore that possibility tomorrow, it’s my night at home. Thanks anyway and I’ll let you know!

[Jamy]

I see a lot of documentation with Cloud Engine but nothing for App Engine...

[Johannes Zick]

Yeah, same. Good luck!

[mp]

Google has a secret management service now. I'd use that at your service's startup to download any additional properties and mix them in with the non-secret env vars from your app.yaml.

[mp]

I implemented something similar before their secret management tool was available by downloading encrypted properties-formatted objects from GCS and decrypting with KMS.

[mp]

Starting today, though, I'd probably use the secret thing.

[mp]

Also, if you're choosing a Terraform-like tool today (starting a new project, in other words) I'd take a hard look at Pulumi before using Terraform. I've used terraform for many years and Pulumi's model is way better.

[Johannes Zick]

@mp Thanks for the advice! I know Google has some secret management, but I did not try it yet. I'm not sold yet on having another complex integration to manage one or two values for my own small project. I might have a look again though. I have not heard yet of Pulumi though. Need to chec it out.

[mp]

It's not complex. Use the gcloud CLI tool to upload a properties-formatted secret. Set that secret name as an env var, and download it via the Java GCP SDK at startup.

[mp]

Use something like commons-configuration taht lets you merge config sources, and now you have one pool of key-value config to draw from.

[mp]

I agree, it'd be convenient if GAE had a good answer for secrets built in, but I think ☝️ is really not too bad, and it doesn't lock you in to GAE.

[Johannes Zick]

I consider this a bunch more complex compared to setting a variable at deploy time, because I suddenly need a startup script to do various things for me. To be fair, there is probably a Spring integration available. Still, it's weird that this is the two options GCP offers: hardcode it or download it at launch time.

[mp]

No need for a startup script. Just do what you need to do in your main().