You're trying to convert your actual source, not your build scripts, to 1.4, right? Sounds like the commenter on that issue thought differently.
We've had a ton of performance issues with 1.4 in general as well. Afaik everything in Gradle has been fixed as of 6.7, and AGP 4.1.0. There are still a number of kotlin issues that are only fixed in 1.4.20 though, so I'd also try upgrading your Gradle to 6.7 and trying kotlin 1.4.20-M2 (you need to add the repo for pre-release builds:
https://kotlinlang.org/eap/configure-build-for-eap.html)