Kapt (stubs generation + code generation) is the longest task in any project that uses annotation processing widely. 1.1.50 has some improvements according my feelings.
But the only real way for now: split your code to many modules to allow generate code only for changed module, because AP doesn't support incremental code generation.
There is a project that will allow to write incremental APs, but WIP and even after release each AP should add support for that