I'm not sure this is the right place to ask but I'm struggling a bit with API design. I've posted this on stack exchange as well: https://softwareengineering.stackexchange.com/questions/392156/designing-a-builder-as-a-compile-time-state-machine

This doesn't solve your problem, but since you are working with Kotlin I would suggest a Kotlin DSL. Instead of a fluent API you would have something like this:

panel {
    size =  Size(10, 10)
    position = Point(5, 5)
    wrapBox {
        shadow = true
        title = "Test"
        type = BoxType.SINGLE
    }
}

I need to support Java users as well 😞