Yeah, Compose for Desktop sounds like a great fit, especially if you want to go to Android some day.
Most of the logic of your application is obviously not going to be specific to Compose for Desktop, and is just generic java logic. For example, to access an excel sheet, I'd probably suggest something Apache POI:
https://howtodoinjava.com/java/library/readingwriting-excel-files-in-java-poi-tutorial/
When you render the records, keep in mind that you won't want to render all millions of records at once. You will want to use some sort of lazy list or pagination because your hardware graphics buffers aren't going to be large enough to render everything at once (regardless of rendering framework). But with some effort, you could make it look very seamless.