Been there with Python, and totally get that. What I've found helpful, although it may seem intimidating at first, is to start grokking optimization and machine learning algorithms from scratch. When you do that, it really empowers you not necessarily to avoid the need for libraries entirely, but get insight and understanding on how the libraries work. The JVM does have some decent libraries already. I'll DM you.