Hi everyone, we created a little portfolio optimiz...

# datascience

[Adrian Trapletti]

Hi everyone, we created a little portfolio optimization example with Kotlin notebook and ecos4j https://github.com/atraplet/portfolio-optimization/blob/master/notebooks/Main.ipynb

[altavir]

Thanks a lot for your work. It happens that we have a similar task. Your solution was not ideal in our case since ECOS is under GPL. But it happens, that Ojalgo has a pure-java implementation both for conic solvers and for Markowitz problem in particular. I've done a separate implementation on Ojalgo and compared to the results of your solver. They do coincide. I've also compared results with commercial MOSEC solver on our client data (sadly, I can't share the results because of NDA) and they are the same.

[altavir]

Here is the Ojalgo solver: https://github.com/optimatika/ojAlgo/blob/develop/src/main/java/org/ojalgo/data/domain/finance/portfolio/MarkowitzModel.java

Ojalgo does not have a good documentation. We plan to provide KMath bindings to make working with its entities easier.

[Adrian Trapletti]

Hi Alexander,

Thx for the nice words. Let me give some background:

In plain vanilla Markowitz portfolio optimization you may compute points on the efficient frontier basically with three different approaches: 1. Maximize return given an upper limit on the variance 2. Minimize the variance given a lower limit on the return 3. Maximize a linear combination of the return and the negative variance, where the weights of the linear combination determine your risk preference and the exact location on the efficient frontier.

In practice, often 1 is used, since you can directly control the risk. ecos4j/ECOS allows you to solve all three approaches. ojalgo basically can solve 2 and 3 directly, but to solve 1, it needs to do some extra work (searching the right risk preference s.t. the risk is at the desired level https://github.com/optimatika/ojAlgo/blob/develop/src/main/java/org/ojalgo/data/domain/finance/portfolio/MarkowitzModel.java#L169).

The convex solver from ojalgo is not a Second Order Cone Program solver, but is has a quadratic and linear objective and linear constraints https://javadoc.scijava.org/ojAlgo/org/ojalgo/optimisation/convex/ConvexSolver.html. ecos4j/ECOS is a SOCP solver supporting positive orthant, SOC, and exponential cones. With these types of constraints much more is possible than with the ojalgo solver, also outside of finance. The Mosek cookbooks are very nice to see what kind of optimization may be performed with such solvers. You don't want to rewrite the native solvers in e.g. Java as this would be a lot of work. They are quite complex. Reusing the native solvers is much easier.

My starting point was https://www.cvxpy.org/ and I wanted to bring some solvers to the JVM. The first one was ECOS, that was the default solver for years. ECOS is under the GPL. That's why I had to put ecos4j under the GPL as well. The second one is https://clarabel.org/ that is the new default in cvxpy. clarabel4j is already able to call the native Clarabel solver, but needs some more work on the Java interface. clarabel4j/Clarabel will be Apache 2. If I find some time, then I will release it in the next weeks. The third solver, where I already have some code is https://www.scipopt.org/. SCIP is interesting since it can also handle Mixed Integer and non-convex problems.

If you have the budget for commercial solvers in your project, then Mosek and Gurobi are very good solvers with interfaces to Java as well.

[altavir]

@Adrian Trapletti Thanks for a thorough explanation. Our client has a budget for a commercial solver, but they have to mitigate risk of discontinuation of the commercial support. Also MOSEC seems to behave a little bit differently on different OS platforms, which is unacceptable in their case. I am not a specialist on a financial theory, so I have to rely on the problem definition stated by our client. They indeed need to work with fixed variance, but the performance is well within allowed limits so it does not seem to be a problem at the moment.

Anyway, your comment helped me a lot. I will have your solution as a backup in case the problem changes in the future.

Also KMath allows to simplify the wrapper code a bit (especially matrix creation and manipulation. We already have a wrapper for EJML and plan to have a wrapper for Ojalgo as well. If you do not mind, I will rewrite your code in KMath (using your wrapper of course) and add it to KMath examples (not right away, but in several months, when I have time).

[altavir]

And I already found your wrapper for Clarabel (last week) and started to do some experiments with it. Nice work.

[Adrian Trapletti]

New link https://github.com/atraplet/portfolio-optimization/blob/master/notebooks/Ecos4j.ipynb

[altavir]

It seems like we will be making a large project, using ojalgo and probably Clarabel in a few months. So I am looking forward to use your wrapper and probably contribute to it.

[Adrian Trapletti]

In the meantime, clarabel4j is released https://github.com/atraplet/clarabel4j and on Maven Central. The accompanying clarabel4j-native https://github.com/atraplet/clarabel4j-native is also ready and on Maven Central. It contains release binaries for Linux (linux_64), Windows (windows_64), and MacOS (osx_arm64).

Examples can be found in the unit tests of the model (linear, quadratic, second-order cone, exponential cone, power cone, and generalized power cone) https://github.com/atraplet/clarabel4j/blob/master/src/test/java/com/ustermetrics/clarabel4j/ModelTest.java

Concerning portfolio optimization, the Java example can be found here https://github.com/atraplet/portfolio-optimization/blob/master/src/main/java/com/ustermetrics/portfoliooptimization/Clarabel4j.java, the Kotlin notebook example here https://github.com/atraplet/portfolio-optimization/blob/master/notebooks/Clarabel4j.ipynb

[altavir]

Nice!