https://kotlinlang.org logo
Join Slack
Powered by
I built a desktop app that runs on Windows/macOS/L...
# compose-ui-showcase
n
I built a desktop app that runs on Windows/macOS/Linux and organizes photos into collections that are tagged by an offline LLM. It’s a Google Photos replacement for me that is fully offline and open source! I use Google Photos on my phone a lot, but there are thousands of old photos on my laptops as well. So over the week, I played around with Ollama/Llava and wrapped it into a Kotlin Multiplatform desktop app K. In the next version, I want to add face recognition and tagging too. This would be especially useful for me to find various faces from weddings and events. You can install it from GitHub, but I would suggest just checking out the repo, opening it in IntelliJ IDEA, and running 
./gradlew :composeApp:run
.
LimanPhotosDemo.mp4.mov
🔥 5
s
Oh wow, this looks amazing. I need to give it a try. I wanted object detection for my app Ashampoo Photo Organizer since an eternity, but results from open source models were always not satisfactory. Never heard of LLaVA. 👀 Also, using Ollama instead of PyTorch is a great idea, too.
I had this project on my list of the models I wanted to try: https://github.com/openai/CLIP Did you try that, too?
n
Thanks! I haven't heard of clip. I am thinking of adding a face recognition and creating collection of photos from faces too. I belive using python would help me with that
s
For face detecion I use RetinaFace through Deep Java Library (DJL) - they made a decent binding for that using PyTorch. https://github.com/deepjavalibrary/djl/blob/0a91a1903af42b2257d942771abffd188a2bd7[…]in/java/ai/djl/examples/inference/face/RetinaFaceDetection.java Good results, but maybe there is something better out now.
🙌 1
On iOS & Android Ashampoo Photo Organizer just uses what Apple & Google provide. Face detection comes out of the box there.
Apple even has a quality rating of the faces. For the other platforms I needed to do this myself. My Photo Organizer has something that's like Narrative Select, if you happen to know that tool.
n
Oh nice! What's your tool? Do you have a link?
s
I've been working on that since 2021 🙂
It lacks AI object recognition. I now wonder if LLaVA could be the solution or if that won't work with 8 GB - that's what a typical end user has available.
Do you plan to monetize LimanPhotos similar to LimanDoc or is this just a pet project?
Unfortunately, my tool is mostly closed source, but back then I was able to convince my company to release the metadata handling as a open source library. You may have seen one of my posts about https://github.com/Ashampoo/kim/ in #C0BJ0GTE2. 🙂
Maybe my metadata library is interesting to actually write the tags AI identified as keywords to the photos. That's what I plan to do.
n
Wow, amazing work! I can imagine it takes a lot of time. Is it also available for Mac? I can see familiar compose ui 😄 I do not plan to monetize LimanPhotos, rather just a testing AI stuff and a relatively low-effort development compared to LimanDoc. Maybe I will incorporate my learnings/reviews in LimanPhotos to LimanDoc 🙂
👍 1
s
Thank you! 🥰 Yes, it was a lot of work. The subscription version https://www.ashampoo.com/en-us/photo-organizer-pro is available for Mac, too.
n
Nice 🙂 Watched the video, great work, lot’s of functionality! I liked the “album” feature, and I was also thinking of creating album automagically by location, time, and other metadata. For now, only AI “tagging” creates thoe albums/collections.
s
Yes, creating those smart albums is pretty easy if you have the data available.
Ok, I tossed some photos in. LLaVA results are pretty usable. 🔥 Three years ago I tried ResNet50 and the results were a lot of BS.
n
I tried my personal photos with ±3k photos, and it was actually pretty good! The “collections” could be improved, but searching is nice for me. By the way, clicking on a photo will display the tags that AI has detected. I would say they are 75-95% accurate. That also can be improved with a better ollama prompt or better model.
s
I love LLaVA... That's just what I needed. It runs on my personal photos and at my machine (64 GB Ram, RTX 2080) performance is good.
At work we've a testing machine... some i3 with 8 GB RAM... typical customer setup. I'm curious how it works there... if at all.
n
On my mac it takes around 5-15 seconds per image. I have images that are 10-20mb each, so that also affects that. I am sure there are performance optimizations I could do, e.g. downscaling and then sending to llava, etc
s
On my machine it takes one second per image.
n
Agree, this is not for average user, that’s also one of the reasons I opensourced it hehe
s
I guess that's because I have a proper graphics card.
n
Yeah, possible. I believe if it is possible to utilise Apple’s AI chip from jvm…
s
Agree, this is not for average user, that’s also one of the reasons I opensourced it hehe
Let's see... I really want a model like that in my product. ^^
🙌 1
If it's not possible I think I'll send you a PR that integrates my metadata library, so there is an option to directly write all detected tags into the files.
n
Feel free to incorporate LimanPhotos code into your project 😉 It’s Apache v2 license.
🙏 1
Open in Slack
Previous
Powered by