Hello y'all. I'm working on scaling up KMP in my ...

# touchlab-tools

[Siggi Gunnarss]

Hello y'all.

I'm working on scaling up KMP in my current project. It's moderately large with dozens of developers. Our kmp "library" has a large number of modules. We use the library model with a swift package and maven distribution for consuming the library.

Local development is done pretty awkwardly with by publishing a Swift package locally on iOS and a local maven for android. I'm sold on this blog posts about compiling from source:
https://touchlab.co/kmp-teams-use-source

Hopefully we'll soon be compiling from source on iOS soon when doing local development, but I feel like a better integration between our gits is needed. A tighter linking between the 3 gits and building from source by default. I think a monorepo is out of the question due to team/project size. Basically I want submodules, but good. :)

Is anyone here using GitPortal with success? Or a homegrown approach that works in a similar way?

[Filip Wiesner]

I am basically in the same situation (also 3 repos, the same package distribution...) and I am considering the monorepo.

I think a monorepo is out of the question due to team/project size

Why do you think that? 🤔

[kpgalligan]

I have content that I have to locate on why monorepo isn't necessarily going to be a good idea, and isn't the practical gold-standard for KMP. The practical side is the resistance most teams/orgs will have to doing it just for KMP. That turns into a "we'll talk about it after the next release/deadline (whatever)", but keeps getting kicked down the road. The mechanical problem, even if you can do it, is if your android and iOS devs are specialists, and most teams are (and probably should be), you need a lot of guardrails and process because an edit to "Android" or "iOS" can and should impact the shared code, but then you've effectively changed the code underneath the other app.

[Siggi Gunnarss]

I used a monorepo with good result in a previous project, so that was my first thought. But my experience has only been with a smaller team, I feel like with 10 teams with 10 developers each, my experience might not apply entirely.

I know there are some worries about monorepo. Probably because it's a big migration that's very hard to back out of. Big CI changes and some questions about how to handle tags and release management. There were probably also concerns about every developer needing java to compile the ios project. An addition to an already heavy onboarding.

I feel like these can all be overcome with some effort. But we also lose some flexibility with monorepo. Today, you can implement something in kmp and android, but then hold off on incrementing the kmp library version on ios. Then update the kmp library when there's a PR ready on ios that supports the new breaking change.

[kpgalligan]

In practice, that means one of a few mitigations, and at scale, I don't think any of them is great.

[Siggi Gunnarss]

ah, plus what Kevin said

[kpgalligan]

I've talked to a few teams that have done it, and many who were looking at it. The "looking at it" teams often suggested things like "we'll have Android and iOS devs pair code. That will also help with silos (etc)". Possibly true, but my response is generally, "you could try that now and see how it goes. Mandate that all architecture changes in either app needs to happen on a Zoom call." I put it less snarky than that, but that is kind of what the suggestion would imply.

[kpgalligan]

Ideally, you'd want either team to be able to edit their app code, and full in changes and features from the other app in a structured way. You can do that with a monorepo, but it's a branching strategy, or something else beyond just having all the KMP in one module, and both apps build from it directly.

[kpgalligan]

Now, I have other content on GitPortal "bidirectional", which I would not exactly recommend using. It's essentially a prototype of a multi-repo model branching strategy, but I think that implementation is too opinionated. Conceptually, though, it would let you edit iOS or Android, including KMP code, without immediately ripping the rug out from the other app.

[kpgalligan]

There's another unfinished post describing how to do that with binaries and KMMBridge. Also, while I think source is best, we did figure out how to have an iOS app debug and browse Kotlin from a published SPM binary, which removes one of the big reasons I think iOS teams should build with source: https://touchlab.co/spm-kotlin-debugging

[kpgalligan]

On the monorepo discussion, I sort of touch on it here, but I kind of lost steam on talking about it: https://touchlab.co/kmp-teams-piloting-vs-scaling

[Siggi Gunnarss]

I saw the "debugging with SPM builds" post, crazy that you pulled that off.

I'm now working on changing our "local kmp development mode" to compile from xcode with "embedAndSignForXcode". So if If I get that merged, iOS KMP developers can optionally build from source and debug. Not a perfect solution, but removes a pain point.

So I'm still looking at some "git submodules but good" solution and thought of that introduction to GitPortal.

[kpgalligan]

I did a POC of GitPortal v2, but under the hood it used submodules. Submodules are stable, but working with them directly sucks. But, the POC never got finished. I've found it a struggle to convince people that I wasn't saying weird things and overcomplicating the problem, so building a whole different version seemed like a lot of work that wouldn't go anywhere 🙂

[Siggi Gunnarss]

I feel that GitPortal was definitely a good idea, linking the gits with some git subtree + github actions magic. Too bad you didn't get a bigger reaction to that, maybe everyone is just happy with hardcore monorepo migrations...

I wanted to write something similar to GitPortal for our setup (github without actions + jenkins), but it's obviously a big job.

Surprised you were going with submodules! Gnarly

[kpgalligan]

maybe everyone is just happy with hardcore monorepo migrations

I don't think so. I don't think they're doing it. That's the core problem I was focusing on. There aren't many examples of teams with more than a handful of devs using KMP higher than "some libraries", and I think a big part of that is the workflow and repo org problem. So, if the only answer is "merge your repos", that'll be enough to hold back more scaled KMP adoption. It's not a solved problem.

[Siggi Gunnarss]

Got it.

Yeah, in the previous smaller project I worked on we went rapidly from kmp-android + ios repos to a monorepo just to increase the velocity and because the ios devs asked for it. We never felt the scaling pain. Not so for the larger project I'm now on.

Let's see what we figure out with Jenkins and some strange git commands.
...unless you have an epiphany and announce GitPortal 3 at KotlinConf :)

[kpgalligan]

I won't be at KotlinConf, so unlikely 🙂 Had to pick that or Google I/O. Same week, and that flight wasn't looking reasonable. Also, after the dive into building the GitPortal bidirectional thing, it's pretty clear that it would be an endless feature drag. So, it would either get reduced to a wrapper over submodules, or a custom restrictive workflow (which is very much what the current GitPortal bidir is) with a ton of feature requests. The only teams that would want to use it are mid+ teams, and they tend to have specific needs. So, lot's of work, not a lot of users, but all of them with specific situations and a pretty low tolerance for "your thing broke!" It would be like KMMBridge, but an order of magnitude worse. We talk with teams that publish binaries, and they often write their own, usually with "well, KMMBridge didn't do this exact thing." But, it's interesting. This is the most I've seen anybody discuss the topic in months.

[Siggi Gunnarss]

Right, makes sense.

...and yeah, not sure why it's not a bigger topic. I'll keep digging in any case

[Bradleycorn]

@Siggi Gunnarss a little late to the party here ... I'm kind of in the same boat. Pretty big project, about a 10 devs on two teams. I'm interested to hear how things go for your team.

Thus far, we have only put a few things into a KMP library, though it is already growing with several modules (database/storage, networking, models, analytics, feature modules, etc).

We are currently publishing binaries to github packages and consuming them via cocoapods (🤯) for ios and maven (github packages) for android.

We are getting ready to start migrating large chunks of our business logic to the KMP library (from our existing android app) and using KMP much more significantly. To prepare for that, I have been working ahead of the teams for a few months now getting some basic architecture in place.

I have experimented with compiling from source, using gitportal in unidirectional mode. I've also made edits to the KMP code from within the app repos, and pushed it back to the kmp repo (as described here) in an effort to simulate how our dev teams would work with gitportal and compiling from source.

It has worked pretty good for me so far using "feature branches". It does seem like it will be a learning curve for the devs (some of the more junior devs in particular might take awhile to "get it"). I'm not 100% sold that it will be the right solution for us permanently, but it might be. I think I am going to have our dev teams give it a shot and see how it goes. I actually have our kmp repo setup currently so that it can work either way, with gitportal (compile from source) or with kmmbridge (publish binaries).

Honestly, our current setup with binaries and a local maven repo works ok. We have one hurdle to work out with versioning and it might be where we land. But also the fact that our ios app is still using cocoapods helps, cause local dev mode with kmmbridge and cocoapods is pretty easy, and even allows debugging. We'll have to switch over to spm soon though, and that may force changes to our kmp workflow as well.

[kpgalligan]

I should probably write some updated thoughts on the whole topic at some point. GitPortal is a particular implementation, but it was essentially a POC implementation of a more general concept. First, to disambiguate, in this context when I say "GitPortal" I mean the bidirectional mode. The unidirectional mode is very simple, both the understand and in implementation. That mode is essentially versioned publishing, but with source instead of binaries. Bidirectional is quite different. Anyway, GitPortal (again, bidir) is a POC implementation for a more general concept. The idea was to put GitPortal out, around KotlinConf last year (it was a lot of hours trying to get it to a functional state, that's for sure). Then, find some teams to try it out, take feedback and necessary features, and decide what to do next. That phase never really happened. The "scaling" topic has been an interesting one. I've had trouble even getting people to see the problem (or problems) unless they've actually tried to do it. Teams who are planning to try KMP tend to view the whole situation with a mental model of other seemingly familiar situations. There are critical differences, which is why my talks and content shifted in the 2nd half of last year from trying to build clever solutions so the problem to simply explaining the problem and what I was proposing, conceptually. I'd characterize the response to that as similar to the GitPortal response. Unless you've tried to do this, or are about to and have had to really think through, the whole thing doesn't really "click". I would imagine I sound like I'm overcomplicating things 🙂

[kpgalligan]

But, in summary, there are a few major conceptual things about KMP that are distinct, or present distinct problems. First, and the easy one to understand, is the mental model of KMP as "shared code". That seems to almost immediately put it into the "library" bucket. We all have a very clear concept of a library. So, KMP is almost universally added as a library, certainly to an existing project. Generally speaking, I think that's how it should be added to start. It's straightforward, and doesn't really disrupt existing workflow. The problem with that is library code is not app code. You don't change SqlDelight or Ktor to support some change in your app's onboarding flow. Those are extreme examples, but the same is true (less true, but still) for internal libraries. Unless you're actually working on the library directly, you're almost certainly pulling them in by version, and the library code is implementing something reasonably self-contained. Logic like some kind of math calculations, common networking, etc.

[kpgalligan]

If you're trying to use KMP for app code, which I more specifically refer to as "feature code", that's not really a good match for a library. It's now code that's part of your app. It gets changed as you're doing app feature changes. So, you're doing something to the onboarding flow that the product manager and designer wanted to do. That can live in a versioned library, technically, but that a terrible workflow. It'll really slow down development. The KMP Golden Rule if you wouldn't do something without KMP, maybe don't with KMP. If somebody suggested the Android app should pull all of it's UI logic, but just the rendering logic (Compose, let's say) into a separate repo, and keep all view models, state, etc, in another repo, and the link between them was a versioned library, you'd probably have a lot of questions. It's pretty difficult to make screen edits if you can't change the code that sits immediately under the UI.

[kpgalligan]

So, the obvious answer is monorepo. All KMP samples (basically) are a monorepo. The design of KMP and sourcesets essentially implies that this is what you should do. I don't think most people question this. Generally speaking, the only time people do question this is for practical reasons. Moving existing teams and apps to a single repo will be difficult. But, if they could, they would. That's where the next problem comes in.

[kpgalligan]

If you're scaling KMP up into view supporting code*, it's app code. If you weren't using KMP, you'd edit your screens and your app code, do a PR, review and merge, or whatever your team does for its workflow. With KMP, that's a problem. If you're editing the Android app and making changes that hit the KMP code, there's a whole different app who's "internal" code you're now editing. Without any mitigations, as in the Android team just worried about Android, and vice versa, that's a whole lot of potential problems. So, you have to change how your teams work in some fundamental way:
• All devs need to edit both apps (merge teams)
• One dev does one platform, then passed off a branch to another dev (tag team)
• Pair coding, but a lot of it
• Etc
There are many. But, they all have something in common. They couple the teams and apps, which is a dramatic change from how they're currently developed. That might work out OK, or it might significantly impact productivity (and team morale). If there are existing apps and repos, you won't find out until your merge repos and seriously disrupt everybody's timelines. If everybody dislikes the outcome, well, that seems like a pretty risky change to push for. I haven't seen many teams of more than a few devs do it. I haven't seen any teams that you might call medium/large do it.
* I hate saying "view model" because it's a confusing term. "JetPack ViewModel, capital V, or view model the concept?" Anyway...

[kpgalligan]

I have seen people argue that coupling the teams is something they want. They want to break down silos, or have "mobile devs" instead of platform devs, or having both teams pair coding much of the time is good, or whatever. I have lengthy brain dumps about each of these in various places, but I'll just go back to the Golden Rule. You could be doing that without KMP, so why aren't you? Try it for a while, without KMP. See how it goes. Making a tech change in the hopes that it forces a team and people change that the vast majority of native mobile teams have resisted for 15 years, well. It doesn't seem like a great idea when you phrase it that way.

[kpgalligan]

The bidir thing wasn't primarily about letting apps stay in different repos. The primary goal was allowing the Android and iOS teams edit the KMP in isolation. Android edits don't immediately impact iOS and vice-versa. That doesn't wreck workflows, don't risk timelines, etc. I've been on some projects that are going fine when things are calm, but when you have a deadline coming up, the testing isn't really happening, the PRs are either delayed or not deeply reviewed. That's when you start getting KMP changes that aren't really tested on opposing platforms. That has many impacts. Speeding up velocity is not one of them.

[kpgalligan]

Changes to KMP code in one app get made in isolation, but obviously the eventually get into the other app. But, a dev does that intentionally. It's part of the workflow. You pull changes over. Use git. That's what it's for. GitPortal is really just an implementation of a branching strategy.

[kpgalligan]

Ah, I should write this up. This free-form brain dump isn't coming together well on a lazy Saturday afternoon. Summary: GitPortal is a POC of something more conceptual. Using a branching strategy to not couple your mobile teams, for the reasons listed. To get back to a more practical discussion, KMMBridge and binary library publishing can be used as a half measure. We have figured out Xcode source browsing and debugging with binaries. That, and use SPM with branches instead of versions in dev. That gets around many of the library model issues, and doesn't require merging repos or a complex branching strategy. There's a long-unfinished blog post and tutorial explaining how to do that.

[Siggi Gunnarss]

Branches vs versions in the binary distribution system is also an interesting thought. Works for dev, maybe also for releases. Would require a GitPortal like branching strategy.

There's a blog post or two in there for sure! Thanks for clarifying/brain-dumping.

[xiaobailong24]

We have a KMP business repo, and a iOS Xcode repo with a local pod that is KMP framework to integrate all KMP libraries.
We don't use Monorepo. We use Gradle composite builds(https://docs.gradle.org/current/userguide/composite_builds.html).
When we develop locally, we can switch to depend local source code of KMP libraries by Gradle composite builds. And in C/CD pipeline, we use published maven artifact of KMP libraries.

[Bradleycorn]

@kpgalligan I really appreciate all of that info. For me a couple of things you mentioned really inspire confidence that KMP will work well for us.
1. While individual devs are still mostly specialized, we merged our mobile app dev teams a few years ago and haven't looked back. One team works on a feature for both our android and ios apps. That has worked out pretty well for us. In that regard, I think we'll be able to leverage kmp in either a library or source build format.
2. Our goal is to use KMP for business logic, and keep all display logic (ViewModels/view models 😉 and above) written natively in the applications. Mainly for the exact reasons you've laid out above. I've had a few of my devs make an argument for shared view models (because in many cases they are the same/similar). But I've replied with the exact arguments you've made above, along with an extra argument that the whole point a view model is to .... model the view, and the fact is, views (and the way they work) ARE different between ios and android. So while it may seem like a good idea to share those models, in the long run it'll hurt more than it helps.*
3. Perhaps somewhat unique to our business, but the inherit separation that comes from a versioned library is actually helpful for us in 2 ways
a. It helps devs to draw a mental line between app/view logic and business logic, keeping business logic "generic" and "reusable" across multiple/many apps.
b. We actually want to be able to package up our business logic into a library and resell it to partners and other 3rd parties. Our own apps, are just one set of ui's put on top of our core business logic/application. Others can buy it, and build their own UI's. So again, having the clear line of delineation that a library provides really helps our devs ... "Is this something that we want in our app, or is it core application logic that anyone would need if making an app for our business"...
*We do have one or two areas where there is core business logic that is complex, but it kind of needs to live in the view (model) layer. For those few places, we do have a seprate module in the kmp library that contains that logic in a shared place, due to it's complexity and a strong desire to maintain consistency across platforms.

[Joaquim Puyo]

Hello! In a modularized application, I think that monorepo seems the most simple option, otherwise I have to create a umbrella module and generate XCFrameworks for each module.
In my previous job, I worked always with monorepo, but now we are trying to merge 2 different codebases that should be aligned as much as possible.
If anyone found issues with monorepo, it would be great to share here, but I am not able to think in any blocker related to monorepo

[Bradleycorn]

For me there are two primary issues with a monorepo:
1. For small apps, it's probably fine. But for a medium/large application, the repo starts to get really big, and particularly if you have developers who are specialized on one platform or another, they are pulling down an entire application that they never work with.
2. Managing branches and tags and releases etc becomes complicated and cluttered, as one app gets some updates and needs a release, while the other app doesn't need those updates (maybe it's a bug fix or something). This is especially cumbersome if you have multiple teams and/or lots of developers all working on the same application. You start to run into issues where you are trying to figure out things like:
a. "Does that branch also have android updates, or only ios?"
b. "Does that release/tag include any ios updates?"
So, if you have a small team and/or a small app, a monorepo is probably fine. But for larger, more complex team and app setups, the monorepo setup presents significant challenges.

That's just my $0.02.

[kpgalligan]

I don't think there's a technical issue with a monorepo. It's the workflow reasons above that are the issues. They're not blockers. It all depends on the app and team.

[Bradleycorn]

Yeah, monorepo actually works great.
It just doesn't scale very well.

[Joaquim Puyo]

Thanks a lot for your feedback!