So, I'm working on a monorepo POC, trying to incorporate elements from the testing hyperpyramid and http4k-connect. I'm looking for some suggestions from the community: Monorepo Build Tools I'm using Gradle, Bitbucket Pipelines, and hoping to deploy to Amazon ECS. What tools do you suggest to deploy only affected services? I know Bitbucket Pipelines has a

changeSets

condition, but it's not smart enough to detect transitive changes, unless I explicitly define them. I'm hoping for a tool that isn't super over-engineered; something that's a component of the pipeline, rather than a replacement. Testing Hyperpyramid Say I have a layout like:

cats/
  contract/
  server/ (depends on cats:contract)
owners/
  contract/
  server/ (depends on owners:contract)
adoptions/
  contract/ (depends on cats:contract and owners:contract)
  server/ (depends on (adoptions:contract)
gateway/
  contract/ (depends on cats:contract, owners:contract, and adoptions:contract)
  server/ (depends on gateway:contract)

The

*:contract

modules declare the http4k-contract, client, and fake client. If I have an

AdoptionsContractTest

that has

Fake

and

Real

implementations, and an

adopt

operation is supposed to fail if an owner or cat doesn't exist, how complete would my

FakeAdoptionsClient

be? 1. Would it depend on

FakeOwnersClient

and

FakeCatsClient

? 2. Would you have to inform the

FakeAdoptionsClient

that a given cat or owner exists? 3. Would you completely omit this requirement from the contract? Clearly, Option 1 is the most correct, but comes off as fairly unmaintainable to me. Imagine how complex the interconnected dependencies would be once you add a

gateway

service that depends on it, or a

payment

service that depends on

owners

?

What tools do you suggest to deploy only affected services?

The monorepo projects I see rely on code that can generate a standardised pipeline configuration out of a custom service descriptor. Dependencies are either defined in this high-level descriptor or generated based on build tool dependencies (e.g. running

./gradlew dependencies

), or the build is "flat" (i.e. each service is built only when itself changes, similarly to a multi-repo setup). Each setup has a trade-off, including how much the org is willing to invest in platform engineering.

Basically the idea is to have each service have a

server

module that's deployed, and a

contract

module that includes the client and fake client for that service. In my current polyrepo, I've found it cumbersome and error-prone to maintain duplicates of clients and fake clients in every service. I was hoping that a shared client could alleviate those issues without devolving into a distributed monolith. Assuming endpoints don't introduce breaking changes, I figured maintaining backwards compatibility wouldn't be any more difficult than in a polyrepo 🤷 .

I was hoping that a shared client could alleviate those issues without devolving into a distributed monolith

Yes, sharing clients would alleviate some of this and also introduce different challenges, like potentially unnecessary build/deployments of all consumer services when a producer changes. In continuous deployment scenarios we've worked on the last few years, that'd be tricky, but maybe in manually-triggered deployment scenarios, that'd be acceptable. It's a trade-off, and you're right that regardless of this setup, one needs to be attention to runtime compatibility. I don't mind covering this topic here, especially since we've mentioned that approach in our talk. I'd also encourage trying different ways and reporting back, as I'm sure we can all learn from each other 🙂

Yes, in a continuous deployment scenario, you would likely get many unnecessary deployments. Perhaps there are ways to mitigate it with different changeset filters, but there would still be a tradeoff, as you noticed.

I enjoyed reading that discussion and hope there will be more of those in the channel :)