https://kotlinlang.org logo
Join Slack
Powered by
Hello :kotlin_emoji: fans worldwide, We have crea...
# feed
s
Hello K fans worldwide, We have created a new tool which accelerates Kotlin builds, particularly with Gradle, but also with support for Maven, Bazel, and others. We are a cloud service, but now we've released a beta version that works for free on all platforms (Windows, Mac, Linux). It might sound like magic, but it's true: you download it, run it during your build, and things get faster. We're looking for a few beta users. We want to make sure our local tools are ready for prime time before our big launch. We'll even give you an API key so you can use the paid service for free. If you're interested, drop us a line here on DM. The service works anywhere in the world and the download works entirely locally. The best part is, it's all written in Kotlin, of course, from the service to the app! 🙂 Even if you don't want to try it / can't because of work conditions, please throw us an upvote, every bit of help counts. We are a young open source company trying to make your builds faster and your life easier 🦜 Thank you 🙏 - the Buildless / Elide team https://less.build
⚡ 4
mind blown 1
👀 2
👍 1
K 16
P.S., if you are curious about the technical details, please let me know and I can post more here 🙂 But, in essence, it provides an in-memory build cache, with optional Cloud integration, over HTTP/3. It can also accelerate Maven downloads by upgrading to HTTP/2 or HTTP/3. It turns out Gradle can only speak HTTP/1.1, and there is a potentially significant gain just by upgrading this. Since the software installs alongside Gradle, it can work with even ancient versions of Gradle with no modification to your build, aside from one plugin or two settings and no plugin.
a
I still do nor get how HTTP 2 or 3 may change anything. The dependencies are downloaded once. How can it be quicker the first time when the server does not support HTTP 2 or 3?
👍 1
s
@AndreyVanDenHaag dependency metadata is downloaded surprisingly frequently; not just that, but during such downloads, HPACK compression and smart connection management can have an impact Gradle’s default methodology is over HTTP/1.1 with an artificial connection limit set based on the number of IPs returned for the host. These days, with HTTP/2, we don’t do that because we have multiplexing now, and because one IP may actually represent hundreds or thousands of hosts. Through our agent, there is no such limitation. We perform build caching over the same protocols described here so it isn’t just dependency downloads. 🙂
Also, because of the way our proxy works, we can upgrade to HTTP/2 or HTTP/3 without the destination server supporting it. We do this with a caching cloud proxy powered by Cloudflare. That’s a product called Pkgst which is always free.
c
We all want faster builds in typed(/compiled)-languagistan. So how do I switch this on? And how do I benchmark the improvement correctly? I just went to your website: do I need to "join the preview"? Will it work without the cloud option? (i expect our code leaves our office when the cloud option is switched on, so this needs to be discussed) Do you expect there will de an on premise option some day?
s
@Eric Ou all fantastic questions! Thank you. This helps us understand where people's thoughts are. Let me do my best to answer below 👇 > So how do I switch this on? You can use the Buildless Agent local-only, for free forever, and optionally you can hook it to our Cloud, which is backed by Cloudflare and Dragonfly DB (very fast redis) Since we're here in Kotlin I'm going to assume Gradle, which is our best integration, but we have others too. For Gradle, you would: 1. Install the 
build.less
plugin in your build 2. Install the 
buildless
command line tool, and start the Buildless Agent background service 3. Run builds, that's it The plug-in will notice the running agent and use it as a build cache. > To activate cloud (Continuing from above) 1. Set 
BUILDLESS_APIKEY
in your environment 2. Run more builds 3. That's it! > And how do I benchmark the improvement correctly? We recommend using Gradle's built-in reporting for this, via the 
--scan
feature, which shows in the Performance section how build caching helped during the build. > I just went to your website: do I need to "join the preview"? That's one way to do it, yes, if you want to try the Cloud service (which is entirely free btw for all usage right now, and will be for some time). We are sponsored by cloud credits. You do not need any login or API key at all to use the Buildless Agent alone without Cloud enabled. You can also just DM us here if you prefer 🙂 we're just engineers. The form on the website goes to us too. > Will it work without the cloud option? Yes. The Buildless Agent is free forever for unlimited usage. Cloud is entirely optional. We don't even annoy you about it, promise 😄 > (i expect our code leaves our office when the cloud option is switched on, so this needs to be discussed) Yes, but to us this is no different than GitHub over SSH or TLS, and we maintain an extremely strong security posture. We're working on innovative ways to encrypt and sign the cache blobs but that requires sharing with teams so it's a bit complex. You can see our TLS report here as an example. > Do you expect there will be an on premise option some day? Yes, we've architected for this from the beginning. We'll be releasing a Private Cloud version first, with support for AWS, GCP, and Azure, with self-hosted / on-premise coming after that. If you have needs in this area let us know and we'd be happy to hear them so we can tailor the product to fit.
e
Do you have any baseline comparison for build execution time with and without this tool? I'd love to see the term faster quantified.
c
^ with a comparison to the regular build cache
s
@Endre Deak obviously those stats can vary based on project structure, velocity of change in the code, and velocity of underlying dependencies, etc., but yes 🙂 we have some sample projects. We post the sample code publicly and the builds run publicly, so you can examine the Gradle reports yourself. Here is our Android/Gradle sample, for instance: https://github.com/buildless/sample-android-gradle
Screenshot 2023-12-20 at 10.58.58 AM.png
That image is from the README
@CLOVIS when you say "regular build cache," do you mean Gradle's built-in disk caching? There are many cases where disk caching is honestly the most effective option, but we have particular trouble with it in Github Actions, and obviously it can't be shared. In some circumstances, if the Buildless plugin doesn't think it can be profitable, it will back out and enable disk caching instead, so I have to admit we are fans of Gradle's built-in cache.
On the other hand, if you are going to use a shared remote cache, you want it to be fast, and you want it to backhaul to the Cloud async so you don't have to wait for uploads. You want the best compression tradeoff and connection management possible. That's what the Buildless Agent does
(We give away most of our solution entirely for free, and we are planning to open source it -- the CLI and Agent -- so in this regard we really are just here to make all of our builds faster)
One more thing -- of course the Cloud service is integrated with SSO, SCIM, and so on, and you can manage permissions for your caches or segment them. That's the sort of thing our paid plans are intended to do.
c
s
@Endre Deak also one more point re/above -- Gradle's materials say that a properly configured remote build cache can save up to 85% wasted compile time. We're simply a fast implementation of that same protocol.
c
s
Ah yes, gotcha @CLOVIS. The difference here is that you don't have to host yourself. I actually started with that and tried it about a year ago, but management of the Docker container... making sure it stayed up, pruned, with enough space, was a challenge
Buildless solves all of that by bringing it to an affordable managed service. We want to hit a price point that lets you sign up on Github for about $20 or less.
So: you can't host Buildless in a container like you can the Build Cache Node (yet), but it is much easier to use.
On a performance basis, it's not even a comparison: we speak HTTP/2, HTTP/3, many more compression options, and we edge-cache with Cloudflare aside from our core cache with Dragonfly
Many transport-layer, TLS-layer... etc., optimizations, which I'm happy to detail if anyone is curious. Ultimately I am a transport nerd. Lol
(On another note, Gradle is great, but we've found their non-Enterprise offerings a bit lacking in this category, which makes sense -- they want to push customers to convert! We want to be a more affordable, shared-cloud alternative.)
c
Ultimately I am a transport nerd.
Not gonna lie, I would be disappointed if the person advertising a remote cache product wasn't 😅
s
Lol
it takes a certain person to be excited about build caching every day 😛
Here's an example of how we really care about the transport layer. We can actually speak pure Redis. https://docs.less.build/docs/redis
It's not Just Redis either! Lol. That would be lazy.
c
I actually started with that and tried it about a year ago, but management of the Docker container... making sure it stayed up, pruned, with enough space, was a challenge
We've been using it in K8s for about a year now with no issues whatsoever, but maybe we're lucky
s
@CLOVIS I can imagine with K8S it wouldn't be so bad, because at least you'd have lifecycle. I think I was just running a simple container.
c
On a performance basis, it's not even a comparison: we speak HTTP/2, HTTP/3, many more compression options, and we edge-cache with Cloudflare aside from our core cache with Dragonfly
That's very interesting. Do you have metrics on what happens if you run your own benchmarks on your own server, but with HTTP1.1?
s
Part of the problem there is connection management; because Gradle pools connections, which is a good practice in HTTP/1.1, there is no chance for HPACK if you don't have this agent in between
👀 1
HPACK header compression can save a lot especially for burstable traffic with many similar headers (like a build cache)
HTTP/3's performance gains are a bit murky, we just released that, but my theory is that the better retry logic of QUIC is helping us on higher-latency links
The answer is no, honestly, we need more visibility
c
it takes a certain person to be excited about build caching every day 😛
I mean… I made https://opensavvy.gitlab.io/pedestal/api-docs/cache/index.html
🚀 1
s
This is awesome
I see Arrow compatibility too lol
nice
honestly I may download this, we see a lot of friction using Coroutines with Caffeine
(also, btw, Gradle limits that connection pool by the # of IPs returned by a Maven host lol)
🤔 1
(which is... insane in the HTTP/2 era)
I tried shipping a change for it but it was beyond my qualifications
c
It's probably going to be a bit slower than Caffeine because I don't have their experience, but it's fully MPP and fully built upon coroutines. If you're interested, please do join https://discord.com/channels/1170433881174986752/1184241127293984941, I'm searching for people with experience in the caching world to fix some API issues 🙂
s
The bottleneck is never caffeine anyway lol
c
(or if you don't like discord, I also have #decouple, though it's for another project)
s
I will join 🙂
thank you color 1
Oh, oops, the link took me to a channel, not a server. But if you have an invite, I hang in Discord all day 🙂
c
Oh, links to channels don't let you join?
s
We'd love to know what magic we'd need to pull to make you prefer buildless over your k8s cluster 😂 lol
a high bar perhaps
Screenshot 2023-12-20 at 11.16.34 AM.png
it just gave me this
c
s
ah, that worked!
c
We'd love to know what magic we'd need to pull to make you prefer buildless over your k8s cluster 😂 lol
That's… going to be difficult 😅 Our K8s is completely home-grown If you ever release an open source self-hosted version of your app though…
s
We very well might! Or at least a private cloud version on the way that is essentially adjacent. We just haven't spoken to users who want it yet so we don't know what to include.
The software is very amenable to it
It is designed, in fact, to host in a containerized environment anywhere. All secrets etc are remote, it's a native GraalVM binary, baked into a very tiny image
👀 1
So we aren't even that far, I just don't think we've been asked to explore it yet. Maybe we build it for you guys or in collaboration 🙂 who knows
I see you also do OSS work
c
By the way if you want feedback, my main two points of concern with the project how you originally announced it were: • why should I use it instead of a build cache node? Either it's implemented as a build cache node, but then why would it be better (you addressed this), or it's something entirely different, but that would require major impacts on the Gradle side • this is a build cache server, it by nature has access to all code, can I trust a SaaS solution?
That second one will the hardest point to convince me 🙂 But I don't believe it to be much of a problem for most people
s
That's fantastic feedback, thank you. We're still in the vortex of product development so any external feedback helps a ton and we are always looking for it. We have a really good answer for security; a cryptographic answer, but one that requires a bit more refining before release. Multi-party signing and encryption isn't something one should usually do themselves so we're designing the solution carefully and picking trusted code to facilitate implementation
Ultimately, though, since you do trust keys you generate yourself, and escrow for others (yourself), we think it can be done
We don't actually inspect the blobs we cache. Gradle, for example, encodes with a gzip-like format that includes certain object metadata, most notably the time it took to build the artifact uncached
👀 1
We could be using that to show metrics, but we don't yet, and we need an encryption scheme that lets you opt-in to that, or keep it entirely black-box encrypted
c
See, that's very interesting
s
So, even though the security isn't perfect yet, we aren't guaranteeing it because it isn't and it will be soon
We could probably encrypt the blob black-box and ship the metrics ourselves with our plug-in, for example
there are different ways to do it
But we're early 🙂 it's only just now working smoothly. All of that is in the works!
plus, when you onboard/offboard developers, you'll want to revoke keys, auto-issue keys, etc.
Also, just generally, we are an OSS company, and so we're always open to collaboration
you may have seen some of our other projects, like Elide, a kotlin-first JS runtime
c
Ohhh, I didn't make the link with Elide
❤️ 1
s
the theme here, really, is all about getting code to market faster with less headache
c
yeah, I've been really curious about it 🙂
s
Elide is very new, but because it has fully instrumented I/O, it is an incredibly healthy runtime for build caching
😉 lol
it greatly upsets me when i push a readme change and it triggers a full build/test
🤣 1
c
oh, if you're interested in KJS, I'm currently working on https://gitlab.com/opensavvy/automation/kotlin-vite 😇
s
uh yes, in fact, big fan of kjs
hopefully you have seen the kotlin 2.0 ES6 stuff coming
c
I have lightly heard of it, but not hard to chance to actually delve into it
s
big change bc ES5 is so outdated, doesn't even have classes as you may know
so i think that will really unlock KJS for the world
https://elide.dev is actually KJS react lol
but executing server side, ssr, inside Elide
c
> uh yes, in fact, big fan of kjs You may notice that among the other people in my discord are the creator and one of the contributors to Kobweb 🙂
s
ha ha small world 🙂 man we should have a conference someday
c
lol we definitely should
s
kotlin server side conf when
c
what about towards the end of may?
Copenhagen sounds great, I've never been
😇
s
Lol
if we're funded by then definitely
c
(I'll attend this year)
s
ahhh yes, i think this may be our year
we almost went in 2023
c
See, the problem is you have exactly the profile for me to advertise everything that I've made
Well, my main project is https://gitlab.com/opensavvy/decouple, everything else is support for it
s
Ahahaha, same, apparently
this is also very cool
i wonder if we could integrate this, we probably could, we already use the Compose compiler
I've seen or used adjacent work before like Mosaic
c
do advertise Elide and Buildless in the #external channel in the discord, I'm very interested by what the Kobweb teams thinks about it
Yeah, Decouple is basically meant as a layer between Mosaic, Compose for Web HTML, Compose for Desktop etc, so you can use them directly to create your design system(s), and then use regular composable functions for business rules
this way you have access to all features of the platform in your design components, and are fully multiplatform in your business components
Personally, I truly believe in the tech of Compose HTML, but I can't use it as is since the APIs are too different from all the other platforms
s
yep, I hear you there
we should move this to the other chats so as not to spam #feed, but actually i have some questions 🙂 i'll ask in your discord
c
perfect for me 🙂
c
Thanks @Samuel Gammon for the answers. As said: all for quicker builds. I'll give yr product a spin, inshallah today.
🚀 1
s
@Cies that sounds great 😄 I'm at 
sam@less.build
, we can get you keys provisioned in only a couple hours if you want to try cloud. the Agent binary can be downloaded here
Docs here
Even the Cloud is all free during the beta, and honestly probably forever for our early adopters, I just can't say that in public yet 😉
👀 1
4 Views
Open in Slack
PreviousNext
Powered by