I noticed some failures to fetch the bindings (<ex...
# github-workflows-kt
p
I noticed some failures to fetch the bindings (example 1, example 2) but the failure metric is constantly at 0. However, the cache size metric has been creeping up for the last hours, not really sure why. Something weird is happening, and it seems to be related to the traffic pattern
the next step would be probably to check the logs for any clues, I can take a look tomorrow. For now things seem to be stable again, but it's worrying that no failure metric was recorded, as if the problem was with some other piece of infra than the server? @LeoColman are you performing some maintenance of your infra?
v
Is that also why the CI is yelling at me? I don't see any reason why it should
I mainly see many 404s in the log. Many for
org/jetbrains/kotlin/kotlin-stdlib/unspecified/kotlin-stdlib-unspecified.pom
which seems strange anyway. But also quite some for ktor and whatnot, which should probably be resolved from Maven Central instead. To speak with your first example, the only
fwilhe2
I find in the log are
200 OK
. So if something fails, then somewhere in-between. No 500s. Though https://github.com/typesafegithub/github-workflows-kt/pull/1924 could hide them but then it would have been 404 not 200.
l
Negative, no different flow on the server
v
But what is broken then? 😞
p
@LeoColman what's the name of the datacenter you rent your server? maybe they had some network-related outage
for GitHub, https://www.githubstatus.com/ is all green, and https://downdetector.com/status/github/ shows a spike in reports
l
It's Hetzner, let me check
p
nothing obvious to me?
l
nothing obvious to me either
p
maybe we should also surface a metric about # of 404s - I know that we normally get a steady level of these because of what requests come to our service, but e.g. if we see a spike there, it may give us some clue
(yeah, I'm aware of Do not swallow exceptions while generating bindings but log and show internal server error, thanks for this - will review shortly)
❤️ 1
v
None of these 404s is what is fixed there. I used my custom types branch to develop a typing file and got 404 even though it should have worked, just then with debugging found, that there was an exception and it should have been 500
👍 1
But yeah, as a separate metric 404s might make sense maybe
👍 1
Also, incident seems over. Rerun without change, now CI is green
👍 1