404 on last_green about bazelisk HOT 4 CLOSED

It's happening again with Bazel 3.1.0 for Linux ARM 64: https://releases.bazel.build/3.1.0/release/bazel-3.1.0-linux-arm64

I'm using Bazelisk version 1.7.3. I've also tried rolling back to 1.7.1 (by npm installing it) and the same issue persists. Is there any sort of workaround? It seems like no matter what version of Bazelisk is used, Bazelisk tries to install the latest Bazel, and there's no option otherwise.

from bazelisk.

Our pipeline for publishing Bazel binaries had some issue during the process of adding Apple Silicon machines, now it's resolved.

from bazelisk.

@KeithMoyer I can confirm that this happened a few times in the past. I'm looking into how to fix this reliably - we probably have to do this in the backend mechanism that updates the last green commit on CI. I'll update this bug when there is progress.

How long are "last green" builds kept available after a new green commit is added?

The last green builds are currently kept forever. However, there is a known race condition where the "last green" pointer can get updated before the binaries have been uploaded (because both happen in different pipelines at the moment).

from bazelisk.

This also affects the CI of the Bazel Central Registry (last in https://buildkite.com/bazel/bcr-presubmit/builds/569).

It looks like Bazelisk is using the existence of a directory as the indicator for the availability of a particular version. Is it possible that the build job uploads the individual binaries into this location as they are built rather than atomically after all have been built? Could the build jobs upload into a temporary location and then trigger a final job that moves the bucket directory into the expected location atomically?

Alternatively, perhaps Bazelisk could check the contents of the directory for the particular binary it would download.

CC @meteorcloudy

from bazelisk.