Comments (16)
I hit this issue the other day when I try to use bazel to run unit tests while offline, and I have to temporarily create a .bazelversion
file to make it work.
I also didn't realize about the CI issue (we build a base docker image contains bazelisk and caches the latest bazel version when building it so we don't have to download the latest bazel version on every CI run), but that does make a lot of sense for an offline use case.
Considering the two separated issues, I think maybe two environment variables/.bazeliskrc
options would makes sense:
BAZELISK_FORCE_CACHED_BAZEL
for CI use cases: treat the latest locally cached bazel version aslatest
and bypass github.BAZELISK_ALLOW_OFFLINE
: when it cannot talk to github but there are locally cached bazel versions, print a big warning and use the locally cached version as a fallback.
from bazelisk.
Our CI is failing regularly with a similar error message even with internet access. It seems like api.github.com is unreliable?
This is a concern because GitHub installs bazelisk at least on their Mac workers as bazel
, resulting in a bad user experience for projects that use GitHub actions w/ bazel for CI.
from bazelisk.
This is a concern because GitHub installs bazelisk at least on their Mac workers as
bazel
, resulting in a bad user experience for projects that use GitHub actions w/ bazel for CI.
Oh, wow - they install Bazelisk on their machines? By default? 😃
I agree that their API endpoint reliability is somewhat lacking. I'll make sure that we improve Bazelisk to not rely on it for use cases where we can get the information from a more reliable source.
from bazelisk.
@madhavajay Great that you got it working! I haven't played with GitHub Actions and their specific environment yet, but if I do and find out something, we can update the docs here with some guidance.
Regarding overriding the mechanism - if you put your downloaded binary into $workspace/tools/bazel
, Bazelisk should detect it and automatically use it. :)
$ cd ~/workspace
$ bazel version
Build label: 3.1.0
$ curl -LO https://github.com/bazelbuild/bazel/releases/download/2.1.0/bazel-2.1.0-darwin-x86_64
$ mv bazel-2.1.0-darwin-x86_64 tools/bazel
$ chmod +x tools/bazel
$ bazel version
Build label: 2.1.0
from bazelisk.
For Github Actions one should set the env BAZELISK_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
or rate limitting will hit eventually (it hits sooner on macos workers as there are fewer of them and RL is by IP).
Example usage: https://github.com/fenollp/bazel_upgradable/blob/master/.github/workflows/workflow.yml#L5
from bazelisk.
@fenollp Is that documented anywhere?
from bazelisk.
Is there a way to run Bazelisk offline? We also install Bazelisk as default
bazel
on our developer machines.
There is if you want to peg your Bazel version to a specific version, like I mentioned in above comment. For now though, you're out of luck if you try and use the latest version.
I hope this issue gets fixed in one way or another.
from bazelisk.
Hi @HemangLavana,
that's a great feature request and something I definitely would like to add. Happy to review PRs for this in case anyone wants to give it a try, otherwise I'll take it on the next time I have time to work on Bazelisk. :)
from bazelisk.
This is maddening!?
I don't seem to be able to bypass bazelisk and just use my own copy of bazel.
https://github.com/OpenMined/SwiftDP/runs/649228808?check_suite_focus=true
curl -LO https://github.com/bazelbuild/bazel/releases/download/3.0.0/bazel-3.0.0-darwin-x86_64
mkdir -p "${GITHUB_WORKSPACE}/bin/"
mv bazel-3.0.0-darwin-x86_64 "${GITHUB_WORKSPACE}/bin/bazel"
chmod +x "${GITHUB_WORKSPACE}/bin/bazel"
${GITHUB_WORKSPACE}/bin/bazel --version
which bazel
set -o pipefail
/usr/local/bin/bazel build //src:SwiftDP --ios_multi_cpus=x86_64,arm64 -apple_bitcode=embedded --copt=-fembed-bitcode
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 636 100 636 0 0 6000 0 --:--:-- --:--:-- --:--:-- 6000
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
39 39.7M 39 15.5M 0 0 12.0M 0 0:00:03 0:00:01 0:00:02 15.8M
100 39.7M 100 39.7M 0 0 17.8M 0 0:00:02 0:00:02 --:--:-- 20.8M
bazel 3.0.0
/usr/local/bin/bazel
2020/05/06 11:01:23 could not resolve the version 'latest' to an actual version number: could not get releases from github.com/bazelbuild/bazel: could not download list of Bazel releases from github.com/bazelbuild: unexpected status code while reading https://api.github.com/repos/bazelbuild/bazel/releases: 403
##[error]Process completed with exit code 1.
Huh? 🤷♀️
from bazelisk.
@madhavajay In your code snippet, you download Bazel 3.0.0 manually using curl and put it into a folder called bin
underneath your workspace. Then you go on and don't actually run it, but instead run /usr/local/bin/bazel
(which seems to be Bazelisk).
Of course this will run Bazelisk and not the binary you just downloaded. If you want to run your download copy of Bazel, you'll have to run that and not the one in /usr/local/bin
.
If you expected that Bazelisk would detect that a file called $workspace/bin/bazel
exists, is the correct version and then use that: This feature sounds interesting, but it doesn't exist - is there a reason you assumed that would work?
from bazelisk.
@philwo I hadn't heard of bazelisk till yesterday. :) I love the idea, just wondering if there is a way to override this functionality by say passing it a local binary for instance (or passing it a preferred download url for a specific bazel binary) like this where the local install is just stomping my usage but also failing. I figured out that I wasn't using the paths correctly thanks to your comment, it was late last night when I was just trying to get the build to pass. I haven't used GitHub Workflow before so I didn't realise it seems like that downloaded binary of mine wasn't available in the subsequent task, so for now im just running it in 1 step.
The good news is I have my tests passing for now:
https://github.com/OpenMined/SwiftDP/runs/651254212?check_suite_focus=true
Do you know if this is just an issue at github's end and that it should normally work?
from bazelisk.
@philwo thats a great work around I will investigate that.
@fenollp that makes more sense. It would be nice to see that in the error message actually e.g.:
could not download list of Bazel releases from github.com/bazelbuild: ...
Do you have a BAZELISK_GITHUB_TOKEN set?
Or something similar.
from bazelisk.
Just wanted to know if this issue is still being looked at because I think this is an issue for bazelisk when the internet may be temporarily unavailable
Notes:
- I have copied bazelisk as bazel and put it in my system path.
- bazelisk is trying to get the latest version
I did a quick test to confirm my suspicions which is shown below.
family@family-desktop:~/Development/stm32$ bazel build //test
INFO: Analyzed target //test:test (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //test:test up-to-date:
bazel-bin/test/test
INFO: Elapsed time: 0.269s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
At this point, I switched off the network connection and ran the command again
family@family-desktop:~/Development/stm32$ bazel build //test
2022/05/14 16:51:07 could not resolve the version 'latest' to an actual version number: unable to determine latest version: could not list Bazel versions in GCS bucket: could not list GCS objects at https://www.googleapis.com/storage/v1/b/bazel/o?delimiter=/: could not fetch https://www.googleapis.com/storage/v1/b/bazel/o?delimiter=/: Get "https://www.googleapis.com/storage/v1/b/bazel/o?delimiter=/": dial tcp: lookup www.googleapis.com on 127.0.0.53:53: server misbehaving
If I specify a version by giving USE_BAZEL_VERSION = 4.2.0, then it will use the cached version regardless of my connection status, however I do think it is an issue that bazelisk aborts when there is no network connection when it is trying to use the latest Bazel version.
Perhaps a change in the way the algorithm determines which version of Bazel to run is needed so that a network connectivity issue will not abort a build.
- If the environment variable USE_BAZEL_VERSION is set, it will use the version specified in the value.
- Otherwise, if a .bazeliskrc file exists in the workspace root and contains the USE_BAZEL_VERSION variable, this version will be used.
- Otherwise, if a .bazelversion file exists in the current directory or recursively any parent directory, it will read the file and use the version specified in it.
- Otherwise it will use the official latest Bazel release.
- Otherwise it will use the cached Bazel release.
from bazelisk.
Is there a way to run Bazelisk offline? We also install Bazelisk as default bazel
on our developer machines.
from bazelisk.
strategic comment to track this PR .... would be great If I had this feature included in my bazelisk instance :)
from bazelisk.
It's sad this didn't get addressed in almost 5 years :(
For now, I will add the work arounds, and code pointers that could help others who stumble upon.
-
First find the latest version.
a. Find the caches directory the user caches directory for your OS. For example, if it is Mac OS, it will be$HOME/Library/Caches
b. List the files in the subdirectorybazelisk/downloads/metadata/bazelbuild/
under the caches directory. For example, run,ls -l $HOME/Library/Caches/bazelisk/downloads/metadata/bazelbuild/
ls -l $HOME/Library/Caches/bazelisk/downloads/metadata/bazelbuild/ total 56 -rw-r--r-- 1 jp staff 64 Nov 28 11:37 bazel-6.4.0-darwin-arm64 -rw-r--r-- 1 jp staff 64 Dec 11 10:54 bazel-7.0.0-darwin-arm64 -rw-r--r-- 1 jp staff 64 Jan 18 13:18 bazel-7.0.1-darwin-arm64 -rw-r--r-- 1 jp staff 64 Jan 25 10:21 bazel-7.0.2-darwin-arm64 -rw-r--r-- 1 jp staff 64 Mar 11 11:25 bazel-7.1.0-darwin-arm64 -rw-r--r-- 1 jp staff 64 Mar 21 19:01 bazel-7.1.1-darwin-arm64 -rw-r--r-- 1 jp staff 64 May 8 14:43 bazel-7.1.2-darwin-arm64
-
Simply run as,
USE_BAZEL_VERSION=7.1.2 bazel build //...
Some code pointers if anyone wants to take a stab at it. Note the code might change, so use it as a reference and search for the similar names in the latest commit.
The bazel version lookup and download happens here at the downloadBazelIfNecessary function,
Line 355 in 47f6047
If there should be a logic for fallback on download failure, it should most likely be in the function RunBazeliskWithArgsFuncAndConfigAndOut
around this
Line 133 in 47f6047
from bazelisk.
Related Issues (20)
- Bazelisk run doesn't propagate signals to the child process
- Bazelisk 1.18 doesn't handle WORKSPACE-less repositories HOT 3
- Bazelisk does not print its version anymore in `1.18.0` when calling `bazelisk version` HOT 2
- py_bazelisk_test & py3_bazelisk_test is failing with Bazel@HEAD HOT 2
- Support BAZELISK_VERIFY_SHA256 for multiplatform projects HOT 1
- .bazelversion not being read in parent directory outside of WORKSPACE HOT 2
- Bazelisk is failing with "httputil:go_default_test" at Bazel@HEAD HOT 2
- `latest-1` is resolved as 7.0.0 instead of 6.4.0 HOT 1
- could not download Bazel when I want to execute the command:bazel build onos HOT 9
- bazelisk run/test hide segmentation violations
- Error loading package under directory error with Bazel@HEAD in CI HOT 6
- How to avoid bazel installation? HOT 6
- Could not download Bazel - bad file descriptor HOT 1
- caoky HOT 1
- Use cached executable if no network available
- Steps for installing Bazelisk are not new-developer friendly HOT 2
- Bazelisk pulls incorrect Bazel on Windows ARM64 HOT 2
- bazel version 6.5.0 install issue HOT 3
- Add an automatic workflow to update winget pkgs
- Support `Tools/` in case-sensitive file systems HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bazelisk.