🎰 Tests the full lifecycle of a Homebrew change.
License: BSD 2-Clause "Simplified" License
This repository was deprecated and split into two repositories:
Library/Formula)
![dependabot-preview[bot] avatar](https://avatars.githubusercontent.com/in/2141?v=4 "dependabot-preview[bot]")
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Reffer:https://travis-ci.org/Homebrew/homebrew-science/builds/180950856
CI tests on Homebrew/homebrew-science fails constantly due to the following error in executing brew test-bot, on line 770 and 771:
==> brew --env
/home/linuxbrew/.linuxbrew/Library/Homebrew/brew.sh: line 353: /home/linuxbrew/.linuxbrew/Library/Homebrew/vendor/portable-ruby/current/bin/ruby: No such file or directory
I think this is caused by the following process, from line 761 to 764:
git clean -ffdx --exclude=Library/Taps
Removing Library/Homebrew/vendor/portable-ruby/
Removing share/
Removing var/
Why brew deletes portable-ruby by itself?
Hi,
Going by @dawidd6's homebrew-tap workflows, I noticed that the command does not pick up my tap, and defaults to Homebrew/core.
Here's the failing yaml:
https://github.com/ddelange/brewformulae/blob/594254e1cbd2c4a906f3308d2633e838fd7f6bcb/.github/workflows/brew-test-bot.yml
Here's an example run of the command correctly picking up the tap it's in:
https://github.com/dawidd6/homebrew-tap/runs/975810574#step:4:7
My hunch is that the command looks for the homebrew- prefix in the repo it's in, but doesn't find it on my repo, because it's no longer called https://github.com/ddelange/homebrew-brewformulae (and instead redirects, so no problem for brew when I'd do a brew install ddelange/brewformulae/yt.
Any chance I can trick the test-bot command into picking up my tap, and doing the checks?
I maintain the osrf/simulation tap and have noticed a recent behavior change in test-bot. It used to be that an _or_later suffix could be appended to the macOS version describing the sha256 data in a bottle do block, which would allow a bottle to be used with newer versions of macOS. At some point, the _or_later suffix was removed as any bottle could be used with a newer version of macOS. Our jenkins instance at build.osrfoundation.org currently has Catalina and Big Sur nodes for building bottles. Until fairly recently, we were building bottles for both of these versions. Starting on Nov 3rd (first noticed in osrf/homebrew-simulation@fb7743c) any formula that didn't have all dependencies bottled for a given version started using --build-from-source instead of --build-bottle in test-bot. One of our formulae (ignition-cmake2) without a big_sur bottle does not have compiled code, just text files, so it is truly reusable on any macOS version. When _or_later suffixes were part of brew, I used that suffix with this package. Now, since it doesn't have a big_sur bottle, test-bot will not build big_sur bottles for any of its dependents.
I'm not asking for anything to be reverted; this has mainly spurred me to be more proactive about building big_sur bottles. I did want to mention it though because it feels like a behavior change to me, and I haven't noticed any explicit discussion about it.
As a side note, it would be really helpful to be able to generate a console message from the output of build_bottle? in lib/tests/formulae.rb to indicate which dep is preventing bottles from building built. Perhaps an API similar to unsatisfied_requirements_messages would work.
I'm using the GitHub Actions workflow that is created by brew tap-new and I'm experiencing a strange issue when testing/building a new simple maven/java formula.
The formula in question can be viewed here: https://github.com/Dekker1/homebrew-minizinc/pull/3/files
Another Java formula that that has the same formula can be viewed here: https://github.com/Dekker1/homebrew-minizinc/pull/2/files
In the ubuntu build of the formula the following step during Run brew test-bot --only-formulae consistently reports failure:
==> brew install --only-dependencies --verbose --build-bottle dekker1/minizinc/fzn-choco
==> FAILED
When looking further through the log the only kind of failure is a warning in the installation of fontconfig (which seems to be somehow required for openjdk):
==> /home/linuxbrew/.linuxbrew/Cellar/fontconfig/2.13.1_1/bin/fc-cache -frv
Last 150 lines from /home/runner/.cache/Homebrew/Logs/fontconfig/post_install.01.fc-cache:
2020-10-22 12:15:12 +0000
/home/linuxbrew/.linuxbrew/Cellar/fontconfig/2.13.1_1/bin/fc-cache
-frv
Unable to revert mtime: /usr/share/fonts
Unable to revert mtime: /usr/share/fonts/cMap
Unable to revert mtime: /usr/share/fonts/cmap
Unable to revert mtime: /usr/share/fonts/truetype
Unable to revert mtime: /usr/share/fonts/type1
Unable to revert mtime: /usr/share/fonts/cmap/adobe-cns1
Unable to revert mtime: /usr/share/fonts/cmap/adobe-gb1
Unable to revert mtime: /usr/share/fonts/cmap/adobe-japan1
Unable to revert mtime: /usr/share/fonts/cmap/adobe-japan2
Unable to revert mtime: /usr/share/fonts/cmap/adobe-korea1
Unable to revert mtime: /usr/share/fonts/truetype/dejavu
Unable to revert mtime: /usr/share/fonts/truetype/droid
Unable to revert mtime: /usr/share/fonts/truetype/lato
Unable to revert mtime: /usr/share/fonts/truetype/liberation
Unable to revert mtime: /usr/share/fonts/truetype/noto
Unable to revert mtime: /usr/share/fonts/type1/gsfonts
/usr/share/fonts: Unable to revert mtime: /usr/share/fonts
caching, new cache contents: 0 fonts, 4 dirs
/usr/share/fonts/cMap: Unable to revert mtime: /usr/share/fonts/cMap
caching, new cache contents: 0 fonts, 0 dirs
/usr/share/fonts/cmap: Unable to revert mtime: /usr/share/fonts/cmap
caching, new cache contents: 0 fonts, 5 dirs
/usr/share/fonts/cmap/adobe-cns1: Unable to revert mtime: /usr/share/fonts/cmap/adobe-cns1
/usr/share/fonts/cmap/adobe-cns1: failed to write cache
caching, new cache contents: 0 fonts, 0 dirs
/usr/share/fonts/cmap/adobe-gb1: Unable to revert mtime: /usr/share/fonts/cmap/adobe-gb1
caching, new cache contents: 0 fonts, 0 dirs
/usr/share/fonts/cmap/adobe-japan1: Unable to revert mtime: /usr/share/fonts/cmap/adobe-japan1
/usr/share/fonts/cmap/adobe-japan1: failed to write cache
caching, new cache contents: 0 fonts, 0 dirs
/usr/share/fonts/cmap/adobe-japan2: Unable to revert mtime: /usr/share/fonts/cmap/adobe-japan2
caching, new cache contents: 0 fonts, 0 dirs
/usr/share/fonts/cmap/adobe-korea1: Unable to revert mtime: /usr/share/fonts/cmap/adobe-korea1
caching, new cache contents: 0 fonts, 0 dirs
/usr/share/fonts/truetype: Unable to revert mtime: /usr/share/fonts/truetype
caching, new cache contents: 0 fonts, 5 dirs
/usr/share/fonts/truetype/dejavu: Unable to revert mtime: /usr/share/fonts/truetype/dejavu
caching, new cache contents: 6 fonts, 0 dirs
/usr/share/fonts/truetype/droid: Unable to revert mtime: /usr/share/fonts/truetype/droid
caching, new cache contents: 1 fonts, 0 dirs
/usr/share/fonts/truetype/lato: Unable to revert mtime: /usr/share/fonts/truetype/lato
caching, new cache contents: 18 fonts, 0 dirs
/usr/share/fonts/truetype/liberation: Unable to revert mtime: /usr/share/fonts/truetype/liberation
caching, new cache contents: 16 fonts, 0 dirs
/usr/share/fonts/truetype/noto: Unable to revert mtime: /usr/share/fonts/truetype/noto
caching, new cache contents: 1 fonts, 0 dirs
/usr/share/fonts/type1: Unable to revert mtime: /usr/share/fonts/type1
caching, new cache contents: 0 fonts, 1 dirs
/usr/share/fonts/type1/gsfonts: Unable to revert mtime: /usr/share/fonts/type1/gsfonts
caching, new cache contents: 35 fonts, 0 dirs
/System/Library/Fonts: skipping, no such directory
/Library/Fonts: skipping, no such directory
/home/runner/Library/Fonts: skipping, no such directory
/home/runner/.local/share/fonts: skipping, no such directory
/home/runner/.fonts: skipping, no such directory
/usr/share/fonts/cMap: skipping, looped directory detected
/usr/share/fonts/cmap: skipping, looped directory detected
/usr/share/fonts/truetype: skipping, looped directory detected
/usr/share/fonts/type1: skipping, looped directory detected
/usr/share/fonts/cmap/adobe-cns1: skipping, looped directory detected
/usr/share/fonts/cmap/adobe-gb1: skipping, looped directory detected
/usr/share/fonts/cmap/adobe-japan1: skipping, looped directory detected
/usr/share/fonts/cmap/adobe-japan2: skipping, looped directory detected
/usr/share/fonts/cmap/adobe-korea1: skipping, looped directory detected
/usr/share/fonts/truetype/dejavu: skipping, looped directory detected
/usr/share/fonts/truetype/droid: skipping, looped directory detected
/usr/share/fonts/truetype/lato: skipping, looped directory detected
/usr/share/fonts/truetype/liberation: skipping, looped directory detected
/usr/share/fonts/truetype/noto: skipping, looped directory detected
/usr/share/fonts/type1/gsfonts: skipping, looped directory detected
/home/linuxbrew/.linuxbrew/var/cache/fontconfig: cleaning cache directory
/home/runner/.cache/fontconfig: not cleaning non-existent cache directory
/home/runner/.fontconfig: not cleaning non-existent cache directory
/home/linuxbrew/.linuxbrew/Cellar/fontconfig/2.13.1_1/bin/fc-cache: failed
Warning: The post-install step did not complete successfully
You can try again using `brew postinstall fontconfig`
==> Summary
/home/linuxbrew/.linuxbrew/Cellar/fontconfig/2.13.1_1: 535 files, 3.7MB
I'm, however, unable recreate this problem in any docker images; neither using the brew test-bot commands or the commands listed in the output of the test-bot commands. Any indication of where the problem might lie would be very welcome.
P.S. I hope this is the right place to post this problem. I though since it concerned the GitHub test-bot actions, I thought it would be.
Currently, we build bottles and test these bottles all in one test-bot call. Most of the code for this call lives in lib/tests/formulae.rb.
I suggest we split this up so that this can be done in separate steps in a GitHub workflow:
I see at least three benefits from doing this:
:all bottles on the Linux runner and then test them on our macOS runners.homebrew-test-bot/lib/tests/formulae_dependents.rb
Lines 21 to 24 in 6a82b40
formulae_dependents.rb. This could potentially be done in the bottle building step.)The last benefit is pretty important: I think we're going to start regularly hitting the three-day workflow time limit on Linux fairly soon given that dependency trees tend to much larger on Linux.
So I have a Brewfile in my homebrew-dotfiles tap… which is the output from brew bundle dump --force --describe. I now also have updated my GitHub Actions CI to make use of brew test-bot (based on the tests.yml output from brew tap-new). However the brew test-bot --only-tap-syntax step obviously includes this Brewfile when running brew style on the whole tap, which results in the following errors:
Brewfile:1:1: C: Missing frozen string literal comment. (which is not output by brew bundle dump) and Use underscores(_) as thousands separator and separate every 3 digits with them. due to the long id: integer output from mas installed apps.
How can I fix this? Could/should the brew style RuboCop checks skip these rules for Brewfiles, or could the output from brew bundle dump be altered so that brew test-bot --only-tap-syntax can pass?
I encountered this while trying to set up Travis CI for a tap. Travis job log
I have only seen this failure happen if the Travis CI build is triggered by a commit to an non-master branch, not by PRs. Did not test if the same will happen if commits directly to master.
Looks like it is not isolated. Similar failures occur in denji/homebrew-nginx's log & dunn/homebrew-emacs's log. (Note that these two repos' builds also contains an Linuxbrew job that does not fail. )
The only "Error:" message that is unique to the macOS jobs is:
Error: File name too long @ dir_s_mkdir - /Users/travis/build/tommyang/homebrew-gerbera/brewbot/ (might need to click the raw log button to see, somehow hidden by Travis)
Hi
I am running
brew test-bot \
--bintray-org="$BINTRAY_ORG" \
--root-url="$BINTRAY_DL_URL" \
--tap=purplebooth/homebrew-repo \
--git-name="$GITHUB_ACTOR" \
--git-email="[email protected]" \
"purplebooth/repo/fix-compaudit"
on github actions and I'm getting audit failures from formulas that aren't "purplebooth/repo/fix-compaudit", that are installed on the base image I am using (the default macos one). I can possibly eventually work something out where I either patch these formulas or delete them or something, but it feels like they shouldn't be being audited at all, since I provide the formula under test.
Is this a bug, or am I doing something wrong?
2020-08-06T05:53:58.0482810Z ##[group]Run brew test-bot \
2020-08-06T05:53:58.0483290Z �[36;1mbrew test-bot \�[0m
2020-08-06T05:53:58.0483450Z �[36;1m --bintray-org="$BINTRAY_ORG" \�[0m
2020-08-06T05:53:58.0483580Z �[36;1m --root-url="$BINTRAY_DL_URL" \�[0m
2020-08-06T05:53:58.0483870Z �[36;1m --tap=purplebooth/homebrew-repo \�[0m
2020-08-06T05:53:58.0484020Z �[36;1m --git-name="$GITHUB_ACTOR" \�[0m
2020-08-06T05:53:58.0484170Z �[36;1m --git-email="[email protected]" \�[0m
2020-08-06T05:53:58.0484310Z �[36;1m "purplebooth/repo/$REPOSITORY_NAME"�[0m
2020-08-06T05:53:58.0643850Z shell: /bin/bash --noprofile --norc -e -o pipefail {0}
2020-08-06T05:53:58.0644050Z env:
2020-08-06T05:53:58.0644200Z REPOSITORY_NAME: fix-compaudit
2020-08-06T05:53:58.0644400Z BINTRAY_DL_URL: https://dl.bintray.com/purplebooth/bottles-repo
2020-08-06T05:53:58.0644570Z BINTRAY_ORG: purplebooth
2020-08-06T05:53:58.0644710Z ##[endgroup]
2020-08-06T05:54:00.5357840Z ==> Tapping homebrew/test-bot
2020-08-06T05:54:00.5698730Z Cloning into '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot'...
2020-08-06T05:54:02.8539070Z Tapped 1 command and 1 formula (99 files, 126.2KB).
2020-08-06T05:54:05.7272550Z �[36m==>�[0m �[1mUsing Homebrew/homebrew-test-bot a1f8223 (Merge pull request #462 from MikeMcQuaid/allowed_taps)�[0m
2020-08-06T05:54:05.7272960Z
2020-08-06T05:54:05.7273840Z �[35m==>�[0m �[1mRunning CleanupBefore#run!�[0m
2020-08-06T05:54:05.7275320Z �[34m==>�[0m �[1mgit -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --depth=1 origin�[0m
2020-08-06T05:54:07.5271720Z �[34m==>�[0m �[1mgit -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core reset --hard origin/master�[0m
2020-08-06T05:54:07.5978920Z �[34m==>�[0m �[1mbash -c mv /usr/local/Cellar/* /tmp�[0m
2020-08-06T05:54:07.7131980Z �[36m==>�[0m �[1mDetermining /usr/local files to purge...�[0m
2020-08-06T05:55:42.2669690Z �[36m==>�[0m �[1mPurging...�[0m
2020-08-06T05:57:17.8449370Z �[34m==>�[0m �[1mbrew untap adoptopenjdk/openjdk�[0m
2020-08-06T05:57:20.0315530Z �[34m==>�[0m �[1mbrew untap aws/tap�[0m
2020-08-06T05:57:21.6952250Z �[34m==>�[0m �[1mbrew untap github/gh�[0m
2020-08-06T05:57:23.2925490Z �[34m==>�[0m �[1mbrew untap mongodb/brew�[0m
2020-08-06T05:57:24.9459490Z �[34m==>�[0m �[1mbrew untap purplebooth/repo�[0m
2020-08-06T05:57:26.5423200Z �[34m==>�[0m �[1mgit -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot checkout -f master�[0m
2020-08-06T05:57:26.6385090Z �[34m==>�[0m �[1mgit -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask checkout -f master�[0m
2020-08-06T05:57:26.8351900Z �[34m==>�[0m �[1mgit -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-services checkout -f master�[0m
2020-08-06T05:57:26.8991460Z �[34m==>�[0m �[1mbrew cleanup --prune=3�[0m
2020-08-06T05:57:28.9228260Z
2020-08-06T05:57:28.9233610Z �[36m==>�[0m �[1mTesting Homebrew/brew 2.4.9 (Merge pull request #8108 from mistydemeo/upgrade_avoid_unrequested_casks)�[0m
2020-08-06T05:57:28.9234240Z
2020-08-06T05:57:28.9237140Z �[35m==>�[0m �[1mRunning Setup#run!�[0m
2020-08-06T05:57:28.9238110Z �[34m==>�[0m �[1mbrew install-bundler-gems�[0m
2020-08-06T05:59:51.0437570Z �[34m==>�[0m �[1mbrew config�[0m
2020-08-06T05:59:52.7281360Z HOMEBREW_VERSION: 2.4.9
2020-08-06T05:59:52.8432440Z ORIGIN: https://github.com/Homebrew/brew
2020-08-06T05:59:52.9028300Z HEAD: e6b066a3d9fc5435009428fcd2855fddf9b455c0
2020-08-06T05:59:52.9757880Z Last commit: 9 days ago
2020-08-06T05:59:53.0321580Z Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
2020-08-06T05:59:53.0864670Z Core tap HEAD: 85250e720999af2009cdf2128d66bd30b75423ab
2020-08-06T05:59:53.1645840Z Core tap last commit: 9 hours ago
2020-08-06T05:59:53.1646280Z HOMEBREW_PREFIX: /usr/local
2020-08-06T05:59:53.1646880Z HOMEBREW_COLOR: set
2020-08-06T05:59:53.1647120Z HOMEBREW_DEVELOPER: set
2020-08-06T05:59:53.1647340Z HOMEBREW_FAIL_LOG_LINES: 150
2020-08-06T05:59:53.1648570Z HOMEBREW_GIT_EMAIL: [email protected]
2020-08-06T05:59:53.1648860Z HOMEBREW_GIT_NAME: PurpleBooth
2020-08-06T05:59:53.1730030Z HOMEBREW_MAKE_JOBS: 4
2020-08-06T05:59:53.1736020Z HOMEBREW_NO_AUTO_UPDATE: set
2020-08-06T05:59:53.1736180Z HOMEBREW_NO_EMOJI: set
2020-08-06T05:59:53.1779500Z CPU: quad-core 64-bit ivybridge
2020-08-06T05:59:53.1780010Z Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
2020-08-06T05:59:53.2314100Z Clang: 11.0 build 1103
2020-08-06T05:59:53.3369920Z Git: 2.24.3 => /Applications/Xcode_11.6.app/Contents/Developer/usr/bin/git
2020-08-06T05:59:53.3482980Z Curl: 7.64.1 => /usr/bin/curl
2020-08-06T05:59:53.5080930Z Java: 14.0.2, 13.0.2, 12.0.2, 11.0.8, 1.8.0_262, 1.7.0_272
2020-08-06T05:59:53.5130960Z macOS: 10.15.6-x86_64
2020-08-06T05:59:53.6426870Z CLT: 11.5.0.0.1.1588476445
2020-08-06T05:59:53.8213360Z Xcode: 11.6 => /Applications/Xcode_11.6.app/Contents/Developer
2020-08-06T05:59:53.9057090Z
2020-08-06T05:59:53.9058890Z �[34m==>�[0m �[1mbrew doctor�[0m
2020-08-06T05:59:57.1306680Z
2020-08-06T05:59:57.1309170Z �[35m==>�[0m �[1mRunning TapSyntax#run!�[0m
2020-08-06T05:59:57.1310240Z �[34m==>�[0m �[1mbrew readall --aliases homebrew/core�[0m
2020-08-06T06:00:06.0232920Z �[34m==>�[0m �[1mbrew style homebrew/core�[0m
2020-08-06T06:01:40.1275550Z �[34m==>�[0m �[1mbrew audit --skip-style�[0m
2020-08-06T06:02:35.4287060Z �[31m==>�[0m �[1m�[31mFAILED�[0m�[0m
2020-08-06T06:02:35.4287910Z �[31mError:�[0m 4 problems in 4 formulae detected
2020-08-06T06:02:35.4288210Z duti:
2020-08-06T06:02:35.4288370Z * public_domain is not a standard SPDX license.
2020-08-06T06:02:35.4288520Z fmdiff:
2020-08-06T06:02:35.4288830Z * public_domain is not a standard SPDX license.
2020-08-06T06:02:35.4289430Z rbenv-gemset:
2020-08-06T06:02:35.4289860Z * public_domain is not a standard SPDX license.
2020-08-06T06:02:35.4290070Z re2c:
2020-08-06T06:02:35.4290280Z * public_domain is not a standard SPDX license.
2020-08-06T06:02:35.4290360Z
2020-08-06T06:02:35.4290450Z
2020-08-06T06:02:35.4291030Z �[35m==>�[0m �[1mRunning Formulae#detect_formulae!�[0m
2020-08-06T06:02:35.4291530Z �[34m==>�[0m �[1mbrew tap purplebooth/repo�[0m
2020-08-06T06:02:38.1280510Z �[36m==>�[0m �[1mUsing Homebrew/homebrew-core 85250e7209 ([email protected]: add linux resource)�[0m
2020-08-06T06:02:38.1281040Z url (undefined)
2020-08-06T06:02:38.1281200Z origin/master (undefined)
2020-08-06T06:02:38.1281340Z HEAD (undefined)
2020-08-06T06:02:38.1281480Z diff_start_sha1 85250e720999af2009cdf2128d66bd30b75423ab
2020-08-06T06:02:38.1281640Z diff_end_sha1 85250e720999af2009cdf2128d66bd30b75423ab
2020-08-06T06:02:38.1282570Z �[36m==>�[0m �[1mTesting Formula changes:�[0m
2020-08-06T06:02:38.1283080Z added (empty)
2020-08-06T06:02:38.1283390Z modified (empty)
2020-08-06T06:02:38.1283730Z deleted (empty)
2020-08-06T06:02:39.5792110Z
2020-08-06T06:02:39.5795500Z �[35m==>�[0m �[1mRunning Formulae#formula!(purplebooth/repo/fix-compaudit)�[0m
2020-08-06T06:02:39.6371240Z �[36m==>�[0m �[1mDetermining dependencies...�[0m
2020-08-06T06:02:42.5646620Z �[34m==>�[0m �[1mbrew fetch --retry cmake gdbm libssh2 ncurses [email protected] pcre pkg-config [email protected] readline rust sphinx-doc sqlite xz zsh�[0m
2020-08-06T06:02:56.6501460Z �[36m==>�[0m �[1mDetermining dependents...�[0m
2020-08-06T06:03:13.0045240Z �[36m==>�[0m �[1mStarting build of purplebooth/repo/fix-compaudit�[0m
2020-08-06T06:03:13.0046510Z �[34m==>�[0m �[1mbrew fetch --retry purplebooth/repo/fix-compaudit --build-bottle --force�[0m
2020-08-06T06:03:14.8036750Z �[34m==>�[0m �[1mbrew install --only-dependencies --verbose --build-bottle purplebooth/repo/fix-compaudit�[0m
2020-08-06T06:04:34.1058980Z �[34m==>�[0m �[1mbrew install --verbose --build-bottle purplebooth/repo/fix-compaudit�[0m
2020-08-06T06:05:13.7684200Z �[34m==>�[0m �[1mbrew audit purplebooth/repo/fix-compaudit --online --git --skip-style�[0m
2020-08-06T06:05:16.5095180Z �[34m==>�[0m �[1mbrew bottle --verbose --json fix-compaudit --root-url=https://dl.bintray.com/purplebooth/bottles-repo�[0m
2020-08-06T06:05:20.7648670Z �[34m==>�[0m �[1mbrew bottle --merge --write --no-commit ./fix-compaudit--0.46.0.catalina.bottle.json�[0m
2020-08-06T06:05:23.4941730Z �[34m==>�[0m �[1mbrew uninstall --force fix-compaudit�[0m
2020-08-06T06:05:25.9888860Z �[34m==>�[0m �[1mbrew uninstall --force cmake gdbm libssh2 [email protected] pkg-config [email protected] readline rust sphinx-doc sqlite xz�[0m
2020-08-06T06:05:34.5551190Z �[34m==>�[0m �[1mbrew install --only-dependencies ./fix-compaudit--0.46.0.catalina.bottle.tar.gz�[0m
2020-08-06T06:05:36.2854690Z �[34m==>�[0m �[1mbrew install ./fix-compaudit--0.46.0.catalina.bottle.tar.gz�[0m
2020-08-06T06:05:40.0142620Z �[34m==>�[0m �[1mbrew linkage --test purplebooth/repo/fix-compaudit�[0m
2020-08-06T06:05:41.5794940Z �[34m==>�[0m �[1mbrew install --only-dependencies --include-test purplebooth/repo/fix-compaudit�[0m
2020-08-06T06:05:43.2863180Z �[34m==>�[0m �[1mbrew test --verbose purplebooth/repo/fix-compaudit�[0m
2020-08-06T06:05:47.5944280Z �[34m==>�[0m �[1mbrew uninstall --force ncurses pcre zsh�[0m
2020-08-06T06:05:52.5813580Z Error: 1 failed steps!
2020-08-06T06:05:52.5826040Z brew audit --skip-style
2020-08-06T06:05:52.6611500Z ##[error]Process completed with exit code 1.
Currently, test-bot deletes bottles that fail the post-build tests (e.g. brew test or brew linkage --test). This is designed to avoid the shipment of broken bottles. However, it also removes one avenue of investigating these failures: manually downloading the bottle for inspection and local testing.
I propose that test-bot rename the broken bottles instead. I believe the relevant piece of code is
homebrew-test-bot/lib/tests/formulae.rb
Lines 657 to 660 in 328a7db
I'll try to get around to this when I have the time for it, but anyone else can feel free to work on this in the meantime.
I am trying to find a way to build arm64 bottles on a tap, using this utility, but I couldn't find out how. I had a look to the homebrew-core repository to find out how it's doing it, but got lost.
I tried to keep it simple.
My formula is targeting a golang project, so I can cross-compile by setting the GOARCH env var. I assume the project is ccorrectly compiled for arm64, at this point. However, the default file name of the bottle, when creating it, resolves on the underlaying CPU architecture (x86-64).
Is there a way to make brew test-bot resolve to arm64 during the bottling ?
Is this possible to reproduce what homebrew-core is doing to auto create the arm64 bottles ?
Supporting this out of the box could be nice as well.
Hi there,
I have a handful of taps I maintain, and I've been using the Homebrew-ordained CI solution created by brew tap-new.
In the past few days, I've had two unrelated formula on separate taps fail to build in the Linux portion of this CI run, with no clear way of fixing or debugging further, both during the brew test-bot --only-formulae step:
==> brew uninstall --force ticky/dreamcast/gditools3
Error: No such file or directory @ rb_file_s_link - (./gditools3--3.0.1.x86_64_linux.bottle.tar.gz, /home/runner/.cache/Homebrew/downloads/7b2a5e8cc188a648502bb6ef03004a5db2c384852c167ee93d9ca8a02a10a220--gditools3-3.0.1.x86_64_linux.bottle.tar.gz)
Error: Process completed with exit code 1.
GitHub Actions run (Will likely expire soon) • Full run log archive
==> brew uninstall --force ticky/playstation/paladin
Error: No such file or directory @ rb_file_s_link - (./paladin--1.3_1.x86_64_linux.bottle.2.tar.gz, /home/runner/.cache/Homebrew/downloads/a48a404125187f63ff6bb4915949b8ae21508e7ef8aa8b2090fa05ba519e3770--paladin-1.3_1.x86_64_linux.bottle.2.tar.gz)
Error: Process completed with exit code 1.
GitHub Actions run • Full run log archive
There's no obvious connection between these two formulae, and they appear to fail in the same way. There are also no other errors or warnings in the output that I can see which would suggest what's gone wrong here. In the case of the gditools3 run, this occurred repeatedly under the same conditions.
What's the best way for me to help debug this issue and get things working?
Thanks!
I think it would be nice to have an option like --exit-if-no-bottles. When passed, test-bot would just call exit for example if no bottles were found for uploading.
Relevant line of code:
homebrew-test-bot/cmd/brew-test-bot.rb
Line 1406 in bb48704
Possible patch:
diff --git a/cmd/brew-test-bot.rb b/cmd/brew-test-bot.rb
index f436505..b91a109 100755
--- a/cmd/brew-test-bot.rb
+++ b/cmd/brew-test-bot.rb
@@ -1403,7 +1403,17 @@ module Homebrew
copy_bottles_from_jenkins unless ENV["JENKINS_HOME"].nil?
- raise "No bottles found in #{Dir.pwd}!" if Dir["*.bottle*.*"].empty? && !ARGV.include?("--dry-run")
+ if Dir["*.bottle*.*"].empty?
+ message = "No bottles found in #{Dir.pwd}!"
+ unless ARGV.include?("--dry-run")
+ if ARGV.include?("--exit-if-no-bottles")
+ puts message
+ exit
+ else
+ raise message
+ end
+ end
+ end
json_files = Dir.glob("*.bottle.json")
bottles_hash = json_files.reduce({}) do |hash, json_file|
Related code:
homebrew-test-bot/lib/test_cleanup.rb
Lines 144 to 158 in 9b5f583
GitHub Actions:
Run brew test-bot --only-cleanup-after
brew test-bot --only-cleanup-after
rm -rvf bottles
shell: /bin/bash -e {0}
env:
HOMEBREW_DEVELOPER: 1
HOMEBREW_GITHUB_ACTIONS: 1
HOMEBREW_NO_AUTO_UPDATE: 1
HOMEBREW_CHANGE_ARCH_TO_ARM: 1
HOMEBREW_RELOCATE_RPATHS: 1
GITHUB_TOKEN: ***
HOMEBREW_GITHUB_API_TOKEN: ***
PATH: /opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
GITHUB_ACTIONS_HOMEBREW_MACOS_SELF_HOSTED: 1
==> Using Homebrew/homebrew-test-bot 6cc7d8d (Merge pull request #728 from carlocab/whitespace)
==> Using Homebrew/brew 3.3.5-28-g3ab140e39 (Merge pull request #12470 from Homebrew/dependabot/bundler/Library/Homebrew/sorbet-0.5.9368)
==> Testing Homebrew/homebrew-core e1adad68e54 (Merge 6ca43aaa001d6915a504c6f2cd4d5243bc69e7f9 into 679db0e12375cac637eaf6a709cc8639d163b355):
==> Running CleanupAfter#run!
==> git -C /opt/homebrew clean -ff -dx
==> FAILED
Full /usr/bin/git clean -ff -dx --exclude=*.bottle*.* --exclude=Library/Taps --exclude=Library/Homebrew/vendor output
warning: could not open directory 'Cellar/cargo-c/': No such file or directory
fatal: Cannot lstat 'bin/cargo-capi': No such file or directory
==> brew cleanup --prune=3
==> FAILED
Full cleanup --prune=3 output
Error: No such file or directory @ dir_s_rmdir - /Users/brew/Library/Caches/Homebrew/downloads/1108ef5651c0479f5e5dc041b8671d3f1b3aad4e04bc2e13f6ba52dacbe31342--cargo-edit--0.8.0_1.arm64_monterey.bottle.tar.gz.incomplete
Error: 2 failed steps!
git -C /opt/homebrew clean -ff -dx
brew cleanup --prune=3
Error: Process completed with exit code 1.
Run brew test-bot --only-cleanup-after
brew test-bot --only-cleanup-after
rm -rvf bottles
shell: /bin/bash -e {0}
env:
HOMEBREW_DEVELOPER: 1
HOMEBREW_GITHUB_ACTIONS: 1
HOMEBREW_NO_AUTO_UPDATE: 1
HOMEBREW_CHANGE_ARCH_TO_ARM: 1
HOMEBREW_RELOCATE_RPATHS: 1
GITHUB_TOKEN: ***
HOMEBREW_GITHUB_API_TOKEN: ***
PATH: /opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
GITHUB_ACTIONS_HOMEBREW_MACOS_SELF_HOSTED: 1
==> Using Homebrew/homebrew-test-bot 6cc7d8d (Merge pull request #728 from carlocab/whitespace)
==> Using Homebrew/brew 3.3.5-28-g3ab140e39 (Merge pull request #12470 from Homebrew/dependabot/bundler/Library/Homebrew/sorbet-0.5.9368)
==> Testing Homebrew/homebrew-core e1adad68e54 (Merge 6ca43aaa001d6915a504c6f2cd4d5243bc69e7f9 into 679db0e12375cac637eaf6a709cc8639d163b355):
==> Running CleanupAfter#run!
==> git -C /opt/homebrew clean -ff -dx
==> FAILED
Full /usr/bin/git clean -ff -dx --exclude=*.bottle*.* --exclude=Library/Taps --exclude=Library/Homebrew/vendor output
warning: failed to remove Cellar/: Directory not empty
Removing Cellar/rubberband
Removing Cellar/libpsl
Removing Cellar/cjson
...
Removing bin/zstdmt
Removing etc/
Removing include/
Removing lib/
Removing opt/
Removing sbin/
Removing share/
Removing var/
==> brew cleanup --prune=3
Error: 1 failed step!
git -C /opt/homebrew clean -ff -dx
Error: Process completed with exit code 1.
See Homebrew/homebrew-core#16205 : testbot skips build of ffmpeg on Yosemite because one of the optional deps requires El Capitan or later.
I think behavior has changed recently so that unbottled dependents are now built from source instead of being skipped. Is there an option to skip building unbottled dependents to speed up test-bot runs?
When running brew test-bot homebrew-ffmpeg/ffmpeg/ffmpeg, recursively perl's post install step fails (ref homebrew-ffmpeg/homebrew-ffmpeg#49), and so tests are failing on the 3rdf party ffmpeg test-bot run on ubuntu (yaml).
Also beforehand installing perl from source (successfully, homebrew-ffmpeg/homebrew-ffmpeg@77e9d7f) won't hinder brew test-bot to incorporate the failing perl bottle.
As I'm not sure if this a bottling issue (ref https://github.com/Homebrew/linuxbrew-core/issues/4808) that might be re-opened, or a caveat of test-bot, I'm not sure where to raise this issue.
Curious about your advise/ideas @dawidd6 :)
Since approx. 4 weeks the build check in GitHub does not work anymore. I see always the error:
Error: Invalid usage: Did not find any formulae or commits to test!
Error: Process completed with exit code 1.
Here is the failed build: https://github.com/kaoh/homebrew-globalplatform/runs/1168873594?check_suite_focus=true
I have not changed the GitHub workflow, so it must be an incompatible upstream change.
name: brew test-bot
on:
push:
branches: master
pull_request: []
jobs:
test-bot:
runs-on: macos-latest
steps:
- name: Set up Git repository
uses: actions/checkout@v2
- name: Run brew test-bot
run: |
set -e
brew update
HOMEBREW_TAP_DIR="/usr/local/Homebrew/Library/Taps/kaoh/homebrew-globalplatform"
mkdir -p "$HOMEBREW_TAP_DIR"
rm -rf "$HOMEBREW_TAP_DIR"
ln -s "$PWD" "$HOMEBREW_TAP_DIR"
brew test-bot
https://github.com/kaoh/homebrew-globalplatform/blob/master/.github/workflows/main.yml
homebrew-test-bot has code to perform prune, gc, clean etc. on the Git repository, called from here:
Lines 836 to 838 in af6fcf3
This code however does not run when --ci-upload is used.
It perhaps should be since the Homebrew Bottle job is asking for it:
==> Fetching /var/lib/jenkins/workspace/Homebrew Bottles/Library/Taps/homebrew/homebrew-core...
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.
warning: There are too many unreachable loose objects; run 'git prune' to remove them.
I'm trying to get test-bot up and running for my own tap and it seems to work when GITHUB_ACTIONS environment variable is not set. When it is set, test-bot is automatically appending some options and it looks like they break the formula detection.
Logs from workflow run when GITHUB_ACTIONS is not set:
2019-09-26T00:27:10.6753530Z ==> Tapping homebrew/test-bot
2019-09-26T00:27:10.6955133Z Cloning into '/home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/homebrew/homebrew-test-bot'...
2019-09-26T00:27:11.3288993Z Tapped 1 command and 1 formula (50 files, 260.6KB).
2019-09-26T00:27:20.0412462Z Homebrew/homebrew-test-bot fb05b68 (Merge pull request #279 from dawidd6/tap)
2019-09-26T00:27:20.0414495Z ARGV: --verbose --root-url=https://dl.bintray.com/dawidd6/bottles-tap
2019-09-26T00:27:20.0420232Z brew tap dawidd6/tap --full
2019-09-26T00:27:20.8428157Z ==> Tapping dawidd6/tap
2019-09-26T00:27:20.8632130Z Cloning into '/home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/dawidd6/homebrew-tap'...
2019-09-26T00:27:21.2177880Z Tapped 7 formulae (34 files, 214.6KB).
2019-09-26T00:27:21.6552743Z
2019-09-26T00:27:21.6554703Z ==> brew pull --clean https://github.com/dawidd6/homebrew-tap/pull/31/checks
2019-09-26T00:27:22.7760544Z ==> Fetching patch
2019-09-26T00:27:22.7761538Z Patch: https://github.com/dawidd6/homebrew-tap/pull/31.patch
2019-09-26T00:27:24.8099061Z ==> Applying patch
2019-09-26T00:27:24.8364312Z Applying: neofetch: test
2019-09-26T00:27:24.9047579Z Warning: dawidd6/tap/neofetch has a bottle: do you need to update it with --bottle?
2019-09-26T00:27:24.9252666Z ==> Patch changed:
2019-09-26T00:27:24.9466862Z Formula/neofetch.rb | 1 +
2019-09-26T00:27:24.9467143Z 1 file changed, 1 insertion(+)
2019-09-26T00:27:25.0428833Z Homebrew/brew 2.1.11-123-ge6e569fd1 (Merge pull request #6489 from dawidd6/create-rust)
2019-09-26T00:27:25.6766291Z Homebrew/linuxbrew-core 9a3e849 (llvm: Fix build for Linuxbrew (#15764))
2019-09-26T00:27:25.6767617Z
2019-09-26T00:27:25.6768341Z Testing tap dawidd6/tap with:
2019-09-26T00:27:25.6768676Z origin/master 1596045 (workflow: diverge build jobs steps)
2019-09-26T00:27:25.6769026Z HEAD 8b12975 (neofetch: test)
2019-09-26T00:27:25.6769649Z diff_start_sha1 1596045
2019-09-26T00:27:25.6769930Z diff_end_sha1 8b12975
2019-09-26T00:27:25.6770148Z
2019-09-26T00:27:25.6770401Z ==> brew tap linuxbrew/xorg
2019-09-26T00:27:26.0041580Z ==> Tapping linuxbrew/xorg
2019-09-26T00:27:26.0256999Z Cloning into '/home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/linuxbrew/homebrew-xorg'...
2019-09-26T00:27:26.7214376Z Tapped 155 formulae (196 files, 1.4MB).
2019-09-26T00:27:27.0742712Z
2019-09-26T00:27:27.0744485Z Formula changes to be tested:
2019-09-26T00:27:27.0745675Z added formulae (empty)
2019-09-26T00:27:27.0745875Z modified formulae dawidd6/tap/neofetch
2019-09-26T00:27:27.0746025Z deleted formulae (empty)
One can see that test-bot successfully detected modified formula.
Logs from workflow run when GITHUB_ACTIONS is set:
2019-09-26T00:26:44.4664380Z ==> Tapping homebrew/test-bot
2019-09-26T00:26:44.4933090Z Cloning into '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot'...
2019-09-26T00:26:44.8894160Z Tapped 1 command and 1 formula (50 files, 117.6KB).
2019-09-26T00:26:46.1650960Z Homebrew/homebrew-test-bot fb05b68 (Merge pull request #279 from dawidd6/tap)
2019-09-26T00:26:46.1651900Z ARGV: --root-url=https://dl.bintray.com/dawidd6/bottles-tap --verbose --ci-auto --no-pull --ci-pr --cleanup --test-default-formula
2019-09-26T00:26:46.5129270Z
2019-09-26T00:26:46.5130690Z ==> brew untap caskroom/versions
2019-09-26T00:26:47.4368830Z Untapping caskroom/versions...
2019-09-26T00:26:47.4369260Z Untapped 161 casks (208 files, 315.5KB).
2019-09-26T00:26:47.4446680Z
2019-09-26T00:26:47.4449750Z ==> brew untap homebrew/cask
2019-09-26T00:26:49.1238310Z Untapping homebrew/cask...
2019-09-26T00:26:49.1238890Z Untapped 1 command and 3532 casks (3,647 files, 3.9MB).
2019-09-26T00:28:55.2793280Z
2019-09-26T00:28:55.2799380Z ==> git -C Homebrew checkout master -f
2019-09-26T00:28:55.4259830Z Switched to branch 'master'
2019-09-26T00:28:55.4270640Z Your branch is up to date with 'origin/master'.
2019-09-26T00:28:56.3861280Z Homebrew/brew 2.1.11-77-gd40a4173d (Merge pull request #6453 from reitermarkus/brew-ruby)
2019-09-26T00:28:56.3861600Z
2019-09-26T00:28:56.3866720Z ==> git -C homebrew/homebrew-core fetch --depth=1 origin
2019-09-26T00:28:58.1709860Z From https://github.com/Homebrew/homebrew-core
2019-09-26T00:28:58.1710730Z + 91de426...692c405 master -> origin/master (forced update)
2019-09-26T00:28:58.1805230Z
2019-09-26T00:28:58.1806600Z ==> git -C homebrew/homebrew-core reset --hard origin/master
2019-09-26T00:28:58.3597820Z HEAD is now at 692c405 rabbitmq 3.7.18
2019-09-26T00:28:58.4103310Z Homebrew/homebrew-core 692c405 (rabbitmq 3.7.18)
2019-09-26T00:28:58.5086510Z
2019-09-26T00:28:58.5087080Z Testing tap dawidd6/tap with:
2019-09-26T00:28:58.5087250Z origin/master 1596045 (workflow: diverge build jobs steps)
2019-09-26T00:28:58.5087400Z HEAD 83cf3be (neofetch: test)
2019-09-26T00:28:58.5087540Z diff_start_sha1 (undefined)
2019-09-26T00:28:58.5089060Z diff_end_sha1 82caf247338317b8fa660665a54dc4f2356aed22
2019-09-26T00:28:58.7057350Z
2019-09-26T00:28:58.7057920Z Formula changes to be tested:
2019-09-26T00:28:58.7058070Z added formulae (empty)
2019-09-26T00:28:58.7058240Z modified formulae (empty)
2019-09-26T00:28:58.7058380Z deleted formulae (empty)
In the job above, before executing test-bot I had to git pull the pull request head branch, cause without that HEAD == origin/master.
I already spent a great time on this issue. I'm still learning Ruby and sometimes can't understand what is going on in the code.
Can only guess that --no-pull option is messing something up.
Took a look at homebrew-core jenkins and linuxbrew-core azure but still can't figure it out.
OT: fun fact, GITHUB_ACTIONS environment variable is not set in containers, only in VMs the job is run on.
When anything using the squashfuse formula is tested, two of its symlinks (libfuseprivate.0.dylib and libfuseprivate.dylib) are left in /usr/local/lib, although libfuseprivate.a does get removed.
1014-Mojave:~ serveradmin$ brew test-bot --ci-pr squashfs
Homebrew/homebrew-test-bot af6fcf3 (Merge pull request #300 from MikeMcQuaid/delete_more_code)
ARGV: --ci-pr squashfs --cleanup --test-default-formula
==> git -C homebrew/homebrew-core reset --hard origin/master
==> brew untap homebrew/cask
Error: No known CI provider detected! If you are using GitHub Actions, Jenkins,
Azure Pipelines, Travis CI or Circle CI then we cannot find the expected
environment variables! Check you have e.g. exported them to a Docker container.
Homebrew/brew 2.2.0 (Merge pull request #6783 from MikeMcQuaid/update-test-pr)
==> git -C homebrew/homebrew-core fetch --depth=1 origin
==> git -C homebrew/homebrew-core reset --hard origin/master
Homebrew/homebrew-core 6807fd8 (teleport: update 4.1.5 bottle.)
Testing with:
origin/master (undefined)
HEAD (undefined)
diff_start_sha1 6807fd8
diff_end_sha1 6807fd8
==> brew doctor
==> brew --env
==> brew config
==> brew readall --aliases
==> brew fetch --retry cmake gdbm lz4 lzo [email protected] pkg-config python readline sphinx-doc sqlite xz zstd
==> brew fetch --retry squashfs --build-bottle --force
==> brew install --only-dependencies --verbose --build-bottle squashfs
==> brew install --verbose --build-bottle squashfs
==> brew audit squashfs --online
==> brew style squashfs
==> brew bottle --verbose --json squashfs
==> brew bottle --merge --write --no-commit ./squashfs--4.4.mojave.bottle.json
==> brew uninstall --force squashfs
==> brew uninstall --force cmake gdbm [email protected] pkg-config python readline sphinx-doc sqlite
==> brew install --only-dependencies ./squashfs--4.4.mojave.bottle.1.tar.gz
==> brew install ./squashfs--4.4.mojave.bottle.1.tar.gz
==> brew linkage --test squashfs
==> brew install --only-dependencies --include-test squashfs
==> brew test squashfs --verbose
==> brew fetch --retry snapcraft
==> brew install --only-dependencies snapcraft
==> brew install snapcraft
==> brew install --only-dependencies snapcraft
==> brew linkage --test snapcraft
==> brew install --only-dependencies --include-test snapcraft
==> brew test --verbose snapcraft
==> brew fetch --retry squashfuse
==> brew install --only-dependencies squashfuse
==> brew install squashfuse
==> brew install --only-dependencies squashfuse
==> brew linkage --test squashfuse
==> brew install --only-dependencies --include-test squashfuse
==> brew test --verbose squashfuse
==> brew uninstall --force lz4 lzo xz zstd
==> git -C homebrew/homebrew-core reset --hard origin/master
==> brew cleanup --prune=3
1014-Mojave:~ serveradmin$ brew list
1014-Mojave:~ serveradmin$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!
Warning: Broken symlinks were found. Remove them with `brew cleanup`:
/usr/local/lib/libfuseprivate.0.dylib
/usr/local/lib/libfuseprivate.dylib
Past occurrences:
Magic comments have no effect for commands in a tap, so
brew test-bot --help
now runs actual bot actions right away.
https://jenkins.brew.sh/job/Homebrew%20Testing/5/
HOMEBREW_VERSION: 1.2.0-39-g790a11337
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: 790a113371ec6cdab38a29c18c835726054e4f78
Last commit: 12 minutes ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core/
Core tap HEAD: 42eecd82cd3c13064daeaae2a54c485b3ab5019e
Core tap last commit: 12 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_REPOSITORY: /usr/local/Homebrew
HOMEBREW_CELLAR: /usr/local/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://homebrew.bintray.com
CPU: quad-core 64-bit nehalem
Homebrew Ruby: 2.0.0-p648
Clang: 8.1 build 802
Git: 2.11.0 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Perl: /usr/bin/perl
Python: /usr/bin/python
Ruby: /usr/bin/ruby => /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby
Java: 1.8.0_121
macOS: 10.12.4-x86_64
Xcode: 8.3.2
CLT: 8.3.2.0.1.1492020469
X11: 2.7.11 => /opt/X11
BOT_PARAMS:
[email protected] [email protected] [email protected] percona-server
Only the percona-server bottle is preserved and passed along to the downstream job:
https://jenkins.brew.sh/job/Homebrew%20Bottles/977/console
For example, when uploading bottles is not desired, or when I want to upload to a repository that is not bintray (e.g. to AWS S3).
I see that I can specify root-url for how to set it in the recipe, but I don't see:
--help - which does not provide high-level docs, just descriptions of each argApologies if I've missed something.
Currently test-bot is set up to only work on formulae, yet seems designed (as output by tap-new) to be for Taps in general… So surely it should also check Casks?
Using Github Actions, if one runs brew test-bot --ci-upload, the tap is automatically resetted to master. Which is undesirable in situations when one checkouts the repo (tap) manually to, let's say, another branch, before uploading the bottles. This is of course not a problem for taps, that rely on test-bot for pushing tags, but it is broken for the ones that push branches for example.
Interesting lines of code:
homebrew-test-bot/lib/test_bot.rb
Lines 161 to 178 in af6fcf3
What I'm talking about is demonstrated here: https://github.com/dawidd6/homebrew-tap/pull/71/checks?check_run_id=329233117#step:8:1
I didn't encounter this issue before, cause I didn't try to add a new formula and build a bottle for it via PR.
I noticed recently that using --skip-recursive-dependents flag on Linux CI (to reduce build time) does not work anymore.
Right now the full list of dependants is being tested. I checked locally and the --recursive flags is removed from the brew uses call. But nevertheless, the full list of dependants is being tested. So something else is going on.
Running brew test-bot results in the following error:
==> Purging...
Error: undefined local variable or method `branch' for #<Homebrew::Tests::CleanupBefore:0x00007fa08d116298>
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot/lib/test_cleanup.rb:125:in `checkout_branch_if_needed'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot/lib/test_cleanup.rb:81:in `cleanup_shared'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot/lib/tests/cleanup_before.rb:38:in `run!'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot/lib/test_runner.rb:115:in `run_tests'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot/lib/test_runner.rb:36:in `block in run!'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot/lib/test_runner.rb:25:in `each'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot/lib/test_runner.rb:25:in `run!'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot/lib/test_bot.rb:125:in `run!'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot/cmd/test-bot.rb:75:in `test_bot'
/usr/local/Homebrew/Library/Homebrew/brew.rb:112:in `<main>'
Looking at
homebrew-test-bot/lib/test_cleanup.rb
Line 125 in c53fa90
branch is not defined anywhere?
A recent change (3c78e44#diff-f8070d5310a565d61333c6a5a1eb5024) seems to have removed the default definition of branch (branch = master) of the method checkout_branch_if_needed(), so that's likely the root cause?
Hello 👋
Some core formula like sqlite (Homebrew/homebrew-core#74485) hit a recurring issue where builds take forever. Here's one that's been going for over 9 hours. It seems like it would be trivial to parallelize the brew test-bot command, especially as we are using Github Actions, and reduce the time it takes to check these changes in. For context, SQLite 3.35.3 has a nasty bug that is fixed in 3.35.4 and it would be nice to be able to release this without running into CI timeouts.
Github Actions allows you to create dynamic matrices based on prior steps. For example:
jobs:
whatever:
steps:
run: echo "::set-output name=matrix::$(some-command)"
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
run_something:
needs: whatever
strategy:
matrix:
something: ${{ fromJson(needs.whatever.outputs.matrix) }}
steps:
....In the above example we have a step that executes some-command, and uses the JSON output from this command to feed into the run_something matrix.
I propose that we adapt the brew test-bot command to support a dynamic matrix. We can do this quite simply: We add a --matrix flag of some kind that outputs a JSON array of shards. If sqlite3 tests needs to build 1,000 packages, and we specify that each job should handle 100 packages then this would be a simple array of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]. If we're building a package that needs to build only 200 packages to test, then it might just be [1, 2]. As this only really makes a difference on some core packages we don't want to always launch a matrix of 10 jobs if each one is only going to run 3 or 4 tests. So there's some smartness that could be added to account for this which is different from just idx % total.
The downstream test would depend on this output and would use the value to shard tests. So, given the above 1,000 packages that need to be tested across 10 different runners then we'd run brew test-bot ${{ matrix.shard }} ${{ matrix.total_shards }}, which would run 10% of the packages it needs to.
In theory this would result in a lot faster builds for packages that are heavily depended on. We could also run the brew test-bot --matrix command on a Ubuntu runner to save resources.
I'm not that familiar with Homebrew or the intricacies of how it's tested, so I hope this makes sense!
This build is green: https://github.com/Homebrew/linuxbrew-core/pull/14737
But we had a bunch of failures in the logs: https://dev.azure.com/Homebrew/56a87eb4-3180-495a-9117-5ed6c79da737/_apis/build/builds/17317/logs/16
It looks like circle.ci and jenkins where looking for the "FAILED" string in uppercase (to be confirmed?). Azure does not do that.
@sjackman @MikeMcQuaid maybe you have a quick idea on what is going on here? If not I'll dig through the code. It looks like we already have something in place in the run function, with a failure check: https://github.com/Homebrew/homebrew-test-bot/blob/master/cmd/brew-test-bot.rb#L1345-L1362. But maybe the exit code is not correctly propagated ...
Hi there!
Following on my quest to automatize the process of building and publishing bottles for taps, I've spent quite some time studying Jenkins' console outputs and the brew-test-bot.rb source code.
One thing I can't get my head around is this:
How does Jenkins know which formula to bottle?
If we take for example Homebrew/homebrew-core#39211, the corresponding logs are:
In the High Sierra log, one can find this line...
brew pull --clean https://github.com/Homebrew/homebrew-core/pull/39211
...where the pull url comes from the ghprbPullLink variable.
And then it magically seems to know which formula to test and bottle (qbs)!
brew uses --recursive qbs
How could I reproduce the same behavior in Azure Pipelines?
For the moment I'm passing the formula name as a parameter to brew test-bot but what if I have multiple formulae?
Thanks a lot for your help!
Best,
-- Ladislas
With a GH Actions template taken/adapted from the default one coming from brew tap-new, several errors occur when running on ubuntu-latest.
Maybe it's a mix up (set -e is part of the macos script), where the ubuntu setup originated elsewhere.
Maybe I'm missing something obvious, or missed the right documentation. Do you have a correct workflow example of testing a 3rd party tap on both macos and ubuntu?
set -e
brew update
HOMEBREW_TAP_DIR="/usr/local/Homebrew/Library/Taps/homebrew-ffmpeg/homebrew-ffmpeg"
mkdir -p "$HOMEBREW_TAP_DIR"
rm -rf "$HOMEBREW_TAP_DIR"
ln -s "$PWD" "$HOMEBREW_TAP_DIR"
brew test-bot
shell: /bin/bash -e {0}
env:
HOMEBREW_COLOR: 1
/usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:10: warning: constant Gem::ConfigMap is deprecated
/usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:10: warning: constant Gem::ConfigMap is deprecated
/usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:29: warning: constant Gem::ConfigMap is deprecated
/usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:30: warning: constant Gem::ConfigMap is deprecated
/usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:10: warning: constant Gem::ConfigMap is deprecated
/usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:10: warning: constant Gem::ConfigMap is deprecated
/usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:29: warning: constant Gem::ConfigMap is deprecated
/usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:30: warning: constant Gem::ConfigMap is deprecated
error: Not a valid ref: refs/remotes/origin/master
fatal: ambiguous argument 'refs/remotes/origin/master': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
==> Homebrew is run entirely by unpaid volunteers. Please consider donating:
https://github.com/Homebrew/brew#donations
Updated 1 tap (ddelange/ffmpeg).
No changes to formulae.
mkdir: cannot create directory ‘/usr/local/Homebrew’: Permission denied
##[error]Process completed with exit code 1.
clazy passed CI in Homebrew/homebrew-core#77005, but then failed the linkage check in this CI run (Homebrew/homebrew-core#77434).
It was then given a revision bump to rebuild it, and I thought that shook off the linkage (as CI passed after the revision bump), but the new bottle still shows broken linkage:
❯ brew linkage clazy
System libraries:
/usr/lib/libSystem.B.dylib
/usr/lib/libncurses.5.4.dylib
/usr/lib/libxml2.2.dylib
/usr/lib/libz.1.dylib
Variable-referenced libraries:
@rpath/ClazyPlugin.dylib
Broken dependencies:
/usr/local/opt/llvm@11/lib/libc++.1.dylib (llvm@11)
Dependencies with no linkage:
qt
This seems to be the step that causes it: https://github.com/Homebrew/homebrew-core/runs/2597888223#step:7:768
The first bottle builds that we have tried since environment filtering was enabled in #129 have been failing on all our build machines with the following error:
==> brew pull --clean https://github.com/osrf/homebrew-simulation/pull/312
==> FAILED
==> Fetching patch
Patch: https://github.com/osrf/homebrew-simulation/pull/312.patch
==> Applying patch
*** Please tell me who you are.
Run
git config --global user.email "[email protected]"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
fatal: unable to auto-detect email address (got 'jenkins@t1000.(none)')
*** Please tell me who you are.
Run
git config --global user.email "[email protected]"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
fatal: unable to auto-detect email address (got 'jenkins@t1000.(none)')
Error: Patch failed to apply: aborted.
I traced this error to the invocation of git am in brew pull and then discovered that git am wants a valid committer identity (including email). The build machines have a ~/.gitconfig with name and email, and I also set the GIT_COMMITTER_EMAIL before invoking test-bot, but environment filtering wipes those out. Note that a direct way to test the committer identity is with git var GIT_COMMITTER_IDENT.
This is where it gets weird. When an email isn't provided, git tries to construct one from the username and hostname, but it declares the hostname invalid if it doesn't contain any . characters. That seems weird to me since many of our machines don't have . characters in their hostnames. I was able to workaround the issue by appending .local to the hostnames of these machines, but it seems like that shouldn't be necessary.
I'm guessing that git am wants the committer identity in case --signoff is passed, but I don't think it needs to check it in every case. This could be a git bug, but it exists in all the released versions of git, so it would be nice to have a workaround that doesn't require changing build machine hostnames.
Take this PR: Homebrew/homebrew-core#36334
On all three CI nodes, the testbot is failing because it is doing the following:
libraw depslibrawlibraw (which I think is wrong)efl dependencies, which include libraw. This fails, because libraw bottles are not published.because
04:43:42 ==> Downloading https://homebrew.bintray.com/bottles/libraw-0.19.2.mojave.bottle.tar.gz
04:43:42 curl: (22) The requested URL returned error: 401 Unauthorized
04:43:42 ==> Retrying download
04:43:42 ==> Downloading https://homebrew.bintray.com/bottles/libraw-0.19.2.mojave.bottle.tar.gz
04:43:42 curl: (22) The requested URL returned error: 401 Unauthorized
04:43:42 Error: Failed to download resource "libraw"
04:43:42 Download failed: https://homebrew.bintray.com/bottles/libraw-0.19.2.mojave.bottle.tar.gz
04:44:03
I have no idea what causes this. The only change I'm seeing in recent testbot history is ENV["CI"] = "1" if jenkins.
As a user I created a issue, and got the label awaiting user reply after a response
I don't have permission to remove the label nor an automated command to remove it.
BrewTestBot or it's github repo doesn't hold any github action / README related to an action like soWe need a better sense of which Homebrew/core tests are flaky, and the best way to do that is to use BuildPulse, which we already do in Homebrew/brew.
We already have some code in place that enables this. All that remains to be done is to upload this to BuildPulse. See discussion at #699.
I hope to be able to work on this at some point, but I'd also love it if someone else picked it up before I did.
For some reason I haven't yet worked out, neither annotations or the ignoring of build/linkage/test failures for unbottled formulae isn't working on Linux.
As best I can tell, the Linux runner seems to exit with an error immediately upon encountering one around here:
Lines 104 to 107 in 96edbe6
The ignore logic and annotations come after that, so this would explain why neither is working. I don't know why this is happening specifically on Linux, though.
For examples of this behaviour: Homebrew/homebrew-core#89550 (annotations and ignore), Homebrew/homebrew-core#89042 (annotations and ignore), Homebrew/homebrew-core#89445 (annotations only)
We don't have this problem on macOS. See Homebrew/homebrew-core#89518.
I have a fomula like this:
if MacOS.version <= :el_capitan
url "https://github.com/MaskRay/ccls.git",
:tag => "0.20180812",
:revision => "06aa25233567ae16a58a6f5237aacfd9e02aa29d"
else
url "https://github.com/MaskRay/ccls.git",
:tag => "0.20180913",
:revision => "56c6ec43dfd87ed2e44418b8d172307d87327a64"
endAnd then build each version with independent image:
matrix:
include:
- env: OSX=10.11
compiler: clang
osx_image: xcode8
- env: OSX=10.12
compiler: clang
osx_image: xcode9.2So far, everything is going well, all build jobs passed:
But after increasing the reversion from 0.20180913 to 0.20180924 for osx 10.12:
if MacOS.version <= :el_capitan
url "https://github.com/MaskRay/ccls.git",
:tag => "0.20180812",
:revision => "06aa25233567ae16a58a6f5237aacfd9e02aa29d"
else
url "https://github.com/MaskRay/ccls.git",
- :tag => "0.20180913",
+ :tag => "0.20180924",
- :revision => "56c6ec43dfd87ed2e44418b8d172307d87327a64"
+ :revision => "41fcc0272c8c93fcbe52f24ef587e633a9dcc7e1"
endI've got failure from job(OSX=10.11):
$ export OSX=10.11
...
$ if [ "$OSX" == "10.12" ]; then brew test-bot --skip-setup; else brew test-bot; fi
...
==> brew audit twlz0ne/ccls/ccls --online
...
Error: 1 problem in 1 formula detected
twlz0ne/ccls/ccls:
* stable version should not decrease (from 0.20180913 to 0.20180812)
==> FAILED
I didn't change the reversion for osx 10.11, why prompting these? Seems the reversions are missed up.
And at the same time, the job(OSX=10.12) passed.
This follows from Homebrew/homebrew-core#69708.
Currently, test-bot does not attempt to build or test formulae using (I think) the following logic:
homebrew-test-bot/lib/tests/formulae.rb
Lines 578 to 585 in 86c53a3
This requires adding the CI-force-arm label so that test-bot will attempt an ARM build for new formulae. We shouldn't need to do this, and it would be good if test-bot recognises that a formula is new and attempt a build even if the CI-force-arm label is not set.
I'm still mulling over the right approach, so suggestions here would be appreciated.
While investigating CI failures in Homebrew/homebrew-core#67615 (applying an upstream patch to boost), I noticed that brew test pcl failed for all architectures, even though it had passed very recently in Homebrew/homebrew-core#66792. The tests in Homebrew/homebrew-core#67615 failed to find eigen when running cmake to configure some sample code that links against pcl. Searching earlier in the console log, I noticed brew unlink eigen just before freeling was tested. Looking at the console logs from Homebrew/homebrew-core#66792, I noticed the same call to brew unlink eigen just before freeling, but there was also a call to brew link eigen just before pcl was handled. The difference between those two pull requests is that pcl received a revision bump in the earlier one, and there is logic in setup_formulae_deps_instances() in lib/tests/formulae.rb that calls brew link on unlinked dependencies for any formulae to be built. I believe this logic should be added to link dependencies for any dependents that are installed, not just the explicitly named formulae to be tested.
I'm really struggling to work out why brew test-bot is resetting the changes I want to test in a formula. (Note: this works locally but not on Github's actions, which is frustrating.).
brew update-reset
brew tap purplebooth/repo
cp formula/formula.rb "$(brew --repo)/Library/Taps/purplebooth/homebrew-repo/Formula/$REPOSITORY_NAME.rb"
brew install "purplebooth/repo/$REPOSITORY_NAME"
brew test-bot \
--tap=purplebooth/repo \
"purplebooth/repo/$REPOSITORY_NAME"
brew test-bot \
--ci-upload \
--bintray-org="$BINTRAY_ORG" \
--root-url="$BINTRAY_DL_URL" \
--tap=purplebooth/repo \
--git-name="$GITHUB_ACTOR" \
--git-email="[email protected]" \
--publish \
"purplebooth/repo/$REPOSITORY_NAME"
This ticket isn't about this individual problem, but rather the lack of a resource that tells me the expected workflow for the tool. I assume I can get to the bottom of this, but it'd be really nice if the help page gave some indication of the workflow you expect with the tool.
To clarify:
It's clear from the code that different things happen if environment variables like "UPSTREAM_PULL_REQUEST" are set. So it looks like from the code you're assuming we're not using a CD pipeline, but a pull request flow. How do these environment variables change the behaviour of the tool, depending on if they are or are not set.
I'm seeing an error I cannot understand when running brew test-bot --only-formulae for my taps repository.
This is the error message:
==> brew linkage --test miguelaferreira/tools/devex-cli
==> FAILED
Unwanted system libraries:
/lib/x86_64-linux-gnu/libz.so.1
Here's the workflow full log.
I could use any help figuring out what this error means and how can I fix it.
In Homebrew/homebrew-core#39225, both python and python@2 are updated to use a new version of setuptools. python gets built first and all its dependencies are tested without any failures. Then it is uninstalled, the python@2 bottle is built (successfully), and the dependencies of python@2 are tested. The first formula to fail is alot, which depends directly on python@2, but not directly on python:
alot has multiple direct dependencies that depend indirectly on python, such as gpgme and notmuch. For some reason, test-bot ends up trying to find the new python bottle on bintray instead of installing the recently built and unpublished one from the filesystem.
==> Installing alot dependency: python
==> Downloading https://homebrew.bintray.com/bottles/python-3.7.3.mojave.bottle.1.tar.gz
curl: (22) The requested URL returned error: 404 Not Found
Error: Failed to download resource "python"
Download failed: https://homebrew.bintray.com/bottles/python-3.7.3.mojave.bottle.1.tar.gz
Note that alot was also tested successfully when python's dependencies were tested, but failed later when python@2 dependencies were tested.
I think a workaround is to split up changes into separate pull requests, though this may not always be feasible or desired. I have observed this issue at times while maintaining the osrf/simulation tap, but I haven't reported it until now because I've found it difficult to describe succinctly with core formulae. I've observed it when building multiple bottles in a pull request and some downstream formulae depend on one of the formula directly, and the other indirectly.
I've saved the failed jenkins build log from https://jenkins.brew.sh/job/Homebrew%20Core%20Pull%20Requests/43640/version=mojave/ in the following gist:
The qt formula has a :build requirement on :xcode, but recently (possibly since #14), I am unable to build a bottle for a formula that depends on qt ( osrf/homebrew-simulation#261 ) on a build machine that only has the command-line tools installed. The build fails with the following console message:
==> SKIPPING: osrf/simulation/ignition-gui
A full installation of Xcode.app is required to compile this software.
Installing just the Command Line Tools is not sufficient.
Xcode can be installed from the App Store.
It should be fine to build without full Xcode since qt is being installed from a bottle. I can try to help with some guidance. I'm not sure if the problem is here or in brew.
I have some bottle builds that are failing with an error that I can't figure out (I've removed /usr/local/Homebrew/Library/Taps/homebrew/homebrew-test-bot/ from the console log to improve readability):
+ brew test-bot --tap=osrf/simulation --ci-pr https://github.com/osrf/homebrew-simulation/pull/418
Homebrew/homebrew-test-bot ed0844f (Merge pull request #156 from MikeMcQuaid/test-bot-new-formula-pr-url)
ARGV: --tap=osrf/simulation --ci-pr https://github.com/osrf/homebrew-simulation/pull/418 --cleanup --test-default-formula --local --junit
==> Tapping osrf/simulation
Cloning into '/usr/local/Homebrew/Library/Taps/osrf/homebrew-simulation'...
Tapped 48 formulae (126 files, 745.5KB)
==> git -C Homebrew checkout master -f
==> git -C Homebrew reset --hard origin/master
==> brew pull --clean https://github.com/osrf/homebrew-simulation/pull/418
==> FAILED
cmd/brew-test-bot.rb:301:in `exec': no implicit conversion of MatchData into String (TypeError)
from cmd/brew-test-bot.rb:301:in `block in run'
from cmd/brew-test-bot.rb:296:in `fork'
from cmd/brew-test-bot.rb:296:in `run'
from cmd/brew-test-bot.rb:1185:in `test'
from cmd/brew-test-bot.rb:485:in `download'
from cmd/brew-test-bot.rb:1253:in `run'
from cmd/brew-test-bot.rb:1626:in `block in test_bot'
from cmd/brew-test-bot.rb:1609:in `each'
from cmd/brew-test-bot.rb:1609:in `test_bot'
from cmd/brew-test-bot.rb:1701:in `<top (required)>'
from /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /usr/local/Homebrew/Library/Homebrew/utils.rb:18:in `require?'
from /usr/local/Homebrew/Library/Homebrew/brew.rb:101:in `<main>'
==> brew cleanup --prune=7
Error: undefined method `gsub' for #<MatchData:0x000001019e8390>
cmd/brew-test-bot.rb:488:in `download'
cmd/brew-test-bot.rb:1253:in `run'
cmd/brew-test-bot.rb:1626:in `block in test_bot'
cmd/brew-test-bot.rb:1609:in `each'
cmd/brew-test-bot.rb:1609:in `test_bot'
cmd/brew-test-bot.rb:1701:in `<top (required)>'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/local/Homebrew/Library/Homebrew/utils.rb:18:in `require?'
/usr/local/Homebrew/Library/Homebrew/brew.rb:101:in `<main>'
I can reproduce this interactively on my build machines. brew doctor is clean:
$ brew doctor
Your system is ready to brew.
$ brew config
HOMEBREW_VERSION: 1.6.4-24-g24025a3
ORIGIN: https://github.com/Homebrew/brew
HEAD: 24025a3ab52822bd8395db47a61b73e45940cc28
Last commit: 5 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: e8561625765c1a3100a04ad8c73ada213f8c2f77
Core tap last commit: 5 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_DEV_CMD_RUN: 1
CPU: octa-core 64-bit ivybridge
Homebrew Ruby: 2.3.3 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/bin/ruby
Clang: 8.0 build 800
Git: 2.10.1 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.43.0 => /usr/bin/curl
Java: 1.8.0_131, 1.8.0_77
macOS: 10.11.6-x86_64
CLT: 8.2.0.0.1.1480973914
Xcode: 8.2.1
XQuartz: 2.7.11 => /opt/X11
I can execute brew pull --clean https://github.com/osrf/homebrew-simulation/pull/418 from the command-line but not through test-bot. Our last successful bottle builds were a week ago. Any ideas?
I'm having issue with brew-test-bot, as it fails with the unreferenced in the pull request formulae -- aptible. I don't understand what is going and how to troubleshoot it. If you could point me in the right direction, I would appreciate. Run on the local workstation shows no errors, problem happens only when using Github actions.
Pull request in questions -- sashkab/homebrew-python#93
2020-09-16T13:27:51.6220050Z ##[group]Run brew test-bot --only-tap-syntax
2020-09-16T13:27:51.6220800Z brew test-bot --only-tap-syntax
2020-09-16T13:27:51.6301860Z shell: /bin/bash -e {0}
2020-09-16T13:27:51.6302170Z ##[endgroup]
2020-09-16T13:27:53.1133600Z Using Homebrew/homebrew-test-bot b1d6911 (Merge pull request #501 from scpeters/fix_tap_arg)
2020-09-16T13:27:53.1656030Z Using Homebrew/brew 2.5.1-35-g76363bebc (Merge pull request #8734 from Homebrew/python39)
2020-09-16T13:27:53.1780550Z Using Homebrew/homebrew-core 5be0d5bad1 (octave: update 5.2.0_10 bottle.)
2020-09-16T13:27:53.1903500Z Testing sashkab/homebrew-python d9aef56 (Merge cd5b28c18f38d486d91424e9395b661bf253e38c into 29cf9f29444645e779eabfd0ed14867198da441b):
2020-09-16T13:27:53.1905110Z
2020-09-16T13:27:53.1905890Z Running TapSyntax#run!
2020-09-16T13:27:53.1906810Z brew readall --aliases sashkab/python
2020-09-16T13:27:54.5860210Z brew style sashkab/python
2020-09-16T13:28:00.6077330Z brew audit --tap=sashkab/python
2020-09-16T13:28:02.0926100Z All steps passed!
2020-09-16T13:28:02.1270540Z ##[group]Run brew test-bot --only-formulae
2020-09-16T13:28:02.1271090Z brew test-bot --only-formulae
2020-09-16T13:28:02.1347990Z shell: /bin/bash -e {0}
2020-09-16T13:28:02.1348260Z ##[endgroup]
2020-09-16T13:28:03.6332710Z Using Homebrew/homebrew-test-bot b1d6911 (Merge pull request #501 from scpeters/fix_tap_arg)
2020-09-16T13:28:03.6854920Z Using Homebrew/brew 2.5.1-35-g76363bebc (Merge pull request #8734 from Homebrew/python39)
2020-09-16T13:28:03.6972290Z Using Homebrew/homebrew-core 5be0d5bad1 (octave: update 5.2.0_10 bottle.)
2020-09-16T13:28:03.7095960Z Testing sashkab/homebrew-python d9aef56 (Merge cd5b28c18f38d486d91424e9395b661bf253e38c into 29cf9f29444645e779eabfd0ed14867198da441b):
2020-09-16T13:28:03.7097020Z
2020-09-16T13:28:03.7098040Z Running Formulae#detect_formulae!
2020-09-16T13:28:03.7099870Z git -C /usr/local/Homebrew/Library/Taps/sashkab/homebrew-python fetch origin +refs/heads/master
2020-09-16T13:28:03.9488670Z url https://github.com/sashkab/homebrew-python/pull/93/checks
2020-09-16T13:28:03.9489950Z origin/master 29cf9f2 (Merge pull request #94 from sashkab/update-test-bot)
2020-09-16T13:28:03.9490790Z HEAD d9aef56 (Merge cd5b28c18f38d486d91424e9395b661bf253e38c into 29cf9f29444645e779eabfd0ed14867198da441b)
2020-09-16T13:28:03.9491610Z diff_start_sha1 29cf9f29444645e779eabfd0ed14867198da441b
2020-09-16T13:28:03.9492350Z diff_end_sha1 d9aef569f6af94595b12f02b782af12121115782
2020-09-16T13:28:03.9855090Z Testing Formula changes:
2020-09-16T13:28:03.9855630Z added (empty)
2020-09-16T13:28:03.9855990Z modified sashkab/python/[email protected]
2020-09-16T13:28:03.9856410Z deleted (empty)
2020-09-16T13:28:05.4366470Z
2020-09-16T13:28:05.4369650Z Running Formulae#formula!(sashkab/python/[email protected])
2020-09-16T13:28:05.4925430Z Determining dependencies...
2020-09-16T13:28:08.2492450Z brew fetch --retry gdbm [email protected] pkg-config readline sqlite xz
2020-09-16T13:28:13.2296590Z Determining dependents...
2020-09-16T13:28:41.2465620Z brew uninstall --force gdbm [email protected] pkg-config readline sqlite xz
2020-09-16T13:28:42.2647910Z Error: No available formula with the name "aptible"
2020-09-16T13:28:42.2697970Z ##[error]Process completed with exit code 1.
2020-09-16T13:28:42.2734730Z Cleaning up orphan processes
We've been merging failing git builds for far too long - ideally we wouldn't be merging anything that's red. Here is an example failure from git - please ignore the brew test failure here as that is something specific to that version bump. I'm talking about this at the end:
==> brew uninstall --force gettext pcre2
==> git -C homebrew/homebrew-core checkout master -f
==> FAILED
dyld: Library not loaded: /usr/local/opt/pcre2/lib/libpcre2-8.0.dylib
Referenced from: /usr/local/bin/git
Reason: image not found
dyld: Library not loaded: /usr/local/opt/pcre2/lib/libpcre2-8.0.dylib
Referenced from: /usr/local/bin/git
Reason: image not found
==> git -C homebrew/homebrew-core reset --hard origin/master
==> FAILED
dyld: Library not loaded: /usr/local/opt/pcre2/lib/libpcre2-8.0.dylib
Referenced from: /usr/local/bin/git
Reason: image not found
homebrew-test-bot uses whatever git is in the path (or Xcode.app if there's nothing). At the point of failure above, brew git is installed but non-functional because pcre2 is uninstalled. This uninstall is intentional as if there is subsequent formulae being built then we want a cleaner environment, and to also make better use of disk space. The git formula isn't uninstalled to avoid rebuilds if there was a dependent due to be tested in the same job.
In addition to the situation above, the situation is worsened in the event that a git build is cancelled. This doesn't have to be manual - adding another commit to the pull request or force-pushing will trigger a cancellation. If it is cancelled in the middle of cleanup, the brew git install might not be wiped. This issue resulted in mojave_orka_1 having to be taken offline earlier today as homebrew-test-bot currently cannot reach the point of wiping previous installs without a functional git. It requires manually going in and cleaning it up yourself - something I had hoped to have done by now if I wasn't having issues connecting to the Orka service endpoint.
Here is me brainstorming possible solutions:
/usr/bin/git in homebrew-test-bot.
/usr/local/bin/git.git-filter-repo requires a newer git so its tests may fail if we change the default as it may go through the shim.git binary in HOMEBREW_PREFIX/"bin" or more advanced checks to see whether this is actually a brew-installed git rather than one installed from another location?
git in cleanup_after before the brew git is wiped.
git as a part of its download & install process?I can do some work on this and send a pull request if you want.
If --local is passed, perform only local operations (i.e. don't
push or create PR).
If --local is passed, ask Homebrew to write verbose logs under
./logs/ and set $HOME to ./home/.
Hi there, I’ve been trying to follow the steps outlined in this recent blog post to test and build bottles for some taps I maintain, and I’ve run into a snag.
I’ve managed to adapt the workflows and get those set up, and created a PR which modifies the formulae so that they’re built, but at the point of asking it to publish the bottles and merge it, I’ve run into this error:
==> Uploading cxmon-2020-07-05T515644Z.x86_64_linux.bottle.tar.gz
/usr/bin/curl --disable --globoff --show-error --user-agent Linuxbrew/2.6.0-51-gedb7496\ \(Linux\;\ x86_64\ Ubuntu\ 18.04.5\ LTS\)\ curl/7.58.0 --header Accept-Language:\ en --retry 3 --location https://uploads.github.com/repos/ticky/homebrew-utilities/releases/34729864/assets\?name=cxmon-2020-07-05T515644Z.x86_64_linux.bottle.tar.gz --header Accept:\ application/vnd.github.v3\+json --write-out '
'\%\{http_code\} --header Accept:\ application/vnd.github.antiope-preview\+json --header Authorization:\ token\ ****** --data-binary @cxmon--2020-07-05T515644Z.x86_64_linux.bottle.tar.gz --header Content-Type:\ application/gzip --dump-header /tmp/github_api_headers20201203-3199-16b571v
Error: Validation Failed: [{"resource"=>"ReleaseAsset", "code"=>"custom", "field"=>"size", "message"=>"size is not included in the list"}]
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils/github.rb:289:in `raise_api_error'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils/github.rb:234:in `open_api'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils/github.rb:506:in `upload_release_asset'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/pr-upload.rb:124:in `block (2 levels) in pr_upload'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/pr-upload.rb:120:in `each_value'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/pr-upload.rb:120:in `block in pr_upload'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/pr-upload.rb:104:in `each_value'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/pr-upload.rb:104:in `pr_upload'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:119:in `<main>'
Error: Failure while executing; `/home/linuxbrew/.linuxbrew/bin/brew pr-upload --debug --bintray-org=homebrew` exited with 1.
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils.rb:294:in `safe_system'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/pr-pull.rb:445:in `block (3 levels) in pr_pull'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_2/lib/ruby/2.6.0/fileutils.rb:128:in `chdir'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_2/lib/ruby/2.6.0/fileutils.rb:128:in `cd'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/pr-pull.rb:396:in `block (2 levels) in pr_pull'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_2/lib/ruby/2.6.0/tmpdir.rb:93:in `mktmpdir'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/pr-pull.rb:395:in `block in pr_pull'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/pr-pull.rb:381:in `each'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/dev-cmd/pr-pull.rb:381:in `pr_pull'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:119:in `<main>'
Now, this seems to be coming from the GitHub API, and from searching for that error, indicate GitHub is rejecting the asset as too big. However, all of the bottles built in this PR add up to 20 megabytes per platform, so I’m not sure how it could be over the line there.
Unfortunately, while this build has uploaded several bottles, due to this failure it has not committed any of them into the formula files.
I’m hoping you can help with this tooling, as the idea of first-class support for this is really exciting!
I have been trying to build bottles for my homebrew tap. I am using the github actions created by the brew tap-new command. There seems to be an issue with the default workflows that get created using brew tap-new.
When I run, brew tap -h, I see that --full is a valid flag to pass to that command, but when trying to use it brew tap aws/tap --full, I get the error
Calling `brew tap --full` is disabled! There is no replacement.
In doing some research, it appears the brew tap USER/TAP already does a full clone by default, and only does a shallow clone if you pass in the --shallow flag.
This issue is resulting in my pr builds to fail because under the hood brew test-bot calls brew tap USER/TAP --full.
homebrew-test-bot/lib/test_bot.rb
Line 79 in e86c0e6
I think this issue can be solved just by removing --full from the above line, but I don't have enough homebrew knowledge to be sure of that.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.