Comments (18)
Bo98
commented on June 23, 2024
2
Yes, for tabs we don't store this stuff in the bottles - we store them in GitHub Packages manifest annotations instead.
from brew.
MikeMcQuaid
commented on June 23, 2024
1
Except that the download location (and checksum) is not for the version indicated (20.13.1). Instead, it points to the location and checksum of 20.13.0. Which kinda makes sense, because you can't really write a file containing the checksum of the bottle inside the bottle. (Or I could just be very confused about what's going on here, which is also a possibility.)
This also should be removed at bottling time and restored at install time.
from brew.
carlocab
commented on June 23, 2024
CC @SMillerDev
from brew.
SMillerDev
commented on June 23, 2024
I'll have to find where this is ignored for tabs since we can't really make sure it's the same. I can probably fix the build date though
from brew.
carlocab
commented on June 23, 2024
I'll have to find where this is ignored for tabs since we can't really make sure it's the same.
Yup, they will indeed be different. It isn't ignored for tabs -- they're just not stored in the bottle, so they don't affect the bottle checksum.
from brew.
MikeMcQuaid
commented on June 23, 2024
Yes, for tabs we don't store this stuff in the bottles - we store them in GitHub Packages manifest annotations instead.
We should do the same thing for SBOMs dates/times as we do for Tab runtime dependencies: update them after installation (based on the dates/times from the tab):
brew/Library/Homebrew/formula_installer.rb
Lines 825 to 830 in 1e4d119
from brew.
SMillerDev
commented on June 23, 2024
Not sure how to resolve this. We could not write the field if the compiler is the system one maybe? Or, which affects the usefulness iyam, we could drop the bottle inclusion of the file and only write it on install.
from brew.
MikeMcQuaid
commented on June 23, 2024
I think in an ideal world we'd detect if the compiler was actually used somehow e.g. write a temporary file on first usage of one of the compiler shims.
In cases like this, it's pretty clear that the compiler isn't actually used or a dependency.
from brew.
Bo98
commented on June 23, 2024
If compiler information needs to be available in the bottle archive via brew fetch (though this archive isn't necessarily representative of a complete install as it's pre-relocation): avoiding system compiler makes sense
If compiler information only needs to be available in the Cellar after brew install: SBOM already fetches the compiler from the tab, so we can exclude it and attach it back again on install.
from brew.
MikeMcQuaid
commented on June 23, 2024
If compiler information only needs to be available in the Cellar after
brew install: SBOM already fetches the compiler from the tab, so we can exclude it and attach it back again on install.
Yes, this seems best for now.
from brew.
carlocab
commented on June 23, 2024
If compiler information only needs to be available in the Cellar after
brew install: SBOM already fetches the compiler from the tab, so we can exclude it and attach it back again on install.Yes, this seems best for now.
This is fine, but it might not be enough. The sbom.spdx.json files also reference bottle checksum of dependencies, which in general be different across OS versions even for existing :all bottles. Unless this information is generated during brew fetch?
from brew.
carlocab
commented on June 23, 2024
Confusingly, the SBOM also seems to contain this snippet:
{
"SPDXID": "SPDXRef-Bottle-node@20",
"name": "node@20",
"versionInfo": "20.13.1",
"filesAnalyzed": false,
"licenseDeclared": "NOASSERTION",
"builtDate": "2024-05-09 05:20:38 -0400",
"licenseConcluded": "MIT",
"downloadLocation": "https://ghcr.io/v2/homebrew/core/node/20/blobs/sha256:a865d16c32d50cdffe26e341fb6a8d52b7c3f95daf10e2a390fb988c4fba0ab3",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceLocator": "pkg:brew/homebrew/core/node@[email protected]",
"referenceType": "purl"
}
],
"checksums": [
{
"algorithm": "SHA256",
"checksumValue": "a865d16c32d50cdffe26e341fb6a8d52b7c3f95daf10e2a390fb988c4fba0ab3"
}
]
}Except that the download location (and checksum) is not for the version indicated (20.13.1). Instead, it points to the location and checksum of 20.13.0. Which kinda makes sense, because you can't really write a file containing the checksum of the bottle inside the bottle. (Or I could just be very confused about what's going on here, which is also a possibility.)
from brew.
MikeMcQuaid
commented on June 23, 2024
Confirmed rebottling in Homebrew/homebrew-core#171540 post #17284 fixes the bottles 🎉
from brew.
carlocab
commented on June 23, 2024
Thanks @MikeMcQuaid ❤️
from brew.
carlocab
commented on June 23, 2024
This is still happening. See Homebrew/homebrew-core@fd1c80d.
Diffoscope Output
--- ~/Library/Caches/Homebrew/downloads/10141ca57ddfafa18b2a0efeaac4f8206739805a36d435c886951987fb4ccaa8--ruby-build--20240517.arm64_ventura.bottle.tar.gz
+++ ~/Library/Caches/Homebrew/downloads/aa750a673b84ed5679172e6c6b4a944dce68ee6f7dc6e1b310c5d63c4df83e0a--ruby-build--20240517.arm64_sonoma.bottle.tar.gz
│ --- 10141ca57ddfafa18b2a0efeaac4f8206739805a36d435c886951987fb4ccaa8--ruby-build--20240517.arm64_ventura.bottle.tar
├── +++ aa750a673b84ed5679172e6c6b4a944dce68ee6f7dc6e1b310c5d63c4df83e0a--ruby-build--20240517.arm64_sonoma.bottle.tar
│ ├── ruby-build/20240517/sbom.spdx.json
│ │ ├── Pretty-printed
│ │ │ @@ -40,19 +40,19 @@
│ │ │ "versionInfo": "20240517"
│ │ │ },
│ │ │ {
│ │ │ "SPDXID": "SPDXRef-Package-SPDXRef-m4-1.4.19",
│ │ │ "checksums": [
│ │ │ {
│ │ │ "algorithm": "SHA256",
│ │ │ - "checksumValue": "11308abe8d607be35da9e88a1d789f191914bf043bca4fdde2b50a6cbf1713cc"
│ │ │ + "checksumValue": "f42d89db519a07d67bcaead6c8dfb2da45e8266bebb996dd8b3f19b1ca13b8a0"
│ │ │ }
│ │ │ ],
│ │ │ "copyrightText": "NOASSERTION",
│ │ │ - "downloadLocation": "https://ghcr.io/v2/homebrew/core/m4/blobs/sha256:11308abe8d607be35da9e88a1d789f191914bf043bca4fdde2b50a6cbf1713cc",
│ │ │ + "downloadLocation": "https://ghcr.io/v2/homebrew/core/m4/blobs/sha256:f42d89db519a07d67bcaead6c8dfb2da45e8266bebb996dd8b3f19b1ca13b8a0",
│ │ │ "externalRefs": [
│ │ │ {
│ │ │ "referenceCategory": "PACKAGE-MANAGER",
│ │ │ "referenceLocator": "pkg:brew/[email protected]",
│ │ │ "referenceType": "purl"
│ │ │ }
│ │ │ ],
│ │ │ @@ -86,19 +86,19 @@
│ │ │ "versionInfo": "2.72"
│ │ │ },
│ │ │ {
│ │ │ "SPDXID": "SPDXRef-Package-SPDXRef-libyaml-0.2.5",
│ │ │ "checksums": [
│ │ │ {
│ │ │ "algorithm": "SHA256",
│ │ │ - "checksumValue": "11239e8f5066c6d0d0718208d4eab518da00c7289f33c9c76c0a09ba5c0417c9"
│ │ │ + "checksumValue": "98c0cf81bcdf7577d5fdc8cc18732970b9ae7e0e7423a733f88f0f566ba483ad"
│ │ │ }
│ │ │ ],
│ │ │ "copyrightText": "NOASSERTION",
│ │ │ - "downloadLocation": "https://ghcr.io/v2/homebrew/core/libyaml/blobs/sha256:11239e8f5066c6d0d0718208d4eab518da00c7289f33c9c76c0a09ba5c0417c9",
│ │ │ + "downloadLocation": "https://ghcr.io/v2/homebrew/core/libyaml/blobs/sha256:98c0cf81bcdf7577d5fdc8cc18732970b9ae7e0e7423a733f88f0f566ba483ad",
│ │ │ "externalRefs": [
│ │ │ {
│ │ │ "referenceCategory": "PACKAGE-MANAGER",
│ │ │ "referenceLocator": "pkg:brew/[email protected]",
│ │ │ "referenceType": "purl"
│ │ │ }
│ │ │ ],
│ │ │ @@ -109,19 +109,19 @@
│ │ │ "versionInfo": "0.2.5"
│ │ │ },
│ │ │ {
│ │ │ "SPDXID": "SPDXRef-Package-SPDXRef-pkg-config-0.29.2_3",
│ │ │ "checksums": [
│ │ │ {
│ │ │ "algorithm": "SHA256",
│ │ │ - "checksumValue": "3ff612c5e44b945c8c0cc6df7d3edb407ca67cddad9c89f9ab99ced494b7a8c2"
│ │ │ + "checksumValue": "7b59abc0b5381065b1eab174217307af9324e0d02edf903171b29250ae58aeaf"
│ │ │ }
│ │ │ ],
│ │ │ "copyrightText": "NOASSERTION",
│ │ │ - "downloadLocation": "https://ghcr.io/v2/homebrew/core/pkg-config/blobs/sha256:3ff612c5e44b945c8c0cc6df7d3edb407ca67cddad9c89f9ab99ced494b7a8c2",
│ │ │ + "downloadLocation": "https://ghcr.io/v2/homebrew/core/pkg-config/blobs/sha256:7b59abc0b5381065b1eab174217307af9324e0d02edf903171b29250ae58aeaf",
│ │ │ "externalRefs": [
│ │ │ {
│ │ │ "referenceCategory": "PACKAGE-MANAGER",
│ │ │ "referenceLocator": "pkg:brew/[email protected]_3",
│ │ │ "referenceType": "purl"
│ │ │ }
│ │ │ ],
│ │ │ @@ -132,19 +132,19 @@
│ │ │ "versionInfo": "0.29.2_3"
│ │ │ },
│ │ │ {
│ │ │ "SPDXID": "SPDXRef-Package-SPDXRef-readline-8.2.10",
│ │ │ "checksums": [
│ │ │ {
│ │ │ "algorithm": "SHA256",
│ │ │ - "checksumValue": "90351660d5ceca72a4c0a287555f2045db95f78aa5f65011b94213429f729cde"
│ │ │ + "checksumValue": "713fd1fa8544426b7e97eb21d13153195fea4c407db8a174bd183777b81c9192"
│ │ │ }
│ │ │ ],
│ │ │ "copyrightText": "NOASSERTION",
│ │ │ - "downloadLocation": "https://ghcr.io/v2/homebrew/core/readline/blobs/sha256:90351660d5ceca72a4c0a287555f2045db95f78aa5f65011b94213429f729cde",
│ │ │ + "downloadLocation": "https://ghcr.io/v2/homebrew/core/readline/blobs/sha256:713fd1fa8544426b7e97eb21d13153195fea4c407db8a174bd183777b81c9192",
│ │ │ "externalRefs": [
│ │ │ {
│ │ │ "referenceCategory": "PACKAGE-MANAGER",
│ │ │ "referenceLocator": "pkg:brew/[email protected]",
│ │ │ "referenceType": "purl"
│ │ │ }
│ │ │ ],
│ │ │ @@ -178,19 +178,19 @@
│ │ │ "versionInfo": "2024-03-11"
│ │ │ },
│ │ │ {
│ │ │ "SPDXID": "[email protected]",
│ │ │ "checksums": [
│ │ │ {
│ │ │ "algorithm": "SHA256",
│ │ │ - "checksumValue": "58665ec9e2873dba2799be5992eab3973f230acc352d09bd4a69131ac3ccd2d4"
│ │ │ + "checksumValue": "ec6f9daf8e32d96f4a2f4cd56d18533ee47bb8d9e7cb3d832ac64115d8a1a4ca"
│ │ │ }
│ │ │ ],
│ │ │ "copyrightText": "NOASSERTION",
│ │ │ - "downloadLocation": "https://ghcr.io/v2/homebrew/core/openssl/3/blobs/sha256:58665ec9e2873dba2799be5992eab3973f230acc352d09bd4a69131ac3ccd2d4",
│ │ │ + "downloadLocation": "https://ghcr.io/v2/homebrew/core/openssl/3/blobs/sha256:ec6f9daf8e32d96f4a2f4cd56d18533ee47bb8d9e7cb3d832ac64115d8a1a4ca",
│ │ │ "externalRefs": [
│ │ │ {
│ │ │ "referenceCategory": "PACKAGE-MANAGER",
│ │ │ "referenceLocator": "pkg:brew/openssl@[email protected]",
│ │ │ "referenceType": "purl"
│ │ │ }
│ │ │ ],
This is basically the problem I describe at #17281 (comment).
from brew.
MikeMcQuaid
commented on June 23, 2024
This is still happening. See Homebrew/homebrew-core@fd1c80d.
This is basically the problem I describe at #17281 (comment).
That problem was fixed. I cannot reproduce this locally. If I run brew bottle ack --json --only-json-tab I get:
- the same checksum every time
- a
ack/3.7.0/sbom.spdx.jsonthat does not contain any of the fields you mention
This was not the case before that was fixed.
This is because we're passing bottling: to skip these at brew bottle time:
brew/Library/Homebrew/dev-cmd/bottle.rb
Line 511 in cb168df
These values are only being added at brew install time:
brew/Library/Homebrew/formula_installer.rb
Line 835 in 610b80e
So this is an issue with either brew test-bot or our homebrew-core CI workflows that is somehow resulting in attempting to double-bottle or use older/cached, broken SBOMs in bottles or something. An issue should probably be opened somewhere but I don't think it's this issue and I don't think (for now) it's Homebrew/brew.
from brew.
carlocab
commented on June 23, 2024
That problem was fixed. I cannot reproduce this locally. If I run
brew bottle ack --json --only-json-tab
No, it is not fixed. You cannot reproduce this problem with ack because ack has no dependencies. You need to do it with something like ruby-build.
Doing brew bottle ruby-build --json --only-json-tab produces a ruby-build/20240517/sbom.spdx.json that contains fields like
{
"SPDXID": "[email protected]",
"name": "openssl@3",
"versionInfo": "3.3.0",
"filesAnalyzed": false,
"licenseDeclared": "NOASSERTION",
"licenseConcluded": "Apache-2.0",
"downloadLocation": "https://ghcr.io/v2/homebrew/core/openssl/3/blobs/sha256:ec6f9daf8e32d96f4a2f4cd56d18533ee47bb8d9e7cb3d832ac64115d8a1a4ca",
"copyrightText": "NOASSERTION",
"checksums": [
{
"algorithm": "SHA256",
"checksumValue": "ec6f9daf8e32d96f4a2f4cd56d18533ee47bb8d9e7cb3d832ac64115d8a1a4ca"
}
],
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceLocator": "pkg:brew/openssl@[email protected]",
"referenceType": "purl"
}
]
}
which will, in general, prevent the creation of :all bottles.
from brew.
MikeMcQuaid
commented on June 23, 2024
Doing
brew bottle ruby-build --json --only-json-tab
Ok, thanks for the reproduction command. It was not clear how to reproduce this before and not clear to me until rereading that this was an additional issue unrelated to reproducibility but related to :all bottles specifically.
#17370 should address this.
from brew.
Related Issues (20)
- Leave manifest platform empty for `:all` bottles HOT 1
- Extract Portable Ruby download information into a templatable file HOT 3
- Bump resources with livechecks on `brew bump`
- Add `brew cleanup --scrub` flag variation
- Wrong tap when listing installed casks from custom taps using `--json=v2 --installed` HOT 1
- `==> ` prefix for installation status and/or deprecation status (and possibly others) HOT 1
- portable-ruby-3.3.2.arm64_big_sur.bottle.tar.gz "The use of 'ruby' has been blocked because it comes from an unidentified developer." HOT 6
- Include language package manager (e.g. `cargo`, `npm`, go modules) information in SBOMs HOT 1
- Tap with :latest does not download new application on explicit "update" command. HOT 1
- brew upgrade can actually downgrade casks in some configurations HOT 4
- Autoremove incorrectly removes build dependencies HOT 9
- `brew list` sometimes returns 1 when all arguments are installed HOT 1
- Cannot upgrade redis-stack-server HOT 1
- regression: Exit code is no longer `1` when listing versions of nonexistent formula/cask HOT 1
- Remove most global "monkey-patches" HOT 2
- Add a way to append linuxbrew to the end of path. HOT 3
- Failed to install Homebrew Portable Ruby HOT 11
- Add color to the instructions after install. HOT 5
- `brew cleanup` removes the current version of `portable-ruby` HOT 3
- Import
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from brew.