canonical / charming-actions Goto Github PK
View Code? Open in Web Editor NEWA collection of Github Actions for interaction with Charmhub, and validating Charmed Operator code
A collection of Github Actions for interaction with Charmhub, and validating Charmed Operator code
Ideally the action should adjust its output based on the differences between the repo and charmhub:
Diff in lib repo vs ch | Lib version | Output |
---|---|---|
No | repo = ch | None (all ok) |
No | repo ≠ ch | Inconsistency warning |
Yes | repo = ch + 1 | None (all ok) |
Yes | repo ≠ ch + 1 | Inconsistency warning |
Github has deprecated the save-state
and set-output
commands. Using these commands will now notify with a deprecation message, and in future not work at all. There is a new method available for the same effect, as described in the linked post.
The following items need to be updated to fix this:
N/A
N/A
N/A
No response
When the major version increases and minor decreases, the library skips releasing the update.
See promethus-k8s-operator CI log output below. prometheus_remote_write 1.1 should have been published.
Bump major version. Set minor to 0.
Github
Run canonical/charming-actions/release-libraries@main
/usr/bin/sudo snap install charmcraft --classic --channel latest/stable
charmcraft 2.4.1 from Canonical** installed
/snap/bin/charmcraft list-lib prometheus-k8s --format=json
[
{
"charm_name": "prometheus-k8s",
"library_name": "prometheus_remote_write",
"library_id": "f783823fa75f4b7880eb70f2077ec259",
"api": 0,
"patch": 15,
"content_hash": "477f3dc99094dac025835a3691df3462ceeb360ee0ade59149bbcabcf08b5c88"
},
{
"charm_name": "prometheus-k8s",
"library_name": "prometheus_scrape",
"library_id": "bc84295fef5f4049878f07b131968ee2",
"api": 0,
"patch": 39,
"content_hash": "dd816a001c52ca853537e8a772f4548a090e7d441b126b71bbedc58e0d226c76"
}
]
found lib file: ./lib/charms/prometheus_k8s/v0/prometheus_scrape.py. Parsing...
found lib file: ./lib/charms/prometheus_k8s/v1/prometheus_remote_write.py. Parsing...
checking status for prometheus_scrape
remote lib: {"libName":"prometheus_scrape","version":0,"revision":39}
prometheus_scrape will be updated from 0.39 to 0.40
checking status for prometheus_remote_write
remote lib: {"libName":"prometheus_remote_write","version":0,"revision":15}
prometheus_remote_write has a lower version or revision than prometheus_remote_write
Errors found during lib comparison.
Publishing changes:
[{"libName":"prometheus_scrape","old":{"major":0,"minor":39},"new":{"minor":40,"major":0}}]
publishing prometheus_scrape
Successfully published libraries
/snap/bin/charmcraft publish-lib charms.prometheus_k8s.v0.prometheus_scrape
Library charms.prometheus_k8s.v0.prometheus_scrape sent to the store with version 0.40
No response
Errors on upload-charm:
Error: Validation Failed: {"resource":"Release","code":"already_exists","field":"tag_name"}
Error: HttpError: Validation Failed: {"resource":"Release","code":"already_exists","field":"tag_name"}
Rerun this job:
https://github.com/canonical/alertmanager-k8s-operator/runs/7455137256?check_suite_focus=true
GH runner.
Run canonical/charming-actions/[email protected]
/usr/bin/sudo snap install charmcraft --classic --channel latest/edge
charmcraft (edge) 2.0.0+6.g7563ce5 from Canonical** installed
/snap/bin/charmcraft pack --destructive-mode --quiet
/usr/bin/docker pull ubuntu/prometheus-alertmanager:0.23-22.04_beta
0.23-22.04_beta: Pulling from ubuntu/prometheus-alertmanager
bdc5a07a7dd2: Pulling fs layer
aa9fa10e633d: Pulling fs layer
e5dd500af9cc: Pulling fs layer
676141a7b9c0: Pulling fs layer
3c2910da3d19: Pulling fs layer
5fdd3cbd15aa: Pulling fs layer
78b2bc159136: Pulling fs layer
3c2910da3d19: Waiting
5fdd3cbd15aa: Waiting
78b2bc159136: Waiting
676141a7b9c0: Waiting
e5dd500af9cc: Verifying Checksum
e5dd500af9cc: Download complete
bdc5a07a7dd2: Verifying Checksum
bdc5a07a7dd2: Download complete
3c2910da3d19: Verifying Checksum
3c2910da3d19: Download complete
676141a7b9c0: Verifying Checksum
676141a7b9c0: Download complete
78b2bc159136: Verifying Checksum
78b2bc159136: Download complete
5fdd3cbd15aa: Verifying Checksum
5fdd3cbd15aa: Download complete
aa9fa10e633d: Verifying Checksum
aa9fa10e633d: Download complete
bdc5a07a7dd2: Pull complete
aa9fa10e633d: Pull complete
e5dd500af9cc: Pull complete
676141a7b9c0: Pull complete
3c2910da3d19: Pull complete
5fdd3cbd15aa: Pull complete
78b2bc159136: Pull complete
Digest: sha256:8f30b70d39053bf979ba646c1444287d71f7ece2342dea518106a6077197cd15
Status: Downloaded newer image for ubuntu/prometheus-alertmanager:0.23-22.04_beta
docker.io/ubuntu/prometheus-alertmanager:0.23-22.04_beta
/snap/bin/charmcraft upload-resource --quiet alertmanager-k8s alertmanager-image --image ubuntu/prometheus-alertmanager:0.23-22.04_beta
/snap/bin/charmcraft resource-revisions alertmanager-k8s alertmanager-image
Revision Created at Size
15 2022-07-21T18:08:50Z 515B
14 2022-07-20T15:27:02Z 515B
13 2022-07-15T11:48:59Z 515B
12 2022-07-06T15:54:06Z 515B
11 2022-07-05T23:27:14Z 515B
10 2022-07-03T20:54:42Z 515B
9 2022-06-30T09:06:59Z 515B
8 2022-06-28T03:41:22Z 515B
7 2022-05-30T21:35:04Z 515B
6 2022-04-28T20:38:02Z 515B
5 2022-03-24T14:04:40Z 515B
4 2022-03-22T15:37:00Z 515B
3 2022-03-17T20:09:33Z 515B
2 2022-03-16T21:47:46Z 515B
1 2021-07-21T18:28:17Z 515B
/snap/bin/charmcraft upload --quiet --release latest/edge /home/runner/work/alertmanager-k8s-operator/alertmanager-k8s-operator/alertmanager-k8s_ubuntu-20.04-amd64.charm --resource=alertmanager-image:15
Error: Validation Failed: {"resource":"Release","code":"already_exists","field":"tag_name"}
Error: HttpError: Validation Failed: {"resource":"Release","code":"already_exists","field":"tag_name"}
at /home/runner/work/_actions/canonical/charming-actions/1.0.0/dist/upload-charm/index.js:7440:21
at processTicksAndRejections (internal/process/task_queues.js:97:5)
Total size of all the files uploaded is 9165 bytes
Finished uploading artifact charmcraft-logs. Reported size is 9165 bytes. There were 0 items that failed to upload
Artifact upload result: {"artifactName":"charmcraft-logs","artifactItems":["/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220721-175635.208482.log","/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220721-180804.853589.log","/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220721-180853.491465.log","/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220721-180855.218908.log"],"size":9165,"failedItems":[]}
No response
For tracks that have charmhub branches, such as:
charmcraft status metacontroller-operator
Track Base Channel Version Revision Expires at
latest ubuntu 20.04 (amd64) stable 2 2
candidate 5 5
beta ↑ ↑
edge 13 13
edge/add-charming-actions 9 9 2022-06-30T01:00:00Z
edge/resources-removal 12 12 2022-07-09T01:00:00Z
the release-charm
action fails due to incorrect parsing of the available channels.
See this action run
See this action run
See [this action run](https://github.com/canonical/metacontroller-operator/runs/7073697720?check_suite_focus=true)
charmcraft
edge now allows for structured output using charmcraft status charmName --format json
. This would let us avoid parsing the human readable text. There are some growing-pains with this output, however.
Running charmcraft 2.0.0, the upload-charm action published this release with undefined version number.
I believe the source of this issue is because the --quiet
flag's behaviour has changed:
--quiet
removed all output except for the final message ("Revision X ..." for success, or "Upload failed with status ..." for failure)--quiet
silences all output, both for success and failure. So revision
cannot be pulled from the stdout, and an error cannot be inferred from the text either (but can be inferred from $?
)see this ci run
You can also try yourself by charmcraft upload somecharm --quiet
on 1.7.x and 2.0.0
charmcraft 2.0.0
see [this ci run](https://github.com/canonical/alertmanager-k8s-operator/actions/runs/2676384442)
Fixing this so it properly parses charmcraft 2.0.0 would be excellent, but we should also add checks that would have marked rev=undefined
as a failure in the action as well. That way it fails loudly, rather than "works" incorrectly
As long as the file resource in question is available in the charm root folder, uploading file resources should be a quite trivial addition. To be clear, I don't think the action needs to be able to download the resource itself - that could be done in a prior step, just upload them to charmhub and add them to the release.
This is probably a lot to ask for, but:
A revision is created when all I changed was tests, which are not included in the charm.
Parsing charmcraft.yaml
and re-implementing charmcraft's logic is probably too much to ask for though.
Unless this could be turned into a bot with commands..? In that case we'd use commands such as $$merge$$
and $$merge-no-upload$$
.
HI,
The release-libraries
silently keeps library non-published if LIBPATCH is too high due to human mistake
(e.g. the latest published LIBPATCH is 3 and the currently committed is 5):
> charmcraft publish-lib charms.mongodb.v0.mongodb_tls
Library charms.mongodb.v0.mongodb_tls has a wrong LIBPATCH number,
it's too high and needs to be consecutive, Charmhub highest version is 0.3.
> echo $?
0
Charmhub enforces LIBPATCH needs to be consecutive. Unfortunately exit code there is zero, so it is silently missing.
charmhub 2.2.0
charming-actions 2.2.2
> charmcraft publish-lib charms.mongodb.v0.mongodb_tls
Library charms.mongodb.v0.mongodb_tls has a wrong LIBPATCH number, it's too high and needs to be consecutive, Charmhub highest version is 0.3.
> echo $?
0
No response
It would be handy if there was a dependabot-like scheduled action to automatically open a PR when fetch-lib results in an update.
For the observability bundle, we periodically open "upkeep" PRs manually for this purpose.
Interesting, Charmcraft seems to fail to upload a charmhub branch when we add a /
in the git branch. That's something we should probably address.
Originally posted by @DomFleischmann in #29 (review)
Based on the readme.md files in the root and /channel
, it is undefined what CI outcomes are expected when a push/PR is out of scope for the channel
action. channel/readme.md
notes that pushes to and PRs against any branch other than default
or track/*
are Ignored
, but does not state whether an error is thrown or if a value is returned in steps.<ID>.outputs.name
in that event. Looking at the code I believe an error is thrown, but my javascript is poor :)
If errors are thrown for out-of-scope channels, is that the desired outcome? A failure shows as an :X: in the CI (and will block merges in github if repos have that enabled). I think we either need to run successfully even when this is an out-of-scope channel (maybe name=""
is defined as the return for out-of-scope events?), or in our workflow snippet in the root readme.md
we need to advise on how to handle these events in a way that doesn't fail the CI. If we return name=""
, we'd also need an if
statement for the upload-action
so we skipped it if name was empty (otherwise either upload-charm
will fail or will send to the default channel, not sure how github would handle that)
However we handle failures for out-of-scope channels, I think we need to set up the workflow so it gives a clear message of what happened in the CI/checks summary. I'd expect outcomes to be:
Set destination channel
and Publish to selected channel
(or better names :) ). Then Publish to selected channel
will depend on (need
) the Set destination channel
, and the job can run only if
Set destination channel
was successful (or, if it returned name != ""
). I think that would then achieve a clear picture in CI, where the publish will only show :X: if it actually broke, rather than just should not have ranThis uses charmcraft resource-revisions
to get all revisions, then takes the top line as the resource we just uploaded. This will often be true, but might not be if:
Suggested fix: parse revision number from the output from charmcraft upload-resource
which says something like Revision N created of resource ...
.
Kubeflow charms also have microk8s.inspect
which could be handy.
- name: Generate inspect tarball
run: >
sg microk8s <<EOF
microk8s inspect | \
grep -Po "Report tarball is at \K.+" | \
xargs -I {} cp {} inspection-report-${{ strategy.job-index }}.tar.gz
EOF
if: failure()
- name: Upload inspect tarball
uses: actions/upload-artifact@v2
with:
name: inspection-reports
path: ./inspection-report-${{ strategy.job-index }}.tar.gz
if: failure()
It is a minor report, just to keep the house clean.
GitHub reports warning:
Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: canonical/charming-actions/[email protected]. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.
You can find it here: https://github.com/canonical/s3-integrator/actions/runs/4004942011
Use "charming-actions/[email protected]" is GH CI/CD and check Actions->Summary->Warnings.
GH Action with the latest availabe [email protected]
Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: canonical/charming-actions/[email protected]. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.
The warning is obliviously reported for other actions too, e.g. channel
:
Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: actions/checkout@v2, canonical/charming-actions/[email protected], canonical/charming-actions/[email protected]. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.
in https://github.com/canonical/mysql-operator/actions/runs/3986561030
Upload-charm action fails when charm is updated from not having any resources to having a resource. When uploading the resources, charmcraft is verifying resources specification against the latest metadata on charmcraft (which is outdated in this scenario). The new charm needs to be uploaded first before uploading a resource.
metadata.yaml
On github
failed github action run for reference
/snap/bin/charmcraft upload-resource --quiet kratos oci-image --image 9f4e08abbf6a
Store operation failed:
- resource-not-found: No oci-image type "oci-image" resource found
No response
Rather than selectively uploading intermediate work to charmhub branches (eg: latest/edge/someBranch
), we could upload all PR updates to charmhub. charmhub branches are ephemeral (ttl=30day) anyway so it won't make too much garbage, and this way we can easily test deployments.
There might be some logic that needs working through as we still need to know the destination track/channel that the branch should go to, but this could be the same as we currently use (if git branch is called /track/...
, push to charmhub at /track/edge/branchName
, else push to latest/edge/branchname
)
When executing the the upload-action for a charm specifying an upstream-resource the exception attached occurs. This happens independently of the upload
and resource-overrides
options specified:
- name: Upload charm to charmhub
uses: canonical/charming-actions/[email protected]
with:
credentials: ${{ secrets.CHARMHUB_TOKEN }}
github-token: ${{ secrets.GITHUB_TOKEN }}
channel: edge
upload: false
The resources section of the charm metadata file is the following:
resources:
indico-image:
type: oci-image
description: Docker image for Indico
auto-fetch: true
indico-nginx-image:
type: oci-image
description: Docker image for nginx Indico
auto-fetch: true
nginx-prometheus-exporter-image:
type: oci-image
description: Prometheus exporter for nginx
auto-fetch: true
upstream-source: nginx/nginx-prometheus-exporter:0.10.0
Run the following workflow of the indico operator changing the existing metadata contents for
resources:
indico-image:
type: oci-image
description: Docker image for Indico
auto-fetch: true
indico-nginx-image:
type: oci-image
description: Docker image for nginx Indico
auto-fetch: true
nginx-prometheus-exporter-image:
type: oci-image
description: Prometheus exporter for nginx
auto-fetch: true
upstream-source: nginx/nginx-prometheus-exporter:0.10.0
The issue popped-up while running a github workflow
/usr/bin/sudo snap install charmcraft --classic --channel latest/stable
charmcraft 2.0.0 from Canonical** installed
/usr/bin/sudo charmcraft pack --destructive-mode --quiet
/usr/bin/docker pull undefined
/usr/bin/docker pull nginx/nginx-prometheus-exporter:0.10.0
Using default tag: latest
***Error response from daemon: pull access denied for undefined, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
Error: The process '/usr/bin/docker' failed with exit code 1
Error: Error: The process '/usr/bin/docker' failed with exit code 1
at ExecState._setResult (/home/runner/work/_actions/canonical/charming-actions/1.0.3/dist/upload-charm/index.js:3276:25)
at ExecState.CheckComplete (/home/runner/work/_actions/canonical/charming-actions/1.0.3/dist/upload-charm/index.js:3259:18)
at ChildProcess.<anonymous> (/home/runner/work/_actions/canonical/charming-actions/1.0.3/dist/upload-charm/index.js:3153:27)
at ChildProcess.emit (events.js:314:20)
at maybeClose (internal/child_process.js:1022:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
No charmcraft logs generated, skipping artifact upload.***
0.10.0: Pulling from nginx/nginx-prometheus-exporter
6ca88092a[12](https://github.com/canonical/indico-operator/runs/7970721141?check_suite_focus=true#step:3:13)c: Pulling fs layer
aaea71c792be: Pulling fs layer
6ca88092a12c: Verifying Checksum
6ca88092a12c: Download complete
aaea71c792be: Verifying Checksum
aaea71c792be: Download complete
6ca88092a12c: Pull complete
aaea71c792be: Pull complete
Digest: sha256:1c6eba17d7768401ba1ed056101fc56ec3e3b5c82e508fdd8968fbcd96ec9574
Status: Downloaded newer image for nginx/nginx-prometheus-exporter:0.10.0
docker.io/nginx/nginx-prometheus-exporter:0.10.0
/snap/bin/charmcraft upload-resource --quiet indico nginx-prometheus-exporter-image --image nginx/nginx-prometheus-exporter:0.10.0
/snap/bin/charmcraft resource-revisions indico nginx-prometheus-exporter-image
Revision Created at Size
***5 2022-08-23T10:06:10Z 545B***
4 2022-08-22T09:51:04Z 545B
3 2022-08-19T15:10:27Z 545B
2 2022-08-19T[13](https://github.com/canonical/indico-operator/runs/7970721141?check_suite_focus=true#step:3:14):35:[32](https://github.com/canonical/indico-operator/runs/7970721141?check_suite_focus=true#step:3:33)Z 5[45](https://github.com/canonical/indico-operator/runs/7970721141?check_suite_focus=true#step:3:46)B
1 2022-08-19T13:32:26Z 5[45](https://github.com/canonical/indico-operator/runs/7970721141?check_suite_focus=true#step:3:46)B
No response
Recently the snap library was bumped to v2 in canonical/operator-libs-linux#92.
The CI didn't seem to catch that, which you can see here
Update a library LIBAPI version, moving the file into the new version
operator-libs-linux repo
/usr/bin/sudo snap install charmcraft --classic --channel latest/stable
charmcraft 2.3.0 from Canonical** installed
/snap/bin/charmcraft list-lib operator-libs-linux --format=json
[
{
"charm_name": "operator-libs-linux",
"library_name": "apt",
"library_id": "7c3dbc9c2ad44a47bd6fcb25caa270e5",
"api": 0,
"patch": 11,
"content_hash": "e16de26b23a6b35ef565c99d49d511f3e781b8d56a58a6f1edf2f4e0e3f6f3a4"
},
{
"charm_name": "operator-libs-linux",
"library_name": "dnf",
"library_id": "1e93f444444d4a4a8df06c1c16b33aaf",
"api": 0,
"patch": 1,
"content_hash": "c030c118a493f355fca303f07ca4795598f708d8931af0d9b4aa1eee92156880"
},
{
"charm_name": "operator-libs-linux",
"library_name": "passwd",
"library_id": "cf7655b2bf914d67ac963f72b930f6bb",
"api": 0,
"patch": 4,
"content_hash": "441ad5d92530c227a78e2207f5650f86a8b842134545e222514b3d6425ed9673"
},
{
"charm_name": "operator-libs-linux",
"library_name": "snap",
"library_id": "05394e5893f94f2d90feb7cbe6b633cd",
"api": 1,
"patch": 12,
"content_hash": "ab4190b96a2bddfa60caf1c14a74868f378b5243396e6e4a2b6604c6b9b10976"
},
{
"charm_name": "operator-libs-linux",
"library_name": "systemd",
"library_id": "045b0d179f6b4514a8bb9b48aee9ebaf",
"api": 1,
"patch": 3,
"content_hash": "56e0dad863d86816ef55ac1fae4f61a42d8d66a73d49620ea8d6768282480575"
}
]
found lib file: ./lib/charms/operator_libs_linux/v0/apt.py. Parsing...
found lib file: ./lib/charms/operator_libs_linux/v0/dnf.py. Parsing...
found lib file: ./lib/charms/operator_libs_linux/v0/passwd.py. Parsing...
found lib file: ./lib/charms/operator_libs_linux/v1/systemd.py. Parsing...
found lib file: ./lib/charms/operator_libs_linux/v2/snap.py. Parsing...
checking status for apt
remote lib: {"libName":"apt","version":0,"revision":11}
apt and apt are equal
checking status for dnf
remote lib: {"libName":"dnf","version":0,"revision":1}
dnf and dnf are equal
checking status for passwd
remote lib: {"libName":"passwd","version":0,"revision":4}
passwd and passwd are equal
checking status for systemd
remote lib: {"libName":"systemd","version":1,"revision":3}
systemd and systemd are equal
checking status for snap
remote lib: {"libName":"snap","version":1,"revision":12}
snap has a lower version or revision than snap
Errors found during lib comparison.
Nothing to update. Exiting...
### Additional context
_No response_
Hi,
We are running check-libraries
on PRs to make sure our charms are using the latest shared libraries.
The approach with green/red labels works perfectly: example.
What is missing is a human protection to inform author about forgotten LIBPATCH bump on the shared library change: example.
What about to set red label: Libraries: bump LIBPATCH
if author changes shared library and forget to bump it?
At the moment the action simply prints notice and sets green label Libraries: OK
:
Library charms.mongodb.v0.helpers has local changes, cannot be updated.
Library charms.mongodb.v0.mongodb_backups has local changes, cannot be updated.
Library charms.mongodb.v0.mongodb_vm_legacy_provider has local changes, cannot be updated.
We have tried to use option fail-build
, but it didn't fail on locally changed library.
Is it a broken option or it was designed for something else? Tnx!
When running in a multi-charm repo like kfp-operators, we execute check-libraries multiple times in the same CI run. When one of those runs returns with a library needing a version bump, it is unclear from the message which one it is.
It would be great if the returned message included the path that the check-libraries function was run on.
Publishing a charm specifying an uptream resource hosted on a private repository that requires authentication is not currently supported. Specifically, I'm trying to use an image uploaded to charmhub
upstream-source: 'registry.jujucharms.com/charm/e7cgxprwp5telxhlb24qmbxuoi9a5e3m40iv/indico-image@sha256:c40b5291a0346132d1f6ed5a38f92fafdefd7da64655742881729a8cbaf6a807'
getting the following error message:
Error response from daemon: pull access denied for undefined, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
Nice work so far! 🚀
It would be great if this action could also detect changes in lib/charms/<charm_name>/v#/<lib_name>
and publish updated version of charm libs. Thinking out loud, there are some gotchas:
LIBPATCH
has actually been bumped and fail gracefully/warn if notTagging in @petevg and @simskij for the discussion too.
Just a note for @DomFleischmann that we need to think
carefully about using this, and what the succession plan is > for that tool now that Ken has likely stopped maintaining it.
Originally posted by @jnsgruk in #21 (comment)
Every time we open a PR we should publish the built charm to a Charmhub branch.
For example, if we open a PR-25 for the git branch track/1.4
the charm built in that PR should be pushed to the charmhub branch 1.4/edge/pr-25
everytime the CI of the PR is running.
This will allow for easier testing of PRs as we will just have to pull the build from charmhub. Additionally, it will test if the charm is publishable before merging the PR.
this messaage shows when, for example, the channel
action is used on a PR that is against a feature branch (eg: PR from some-branch
to some-other-branch
, where the target is not the default branch or a track/*
branch). This message does not explain what is happening in a way someone without extra context could interpret.
Originally posted by @simskij in #21 (comment)
charming-actions/dist/channel/index.js
Lines 23192 to 23207 in ee30262
if one works on a feature branch and wants to publish a charm, then the action will fail.
for example, I want to merge feat-999-add-relation-psql
into feat-999
branch, then action will fail because Unhandled PR base name feat-999
while would be good to publish charm (according to the internal logic) to latest/edge/feat-999-add-relation-psql
real use case: https://github.com/canonical/mlflow-operator/actions/runs/4344924048/jobs/7588922599
If a lib version / libpatch is updated, I'd like an action to publish it to charmhub.
If you could add me to the contributors and assign this to me, I'll open a PR soon :)
Every time we merge a PR to main
or a track/foo
branch a charm will be published to latest/edge
or foo/edge
in charmhub.
It is easy to know what code corresponds to the charm in edge
as it will always be the latest code. But once we start promoting different revisions we will lose track of that.
Therefore, when publishing a new charm we should also tag the latest commit in our git history with the charm revision of the published charm. That way we can search for these revisions easily in our git history.
This should only apply for when we publish to a channel, not for a charmhub branch
get an issue in CI with unclear fail message:
Also tested with 2.0.0 stable
Run canonical/charming-actions/[email protected]
/usr/bin/sudo snap install charmcraft --classic --channel latest/stable
charmcraft 2.0.0 from Canonical** installed
/snap/bin/charmcraft fetch-lib
No keyring found to store or retrieve credentials from.
Full execution log: '/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220907-085012.313435.log'
Error: The process '/snap/bin/charmcraft' failed with exit code 1
Error: Error: The process '/snap/bin/charmcraft' failed with exit code 1
at ExecState._setResult (/home/runner/work/_actions/canonical/charming-actions/2.0.0-rc2/dist/check-libraries/index.js:3276:25)
at ExecState.CheckComplete (/home/runner/work/_actions/canonical/charming-actions/2.0.0-rc2/dist/check-libraries/index.js:3259:18)
at ChildProcess.<anonymous> (/home/runner/work/_actions/canonical/charming-actions/2.0.0-rc2/dist/check-libraries/index.js:3153:27)
at ChildProcess.emit (events.js:314:20)
at maybeClose (internal/child_process.js:1022:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
we run following pipe on https://github.com/canonical/grafana-k8s-operator
lib-check:
name: Check libraries
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Check libs
uses: canonical/charming-actions/[email protected]
with:
credentials: "${{ secrets.CHARMHUB_TOKEN }}"
github-token: "${{ secrets.GITHUB_TOKEN }}"
for example see checks in canonical/grafana-k8s-operator#118
see the bug description
No response
The upload charm action doesn't have an input for setting the Charmhub upload name (analogous to charmcraft upload --name
). We want to upload the charm under a different name for testing.
- name: Upload charm to charmhub
uses: canonical/charming-actions/[email protected]
with:
name: ....
Charmcraft's charmcraft upload-resource
by design requires any image to be upload be specified by digest, not by image name. In charmcraft <=2.1.0, charmcraft upload-resource
did not check for this and by chance used the input in a way that successfully uploaded the image even though it was not specified by digest.
Charmcraft 2.1.0 fixes a bug in handling image digests in a way that
breaks using tagged images (specifically, charmcraft treats any input with a :
as a digest, like sha256:digest
). This means the upload-charm action fails because it provides an image name instead of a digest.
Use upload-action in any CI with charmcraft >2.1.0
Github runners
Example of failure: https://github.com/canonical/kubeflow-profiles-operator/actions/runs/3235732172/jobs/5300571800
No response
If a charm has dependencies defined in its charmcraft.yaml
file, it will try to apt-get install
them. However, the upload-charm
is hitting a Permission Denied error, which can be solved by just running sudo apt-get install
instead.
sample build-packages in charmcraft.yaml:
parts:
charm:
build-packages:
- git
- python3-dev
- libffi-dev
- cargo
- libssl-dev
Charm: https://github.com/claudiubelu/legend-juju-sdlc-server-operator
Job: https://github.com/claudiubelu/legend-juju-sdlc-server-operator/runs/5589471448?check_suite_focus=true
Logs: https://paste.ubuntu.com/p/pDgzgQMNm6/
This charm has logic to skip publishing when called from branches/events that do not meet these conditions. If called from a non-publishable event, the action returns as passing (green in github actions page) even though nothing was published.
When CI tests this charm in this repo, it is subject to the same conditions as above. Thus, testing a PR from a branch other than the allowed names (eg: branch/my-branch
) results in the CI showing green but not having been run. This should be changed
Possible fixes:
If we don't do something that's really obvious, we should also add notes in the readme for developers (eg: when making mods to this repo, do them from branches called branch/my-new-feature
)
The upload-charm action fails when it tries to upload any new resource. For example, if you have an existing charm and add a resource to that charm, you must manually upload an instance of that resource before the action will work. It would be good if the action caught this and uploaded the resource for you as expected.
Add a resource to a charm, then try to upload the charm using this action
github action run for any environment
Logs from a past action run, which attempted to upload resource `myfile` for the first time:
Error: Resource 'myfile' does not have any uploaded revisions.
Error: Error: Resource 'myfile' does not have any uploaded revisions.
at Charmcraft.<anonymous> (/home/runner/work/_actions/canonical/charming-actions/2.0.0-rc/dist/upload-charm/index.js:21737:23)
at Generator.next (<anonymous>)
at fulfilled (/home/runner/work/_actions/canonical/charming-actions/2.0.0-rc/dist/upload-charm/index.js:21633:58)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
No response
The Kubeflow team had CI using this action recently fail when executed from a fork. We believe this is because the fork did not define the credentials secret, thus meaning the credentials argument passed to upload-charm
was an empty string.
Assuming this is the correct failure mode, upload-charm
should catch and fail with an obvious error when it is passed blank or obviously incorrect credentials. Calling the action with an empty credential is an especially easy error to make in github actions, as actions will silently use an uninstantiated variable rather than raise an error.
We recently added the canonical/charming-actions/release-libraries
action to our reusable workflows and have isolated an issue we are seeing to the inclusion of this action in the reusable workflow. With this job:
release-charm-libs:
name: Release charm libs
uses: canonical/charming-actions/[email protected]
needs: [ lib-check, tests, integration-tests ]
with:
credentials: ${{ secrets.CHARMHUB_TOKEN }}
github-token: ${{ secrets.GITHUB_TOKEN }}
To https://github.com/canonical/operator-workflows/blob/main/.github/workflows/test_and_publish_charm.yaml (lines 90-96), we get the following failure:
[Invalid workflow file: .github/workflows/test_and_publish_charm.yaml#L19](https://github.com/canonical/indico-operator/actions/runs/4000676362/workflow)
error parsing called workflow
".github/workflows/test_and_publish_charm.yaml"
-> "canonical/operator-workflows/.github/workflows/test_and_publish_charm.yaml@main" (source branch with sha:a914f5ce827f00cc5e314b2d54ff9227a79c95ff)
: invalid value workflow reference: references to workflows must be rooted in '.github/workflows'
Without it, the workflow runs fine. Obviously the above error is a red herring, I'm not sure what is causing the problem. Do you have any ideas?
Add:
release-charm-libs:
name: Release charm libs
uses: canonical/charming-actions/[email protected]
needs: [ lib-check, tests, integration-tests ]
with:
credentials: ${{ secrets.CHARMHUB_TOKEN }}
github-token: ${{ secrets.GITHUB_TOKEN }}
To a reusable workflow and call it from a repository
This is running on GitHub actions
[Invalid workflow file: .github/workflows/test_and_publish_charm.yaml#L19](https://github.com/canonical/indico-operator/actions/runs/4000676362/workflow)
error parsing called workflow
".github/workflows/test_and_publish_charm.yaml"
-> "canonical/operator-workflows/.github/workflows/test_and_publish_charm.yaml@main" (source branch with sha:a914f5ce827f00cc5e314b2d54ff9227a79c95ff)
: invalid value workflow reference: references to workflows must be rooted in '.github/workflows'
### Additional context
_No response_
As stated in the title, the release-charm
action currently fails for resourceless charms. The reason seems to be that we're not guarding against empty resource blocks in the charmcraft output. This should solve itself when we move over to using the programmatic output, but we still need to track it here.
charming-actions/src/services/charmcraft/charmcraft.ts
Lines 261 to 275 in 39aea17
(cc @agathanatasha)
Currently, the juju-bundle
implementation in the upload-bundle
action changes directories to the specified path and looks for a bundle.yaml
file as seen in here:
import * as core from '@actions/core';
import * as exec from '@actions/exec';
class Bundle {
async publish(path: string, channel: string) {
core.exportVariable('CHARMCRAFT_AUTH', core.getInput('credentials'));
process.chdir(path);
await exec.exec('juju-bundle', [
'publish',
'--destructive-mode',
'--serial',
'--release',
channel,
]);
}
}
export { Bundle };
This unnecessarily enforces bundle names which creates unnecessary rigidity to how bundle repos have to be structured. Not only do the bundles have to be named bundle.yaml
, but they also will have to be in separate directories. This makes little sense when a path to a specific bundle can be given straight to juju-bundle
using --bundle
.
It would be useful to return structured output when executing the upload-charm action including the uploaded charm revision and the charm resources revisions to be able to use them in future steps pf the workflow.
Thank you
The label "Libraries: Out of sync" is remain for PR with updated libs.
Expected results: label replaced from "Libraries: Out of sync" to "Libraries: OK"
Proposal: update the label if test result status != GitHub label.
PR: canonical/postgresql-k8s-operator#86
Test where old label remain: https://github.com/canonical/postgresql-k8s-operator/actions/runs/4024638068/jobs/6916898343
Run canonical/charming-actions/[email protected]
with:
credentials: ***
github-token: ***
charm-path: .
charmcraft-channel: latest/stable
use-labels: true
label-success: Libraries: OK
label-fail: Libraries: Out of sync
comment-on-pr: false
fail-build: false
/usr/bin/sudo snap install charmcraft --classic --channel latest/stable
charmcraft 2.2.0 from Canonical** installed
/snap/bin/charmcraft fetch-lib
Library charms.data_platform_libs.v0.database_provides was already up to date in version 0.2.
Library charms.observability_libs.v1.kubernetes_service_patch was already up to date in version 1.5.
Library charms.postgresql_k8s.v0.postgresql was already up to date in version 0.7.
Library charms.postgresql_k8s.v0.postgresql_tls was already up to date in version 0.5.
Library charms.rolling_ops.v0.rollingops was already up to date in version 0.2.
Library charms.tls_certificates_interface.v1.tls_certificates was already up to date in version 1.12.
/usr/bin/git checkout HEAD -- lib
### Additional context
_No response_
The charming actions v2.3.0 use deprecated node12. Github started enforcing migration to node16 (see this article). The main branch is already updated with the new version, but would it be possible to create a new tag for it?
With a new tag, renovate would open PRs to update repos which use the charming actions in CI.
The current logdump action is not very good and has quite a lot of annoying limitations. We should switch it out to an action in here, powered by https://github.com/juju/juju-crashdump.
Hi,
At the moment upload-charm
is always building charms using --destructive-mode
for the speed reason.
Unfortunately it cause issues when GH runners (e.g. ubuntu-latest
) are migrated to new versions and charms are still using old base (#1, #2). It would be great to avoid --destructive-mode
if built-on
doesn't match the runner's host OS version.
The proposal came from @jnsgruk here. Tnx!
P.S. it is not always possible to update base-on
for subordinate charms as we need to support multiple serials in parallel.
Using the following upload-bundle
action fails with the following error: Parts processing error: Failed to copy '/tmp/.tmpDpiw3w/build/stage/icon.svg': no such file or directory.
But the repository [1] has an icon.svg
file, and regular charmcraft pack
works.
[1] https://github.com/claudiubelu/waltz-juju-bundle
Github action:
- name: Upload bundle to edge
uses: canonical/charming-actions/[email protected]
with:
credentials: "${{ secrets.CHARMCRAFT_AUTH }}"
github-token: "${{ secrets.GITHUB_TOKEN }}"
channel: "${{ steps.release-short.outputs.short_version }}/edge"
Github action: https://github.com/claudiubelu/waltz-juju-bundle/blob/560951f1982fdc6bc37829186c40681d22adfced/.github/workflows/create_release.yaml
Error: Error running subcommand ``charmcraft pack -p /tmp/.tmpDpiw3w``: `Packing the bundle.
Parts processing error: Failed to copy '/tmp/.tmpDpiw3w/build/stage/icon.svg': no such file or directory.
Full execution log: '/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220416-011600.638784.log'
Failure: https://github.com/claudiubelu/waltz-juju-bundle/runs/6044583147?check_suite_focus=true
### Additional context
_No response_
The code here uploads the single charm only.
Hovewer the charmcraft.yaml supports multiple bases
(which is critical for subordinate operators).
It is an enhancement request to improve 'upload-charm' action to upload all *.charm files produced by charmcraft.
Thank you in advance!
This request is similar to #16, in the sense that it is looking to address a gap currently in the existing actions for charms with file resources.
The upload-charm
readme contains the following,
When uploading a charm with file resources, the most recent resource will be attached to the release. If you want to use a new resource, you'll have to cut a new resource revision prior to running the action.
If I'm not mistaken this would mean a workflow that wants to upload a resource + a charm in one go would need to look something like the below. This feels like extra boilerplate and a lack of consistency between the manual steps and the action. My suggestion would be a separate action dedicated to uploading file resources, that would prevent any breaking changes to the upload-charm
action.
- uses: actions/download-artifact@master
with:
name: <snap-name>
- name: Install charmcraft
run: sudo snap install charmcraft --channel=2.x/stable
- name: Publish Charm Resource
run: charmcraft upload-resource <charm-name> <resource-name> --filepath ./<file-name>
env:
CHARMCRAFT_AUTH: "${{ secrets.CHARMHUB_TOKEN }}"
- name: Upload charm to charmhub
uses: canonical/charming-actions/[email protected]
with:
credentials: "${{ secrets.CHARMHUB_TOKEN }}"
github-token: "${{ secrets.GITHUB_TOKEN }}"
channel: "${{ steps.channel.outputs.name }}"
Interested to hear alternate approaches to this issue.
One of the biggest bottlenecks of charm development is building charm sin the CI: it takes forever, and it only gets worse as the list of charms to build grows. The data-platform-workflows (by @carlcsaposs-canonical) repository contains a series of GH actions that can be leveraged for building the charms with a cache and reuse them across different jobs in a workflow.
The most obvious example where this is useful is the publish job, which also packs the charm(s) in the repo on the same CI where other jobs are running, but there could be other actions that may be benefited from this.
For more information, go here
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.