circleci-public / circleci-cli Goto Github PK
View Code? Open in Web Editor NEWUse CircleCI from the command line
Home Page: https://circleci-public.github.io/circleci-cli/
License: MIT License
Use CircleCI from the command line
Home Page: https://circleci-public.github.io/circleci-cli/
License: MIT License
I am using version 0.1.3257+b197d90 of the CLI.
I have a parameter called 'myparam'. When I use this parameter, and ran validation, the following happened:
The only alternative that actually works is number 3. Number 1 will not work. When I ran 'circleci config process', it was not able to find the parameter.
Should number 1 result in a validation error, or should 'circleci config process' be able to find the parameter?
Btw, when I check in number 1, the circleci.com processing result in the same as the CLI processing.
I have a small script that drops me into a bash session inside a container using the same image I use for builds on CircleCI. I use the circleci
CLI to canonicalize the YAML configuration so I can slice up the output and figure out the name of the image. But it's slow because the CLI will do an update check each time before it processes the YAML.
Is there a way to disable the update check?
$ circleci version
0.1.2347+39dfda0
$ circleci
This project is the seed for CircleCI's new command-line application.
Usage:
circleci [command]
Available Commands:
config Operate on build config files
diagnostic Check the status of your CircleCI CLI.
help Help about any command
local Debug jobs on the local machine
namespace Operate on namespaces
orb Operate on orbs
query Query the CircleCI GraphQL API.
setup Setup the CLI with your credentials
update Update the tool
version Display version information
Flags:
-h, --help help for circleci
--host string URL to your CircleCI host (default "https://circleci.com")
--token string your token for using CircleCI
--verbose Enable verbose logging.
Use "circleci [command] --help" for more information about a command.
It looks like circleci tests split
command is missing.
https://circleci.com/docs/2.0/parallelism-faster-jobs/#splitting-test-files
Did you break support of it?
The documented way to pack a config folder into one file is using the output stream of the cli
. Using >
, however, the cli
recently adds Checking for updates...
with special chars to the file which makes it invalid.
I see three possible solutions:
config pack
Moreover, you may add a test case for the config pack that validates the output.
I see that there is a circleci config pack <dir>
command but no documentation on how to use it or what it is for.
My configuration has a store_artifacts
step. I'm wondering how to get the artifacts out when I do a local build. Is it possible?
Uploading /root/deb to root/deb
Uploading /root/deb/blahblah.deb (54 MB): Error: FAILED with error not supported
I am trying out nixorg/nix:circleci
Docker image for Circle CI, locally on macOS, using circleci
CLI tool. However, it fails at the code checkout step with /bin/sh: mkdir: command not found
error.
NOTE: it works fine on Circle CI platform.
$ circleci build
====>> Spin up Environment
Build-agent version 0.0.5895-01cdb92 (2018-05-29T20:07:10+0000)
Starting container nixorg/nix:circleci
using image nixorg/nix@sha256:f3fff0fca5b41722a7c41b88b0879accd6f314307eb408601c1caaa9c1b71e3d
Using build environment variables:
BASH_ENV=/tmp/.bash_env-localbuild-1533558734
CI=true
CIRCLECI=true
CIRCLE_BRANCH=nix
CIRCLE_BUILD_NUM=
CIRCLE_JOB=build
CIRCLE_NODE_INDEX=0
CIRCLE_NODE_TOTAL=1
[email protected]:owner/repo.git
CIRCLE_SHA1=dcb3199a9bf730c351918d8f2322301926bdccba
CIRCLE_SHELL_ENV=/tmp/.bash_env-localbuild-1533558734
CIRCLE_WORKING_DIRECTORY=~/project
====>> Checkout code
#!/bin/sh
mkdir -p /root/project && cp -r /tmp/_circleci_local_build_repo/. /root/project
/bin/sh: mkdir: command not found
Error: Exited with code 127
Step failed
Task failed
Circle CI config:
# .circleci/config.yml
version: 2
jobs:
build:
docker:
- image: nixorg/nix:circleci
steps:
- checkout
- run:
command: nix-build
workflows:
version: 2
build_and_release:
jobs:
- build
Tried with these circleci
versions:
$ brew install circleci
$ circleci --version
0.1.0
$ curl -o /usr/local/bin/circleci https://circle-downloads.s3.amazonaws.com/releases/build_agent_wrapper/circleci && chmod +x /usr/local/bin/circleci
$ circleci --version
0.0.5894-01cdb92
Documentation for build
says:
circleci build --help
Run a full build locally
Usage:
circleci build [flags]
Flags:
...
--skip-checkout use local path as-is (default true)
...
However it appears that the tool still tries to copy files over when doing
#!/bin/sh
mkdir -p /root/project && cp -r /tmp/_circleci_local_build_repo/. /root/project
I would appreciate any help with this, thanks!
Hi,
sometimes I need to trigger a new build for the last commit on a particular project, it would be great to do this with the CLI instead to use the GUI or to maintain an ugly curl script.
Thanks.
When a job is listed in a workflow but missing from the jobs section, the config validator will report the config as valid but will fail immediately when pushed to CircleCI:
* Cannot find a job named `build` to run in the `jobs:` section of your configuration file.
The validator should ensure that all jobs listed in workflows are actually defined.
I'm working on a CircleCI config generator (amongst other things) so I want to test that:
circleci config validate
)circleci local execute --job myjob
)From within CircleCI, so essentially I'm running those CircleCI commands in CircleCI. Very meta but it sounds like it should work.
First command (config validation) works fine.
I get the following error when running circleci local execute --job myjob
(within a CircleCI job):
$ circleci local execute --job myjob
Docker image digest: sha256:0ae15d9b8329446dd5d241b1d654774871caa89c1cc47d513063c23037aae917
Error: failed to start event processor: failed to compute task config: failed to read config file: open circle.yml: no such file or directory
It's confusing because I'm using .circleci/config.yml
and not using circle.yml
.
The same command runs fine locally.
Running version 0.1.4211+14dfd68
--config
argument ❌chmod 777 .circleci/config.yml
❌Here's the smallest generator config file I've been able to reproduce teh bug with: https://gist.github.com/n6g7/f11008ee351b3dfd9c56f6c23b5ce7d6, I don't think the generated config file matters but can share it too if necessary.
Any idea where this could be coming from?
According to the CircleCI's document, development orbs can be tagged with the format like dev:<string>
, and the following tag is a valid:
"dev:myinitials/mybranch"
However, if you try to publish an orb using the above tag, following error message will be shown:
$ circleci orb publish src/orb.yml hoge/fuga@dev:myinitials/mybranch
Error: Invalid orb reference 'hoge/fuga@dev:myinitials/mybranch': Expected a namespace, orb and version in the format 'namespace/orb@version'
Consider you make CircleCI to automatically publish the development orb tagged with a git branch name. If you create a branch named feature/prepare-v1.0.0
, publishing will be failed because the branch name contains the character /
.
macOS 10.14.1
$ circleci version
0.1.4211+14dfd68
System Configuration Information:
Macbook Pro Laptop Mid 2015
Macos High Sierra Version 10.13.5
CircleCI CLI Version:
I am running the command:
circleci build
and getting these errors in the terminal:
Note I got this same error with this version of the local cli that is in the circle ci docs:
$ curl -o /usr/local/bin/circleci https://circle-downloads.s3.amazonaws.com/releases/build_agent_wrapper/circleci && chmod +x /usr/local/bin/circleci
I tried this with the following docker images:
From build.go
:
flags.StringSliceVarP(&args.taskInfo.Volumes, "volume", "v", nil, "Volume bind-mounting")
This setting doesn't seem to be read anywhere.
Hi, I'm trying to use circleci-cli for checking on local, but circleci update
command doesn't work.
OS is MacOS 10.14, homebrew version is 1.8.6.
Error is below... Let me know how to update.
% circleci update check
`update` is not available because this tool was installed using `homebrew`.
Please consult the package manager's documentation on how to update the CLI.
# use sudo
% sudo circleci update install
Password:
`update` is not available because this tool was installed using `homebrew`.
Please consult the package manager's documentation on how to update the CLI.
Installation command
% brew install circleci
% circleci version
0.1.4786+bad101f
I think it is useless to check for new circleci-cli
version each time we run the Docker image. I am not even sure that it makes sense to verify circleci-cli
version at all inside Docker.
Something as basic as circleci config validate
now requires network connectivity to work. I believe this is a change from the Local CLI which did not.
Perhaps we should document this somewhere?
Running docker on windows:
> docker version
Client:
Version: 18.06.1-ce
API version: 1.38
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:21:34 2018
OS/Arch: windows/amd64
Experimental: false
Server:
Engine:
Version: 18.06.1-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:29:02 2018
OS/Arch: linux/amd64
Experimental: true
Attempting to build locally results in:
There ins't any extra information when running it with the --verbose
flag.
Docker image digest: sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08
Error: failed to execute docker: not supported by windows
Which would make sense given that "--volume", "/var/run/docker.sock:/var/run/docker.sock"
wouldn't work on windows.
After some googlefoo -
-v //var/run/docker.sock:/var/run/docker.sock
seems to work from windows
But I still have issues getting the docker to build by manually running it for now with:
docker run --interactive --tty --rm ^
-v //var/run/docker.sock:/var/run/docker.sock ^
-v %cd%:/root/.circleci ^
--workdir /%cd%\ ^
circleci/picard circleci build -c /root/.circleci/config.yml
Error: error starting container <>: Error response from daemon: invalid mode: /tmp/_circleci_local_build_repo
Error:
Unexpected environment preparation error: Error response from daemon: invalid mode: /tmp/_circleci_local_build_repo
Step failed
Task failed
Tried using the install script and it does not work, as is. I believe the issue is the due to the formatting of the release.json
file that is downloaded. The JSON text is all on one line, which causes issues with the grep
commands as they return the entire JSON object instead of individual lines.
I have updated the script to fix this issue and will create a PR today.
Before:
Finding latest release.
Downloading CircleCI ## Changelog\n\n1f1866a Merge pull request #84 from CircleCI-Public/use-latest-codecov-orb\n229aaf5 Use released 0.0.4 codecov-clojure orb\n\n
An error occured installing the tool.
After:
Finding latest release.
Downloading CircleCI v0.1.1789
tried update circleci-cli, but can't.
Error message is this.
[]:~/ circleci update
Error: error finding latest release: GET https://api.github.com/repos/CircleCI-Public/circleci-cli/releases: 403 API rate limit exceeded for XXX.XXX.XXX.XXX. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 27m17s]
This is GitHub API Limit, but if circleci cli can set GitHub personal token, I can avoid this error.
I want to can set GitHub token option like --github-token
circleci version
0.1.3341+6be51e4
circleci update check
A new release is available (0.1.4427)
You are running 0.1.3341
You can update with `circleci update install`
circleci update install
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x127d962]
goroutine 1 [running]:
github.com/CircleCI-Public/circleci-cli/logger.(*Logger).Debug(0x0, 0x1539208, 0x12, 0xc420143b60, 0x1, 0x1)
/go/src/github.com/CircleCI-Public/circleci-cli/logger/logger.go:37 +0x22
github.com/CircleCI-Public/circleci-cli/cmd.update(0xc420155340, 0x0, 0x18450f0, 0x0, 0x0, 0xc420143c00, 0x12e4654, 0xc420133100)
/go/src/github.com/CircleCI-Public/circleci-cli/cmd/update.go:149 +0x228
github.com/CircleCI-Public/circleci-cli/cmd.installUpdate(0xc420155340, 0x0, 0x18450f0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/CircleCI-Public/circleci-cli/cmd/update.go:124 +0x4d
github.com/CircleCI-Public/circleci-cli/cmd.newUpdateCommand.func4(0xc42017ec80, 0x18450f0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/CircleCI-Public/circleci-cli/cmd/update.go:55 +0x49
github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra.(*Command).execute(0xc42017ec80, 0x18450f0, 0x0, 0x0, 0xc42017ec80, 0x18450f0)
/go/src/github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra/command.go:762 +0x468
github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc420160500, 0xc420160780, 0xc420155360, 0xc420155350)
/go/src/github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra/command.go:852 +0x30a
github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra.(*Command).Execute(0xc420160500, 0x0, 0xc420093740)
/go/src/github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra/command.go:800 +0x2b
github.com/CircleCI-Public/circleci-cli/cmd.Execute()
/go/src/github.com/CircleCI-Public/circleci-cli/cmd/root.go:26 +0x27
main.main()
/go/src/github.com/CircleCI-Public/circleci-cli/main.go:9 +0x20
It would be very helpful when troubleshooting with CLI users to know when it was installed via snap
or brew
. In fact, the OS as well.
I figure this information could be:
circleci version
circleci version --debug
circleci diagnostic
circleci about
I'd like to see a line that shows something like OS: <OS> <OS version>
where would be macOS, Ubuntu, Fedora, etc and then OS version number. If we really needed to keep the logic to a minimum, the OS distinguishing between darwin
and linux
would be enough.
Then a line such as Installation method: <method>
where could be binary, snap, or brew.
When troubleshooting, this allows us to tell users to copy & paste the output of whichever command we use and have the relevant information easily, such as with circleci version --debug | xsel -b
.
Related to #152
When executing circleci local execute
or circleci build
I get stuck on the step Checkout code.
In verbose
mode this shows up:
…
DEBU[2018-10-24T09:35:41Z] Restarting ListTaskSignals stream
INFO[2018-10-24T09:35:41Z] checking task signals
DEBU[2018-10-24T09:35:41Z] EOF in ListTaskSignals stream
DEBU[2018-10-24T09:35:51Z] Restarting ListTaskSignals stream
INFO[2018-10-24T09:35:51Z] checking task signals
DEBU[2018-10-24T09:35:51Z] EOF in ListTaskSignals stream
DEBU[2018-10-24T09:36:01Z] Restarting ListTaskSignals stream
INFO[2018-10-24T09:36:01Z] checking task signals
DEBU[2018-10-24T09:36:01Z] EOF in ListTaskSignals stream
… (Until I exit the process)
I think it'll be useful to add a create subcommand to config that allows the user to create new configs from scratch. Here's examples of how this might be use:
circleci config create [flags]
Flags:
--executor string Can be 'docker', 'machine', or 'macos'. Defaults to 'docker'.
--template string Path to a template file to be used when creating config.
```
I would image running `circleci config create` with defaults would:
1. Create the `./circleci` directory if it doesn't exists.
2. Create the file ./circleci/config.yml` if it doesn't exists.
3. Create a starting config like below:
```
version: 2.1
workflows:
main:
jobs:
- build
jobs:
build:
docker:
- image: cibuilds/base:2018.10
steps:
- checkout
- run:
name: "First Step"
command: echo "Run a Bash command."
```
When adding a new project to CircleCI, the webapp shows a config (very verbose though) that the user could use for that project based on language. In addition to or instead of `--template`, we could have a language keyboard that pulls that same config for the user to use it as a staring point.
I'm on MacOS 10.13 and the quick install instructions seem to be failing for me. I've run:
bash -c "$(curl -fSl https://raw.githubusercontent.com/CircleCI-Public/circleci-cli/master/install.sh)"
and
sudo bash -c "$(curl -fSl https://raw.githubusercontent.com/CircleCI-Public/circleci-cli/master/install.sh)"
Both result in this output:
Finding latest release.
An error occured installing the tool.
The contents of the directory /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.OEVuIyT7 have been left in place to help to debug the issue.
The folder it's saying has been left behind doesn't exist when trying to inspect it afterwards. Is this an issue with my system, or reproducible?
On version 0.1.3469 (the latest as of this posting):
$ circleci local execute --help
Usage of configFlags:
-c, --config string config file (default ".circleci/config.yml")
Error: Unable to parse flags
Hello!
As a means of exploring with optimizations for our workflow pipeline, I've recently started using the circleci
CLI locally. I ran into a problem when testing the circleci tests split
command.
~/workspace/project (branch-name *) $ circleci tests glob "{spec,engines}/**/*_spec.rb" | circleci tests split --split-by=timings
Error: Please ensure that circleci-agent is installed, expected this to be called inside a job: exec: "circleci-agent": executable file not found in $PATH
Where can I download the circleci-agent
? And is there really a strict necessity for the additional binary?
Running circleci config validate
with the new CLI installed via snap
no longer works. Returns the error:
Error: Could not load config file at .circleci/config.yml: open .circleci/config.yml: permission denied
$> circleci version
0.1.3341+6be51e4
(I know it's not the latest version, trying to update the normal way instead of reinstalling latest version over it is what this issue is about.)
$> circleci update check
A new release is available (0.1.3923)
You are running 0.1.3341
You can update with `circleci update install`
$> sudo circleci update install
[sudo] password for karfau:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x67a122]
goroutine 1 [running]:
github.com/CircleCI-Public/circleci-cli/logger.(*Logger).Debug(0x0, 0x93792b, 0x12, 0xc420135b60, 0x1, 0x1)
/go/src/github.com/CircleCI-Public/circleci-cli/logger/logger.go:37 +0x22
github.com/CircleCI-Public/circleci-cli/cmd.update(0xc420147340, 0x0, 0xc437a0, 0x0, 0x0, 0xc420135c00, 0x6e0e14, 0xc420125100)
/go/src/github.com/CircleCI-Public/circleci-cli/cmd/update.go:149 +0x228
github.com/CircleCI-Public/circleci-cli/cmd.installUpdate(0xc420147340, 0x0, 0xc437a0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/CircleCI-Public/circleci-cli/cmd/update.go:124 +0x4d
github.com/CircleCI-Public/circleci-cli/cmd.newUpdateCommand.func4(0xc420164c80, 0xc437a0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/CircleCI-Public/circleci-cli/cmd/update.go:55 +0x49
github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra.(*Command).execute(0xc420164c80, 0xc437a0, 0x0, 0x0, 0xc420164c80, 0xc437a0)
/go/src/github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra/command.go:762 +0x468
github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc420152500, 0xc420152780, 0xc420147360, 0xc420147350)
/go/src/github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra/command.go:852 +0x30a
github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra.(*Command).Execute(0xc420152500, 0x0, 0xc42001d770)
/go/src/github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra/command.go:800 +0x2b
github.com/CircleCI-Public/circleci-cli/cmd.Execute()
/go/src/github.com/CircleCI-Public/circleci-cli/cmd/root.go:26 +0x27
main.main()
/go/src/github.com/CircleCI-Public/circleci-cli/main.go:9 +0x20
(running with or without sudo
makes no difference)
$ uname -a
Linux karfau-manjaro 4.18.16-2-MANJARO #1 SMP PREEMPT Thu Nov 1 15:47:06 UTC 2018 x86_64 GNU/Linux
If more details are needed to understand/fix the issue please let me know
I have the following config file:
version: 2
jobs:
build:
docker:
- image: cibuilds/hugo:0.46
docker:jj:
steps:
- checkout
- run:
name: "Build Website With Hugo"
command: echo "Hi"
There's at least two errors here. Line 3 has 3 spaces and line 4 is an invalid key as well as doesn't have a value. This file validates.
CLI version: 0.1.1640+4a9a3c9
Installed method: cURL
Build-agent version: The CLI doesn't say
Technically this is valid YAML, but not a CircleCI config. I guess we haven't gotten that far yet?
I am using the latest version of CLI tool
etc :: ~/projects/scio » circleci version
0.1.3667+073b0ec
And validate command is perfectly fine with my config
etc :: ~/projects/scio » circleci config validate
Config file at .circleci/config.yml is valid.
But once I try to spin local container, provision fails.
etc :: ~/projects/scio » circleci local execute
Docker image digest: sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08
====>> Spin up Environment
Build-agent version 0.1.750-336e0196 (2018-09-13T19:48:06+0000)
Error: Configuration errors: 1 error occurred:
* Cannot find a job named `build` to run in the `jobs:` section of your configuration file.
If you expected a workflow to run, check your config contains a top-level key called 'workflows:'
Step failed
Task failed
Even though it should not:
etc :: ~/projects/scio » grep "workflows" -A 17 .circleci/config.yml
workflows:
version: 2
# Per commit build
build:
jobs:
- build_211
- build_212
- deploy:
requires:
- build_211
- build_212
filters:
tags:
only: /^v.*/
branches:
ignore: /.*/
As far as I can tell, the problem is on the picard side, which is closed sourced. How can I see the piece of code where this parsing and validation happens to somehow contribute the fix? Any other directions?
It'd be handy to be able to install this via Homebrew, if possible.
cans@ws:~/src/circleci-cli$ bash -c "$(curl -fSl https://raw.githubusercontent.com/CircleCI-Public/circleci-cli/master/install.sh)"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1216 100 1216 0 0 8386 0 --:--:-- --:--:-- --:--:-- 8386
Finding latest release.
Downloading CircleCI v0.1.3139
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 631 0 631 0 0 1561 0 --:--:-- --:--:-- --:--:-- 1558
100 3259k 100 3259k 0 0 1221k 0 0:00:02 0:00:02 --:--:-- 2096k
circleci-cli_0.1.3139_linux_amd64/client/LICENSE
circleci-cli_0.1.3139_linux_amd64/md_docs/LICENSE
circleci-cli_0.1.3139_linux_amd64/circleci
Installing to /usr/local/bin/circleci
mv: impossible de créer le fichier standard '/usr/local/bin/circleci': Permission non accordée
An error occured installing the tool.
The contents of the directory /tmp/tmp.RhbLUmMxEx have been left in place to help to debug the issue.
cans@ws:~/src/circleci-cli$ ls /tmp/tmp.RhbLUmMxEx
ls: impossible d'accéder à '/tmp/tmp.RhbLUmMxEx': Aucun fichier ou dossier de ce type
I belive this is because you use set -o errexit
and use a trap on EXIT
to clean the directory.
As a side note, though no 100% sure, I also noticed the following: I am not sure it makes sense to use set -o pipefail
and set -o errexit
together. Indeed former asks to "keep going (but still report the error)" and the latter to "not keep going". Again, I might be wrong about that.
(This is somewhat related to a support request I've already submitted, but thought this would be useful for visibility)
If I have a .circleci/config.yml
file with an emoji in it, the circleci config validate
check says it's valid, but it's never picked up by CircleCI, as something is silently failing.
Emoji may not be supposed by all yaml parsers, for example in Python:
>>> import yaml
>>> with open(".circleci/config.yml") as f:
... y = yaml.load(f)
...
yaml.reader.ReaderError: unacceptable character #x1f527: special characters are not allowed
However, I assume this golang CLI does yaml parsing, and thinks the config is valid.
Somewhere in the pipeline this is silently failing. That is: the build is never picked up by CircleCI
Basic issue reproduction: https://github.com/glasnt/verbose-journey/commits/master (note the change made in f3821380 and how there's no CI status badge on the commit)
I'm not a golang dev, but I'd suggest:
validate
fail if emoji are spotted (not a complete workaround, as validate
doesn't block building, but it might be usefulWe currently store user API tokens in a dotfile.
We should store them in the system provided storage for such items, such as the Keychain on macOS.
This library looks promising: https://github.com/zalando/go-keyring
There are many instances of fmt.Pritnf()
that is missing a trailing newline, which causes the output to incorrectly run together. For example, circleci namespace create
:
Namespace `...` created.Please note that any orbs you publish in this namespace are open orbs and are world-readable.
Most print statements that do not contain formatting directives use Println
and there are Printf
statements that do end in \n
, so it appears that all of the instances in which this is missing are simply errors/bugs.
I created a Discord channel that I think might be helpful in communication, what do we all think?
Following on from this thread.
When running locally (eg circleci local execute --job build
), all files in the current working directory are copied over to the docker container, causing errors that don't occur when run on circleci.com.
(note: a reproducible case is available at jesstelford/local-circleci-test#1)
My project is written in nodejs, and has a dependency on the bcrypt
package:
package.json
{
"name": "test-project",
"version": "1.0.0",
"main": "index.js",
"license": "MIT",
"dependencies": {
"bcrypt": "^3.0.1"
}
}
The bcrypt
package gets installed into node_modules/bcrypt
, so we add it to our .gitignore
:
node_modules
Then we setup a job to run on circleci:
.circleci/config.yml
version: 2
jobs:
tests:
docker:
# specify the version you desire here
- image: circleci/node:carbon-browsers
working_directory: /home/circleci/repo
steps:
- checkout
- run: yarn
- run: node -p "require('bcrypt').hashSync('password', 5)"
Then when we commit these 3 files, and run it on CircleCI.com, everything runs fine.
But if we try to run the following commands on an OSX machine, there are errors:
# creates the `node_modules` directory which is incorrectly copied to the docker image
yarn
# run the local circleci command
circleci local execute --job build
❯ circleci local execute --job build
Docker image digest: sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08
====>> Spin up Environment
Build-agent version 0.1.750-336e0196 (2018-09-13T19:48:06+0000)
Starting container circleci/node:carbon-browsers
using image circleci/node@sha256:e116faa71d34d76287d6b2a81abeeadd5897f508e14b41391b29ed882c820022
Using build environment variables:
BASH_ENV=/tmp/.bash_env-localbuild-1538887853
CI=true
CIRCLECI=true
CIRCLE_BRANCH=the-error
CIRCLE_BUILD_NUM=
CIRCLE_JOB=build
CIRCLE_NODE_INDEX=0
CIRCLE_NODE_TOTAL=1
[email protected]:jesstelford/local-circleci-test.git
CIRCLE_SHA1=bbde846be5368499be3eb79f3154d4c017ab9122
CIRCLE_SHELL_ENV=/tmp/.bash_env-localbuild-1538887853
CIRCLE_WORKING_DIRECTORY=/home/circleci/repo
====>> Checkout code
#!/bin/bash -eo pipefail
mkdir -p /home/circleci/repo && cp -r /tmp/_circleci_local_build_repo/. /home/circleci/repo
====>> yarn
#!/bin/bash -eo pipefail
yarn
yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 2.34s.
====>> node -p "require('bcrypt').hashSync('password', 5)"
#!/bin/bash -eo pipefail
node -p "require('bcrypt').hashSync('password', 5)"
module.js:682
return process.dlopen(module, path._makeLong(filename));
^
Error: /home/circleci/repo/node_modules/bcrypt/lib/binding/bcrypt_lib.node: invalid ELF header
at Object.Module._extensions..node (module.js:682:18)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/circleci/repo/node_modules/bcrypt/bcrypt.js:6:16)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
Error: Exited with code 1
Step failed
Error: runner failed
Task failed
However, if you remove node_modules
, everything runs fine:
# clean up files that aren't meant to be copied to the docker image
rm -rf node_modules
# run the local circleci command
circleci local execute --job build
❯ circleci local execute --job build
Docker image digest: sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08
====>> Spin up Environment
Build-agent version 0.1.750-336e0196 (2018-09-13T19:48:06+0000)
Starting container circleci/node:carbon-browsers
using image circleci/node@sha256:e116faa71d34d76287d6b2a81abeeadd5897f508e14b41391b29ed882c820022
Using build environment variables:
BASH_ENV=/tmp/.bash_env-localbuild-1538887959
CI=true
CIRCLECI=true
CIRCLE_BRANCH=the-error
CIRCLE_BUILD_NUM=
CIRCLE_JOB=build
CIRCLE_NODE_INDEX=0
CIRCLE_NODE_TOTAL=1
[email protected]:jesstelford/local-circleci-test.git
CIRCLE_SHA1=bbde846be5368499be3eb79f3154d4c017ab9122
CIRCLE_SHELL_ENV=/tmp/.bash_env-localbuild-1538887959
CIRCLE_WORKING_DIRECTORY=/home/circleci/repo
====>> Checkout code
#!/bin/bash -eo pipefail
mkdir -p /home/circleci/repo && cp -r /tmp/_circleci_local_build_repo/. /home/circleci/repo
====>> yarn
#!/bin/bash -eo pipefail
yarn
yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 5.15s.
====>> node -p "require('bcrypt').hashSync('password', 5)"
#!/bin/bash -eo pipefail
node -p "require('bcrypt').hashSync('password', 5)"
$2b$05$XhiGPRaOV607/aGp9Th08OdyJPKFVQztuqWREyjTk4IYTPX1W5XXu
Success!
This is because:
bcrypt
package installs OS-specific binaries into node_modules
circleci-cli
then copies over node_modules
to the docker image (ignoring my .gitignore
)bcrypt
executable is run on a linux docker imageinvalid ELF header
error.Today, I noticed that recently, unexpected line breaks occur with long commands.
This is a shortened example from the circleci orb source
of a dev orb I'm working on:
steps:
- run:
command: |
circleci orb publish << parameters.orb-path >> << parameters.orb-ref >> <<# parameters.publish-token-variable >>--token << parameters.publish-token-variable >> <</ parameters.publish-token-variable >>
Here the config-expanded version from the online console:
circleci orb publish ./.orbspace/orb.yml ${ORB_PUBLISHING_NAMESPACE}/${ORB_PUBLISHING_NAME}@dev:${CIRCLE_BUILD_NUM}
--token ${ORB_PUBLISHING_TOKEN}
Notice the unexpected new line break before --token
.
The confusing thing about this issue is that the same orb source code works from a previous orb version. That only makes sense to me if you choose the cli version depending on the orb version, otherwise I'm confused why the same config, same orb source and same command should be parsed differently just because the version is newer.
This is very similar to the bug on the UI where any newly configured job automatically opts one into an experimental 2.1 config feature set. This is not desired or expected.
CircleCI's cli version is 0.1.1430+9788d75
One can use these dummy config to reproduce by running circleci config validate
# generic v2.x config with filter
version: 2
jobs:
build:
docker:
- image: circleci/python:3.6
branches:
only:
- master
steps:
- checkout
- run:
name: Test
command: |
echo "this is just a test"
Unexpected output:
Error: Job "build" has filters configured in the job definition. These filters are incompatible with workflows.
# generic v2.x config with filter
version: 2
jobs:
build:
docker:
- image: circleci/python:3.6
# branches:
# only:
# - master
steps:
- checkout
- run:
name: Test
command: |
echo "this is just a test"
Expected output:
Error: Job "build" has filters configured in the job definition. These filters are incompatible with workflows.
# explicitly use v2.0 config with filter
version: 2.0
jobs:
build:
docker:
- image: circleci/python:3.6
branches:
only:
- master
steps:
- checkout
- run:
name: Test
command: |
echo "this is just a test"
Expected output:
Error: Job "build" has filters configured in the job definition. These filters are incompatible with workflows.
# explicitly use v2.1 config with filter
version: 2.1
jobs:
build:
docker:
- image: circleci/python:3.6
branches:
only:
- master
steps:
- checkout
- run:
name: Test
command: |
echo "this is just a test"
Expected output:
Error: jobs: build: extraneous key [branches] is not permitted
Previous releses such as v0.1.4005 always included a linux asset eg:
circleci-cli_0.1.4005_linux_amd64.tar.gz
The latest release is missing the linux asset, the install.sh
also fails because of this on linux:
curl -fLSs https://circle.ci/cli | bash
Installing CircleCI CLI
Finding latest release.
Downloading CircleCI v0.1.4029
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
An error occured installing the tool.
The contents of the directory /tmp/tmp.Hx8hnDm6Xc have been left in place to help to debug the issue.
The asset should be added to that release if this was not intended. Otherwise the installation process should be adapted to support the situation properly.
I just downloaded circleci from the usual:
curl https://raw.githubusercontent.com/CircleCI-Public/circleci-cli/master/install.sh --fail --show-error | bash
It reports version:
$ circleci version
0.1.1568+255077c
Possible aside: I noted that circleci switch
does nothing.
When I run something locally with the config-preview-sdk (i.e. version: 2.1
) it errors with config parsing issues that indicate that it's not using the preview SDK e.g.
version: 2.1
...
jobs:
deploy:
executor:
name: some-custom-executor
...
* Error in config file: The schema/shape of the YAML is incorrect: json: cannot unmarshal object into Go struct field JobDescription.Executor of type string
I seem to be missing something, and guidance would be appreciated.
CLI Version 0.1.3277+950a73e
Error:
Error: orbs: deploy: -schedulerer0 subschemas matched instead of one
orbs: deploy: -schedulererexpected type: String, found: Mapping
orbs: deploy: -schedulerercommands: deploy: parameters: name: 2 schema violations found
orbs: deploy: -schedulerercommands: deploy: parameters: name: required key [type] not found
orbs: deploy: -schedulerercommands: deploy: parameters: name: extraneous key [cluster] is not permitted
Config sample:
orbs:
deploy-schedulerer:
...
It seems that somehow the word deploy-schedulerer
is being split.
Changing the config to deploy_scheduler
made the errors more readable. I'm still the other errors.
I'm trying to see if I can use this version of the circleci binary to get test parallelization working with remote docker but my default ci container is based on alpine linux. Go makes it pretty easy to generate a static binary which would make it far more portable.
When installing install.sh
, the install location is hard coded to /usr/local/bin
. Users should be able to override this location.
Perhaps specifying an argument to the script, or by setting an env var before executing it.
I've tried to pull the required docker image with $ docker pull {image}
and that was fine. However, trying to $ circleci local execute
leads to such error. There is no more info about it, even in verbose mode.
What I also tried to do: $ sudo circleci local execute
because docker
itself runs only under sudo
. However I got this very strange error: Error: Unable to read config file: open .circleci/config.yml: permission denied
, but that folder and file have all permissions for all users. I think I missed something because of weak Linux skill maybe. But it can be also a problem on CircleCI CLI side. Would appreciate any help, thank you.
When running circleci local execute
in a python project, I got an access denied error bellow:
====>> Checkout code
#!/bin/bash -eo pipefail
mkdir -p /home/circleci/alpe-skytef-gw && cp -r /tmp/_circleci_local_build_repo/. /home/circleci/alpe-skytef-gw
cp: cannot open '/tmp/_circleci_local_build_repo/./tests/__pycache__/test_inicializacao.cpython-36-PYTEST.pyc' for reading: Permission denied
Error: Exited with code 1
Step failed
Error: runner failed
Task failed
The thing is, when I clean all __pycache__
dirs, it runs flawlessly.
Attached is the config.yml file, and the full error output running in verbose...
error_report.txt
config.txt
We need development instructions for outside contributors and new CircleCI employees on how to config and build the CLI.
Can be in the readme, or if very verbose, it's own dev Markdown file.
[ben@localhost directory]$ circleci-cli local execute
Downloading latest CircleCI build agent...
Docker image digest: sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08: Pulling from docker.io/circleci/picard
/usr/bin/docker-current: Error parsing reference: "circleci/picard@sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08: Pulling from docker.io/circleci/picard" is not a valid repository/tag: invalid reference format.
See '/usr/bin/docker-current run --help'.
That sha appears to sit in ~/.circleci/build_agent_settings.json
. Here is the contents of that file:
{"LatestSha256":"sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08: Pulling from docker.io/circleci/picard"}
Tried with the latest release version, as well as built from master just now.
Relevant other links on the internet:
https://discuss.circleci.com/t/circleci-cli-circleci-picard-sha256-is-not-a-valid-repository-tag/24807/3
https://discuss.circleci.com/t/circleci-cli-on-linux-system-issues/18584
I recently installed the CLI it seems like my machine doesn't natively trust github (might be due to certificate validation that my organization does), so when I run circleci update check
the following error occurs:
Error: error finding latest release: Get https://api.github.com/repos/CircleCI-Public/circleci-cli/releases: x509: certificate signed by unknown authority
On a semi-related note, when I run circleci config validate <my_config.yml>
it fails with:
Error: Unable to validate config: Something unexpected happened.
Would anyone have any insight for solving this?
Without Docker running the local execute
command fails with a rather unhelpful error.
$ circleci local execute
Downloading latest CircleCI build agent...
Error: Could not find picard image: failed to pull latest docker image: exit status 1
Starting Docker fixes it, but I had to read the source before I figured that out 😅
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.