Something as basic as circleci config validate now requires network connectivity to work. I believe this is a change from the Local CLI which did not.

Perhaps we should document this somewhere?

Without Docker running the local execute command fails with a rather unhelpful error.

$ circleci local execute
Downloading latest CircleCI build agent...
Error: Could not find picard image: failed to pull latest docker image: exit status 1

Starting Docker fixes it, but I had to read the source before I figured that out 😅

Running circleci config validate with the new CLI installed via snap no longer works. Returns the error:

Error: Could not load config file at .circleci/config.yml: open .circleci/config.yml: permission denied

Hello!

As a means of exploring with optimizations for our workflow pipeline, I've recently started using the circleci CLI locally. I ran into a problem when testing the circleci tests split command.

~/workspace/project (branch-name *) $ circleci tests glob "{spec,engines}/**/*_spec.rb" | circleci tests split --split-by=timings
Error: Please ensure that circleci-agent is installed, expected this to be called inside a job: exec: "circleci-agent": executable file not found in $PATH

Where can I download the circleci-agent? And is there really a strict necessity for the additional binary?

I created a Discord channel that I think might be helpful in communication, what do we all think?

It would be very helpful when troubleshooting with CLI users to know when it was installed via snap or brew. In fact, the OS as well.

I figure this information could be:

1. added to the output of the version command circleci version
2. added as a flag to the version command circleci version --debug
3. added to the diagnostic command circleci diagnostic
4. added to its own command circleci about

I'd like to see a line that shows something like OS: <OS> <OS version> where would be macOS, Ubuntu, Fedora, etc and then OS version number. If we really needed to keep the logic to a minimum, the OS distinguishing between darwin and linux would be enough.

Then a line such as Installation method: <method> where could be binary, snap, or brew.

When troubleshooting, this allows us to tell users to copy & paste the output of whichever command we use and have the relevant information easily, such as with circleci version --debug | xsel -b.

I have the following config file:

version: 2
jobs:
   build:
    docker:
      - image: cibuilds/hugo:0.46
    docker:jj:
    steps:
      - checkout
      - run:
          name: "Build Website With  Hugo"
          command: echo "Hi"

There's at least two errors here. Line 3 has 3 spaces and line 4 is an invalid key as well as doesn't have a value. This file validates.

CLI version: 0.1.1640+4a9a3c9
Installed method: cURL
Build-agent version: The CLI doesn't say

Technically this is valid YAML, but not a CircleCI config. I guess we haven't gotten that far yet?

System Configuration Information:

Macbook Pro Laptop Mid 2015
Macos High Sierra Version 10.13.5

CircleCI CLI Version:

Docker Version:

I am running the command:

circleci build and getting these errors in the terminal:

Note I got this same error with this version of the local cli that is in the circle ci docs:

$ curl -o /usr/local/bin/circleci https://circle-downloads.s3.amazonaws.com/releases/build_agent_wrapper/circleci && chmod +x /usr/local/bin/circleci

I tried this with the following docker images:

• circleci/golang:1.9
• circleci/node:7.10

Previous releses such as v0.1.4005 always included a linux asset eg:
circleci-cli_0.1.4005_linux_amd64.tar.gz

The latest release is missing the linux asset, the install.sh also fails because of this on linux:

curl -fLSs https://circle.ci/cli | bash
Installing CircleCI CLI
Finding latest release.
Downloading CircleCI v0.1.4029
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
An error occured installing the tool.
The contents of the directory /tmp/tmp.Hx8hnDm6Xc have been left in place to help to debug the issue.

The asset should be added to that release if this was not intended. Otherwise the installation process should be adapted to support the situation properly.

I'm on MacOS 10.13 and the quick install instructions seem to be failing for me. I've run:

bash -c "$(curl -fSl https://raw.githubusercontent.com/CircleCI-Public/circleci-cli/master/install.sh)"
and
sudo bash -c "$(curl -fSl https://raw.githubusercontent.com/CircleCI-Public/circleci-cli/master/install.sh)"

Both result in this output:

Finding latest release.
An error occured installing the tool.
The contents of the directory /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/tmp.OEVuIyT7 have been left in place to help to debug the issue.

The folder it's saying has been left behind doesn't exist when trying to inspect it afterwards. Is this an issue with my system, or reproducible?

I recently installed the CLI it seems like my machine doesn't natively trust github (might be due to certificate validation that my organization does), so when I run circleci update check the following error occurs:
Error: error finding latest release: Get https://api.github.com/repos/CircleCI-Public/circleci-cli/releases: x509: certificate signed by unknown authority

On a semi-related note, when I run circleci config validate <my_config.yml> it fails with:
Error: Unable to validate config: Something unexpected happened.

Would anyone have any insight for solving this?

My configuration has a store_artifacts step. I'm wondering how to get the artifacts out when I do a local build. Is it possible?

Uploading /root/deb to root/deb
Uploading /root/deb/blahblah.deb (54 MB): Error: FAILED with error not supported

I am using version 0.1.3257+b197d90 of the CLI.

I have a parameter called 'myparam'. When I use this parameter, and ran validation, the following happened:

1. echo << myparam >> # Validated ok
2. echo << notmyparam >> # Validated with error
3. echo << parameters.myparam >> # Validated ok
4. echo << parameters.notmyparam >> # Validated with error

The only alternative that actually works is number 3. Number 1 will not work. When I ran 'circleci config process', it was not able to find the parameter.

Should number 1 result in a validation error, or should 'circleci config process' be able to find the parameter?

Btw, when I check in number 1, the circleci.com processing result in the same as the CLI processing.

Hi,

sometimes I need to trigger a new build for the last commit on a particular project, it would be great to do this with the CLI instead to use the GUI or to maintain an ugly curl script.

Thanks.

I see that there is a circleci config pack <dir> command but no documentation on how to use it or what it is for.

Related to #152

When executing circleci local execute or circleci build I get stuck on the step Checkout code.

In verbose mode this shows up:

…
DEBU[2018-10-24T09:35:41Z] Restarting ListTaskSignals stream
INFO[2018-10-24T09:35:41Z] checking task signals
DEBU[2018-10-24T09:35:41Z] EOF in ListTaskSignals stream
DEBU[2018-10-24T09:35:51Z] Restarting ListTaskSignals stream
INFO[2018-10-24T09:35:51Z] checking task signals
DEBU[2018-10-24T09:35:51Z] EOF in ListTaskSignals stream
DEBU[2018-10-24T09:36:01Z] Restarting ListTaskSignals stream
INFO[2018-10-24T09:36:01Z] checking task signals
DEBU[2018-10-24T09:36:01Z] EOF in ListTaskSignals stream
… (Until I exit the process)

Following on from this thread.

When running locally (eg circleci local execute --job build), all files in the current working directory are copied over to the docker container, causing errors that don't occur when run on circleci.com.

Example

(note: a reproducible case is available at jesstelford/local-circleci-test#1)

My project is written in nodejs, and has a dependency on the bcrypt package:

package.json

{
  "name": "test-project",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "bcrypt": "^3.0.1"
  }
}

The bcrypt package gets installed into node_modules/bcrypt, so we add it to our .gitignore:

node_modules

Then we setup a job to run on circleci:

.circleci/config.yml

version: 2

jobs:
  tests:
    docker:
      # specify the version you desire here
      - image: circleci/node:carbon-browsers
    working_directory: /home/circleci/repo
    steps:
      - checkout
      - run: yarn
      - run: node -p "require('bcrypt').hashSync('password', 5)"

Then when we commit these 3 files, and run it on CircleCI.com, everything runs fine.

But if we try to run the following commands on an OSX machine, there are errors:

# creates the `node_modules` directory which is incorrectly copied to the docker image
yarn

# run the local circleci command
circleci local execute --job build

❯ circleci local execute --job build
Docker image digest: sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08
====>> Spin up Environment
Build-agent version 0.1.750-336e0196 (2018-09-13T19:48:06+0000)
Starting container circleci/node:carbon-browsers
  using image circleci/node@sha256:e116faa71d34d76287d6b2a81abeeadd5897f508e14b41391b29ed882c820022
Using build environment variables:
  BASH_ENV=/tmp/.bash_env-localbuild-1538887853
  CI=true
  CIRCLECI=true
  CIRCLE_BRANCH=the-error
  CIRCLE_BUILD_NUM=
  CIRCLE_JOB=build
  CIRCLE_NODE_INDEX=0
  CIRCLE_NODE_TOTAL=1
  [email protected]:jesstelford/local-circleci-test.git
  CIRCLE_SHA1=bbde846be5368499be3eb79f3154d4c017ab9122
  CIRCLE_SHELL_ENV=/tmp/.bash_env-localbuild-1538887853
  CIRCLE_WORKING_DIRECTORY=/home/circleci/repo
====>> Checkout code
  #!/bin/bash -eo pipefail
mkdir -p /home/circleci/repo && cp -r /tmp/_circleci_local_build_repo/. /home/circleci/repo
====>> yarn
  #!/bin/bash -eo pipefail
yarn
yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 2.34s.
====>> node -p "require('bcrypt').hashSync('password', 5)"
  #!/bin/bash -eo pipefail
node -p "require('bcrypt').hashSync('password', 5)"
module.js:682
  return process.dlopen(module, path._makeLong(filename));
                 ^

Error: /home/circleci/repo/node_modules/bcrypt/lib/binding/bcrypt_lib.node: invalid ELF header
    at Object.Module._extensions..node (module.js:682:18)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/circleci/repo/node_modules/bcrypt/bcrypt.js:6:16)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
Error: Exited with code 1
Step failed
Error: runner failed
Task failed

However, if you remove node_modules, everything runs fine:

# clean up files that aren't meant to be copied to the docker image
rm -rf node_modules

# run the local circleci command
circleci local execute --job build

❯ circleci local execute --job build
Docker image digest: sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08
====>> Spin up Environment
Build-agent version 0.1.750-336e0196 (2018-09-13T19:48:06+0000)
Starting container circleci/node:carbon-browsers
  using image circleci/node@sha256:e116faa71d34d76287d6b2a81abeeadd5897f508e14b41391b29ed882c820022
Using build environment variables:
  BASH_ENV=/tmp/.bash_env-localbuild-1538887959
  CI=true
  CIRCLECI=true
  CIRCLE_BRANCH=the-error
  CIRCLE_BUILD_NUM=
  CIRCLE_JOB=build
  CIRCLE_NODE_INDEX=0
  CIRCLE_NODE_TOTAL=1
  [email protected]:jesstelford/local-circleci-test.git
  CIRCLE_SHA1=bbde846be5368499be3eb79f3154d4c017ab9122
  CIRCLE_SHELL_ENV=/tmp/.bash_env-localbuild-1538887959
  CIRCLE_WORKING_DIRECTORY=/home/circleci/repo
====>> Checkout code
  #!/bin/bash -eo pipefail
mkdir -p /home/circleci/repo && cp -r /tmp/_circleci_local_build_repo/. /home/circleci/repo
====>> yarn
  #!/bin/bash -eo pipefail
yarn
yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 5.15s.
====>> node -p "require('bcrypt').hashSync('password', 5)"
  #!/bin/bash -eo pipefail
node -p "require('bcrypt').hashSync('password', 5)"
$2b$05$XhiGPRaOV607/aGp9Th08OdyJPKFVQztuqWREyjTk4IYTPX1W5XXu
Success!

This is because:

• I am running on OSX
• the bcrypt package installs OS-specific binaries into node_modules
• circleci-cli then copies over node_modules to the docker image (ignoring my .gitignore)
• The OSX bcrypt executable is run on a linux docker image
• resulting in the invalid ELF header error.

Problem:

According to the CircleCI's document, development orbs can be tagged with the format like dev:<string>, and the following tag is a valid:

"dev:myinitials/mybranch"

However, if you try to publish an orb using the above tag, following error message will be shown:

$ circleci orb publish src/orb.yml hoge/fuga@dev:myinitials/mybranch
Error: Invalid orb reference 'hoge/fuga@dev:myinitials/mybranch': Expected a namespace, orb and version in the format 'namespace/orb@version'

Will be in trouble when:

Consider you make CircleCI to automatically publish the development orb tagged with a git branch name. If you create a branch named feature/prepare-v1.0.0, publishing will be failed because the branch name contains the character /.

Environment:

macOS 10.14.1

$ circleci version
0.1.4211+14dfd68

When running circleci local execute in a python project, I got an access denied error bellow:

====>> Checkout code
  #!/bin/bash -eo pipefail
mkdir -p /home/circleci/alpe-skytef-gw && cp -r /tmp/_circleci_local_build_repo/. /home/circleci/alpe-skytef-gw
cp: cannot open '/tmp/_circleci_local_build_repo/./tests/__pycache__/test_inicializacao.cpython-36-PYTEST.pyc' for reading: Permission denied
Error: Exited with code 1
Step failed
Error: runner failed
Task failed

The thing is, when I clean all __pycache__ dirs, it runs flawlessly.

Attached is the config.yml file, and the full error output running in verbose...
error_report.txt
config.txt

We need development instructions for outside contributors and new CircleCI employees on how to config and build the CLI.

Can be in the readme, or if very verbose, it's own dev Markdown file.

[ben@localhost directory]$ circleci-cli local execute
Downloading latest CircleCI build agent...
Docker image digest: sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08: Pulling from docker.io/circleci/picard                                        
/usr/bin/docker-current: Error parsing reference: "circleci/picard@sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08: Pulling from docker.io/circleci/picard" is not a valid repository/tag: invalid reference format.
See '/usr/bin/docker-current run --help'.

That sha appears to sit in ~/.circleci/build_agent_settings.json. Here is the contents of that file:

{"LatestSha256":"sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08: Pulling from docker.io/circleci/picard"}

Tried with the latest release version, as well as built from master just now.

Relevant other links on the internet:
https://discuss.circleci.com/t/circleci-cli-circleci-picard-sha256-is-not-a-valid-repository-tag/24807/3
https://discuss.circleci.com/t/circleci-cli-on-linux-system-issues/18584

I am using the latest version of CLI tool

etc :: ~/projects/scio » circleci version
0.1.3667+073b0ec

And validate command is perfectly fine with my config

etc :: ~/projects/scio » circleci config validate
Config file at .circleci/config.yml is valid.

But once I try to spin local container, provision fails.

etc :: ~/projects/scio » circleci local execute
Docker image digest: sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08
====>> Spin up Environment
Build-agent version 0.1.750-336e0196 (2018-09-13T19:48:06+0000)
Error: Configuration errors: 1 error occurred:

* Cannot find a job named `build` to run in the `jobs:` section of your configuration file.
If you expected a workflow to run, check your config contains a top-level key called 'workflows:'
Step failed
Task failed

Even though it should not:

etc :: ~/projects/scio » grep "workflows" -A 17 .circleci/config.yml
workflows:
  version: 2

  # Per commit build
  build:
    jobs:
      - build_211
      - build_212
      - deploy:
          requires:
            - build_211
            - build_212
          filters:
            tags:
              only: /^v.*/
            branches:
              ignore: /.*/

As far as I can tell, the problem is on the picard side, which is closed sourced. How can I see the piece of code where this parsing and validation happens to somehow contribute the fix? Any other directions?

I've tried to pull the required docker image with $ docker pull {image} and that was fine. However, trying to $ circleci local execute leads to such error. There is no more info about it, even in verbose mode.

What I also tried to do: $ sudo circleci local execute because docker itself runs only under sudo. However I got this very strange error: Error: Unable to read config file: open .circleci/config.yml: permission denied, but that folder and file have all permissions for all users. I think I missed something because of weak Linux skill maybe. But it can be also a problem on CircleCI CLI side. Would appreciate any help, thank you.

There are many instances of fmt.Pritnf() that is missing a trailing newline, which causes the output to incorrectly run together. For example, circleci namespace create:

Namespace `...` created.Please note that any orbs you publish in this namespace are open orbs and are world-readable.

Most print statements that do not contain formatting directives use Println and there are Printf statements that do end in \n, so it appears that all of the instances in which this is missing are simply errors/bugs.

When a job is listed in a workflow but missing from the jobs section, the config validator will report the config as valid but will fail immediately when pushed to CircleCI:

* Cannot find a job named `build` to run in the `jobs:` section of your configuration file.

The validator should ensure that all jobs listed in workflows are actually defined.

CircleCI-Archived/local-cli#20 (comment)

cc @federicobond

I'm trying to see if I can use this version of the circleci binary to get test parallelization working with remote docker but my default ci container is based on alpine linux. Go makes it pretty easy to generate a static binary which would make it far more portable.

I have a small script that drops me into a bash session inside a container using the same image I use for builds on CircleCI. I use the circleci CLI to canonicalize the YAML configuration so I can slice up the output and figure out the name of the image. But it's slow because the CLI will do an update check each time before it processes the YAML.

Is there a way to disable the update check?

Tried using the install script and it does not work, as is. I believe the issue is the due to the formatting of the release.json file that is downloaded. The JSON text is all on one line, which causes issues with the grep commands as they return the entire JSON object instead of individual lines.

I have updated the script to fix this issue and will create a PR today.

Before:

Finding latest release.
Downloading CircleCI ## Changelog\n\n1f1866a Merge pull request #84 from CircleCI-Public/use-latest-codecov-orb\n229aaf5 Use released 0.0.4 codecov-clojure orb\n\n
An error occured installing the tool.

After:

Finding latest release.
Downloading CircleCI v0.1.1789

It'd be handy to be able to install this via Homebrew, if possible.

$> circleci version
0.1.3341+6be51e4

(I know it's not the latest version, trying to update the normal way instead of reinstalling latest version over it is what this issue is about.)

$> circleci update check
A new release is available (0.1.3923)
You are running 0.1.3341
You can update with `circleci update install`

$> sudo circleci update install
[sudo] password for karfau: 
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x67a122]

goroutine 1 [running]:
github.com/CircleCI-Public/circleci-cli/logger.(*Logger).Debug(0x0, 0x93792b, 0x12, 0xc420135b60, 0x1, 0x1)
	/go/src/github.com/CircleCI-Public/circleci-cli/logger/logger.go:37 +0x22
github.com/CircleCI-Public/circleci-cli/cmd.update(0xc420147340, 0x0, 0xc437a0, 0x0, 0x0, 0xc420135c00, 0x6e0e14, 0xc420125100)
	/go/src/github.com/CircleCI-Public/circleci-cli/cmd/update.go:149 +0x228
github.com/CircleCI-Public/circleci-cli/cmd.installUpdate(0xc420147340, 0x0, 0xc437a0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/CircleCI-Public/circleci-cli/cmd/update.go:124 +0x4d
github.com/CircleCI-Public/circleci-cli/cmd.newUpdateCommand.func4(0xc420164c80, 0xc437a0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/CircleCI-Public/circleci-cli/cmd/update.go:55 +0x49
github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra.(*Command).execute(0xc420164c80, 0xc437a0, 0x0, 0x0, 0xc420164c80, 0xc437a0)
	/go/src/github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra/command.go:762 +0x468
github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc420152500, 0xc420152780, 0xc420147360, 0xc420147350)
	/go/src/github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra/command.go:852 +0x30a
github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra.(*Command).Execute(0xc420152500, 0x0, 0xc42001d770)
	/go/src/github.com/CircleCI-Public/circleci-cli/vendor/github.com/spf13/cobra/command.go:800 +0x2b
github.com/CircleCI-Public/circleci-cli/cmd.Execute()
	/go/src/github.com/CircleCI-Public/circleci-cli/cmd/root.go:26 +0x27
main.main()
	/go/src/github.com/CircleCI-Public/circleci-cli/main.go:9 +0x20

(running with or without sudo makes no difference)

$ uname -a
Linux karfau-manjaro 4.18.16-2-MANJARO #1 SMP PREEMPT Thu Nov 1 15:47:06 UTC 2018 x86_64 GNU/Linux

If more details are needed to understand/fix the issue please let me know

Running docker on windows:

> docker version
Client:
 Version:           18.06.1-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        e68fc7a
 Built:             Tue Aug 21 17:21:34 2018
 OS/Arch:           windows/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.1-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       e68fc7a
  Built:            Tue Aug 21 17:29:02 2018
  OS/Arch:          linux/amd64
  Experimental:     true

Attempting to build locally results in:
There ins't any extra information when running it with the --verbose flag.

Docker image digest: sha256:3d88cfffd448da07ff8ced22ccdca5adc09ff18e87dc7db9b8af5b8a47647c08
Error: failed to execute docker: not supported by windows

Which would make sense given that "--volume", "/var/run/docker.sock:/var/run/docker.sock" wouldn't work on windows.

After some googlefoo -
-v //var/run/docker.sock:/var/run/docker.sock seems to work from windows

But I still have issues getting the docker to build by manually running it for now with:

docker run --interactive --tty --rm ^
    -v //var/run/docker.sock:/var/run/docker.sock ^
    -v %cd%:/root/.circleci ^
    --workdir /%cd%\ ^
      circleci/picard circleci build -c /root/.circleci/config.yml

Error:   error starting container <>: Error response from daemon: invalid mode: /tmp/_circleci_local_build_repo

Error:
Unexpected environment preparation error: Error response from daemon: invalid mode: /tmp/_circleci_local_build_repo

Step failed
Task failed

On version 0.1.3469 (the latest as of this posting):

$ circleci local execute --help
Usage of configFlags:
  -c, --config string   config file (default ".circleci/config.yml")
Error: Unable to parse flags

Context

I'm working on a CircleCI config generator (amongst other things) so I want to test that:

• the generated config is valid (circleci config validate)
• the generated tests pass (circleci local execute --job myjob)

From within CircleCI, so essentially I'm running those CircleCI commands in CircleCI. Very meta but it sounds like it should work.
First command (config validation) works fine.

Problem

I get the following error when running circleci local execute --job myjob (within a CircleCI job):

$ circleci local execute --job myjob
Docker image digest: sha256:0ae15d9b8329446dd5d241b1d654774871caa89c1cc47d513063c23037aae917
Error: failed to start event processor: failed to compute task config: failed to read config file: open circle.yml: no such file or directory

It's confusing because I'm using .circleci/config.yml and not using circle.yml.
The same command runs fine locally.

Running version 0.1.4211+14dfd68

What I tried

• Using the --config argument ❌
• Moving the generated code outside of the generator repo (to avoid git conflicts or even multiple circle ci config files in parent directories) ❌
• Running chmod 777 .circleci/config.yml ❌

Here's the smallest generator config file I've been able to reproduce teh bug with: https://gist.github.com/n6g7/f11008ee351b3dfd9c56f6c23b5ce7d6, I don't think the generated config file matters but can share it too if necessary.

Any idea where this could be coming from?

tried update circleci-cli, but can't.
Error message is this.

[]:~/ circleci update 
Error: error finding latest release: GET https://api.github.com/repos/CircleCI-Public/circleci-cli/releases: 403 API rate limit exceeded for XXX.XXX.XXX.XXX. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 27m17s]

This is GitHub API Limit, but if circleci cli can set GitHub personal token, I can avoid this error.
I want to can set GitHub token option like --github-token

$ circleci version                                                                              
0.1.2347+39dfda0

$ circleci                                                                                      
This project is the seed for CircleCI's new command-line application.

Usage:
  circleci [command]

Available Commands:
  config      Operate on build config files
  diagnostic  Check the status of your CircleCI CLI.
  help        Help about any command
  local       Debug jobs on the local machine
  namespace   Operate on namespaces
  orb         Operate on orbs
  query       Query the CircleCI GraphQL API.
  setup       Setup the CLI with your credentials
  update      Update the tool
  version     Display version information

Flags:
  -h, --help           help for circleci
      --host string    URL to your CircleCI host (default "https://circleci.com")
      --token string   your token for using CircleCI
      --verbose        Enable verbose logging.
Use "circleci [command] --help" for more information about a command.

It looks like circleci tests split command is missing.

https://circleci.com/docs/2.0/parallelism-faster-jobs/#splitting-test-files

Did you break support of it?

From build.go:

flags.StringSliceVarP(&args.taskInfo.Volumes, "volume", "v", nil, "Volume bind-mounting")

This setting doesn't seem to be read anywhere.

This is very similar to the bug on the UI where any newly configured job automatically opts one into an experimental 2.1 config feature set. This is not desired or expected.

CircleCI's cli version is 0.1.1430+9788d75

One can use these dummy config to reproduce by running circleci config validate

# generic v2.x config with filter
version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.6
    branches:
      only:
        - master
    steps:
      - checkout
      - run:
          name: Test
          command: |
            echo "this is just a test"

Unexpected output:
Error: Job "build" has filters configured in the job definition. These filters are incompatible with workflows.

# generic v2.x config with filter
version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.6
    # branches:
    #   only:
    #     - master
    steps:
      - checkout
      - run:
          name: Test
          command: |
            echo "this is just a test"

Expected output:
Error: Job "build" has filters configured in the job definition. These filters are incompatible with workflows.

# explicitly use v2.0 config with filter
version: 2.0
jobs:
  build:
    docker:
      - image: circleci/python:3.6
    branches:
      only:
        - master
    steps:
      - checkout
      - run:
          name: Test
          command: |
            echo "this is just a test"

Expected output:
Error: Job "build" has filters configured in the job definition. These filters are incompatible with workflows.

# explicitly use v2.1 config with filter
version: 2.1
jobs:
  build:
    docker:
      - image: circleci/python:3.6
    branches:
      only:
        - master
    steps:
      - checkout
      - run:
          name: Test
          command: |
            echo "this is just a test"

Expected output:
Error: jobs: build: extraneous key [branches] is not permitted

CLI Version 0.1.3277+950a73e

Error:

Error: orbs: deploy: -schedulerer0 subschemas matched instead of one
  orbs: deploy: -schedulererexpected type: String, found: Mapping
  orbs: deploy: -schedulerercommands: deploy: parameters: name: 2 schema violations found
    orbs: deploy: -schedulerercommands: deploy: parameters: name: required key [type] not found
    orbs: deploy: -schedulerercommands: deploy: parameters: name: extraneous key [cluster] is not permitted

Config sample:

orbs:                                                                                                                                                          
  deploy-schedulerer:
    ...

It seems that somehow the word deploy-schedulerer is being split.

Changing the config to deploy_scheduler made the errors more readable. I'm still the other errors.

I am trying out nixorg/nix:circleci Docker image for Circle CI, locally on macOS, using circleci CLI tool. However, it fails at the code checkout step with /bin/sh: mkdir: command not found error.

NOTE: it works fine on Circle CI platform.

$ circleci build

====>> Spin up Environment
Build-agent version 0.0.5895-01cdb92 (2018-05-29T20:07:10+0000)
Starting container nixorg/nix:circleci
  using image nixorg/nix@sha256:f3fff0fca5b41722a7c41b88b0879accd6f314307eb408601c1caaa9c1b71e3d

Using build environment variables:
  BASH_ENV=/tmp/.bash_env-localbuild-1533558734
  CI=true
  CIRCLECI=true
  CIRCLE_BRANCH=nix
  CIRCLE_BUILD_NUM=
  CIRCLE_JOB=build
  CIRCLE_NODE_INDEX=0
  CIRCLE_NODE_TOTAL=1
  [email protected]:owner/repo.git
  CIRCLE_SHA1=dcb3199a9bf730c351918d8f2322301926bdccba
  CIRCLE_SHELL_ENV=/tmp/.bash_env-localbuild-1533558734
  CIRCLE_WORKING_DIRECTORY=~/project

====>> Checkout code
  #!/bin/sh
mkdir -p /root/project && cp -r /tmp/_circleci_local_build_repo/. /root/project
/bin/sh: mkdir: command not found
Error: Exited with code 127
Step failed
Task failed

Circle CI config:

# .circleci/config.yml
version: 2

jobs:
  build:
    docker:
      - image: nixorg/nix:circleci
    steps:
      - checkout
      - run:
          command: nix-build

workflows:
  version: 2
  build_and_release:
    jobs:
      - build

Tried with these circleci versions:

$ brew install circleci
$ circleci --version
0.1.0

$ curl -o /usr/local/bin/circleci https://circle-downloads.s3.amazonaws.com/releases/build_agent_wrapper/circleci && chmod +x /usr/local/bin/circleci
$ circleci --version
0.0.5894-01cdb92

Documentation for build says:

circleci build --help

Run a full build locally

Usage:
  circleci build [flags]

Flags:
      ...
      --skip-checkout         use local path as-is (default true)
      ...

However it appears that the tool still tries to copy files over when doing

#!/bin/sh
mkdir -p /root/project && cp -r /tmp/_circleci_local_build_repo/. /root/project

I would appreciate any help with this, thanks!

I just downloaded circleci from the usual:

curl https://raw.githubusercontent.com/CircleCI-Public/circleci-cli/master/install.sh --fail --show-error | bash

It reports version:

$ circleci version
0.1.1568+255077c

Possible aside: I noted that circleci switch does nothing.

When I run something locally with the config-preview-sdk (i.e. version: 2.1) it errors with config parsing issues that indicate that it's not using the preview SDK e.g.

version: 2.1
...
jobs:
  deploy:
     executor:
        name: some-custom-executor
...      

* Error in config file: The schema/shape of the YAML is incorrect: json: cannot unmarshal object into Go struct field JobDescription.Executor of type string

I seem to be missing something, and guidance would be appreciated.

I think it'll be useful to add a create subcommand to config that allows the user to create new configs from scratch. Here's examples of how this might be use:

circleci config create [flags]

Flags:
  --executor  string   Can be 'docker', 'machine', or 'macos'. Defaults to 'docker'.
  --template  string   Path to a template file to be used when creating config.
```

I would image running `circleci config create` with defaults would:

1. Create the `./circleci` directory if it doesn't exists.
2. Create the file ./circleci/config.yml` if it doesn't exists.
3. Create a starting config like below:

```
version: 2.1
workflows:
  main:
    jobs:
      - build

jobs:
  build:
    docker:
      - image: cibuilds/base:2018.10
    steps:
      - checkout
      - run:
          name: "First Step"
          command: echo "Run a Bash command."
```

When adding a new project to CircleCI, the webapp shows a config (very verbose though) that the user could use for that project based on language. In addition to or instead of `--template`, we could have a language keyboard that pulls that same config for the user to use it as a staring point.

When installing install.sh, the install location is hard coded to /usr/local/bin. Users should be able to override this location.

Perhaps specifying an argument to the script, or by setting an env var before executing it.

cans@ws:~/src/circleci-cli$ bash -c "$(curl -fSl https://raw.githubusercontent.com/CircleCI-Public/circleci-cli/master/install.sh)"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1216  100  1216    0     0   8386      0 --:--:-- --:--:-- --:--:--  8386
Finding latest release.
Downloading CircleCI v0.1.3139
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   631    0   631    0     0   1561      0 --:--:-- --:--:-- --:--:--  1558
100 3259k  100 3259k    0     0  1221k      0  0:00:02  0:00:02 --:--:-- 2096k
circleci-cli_0.1.3139_linux_amd64/client/LICENSE
circleci-cli_0.1.3139_linux_amd64/md_docs/LICENSE
circleci-cli_0.1.3139_linux_amd64/circleci
Installing to /usr/local/bin/circleci
mv: impossible de créer le fichier standard '/usr/local/bin/circleci': Permission non accordée
An error occured installing the tool.
The contents of the directory /tmp/tmp.RhbLUmMxEx have been left in place to help to debug the issue.
cans@ws:~/src/circleci-cli$ ls /tmp/tmp.RhbLUmMxEx
ls: impossible d'accéder à '/tmp/tmp.RhbLUmMxEx': Aucun fichier ou dossier de ce type

I belive this is because you use set -o errexit and use a trap on EXIT to clean the directory.

As a side note, though no 100% sure, I also noticed the following: I am not sure it makes sense to use set -o pipefail and set -o errexit together. Indeed former asks to "keep going (but still report the error)" and the latter to "not keep going". Again, I might be wrong about that.

The documented way to pack a config folder into one file is using the output stream of the cli. Using >, however, the cli recently adds Checking for updates... with special chars to the file which makes it invalid.

I see three possible solutions:

1. Remove the version check from the config pack
2. Redirect the version check output to the stderr so that it is separate from the stdout and shown in the console, not the file
3. Add a possibility to pack a config into a file without using stdout.

Moreover, you may add a test case for the config pack that validates the output.

Hi, I'm trying to use circleci-cli for checking on local, but circleci update command doesn't work.
OS is MacOS 10.14, homebrew version is 1.8.6.

Error is below... Let me know how to update.

% circleci update check
`update` is not available because this tool was installed using `homebrew`.
Please consult the package manager's documentation on how to update the CLI.

# use sudo
% sudo circleci update install
Password:
`update` is not available because this tool was installed using `homebrew`.
Please consult the package manager's documentation on how to update the CLI.

Installation command

% brew install circleci

% circleci version
0.1.4786+bad101f

(This is somewhat related to a support request I've already submitted, but thought this would be useful for visibility)

If I have a .circleci/config.yml file with an emoji in it, the circleci config validate check says it's valid, but it's never picked up by CircleCI, as something is silently failing.

Emoji may not be supposed by all yaml parsers, for example in Python:

>>> import yaml
>>> with open(".circleci/config.yml") as f:
...     y = yaml.load(f)
...
yaml.reader.ReaderError: unacceptable character #x1f527: special characters are not allowed

However, I assume this golang CLI does yaml parsing, and thinks the config is valid.

Somewhere in the pipeline this is silently failing. That is: the build is never picked up by CircleCI

Basic issue reproduction: https://github.com/glasnt/verbose-journey/commits/master (note the change made in f3821380 and how there's no CI status badge on the commit)

I'm not a golang dev, but I'd suggest:

• upstream investigations, and/or, as a workaround
• make the validate fail if emoji are spotted (not a complete workaround, as validate doesn't block building, but it might be useful

We currently store user API tokens in a dotfile.

We should store them in the system provided storage for such items, such as the Keychain on macOS.

This library looks promising: https://github.com/zalando/go-keyring

Today, I noticed that recently, unexpected line breaks occur with long commands.

This is a shortened example from the circleci orb source of a dev orb I'm working on:

steps:
  - run:
        command: |
          circleci orb publish << parameters.orb-path >> << parameters.orb-ref >> <<# parameters.publish-token-variable >>--token << parameters.publish-token-variable >> <</ parameters.publish-token-variable >>

Here the config-expanded version from the online console:

circleci orb publish ./.orbspace/orb.yml ${ORB_PUBLISHING_NAMESPACE}/${ORB_PUBLISHING_NAME}@dev:${CIRCLE_BUILD_NUM}
 --token ${ORB_PUBLISHING_TOKEN} 

Notice the unexpected new line break before --token.

The confusing thing about this issue is that the same orb source code works from a previous orb version. That only makes sense to me if you choose the cli version depending on the orb version, otherwise I'm confused why the same config, same orb source and same command should be parsed differently just because the version is newer.

I think it is useless to check for new circleci-cli version each time we run the Docker image. I am not even sure that it makes sense to verify circleci-cli version at all inside Docker.