buildkite / cli Goto Github PK

View Code? Open in Web Editor NEW

164.0 22.0 47.0 2.19 MB

A command line interface for Buildkite.

License: MIT License

Go 98.75% Dockerfile 0.22% Shell 1.03%

cli's Introduction

bk - The Buildkite CLI

A command line interface for Buildkite.

Use bk to interact with your Buildkite organization without leaving the terminal 🙌.

Note

The 3.x (default) branch is under current active development. If you'd like to use the most recent released version of the Buildkite CLI, please refer to the main branch and releases page for details and installation instructions.

Installing

bk is available as a downloadable binary from the releases page.

Usage

$ bk --help
Work with Buildkite from the command line.

Usage:
  bk [command]

Examples:
$ bk build view
$ bk agent stop <agent ID>


Available Commands:
  agent       Manage agents
  build       Manage pipeline builds
  completion  Generate the autocompletion script for the specified shell
  configure   Configure Buildkite API token
  help        Help about any command
  init        Initialize a pipeline.yaml file
  use         Select an organization

Flags:
  -h, --help   help for bk

Use "bk [command] --help" for more information about a command.

Configure

You'll need to run bk configure first to set up your organization and API token.

💬 Feedback

We'd love to hear any feedback and questions you might have. Please file an issue on GitHub and let us know!

🔨 Development

Developed using Golang 1.20+ with modules.

git clone [email protected]:buildkite/cli.git
cd cli/
export BUILDKITE_GRAPHQL_TOKEN="<token>"
go generate
go run cmd/bk/main.go --help

cli's People

Contributors

Stargazers

Watchers

cli's Issues

bk init - GraphQL error: Authentication required - but it might actually be a permissions issue

I create a GraphQL token for my org but I don't give it any scopes (because I don't know who/when they are needed, and there are so many of them, so I just ignore it).

Then when I run bk init I get a GraphQL error: Authentication required error:

$ bk configure
...
$ bk init
Checking for pipeline file: .buildkite/pipeline.yml ✅
Checking for git repository and remote: https://github.com/drnic/buildkite-envvar-playtime ✅
Checking for buildkite pipeline: GraphQL error: Authentication required. Please supply a valid API Access Token: https://buildkite.com/docs/apis/graphql-api#authentication ❌
🚨 GraphQL error: Authentication required. Please supply a valid API Access Token: https://buildkite.com/docs/apis/graphql-api#authentication

My guess is that it isn't an authentication issue (authentication = who is this?) but an authorization issue (is this user allowed to do operation xyz?)

Is it possible for the bk init command error nicely and educate me in future?

Which scopes do I need for bk init and other bk operations?

agent env variables

Hi,

I'm sorry if I'm not posting in the right place but I don't seem to be able to figure out how to do this:

I am using Buildkite on OsX to build ReactNative apps. All good and I'm very happy but I have the case where I need to pass some environment variables to the agent that are not the same with the machine I'm working on.

For example ANDROID_HOME

Setting this in the pipeline file is not an option as I want to be able to build the code without the need of extra changes and I don't know how to pass to the agent the variables already existent to a "normal" user.

Thanks for any help you could give 🤗

Suggestion: easy install for any OS that has curl

GoBinaries is an awesome service that lets non-go users start using go-developed tools right away.

➜ curl -sf https://gobinaries.com/buildkite/cli/cmd/bk | sh

  ==> Downloading github.com/buildkite/cli/cmd/bk@master
  ==> Resolved version master to v1.0.0
  ==> Downloading binary for linux amd64
  ==> Installing bk to /usr/local/bin
  ==> Installation complete

Stalls on pipeline upload when used inside Docker container

I tried using this from a custom GitHub Action, but it seemed to stall. I tried running the same thing locally, and I have the same problem — it just stalls on the pipeline upload:

$ docker run -it --rm -v "$(PWD):/app" bk run local
>>> Starting local agent 🤖
>>> Starting build 👟
>>> Executing initial command: buildkite-agent pipeline upload
^C
>>> Gracefully shutting down...
🚨 Initial command failed: context canceled

The Dockerfile I'm using is here:
https://gist.github.com/toolmantim/c731730b4b4b7989f8f045956346bcfe

Any hints on how I'd be able to debug?

`bk init` parameters

It would be ideal to be able to provide either a config file (yml, toml, or other) to the bk init command to configure the pipeline from the CLI, or even better a bk pipeline config or similar command to do so.
That way we can create a "standard" pipeline config and easily bootstrap a bunch of repos. eg: GH settings, skip settings, etc.

some emoji missing or different set supported?

What is bk's relationship with https://github.com/buildkite/emojis?

In the example pipeline below, only :thumbsup: is supported by bk:

steps:
  - command: "sleep 2"
    label: ":thumbsup:"
  - command: "sleep 2"
    label: ":thumbsup::skin-tone-2:"
  - command: "sleep 2"
    label: ":+1:"
  - command: "sleep 2"
    label: ":+1::skin-tone-2:"

The output includes:

>>> Executing command step 👍
...
>>> Executing command step 👍:skin-tone-2:
...
>>> Executing command step :+1:
...
>>> Executing command step :+1::skin-tone-2:

fork/exec text file busy

I'm attempting to use bk run local to test my pipeline, but am having issues.

I ran this with --debug and here are some relevant lines.

2019/01/15 15:59:08 [http] POST /jobs/dc7e0e82-3e6b-484e-a553-94a05bb8b1cf/chunks?sequence=1&offset=0&size=50 127.0.0.1:49280
2019-01-15 15:59:08 DEBUG  local ↳ POST http://127.0.0.1:37989/jobs/dc7e0e82-3e6b-484e-a553-94a05bb8b1cf/chunks?sequence=1&offset=0&size=50 (HTTP/1.1 200 OK 1.311835ms)
2019-01-15 15:59:08 DEBUG  local [LogStreamer] Shutting down all workers
2019-01-15 15:59:08 DEBUG  local [JobRunner] Waiting for all other routines to finish
2019-01-15 15:59:08 DEBUG  local [JobRunner] Deleted env file: /tmp/job-env-dc7e0e82-3e6b-484e-a553-94a05bb8b1cf033464002
2019-01-15 15:59:08 DEBUG  local [LogStreamer/Worker#1] Worker has shutdown
2019-01-15 15:59:08 DEBUG  local [LogStreamer/Worker#0] Worker has shutdown
2019-01-15 15:59:08 DEBUG  local [LogStreamer/Worker#2] Worker has shutdown
2019-01-15 15:59:08 DEBUG  local PUT http://127.0.0.1:37989/jobs/dc7e0e82-3e6b-484e-a553-94a05bb8b1cf/finish
2019/01/15 15:59:08 [http] PUT /jobs/dc7e0e82-3e6b-484e-a553-94a05bb8b1cf/finish 127.0.0.1:49280
fork/exec /tmp/bootstrap-012338360: text file busy
2019-01-15 15:59:08 DEBUG  local ↳ PUT http://127.0.0.1:37989/jobs/dc7e0e82-3e6b-484e-a553-94a05bb8b1cf/finish (HTTP/1.1 200 OK 1.007612ms)
2019-01-15 15:59:08 DEBUG  local Received signal `INT`
2019-01-15 15:59:08 INFO   local Received CTRL-C, send again to forcefully kill the agent
2019-01-15 15:59:08 INFO   local Gracefully stopping agent. Waiting for current job to finish before disconnecting...
2019-01-15 15:59:08 INFO   local Finished job dc7e0e82-3e6b-484e-a553-94a05bb8b1cf
2019-01-15 15:59:08 INFO   local Disconnecting local...
2019-01-15 15:59:08 DEBUG  local POST http://127.0.0.1:37989/disconnect
2019/01/15 15:59:08 [http] POST /disconnect 127.0.0.1:49276
2019-01-15 15:59:08 DEBUG  local ↳ POST http://127.0.0.1:37989/disconnect (HTTP/1.1 200 OK 1.091944ms)
🚨 Initial command failed: Job failed with code 1

It almost looks like everything worked, but it had issues finalizing?

My pipeline:

steps:
  - label: Example Test
    command: echo "Hello"

Environment:

$ buildkite-agent --version
buildkite-agent version 3.8.2, build 2742

$ uname -a
Linux jerome-arch 4.20.0-1-MANJARO #1 SMP PREEMPT Mon Dec 24 08:20:48 UTC 2018 x86_64 GNU/Linux

brew failed: Download failed: $STABLE_URL

$ brew upgrade
...
==> Upgrading buildkite/cli/bk v1.0.0 -> v1.1.0
==> Downloading $STABLE_URL
Warning: Transient problem: timeout Will retry in 1 seconds. 3 retries left.
Warning: Transient problem: timeout Will retry in 2 seconds. 2 retries left.
Warning: Transient problem: timeout Will retry in 4 seconds. 1 retries left.
##O=#  #
curl: (6) Could not resolve host: $STABLE_URL
Error: Failed to download resource "bk"
Download failed: $STABLE_URL

Looks like https://github.com/buildkite/cli/blob/master/.github/tap-release.yml#L11 $STABLE_URL and $STABLE_SHA256 are not being replaced with a value; but $STABLE_VERSION is correctly being given a value.

My local bk formula is:

# Generated by https://github.com/buildkite/cli/blob/master/.github/tap-release.yml

class Bk < Formula
  homepage "https://github.com/buildkite/cli"
  desc "A command line interface for Buildkite."
  url "$STABLE_URL"
  version "v1.1.0"
  sha256 "$STABLE_SHA256"

  def install
    mv "bk-darwin-amd64-1.1.0", "bk"
    bin.install "bk"
  end

  test do
    assert_match "bk #{version}", shell_output("#{bin}/bk --version 2>&1")
  end
end

local run doesn't pass environment variables in from steps

Hi there, I've got a steps file which includes steps with elements in the env field:

  - label: "build :android:"
    command: bash -c ci/build.sh
    <<: *common
    artifact_paths:
      - logs/**/*
    env: 
      WORKER_TYPE: "AndroidClient"
      BUILD_TARGET: "local"
      SCRIPTING_TYPE: "mono"

However, when I run bk-cli local run steps.yaml this step fails with:

ci/build.sh: line 8: WORKER_TYPE: unbound variable

Is this the intended behaviour?

OS: macOS 10.13.6
CLI version: 0.4.0
buildkite-agent version: 3.8.4, build 2783

Create build should trigger the pipeline chooser?

"local run" does not interpolate environment variables in pipeline.yml

Version: d57d7d87d471c114eaf0f6ab91743568abad56c7

I have this in my pipeline.yml file:

steps:
 - label: foo
   command: 'echo hi'
   plugins:
    - docker-compose#${COMPOSE_VERSION}:
        run: app

And I run this:

$ export COMPOSE_VERSION=v2.6.0

$ bk local run
2019/02/06 10:19:41 Error getting git commit: exit status 128
2019/02/06 10:19:41 Error getting git branch: exit status 128
>>> Starting local agent 🤖
>>> Starting build 👟
>>> Executing initial command: buildkite-agent pipeline upload
>>> Executing command step foo

~~~ Setting up plugins
$ cd /Users/ssuter/workspace/bktest
🚨 Error: Failed to checkout plugin docker-compose: Incomplete plugin path "docker-compose"

>>> 🚨 Command failed in 3.069498517s
🚨 Job failed with code 1

[feature] bk run - allow running local pipeline using remote agents

Whilst discovering buildkite/agent#1181 I came across the desire for bk run or bk local run to split out its behaviour and run a local pipeline, but use remote agents.

I like bk local run -- it allows me to test a new pipeline before I "commit" all future PRs/commits/team members to the pipeline changes. You can iterate on the pipeline.

The command happens to run the steps locally; though to me this is tangential.

I can imagine this is not a small PR. I wanted to raise the topic and perhaps someone can contribute if its a good or bad idea.

bk gets stuck

I tried downloading and running the cli for a local build, but it just sat there doing nothing for 10 minutes until I figured it got stuck somehow and hit ^C:

karl@chrx:~/work/goprofile_test$ bk local run
>>> Starting local agent 🤖
>>> Starting build 👟
>>> Executing initial command: buildkite-agent pipeline upload
^C
>>> Gracefully shutting down...
🚨 Initial command failed: context canceled

When I ran it with debug, it complained about permission problems, and then got stuck again:

karl@chrx:~/work/goprofile_test$ bk local run --debug
2019/12/03 13:25:51 [keyring] Considering backends: [pass file]
2019/12/03 13:25:51 [keyring] Failed backend pass: The pass program is not available
2019/12/03 13:25:51 Serving API on http://127.0.0.1:35269
2019/12/03 13:25:51 Creating bootrap script at /tmp/bootstrap-458222797
>>> Starting local agent 🤖
>>> Starting build 👟
>>> Executing initial command: buildkite-agent pipeline upload
open /etc/buildkite-agent/buildkite-agent.cfg: permission denied^C
>>> Gracefully shutting down...

🚨 Initial command failed: context canceled

How do I fix this?

Artifact download

I often want to download all artifacts for a particular build or job to my filesystem for local build pipeline debugging. It'd be neat if bk download artifacts https://buildkite.com/acme/widgets/builds/123 worked, or something.

Unable to configure on WSL

I'm running Linux Subsystem on Windows, and when running bk configure, I get the below error, since WLS isn't able to open a browser. Is it possible to output the URL so that I can copy and paste into the browser. (Similar behaviour to how composer handles this.)

We need to authorize this app to access your repositories. This authorization is stored securely locally, buildkite.com never gets access to it.

When you press enter, your default browser will open and authenticate to github.com
❌

🚨 Github OAuth error: exec: "xdg-open": executable file not found in $PATH```

Running 0.3.0

Configuration path output is `%!s(func() (string, error)=0xxxx)`

When you run bk configure, the output you get is:

Enable retrying failed job based on BUILDKITE_JOB_ID

It would be very useful to be able to retry a failed job locally via bk by referencing the BUILDKITE_JOB_ID. I'd imagine bk fetching the job details from the API and starting a local build with the same environment variables, etc.

bk run supports docker plugin, but is not propagating env vars

Consider this pipeline:

https://gist.github.com/drnic/a042bd719ed9213c5af7fd1605f8bf93#file-pipeline-bk-run-docker-fail-yml

When you run it, the -E AGENT_ENV=foobar value is not propagated into Docker:

git clone https://gist.github.com/drnic/a042bd719ed9213c5af7fd1605f8bf93 ~/workspace/pipeline.bk-run-docker-fail
cd ~/workspace/pipeline.bk-run-docker-fail

bk run pipeline.bk-run-docker-fail.yml -E AGENT_ENV=where-is-this-env

The output should include the two lines:

STEP_ENV=found-step-env
AGENT_ENV=where-is-this-env

But it only includes the first one:

STEP_ENV=found-step-env

Why does bk local run -E AGENT_ENV=xyz not pass $AGENT_ENV into the docker plugin env?

Choose step as a default for local run?

I am coming from bksr for local run where a default bksr would allow you to choose step from your pipeline.yml.

Can the default for local run retain the same behavior? Currently bk local run doesnt (appear to) allow you to choose or specify the specific step.

Where are my tokens securely stored?

I love that the app is very explicit about tokens and securely storing them, and tells me that it "Securely stored GraphQL token! 💪"

But I'm puzzled about where they are stored, and how to remove them later. Maybe "Securely stored GraphQL token in your Keychain! 💪" or something?

Fix kwallet dbus

As applied on: 99designs/aws-vault@cb88446 we should update the keyring deps to include this fix: 99designs/keyring#52 otherwise kwallet backend does not work.

BK Init Command Fails When Not Admin-Level User

In trying to initialize a new pipeline on Buildkite, I was attempting to run the bk init command.
Nothing had been setup in BuildKite at this stage.
When I ran the command at a user level (just part of the default "Everyone" team), the command failed with:
"Checking for buildkite pipeline: GraphQL error: Please ensure at least one team is added to this pipeline"
When running as an admin-level user, the init succeeded and I noticed when going to the web ui no team was assigned.
So it seems two things are happening:

A non-admin user doesn't have the ability to create a pipeline without a team assignment. (whether anyone of any level is supposed to be able to do this I do not know)
There doesn't appear to be any flag to provide a team for the CLI and thus it just fails with the above error.

Also, just to note: a user just part of the default everyone group, is able to select a team and successfully make a pipeline in the web ui, so I don't believe it's some sort of permissions issue. Though maybe there is some permission we can provide users so they can...not pass a team?

Ctrl-c'ing out of a menu probably shouldn't show error

If you hit ctrl-c after hitting up/down in the bk browse pipeline chooser, you get this message:

`bk local run` doesn't respect skip

Background

Description

Currently, when running bk local run with a step containing skip: true the agent still runs the commands

Steps to reproduce

Create a basic pipeline with the below under .buildkite/pipeline.yml:

steps:
  - label: ":coffee: i need coffee :coffee:"
    skip: true
    commands:
      - echo "coffee is better than tea" > file.out

  - wait

  - label: ":hammer: file.out is created"
    commands:
      - echo "There shouldn't be a file.out but there is :("

Run bk local run and observe the creation of the file.out which shouldn't exist as it should be skipped

Desired Outcome

The key skip is respected, with its bool value or string

Run bk configure in browserless environment

Currently one of the step for bk configure requires configuration of github cred within a browser:

──── Let's configure your github.com credentials 💻

We need to authorize this app to access your repositories. This authorization is stored securely locally, buildkite.com never gets access to it.

When you press enter, your default browser will open and authenticate to github.com

How to do this step in the command line in a browserless/headless environment?

Pipeline can be found by fuzzy search but cannot browse to that pipeline

Hi there,

Thank you very much for implementing buildkite cli.

Currently, I got an issue running bk browse.
The pipeline can be found by running

bk pipline list --fuzzy=pipeline-name

However, when im in pipeline-name directory and run

bk browse

It outputs an error

🚨 Pipeline doesn't exist

This is the debug version of the command

2019/09/16 10:18:16 [keyring] Considering backends: [pass file]
2019/09/16 10:18:16 [keyring] Failed backend pass: The pass program is not available
2019/09/16 10:18:16 [keyring] Expanded file dir to /home/bangn/.buildkite/keyring/
2019/09/16 10:18:16 [keyring] Expanded file dir to /home/bangn/.buildkite/keyring/
Enter passphrase to unlock /home/user/.buildkite/keyring/:
🚨 Pipeline doesn't exist

Im using bk cli version 1.0.0 on debian 10.

Do you have any idea on why it cannot browse to the pipeline?

Cheers,
Bang

Specifying --pipeline option to init causes panic

CLI version: 0.3.0
OS: Arch linux

→ bk init --pipeline=research
Checking for pipeline file: .buildkite/pipeline.yml ✅
Checking for git repository and remote: https://github.com/quantopian/research_env ✅
Enter passphrase to unlock /home/irl/.buildkite/keyring/: 
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/buildkite/cli.InitCommand(0xa67ac0, 0xe56ae8, 0xa65020, 0xc00017dd70, 0x0, 0x0, 0x0, 0x7ffe0394a4e7, 0x8, 0x7f3124a12290, ...)
        /Users/lachlan/go/src/github.com/buildkite/cli/cmd_init.go:99 +0x1c5f
main.run.func5(0xc0001ea000, 0x409dc3, 0x936360)
        /Users/lachlan/go/src/github.com/buildkite/cli/cmd/bk/main.go:192 +0xb5
gopkg.in/alecthomas/kingpin%2ev2.(*actionMixin).applyActions(0xc0001b0498, 0xc0001ea000, 0x0, 0x0)
        /Users/lachlan/go/pkg/mod/gopkg.in/alecthomas/[email protected]/actions.go:28 +0x6d
gopkg.in/alecthomas/kingpin%2ev2.(*Application).applyActions(0xc0001e4690, 0xc0001ea000, 0x0, 0x0)
        /Users/lachlan/go/pkg/mod/gopkg.in/alecthomas/[email protected]/app.go:557 +0xdf
gopkg.in/alecthomas/kingpin%2ev2.(*Application).execute(0xc0001e4690, 0xc0001ea000, 0xc000176b50, 0x1, 0x1, 0x0, 0x0, 0x1, 0xc00019c2e8)
        /Users/lachlan/go/pkg/mod/gopkg.in/alecthomas/[email protected]/app.go:390 +0x8f
gopkg.in/alecthomas/kingpin%2ev2.(*Application).Parse(0xc0001e4690, 0xc0000a0010, 0x3, 0x3, 0x1, 0xc00019c2b8, 0x0, 0x1)
        /Users/lachlan/go/pkg/mod/gopkg.in/alecthomas/[email protected]/app.go:222 +0x1fa
main.run(0xc0000a0010, 0x3, 0x3, 0x9f9d88)
        /Users/lachlan/go/src/github.com/buildkite/cli/cmd/bk/main.go:356 +0x2a16
main.main()
        /Users/lachlan/go/src/github.com/buildkite/cli/cmd/bk/main.go:21 +0x6e

Without specifying pipeline:

→ bk init
Checking for pipeline file: .buildkite/pipeline.yml ✅
Checking for git repository and remote: https://github.com/quantopian/research_env ✅
Enter passphrase to unlock /home/irl/.buildkite/keyring/: 
Checking for buildkite pipeline: Created https://buildkite.com/quantopian/research-env ✅
Checking github repository config for quantopian/research_env: Created webhook ✅

Ok! Your project is ready to go at https://buildkite.com/quantopian/research-env 🚀

[REQUEST] Support `DOCKER_HOST` etc

Because of crippling performance issues with Docker on macOS Mojave, I use dinghy to run Docker on macOS, which essentially runs it in a VM then mounts local directories over NFS into the VM. It's hacky, but it is about a 10x performance improvement in IO-heavy projects :(.

I'm not sure if there are inherent limitations that would make this impossible, but it'd be great if I could use bk in this context. Currently, I get the following bk local run error:

docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.

Infinite builds

Hello BK :)

Using the bk init function to set up scaffolding and pushing up the skeleton init commit results in the agent running infinite builds.

I used the client to manually init a repo I had to test the bk agent using the AWS stack set up. When I checked the bk pipeline I noticed that it was continually building and had to manually stop the build.

I am happy to put in a fix for this - any preferences?

exec: "buildkite-agent": executable file not found in $PATH

First of all hugs ❤️
Secondly, wondering the definition of the 'interface' to BK is?
Our workflow tends to involve developers working at their local machines, pushing changes up to a remote repository and having remote bk agents process the pipelines.

Intially, was hoping that bk local run would simulate the running of an agent but after a brew install I received

exec: "buildkite-agent": executable file not found in $PATH

How do you envision this local run working?
Is there scope for including the buildkite-agent install step as part of the brew formulae?
Ideally I'd love to run bk local run as part of a git commit hook on devs locals.
Is the executable just a by-product of needing a BK API client?

I'm currently using a cobra client with github.com/buildkite/go-buildkite/buildkite.
It performs extra stuff like emitting statsd metrics based on the results of the api calls.
Are you imagining people use this tool in their pipelines for similar purposes?

If you'd like a customer to do user stories (https://www.mountaingoatsoftware.com/articles/questioning-your-users) with let me know, I would love to help out.

Again, lots of love for BK ❤️
Thank you.

buildkite-agent: command not found - it is installed; how do I get bk to use it?

I have a step that fetches meta-data, but bk run local fails.

My bash script invokes:

CF_ORGANIZATION=$(buildkite-agent meta-data get cf-organization)
CF_SPACE=$(       buildkite-agent meta-data get cf-space)

But bk fails with:

ci/scripts/cf-push.sh: line 5: buildkite-agent: command not found

I do have buildkite-agent installed locally. How do I get bk to use it?

Ability to choose individual step(s)

We have a monorepo with a pre-merge pipeline that runs many disparate steps.

For local iteration it would be really convenient to run a subset of those steps.

Could bk local run ..... --steps foo bar baz become possible? Run the specified step(s) in the order given?

Output from a specific build and step

Somewhat related to #11

In my case, I've got a Rails app (with Docker) and develop locally (also with Docker). Sometimes I build something and CI fails way more than excepted. 😰

What I'd love to be able to do is to locally grab just the output of the failures for one step of our builds. From there I could pass this to RSpec's --only-failures option and easily iterate on anything broken.

PANIC after bk configure

After successfully run bk configure, any other commands such bk browse errors with message:

🚨 %!!(MISSING)v(PANIC=runtime error: invalid memory address or nil pointer dereference)

Running on MacOs 10.14.3
Installed using homebrew
bk --version
0.3.0

Local Pipeline run failing on Windows

Hi there, I'm attempting to run a pipeline locally on Windows, and am having some trouble. I've pasted the output below and you can find a copy of the static steps here.

Any ideas where I'm going wrong?

Thanks

λ bk local run .buildkite/premerge.steps.yaml --debug
2019/02/06 18:29:18 [keyring] Considering backends: [file]
2019/02/06 18:29:18 Serving API on http://127.0.0.1:49256
2019/02/06 18:29:18 Creating bootrap script at C:\Users\JAMIEB~1\AppData\Local\Temp\bootstrap-450177923
>>> Starting local agent 🤖
>>> Starting build 👟
>>> Executing initial command: buildkite-agent pipeline upload ".buildkite/premerge.steps.yaml"

  _           _ _     _ _    _ _                                _
 | |         (_) |   | | |  (_) |                              | |
 | |__  _   _ _| | __| | | ___| |_ ___    __ _  __ _  ___ _ __ | |_
 | '_ \| | | | | |/ _` | |/ / | __/ _ \  / _` |/ _` |/ _ \ '_ \| __|
 | |_) | |_| | | | (_| |   <| | ||  __/ | (_| | (_| |  __/ | | | |_
 |_.__/ \__,_|_|_|\__,_|_|\_\_|\__\___|  \__,_|\__, |\___|_| |_|\__|
                                                __/ |
 http://buildkite.com/agent                    |___/

2019-02-06 18:29:18 NOTICE Starting buildkite-agent v3.8.4 with PID: 40328
2019-02-06 18:29:18 NOTICE The agent source code can be found here: https://github.com/buildkite/agent
2019-02-06 18:29:18 NOTICE For questions and support, email us at: [email protected]
2019-02-06 18:29:18 INFO   Configuration loaded from: C:\buildkite-agent\buildkite-agent.cfg
2019-02-06 18:29:18 DEBUG  Bootstrap command: C:\Users\JAMIEB~1\AppData\Local\Temp\bootstrap-450177923
2019-02-06 18:29:18 DEBUG  Build path: C:\Users\JAMIEB~1\AppData\Local\Temp\buildkite-build-735988486
2019-02-06 18:29:18 DEBUG  Hooks directory: C:\buildkite-agent\hooks
2019-02-06 18:29:18 DEBUG  Plugins directory: C:\buildkite-agent\plugins
2019-02-06 18:29:18 INFO   Running builds within a pseudoterminal (PTY) has been disabled
2019-02-06 18:29:18 INFO   Registering agent with Buildkite...
2019-02-06 18:29:18 INFO   Started 1 Agent(s)
2019-02-06 18:29:18 INFO   You can press Ctrl-C to stop the agents
2019-02-06 18:29:18 DEBUG  POST http://127.0.0.1:49256/register
2019-02-06 18:29:18 WARN   Post http://127.0.0.1:49256/register: dial tcp 127.0.0.1:49256: socket: The requested service provider could not be loaded or initialized. (Attempt 1/30 Retrying in 10s)
2019-02-06 18:29:28 DEBUG  POST http://127.0.0.1:49256/register
2019-02-06 18:29:28 WARN   Post http://127.0.0.1:49256/register: dial tcp 127.0.0.1:49256: socket: The requested service provider could not be loaded or initialized. (Attempt 2/30 Retrying in 10s)
2019/02/06 18:29:30 [http] GET /register 127.0.0.1:49257
>>> 😓 An unknown agent API endpoint was requested (GET /register).
File an issue at https://github.com/buildkite/cli/issues and we'll see what we can do!
2019/02/06 18:29:30 [http] GET /favicon.ico 127.0.0.1:49257
>>> 😓 An unknown agent API endpoint was requested (GET /favicon.ico).
File an issue at https://github.com/buildkite/cli/issues and we'll see what we can do!
2019-02-06 18:29:38 DEBUG  POST http://127.0.0.1:49256/register
2019-02-06 18:29:38 WARN   Post http://127.0.0.1:49256/register: dial tcp 127.0.0.1:49256: socket: The requested service provider could not be loaded or initialized. (Attempt 3/30 Retrying in 10s)
2019-02-06 18:29:48 DEBUG  POST http://127.0.0.1:49256/register
2019-02-06 18:29:48 WARN   Post http://127.0.0.1:49256/register: dial tcp 127.0.0.1:49256: socket: The requested service provider could not be loaded or initialized. (Attempt 4/30 Retrying in 10s)
2019-02-06 18:29:58 DEBUG  POST http://127.0.0.1:49256/register
2019-02-06 18:29:58 WARN   Post http://127.0.0.1:49256/register: dial tcp 127.0.0.1:49256: socket: The requested service provider could not be loaded or initialized. (Attempt 5/30 Retrying in 10s)
2019-02-06 18:30:08 DEBUG  POST http://127.0.0.1:49256/register
2019-02-06 18:30:08 WARN   Post http://127.0.0.1:49256/register: dial tcp 127.0.0.1:49256: socket: The requested service provider could not be loaded or initialized. (Attempt 6/30 Retrying in 10s)

>>> Gracefully shutting down...

🚨 Initial command failed: context canceled

cc @petemounce @DoomGerbil

Update logo of Buildkite CLI on GitHub apps

Right now it shows a randomly generated Avatar asking for a lot of things, the user may be confused if this is really from Buildkite.

Would be great if the avatar is a Buildkite logo.

No directory provided for file keyring

What is the proper option to get the initial install working? The current docs result in the below error:

ubuntu@ip-10-221-102-20 ~ $ bk configure
──── Ok! Let's get started with configuring bk 🚀


──── Configuring Buildkite GraphQL credentials

Create a GraphQL token at https://buildkite.com/user/api-access-tokens/new. Make sure to tick the GraphQL scope at the bottom.

GraphQL Token: anthonyXXXXXion.com ✅

🚨 No directory provided for file keyring

Keyring-backend option mentions file but if I specify a file I get a confusing error:

22:24 $ bk configure --keyring-backend=$HOME/test
🚨 enum value must be one of file, got '/home/ubuntu/test'

Failed to find organization id for slug

After authentication; bk init process failed at this step:

Checking for pipeline file: .buildkite/pipeline.yml ✅
Checking for git repository and remote: https://github.com/XXXXX/XXXX ✅
Enter passphrase to unlock /Users/antho/.buildkite/keyring/:
Checking for buildkite pipeline: Failed to find organization id for slug "XXXXX" ❌

Issue using the junit plugin on osx

Hi
First, thank you for the great tool which allows to test our pipelines on local.

Running the tool installed from brew on osx, I encountered an issue that seems to be linked with the peculiar status of the platform.

When running the sample pipeline of the plugin: junit-annotate, I see the following error happening:

 bk local run .buildkite/pipeline.integration_tests.yml
(...)

--- :junit: Processing the junits
docker: Error response from daemon: Mounts denied: 
The path /var/folders/kf/fsdklxv57_v6kq5ydjxpj6g00000gp/T/buildkite-plugins-670558647/github-com-buildkite-plugins-junit-annotate-buildkite-plugin-v1-7-0/ruby
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.
.
ERRO[0000] error waiting for container: context canceled 
🚨 Error: The command exited with status 125

Apparently, when trying to mount such a folder on a mac, one would need to prefix /var with /private and use instead /private/var according to this comment

Is this a problem of the plugin or of the bk local agent?

Inferred file keyring backend can't easily be overridden

On macOS I'm trying to use the keychain backend, but file is always the one chosen even when I explicitly specify --keyring-backend=keychain.

This seems to have been introduced in 3e9370c. Following that change, inferring the file backend from keyringFileDir is no longer valid because the latter string now defaults to non-empty.

As confirmation of this behaviour, I'm able to work around the problem and use keychain by specifying --keyring-file-dir "".

I think the solution is to only try to infer a backend if it's not explicitly specified. The inference of file from keyringFileDir will also need rethinking.

Adding a license

What is the licensing on this repository? Is it MIT like the rest of your software? If so, could we add a LICENSE file?

GitHub Enterprise Support?

currently bk seems to point only at github.com... any chance there will be support for GitHub Enterprise and on-prem installations?

Thanks!

bk local run - support if conditions - currently ignoring them

https://buildkite.com/docs/pipelines/conditionals

steps:
  - command: "echo --- All good. Nice one."
    label: ":git:"
    if: build.branch == 'master'
  - command: "echo --- ERROR this shouldn't run; exit 1"
    label: ":git:"
    if: build.branch != 'master'

When I run this pipeline.yml it runs both steps; but it should only run one or the other.

@keithpitt is this something that should work, or one of those "its annoying to implement everything in ruby + golang" (my words not yours) gaps ;)

"bk init" assumes your GitHub org matches your Buildkite org

I should be able to override the Buildkite org for init - it assumes it will match the GitHub org.

Provide ability to set BUILDKITE_* environment variables

I run agents for a few different Buildkite orgs on my machine, and I'd like to use bk run with the non-default one. To that end, I'd like to be able to do things like this without the env vars intended for the agent being overriden:

BUILDKITE_CONFIG_PATH=/etc/buildkite-agent/bt-buildkite-agent.cfg bk run

GitHub scopes too large?

Is all we need a list of repos? There's a new GitHub App permissions scope that can give you that, without all this:

If we are asking for a god token, perhaps we need to spell out why a little more?

Also perhaps we need to say that the GitHub authorisation is only valid to your cli, and can't be used outside of the cli (if we were, say, to update the GitHub App settings or something). Assuming that's correct? I was only a little more comfortable once I noticed the 127.0.0.1 redirect at the bottom of the authorise screen.

Secrets backend choosing doesn't seem to work

On linux, attempting to specify a secret-backend other than kwallet (which isn't installed, because people tend to not use KDE here) fails:

# me on my-box in ~/mydir on git:master ✖︎ [10:04:40]
→ go run cmd/bk/main.go configure buildkite
──── Configuring Buildkite GraphQL credentials

Create a GraphQL token at https://buildkite.com/user/api-access-tokens/new. Make sure to tick the GraphQL scope at the bottom.

GraphQL Token: [email protected] ✅

�� The name org.kde.kwalletd was not provided by any .service files
exit status 1
[1]    2541 exit 1     go run cmd/bk/main.go configure buildkite

# me on my-box in ~/mydir on git:master ✖︎ [10:06:01]
→ go run cmd/bk/main.go configure buildkite --keyring-backend=secret-service
──── Configuring Buildkite GraphQL credentials

Create a GraphQL token at https://buildkite.com/user/api-access-tokens/new. Make sure to tick the GraphQL scope at the bottom.

GraphQL Token: [email protected] ✅

�� The name org.kde.kwalletd was not provided by any .service files
exit status 1
[1]    2918 exit 1     go run cmd/bk/main.go configure buildkite --keyring-backend=secret-service

# me on my-box in ~/mydir on git:master ✖︎ [10:06:44]
→ go run cmd/bk/main.go configure buildkite --keyring-backend=file          
──── Configuring Buildkite GraphQL credentials

Create a GraphQL token at https://buildkite.com/user/api-access-tokens/new. Make sure to tick the GraphQL scope at the bottom.

GraphQL Token: [email protected] ✅

�� The name org.kde.kwalletd was not provided by any .service files
exit status 1
[1]    3048 exit 1     go run cmd/bk/main.go configure buildkite --keyring-backend=file

# me on my-box in ~/mydir on git:master ✖︎ [10:07:04]
→ go run cmd/bk/main.go --keyring-backend=file configure backend
�� unexpected backend
exit status 1
[1]    3225 exit 1     go run cmd/bk/main.go --keyring-backend=file configure backend

# me on my-box in ~/mydir on git:master ✖︎ [10:07:09]
→ go run cmd/bk/main.go --keyring-backend=file configure buildkite
──── Configuring Buildkite GraphQL credentials

Create a GraphQL token at https://buildkite.com/user/api-access-tokens/new. Make sure to tick the GraphQL scope at the bottom.

GraphQL Token: [email protected] ✅

�� The name org.kde.kwalletd was not provided by any .service files
exit status 1
[1]    3378 exit 1     go run cmd/bk/main.go --keyring-backend=file configure buildkite

Pass environment variables when creating a build

I notice the 'Create a build' REST API call includes the capability to set environment variables.

It would be awesome if this CLI app exposed this feature so we could pass environments variables to builds from the command line.

bk build create exit status 128 on osx

I'm unable to successfully create a build on osx High Sierra. I've tried using both the keyring and a file backend but I don't have enough documentation and the error messages are too opaque to know what I'm doing wrong. Please help:

> bk configure
──── Ok! Let's get started with configuring bk 🚀


──── Configuring Buildkite GraphQL credentials

Create a GraphQL token at https://buildkite.com/user/api-access-tokens/new. Make sure to tick the GraphQL scope at the bottom.

GraphQL Token:  [redacted] ✅

Securely stored GraphQL token! 💪
Wrote configuration to /Users/[redacted]/.buildkite/config.json 📝

For now, we will assume you are using GitHub. Support for others is coming soon! 😓

──── Let's configure your github.com credentials 💻

We need to authorize this app to access your repositories. This authorization is stored securely locally, buildkite.com never gets access to it.

When you press enter, your default browser will open and authenticate to github.com

Authenticated as [redacted] ✅
Securely stored GitHub token! 💪

Ok, you are good to go!

^^ This seems to work and I've also tried the following commands using both the osx keychain and a file keychain so I don't think this is the issue.

> bk init
Checking for pipeline file: .buildkite/pipeline.yml ✅
Checking for git repository and remote: exit status 128 ❌
🚨 exit status 128

> git status
On branch uavcan-v1/issues/235
nothing to commit, working tree clean

As you can see, this is a fully functional repository that I use to submit to github all the time.

> bk build create --debug --debug-graphql
2019/07/27 11:05:04 [keyring] Considering backends: [keychain pass file]
2019/07/27 11:05:04 [keyring] Querying keychain for service="buildkite", account="graphql-token", keychain=""
2019/07/27 11:05:04 [keyring] Found item "Buildkite GraphQL Token"
🚨 exit status 128

Again, no useful debug information here, just "exit status 128".

Other commands:

> bk browse
🚨 exit status 128
> bk pipeline list
uavcan/libuavcan-v1
> bk local run
(this works)

I'd really like the ability to initiate a build from local changes! Thanks for helping (You all are rockstars🎸 by the way).

Ability to run pipelines locally without giving out org's agent token?

Hey!
We have a handful of folks within our org who would like to use the local pipeline running functionality of this, however they need to be running an instance of the agent, which requires our agent token. I'm hesitant to give out our org's agent token to every developer who wants to do local testing, given that it allows registration of new agents, which may unintentionally cause issues. Is there any way to get the local pipeline functionality without giving out our agent token?

I tried using a fake token to see if it would still work, but unfortunately it appears to error out if the token is invalid.