We are often asked how to restart the agent to pick up new configuration or how to shutdown agents without killing running tasks. This needs to be documented somewhere.

The AWS SNS guide is outdated and the linked document no longer gives enough detail to follow through with!

I've noticed that many of the environment variables listed when doing a buildkite-agent start --help are not documented here: https://buildkite.com/docs/guides/environment-variables, although the ones I've tried can be queried in the build step.
A useful one is BUILDKITE_BUILD_PATH

BUILDKITE_AGENT_CONFIG
BUILDKITE_AGENT_DEBUG
BUILDKITE_AGENT_DEBUG_HTTP
BUILDKITE_AGENT_ENDPOINT
BUILDKITE_AGENT_META_DATA
BUILDKITE_AGENT_NO_COLOR
BUILDKITE_AGENT_PRIORITY
BUILDKITE_AGENT_TOKEN
BUILDKITE_BOOTSTRAP_SCRIPT_PATH
BUILDKITE_BUILD_PATH
BUILDKITE_HOOKS_PATH
BUILDKITE_NO_AUTOMATIC_SSH_FINGERPRINT_VERIFICATION
BUILDKITE_NO_COMMAND_EVAL
BUILDKITE_NO_PTY

https://buildkite.com/docs/pipelines/wait-step

- command: "command.sh"
- wait: ~
  continue_on_failure: true
- command: "echo This runs regardless of the success or failure"
- wait
- command: "echo The command passed"

is listed as an example of a wait step-including-pipeline that continues. Is the wait: ~ mandatory? What if we attached a blank key like

- command: "command.sh"
- wait: 
  continue_on_failure: true
- command: "echo This runs regardless of the success or failure"
- wait
- command: "echo The command passed"

Instead? Should that still continue? It's not clear according to the docs what role ~ has here.

Buildkite team can check this pipeline: https://buildkite.com/gusto/buildkite-superpull/builds/95#772d22ee-e9f1-44b7-a63b-f89577f91624 for an example of a wait: key with continue_on_failure set that doesn't have~

Hi Team,

wondering how to schedule build using pipeline.yml file.

Thanks,
Aparna

I'm looking at the API docs and trying to work out how to create a pipeline in a "team"
https://buildkite.com/docs/rest-api/pipelines#create-a-pipeline

It's unclear how to do this.

The docs refer to BUILDKITE_NO_AUTOMATIC_SSH_FINGERPRINT_VERIFICATION, which is only in Agent v3. In Agent v2 it is BUILDKITE_AUTO_SSH_FINGERPRINT_VERIFICATION . See https://github.com/buildkite/agent/blob/2b8f1d569b659e07de346c0e3ae7090cb98e49ba/templates/bootstrap.sh#L226

E.g https://buildkite.com/macstadium/vmkite-images/settings/setup/github

New ones look like this:

The docs are very useful, but would be even more useful if they were searchable.

Please make it happen!

I was specifically looking to convert a CircleCI build over, and needed some guidelines for mapping over the various bits. Especially caching things like asdf and yarn etc.

https://github.com/buildkite/docs/blob/master/pages/guides/docker_containerized_builds.md.erb#L9

to define it’s own

"its" is the possessive.

Hi,

Just to be sure. We've had a member in our organization who set up the most of our buildkite environment. Now that he has left the organization I want to delete him from the users. What are the consequences of this? I previously had some bad experiences where deleting a user, automatically deleted all the things that he had set up for us.

Bye, Peter

There are a few broken images on the parallel builds page.

Hi!

The section on Google Cloud Storage [1] is missing documentation on the below two environment variables (which I found through the agent source code):

• BUILDKITE_GS_ACL
• BUILDKITE_GS_APPLICATION_CREDENTIALS

Also, it doesn't mention that GCS is only supported with agent 3.x.

Thanks!

[1] https://buildkite.com/docs/agent/gcloud#uploading-artifacts-to-google-cloud-storage

A customer reported that the GitLab docs are out of date:

As a side note, the links in the Gitlab setup settings/setup/gitlab?setup=true for automatic builds are invalid. After the recent gitlab UI changes, both services and webhooks are under /settings/integrations

I run a Builtkite agent behind an HTTP proxy. The proxy only allows connections to whitelisted domains. Which domains should I have whitelisted for the Buildkite agent to work correctly?

https://agent.buildkite.com
... anything else?

It seems parts of the docs are scattered all over the place with big chunks missing

I can see definition of the main keys in a step, then followed by the main plugins and what they can do, and what the different parameters are.

Hi,

Loving Buildkite. Originally provided the below info in a ticket, but @keithpitt asked me to open an issue here, so here it is:

I had some confusion trying to understand exactly how to properly get an environment hook set up in the docker agent, to ensure my github ssh deploy key and other secrets were properly set for a pipeline which builds and publishes a docker image.

It would be great if there were more of a "step by step" example in the docker agent README for these cases.

At least, it would be good to explain exactly where to stick the environment hook (e.g. ensuring you have mounted /buildkite/hooks with your environment hook, which can also contain a mechanism to copy your private key to the right place).

Also, I was trying to mount the entire /buildkite volume while using the 'stable' image, which didn't work because it blows away the bootstrap. So, you have to either mount only /buildkite/hooks or ensure you use the beta image.

I've documented the steps I took here, feel free to steal from them - below are links to two "Buildkite-specific" sections I have in the notes I took during my setup. They are a little specific to what I did (on Rancher), but should be a good start on a more detailed step-by-step walkthrough. They are also to specific commits, so the line numbers don't get stale, but you can look at the latest raw version on master for easier copy-paste.

https://github.com/thewoolleyman/continuous-everything/blame/5a66bb218a2230a36513171ceff0f9e9195ed55f/README.md#L72

https://github.com/thewoolleyman/continuous-everything/blame/5a66bb218a2230a36513171ceff0f9e9195ed55f/README.md#L540

Thanks,
-- Chad

Hi peeps

I'm a little confused about the environment variables & interpolation logic. For example -

env: 
  A: 'somevar'
steps;
  - name: 'test'
  env:
    A: 'anothervar'

and I fire a build and set an environment within the build of
A=foobar

What does $A result in?

It appears that the result is A=foobar.
It seems the logic is the environment within a step, and then the global environment set within a pipeline. Any pass through environment variables that are set are ignored if there is a global environment set within the pipeline.yml

I'm trying to configure a docker agent using a config file, but it seems the documented config file location is incorrect. In the documentation it says to map your config file to /buildkite/buildkite-agent.cfg

So I've got the following config file:

# file: /devops/buildkite/buildkite-agent.cfg
token="<secret>"
name="docker-test"
meta-data="ci=true,docker=true"
debug=true

And spin up a container with:
docker run -it -v "$HOME/buildkite/buildkite-agent.cfg:/buildkite/buildkite-agent.cfg:ro" buildkite/agent

It tells me: 2018-03-15 08:29:37 FATAL Missing token. See: buildkite-agent start --help

However if I run it detached and docker exec into the container to check the config file, it is correctly mounted, so that doesn't appear to be the problem.
I've also tried running buildkite/agent:3 but the issue persists

The https://buildkite.com/docs/pipelines/branch-configuration page should also provide some sample pipeline.yml with step-level branch filtering.

Hi BK team!
We're currently evaluating BuildKite to see if it's a good fit for our build-deploy process long-term. It's been a good experience so far, but right now we're in the following situation we need to account for:

1. We have an error or potential security issue with the buildkite-agent.cfg config file / environment config
2. We want to reload the hosts
3. We don't want to interrupt any builds that are in-flight

Right now our only option is to force-stop / start agents en masse, but this runs the risk of killing inflight jobs.

Is there a mechanism to either

1. Hot-reload the configuration (buildkite-agent.cfg in our case)
2. Be able to issue a command to live running agents to restart and pick up latest settings/environment/etc once the current build they're running has completed?

Thank you!

The documentation for global/build hooks mentions that:
"Each agent installer comes with a hooks directory containing a sample set of hooks"
https://buildkite.com/docs/agent/v3/hooks

But there are no sample hook files with the Docker image:
https://github.com/buildkite/docker-buildkite-agent

https://buildkite.com/docs/graphql-api#relay-compatibility mentions:

The Buildkite GraphQL API adheres to the Relay Specification, which defines standards for querying paginated collections ("Connections" and "Edges")

The pagination doesn't seem to work though. I can see that the number of builds returned is limited to 500, however the pageInfo element never sets hasNextPage to true. Even though BuildConnection.count is higher than the number of returned edges.

This may be either a lack of documentation about how to handle paging results, or a bug in the implementation.

Old

The stack connects to the Buildkite API to retrieve the autoscaling metrics to spins up and down EC2 instances based on the build activity. You’ll need to create a Buildkite API Access Token for the Elastic CI Stack to use:

New

The stack connects to the Buildkite API to gather autoscaling metrics and build activity to scale up or scale down the number of EC2 instances. You’ll need to create a Buildkite API Access Token for the Elastic CI Stack to use:

When I export a pipeline it says "label:" on the steps. https://buildkite.com/docs/guides/uploading-pipelines has something that says "name:", but never mentions "label:". Are they the same? Is there reference documentation for this somewhere?

There really should be only one place for official documentation.

For Docker it exists:
https://buildkite.com/docs/agent/v3/docker
https://github.com/buildkite/docker-buildkite-agent

Maybe just move Github docs into your official one ?

On this page here:
https://buildkite.com/docs/agent/v3/docker

It is has a link to "Ubuntu Dockerfile" which is broken.

It’d be helpful if the trigger step documentation included some mention that the user who starts the triggering build must have access to build in the triggered build.

We don't document how to configure buildkite-agent’s HTTP proxy settings. This was just submitted by a user who went through the pain of figuring it out:

## Proxy.

If you are behind a proxy server you need to create a folder "/etc/system/system/buildkite-agent.service.d" and in there create a file called proxy.conf with your proxy settings inside it. You'll need to stop and start the service for the changes to take affect.

My company is tarring up the node_modules and uploading this as an artifact. We then have to check in each build step if a node_modules exists and if not to download the artifact & untar it. None of the examples presented are even close to real world, since 1 step isn't close to how people use CI, which makes it quite difficult to understand whether this is the proper way to handle it.

Hi,

We're trying to implement elastic agents "aws", want to know how exactly the ssh pairing to be done. There is an error occurring when the build is started.

PS:FTA

https://github.com/buildkite/docs/blob/master/pages/agent/aws.md.erb

parallizing should be parallelizing or parallelising depending on your regional bias...

On the following pages:
https://buildkite.com/docs/builds/artifacts
https://buildkite.com/docs/agent/v2/cli-artifact
https://buildkite.com/docs/pipelines/defining-steps

There is either no mention (first 2 pages) or little mention (last page) of the artifact_paths property for pipelines or really automatic uploading in general.

The directions here https://buildkite.com/docs/builds/parallel-builds where it talks about running multiple agents is not accurate for newer Ubuntu systems that use systemd based configurations.

Are there updates or other documentation available?

This agent install doc: https://buildkite.com/docs/agent/v3/redhat

References: https://yum.buildkite.com/buildkite-agent/unstable/x86_64/

But I assume that since 3.0 is now stable, that should be updated. Same thing seem to be the case for most other distros.

In regards to this section:
https://buildkite.com/docs/agent/v3/cli-artifact#downloading-artifacts-outside-a-running-build

It isn't clear what this means. Does it mean:

(1) Downloading artifacts outside the current build
(2) Downloading artifacts outside any build

Stumbled upon the incredibly useful --default flag in the source. I wouldn't have been able to successfully use meta-data without it. Would be great to see it reflected in the documentation for others to benefit from it.

We need to update this for Agent v3:

https://buildkite.com/docs/builds/docker-containerized-builds

BUILDKITE_DISABLE_GIT_SUBMODULES buildkite/agent#168 isn’t listed on:

• https://buildkite.com/docs/builds/environment-variables, or
• https://buildkite.com/docs/agent/configuration

Just signed up for https://githost.io/, trying to get it integrated with BK. Have run into some issues, would certainly be good to get it mentioned on here https://buildkite.com/docs/how-tos/gitlab

I'm personally lacking of list of all possible states for Jobs.
Maybe something else isn't documented too

There is a missing image at https://buildkite.com/docs/quickstart/getting-started

Specifically email-address.png is missing, the one reference here:

<%= image "email-address.png", size: "251x52", alt: "Screenshot of email address section of Buildkite settings" %>

https://github.com/buildkite/docs/blob/master/pages/quickstart/getting_started.md.erb#L25

Currently this results in no external network connectivity, because there is no NAT gateway. This either needs to be documented or a NAT gateway conditionally provisioned.

Does BUILDKITE_BUILD_NUMBER ever wrap around or does it increase monotonically? It would be great if this could be documented. Thanks :-)

I'm looking at:
https://buildkite.com/docs/agent/hooks

An example hook is given as:

#!/bin/bash
set -eu
echo '--- :house_with_garden: Setting up the environment'
export GITHUB_RELEASE_ACCESS_KEY='xxx'

The first line "#!/bin/bash" implies this is a script which will get executed. If it is executed, the export won't do anything. A similar issue exists with the pre-command hook.
It's a small picky issue, but caused me some confusion and extra work when someone asked me to create an env hook.

We have recently provisioned a brand new Mac mini for iOS builds, and have found that the instructions to start the service in the background per https://buildkite.com/docs/agent/osx don't work:

buildkite-mac-1:~$ brew info buildkite-agent
buildkite/buildkite/buildkite-agent: stable 2.6.6, devel 3.0-beta.34
Build runner for use with Buildkite
https://buildkite.com/docs/agent
/usr/local/Cellar/buildkite-agent/2.6.6 (5 files, 9.9MB) *
  Built from source on 2017-10-26 at 07:52:25 with: --token=
From: https://github.com/buildkite/homebrew-buildkite/blob/master/buildkite-agent.rb
==> Options
--token=
	Your account's agent token to add to the config on install
--devel
	Install development version 3.0-beta.34
==> Caveats
buildkite-agent is now installed!

Configuration file (to configure agent meta-data, priority, name, etc):
    /usr/local/etc/buildkite-agent/buildkite-agent.cfg

Hooks directory (for customising the agent):
    /usr/local/etc/buildkite-agent/hooks

Builds directory:
    /usr/local/var/buildkite-agent/builds

Log paths:
    /usr/local/var/log/buildkite-agent.log
    /usr/local/var/log/buildkite-agent.error.log

If you set up the LaunchAgent, set your machine to auto-login as
your current user. It's also recommended to install Caffeine
(http://lightheadsw.com/caffeine/) to prevent your machine from going to
sleep or logging out.

To run multiple agents simply run the buildkite-agent start command
multiple times, or duplicate the LaunchAgent plist to create another
that starts on login.

To have launchd start buildkite/buildkite/buildkite-agent now and restart at login:
  brew services start buildkite/buildkite/buildkite-agent
Or, if you don't want/need a background service you can just run:
  buildkite-agent start

Following the instructions yields:

buildkite-mac-1:~$ brew services start buildkite/buildkite/buildkite-agent
Could not find domain for
Error: Failure while executing: /bin/launchctl enable gui/502/homebrew.mxcl.buildkite-agent

Its unclear how to install v3 from the Manual installation instructions.

Specifically, where does the bootstratp.sh come from? Or is this relevant to v3

In the docs we show how to run multiple agents but we don't show any way to customise the agent configuration files for each agent… say, for giving them different tags/meta-data.

Someone just submitted the process it took for them:

To run multiple agents, on the same host, with different configurations, and behind a proxy server.

Once you have installed and tested an agent as per the installation instructions.

## Kill the single agent.

Stop the service and disable it "sudo systemctl stop buildkite-agent" and "sudo systemctl disable buildkite-agent"

## Config Files.

In the buildkite-agent home directory, `/ect/buildkite-agent` make a copy of buildkite-agent.config for each different config you want, e.g. `buildkite-agent-1.cfg`, `buildkite-agent-2.cfg`, etc. The name is irrelevant so call it what you like.

Then edit the new config files and set whatever configuration you want - I just set different queue names in the meta-data/tags.

## System launch file.

This file is `/lib/system/system/buildkite-agent.service`

Make a copy for each agent you want to run, i.e. `[email protected]`, `[email protected]`, etc., and edit this file.

There is an ExecStart line that launches the service, add to the end of this, in each file, ` -config /etc/buildkite-agent/buildkite-agent-1.cfg` or whichever conf you want to use.

Now you can enable the service, `sudo systemctl enable buildkite-agent@1`, and it should run.

## Repeat for however many agent you have setup.

This creates a symlink called `/etc/system/system/mutli-user.target.wants/[email protected]` that should link to the system launch file `/lib/sytemd/system/[email protected]` that was created earlier.

## Start the agents

Finally start the agents: `sudo systemctl start buildkite-agent@1`

Referring to this page:
https://buildkite.com/docs/agent/v3/cli-artifact#uploading-artifacts

Within the "Uploading artifacts" section it shows this:
$ export BUILDKITE_S3_ACCESS_KEY_ID=xxx $ export BUILDKITE_S3_SECRET_ACCESS_KEY=yyy $ export BUILDKITE_S3_DEFAULT_REGION=eu-central-1 # default is us-east-1 $ export BUILDKITE_S3_ACL=private # default is public-read $ buildkite-agent artifact upload "log/**/*.log" s3://name-of-your-s3-bucket/$BUILDKITE_JOB_ID

But then further down in the "Using your own private AWS S3 bucket" section it says:

"To do so, you’ll need to export the following environment variables using an environment agent hook (this can not be set via the Buildkite web interface, API, or during pipeline upload):"

export BUILDKITE_ARTIFACT_UPLOAD_DESTINATION="s3://name-of-your-s3-bucket/$BUILDKITE_JOB_ID" export BUILDKITE_S3_DEFAULT_REGION="eu-central-1" # default: us-east-1

Do I need to use an environment agent hook or can I just export variables from the pipeline code ?