If the agent has a hard time finding the job to run, it should try the call a few more times before giving up.

I don't see any documentation anywhere besides the README file.
Is there one yet?

It's currently UNIX specific.

At https://github.com/buildbox/buildbox-agent it says "The biggest change you'll notice is that you no longer define your build scripts on Buildbox. You instead should write these scripts and save them to your projects source control." and then "Next, go to your Project Settings on Buildbox and update the "Script Path" field with the path to your new build script (here we used scripts/buildbox.sh)."

I can't find a "scripts path" anywhere in my project settings... am I missing something?

--bootstrap-script="~/.boo..." needs to expand the path

I think you need an AgentDisconnect call here:

https://github.com/buildbox/agent/blob/master/buildbox/agent.go#L144

Like:

https://github.com/buildbox/agent/blob/master/buildbox/agent.go#L89

Or maybe disconnection isn't working, because exiting the agent isn't removing it from the interface for me.

Running an agent with something like:

buildbox agent start --token [token] --meta-data ruby=2.1.2 --meta-data ruby=2.0.0 --meta-data mysql=5.5

Yields only the following metadata in the interface:

ruby=2.0.0
mysql=5.5

And won't run builds configure for:

ruby=2.1.2

Hi,

I encountered a situation where I inadvertently started up two copies of the buildbox-agent on the same system, using the same access token. Both appeared to be able to start builds, and there was of course a high potential to conflict if they built the same project.

It would be good if a second agent using an access token that's currently in use by another agent would get rejected.

It would also be good if the log file output would note the PID so this situation can be spotted easily while tailing logs.

If a build is running, you can't edit bootstrap.sh because the file is in use.

I have an agent running on OS X Mavericks. It was originally running the Ruby agent, using a launch daemon .plist that I think I might have written. Here's the contents:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>io.buildbox.buildbox-ruby</string>

  <key>UserName</key>
  <string>_buildbox</string>

  <key>ProgramArguments</key>
  <array>
    <string>/usr/bin/buildbox</string>
    <string>agent:start</string>
  </array>

  <key>KeepAlive</key>
  <true/>

  <key>RunAtLoad</key>
  <true/>

  <key>StandardOutPath</key>
  <string>/var/log/buildbox.log</string>

  <key>StandardErrorPath</key>
  <string>/var/log/buildbox.log</string>

  <key>EnvironmentVariables</key>
  <dict>
    <key>PATH</key>
    <string>/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin</string>
  </dict>
</dict>
</plist>

This worked great. Now I'm upgrading to this Golang agent, so I installed it as the _buildbox user. I modified the launch daemon:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>io.buildbox.buildbox-golang</string>

  <key>UserName</key>
  <string>_buildbox</string>

  <key>ProgramArguments</key>
  <array>
    <string>/Users/_buildbox/.buildbox/buildbox-agent</string>
    <string>start</string>
  </array>

  <key>KeepAlive</key>
  <true/>

  <key>RunAtLoad</key>
  <true/>

  <key>StandardOutPath</key>
  <string>/var/log/buildbox.log</string>

  <key>StandardErrorPath</key>
  <string>/var/log/buildbox.log</string>

  <key>EnvironmentVariables</key>
  <dict>
    <key>PATH</key>
    <string>/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin</string>

    <key>BUILDBOX_AGENT_ACCESS_TOKEN</key>
    <string>secret</string>
  </dict>
</dict>
</plist>

The agent starts up and waits for work. When I trigger a build, the interface on the website indicates that the agent started running the job (the job turns yellow). But there's never any build output, or activity on the machine. The log file contains:

2014/05/08 12:54:17 Started buildbox-agent `Beaver` (version 0.1)
2014/05/08 12:54:24 Starting job f1febf1f-5845-4e52-8ef9-5466f1982aad
2014/05/08 12:54:25 Running script `bootstrap.sh` from inside /Users/_buildbox/.buildbox
2014/05/08 12:54:27 Started buildbox-agent `Beaver` (version 0.1)

launchd says the agent died, so it respawns it:

May  8 12:54:25 Beaver com.apple.launchd[1] (io.buildbox.buildbox-golang[1294]): Exited with code: 2
May  8 12:54:25 Beaver com.apple.launchd[1] (io.buildbox.buildbox-golang): Throttling respawn: Will start in 1 seconds

When I run the agent as the _buildbox user in the foreground, it works fine.

https://github.com/buildbox/agent/blob/master/templates/bootstrap.sh#L144

These sorts of failure scenarios where exit or buildbox-exit-if-failed is called can result in rogue containers staying around and not being cleaned up. I.e fig kill and fig rm are not called since it exited.

Also if a build from the UI is cancelled then depending on where in the build step it is performed, it can also leave around containers.

Chris suggested a trap finish EXIT with a finish function declared as a possible solution to this particular problem.

It should check in "~/.buildbox/bootstrap.sh"

Really not a big one, but:

$ buildbox-agent artifact upload help
buildbox: missing agent access token
See 'buildbox artifact download --help'

upload/download mixed up.

When the build script finishes, an io.Copy error is expected, we shouldn't show it in the logs in this case.

All of our systems are managed via chef. Installing packages is much easier than piping curl to sh, and much easier to make idempotent.

The S3 uploader seems to default to public-read, which is not always desirable.

Seeing this in the log file:

2014-09-04 07:56:43 [INFO ] Starting job 4b797e6e-dd55-4960-a905-770150b3d5d7
2014-09-04 07:56:44 [INFO ] Starting to run script: /opt/buildbox/bootstrap.sh
2014-09-04 07:56:44 [INFO ] Process is running with PID: 23709
2014-09-04 07:57:04 [ERROR] io.Copy failed with error: *os.PathError: read /dev/ptmx: input/output error
2014-09-04 07:57:04 [INFO ] Finished job 4b797e6e-dd55-4960-a905-770150b3d5d7

It seems like the default bootstrap.sh is not behaving well on CentOS 7. I don't have time to debug it, but it gets stuck on the "RSA key fingerprint" step. The "yes yes" piping does not help. The solution is to do this:

ssh -o StrictHostKeyChecking=no github.com ls

Which essentially tries to login without strict key checking, which will populate known_hosts file. Just pipe the output of IN/ERR to /dev/null, because the command will error out.

Hi all,

Should the agent support pull requests for remote repositories? Personally, I think it should.

I've noticed that the pull request phonegap/ios-deploy/pull/68 is failing because the commit cannot be found. From the looks of it, the agent is trying to checkout the commit from the origin repository, rather than adding the remote's repository.

I haven't dug into the code, but I'm open to adding support for remotes. However, I want to make sure this is a legit bug and desired feature of the agent :)

The OSX buildbox-agent/1.0-beta.6 (darwin; 386) stops working after a laptop suspend+resume cycle.
It doesn't exit, it just sits there quietly and wont show up in the UI agent list.

Since always-on OSX hardware is a bit scarce this is a big show stopper for me.

We are running the latest buildbox go-agent.
Currently observing the following messages on the agent output:

Problem with updating job ded6a18c-93f4-44aa-812a-5b58ba05c0bc%!(EXTRA *url.Error=Put https://agent.buildbox.io/v1/6145c05b9566560d1d54187414819cd2eac222500ab27a3776/jobs/ded6a18c-93f4-44aa-812a-5b589f05c0bc: x509: certificate signed by unknown authority (possibly because of "x509: cannot verify signature: algorithm unimplemented" while trying to verify candidate authority certificate "COMODO RSA Certification Authority"))

This seems to put all builds under the "wait" status.

Your docker & fig related clean up is missing some things. You should use --rm here or provide an option to do so. If you do not use --rm here containers will build up on the host machine and may cause problems if people ever try docker rmi. You may also want to consider adding -t. I haven't seen any harm come from this and it will make people's test out put more similar their development environment. You should consider using -v here. -v will also remove docker volumes. It's unfortunate it's disable by default. Without -v the docker VFS will slowly grow over time (depending on how many images have volumes of course). This can create real pain over time because you run out space for when you don't expect it. This problem is definitely more pronounced on CI machines that have high container churn rate. The same issues apply to the default fig commands. fig run should most likely have --rm and fig rm should have -v as well. Also if you use --rm on fig run you do not need to trap and clean up. I offer these suggestion from my experience using docker/fig and various combinations for them for CI things in various arrangements. You maybe able to save less knowledgeable customers some pain.

I get the following failure:

-- https://buildbox.io

Installing Version: v0.1
Destination: /home/ubuntu/.buildbox

Downloading https://github.com/buildboxhq/buildbox-agent/releases/download/v0.1/buildbox-agent-linux-amd64.tar.gz
chmod: cannot access ‘/home/ubuntu/.buildbox/buildbox-data’: No such file or directory

When I created /home/ubuntu/.buildbox/buildbox-data it works.

I know it's probably on your TODO, but as a reminder the calls to buildbox artifact need to be updated:

https://github.com/buildbox/agent/blob/v1.0-beta.1/templates/bootstrap.sh#L62-L82

Just fixed these in our bootstrap. :-)

It would be great if we could set a maximum build time and automatically cancel and kill a build if it goes over this time. We've got a few builds that spawn parallel processes and occasionally one of these hangs, leaving us in a situation where a build goes on indefinitely until it's manually cancelled.

Check the response headers for the latest version, and compare.

This initially seemed similar to #30 however I am not seeing any certificate errors, and I've also upgraded to the version listed in this issue (0.2-beta.3) but I am still seeing the issue.

Basically it seems that artifacts are uploaded to S3 (Amazon) correctly but the process never finishes, and builds will hang forever on this step. Running an strace against the process doesn't yield anything particularly useful, other than it looping through some kind of wait state.

The agent should handle bad responses from the API. It shouldn't crash the agnet.

I have an instance of the agent running on OSX Mavericks. It starts up fine and will pick up a build, but it dies right after logging "running bootstrap.sh" in the log file. No other errors are reported.

It would be great if there was a way to only expand, or auto-expand, the sections of the build output where errors have occurred.

As far as implementing it goes, perhaps another special string can be checked for by buildbox for sections of interest, similar to the way buildbox currently looks for ---, e.g. if the build script hits an error it could trap and output something *** or similar, and buildbox would treat that as an erroring section.

Having an issue with subfolders uploading:
https://buildbox.io/ferocia/johanna/builds/4077

Say I have two agents with different versions of mysql:

buildbox agent start --token [token] --meta-data mysql=5.5
buildbox agent start --token [token] --meta-data mysql=5.1

And I only care about having mysql in general. Maybe in my project settings I could ask for meta data:

mysql=*

And it will run on either?

(I tried this and it doesn't currently work.)

After an otherwise successful build the elastic agent complains about missing buildbox-artifact:

ERROR: buildbox-artifact could not be found in /home/maglun/.buildbox

buildbox-agent version: buildbox-agent/1.0-beta.6 (darwin; 386)

With many, many small files to upload, buildbox-artefact produces the following for me:

2014-06-19 21:04:57 [INFO ] Found 9840 files that match "results/**/*; log/**/*; tmp/capybara/**/*" 
2014-06-19 21:05:56 [FATAL] Failed to upload artifacts: Unexpected error: 504 Gateway Timeout 

(Example build id: "f9b0776ae49392e31dbff3ae6663076a9f54ab9a")

Dpl is an open source software made by Travis that allows you to deploy to multiple targets including S3.
It's completely open source so there is no reason why you shouldn't.

Most build services nowadays use a YAML file in order to generate the bootstrap.sh script.
You can ask the travis support team to generate it for you but I don't see why you should.

The agent needs to be more aggressive in cancelling processes. If it doesn't die after (x) seconds, start -9 killing it.

Hi,

It would be great if we could introduce a on-build-failure step so we can add things like notifications or cleanup steps.

Case study: we had some code merged that left some assets that subsequently broke every build after that. I had to (temporarily) add an asset wiper to the bootstrap script to get around this, however we don't generally want to wipe the assets as these take time to generate and don't need to be completely recreated all the time. If there was an on-fail step we could add the asset wiper into this and only do this type of cleanup when a build has a problem. It would of course need to run on the same CI boxes that ran the broken build step(s).

Thanks.

I know we all hate how github/tools format code with tabs but it would be nicer for contributors if this code was in line with golang formatting.

The reason I mention this is while hacking around on this project to try out some refactoring I discovered I pretty much had to disable every code check tool I have to avoid making massive white space changes. This in turn led to the "big list of compile issues" I used to get..

Note I am interested to hear what tools your using now, and why this code base is formatted the way it is, however it is pretty frustrating to edit with standard tools at the moment.

basically

buildbox-run "git checkout -qf \"$BUILDBOX_COMMIT\""
buildbox-run "git submodule update --init"

I can't use the value ruby=1.9.3+envato4, maybe an issue with the plus symbol?

Any logging or debug-type output from buildbox-agent commands should go to STDERR so that var setting (MY_VAR=$(buildbox-agent build-data get my-var) and redirection (buildbox-agent build-data get my-var > my-var.txt) still work as expected and the logging / debug output still shows up on the console running the script.

• Figure out the best way to install the agent on a windows machine
• Easy way to enter the agent token, it'd suck to have to "right click -> command line options" type deal

Some of the windows/unix code is duplicated. It'd be nice if it was cleaned up a little bit.

The default bootstrap.sh does not support changing git remotes. It will only check the existence of a .git folder and not if it is an related repository.

(I changed from github to gitlab with a project, had to destroy all .git folders to recover)

Remotes could be grepped from running git remotes -v in the clone condition.