The wiki lists the production filename as

/app/kochiku/secrets/kochiku-secret.yml

but it should be

/app/kochiku/secrets/kochiku-github.yaml

Is it possible to support postgresql?

The implementation of individual log_file_globs for each target inside of a kochiku.yml does not currently work. The last log file glob specified will always win and overwrite all of the others.

Example:

test_command: 'script/ci'
targets:
  - type: unit
    glob: spec/unit/**/*_spec.rb
    log_file_globs:
      - log/test.log
  - type: integration
    glob: spec/integration/**/*_spec.rb
    log_file_globs:
      - log/integration.log

Here the unit target will receive a log_file_globs value of "log/integration.log" because that is the last one specified.

This will be difficult to fix so I propose removing the log_file_globs key from the targets section. Users can specify all of their log files in the global key instead.

If the service hook ID stored in the database ends up wrong (say because the hooks were edited by hand), the attempt to update the hook will fail and builds will not proceed.

response: Net::HTTPUnprocessableEntity body: {"message":"Validation Failed","errors":[{"resource":"Hook","code":"custom","message":"Hook already exists on this repository"}]}

How would you recommend handling authentication for private GH repos during the clone operations for both the kochiku server and workers? The GitHub remote server model (https://github.com/square/kochiku/blob/bea210e11d350e3f01495996d71e14078e189a57/app/models/remote_server/github.rb) doesn't accommodate for a password file and auth methods like the stash version.

I have also found that SSH key-based authentication with GH doesn't work in the context of the resque workers.

Thanks in advance for your insight.

Hello,
I am trying to make Kochiku CI work for a git repository that contains a submodule.

For some reason, the BuildPartioningJob fails. Here is what the Resq error queue gives me:

non-0 exit code pid 13187 exit 1 returned from [git submodule --quiet update]
/app/kochiku/kochiku/releases/20130919182453/lib/git_repo.rb:76:in `run!'
/app/kochiku/kochiku/releases/20130919182453/lib/git_repo.rb:30:in `block (2 levels) in inside_copy'
/app/kochiku/kochiku/releases/20130919182453/lib/git_repo.rb:17:in `chdir'
/app/kochiku/kochiku/releases/20130919182453/lib/git_repo.rb:17:in `block in inside_copy'
/usr/local/rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/tmpdir.rb:88:in `mktmpdir'
/app/kochiku/kochiku/releases/20130919182453/lib/git_repo.rb:13:in `inside_copy'
/app/kochiku/kochiku/releases/20130919182453/app/jobs/build_partitioning_job.rb:17:in `perform'
/app/kochiku/kochiku/releases/20130919182453/app/jobs/job_base.rb:22:in `perform'

I can trigger a green build for my repository without the submodule.
Any idea on what could go wrong? Do I need any special configuration to make Kochiku work with submodules?
Thanks

This feature moved into the kochiku.yml config file last summer. Anyone using it has had time to migrate so we can drop the column from the table and any code that references it now.

This is a problem when you work at, say, Square and you use Stash which has a different URL scheme.

In a build with 100% completion rate, currently-active first-try builds on the branch will count against this metric while in-progress (dropping the percentage below 100%) until they eventually pass. Build parts who are in-progress on their first attempt should not be counted against this metric, only those which are on their second or higher attempt.

during the deploy I got this error:

*** [err :: localhost] rake aborted!
*** [err :: localhost] Could not find a JavaScript runtime. See https://github.com/sstephenson/execjs for a list of available runtimes.
*** [err :: localhost] /home/kochiku-web/kochiku/shared/bundle/ruby/2.0.0/gems/execjs-1.4.0/lib/execjs/runtimes.rb:51:in `autodetect'

I could not find anything in the install docs mentioning the need for execjs

Sending a separate email for an auto-merged build in addition to the success email is noisy (especially when you get a third email from the code review tool as well). The success email should be augmented to specify the fact that a build was also merged.

What about recipe to automate server deployment?

Which system community prefer?

If chef is good enough I can write something like this recipe .

I'm trying to do a deploy and i'm getting the following:

** transaction: start
*** [err :: localhost] cp:
*** [err :: localhost] cannot create directory `/home/kochiku-web/kochiku/releases/20130908064030'
*** [err :: localhost] : No such file or directory
*** [err :: localhost]
*** [deploy:update_code] rolling back
failed: "env PATH=/usr/local/bin:$PATH sh -c 'cp -RPp /home/kochiku-web/kochiku/shared/cached-copy /home/kochiku-web/kochiku/releases/20130908064030 && (echo d7c117f > /home/kochiku-web/kochiku/releases/20130908064030/REVISION)'" on localhost

I'm installing on localhost just to test what exactly is deployed and installed.

I know that the cap environment can create directories because it created /home/kochiku-web/shared and all of it's sub-directories.

I worked around the issue by creating the releases/ directory manually.

*** [err :: localhost] Rails Error: Unable to access log file. Please ensure that /home/kochiku-web/kochiku/releases/20130908071331/log/production.log exists and is chmod 0666. The log level has been raised to WARN and the output directed to STDERR until the problem is fixed.
*** [err :: localhost] rake aborted!
*** [err :: localhost] No such file or directory - /home/kochiku-web/kochiku/releases/20130908071331/log/bullet.log

why does it not create the log directory itself?

When multiple parts fail there are specific ones which I know are important to address first but there's no way to know which link is which in the email.

I think it'd be a good idea that we can add a default javascript runtime like 'therubyracer' which makes the installation a bit more flawlessly.
And a person wants to change it to something else, they know what they are doing.

That's just my personal opinion :)

Currently if you don't specify a partitioning strategy in your kochiku.yml, your parts will be partitioned with the "alphabetically" strategy. I am thinking that the "round_robin" strategy makes more sense as the default.

The problem that alphabetically suffers from is that, for instance, if you have the same number of model and controller test, the models test will be in one part and the controller test in the other. If all of your controller test are slow and your models test are fast this defeats the partitioning strategy. "round_robin" would equally distribute the model and controller tests across the build parts.

Become more responsive installation by users.

Thanks!

... TEST_RUNNER=build --parallel ...
env: --parallel: No such file or directory

when specifying

type: "build --parallel"

I think they changed the url format from api.github.com/api/v3 to just api.github.com. There needs to be a change from enterprise to regular github. It worked when I hacked in the url locally. I'll try and do a proper PR in a bit.

I have to delete the build from mysql before it will work again.

Steps to reproduce:

1. create a new build
2. one of the build attempts fails or errors.
3. delete the commit from Git (force push over it)
4. click rebuild in the kochiku UI
5. a new "runnable" build attempt is enqueued but never makes it into the resque queue because kochiku encounters an error. The error does not get propagated.

Currently click the "Abort" button on a build it goes into the aborted state permanently. Kochiku should allow the user to reverse this action if they wish by clicking a "Reactivate build" button. Once the build is active again they can enqueue build parts.

Currently Kochiku sends a build failure email after all build parts have completed. This means the email can get held up by a really long running build part. Developers would like quicker feedback, therefore we should switch from a build failed email to a build is doomed email.

Perhaps provide this as an option at the repository level. Short builds might prefer the current behavior whereas long builds would not.

Merge on Success uses Kochiku Robot as the author for the merge commit, it should use the author from the commits that it is merging in.
Since it shows that the author of the merge is Kochiku Robot, a lot of git tools like git blame and IntelliJ's Annotate Lines become much less useful when your team members are using Merge on Success:

This should be pretty simple to solve, git authors can be set to whatever you like with the flag --author="John Doe <[email protected]>" in most git commands.

I forced the deploy to use a release dir of "bear" and created the log directory just so I could get past the previous blocking point, but now I get this:

• 2013-09-09 08:48:26 executing `deploy:cold'
• 2013-09-09 08:48:26 executing `deploy:update'
  ** transaction: start
• 2013-09-09 08:48:26 executing `deploy:update_code'
  updating the cached checkout on all servers
  executing locally: "git ls-remote https://github.com/square/kochiku.git master"
  command finished in 419ms
• executing "if [ -d /home/kochiku-web/kochiku/shared/cached-copy ]; then cd /home/kochiku-web/kochiku/shared/cached-copy && git fetch -q origin && git fetch --tags -q origin && git reset -q --hard d7c117f && git clean -q -d -x -f; else git clone -q -b master https://github.com/square/kochiku.git /home/kochiku-web/kochiku/shared/cached-copy && cd /home/kochiku-web/kochiku/shared/cached-copy && git checkout -q -b deploy d7c117f; fi"
  servers: ["localhost"]
  [localhost] executing command
  command finished in 1666ms
  copying the cached version to /home/kochiku-web/kochiku/releases/bear
• executing "cp -RPp /home/kochiku-web/kochiku/shared/cached-copy /home/kochiku-web/kochiku/releases/bear && (echo d7c117f > /home/kochiku-web/kochiku/releases/bear/REVISION)"
  servers: ["localhost"]
  [localhost] executing command
  command finished in 49ms
• 2013-09-09 08:48:29 executing deploy:finalize_update' triggering before callbacks fordeploy:finalize_update'
• 2013-09-09 08:48:29 executing `deploy:assets:symlink'
• executing "ls -x /home/kochiku-web/kochiku/releases"
  servers: ["localhost"]
  [localhost] executing command
  command finished in 18ms
• executing "rm -rf /home/kochiku-web/kochiku/releases/bear/public/assets && mkdir -p /home/kochiku-web/kochiku/releases/bear/public && mkdir -p /home/kochiku-web/kochiku/shared/assets && ln -s /home/kochiku-web/kochiku/shared/assets /home/kochiku-web/kochiku/releases/bear/public/assets"
  servers: ["localhost"]
  [localhost] executing command
  command finished in 24ms
• 2013-09-09 08:48:29 executing `bundle:install'
• executing "cd /home/kochiku-web/kochiku/releases/bear && bundle install --gemfile /home/kochiku-web/kochiku/releases/bear/Gemfile --path /home/kochiku-web/kochiku/shared/bundle --deployment --quiet --without development test"
  servers: ["localhost"]
  [localhost] executing command
  ** [out :: localhost] The --deployment flag requires a Gemfile.lock. Please make sure you have checked
  ** [out :: localhost] your Gemfile.lock into version control before deploying.
  command finished in 354ms
  *** [deploy:update_code] rolling back
• executing "rm -rf /home/kochiku-web/kochiku/releases/bear; true"
  servers: ["localhost"]
  [localhost] executing command
  command finished in 25ms
  failed: "env PATH=/usr/local/bin:$PATH sh -c 'cd /home/kochiku-web/kochiku/releases/bear && bundle install --gemfile /home/kochiku-web/kochiku/releases/bear/Gemfile --path /home/kochiku-web/kochiku/shared/bundle --deployment --quiet --without development test'" on localhost

Kochiku workers have been updated to support this however I am 99% certian that the mainline repository does not enqueue a job with the correct (ie the fork) repository url.

I have a branch that starts to add support for this however I haven't had a chance to finish it, wanted to make an issue to record this problem for the public.

The Elapsed Time column seems to increment, but the status does not actually update. I'm pretty sure the Elapsed Time column doesn't increment correctly either (the minute is off).

Curently kochiku derives the project name from the repository name. This breaks, with no workaround that I could find, if two repositories would produce the same name. For example, if you have a [email protected]:legacy/dashboard.git and a [email protected]:hotness/dashboard.git and you add both, kochiku will not show both of them on the home page.

The background jobs should poll QUEUE=partition. Would be great to add some example on how to set it up that would go along with what happens in config/deploy.rb.

If, for example, a large part of your network becomes unavailable and jobs get dropped, it would be nice for kochiku to not consider those jobs still "pending" when they will in fact never be picked up. I'd probably set this to like 1-2 hours.