square / kochiku Goto Github PK
View Code? Open in Web Editor NEWShard your builds for fun and profit
License: Apache License 2.0
Shard your builds for fun and profit
License: Apache License 2.0
The wiki lists the production filename as
/app/kochiku/secrets/kochiku-secret.yml
but it should be
/app/kochiku/secrets/kochiku-github.yaml
Is it possible to support postgresql?
Curently kochiku derives the project name from the repository name. This breaks, with no workaround that I could find, if two repositories would produce the same name. For example, if you have a [email protected]:legacy/dashboard.git
and a [email protected]:hotness/dashboard.git
and you add both, kochiku will not show both of them on the home page.
How would you recommend handling authentication for private GH repos during the clone operations for both the kochiku server and workers? The GitHub remote server model (https://github.com/square/kochiku/blob/bea210e11d350e3f01495996d71e14078e189a57/app/models/remote_server/github.rb) doesn't accommodate for a password file and auth methods like the stash version.
I have also found that SSH key-based authentication with GH doesn't work in the context of the resque workers.
Thanks in advance for your insight.
Hello,
I am trying to make Kochiku CI work for a git repository that contains a submodule
.
For some reason, the BuildPartioningJob
fails. Here is what the Resq
error queue gives me:
non-0 exit code pid 13187 exit 1 returned from [git submodule --quiet update]
/app/kochiku/kochiku/releases/20130919182453/lib/git_repo.rb:76:in `run!'
/app/kochiku/kochiku/releases/20130919182453/lib/git_repo.rb:30:in `block (2 levels) in inside_copy'
/app/kochiku/kochiku/releases/20130919182453/lib/git_repo.rb:17:in `chdir'
/app/kochiku/kochiku/releases/20130919182453/lib/git_repo.rb:17:in `block in inside_copy'
/usr/local/rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/tmpdir.rb:88:in `mktmpdir'
/app/kochiku/kochiku/releases/20130919182453/lib/git_repo.rb:13:in `inside_copy'
/app/kochiku/kochiku/releases/20130919182453/app/jobs/build_partitioning_job.rb:17:in `perform'
/app/kochiku/kochiku/releases/20130919182453/app/jobs/job_base.rb:22:in `perform'
I can trigger a green build
for my repository without the submodule.
Any idea on what could go wrong? Do I need any special configuration to make Kochiku work with submodules?
Thanks
Currently Kochiku sends a build failure email after all build parts have completed. This means the email can get held up by a really long running build part. Developers would like quicker feedback, therefore we should switch from a build failed email to a build is doomed email.
Perhaps provide this as an option at the repository level. Short builds might prefer the current behavior whereas long builds would not.
I'm trying to do a deploy and i'm getting the following:
** transaction: start
*** [err :: localhost] cp:
*** [err :: localhost] cannot create directory `/home/kochiku-web/kochiku/releases/20130908064030'
*** [err :: localhost] : No such file or directory
*** [err :: localhost]
*** [deploy:update_code] rolling back
failed: "env PATH=/usr/local/bin:$PATH sh -c 'cp -RPp /home/kochiku-web/kochiku/shared/cached-copy /home/kochiku-web/kochiku/releases/20130908064030 && (echo d7c117f > /home/kochiku-web/kochiku/releases/20130908064030/REVISION)'" on localhost
I'm installing on localhost just to test what exactly is deployed and installed.
I know that the cap environment can create directories because it created /home/kochiku-web/shared and all of it's sub-directories.
I worked around the issue by creating the releases/ directory manually.
If, for example, a large part of your network becomes unavailable and jobs get dropped, it would be nice for kochiku to not consider those jobs still "pending" when they will in fact never be picked up. I'd probably set this to like 1-2 hours.
What about recipe to automate server deployment?
Which system community prefer?
If chef is good enough I can write something like this recipe .
The implementation of individual log_file_globs
for each target inside of a kochiku.yml does not currently work. The last log file glob specified will always win and overwrite all of the others.
Example:
test_command: 'script/ci'
targets:
- type: unit
glob: spec/unit/**/*_spec.rb
log_file_globs:
- log/test.log
- type: integration
glob: spec/integration/**/*_spec.rb
log_file_globs:
- log/integration.log
Here the unit target will receive a log_file_globs value of "log/integration.log" because that is the last one specified.
This will be difficult to fix so I propose removing the log_file_globs key from the targets section. Users can specify all of their log files in the global key instead.
This is a problem when you work at, say, Square and you use Stash which has a different URL scheme.
I have to delete the build from mysql before it will work again.
*** [err :: localhost] Rails Error: Unable to access log file. Please ensure that /home/kochiku-web/kochiku/releases/20130908071331/log/production.log exists and is chmod 0666. The log level has been raised to WARN and the output directed to STDERR until the problem is fixed.
*** [err :: localhost] rake aborted!
*** [err :: localhost] No such file or directory - /home/kochiku-web/kochiku/releases/20130908071331/log/bullet.log
why does it not create the log directory itself?
In a build with 100% completion rate, currently-active first-try builds on the branch will count against this metric while in-progress (dropping the percentage below 100%) until they eventually pass. Build parts who are in-progress on their first attempt should not be counted against this metric, only those which are on their second or higher attempt.
Kochiku workers have been updated to support this however I am 99% certian that the mainline repository does not enqueue a job with the correct (ie the fork) repository url.
I have a branch that starts to add support for this however I haven't had a chance to finish it, wanted to make an issue to record this problem for the public.
The Elapsed Time column seems to increment, but the status does not actually update. I'm pretty sure the Elapsed Time column doesn't increment correctly either (the minute is off).
Steps to reproduce:
The background jobs should poll QUEUE=partition. Would be great to add some example on how to set it up that would go along with what happens in config/deploy.rb
.
Currently click the "Abort" button on a build it goes into the aborted state permanently. Kochiku should allow the user to reverse this action if they wish by clicking a "Reactivate build" button. Once the build is active again they can enqueue build parts.
I think they changed the url format from api.github.com/api/v3 to just api.github.com. There needs to be a change from enterprise to regular github. It worked when I hacked in the url locally. I'll try and do a proper PR in a bit.
during the deploy I got this error:
*** [err :: localhost] rake aborted!
*** [err :: localhost] Could not find a JavaScript runtime. See https://github.com/sstephenson/execjs for a list of available runtimes.
*** [err :: localhost] /home/kochiku-web/kochiku/shared/bundle/ruby/2.0.0/gems/execjs-1.4.0/lib/execjs/runtimes.rb:51:in `autodetect'
I could not find anything in the install docs mentioning the need for execjs
If the service hook ID stored in the database ends up wrong (say because the hooks were edited by hand), the attempt to update the hook will fail and builds will not proceed.
response: Net::HTTPUnprocessableEntity body: {"message":"Validation Failed","errors":[{"resource":"Hook","code":"custom","message":"Hook already exists on this repository"}]}
... TEST_RUNNER=build --parallel ...
env: --parallel: No such file or directory
when specifying
type: "build --parallel"
This feature moved into the kochiku.yml config file last summer. Anyone using it has had time to migrate so we can drop the column from the table and any code that references it now.
Sending a separate email for an auto-merged build in addition to the success email is noisy (especially when you get a third email from the code review tool as well). The success email should be augmented to specify the fact that a build was also merged.
Currently if you don't specify a partitioning strategy in your kochiku.yml, your parts will be partitioned with the "alphabetically" strategy. I am thinking that the "round_robin" strategy makes more sense as the default.
The problem that alphabetically suffers from is that, for instance, if you have the same number of model and controller test, the models test will be in one part and the controller test in the other. If all of your controller test are slow and your models test are fast this defeats the partitioning strategy. "round_robin" would equally distribute the model and controller tests across the build parts.
I think it'd be a good idea that we can add a default javascript runtime like 'therubyracer' which makes the installation a bit more flawlessly.
And a person wants to change it to something else, they know what they are doing.
That's just my personal opinion :)
Become more responsive installation by users.
Thanks!
I forced the deploy to use a release dir of "bear" and created the log directory just so I could get past the previous blocking point, but now I get this:
deploy:finalize_update' triggering before callbacks for
deploy:finalize_update'Merge on Success uses Kochiku Robot as the author for the merge commit, it should use the author from the commits that it is merging in.
Since it shows that the author of the merge is Kochiku Robot, a lot of git tools like git blame and IntelliJ's Annotate Lines become much less useful when your team members are using Merge on Success:
This should be pretty simple to solve, git authors can be set to whatever you like with the flag --author="John Doe <[email protected]>"
in most git commands.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.