Git Product home page Git Product logo

simplecov-buildkite's Introduction

SimpleCov::Buildkite

Gem Version Build Status

Generate Buildkite annotations from your SimpleCov coverage reports when running your build on Buildkite.

Installation

Add this line to your application's Gemfile, after simplecov:

gem "simplecov"
gem "simplecov-buildkite"

And then execute:

$ bundle

Or install it yourself as:

$ gem install simplecov-buildkite

Usage

Use it alongside your favourite formatter. For example, in Rails with RSpec, add it in your spec_helper.rb like this:

# spec/spec_helper.rb

require "simplecov"
require "simplecov-buildkite"

SimpleCov.start "rails" do
  load_profile "buildkite"

  formatter SimpleCov::Formatter::MultiFormatter.new([
    SimpleCov::Formatter::HTMLFormatter,
    SimpleCov::Buildkite::AnnotationFormatter,
  ])
end

When run on Buildkite with the "buildkite" profile enabled, this will also output a pretty Buildkite annotation, with coverage change breakdowns for the current PR or branch and commit:

Buildkite build showing a SimpleCov report in a Buildkite annotation

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/ticky/simplecov-buildkite. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the SimpleCov::Buildkite project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

simplecov-buildkite's People

Contributors

dependabot[bot] avatar sj26 avatar ticky avatar timriley avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

simplecov-buildkite's Issues

Quiet output running locally

I'm loving having annotations in my Buildkite builds, but is there a way to disable output at the command line when not running this within a buildkite session?

Or perhaps just recommend in the README to only invoke this inside buildkite, perhaps with a custom wrapper script?

Gem not working with ruby < 2.5

I'm in a project with ruby 2.4 and I'm getting

SyntaxError:
   /opt/whatever/webapp/vendor/bundle/ruby/2.4.0/gems/simplecov-buildkite-0.2.0/lib/simplecov/buildkite/profiles.rb:112: syntax error, unexpected keyword_rescue, expecting keyword_end
   rescue RuntimeError => error
   ^

that's because rescue inside blocks is only allowed since ruby 2.5.
Because of this, it would be good to add the restriction to the gemspec file, so the gem isn't installed for uncompatible ruby versions.

Handle no-group case elegantly

Currently, if no groups exist, an empty unordered list is created. This should probably simply return the main summary’s content and nothing else.

Not getting any annotation outputted in build

hi,

I have followed the steps and running the build using buildkite, however not getting an annotation displayed.

This is what I have in my test_helper.rb

`ENV["RAILS_ENV"] ||= "test"

require "simplecov"
require "simplecov-buildkite"

SimpleCov.start "rails" do
load_profile "buildkite"

add_filter "/bin/"
add_filter "/db/"
add_filter "/spec/" # for rspec
add_filter "/test/" # for minitest

formatter SimpleCov::Formatter::MultiFormatter.new(
[
SimpleCov::Formatter::HTMLFormatter,
SimpleCov::Buildkite::AnnotationFormatter,
])
end`

This is what is outputted from my test step, but not seeing any annotation.

image

Spec suite does not exercise any real coverage results

Because the spec suite’s coverage data fixtures aren’t based on any files that exist, and because SimpleCov requires a file to exist on-disk for it to process any coverage data representing it, the coverage data is not actually processed: the data which is present should show 50% (2/4) coverage, not 100% (0/0), if the files existed.

It will become more important to have a proper spec suite which renders realistic results for features to come in the future.

Git-Mirrors experiment breaks git diff output

First -- your plugin is awesome and we really appreciate you releasing it!!!

Having some trouble now that git mirrors is enabled the git commands are broken.. I think this is related buildkite-plugins/docker-compose-buildkite-plugin#225

Sounds like there is some kind of fix for this but we couldn't figure out how to do it. Do you think there is a way to update this plugin to tolerate git mirrors being enabled?

Thank you so much for releasing this OSS!

SimpleCov::Buildkite profile initialising...

  | branch_name=dev-#x79zf1-add-code-coverage-to-ci
  | base_branch_name=master
  | current_commit=6f6a95dec90d54131c042b62332b0126d8e61106
  | error: object directory /var/lib/buildkite-agent/git-mirrors/git-github-com-ollieorder-ollieorder-git/objects does not exist; check .git/objects/info/alternates
  | current_commit_short=6f6a95d
  | error: object directory /var/lib/buildkite-agent/git-mirrors/git-github-com-ollieorder-ollieorder-git/objects does not exist; check .git/objects/info/alternates
  | fatal: bad object 6f6a95dec90d54131c042b62332b0126d8e61106
  | Command exited with status 128: git diff --name-only --diff-filter=d 6f6a95dec90d54131c042b62332b0126d8e61106

Files changed are not compared against origin's version of the base branch

Hi! This is a pretty cool gem! I like the formatting overall, it's nice and simple.

One issue that I've noticed is that the "files changed" metrics are comparing against ENV['BUILDKITE_PULL_REQUEST_BASE_BRANCH'] which might not actually be the upstream's most recent version of that branch. E.g. If my base branch is main, unless I check out that branch, pull from origin, and then check out the target branch, main could be stale, if for instance my CI is running in an image that gets built somewhat infrequently. I'm not sure if it would be enough to just add in origin to this piece of code here, or if it would possibly depend on what the remote repository is named...

      merge_base = git_merge_base(current_commit,
                                  "origin",)
                                  base_branch_name)

I suppose I can work around it by ensuring that I do a git checkout main && git pull origin main before checking out the target branch in the Buildkite checkout hook, but thought it would be nice if this gem handled it automatically - otherwise checkout out the upstream branch isn't strictly necessary so it'd probably save a tiny bit of time to not have to do it.

Thank you!

Edit: Hmm, I am realizing that I don't need to actually check out the main branch to update it, I can just update the local reference by doing git fetch origin main:main. That seems good enough to work around the issue for now. This caught me by surprise when I first ran into it, but it does make sense that it works this way currently.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.