rizalgowandy / octocov Goto Github PK

octocov is a toolkit for collecting code metrics (code coverage, code to test ratio and test execution time).

Key features of octocov are:

• Support multiple coverage report formats.
• Support multiple code metrics.
• Support for even generating coverage report badge.
• Have a mechanism to aggregate reports from multiple repositories.

GitHub Actions for octocov is here !!

First, run test with coverage report output.

For example, in case of Go language, add -coverprofile=coverage.out option as follows

$ go test ./... -coverprofile=coverage.out

Add .octocov.yml ( or octocov.yml ) file to your repository.

# .octocov.yml
coverage:
  paths:
    - coverage.out
codeToTestRatio:
  code:
    - '**/*.go'
    - '!**/*_test.go'
  test:
    - '**/*_test.go'
comment:
  enable: true

And set up a workflow file as follows and run octocov on GitHub Actions.

# .github/workflows/ci.yml
name: Test

on:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      -
        uses: actions/checkout@v2
      -
        uses: actions/setup-go@v2
        with:
          go-version: 1.17
      -
        name: Run tests with coverage report output
        run: go test ./... -coverprofile=coverage.out
      -
        uses: k1LoW/octocov-action@v0

Then, octocov comment the report of the code metrics to the pull request.

Notice: Note that only pull requests from the same repository can be commented on. This is because the workflow token of a forked pull request does not have write permission.

octocov acts as a code metrics viewer on the terminal.

For example, in case of Go language, add -coverprofile=coverage.out option as follows

$ go test ./... -coverprofile=coverage.out

And run octocov ls-files , octocov view [FILE...] and octocov diff [REPORT_A] [REPORT_B]

By setting comment:, comment the reports to pull request.

# .octocov.yml
comment:
  enable: true
  hideFooterLink: false # hide octocov link

octocov checks for "Code Coverage" by default. If it is running on GitHub Actions, it will also measure "Test Execution Time".

If you want to measure "Code to Test Ratio", set codeToTestRatio:.

comment:
  enable: true
codeToTestRatio:
  code:
    - '**/*.go'
    - '!**/*_test.go'
  test:
    - '**/*_test.go'

By setting diff: ( diff.path: or diff.datastores ) additionally, it is possible to show differences from previous reports as well.

comment:
  enable: true
diff:
  datastores:
    - s3://bucket/reports

By setting coverage.acceptable:, the condition of acceptable coverage is specified.

If this condition is not met, the command will exit with exit status 1.

# .octocov.yml
coverage:
  acceptable: 60%

$ octocov
Error: code coverage is 54.9%. the condition in the `coverage.acceptable:` section is not met (`60%`)

By setting codeToTestRatio.acceptable:, the condition of acceptable "Code to Test Ratio" is specified.

If this condition is not met, the command will exit with exit status 1.

# .octocov.yml
codeToTestRatio:
  acceptable: 1:1.2
  code:
    - '**/*.go'
    - '!**/*_test.go'
  test:
    - '**/*_test.go'

$ octocov
Error: code to test ratio is 1:1.1, the condition in the `codeToTestRatio.acceptable:` section is not met (`1:1.2`)

By setting testExecutionTime.acceptable:, the condition of acceptable "Test Execution Time" is specified (on GitHub Actions only) .

If this condition is not met, the command will exit with exit status 1.

# .octocov.yml
testExecutionTime:
  acceptable: 1 min

$ octocov
Error: test execution time is 1m15s, the condition in the `testExecutionTime.acceptable:` section is not met (`1 min`)

By setting *.badge.path:, generate badges self.

# .octocov.yml
coverage:
  badge:
    path: docs/coverage.svg

# .octocov.yml
codeToTestRatio:
  badge:
    path: docs/ratio.svg

# .octocov.yml
testExecutionTime:
  badge:
    path: docs/time.svg

You can display the coverage badge without external communication by setting a link to this badge image in README.md, etc.

# mytool

![coverage](docs/coverage.svg)

By setting push:, git push report badges self.

# .octocov.yml
coverage:
  badge:
    path: docs/coverage.svg
push:
  enable: true

By setting report:, store the reports to datastores and local path.

# .octocov.yml
report:
  datastores:
    - github://owner/coverages/reports
    - s3://bucket/reports

# .octocov.yml
report:
  path: path/to/report.json

• GitHub repository
• S3
• GCS
• BigQuery
• Local

By enabling central:, octocov acts as a central repository for collecting reports ( example ).

# .octocov.yml for central mode
central:
  enable: true
  root:                    .             # root directory or index file path of collected coverage reports pages. default: .
  reports:
    - bq://my-project/my-dataset/reports # datastore paths (URLs) where reports are stored. default: local://reports
  badges: badges                         # directory where badges are generated. default: badges
  push:
    enable: true                         # enable self git push

• GitHub repository
• S3
• GCS
• BigQuery
• Local

octocov ls-files command can be used to list files logged in code coverage report.

octocov view (alias: octocov cat) command can be used to view the file coverage report.

The name of the repository.

It should be in the format owner/repo.

By default, the value of the environment variable GITHUB_REPOSITORY is set.

In case of monorepo, code metrics can be reported to datastore separately by specifying owner/repo/project-a or owner/repo@project-a.

repository: k1LoW/octocov

Configuration for code coverage.

coverage.path: has been deprecated. Please use coverage.paths: instead.

The path to the coverage report file.

If no path is specified, the default path for each coverage format will be scanned.

coverage:
  paths:
    - tests/coverage.xml

acceptable coverage condition.

coverage:
  acceptable: 60%

coverage:
  acceptable: current >= 60% && diff >= 0.5%

The variables that can be used are as follows.

It is also possible to omit the expression as follows

Set this if want to generate the badge self.

The path to the badge.

coverage:
  badge:
    path: docs/coverage.svg

Configuration for code to test ratio.

Files to count.

codeToTestRatio:
  code:                  # files to count as "Code"
    - '**/*.go'
    - '!**/*_test.go'
  test:                  # files to count as "Test"
    - '**/*_test.go'

acceptable ratio condition.

codeToTestRatio:
  acceptable: 1:1.2

codeToTestRatio:
  acceptable: current >= 1.2 && diff >= 0.0

The variables that can be used are as follows.

It is also possible to omit the expression as follows

Set this if want to generate the badge self.

The path to the badge.

codeToTestRatio:
  badge:
    path: docs/ratio.svg

Configuration for test execution time.

acceptable time condition.

testExecutionTime:
  acceptable: 1min

testExecutionTime:
  acceptable: current <= 1min && diff <= 1sec

The variables that can be used are as follows.

It is also possible to omit the expression as follows

Set this if want to generate the badge self.

The path to the badge.

testExecutionTime:
  badge:
    path: docs/time.svg

Configuration for git push badges self.

Enable / disable git push

push:
  enable: false

Set this if want to comment report to pull request

Enable / disable comment.

comment:
  enable: true

enable: true can be omitted if any other parameters are set as follows.

comment:
  hideFooterLink: true

Hide footer octocov link.

comment:
  hideFooterLink: true

Conditions for commenting report.

# .octocov.yml
comment:
  if: github.event_name == 'pull_request'

Configuration for comparing reports.

Path of the report to compare.

diff:
  path: path/to/coverage.yml

diff:
  path: path/to/report.json

Datastores where the report to be compared is stored.

diff:
  datastores:
    - local://.octocov       # Use .octocov/owner/repo/report.json
    - s3://my-bucket/reports # Use s3://my-bucket/reports/owner/repo/report.json

Conditions for comparing reports

# .octocov.yml
report:
  if: github.event_name == 'pull_request'
  path: path/to/report.json

Configuration for reporting to datastores.

Path to save the report.

report:
  path: path/to/report.json

Datastores where the reports are saved.

report:
  datastores:
    - github://owner/coverages/reports
    - s3://bucket/reports

Use github:// scheme.

github://[owner]/[repo]@[branch]/[prefix]

Required environment variables:

• GITHUB_TOKEN or OCTOCOV_GITHUB_TOKEN
• GITHUB_REPOSITORY or OCTOCOV_GITHUB_REPOSITORY
• GITHUB_API_URL or OCTOCOV_GITHUB_API_URL (optional)

Use s3:// scheme.

s3://[bucket]/[prefix]

Required permission:

• s3:PutObject

Required environment variables:

• AWS_ACCESS_KEY_ID or OCTOCOV_AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY or OCTOCOV_AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN or OCTOCOV_AWS_SESSION_TOKEN (optional)

Use gs:// scheme.

gs://[bucket]/[prefix]

Required permission:

• storage.objects.create
• storage.objects.delete

Required environment variables:

• GOOGLE_APPLICATION_CREDENTIALS or GOOGLE_APPLICATION_CREDENTIALS_JSON or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON

Use bq:// scheme.

bq://[project ID]/[dataset ID]/[table]

Required permission:

• bigquery.datasets.get
• bigquery.tables.get
• bigquery.tables.updateData

Required environment variables:

• GOOGLE_APPLICATION_CREDENTIALS or GOOGLE_APPLICATION_CREDENTIALS_JSON or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON

Datastore schema:

Datastore schema

If you want to create a table, execute the following command ( require bigquery.datasets.create ).

$ octocov migrate-bq-table

Use local:// or file:// scheme.

local://[path]

Example:

If the absolute path of .octocov.yml is /path/to/.octocov.yml

• local://reports ... /path/to/reports directory
• local://.reports ... /path/to/reports directory
• local://../reports ... /path/reports directory
• local:///reports ... /reports directory.

Conditions for saving a report.

# .octocov.yml
report:
  if: env.GITHUB_REF == 'refs/heads/main'
  datastores:
    - github://owner/coverages/reports

The variables available in the if section are as follows

Enable / disable central mode.

central:
  enable: false

enable: true can be omitted if any other parameters are set as follows.

central:
  reports:
    datastores:
      - local://reports
      - gs://my-gcs-bucket/reports

:NOTICE: When central mode is enabled, other functions are automatically turned off.

The root directory or index file ( index file example ) path of collected coverage reports pages. default: .

central:
  root: path/to

Datastore paths (URLs) where reports are stored. default: local://reports

central:
  reports:
    datastores:
      - local://reports
      - gs://my-gcs-bucket/reports

When using the central repository as a datastore, perform badge generation via on.push.

# .octocov.yml
report:
  datastores:
    - github://owner/central-repo/reports

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - github://owner/central-repo/reports
  push:
    enable: true

or

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - local://reports
  push:
    enable: true

When using the S3 bucket as a datastore, perform badge generation via on.schedule.

# .octocov.yml
report:
  datastores:
    - s3://my-s3-bucket/reports

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - s3://my-s3-bucket/reports
  push:
    enable: true

Required permission (Central Repo):

• s3:GetObject
• s3:ListObject

Required environment variables (Central Repo):

• AWS_ACCESS_KEY_ID or OCTOCOV_AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY or OCTOCOV_AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN or OCTOCOV_AWS_SESSION_TOKEN (optional)

When using the GCS bucket as a datastore, perform badge generation via on.schedule.

# .octocov.yml
report:
  datastores:
    - gs://my-gcs-bucket/reports

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - gs://my-gcs-bucket/reports
  push:
    enable: true

Required permission (Central Repo):

• storage.objects.get
• storage.objects.list
• storage.buckets.get

Required environment variables (Central Repo):

• GOOGLE_APPLICATION_CREDENTIALS or GOOGLE_APPLICATION_CREDENTIALS_JSON or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON

When using the BigQuery table as a datastore, perform badge generation via on.schedule.

# .octocov.yml
report:
  datastores:
    - bq://my-project/my-dataset/reports

# .octocov.yml for central repo
central:
  reports:
    datastores:
      - bq://my-project/my-dataset/reports
  push:
    enable: true

Required permission (Central Repo):

• bigquery.jobs.create
• bigquery.tables.getData

Required environment variables (Central Repo):

• GOOGLE_APPLICATION_CREDENTIALS or GOOGLE_APPLICATION_CREDENTIALS_JSON or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS or OCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON

Datastore paths (URLs) where badges are generated. default: local://badges

central:
  badges:
    datastores:
      - local://badges
      - s3://my-s3-buckets/badges

Configuration for git push index file and badges self.

Enable / disable git push

push:
  enable: true

Conditions for central mode.

# .octocov.yml
central:
  if: env.GITHUB_REF == 'refs/heads/main'
  reports:
    datastores:
      - s3://my-s3-bucket/reports

octocov supports multiple coverage report formats.

And octocov searches for the default path for each format.

If you want to specify the path of the report file, set coverage.path

coverage:
  paths:
    - /path/to/coverage.txt

Default path: coverage.out

Default path: coverage/lcov.info

Support SF DA only

Default path: coverage/.resultset.json

Default path: coverage.xml

Default path: coverage.xml

• Code Coverage
• Code to Test Ratio
• Test Execution Time (on GitHub Actions only)

deb:

Use dpkg-i-from-url

$ export OCTOCOV_VERSION=X.X.X
$ curl -L https://git.io/dpkg-i-from-url | bash -s -- https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.deb

RPM:

$ export OCTOCOV_VERSION=X.X.X
$ yum install https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.rpm

apk:

Use apk-add-from-url

$ export OCTOCOV_VERSION=X.X.X
$ curl -L https://git.io/apk-add-from-url | sh -s -- https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.apk

homebrew tap:

$ brew install k1LoW/tap/octocov

manually:

Download binary from releases page

docker:

$ docker pull ghcr.io/k1low/octocov:latest