Git Product home page Git Product logo

repository-traffic-action's Introduction

Repository Traffic GitHub Action

Github action that can be used to store repository traffic and clones past the default 2 week period. It pulls traffic and clones data from the GitHub API v3 and stores it into a csv file, which can be commited to your repository or uploaded elsewhere.

Usage

Setting up permissions

You'll first need to create a personal access token (PAT) so the action can access the GitHub API.

You can generate a PAT by going to Settings -> Developer Settings -> Personal Access Tokens -> Generate new token. You will need to grant "repo" permission. For more in depth instructions, see the GitHub documentation

After you have generated the PAT, go to the "Settings" tab of the repository, click on New Secret, name the secret "TRAFFIC_ACTION_TOKEN" and copy the PAT into the box.

Create a work flow

Create a workflow.yml file and place in your .github/workflows folder. You can reference the action from this workflow. The only required parameter is setting the PAT that was generated when setting up the permissions.

    steps:
    # Calculates traffic and clones and stores in CSV file
    - name: Repository Traffic 
      uses: sangonzal/[email protected]
      env:
        TRAFFIC_ACTION_TOKEN: ${{ secrets.TRAFFIC_ACTION_TOKEN }} 

This actions does not store the generated data anywhere by default. It temporarily stores it in ${GITHUB_WORKPLACE}/traffic, but unless it's exported it will be lost. You can integrate other actions into the workflow to upload data elsewhere. Below are two examples.

Sample workflow that runs weekly and commits files to repository.

on:
  schedule: 
    # runs once a week on sunday
    - cron: "55 23 * * 0"
    
jobs:
  # This workflow contains a single job called "traffic"
  traffic:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
    # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
    - uses: actions/checkout@v2
      with:
        ref: "traffic"
    
    # Calculates traffic and clones and stores in CSV file
    - name: GitHub traffic 
      uses: sangonzal/[email protected]
      env:
        TRAFFIC_ACTION_TOKEN: ${{ secrets.TRAFFIC_ACTION_TOKEN }} 
     
    # Commits files to repository
    - name: Commit changes
      uses: EndBug/add-and-commit@v4
      with:
        author_name: Santiago Gonzalez
        message: "GitHub traffic"
        add: "./traffic/*"
        ref: "traffic"  # commits to branch "traffic" 
  • Notes:
    • Ensure there is a branch in your repository with whatever ref value you use before running the action. If using the above values, you would create a branch "traffic".
    • Ensure that the ref used in actions/checkoutv2 is the same in Endbug/add-and-commit@v4.

Sample workflow that runs weekly and uploads files to S3.

If you'd like to avoid commiting the data to the repository, you can use another action to upload elsewhere. For example, you could download and upload files from S3 using other github actions.

on:
  schedule: 
    # runs once a week on sunday
    - cron: "55 23 * * 0"
    
jobs:
  # This workflow contains a single job called "traffic"
  traffic:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
    # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
    - uses: actions/checkout@v2

    # Download from S3
    - uses: prewk/s3-cp-action@master
      env:
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        SOURCE: 's3://some-bucket/something-remote'
        DEST: 'traffic'
    
    # Calculates traffic and clones and stores in CSV file
    - name: Repository Traffic 
      uses: sangonzal/[email protected]
      env:
        TRAFFIC_ACTION_TOKEN: ${{ secrets.TRAFFIC_ACTION_TOKEN }} 
     
     # Upload to S3
    - name: S3 Sync
      uses: jakejarvis/[email protected]
      with:
        args: --acl public-read --follow-symlinks --delete
      env:
        AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        SOURCE_DIR: 'traffic'

Running from a different repository

If you'd like to get stats from a different repository than the one that you are running the github action, you can set the env variable "REPOSITORY_NAME". REPOSITORY_NAME should be formatted as "username/repository_name" or "organization_name/repository_name". The personal access token that you created in the first step should have access to the repository. For example, if I was to set up a a github action in a different repository to store stats for this repo, the workflow file would look like this:

    steps:
    # Calculates traffic and clones and stores in CSV file
    - name: Repository Traffic 
      uses: sangonzal/[email protected]
      env:
        TRAFFIC_ACTION_TOKEN: ${{ secrets.TRAFFIC_ACTION_TOKEN }}
        REPOSITORY_NAME: "sangonzal/repository-traffic-action"

repository-traffic-action's People

Contributors

actions-user avatar andreadega avatar dependabot[bot] avatar dfmore avatar jessicas11 avatar sangonzal avatar simoncw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

repository-traffic-action's Issues

Get traffic data for repo A, store in folder in repo B

Hi Santiago

First of, this looks really good, great work.

I want to know if it is possible to set up a workflow that will get the traffic data from repo A and store it in repo B, under a specific folder/filename.csv file.

My use case is a bunch of repos that I all want to collect stats for, but I would like to store them in a private repo.

Thanks, Egil

github.GithubException.BadCredentialsException: 401

Hi - starting last week I keep getting this error reported below.
I tried to re-create my token but I have not been able to sort the problem.
Any idea? Can you help?

thanks!

Run sangonzal/[email protected]
  with:
    branch: dev
  env:
    TRAFFIC_ACTION_TOKEN: ***
/usr/bin/docker run --name a33c1a32ba7c2905941738f2b48262d66ba50_246ae8 --label 8a33c1 --workdir /github/workspace --rm -e TRAFFIC_ACTION_TOKEN -e INPUT_BRANCH -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/MMM-covid19/MMM-covid19":"/github/workspace" 8a33c1:a32ba7c2905941738f2b48262d66ba50
Repository name:  0m4r/_[REDACTED]_
Traceback (most recent call last):
  File "/main.py", line 9, in <module>
    repo = github.get_repo(repo_name)
  File "/root/-x-v5uFv0/lib/python3.8/site-packages/github/MainClass.py", line 345, in get_repo
    headers, data = self.__requester.requestJsonAndCheck(
  File "/root/-x-v5uFv0/lib/python3.8/site-packages/github/Requester.py", line 315, in requestJsonAndCheck
    return self.__check(
  File "/root/-x-v5uFv0/lib/python3.8/site-packages/github/Requester.py", line 340, in __check
    raise self.__createException(status, responseHeaders, output)
github.GithubException.BadCredentialsException: 401 {"message": "Bad credentials", "documentation_url": "https://docs.github.com/rest"}

Views and Visitors

Can the action report the number of views and unique visitors too since these attributes are also restricted to the 2 weeks limitation

Action will fail in forks

Thanks for this very handy action. The problem I encountered is that the action will fail for anyone forking the repo since it relies on a PAT. Any ideas for a workaround?

where are the data files?

dear @sangonzal your action looks simple and easy and well explained, great work

  • I have done everything you said in the readme
  • the action runs without error in my repo
  • but then nothing more happens, no new file or folder .traffic is visible in my repo...

where is the data now?

Does the action overwrite traffic data csvs every time it runs?

First, thank you for this action - very useful!

In the example this action runs every week. I'm assuming that every time it runs is just overwrites the data it had from the previous week? Or does it work as a log and keeps adding rows to the existing CSV files?

If it overwrites the files, it would be good to mention that in the README. Also, a quick first improvement could be to timestamp each file, so that every week it would generate a file which could be collated into a master.

Job failing ; exception error

Hello,

I have been using this workflow for about 2 years with no issues but the jobs suddenly stopped running in November'23. When I look at the error returned in the job, I see "Resource protected by organization SAML enforcement".

What I have done so far:

  1. I ensured the PAT was not expired and even generated a new one with extended validity date.
  2. I followed the steps in issue 2661 and Github docs to ensure write access by re-authorizing access. This error no longer occurs but the job still fails and now the error below is the current error: "{"message": "Must have push access to repository", "documentation_url": "https://docs.github.com/rest/metrics/traffic#get-page-views"}`

ย 
Traceback (most recent call last): File "/main.py", line 57, in <module> main() File "/main.py", line 25, in main views_frame = repo_stats.get_views(views_path) File "/repostats.py", line 9, in get_views views = self.repo.get_views_traffic() File "/root/-x-v5uFv0/lib/python3.8/site-packages/github/Repository.py", line [19](https://github.com/Azure/Azure-Network-Security/actions/runs/7398228726/job/20126949735#step:4:20)86, in get_views_traffic headers, data = self._requester.requestJsonAndCheck( File "/root/-x-v5uFv0/lib/python3.8/site-packages/github/Requester.py", line 353, in requestJsonAndCheck return self.__check( File "/root/-x-v5uFv0/lib/python3.8/site-packages/github/Requester.py", line 378, in __check raise self.__createException(status, responseHeaders, output) github.GithubException.GithubException: 403 **{"message": "Must have push access to repository", "documentation_url": "https://docs.github.com/rest/metrics/traffic#get-page-views"}**

latest commit in dev branch fails to run

Hi there,

Thanks so much for this very useful tool. I would like to use the latest version, which looking at the code seems much improved. However, setting @afe0206f766a8c1b5350977a377bdbcc6ab8072b instead of @v0.1.4 results in the error:

 Traceback (most recent call last):
  File "/main.py", line 3, in <module>
    from repostats import RepoStats
ModuleNotFoundError: No module named 'repostats'

I wonder if it is just a matter of copying the repostat.py into the docker container, which seems to be missing in the Dockerfile.

quick question

When the action runs, are the previous files in the ./traffic directory overwritten?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.