Git Product home page Git Product logo

ci's Introduction

Py-CI

Generic continuous integration server for interfacing with github and publishing results to a Media Wiki. The basic idea is that continuous integration just requires three things:

  • A repo to monitor for changes.
  • Unit tests to run.
  • A place to publish the results of the tests.

Py-CI interfaces with github using the pygithub API wrapper for interfacing with github's API. It monitors the pull requests for a set of repos. When a new or untested pull request is found:

  • Create a local copy of the repo; merge the proposed changes from the pull request into a working branch.
  • Run all the unit tests specified in the repo.xml configuration file.
  • Post the results of running the unit tets to a media wiki.

Quickstart

If you already have a virtualenv installed as well as virtualenvwrapper, skip this next code block. However, be sure to add the VENV variable to the global configuration file.

pip install virtualenv
pip install virtualenvwrapper
mkvirtualenv ci

Next, copy the global.xml file and put it somewhere on you local disk (say ~/.ci.global.xml). Be sure to edit the values. Then:

export PYCI_XML="~/.ci.global.xml"
workon ci
pip install py-ci
sudo ci.py -setup

If you don't see any errors, your server is ready to have repositories installed. This configures the server cron to run every minute. Use -cronfreq [int minutes] to change that frequency (see cron settings for details). You don't need to use sudo for anything except setting the server up initially (or uninstalling it later with the -rollback switch). Next, create a repo.xml file for repository you want to monitor. Suppose it exists at ~/repos/myrepo/ci.xml, then:

ci.py -install ~/repos/myrepo/ci.xml

Your repository will now be monitored for new pull requests forever untill you either -uninstall the repo.xml file or you -disable the CI server to temporarily suspend all requests. To understand the behavior of the CI server, read through the repository level settings page.

IMPORTANT: if your unit tests require environment variables to be set, they need to be added to a file called ~/.cron_profile that will be loaded by the CI server whenever the cron is run. See cron environment variables for more details.

ci's People

Contributors

rosenbrockc avatar

Stargazers

avatar

Watchers

avatar

ci's Issues

Handling Detailed Output from Unit Tests

The initial CI server is almost ready. However, I just realized that we need to discuss one more aspect. So far, I programmed the server to keep track of the output from stdout when running each unit test. That output is kept in memory until the wiki page with the unit test details is created. My original idea was to upload the contents of stdout for each test to a file on the wiki and then just link to it from the test results page.

That still seems like a viable possibility except that for some of the unit tests the output can be quite large. Do we place limitations on the size of the files we want to upload? If fortpy handles the unit testing, it provides summary information and will write out compilation errors or warnings and that output is usually small. But what about output from other programs? For example, the Wang Landau in UNCLE once produced a file that was 135GB!

Question: if the files are larger than 1MB, do we upload only part of them? If so, which part? If not, do we just say the file was too big? Do we remove it from the server then, or move it to some archive?

Serial Execution of Commands

The default (current) implementation only allows the unit tests to be run in parallel with the multiprocessing module. However, the global settings allow for serial processing. It is implemented in the ci.py script, but not in the pyci.server module.

Awkward question

Hi @rosenbrockc

So this is a bit weird, but i am developing a python application and strangely enough, i called it Py-CI.
I noticed you are the owner of that project name on PyPi.

Is there any chance we can discuss about maybe giving the ownership for that project to me?

Much obliged.

👍

Timeout Implementation

Although the repo settings allow for a timeout to be specified (after which any unit tests still running are forcible terminated), the pyci.server module currently doesn't implement this except in analyzing the run times returned in the execution statistics. The process termination still needs to be implemented.

Verbose Output

The script ci.py has a switch for switching to verbose output mode. However, none of the functions coded so far support any level of verbosity. This is something that isn't critical at this point, but would be nice to have for debugging other user's installations (really, any respectable code has verbose output).

Shields Web URL

It would be useful to have the CI server be able to return the URL to a shield to post on the README page of a repo being monitored (like Travis and all of the others do). This is easy using:

https://img.shields.io/badge/Test%20Coverage-100-green.svg

We just need to have the latest scores written to JSON and then deserialized by the web page to return the URL with the appropriate fields replaced. Probably we would need to add one option to the config file that specifies the command to run on the repo's master to get the coverage etc. and the latest test scores.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.