Git Product home page Git Product logo

zalando / package-build Goto Github PK

View Code? Open in Web Editor NEW
35.0 30.0 8.0 403 KB

A toolset for building system packages using Docker and fpm-cookery

Home Page: https://tech.zalando.com/blog/building-system-packages-from-python-modules-with-dependencies-included/

License: MIT License

Shell 41.83% Ruby 42.38% Makefile 4.35% HCL 0.28% Perl 2.73% Dockerfile 8.43%
fpm-cookery packaging apt rpm deb developer-infrastructure

package-build's Introduction

Package Build

Package-build is a toolset for creating native system packages: .deb for Debian-like and .rpm for RedHat-like OSes. It is useable out-of-the-box.

The Idea

With a few simple scripts, you can use package-build to create isolated, self-contained packages; provide them in your internal repos; and not worry about deployment and dependencies. You can even use these scripts to package tarballs that are randomly dropped into a web folder. Because a simple shell script performs the actual package-building, you can easily use the same commands in a continuous integration context โ€” i.e., to automatically build packages every time a recipe changes or a new one has been added.

Details:

  • the whole build process is triggered from Fabric tasks running on the "build host"
  • build slaves are Docker containers to maintain a clean, well-defined environment
  • actual package building is done with fpm and fpm-cookery

Setup

Clone the repo and install the Python requirements:

sudo pip install -r requirements.txt

cp package-build.yaml-example ~/.config/package-build.yaml
vim ~/.config/package-build.yaml # adapt to your repo server and distributions for which you want to build packages
fab docker_build # this will setup the required Docker images and could take a while

If pycrypto makes trouble here, install Python's headers before: sudo apt-get install python-dev or sudo yum install python-devel depending on your OS.

How-to

Docker containers provide the build environments, so you'll have to create a new subfolder under docker/ with a Dockerfile for all the distributions for which you want to create a package. Create an entry in the config file ~/.config/package-build.yaml for each distribution to map it to a package format ("rpm" or "deb", see the example config file).

As stated above, fpm and fpm-cookery do the actual package build. (fpm-cookery automatically builds [only a package for the distribution/OS where it's running] (https://github.com/bernd/fpm-cookery/blob/master/spec/facts_spec.rb#L72).) Create a recipe.rb under a subfolder in recipes/. Optionally, create a script called prepare.sh, meant to be run before fpm-cook package is executed.

$ Command Line Examples

To start the packaging process for different distributions and packages, see the examples below.

Build facter for Ubuntu 16.04 ("Xenial"):

fab package_build:facter,ubuntu16.04

Build facter for all configured distributions:

fab package_build:facter

For testing: The created package will not automatically upload and publish to your repositories unless you set the parameter upload to True:

fab package_build:debian7,upload=True

Publish a package to the internal repository for ubuntu16.04 (which is the default distribution):

fab repo_deb_add:~/path/to/package.deb

To publish a package in the repos for other distributions, you have to pass them explicitly:

fab repo_deb_add:~/path/to/package.deb,dist=ubuntu12.04

To delete a package from the repo for a specified distribution:

fab repo_deb_del:dist=ubuntu12.04,chimp

More Recipe Examples

Test Coverage

Currently, there are no tests for this project and test contributions are very welcome. Please see also the Contributing section.

Contributing

Thanks for your interest in contributing! There are many ways to contribute to this project. Get started here.

License

The contents of this repository are licensed under the MIT License.

package-build's People

Contributors

apfeiffer85 avatar christianberg avatar lappleapple avatar perploug avatar raffo avatar shinzu avatar skoenig avatar szuecs avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

package-build's Issues

Howler: when a build is done, trying to build again will fail

This is possible due to this two cleanup lines in the recipe:

    rm_rf "#{builddir}/gobuild/pkg", :verbose => true
    rm_rf "#{builddir}/gobuild/bin", :verbose => true

This is the error:

WARNING: Skipping build of howler because build cookie found (/data/recipes/howler/tmp-build/.build-cookie-howler_0_0_8), use "fpm-cook clean" to rebuild!
/var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/path.rb:63:in `install_p': /data/recipes/howler/tmp-build/gobuild/bin/howler does not exist (RuntimeError)
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/path.rb:40:in `install'
    from /data/recipes/howler/recipe.rb:32:in `install'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/packager.rb:160:in `block (2 levels) in dispense'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/packager.rb:138:in `chdir'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/packager.rb:138:in `block in dispense'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/packager.rb:115:in `chdir'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/packager.rb:115:in `dispense'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/cli.rb:113:in `exec'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/cli.rb:77:in `block in execute'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/book.rb:16:in `call'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/book.rb:16:in `load_recipe'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/lib/fpm/cookery/cli.rb:73:in `execute'
    from /var/lib/gems/1.9.1/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'
    from /var/lib/gems/1.9.1/gems/clamp-0.6.5/lib/clamp/subcommand/execution.rb:11:in `execute'
    from /var/lib/gems/1.9.1/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'
    from /var/lib/gems/1.9.1/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'
    from /var/lib/gems/1.9.1/gems/fpm-cookery-0.31.0/bin/fpm-cook:7:in `<top (required)>'
    from /usr/local/bin/fpm-cook:23:in `load'
    from /usr/local/bin/fpm-cook:23:in `<main>'
===> Skipping fetch, rev 0.0.8 exists.
===> Using source directory: /data/recipes/howler/tmp-build/howler-tag-0.0.8
===> Installing into /data/recipes/howler/tmp-dest
task ran 1.61386299133 seconds

The command used:

fab package_build:ubuntu14.04,howler

Zalando-specific README, but not Zalando-dependent project?

This project seems to have no Zalando dependencies--is that right?

The README refers to Zalando internal team names and other things that won't make sense to the outside world, and that suggest that this is Z-dependent. If it's truly out-of-the-box, then we can revise the Readme to show this and still address Zalandos. Thoughts?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.