korenyoni / snowcapper Goto Github PK

View Code? Open in Web Editor NEW

12.0 1.0 0.0 318 KB

Immutable config management for Alpine Linux

Home Page: https://k0ren.com

License: Mozilla Public License 2.0

Go 98.76% Makefile 0.90% Shell 0.34%

alpine linux golang devops configuration configuration-management immutable-infrastructure go

snowcapper's Introduction

Snowcapper - Immutable config management for Alpine Linux

Snowcapper is a single binary for bootstrapping services onto an Alpine Linux image.

Example Config:

extends:
  - src: /tmp/examples/vim.snc
packages:
  - name: vault
    binaries:
      - name: vault
        mode: 0755
        src: https://releases.hashicorp.com/vault/0.10.0/vault_0.10.0_linux_amd64.zip
        src_hash: a6b4b6db132f3bbe6fbb77f76228ffa45bd55a5a1ab83ff043c2c665c3f5a744
        format: zip
    files:
      - path: /etc/vault/config.hcl
        mode: 0700
        content: |
          storage "file" {
            path    = "/mnt/vault/data"
          }

          listener "tcp" {
            address     = "0.0.0.0:8200"
            tls_disable = 1
          }
    services:
      - binary: vault
        args:
          - "server"
          - "-config /etc/vault/config.hcl"
    inits:
      - type: openrc
        content: vault

Usage:

make get
make binary
./snowcapper

To test in an Alpine environment:

make

This builds the binary and provisions an Alpine VM using snowcapper and Vagrant.

snowcapper's People

Contributors

Stargazers

Watchers

snowcapper's Issues

Unit tests

At the moment, this package contains no unit tests. Unit tests should be incorporated to achieve high quality code.

Simplify config files by creating Service struct for OpenRC services

At the moment, to create OpenRC services your config must specify the OpenRC scripts in the files list of a Package struct:

    binaries:
      - name: vault
        mode: 0755
        src: https://releases.hashicorp.com/vault/0.10.0/vault_0.10.0_linux_amd64.zip
        format: zip
    files:
      - path: /etc/vault/config.hcl
        mode: 0700
        content: |
          ...
      - path: /etc/init.d/vault
        mode: 0700
        content: |
          NAME=vault
          DAEMON=/usr/bin/$NAME

          depend() {
          ...
          }

          start() {
          ....
          }

          stop () {
          ...
          }
    inits:
      - type: openrc
        content: vault

This could be simplified by adding a service struct that would go in a Package struct's services list:

    binaries:
      - name: vault
        mode: 0755
        src: https://releases.hashicorp.com/vault/0.10.0/vault_0.10.0_linux_amd64.zip
        format: zip
    files:
      - path: /etc/vault/config.hcl
        mode: 0700
        content: |
          ...
    services:
      - binary: vault
        args:
          - "server"
          - "-config /etc/vault/config.hcl"
    inits:
      - type: openrc
        content: vault

Allow specifying hash for files to be downloaded.

Accepting a src_hash will allow the user to ensure the file is as intended and hasn't been tampered with / changed.

Prior art https://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.append and its source_hash attribute. It's a neat trick that it automatically determines the hash type from the length

Type    Length
======  ======
sha512     128
sha384      96
sha256      64
sha224      56
sha1        40
md5         32

Test that service is running correctly in CI

Snowcapper should have a successful CI test using the alpine test image only if the service it is bootstrapping has been successfuly bootstrapped.

In the case of Vault, running:

VAULD_ADDR=http://127.0.0.1:8200 vault status should print in its stderr:


URL: GET http://127.0.0.1:8200/v1/sys/seal-status
Code: 400. Errors:

* server is not yet initialized

Therefore a successful test would include the command:

VAULT_ADDR=http://127.0.0.1:8200 vault status 2>&1 | grep "server is not yet initialized"

And the CI would fail unless the above command would exit with 0.

Add unit tests for waits in inits package

The following code doesn't have unit tests

https://github.com/yonkornilov/snowcapper/blob/55cf949934c2f5e6035a16c7102f9dee09d04657/inits/inits.go#L102-L139

Allow for local binary source

Atm a binary source must be remote:

binaries:
  - name: vault
    mode: 0755
    src: https://releases.hashicorp.com/vault/0.10.0/vault_0.10.0_linux_amd64.zip
    format: zip

There should be support for a local source:

binaries:
  - name: vault
    mode: 0755
    src: /tmp/vault_0.10.0_linux_amd64.zip
    format: zip

Stat Binary/Extend after downloading from Src or simply returning local Src

On local Extrend retrievals, os.Stat is performed on the Src which is returned (https://github.com/yonkornilov/snowcapper/blob/master/runner/runner.go#L132-L138). This is not done on remote retrievals.

For Binary retrievals, the os.Stat operation is done in neither remote or local retrievals, and should be done for both.

Renaming downloadPath in getBinary and getExtract to something in favour of local retrievals should be done too.

Eliminate else by returning in the if

https://github.com/yonkornilov/snowcapper/blob/e6f70e78beec69ed31ddd805bc99c3245015d513/init/init.go#L14

Fix cgroups issue in CircleCI alpine test container

The alpine test container cannot start openrc:

 * Caching service dependencies ...
Service `hwdrivers' needs non existent service `dev'                                                      [ ok ]                                                                                                    
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/blkio/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/cpu/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/cpuacct/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/cpuset/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/devices/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/freezer/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/hugetlb/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/memory/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/net_cls/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/net_prio/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/perf_event/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/pids/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/systemd/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/blkio/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/cpu/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/cpuacct/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/cpuset/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/devices/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/freezer/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/hugetlb/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/memory/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/net_cls/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/net_prio/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/perf_event/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/pids/tasks: Permission denied
/lib/rc/sh/openrc-run.sh: line 273: can't create /sys/fs/cgroup/systemd/tasks: Permission denied
 * You are attempting to run an openrc service on a
 * system which openrc did not boot.
 * You may be inside a chroot or you may have used
 * another initialization system to boot this system.
 * In this situation, you will get unpredictable results!
 * If you really want to do this, issue the following command:
 * touch /run/openrc/softlevel
 * ERROR: networking failed to start
 * ERROR: cannot start vault as networking would not start

Fix error handling for failed os.Exec

At the moment, the output of a non-0-exitcode command executed with os.Exec does not show:

panic: exit status 1

goroutine 1 [running]:
main.main()
	/go/src/github.com/yonkornilov/snowcapper/main.go:25 +0x17d
Exited with code 2```

Rename config.yaml to config.snc

As per #32 it would be a good idea for dedicating an *.snc file suffix to distinguish Snowcapper configurations in URIs.

The Runner struct needs to be aware of the context the program is running in. This is important for testing as the system-manipulating functions should be dry-run instead of executed if the functions calling them are run by go test.

A context struct will be accessible from runner.Run and these functions and will determine if these functions should be dry-run or executed.

Extensible profiles

It would be interesting to be able to layer snowcapper configurations on one another.

It could be a way to create community-curated bases to build on.

extends:
  - src: https://example.com/cis_hardened.snc 
    src_hash: a6b4b6db132f3bbe6fbb77f76228ffa45bd55a5a1ab83ff043c2c665c3f5a744

packages:
  - name: vault
    binaries:
      - name: vault
        mode: 0755
        src: https://releases.hashicorp.com/vault/0.10.0/vault_0.10.0_linux_amd64.zip
        src_hash: a6b4b6db132f3bbe6fbb77f76228ffa45bd55a5a1ab83ff043c2c665c3f5a744
        format: zip

In that example, you would start with a machine that followed the CIS Benchmarks and then installs your own configuration on top.

packages:
  - name: test
    files:                                
      - path: /etc/vault/config.hcl
        mode: 0700
        content: |
          storage "file" {
            path    = "/mnt/vault/data"
          }

          listener "tcp" {
            address     = "0.0.0.0:8200"
            tls_disable = 1
          }

          disable_mlock = "true"
    inits:
      - type: command
        content: echo nothing is installed

Gives Runtime error: Packages: (0: (Binaries: cannot be blank.).).

In another example,

packages:
  - name: test
    inits:
      - type: command
        content: echo nothing is installed.

Gives Runtime error: Packages: Either Files or Services must be present.

Configurations should be as minimalistic as the user desires.

These can be compressed to a single case statement

https://github.com/yonkornilov/snowcapper/blob/e6f70e78beec69ed31ddd805bc99c3245015d513/extract/extract.go#L60