This is a personal home lab server which I’m trying to run as a practice in the following areas:

1. Infrastructure as Code: Mostly ansible is used at this moment. But knowing its shortcomings, I expect to incrementally replace it with other toolings.
2. Self-hosted cloud
3. Security

Configure resource discovery where:

• hosts will contain all targeted hosts.
• roles will contain all the installed roles.
• remote_user is set to pi as we are using that user on our raspberry pis!

[defaults]
inventory=./hosts
roles_path=./roles
remote_user=pi

Yeah, let’s make it easy to contribute for everyone (hehe, at least on the paper)!

# top-most EditorConfig file
root = true

# Unix-style newlines with a newline ending every file
[*]
end_of_line = lf
insert_final_newline = true
charset = utf-8

# Indentation for all Yaml files
[*.yml]
indent_style = space
indent_size = 2

# Indentation for all Ansible template files
[*.j2]
indent_style = space
indent_size = 4

Currently there is one host targetted by this configuration. I’m going to define it inside the project for now:

192.168.178.94

First thing first, I want to upload my SSH keys into the device, so except for the first time, I don’t need to worry about the password and accessing. Also later we are going to secure SSH connections and as part of that, we are going to disable password login:

---
- hosts: all
  remote_user: pi
  tasks:
    - name: Upload SSH Key
      authorized_key:
        user: pi
        state: present
        manage_dir: yes
        key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"

To run this however, I need to enter the password for the first time, and for that I need to use --ask-pass flag with the ansible-playbook command. That however, needs another dependency, which I need to install like:

sudo apt install sshpass

Probably easier to put all these into a Makefile.

install-deps:
	sudo apt install sshpass

init: install-deps
	ansible-playbook --ask-pass playbooks/upload_ssh_key.yml -i hosts

run: init
	ansible-playbook playbooks/main.yml

Now running make init should satisfy our needs. Let’s see:

make init

Make sure the system is update:

---
- name: Update apt repository and cache
  apt:
    update_cache: yes

- name: Upgrade installed packages
  apt:
    upgrade: dist

Here we are going to manage the applications required by the host. For the self service mechanism however, I tend to use containerized applications as much as possible.

---
- name: Install required applications
  apt:
    pkg:
      - cryptsetup
      - libffi-dev
      - libssl-dev
      - python3
      - python3-pip
      - ufw
      - fail2ban
    state: latest
    update_cache: true

First thing I tried was to use a role from the galaxy, however it turned out to be simpler (it supposed to be, right?) and I thought safer to do this component myself. Main reason being to not being blocked by the upstream on updates.

- name: Remove python-configparser package
  apt:
    name: python-configparser
    state: absent

- name: get docker convenience script
  shell: curl -fsSL https://get.docker.com -o get-docker.sh
  args:
    creates: /home/pi/get-docker.sh

- name: install docker
  shell: sh /home/pi/get-docker.sh
  args:
    creates: /usr/bin/docker

Add pi user to Docker group, so it can execute docker commands without sudo:

- name: make pi user execute docker commands
  shell: usermod -aG docker pi

Install docker-compose command using Python3:

- name: install docker-compose
  shell: pip3 -v install docker-compose
  args:
    creates: /usr/local/bin/docker-compose

• How to install Docker and docker-compose on Raspberry Pi using Ansible

Well, I’m not sure about this application surface yet. Having two options, including Potrainer, I decided to give new kid in the town a try, as it seems to be easier to use, and I don’t want to spend more than necessary time on it for a single node.

- name: Create the Docker volume for Yacht
  docker_volume:
    name: volume_one

- name: Run Yacht container
  docker_container:
    name: yacht
    image: selfhostedpro/yacht
    state: started
    ports:
      - "8000:8000"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "yacht:/config"

I have two disks which I wish to connect to this server. The partitioning scheme I have in mind is quite simple. Create an encryption layer using cryptsetup and create a single ext4 partition inside it.

There will be two keys installed on the encryption layer to open the disks. First one a long password, to be used on ad-hoc mode. And second one a key file, registered in crypttab to allow automatic decryption on the server.

**Caution**: This scheme is not a full circle yet, as whomever has access to the device, would be able to access the disks. In future I plan to give Sakaki’s EFI Install Guide/Configuration Secure Boot a try to resovle this issue.

Generate an encryption file using openssl:

export KEY_PATH=./playbooks/keys/$DISK_NAME
openssl genrsa -out $KEY_PATH

Secure it locally:

chmod -v 0400 $KEY_PATH
chown $USER:$USER $KEY_PATH

Shred the device:

shred -v --iterations=1 $DEVICE

Create the encryption layer:

cryptsetup luksFormat $DEVICE

So far the encryption has been done, and the password key is added. Let’s add the key file as extra way of possibility:

cryptsetup luksAddKey $DEVICE $KEY_PATH

Now open the device using the key file:

cryptsetup luksOpen $DEVICE hdd_$DEVICE --key-file $KEY_PATH

And format it:

mkfs.ext4 $DEVICE

Repeat this process for any new hard drive.

• How to enable Lucks disk encryption with Keyfile in Linux

First let’s create a safe directory to store our key files and upload them:

---
- name: Create keys directory
  file:
    path: /opt/keys
    state: directory
    owner: root
    group: root

# TODO run me only if the appropriate device is installed. Is it possible to use
# /dev/disk/by-uuid to compare the list?
- name: Upload key and secure it [hdd_1]
  copy:
    src: ./keys/hdd_1
    dest: /opt/keys/hdd_1
    owner: root
    group: root
    mode: '0400'

Then we can use the key, and instruct cryptsetup to open the device:

- name: Enable hdd_1 [SEAGATE 1TB]
  community.crypto.luks_device:
    uuid: 010d2738-d903-4b46-87be-91f255c7df37
    keyfile: /opt/keys/hdd_1
    state: opened

However so far, the device won’t be decrypted or mounted on the boot. Let’s take care of it:

- name: Setup crypttab [hdd_1]
  lineinfile:
    path: /etc/crypttab
    regexp: ^hdd_1
    line: hdd_1 UUID=010d2738-d903-4b46-87be-91f255c7df37 /opt/keys/hdd_1 luks

**NOTE** the following module currently doesn’t support the keyfile flag to automatically open the device:

# Note this is not tangled due the note above
 community.general.crypttab:
   backing_device: UUID=010d2738-d903-4b46-87be-91f255c7df37
   name: hdd_1
   keyfile: /opt/keys/hdd_1
   state: present

- name: Mount [hdd_1]
  mount:
    path: /mnt/hdd_1
    src: /dev/mapper/hdd_1
    fstype: ext4
    state: present

First I tried to implement the security by following Ansible Server Security document. However, then realized there is a project called Ansible Collection Hardening which would be more helpful if possible to use. So going to give it a try.

ansible-galaxy collection install devsec.hardening

Let’s put all of these together in a single playbook to run:

---
- name: Install Proper Access
  hosts: all
  remote_user: pi
  become: true
  gather_facts: true
  vars:
    sysctl_overwrite:
      vm.mmap_rnd_bits: 18
  collections:
    - devsec.hardening
  tasks:
    - include_tasks: system.yml
    - include_tasks: applications.yml
    - include_tasks: disks.yml
  roles:
    - os_hardening
    - ssh_hardening

• vm.mmap_rnd_bits is set to 32 on the hardening collection and the document confirms that there might be some issues with some systems. On Raspberry pi, 16 didn’t work either and raised “Invalid Argument” error. Instead I just checked cat /dev/proc/sys/vm/mmap_rnd_bits which showed 18 and set the value to this.