Git Product home page Git Product logo

rpi-nas's Introduction

Raspberry PI NAS

Setup your own NAS on a Raspberry Pi.

Preview

Dashboard

dashboard

Monitoring

grafana

Table of Contents

Hardware

If you want to use a software based RAID setup instead, you need to format your drives first. You can use the following guides to setup a RAID 5 with Btfs:

OS

  • 64 bit Raspbian
  • Vanilla Rasbperry Pi OS Bullseye (e.g. Raspberry Pi OS LITE 64 bit) Installation guide

SSH

More documentation to enable SSH or generate Keys.

# Enable SSH
sudo systemctl enable ssh
sudo systemctl start ssh

# SSH Keys
ssh-keygen -t rsa -b 4096 -C

SMB Shares

Format Drives

# List Disks
sudo fdisk -l

# Open Disk (replace X with the disk letter)
sudo fdisk /dev/sdX 

# Create Partititon
n
default
default
default
w

# Check Table
sudo fdisk -l

# Format the partition (replace X with the disk letter)
sudo mkfs -t ext4 /dev/sdX1

More documentation can be found here.

Mount Drives

Prepare Disks

In order to identify the disks with a descriptive name, we need to add a label to the disks. This can be done with the following commands.

# List Disks
sudo fdisk -l

# Goal state
# /dev/disk/by-label/qnap (8TB)        | LABEL="qnap"

# Add Label to Disks: (replace X with the disk letter)
sudo e2label /dev/sdX1 qnap

Mount Disks manually

# Create Mount Points
sudo mkdir /mnt/qnap

# Check Permissions
stat /mnt/qnap

# Change Owner (replace $USER with your username)
sudo chown -R $USER:users /mnt/qnap/*

# Change Permissions
chmod -R 775 /mnt/qnap/*

# Mount Disks
mount /dev/disk/by-label/qnap /mnt/qnap/

# (Optional) Mount Disks with User Permissions
mount -o user=$USER /dev/disk/by-label/qnap /mnt/qnap/

More documentation can be found here.

Mount Disks automatically

# Identify UUIDs or Labels of the disks
sudo blkid
# Edit /etc/fstab
sudo nano /etc/fstab
# Add the following lines to /etc/fstab
LABEL=qnap /mnt/qnap ext4 nofail,x-systemd.device-timeout=1ms 0 0

More documentation can be found here.

Setup SMB Server

sudo apt-get update
sudo apt-get upgrade

# Install Samba
sudo apt-get install samba samba-common-bin

# Setup Shares
sudo nano /etc/samba/smb.conf

# Add Configuration to smb.conf (replace $USER with your username)
[nas]
    path = /mnt/qnap/nas
    writeable = Yes
    create mask = 0775
    directory mask = 0775
    public = no
    force user = $USER
    force group = $USER

[timemachine]
    path = /mnt/qnap/timemachine
    writeable = Yes
    vfs objects = catia fruit streams_xattr
    fruit:time machine = yes
    public = no

# Setup SMB User (replace $USER with your username)
sudo smbpasswd -a $USER

# Restart Samba service
sudo systemctl restart smbd

More documentation can be found here and here.

Setup TimeMachine Share

sudo nano /etc/avahi/services/samba.service

# Add Configuration to samba.service
  <?xml version="1.0" standalone='no'?><!--*-nxml-*-->
  <!DOCTYPE service-group SYSTEM "avahi-service.dtd">
  <service-group>
    <name replace-wildcards="yes">%h</name>
    <service>
      <type>_smb._tcp</type>
      <port>445</port>
    </service>
    <service>
      <type>_device-info._tcp</type>
      <port>9</port>
      <txt-record>model=Xserve1,1</txt-record>
    </service>
    <service>
      <type>_adisk._tcp</type>
      <port>9</port>
      <txt-record>dk0=adVN=timemachine,adVF=0x82</txt-record>
      <txt-record>sys=adVF=0x100</txt-record>
    </service>
  </service-group>

# Restart Avahi service
sudo systemctl restart avahi-daemon

More documentation can be found here. After modifying the samba.service file, the TimeMachine share should be visible in the Finder.

Other useful things

HD-Idle

Setup HDD spin down times to save energy and increase lifetime of the disks.

# Install hd-idle
sudo apt install hd-idle
# Configure hd-idle
sudo nano /etc/default/hd-idle

# Change line according to your available disks. Replace X with the disk letter.
# The following line will set the default to 0 and the custom idle time for disk sdX to 8 minutes. Save File afterwards.
HD_IDLE_OPTS="-i 0 -a sdX -i 480 -l /var/log/hd-idle.log"

# Restart hd-idle
sudo systemctl restart hd-idle

# Check if hd-idle is running
systemctl status hd-idle

# Check if disks are spinning down
cat /var/log/hd-idle.log

More documentation can be found here

Log2Ram

sudo apt update
sudo apt install log2ram

# Check if log2ram is running (after reboot!)
systemctl status log2ram

More documentation can be found here.

Check Disk Utilization

df -h

Docker Setup

Install Docker

I recommend to install it yourself following a simple guide.

Prepare Docker Environment

All following commands need to be run from your Raspberry PI. Either connect via SSH or direct access.

1. Clone Repository

git clone https://github.com/timoknapp/rpi-nas.git
cd rpi-nas

2. Configure your Setup

Replace Placeholders in docker-compose.yml file:

  • ${PATH_TO_DISK} with related Path on your PI

  • Execute the following command on your PI: id `whoami`

    • Replace ${USER_ID} with the integer value of uid.
    • Replace ${GROUP_ID} with the integer value of gid.
  • Set password postgres user

3. Run Compose-Stack

docker-compose up

4. Try it out

Opening a browser with the IP of your PI should show now the Heimdall dashboard. A configured dashboard would like like one on top. (e.g. http://IP-OF-YOUR-PI)

Components

Following show all the applications of the docker-compose.yml related to their exposed ports on the host.

Application Port URL Optional
Portainer 9000 http://localhost:9000
Heimdall Dashboard 80, 443 http://localhost, https://localhost
Pi-hole 53, 8080 http://localhost:8080
CloudflareD -
Homebridge 8581 http://localhost:8581
Deconz Conbee 8888, 8443 http://localhost:8888, https://localhost:8443
Plex 32400 http://localhost:32400/web/index.html yes
CloudCmd 8008 http://localhost:8008 yes
pyLoad 8088 http://localhost:8088
Nextcloud 8081 http://localhost:8081 yes
Home-Assistant(1) 8123 http://localhost:8123 yes
Grafana 3000 http://localhost:3000
InfluxDB -
Telegraf -
Internet Speedtest - yes

Optional means that the application is not necessary for the NAS to work properly. It is just a nice to have. If you want to use it, you need to uncomment the related lines in the docker-compose.yml file.

(1) In order to expose your external devices using Zigbee/Z-Wave dongles to the Home Assistant container, you can read this guide

Configure Pi-hole

  • Open http://IP-OF-YOUR-PI:8080/admin and login with the password you set in the docker-compose.yml file.

  • Go to Settings -> DNS -> Interface settings and change the setting from Allow only local requests to Permit all origins -> Save. (This could already be set by the environment variable DNSMASQ_LISTENING in the docker-compose.yml file)

    • This will add the following line in /etc/dnsmasq.d/01-pihole.conf:
    except-interface=nonexisting

    Since the Pi-hole is running in a docker container, it is not possible to use the Allow only local requests setting as its only considering the local network of the container.

  • Go to Settings -> System -> Disable Query Logging.

    Since we want to use the Pi-hole as a DNS server only, we don't need to log any queries. This will reduce the amount of writes to the disk and will allow us to run Pi-hole in anonymous mode.

  • Add the following lines to /etc/pihole/pihole-FTL.conf:

    PRIVACYLEVEL=3 #; 0=show everything, 1=hide domains, 2=hide domains and clients, 3=anonymous mode
    #; MAXLOGAGE=24.0 #; up to how many hours of logs to show in pihole web interface
    MAXDBDAYS=7 #; delete entries older than 30 days. Setting this to 0 will disable the database.
    DBINTERVAL=60.0 #; write to the pihole-FTL.db file every 30 minutes (lets the HDD spin down)
    #; DBFILE=/etc/pihole/pihole-FTL.db #; path to the database file. Setting this to DBFILE= disables the database altogether
  • Restart Pi-hole container!

Configure CloudflareD

Tdb.

  • Add your Cloudflare API Token to the TUNNEL_TOKEN variable in the docker-compose.yml file.

More configuration options can be found here or here.

rpi-nas's People

Contributors

timoknapp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

rpi-nas's Issues

volumes needs to be recreated

Hi Timo,

with the stack below I get the error:

Volume "X" needs to be recreated - driver has changed.

=======================

  • Removed Portainer as it is already running onder OMV
  • Removed Plex as I don't use it
  • Removed homeassistant as I don't use it

What needs to be added to get these following Apps running under the Portainer?

volumes:
nextcloud_postgres: ?
nextcloud: ?
grafana-storage: ?
influxdb-storage: ?

networks:
influxdb: ?

=======================
version: "2"

services:
heimdall: # Dashboard for all you Apps
image: linuxserver/heimdall
container_name: heimdall
environment:
- PUID=998
- PGID=100
- TZ=Europe/Amsterdam
volumes:
- /dev-disk-by-label-data/AppData/heimdall:/config
ports:
- 8080:80
- 443:443
restart: unless-stopped

portainer: # UI based management of Docker
image: portainer/portainer
container_name: portainer
command: -H unix:///var/run/docker.sock
restart: unless-stopped
ports:
- 9000:9000
- 8000:8000
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- portainer_data:/data

cloudcmd: # UI based filemanager
image: team0/rpi-cloudcmd # arm-based docker images for cloudcmd
container_name: cloudcmd
ports:
- 8008:8000
volumes:
- ~:/root
- /:/mnt/fs

pyload: # pyload - download manager for your NAS
image: linuxserver/pyload
container_name: pyload
environment:
- PUID=998
- PGID=100
- TZ=Europe/Amsterdam
volumes:
- /dev-disk-by-label-data/AppData/pyload:/config
- /dev-disk-by-label-data/AppData/downloads:/downloads
ports:
- 8088:8000
- 7227:7227 #optional
restart: unless-stopped

nextcloud_postgres: # DB for Nextcloud
image: postgres
container_name: nextcloud_postgres
restart: always
volumes:
- nextcloud_postgres:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=ABCDE12345
- POSTGRES_DATABASE=nextcloud
- POSTGRES_USER=nextcloud

nextcloud: # Nextcloud
image: nextcloud
container_name: nextcloud
ports:
- 8081:80
links:
- postgres
volumes:
- nextcloud:/var/www/html
restart: unless-stopped

Monitoring Stack:

telegraf:
image: telegraf:latest
container_name: telegraf
command: --config-directory /etc/telegraf/telegraf.d
depends_on:
- influxdb
networks:
- influxdb
volumes:
- ./monitoring/telegraf:/etc/telegraf
restart: always

influxdb:
image: influxdb:latest
container_name: influxdb
networks:
influxdb:
volumes:
- influxdb-storage:/var/lib/influxdb
restart: unless-stopped

grafana:
image: grafana/grafana:latest
container_name: grafana
depends_on:
- influxdb
ports:
- 3000:3000
networks:
- influxdb
environment:
- GF_INSTALL_PLUGINS=grafana-piechart-panel
volumes:
- grafana-storage:/var/lib/grafana
- ./monitoring/grafana/provisioning:/etc/grafana/provisioning
restart: unless-stopped

volumes:
nextcloud_postgres:
nextcloud:
grafana-storage:
influxdb-storage:

networks:
influxdb:

=======================

Thank you for you reply!

Is there a solution for 3D printing shell

Argon has launched a new raspberry pie shell Argon eon, but the crowdfunding price is a little expensive, is there a more cost-effective 3D printing solution, want to print a shell with board card deck nas
It is said that the hard disk connected to the raspberry pie USB interface through the USB cable can easily cause data damage, and the omv system does not support wireless access to the USB port. So I want to get a shell, a better group of nas for onboard interface.

pyload misses "services:" ?

Hi Timo,

With your compose docker pyLoad misses the line "services:" ?

=> services:
  pyload: # pyload - download manager for your NAS
    image: linuxserver/pyload
    container_name: pyload
    environment:
      - PUID=${USER_ID}
      - PGID=${GROUP_ID}
      - TZ=Europe/Berlin
    volumes:
      - ${PATH_TO_DISK}/appdata:/config
      - ${PATH_TO_DISK}/downloads:/downloads
    ports:
      - 8088:8000
      - 7227:7227 #optional
    restart: unless-stopped

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.