ibizaman / selfhostblocks Goto Github PK

This would allow blocks or services to define what secrets they need and the user could just fill-in the path to the secret.

https://matrix.org/bridges/

Quoting from the README:

One important goal of SHB is to be the smallest amount of code above what is available in nixpkgs. It should be the minimum necessary to make packages available there conform with the contracts. This way, there are less chance of breakage when nixpkgs gets updated.

However, it looks like the *arr configuration modules go against that ethos:

• the XML generator could be upstreamed to nixpkgs.
• the various configuration options could be upstreamed to each of the *arr modules.

This requires a few things:

• The user’s deploy tool must allow to give this pinned nixpkgs commit to the target host. I know colmena and deploy-rs does that. I didn’t check others.
• Write a manual page explaining how to do that: https://shb.skarabox.com/usage.html
• SHB has a cron job to automate bumping the nixpkgs pinned commit. Let’s be bold and say to run this daily and to merge automatically if tests pass. https://blog.tiserbox.com/posts/2023-12-25-automated-flake-lock-update-pull-requests-and-merging.html
• The demos use that pinned commit too. #86
• Propose nixpkgs pinned commit for unstable and latest two releases.

From everything I read, that's what NixOS is using now.

Also, this makes the doc generator understand markdown which is a nice improvement IMO.

A few links:
https://discourse.nixos.org/t/this-month-in-nix-docs-2-april-2023/27899
https://search.nixos.org/packages?channel=23.05&show=nixos-render-docs&from=0&size=50&sort=relevance&type=packages&query=nixos-render-docs
nix-community/home-manager#4673

This test should spin up Nginx, Authelia and a stub upstream server that returns in a json response all the headers of requests it receives.

The test should configure Authelia to protect some endpoints of the stub server.

The test should make requests to the Nginx instance and validate the received payload from the stub server ha the correct headers.

Interesting links:
https://www.authelia.com/integration/trusted-header-sso/introduction/
https://openidconnect.net/
https://oidcdebugger.com/

No big deal, but thought I'd file this here so we remember to fix it at some point.

There are two missing directories preventing successful start of the jellyfin service when running on a fresh install.

/var/lib/jellyfin/config
/var/lib/jellyfin/plugins/configurations

For anyone who might run into this, you can get around it with

> sudo mkdir -p /var/lib/jellyfin/config
> sudo mkdir -p /var/lib/jellyfin/plugins/configurations
> sudo chown -R jellyfin:jellyfin  /var/lib/jellyfin

Otherwise, the code in the demos could drift.

Test backup and restore. What else?

Make it so the groups are added declaratively.

Bonus points if the groups are read only in LLDAP.

This will require development on LLDAP side, see lldap/lldap#643 (comment)

https://github.com/nextcloud/notify_push

The GitHub job to update the root flake inputs should be extended to also update the demos.

There might be an issue because the demos have their selfhostblocks input URI pointing to the GitHub repository directly. That will of course not have the update just done by the GitHub job. So we'll need to run the following command for each demo but not commit that:

nix flake lock --override-input selfhostblocks ../.. --update-input selfhostblocks

Now that I'm writing this, maybe the above command should instead live inside the GitHub job testing the demos?

I was poking around and just wanted to make a comment.

I think you should enable the recommend proxy settings for nginx:
services.nginx.recommendedProxySettings

There's a few other ones on that same note that are interesting to enable as well.
services.nginx.recommendedZstdSettings
services.nginx.recommendedTlsSettings
services.nginx.recommendedOptimisation
services.nginx.recommendedGzipSettings
services.nginx.recommendedBrotliSettings

https://github.com/janeczku/calibre-web

The demo builds in CI uses a matrix of demo names. We should avoid hardcoding this matrix because we will forget to maintain this. We can use something like shown here: https://github.com/tjtharrison/demo-github-dynamic-matrix/blob/main/scripts/generate-matrix.py

Looks like an interesting link https://community.home-assistant.io/t/anyone-have-authelia-working-with-ha-to-handle-authentication/321233/4

Split from #134. They are currently generated only once. Also, the options related to this should follow the acme options from nixpkgs. And the options should be added to the contract.

This should spin up a real DB and test that we correctly create the required database and user.

https://invoiceninja.com/

Certs are created with read-only group permissions for acme:acme. However, nginx is not part of the acme group, so it does not have access.

Fixed this temporarily on my server by adding nginx to the acme group, but maybe it makes more sense to add a new group?

For example https://invidious.io/
I never used any so I’d line recommendations

The implementation would use dnsmasq as a starter backend (because I know it well).

Functionality should be:

• User defines where a reachable service lives. For example, the Nextcloud service lives on the host 192.168.1.50.
• User defines what the fqdn is to reach the services. For example, the Nextcloud service is accessible at nextcloud.mydomain.com.
• This blocks creates a DNS server that answers to the question nextcloud.mydomain.com with 192.168.1.50.

We should automatically datasources and optionally dashboards and alerts.

Should integrate with Grafana.

They do not point to existing files.

Follow advice at https://christine.website/blog/paranoid-nixos-2021-07-18/

Essentially, there should be a wrapper around restic or borgmatic so we don’t need to put in the credentials.

Also, we should have a way to fix errors like:

• a file that’s going to be backed up cannot be read by the backup user. This leads to restic bailing out.
• repository is locked by unlocking it. Sometimes there’s a good reason for it being locked but sometimes it’s just a lock that’s not removed after an aborted job. Example:

  RESTIC_PASSWORD_FILE=/run/secrets/restic/passphrases/<name> RESTIC_REPOSITORY=<repo> restic unlock

• cache cleanups, same as above but with cache --cleanup
• running repair-index to fix restic/restic#2191

https://www.tubearchivist.com/

An impressive piece of work!

I read that your are having problem with nixops2, that projects seems have be forgotton about.

May i recommend you have a look at nixinate.

It's just a shell script, but it uses nix primitives to deploy machines. I think it would be idea for selfhostblocks.

https://github.com/MatthewCroughan/nixinate

give it a try and see what you think, there are other more sophisticated tools, but for this job it's perfect and simple!!

Making a flake which can deploy to multiple systems should be very easy.

reach out if you got questions!

I see your using colmena for deployment , so that also might be a great option.

https://github.com/axcore/tartube

https://samber.github.io/awesome-prometheus-alerts/rules.html#promtail

https://roundcube.net/

Issue

The authelia OIDC clients each get written to their own yaml files, and Authelia is launched with
authelia --config config.yaml,client1.yaml,client2.yaml,client3.yaml

However, it seems like authelia reads the yamls as if they were all in the same file, meaning that each OIDC client overrides the previous one because each client defines the sequence identity_providers.oidc.clients. I.e. Authelia doesn't implement any merging logic (like nix does).

I've checked that this is the case by removing one and one OIDC client from my SHB config, and checking that only the last one works. (tested through openidconnect.net, and by checking the authelia debug log, which prints registered clients).

The solution would probably be to write out one clients.yaml file containing all clients instead of one file per client.

It would be great to allow the demos to be deployed to without requiring to deploy through Colmena. Or at least keeping that optionally.

A few reasons:

• If we include all the configuration in configuration.nix, we can share the nix store between the VM and the host. This makes for much faster demos to run.
• Some users experienced issues when running build-vm-with-bootloader and could not run the demos.
• Would make #53 possible.

From what I’ve seen on my own server as far as Nextcloud performance goes… having a pooling mechanism in front of Postrgres will be very valuable.

services.openssh.permitRootLogin and -passwordAuthentication have been renamed to -PermitRootLogin and PasswordAuthentication

It should probably integrate with Nextcloud.

Interesting links:

https://www.authelia.com/integration/openid-connect/nextcloud/
https://github.com/lldap/lldap/blob/main/example_configs/nextcloud_oidc_authelia.md

Instead of taking the path to a sops file, take a path to the secrets directly.

• blocks/vpn
• blocks/backup
• #169
• services/jellyfin
• services/home-assistant
• update manual
• update flake.nix

There are commits where I already removed sops dependency:

• 31f9589
• https://github.com/ibizaman/selfhostblocks/pull/74/files

https://github.com/AnalogJ/scrutiny

This is used to monitor hard drive health based on statistics from https://backblaze.com.

Got the following warning while running nix build .#checks.x86_64-linux.vm_postgresql_peerAuth:

trace: warning: 
      `services.postgresql.ensureUsers.*.ensurePermissions` is used in your expressions,
      this option is known to be broken with newer PostgreSQL versions,
      consider migrating to `services.postgresql.ensureUsers.*.ensureDBOwnership` or
      consult the release notes or manual for more migration guidelines.

      This option will be removed in NixOS 24.05 unless it sees significant
      maintenance improvements.

We should get alerts for:

• backup failed X times in a row
• backup did not succeed in the most recent X hours
  Whichever comes first.

Inspirations:
https://nixos.wiki/wiki/Borg_backup#Notifications_when_backup_fails

Currently, the apps and configuration must be done manually.

We should expose the security.acme.extraDomainNames option in the SSL block for letsencrypt.

https://ersatztv.org/

Ideally, Jellyfin should be configured automatically to access the Ersatztv instance https://ersatztv.org/docs/user-guide/configure-clients#jellyfin