Internal or External access to services about ansible-hms-docker HOT 20 CLOSED

Just to clear some things up, you're wanting to expose containers (other than Overseerr) to the public internet, is that correct? Such as exposing your nzbget container to the public internet so that people can go to https://nzbget.<your domain> to login to your nzbget instance?

If so, I assume you're aware of the security risks and requirements of doing this? Such as needing to configure authentication on the exposed services manually (as by default they have no authentication).

Implementing per-container public internet exposure should be pretty straightforward, I just never thought about anyone needing to expose the other containers to the public tbh (as it's a bigger security impact).

If it'd be better, would an external-ipwhitelist variable work? This way you can control which IP's can access your publicly exposed containers, but this would require manual updates to the IP list if your users have dynamic IPs from their ISP. Just a thought, I can likely implement both shortly.

from ansible-hms-docker.

Just to clear some things up, you're wanting to expose containers (other than Overseerr) to the public internet, is that correct? Such as exposing your nzbget container to the public internet so that people can go to https://nzbget. to login to your nzbget instance?

Yes sir :)

If so, I assume you're aware of the security risks and requirements of doing this? Such as needing to configure authentication on the exposed services manually (as by default they have no authentication).

Yes im aware of this and im about to try and include Authelia in your repo aswell :)

If it'd be better, would an external-ipwhitelist variable work?

This would work, but this also needs to update the docker-compose.yml aswell?

Super thanks for the really quick replay!

from ansible-hms-docker.

Just created a new branch to test implementation, I have updated the code but have not yet verified it's working correctly

Link to changes if you're interested: master...per-container-public-exposure

from ansible-hms-docker.

Also good call on the required updating of the docker-compose.yml whenever an external-ipwhitelist value would change. Definitely don't want to possibly cause an outage by updating that variable that would impact all containers in the compose, requiring them to restart. I didn't think about that ;)

from ansible-hms-docker.

This works well, thanks for the help :)

Now i gonna try and intergrate Authelia into this aswell, if a service is exposed to the internet then it should be protected by authelia auth.

from ansible-hms-docker.

One question, how does this work? im trying to understand this.

In defaults.yml i want to expose Sonarr, then i choose "yes"

then in docker-compose.yml file this line is called i think?

{% if not hms_docker_container_map['sonarr']['expose_to_public'] %}

This line is adding to the external ipwhitelist, but when i look at the routes in traefik dashboard, sonarr doesnt have any middleware at all, how can in statement add my own middlewares?

How does that line see if it should expose sonarr or not? Does it has something todo with "if not" statement?

from ansible-hms-docker.

By changing the setting to expose_to_public: yes, this will cause the template to not add in the internal whitelist middleware to the compose file for that container, meaning there will be no IP restrictions set on who can access the container.

So this docker-compose.yml template line essentially says "if the container is not exposed to the public, add in the internal whitelist middleware. If it should be exposed, do not add the internal whitelist middleware".

from ansible-hms-docker.

Okey. so to add my own middleware's i need to add something like

{% if hms_docker_container_map['portainer']['expose_to_public'] %}
"traefik.http.routers.portainer-{{ project_name }}.middlewares=auth@authelia"
another-middleware-here
and-here

Hope you understand what im trying to ask :)

from ansible-hms-docker.

Yeah I think I get the idea of what you're trying to do; it sounds like you're trying to add middlewares on a per-container basis which can be easily controlled by a variable, and still have the ability to specify other middlewares.

I tried to make this project as future-proof and scalable as possible, so here's a general idea of how I would do it.

(And I'm actually in the process of adding in authelia already if you want to wait for me to push that out)

But here's an example of how you could add a simple if statement per container to enable authelia or not:

...
  labels:
      - "traefik.label.here"
      - "another.traefik.label"
{% if hms_docker_container_map['portainer']['authelia'] %}
      - "traefik.http.routers.portainer-{{ project_name }}.middlewares=authelia@docker"
{% endif %}
      - "another.traefik.label"

This snippet means that within the hms_docker_container_map, if portainer has its authelia key set to yes, it will add in that line for the middleware.

This would also require you to add another "key" into the hms_docker_container_map per container with authelia: no (or yes to enable authelia for that container).

from ansible-hms-docker.

I just pushed my basic authelia implementation to a new branch: f394b7f

Try it out and let me know how it works. It should auto-generate a working config (for the most part). I've never used authelia before so this'll be fun to figure out.

from ansible-hms-docker.

After looking at other options, I may switch it over to using authentik instead. It has better SAML and OAuth support, whereas authelia seems to be more of a http basic-auth layer on top of things. Like if you try to login to Portainer with authelia, you'll have 2 layers of authentication.

from ansible-hms-docker.

I just pushed my basic authelia implementation to a new branch: f394b7f

Try it out and let me know how it works. It should auto-generate a working config (for the most part). I've never used authelia before so this'll be fun to figure out.

This is awesome, gonna try this at once!

After looking at other options, I may switch it over to using authentik instead. It has better SAML and OAuth support, whereas authelia seems to be more of a http basic-auth layer on top of things. Like if you try to login to Portainer with authelia, you'll have 2 layers of authentication.

Never played with Authentik, only visit their webpage, gonna read up about it :)

But first im gonna try your latest push :)

from ansible-hms-docker.

Trying this out, and its not working i'm affraid.

TASK [hmsdocker : Ensure docker-compose.yml file.] ***************************************************************************************************************
task path: /home/xxx/ansible-hms-docker/roles/hmsdocker/tasks/main.yml:52

The full traceback is:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/ansible/template/init.py", line 1156, in do_template
res = j2_concat(rf)
File "", line 171, in root
File "/usr/lib/python3/dist-packages/jinja2/runtime.py", line 639, in _fail_with_undefined_error
raise self._undefined_exception(hint)
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'expose_to_public'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/ansible/plugins/action/template.py", line 150, in run
resultant = templar.do_template(template_data, preserve_trailing_newlines=True, escape_backslashes=False)
File "/usr/lib/python3/dist-packages/ansible/template/init.py", line 1193, in do_template
raise AnsibleUndefinedVariable(e)
ansible.errors.AnsibleUndefinedVariable: 'dict object' has no attribute 'expose_to_public'
fatal: [192.168.56.10]: FAILED! => {
"changed": false,
"msg": "AnsibleUndefinedVariable: 'dict object' has no attribute 'expose_to_public'"

Since im not a coder, but from what i can see maybe there is a problem in docker-compose.yml.j2 file?
To this looks like it trying to expose the service and then close it using authelia?

{% if not hms_docker_container_map['portainer']['expose_to_public'] %}
- "traefik.http.routers.portainer-{{ project_name }}.middlewares=internal-ipwhitelist"
{% if (hms_docker_container_map['authelia']['enabled'] or authelia_enabled) and traefik_ssl_enabled and hms_docker_container_map['portainer']['authelia'] %}
- "traefik.http.routers.portainer-{{ project_name }}.middlewares=authelia-{{ project_name }}@docker"

from ansible-hms-docker.

Ahah I actually had the exact same issue when working on it last night!

Basically, if you use a custom variable file, you may need to add the expose_to_public manually to each container within the map like so (example using only the radarr container):

hms_docker_container_map:
  radarr:
    enabled: yes
    directory: yes
    traefik: yes
    expose_to_public: no
...

But you're sorta correct! The issue isn't directly in the compose file, it is throwing an error because it expects a variable to exist in the vars file(s) and it can't render the template since it cannot find these variables. (Sorta my fault, I just realized I don't have something to handle if the variable doesn't exist)

Since it seems like you're just starting to get into this field, here's some helpful resources:
Using markdown to style code blocks (will help format your code in comments and stuff so it doesn't appear as "quoted" text)
Using Jinja2 templates with Ansible (Ansible uses Jinja2 to render template files)
Ansible variables (Just a general ansible variable overview)

from ansible-hms-docker.

Authelia has now been replaced with Authentik in dbd1c8b.

Just follow the instructions at the bottom of the README.md and that should get you going in the right direction.

Edit: Also, I think I fixed the logic for the variables within the template so it should no longer error, it should fail-safely if newer variables are not defined in an existing variables file

from ansible-hms-docker.

Hi again sorry for late reply, been busy @ work this week.

But i have followed the Authentik installation guide and it does not work, im getting an middleware error in traefik

time="2022-06-01T23:31:59+02:00" level=error msg="middleware "authentik-proxy-home-local-sonarr-router@docker" does not exist" entryPointName=web routerName=sonarr-home-local@docker
time="2022-06-01T23:31:59+02:00" level=error msg="middleware "authentik-proxy-home-local-sonarr-router@docker" does not exist" entryPointName=websecure routerName=websecure-sonarr-home-loca@docker

Have no clue to fix this, this is waaay over my head im affraid

from ansible-hms-docker.

Seems to me that either the Outpost (within Authentik) is not running or is not configured correctly. A 404 error (in my experience while developing this) means the Outpost is not working correctly and needs to be changed/rebuilt within Authentik, and a 500 error means there's duplicate Traefik routers/middlewares. Based on your specific error message, it appears the Outpost is not running (but it may be failing to start due to a config error).

Small amount of background info: an Outpost is like an additional proxy-layer. This Outpost/proxy-layer needs to be working order for the rest of the traffic flow to work. Authentik is able to automatically create these new Outpost containers for you (assuming the config is correct).

You can view if the Outpost is running using Portainer (or Docker CLI if you're familiar with it), and then you can use the Traefik UI to see the active routers and middlewares. I will admit that first playing around with Authentik is very confusing, it took me a while to wrap my head around how the Outposts work (and which is why I created a config generator for the Outposts).

I am working on a way to automatically provision everything required within Authentik using its API, but it will take some time to get working correctly. I'd say it's currently at about 30% completed.

from ansible-hms-docker.

In portainer i have the following containers running from Authentik

authentik-proxy-homelab-local-sonarr
authentik-server
authentik-worker
authentik-redis
authentik-geoupdate
authentik-postgresql

So far so good i think.
And in traefik i have

Http Routers:

Host(sonarr.homelab.local) && PathPrefix(/outpost.goauthentik.io)

Middleware:

authentik-proxy-homelab-local-sonarr-router@docker

And the weird thing is that if i go into Sonarr's http router to see middlewares and other stuff there is no middleware to it.

I have copied the config that is generated to /opt/homelab/apps/authentik/outposts/authentik-sonarr.outpost.yml into Authentik -> Applications -> Outposts (when i created the Sonarr outpost).

from ansible-hms-docker.

Hmmmm I'm not too sure what could be going on then without looking at it myself.

A valid Sonarr HTTP router in Traefik with Authentik middleware should look like this. Note that I only have HTTPS enabled and this is the websecure-sonarr-{{ project_name }} HTTP router within Traefik, not the router for the authentik proxy that should also be in the list.

from ansible-hms-docker.

Closing due to inactivity

from ansible-hms-docker.