Autostart fails about ansible-prometheus HOT 15 CLOSED

I just upgraded from ansible 2.6.4 to 2.7.2 and ran the playbook again.

This time the prometheus service is enabled 👍

Always keep your dependencies up-to-date 😅

Thanks for your help!

from ansible-prometheus.

Could you check if /etc/prometheus/prometheus.yml exists and is readable by prometheus user. And also could you paste content of /etc/systemd/system/prometheus.service ?

from ansible-prometheus.

Also you cannot start prometheus just by running prometheus binary without flags, you need to start it with systemctl start prometheus

from ansible-prometheus.

prometheus.yml exists and is readable.

-rw-r-----   1 root       prometheus   465 Oct  3 15:51 prometheus.yml

/etc/systemd/system/prometheus.service

root@liffey:/etc/prometheus# cat /etc/systemd/system/prometheus.service
# Ansible managed
[Unit]
Description=Prometheus
After=network.target

[Service]
Type=simple
Environment="GOMAXPROCS=40"
User=prometheus
Group=prometheus
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus \
  --storage.tsdb.retention=30d \
  --web.console.libraries=/etc/prometheus/console_libraries \
  --web.console.templates=/etc/prometheus/consoles \
  --web.listen-address=0.0.0.0:9090 \
  --web.external-url=
PrivateTmp=true
PrivateDevices=true
ProtectHome=true
NoNewPrivileges=true
ReadWritePaths=/var/lib/prometheus
ProtectSystem=strict
ProtectControlGroups=true
ProtectKernelModules=true
ProtectKernelTunables=true


SyslogIdentifier=prometheus
Restart=always

[Install]
WantedBy=multi-user.target

from ansible-prometheus.

I don't see any issues with your config. Could you run:

systemctl start prometheus

And check its logs by using journalctl -f -u prometheus.service ?

from ansible-prometheus.

I can see though that prometheus service is disabled on your host. Which looks like a manual intervention or badly executed role since we are enabling that service in the last task:

from ansible-prometheus.

I did re-run playbook again to ensure that the last task gets executed and I also didn't manually disable the service 🤔 .

from ansible-prometheus.

After running role you should have status "enabled" when checking prometheus service.
Now you had this:

galen@liffey:/$ systemctl status prometheus
prometheus.service - Prometheus
   Loaded: loaded (/etc/systemd/system/prometheus.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

And important part is:

/etc/systemd/system/prometheus.service; disabled; vendor preset: enabled

Which should look more like:

/etc/systemd/system/prometheus.service; enabled; vendor preset: enabled

from ansible-prometheus.

I manually enabled the prometheus service with systemctl enable prometheus.

It does prompt for authentication though that forces you to choose an identity. Maybe this unusual prompt breaks the last ansible task?

galen@liffey:/$ systemctl enable prometheus
Synchronizing state of prometheus.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable prometheus
==== AUTHENTICATING FOR org.freedesktop.systemd1.reload-daemon ===
Authentication is required to reload the systemd state.
Multiple identities can be used for authentication:
1.  galen
2. <REDACTED>
...
<REDACTED>

from ansible-prometheus.

You need to do it as root. Prometheus is installed as a system service, not a user service.
Ansible uses become: true for similar purposes.

from ansible-prometheus.

Ah, I see thank you 😃

Well since the task ensure prometheus service is started and enabled does have become: true already, I'm not sure why running the role does not automatically enable the service then.

from ansible-prometheus.

Seems like this might be some sort of a bug in ansible systemd module. What OS are you using on your target system (where prometheus is installed)? And which ansible version?

from ansible-prometheus.

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
NAME="Ubuntu"
VERSION="18.04.1 LTS (Bionic Beaver)"

from ansible-prometheus.

Seems like this might be the issue you are running into: ansible/ansible#39116

from ansible-prometheus.

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

from ansible-prometheus.