Git Product home page Git Product logo

Comments (11)

marefr avatar marefr commented on September 17, 2024

Please try doing something along these lines, e.g. import a self-signed CA ssl certificate into the linux version of google chrome. Still think if you have a self signed cert for localhost it will never work in regular Chrome/Chromium, but I may be wrong.

from grafana-image-renderer.

peteeckel avatar peteeckel commented on September 17, 2024

I actually tried something like that in one of my first attempts to fix the problem, see grafana/grafana#19842 (comment).

Following the guidelines in your link, I slightly modfied my second approach (the specific certificate for the Grafana user) to grant even more trust to the certificates, system-wide as well as user specific, in old and new nssdb format, with the root cert and the sub-ca cert just to be really sure I didn't miss anything:

certutil -d sql:/etc/pki/nssdb -A -n hbr-dev-subca01 -t TCP,TCP,TCP -i /etc/pki/tls/certs/hbr-dev-subca01.crt.pem
certutil -d sql:/etc/pki/nssdb -A -n hbr-dev-root-ca -t TCP,TCP,TCP -i /etc/pki/tls/certs/hbr-dev-root-ca.crt.pem
certutil -d /etc/pki/nssdb -A -n hbr-dev-subca01 -t TCP,TCP,TCP -i /etc/pki/tls/certs/hbr-dev-subca01.crt.pem
certutil -d /etc/pki/nssdb -A -n hbr-dev-root-ca -t TCP,TCP,TCP -i /etc/pki/tls/certs/hbr-dev-root-ca.crt.pem
certutil -d sql:/usr/share/grafana/.pki/nssdb -A -n hbr-dev-subca01 -t TCP,TCP,TCP -i /etc/pki/tls/certs/hbr-dev-subca01.crt.pem
certutil -d sql:/usr/share/grafana/.pki/nssdb -A -n hbr-dev-root-ca -t TCP,TCP,TCP -i /etc/pki/tls/certs/hbr-dev-root-ca.crt.pem
certutil -d /usr/share/grafana/.pki/nssdb -A -n hbr-dev-subca01 -t TCP,TCP,TCP -i /etc/pki/tls/certs/hbr-dev-subca01.crt.pem
certutil -d /usr/share/grafana/.pki/nssdb -A -n hbr-dev-root-ca -t TCP,TCP,TCP -i /etc/pki/tls/certs/hbr-dev-root-ca.crt.pem

Checking the success of the operation reveals the following status:

[root@master1 grafana]# certutil -d sql:/etc/pki/nssdb -L

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

hbr-dev-root-ca                                              CT,C,C
hbr-dev-subca01                                              CT,C,C

[root@master1 grafana]# certutil -d /etc/pki/nssdb -L

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

hbr-dev-subca01                                              CT,C,C
hbr-dev-root-ca                                              CT,C,C

[root@master1 grafana]# certutil -d sql:/usr/share/grafana/.pki/nssdb -L

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

hbr-dev-root-ca                                              CT,C,C
hbr-dev-subca01                                              CT,C,C

[root@master1 grafana]# certutil -d /usr/share/grafana/.pki/nssdb -L

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

hbr-dev-root-ca                                              CT,C,C
hbr-dev-subca01                                              CT,C,C

So both relevant CA certificates are in all potential nssdb instances, with all possible trust levels (the link you sent is effectively overdoing it a bit, resp. trying to apply trust levels that don't work, but never mind :-)). If any of these settings were efficient it should work now.

Ah yes, just to make sure the certificates themselves work, are the same the server uses, and are properly installed in the NSS databases:

[root@master1 grafana]# openssl s_client -connect 192.168.106.101:3443 -showcerts
CONNECTED(00000003)
depth=2 C = DE, ST = Hessen, [...], OU = HBR DEV Root CA
verify return:1
depth=1 C = DE, ST = Hessen, [...], OU = HBR DEV SubCA 01
verify return:1
depth=0 C = DE, ST = Hessen, [...], CN = grafana.xxx.xxx
verify return:1
---
Certificate chain
 0 s:/C=DE/ST=Hessen/[...]/CN=grafana.xxx.xxx
   i:/C=DE/ST=Hessen/[...]/OU=HBR DEV SubCA 01
-----BEGIN CERTIFICATE-----
MIIG2jCCBMKgAwIBAgIUMgrJGgTdxzIrMn62yD26YHlYp+UwDQYJKoZIhvcNAQEL
BQAwgYQxCzAJBgNVBAYTAkRFMQ8wDQYDVQQIDAZIZXNzZW4xFDASBgNVBAcMC0Jh
[...]
OVMYrxlAqDuKauqLPM9dvS2xthaYfE30NU80MKQUvnNGgiZJ9nRqIfwlQp45wotp
hUjwhgBW/GIRiyvBU30LgwZTrf3NA/SdqxweLT5cGQzeOVnBQ34cyEo+sTDpKxmN
yPgk6xGx8q7fRdz7FEKOYSNdauWhnwNTZZrHThC+
-----END CERTIFICATE-----
 1 s:/C=DE/ST=Hessen/[...]/OU=HBR DEV SubCA 01
   i:/C=DE/ST=Hessen/[...]/OU=HBR DEV Root CA
-----BEGIN CERTIFICATE-----
MIIFujCCA6KgAwIBAgIUFrsUndqtaf4SUA42htpLHqurylYwDQYJKoZIhvcNAQEL
BQAwgYMxCzAJBgNVBAYTAkRFMQ8wDQYDVQQIDAZIZXNzZW4xFDASBgNVBAcMC0Jh
[...]
0aqc+fTvsK7EBlTBYyCrNNrd68Jd30KaMAw3Tw9CP5LyLdXaaoK7PH08kvb2JbSg
7yHQq+UfA1WNyTRTn8yvvysZEyLKK8CRtUNSU323
-----END CERTIFICATE-----
---
Server certificate
subject=/C=DE/ST=Hessen/[...]/CN=grafana.xxx.xxx
issuer=/C=DE/ST=Hessen/[...]/OU=HBR DEV SubCA 01
---
No client certificate CA names sent
---
SSL handshake has read 3523 bytes and written 887 bytes
---
[...]
    Verify return code: 0 (ok)
---

[root@master1 ~]# openssl x509 -noout -fingerprint -hash -text | egrep -A1 '(Subject Alter|Subject:|Fingerprint)'
-----BEGIN CERTIFICATE-----
MIIG2jCCBMKgAwIBAgIUMgrJGgTdxzIrMn62yD26YHlYp+UwDQYJKoZIhvcNAQEL
BQAwgYQxCzAJBgNVBAYTAkRFMQ8wDQYDVQQIDAZIZXNzZW4xFDASBgNVBAcMC0Jh
[...]
hUjwhgBW/GIRiyvBU30LgwZTrf3NA/SdqxweLT5cGQzeOVnBQ34cyEo+sTDpKxmN
yPgk6xGx8q7fRdz7FEKOYSNdauWhnwNTZZrHThC+
-----END CERTIFICATE-----
SHA1 Fingerprint=46:D9:EE:7F:A6:2C:85:D8:EA:48:BF:D5:39:9E:5C:74:1E:C6:D7:F2
084a673f
--
        Subject: C=DE, ST=Hessen, [...], CN=grafana.xxx.xxx
        Subject Public Key Info:
--
            X509v3 Subject Alternative Name: 
                DNS:master1.xxx.xxx, IP Address:192.168.106.101, DNS:master2.xxx.xxx, IP Address:192.168.106.102, IP Address:192.168.106.52, DNS:grafana.xxx.xxx

[root@master1 ~]# openssl x509 -noout -fingerprint -hash -text | egrep -A1 '(Subject Alter|Subject:|Fingerprint)'
-----BEGIN CERTIFICATE-----
MIIFujCCA6KgAwIBAgIUFrsUndqtaf4SUA42htpLHqurylYwDQYJKoZIhvcNAQEL
BQAwgYMxCzAJBgNVBAYTAkRFMQ8wDQYDVQQIDAZIZXNzZW4xFDASBgNVBAcMC0Jh
[...]
0aqc+fTvsK7EBlTBYyCrNNrd68Jd30KaMAw3Tw9CP5LyLdXaaoK7PH08kvb2JbSg
7yHQq+UfA1WNyTRTn8yvvysZEyLKK8CRtUNSU323
-----END CERTIFICATE-----
SHA1 Fingerprint=16:33:EC:24:27:FB:AB:86:56:A0:91:A2:62:97:8C:82:66:8E:D7:63
95023bec
--
        Subject: C=DE, ST=Hessen, [...], OU=HBR DEV SubCA 01
        Subject Public Key Info:

[root@master1 ~]# certutil -d /etc/pki/nssdb -L -n  hbr-dev-root-ca | grep -A1 'Fingerprint (SHA1)'
    Fingerprint (SHA1):
        E1:1B:7A:6A:88:1F:78:5C:CB:E4:30:D5:A5:6C:0D:5E:A1:DC:CE:F2
        
[root@master1 ~]# certutil -d /etc/pki/nssdb -L -n  hbr-dev-subca01 | grep -A1 'Fingerprint (SHA1)'
    Fingerprint (SHA1):
        16:33:EC:24:27:FB:AB:86:56:A0:91:A2:62:97:8C:82:66:8E:D7:63

I hope I didn't miss anything.

So now I restarted the two Grafana instances, and tried again to use the renderer:

t=2019-12-04T12:39:22+0000 lvl=eror msg="Render request failed" logger=rendering error=map[] url="https://192.168.106.101:3443/d-solo/zxDJxNaZk/graphite-metrics?orgId=1&refresh=1m&from=1575438321300&to=1575459921300&var-Host=master1&panelId=4&width=1000&height=500&tz=Europe%2FBerlin&render=1" timestamp=0001-01-01T00:00:00.000Z
t=2019-12-04T12:39:22+0000 lvl=eror msg="Rendering failed." logger=context userId=1 orgId=1 uname=admin error="Rendering failed: Error: net::ERR_CERT_AUTHORITY_INVALID at https://192.168.106.101:3443/d-solo/zxDJxNaZk/graphite-metrics?orgId=1&refresh=1m&from=1575438321300&to=1575459921300&var-Host=master1&panelId=4&width=1000&height=500&tz=Europe%2FBerlin&render=1"
t=2019-12-04T12:39:22+0000 lvl=eror msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=GET path=/render/d-solo/zxDJxNaZk/graphite-metrics status=500 remote_addr=192.168.106.101 time_ms=310 size=1722 referer="https://grafana.xxx-xxx/d/zxDJxNaZk/graphite-metrics?orgId=1&refresh=1m"

I think we can be fairly certain that this is not an issue of host name vs. certificate subject/SAN, as the error message explicitly says ERR_CERT_AUTHORITY_INVALID, so the renderer does not seem to pick up the custom CA certificates from anywhere I put them to. Which also makes it fairly sure to assume that it does not use the NSS library, at least not in a standard way.

I must admit I'm out of my wits at the moment.

from grafana-image-renderer.

peteeckel avatar peteeckel commented on September 17, 2024

I must admit I'm out of my wits at the moment.

Not quite, as it turned out. It's a permissions problem as it seems. I'll work that out and update this issue as soon as I can come up with precise instructions.

from grafana-image-renderer.

peteeckel avatar peteeckel commented on September 17, 2024

So, i think I got it fixed.

The following actions provided the minimal fix for my setup that still works:

[root@master1 ~]# [ -d ~grafana/.pki/nssdb ] || mkdir -p /usr/share/grafana/.pki/nssdb
[root@master1 ~]# certutil -d sql:/usr/share/grafana/.pki/nssdb -A -n hbr-dev-root-ca -t C -i /etc/pki/tls/certs/hbr-dev-root-ca.crt.pem
[root@master1 ~]# chown -R grafana: /usr/share/grafana/.pki/nssdb

Root certificate alias and file name obviously need to be adapted.

The most important part is that the NSS database needs to be owned by the grafana user, or at least to have write permission for that user (it also works when the files have the Grafana group assigned and are group-writable). Somewhat strange. certutilby default creates the database files with permissions 0600, so having the Grafana user as their owner is probably the most straightforward solution.

from grafana-image-renderer.

peteeckel avatar peteeckel commented on September 17, 2024

Given the fact that it is possible to implant custom certificates in the way I described above I suggest changing this from a "Feature Request" to a "Documentation Improvement Request".

Is there anything I can do?

from grafana-image-renderer.

marefr avatar marefr commented on September 17, 2024

Should probably be added to https://grafana.com/docs/grafana/latest/administration/image_rendering/#troubleshooting. Care to contribute changes needed?

from grafana-image-renderer.

peteeckel avatar peteeckel commented on September 17, 2024

I'll update it as soon as I find the time, this weekend at the earliest.

from grafana-image-renderer.

marefr avatar marefr commented on September 17, 2024

@peteeckel do you happen to know if your instructions would work in the alpine docker image for the rendering service?

from grafana-image-renderer.

peteeckel avatar peteeckel commented on September 17, 2024

Sorry, I can't test it at the moment, but I would expect the situation to be exactly the same - The renderer uses chromium as its engine, chromium uses NSS, and the instructions describe how to add a trusted CA to the NSS cert store.

The only obstacles that come to my mind are that the certutil tool might me missing from the Docker image, or that the chromium instance on the renderer Image might be running under a different user.

from grafana-image-renderer.

albundy83 avatar albundy83 commented on September 17, 2024

Hello,

under Ubuntu 20.04, due to the bug https://bugs.launchpad.net/ubuntu/+source/nss/+bug/1647285 the solution to import with certutil does not work depending of the situation.

In my case, I have a root and an intermediate certificate authorities to import and I never make it works.
Here another solution that works much better (it comes from here https://superuser.com/questions/437330/how-do-you-add-a-certificate-authority-ca-to-ubuntu):

sudo apt-get install -y p11-kit libnss3
sudo mv /usr/lib/x86_64-linux-gnu/nss/libnssckbi.so /usr/lib/x86_64-linux-gnu/nss/libnssckbi.so.bak
sudo ln -s /usr/lib/x86_64-linux-gnu/pkcs11/p11-kit-trust.so /usr/lib/x86_64-linux-gnu/nss/libnssckbi.so
sudo update-ca-certificates

This way finally solved the issue with ERR_CERT_AUTHORITY_INVALID

from grafana-image-renderer.

Nikoos avatar Nikoos commented on September 17, 2024

Hello,
I had the same issue with the net::ERR_CERT_AUTHORITY_INVALID error message.

I tried to create the nssdb inside /usr/share/grafana/.pki/nssdb with my custom CA only and grafana-image-rendere failed to connect my Grafana instance..

However, adding my custom CA certificate in (it's a RHEL box) /usr/share/pki/ca-trust-source/anchors/ and launching update-ca-trust (without creating the nssdb db): I was able to see my custom CA in the trust list command output.

Note 1: Chrome or Chromium may throw you the following error message even if you had the correct CA to be system wide :
error="Error: net::ERR_CERT_COMMON_NAME_INVALID at https:// in my case, after digging, it seems that Chrome needs to have SAN entries for your certificate (one for your FQDN server, and one for your domain).

Once done, I was able to retrieve my panel in PNG.

Best regards,

Nikoos

from grafana-image-renderer.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.