Link open graph metadata preview doesn't work when paste website link about bunkerweb HOT 5 CLOSED

I found it is because default nginx ssl_protocols value is TLSv1.3.

Almost search engine crawlers try to scrap metadata by using TLSv1.2.

PR #18 .

from bunkerweb.

Hello @peterkimzz, version 1.1.1 should now support both TLS 1.2 and 1.3. Please note that crawlers may not work when using the antibot feature through the USE_ANTIBOT env var.

from bunkerweb.

Hello @peterkimzz can you try the new (and current) 1.1.0 version ? The logging bug should now be fixed in this version. Please note that nginx now listens on ports 8080 for HTTP and 8443 for HTTPS. Then try again to see if we have any logs ?

from bunkerweb.

Hi @bunkerity.

I used 1.1.0 version, still got same result.
Looks all of crawler are blocked.

I think the reason why no any log is that some security tool or rule block ONLY for the crawlers before requesting to nginx.

nginx log (still using curl and browser work)

myreverse_1  | [*] Running nginx ...
myreverse_1  | ==> /var/log/access.log <==
myreverse_1  | 
myreverse_1  | ==> /var/log/error.log <==
myreverse_1  | 
myreverse_1  | ==> /var/log/access.log <==
myreverse_1  | 211.176.129.xx - - [29/Oct/2020:02:02:14 +0000] "HEAD / HTTP/1.1" 301 0 "-" "curl/7.64.1"
myreverse_1  | 211.176.129.xx - - [29/Oct/2020:02:06:40 +0000] "GET / HTTP/2.0" 200 11357 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_16_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Whale/2.8.107.17 Safari/537.36"

docker-compose.yml (adapted by v1.1.0)

version: "3.8"

services:
  myreverse:
    image: bunkerity/bunkerized-nginx:1.1.0
    restart: always
    ports:
      - 80:8080
      - 443:8443
    volumes:
      - ./letsencrypt:/etc/letsencrypt
      - ./nginx/http-configs:/http-confs
      - ./nginx/server-configs:/server-confs
    environment:
      - SERVER_NAME=www.example.com
      - SERVE_FILES=no
      - REDIRECT_HTTP_TO_HTTPS=yes
      - AUTO_LETS_ENCRYPT=yes
      - PROXY_REAL_IP=yes # doesn't affect the result.
      - USE_DNSBL=no
      - USE_FAIL2BAN=no
      - USE_MODSECURITY=no
      - USE_MODSECURITY_CRS=no
      - USE_CLAMAV_UPLOAD=no
      - USE_CLAMAV_SCAN=no
      - CLAMAV_SCAN_REMOVE=no
      - BLOCK_USER_AGENT=no
      - BLOCK_TOR_EXIT_NODE=no
      - BLOCK_PROXIES=no
      - BLOCK_ABUSERS=no
      - USE_WHITELIST_IP=no
      - USE_WHITELIST_REVERSE=no
      - USE_BLACKLIST_IP=no
      - USE_BLACKLIST_REVERSE=no
      - USE_LIMIT_REQ=no

  app1:
    image: "${DOCKER_REGISTRY}:www"
    restart: always
    command: yarn start

 # ...

from bunkerweb.

I found when nginx catches a log.

When I paste a link with http:// (not https://) in Slack chat, nginx returns a log with 301 status.
But Slack still doesn't show the preview.

myreverse_1  | 3.95.174.215 - - [29/Oct/2020:02:37:35 +0000] "GET / HTTP/1.1" 301 162 "-" "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)"
myreverse_1  | 3.95.174.215 - - [29/Oct/2020:02:37:36 +0000] "GET / HTTP/1.1" 301 162 "-" "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)"

from bunkerweb.