Inclusion of prometheus leading to socket.error about thumbor HOT 10 CLOSED

Hey @Brian-Myers-Bose 👋

I've never used prometheus, so not sure I can help much. Would you like to post a sample docker-compose.yml file that reproduces the issue you're experiencing? this can certainly help troubleshooting things.

from thumbor.

docker-compose.yml

version: '3.4'

services:

  thumbor:
    build: ./images/thumbor
    ports:
      - 80:80
      - 8000:8000
    environment:
      LOG_LEVEL: DEBUG
      HEALTHCHECK_ROUTE: "/"
      THUMBOR_PORT: "80"
      ALLOW_UNSAFE_URL: "True"
      AUTO_WEBP: "True"
      # set the number of processes to 1-2x of CPU cores for each Node
      THUMBOR_NUM_PROCESSES: "2"
      # MAX_AGE sets the HTTP caching headers
      # depending on your setup, you might want to increase it
      MAX_AGE: "3600"
      MAX_AGE_TEMP_IMAGE: "300"
      RESULT_STORAGE_EXPIRATION_SECONDS: "31536000"
      # Prometheus Metrics
      METRICS: "tc_prometheus.metrics.prometheus_metrics"
      PROMETHEUS_SCRAPE_PORT: "8000"

from thumbor.

Btw. I changed my Dockerfile to

#FROM minimalcompact/thumbor:6.7.0
FROM apsl/thumbor:latest

RUN pip install --upgrade pip \
# \
#    && pip install tc_redis
    && pip install tc_prometheus
# ENV LOG_LEVEL DEBUG

and it worked first time.

from thumbor.

Looks like the Dockerfile is based on apsl/thumbor, so I'm not sure I can help with that one.

btw, did you check that nothing else is binding to port 8000 on your machine? either a local service that listens on it, or another docker container with an exposed port?

from thumbor.

I tried to test it as well, and I'm seeing the same issue. I tried different ports, but it seems to always happen.

Perhaps worth asking on https://github.com/thumbor-community/prometheus ? maybe it's being started more than once? (just a guess, I don't really know).

@kkopachev is this something you could maybe help with?

from thumbor.

@gingerlime Thanks for taking a look. This only appears to be an issue with minimalCompact where the plug-in is included by default. When one moves to another build where one must install plugins, it works. I have run netstat -an | grep LISTEN and don't appear to have anything else listening on 8000.

from thumbor.

very strange... your guess is as good as (or perhaps better than) mine. It was added last year and seemed to have worked? #38 (comment) but not sure what changed that could break it ... maybe try a few older versions to try to pinpoint what changed caused it to start behaving this way?

from thumbor.

Hey @Brian-Myers-Bose I think I found/understand why it happens, although not sure what's the solution.

If you change the number of thumbor processes to 1, then there are no errors. So it looks like each thumbor process tries to start the prometheus http server, and obviously with a static port number, there's a clash. Perhaps there's a way to specify a dynamic port or something? another way around it is to use replicas with docker-compose to run multiple instances of thumbor, each one with only 1 internal process...

This will hopefully get resolved with thumbor 7 (?) which has a built-in multi-process option, but perhaps it's something worth testing with #76

from thumbor.

@gingerlime Thanks. That's pretty interesting. I'm looking forward to moving to 7. 4 months to 2.7 is deprecated

from thumbor.

It's not really resolved, but I'll close this for now, given that there isn't much we can do at the moment.

from thumbor.