Git Product home page Git Product logo

Comments (12)

feikesteenbergen avatar feikesteenbergen commented on August 19, 2024 1

No, changing the namespace + name of the secrets should be enough

from helm-charts.

feikesteenbergen avatar feikesteenbergen commented on August 19, 2024 1

Changing the names of the Secrets seems best to me, however you can specify the names as well here:

https://github.com/timescale/timescaledb-kubernetes/blob/master/charts/timescaledb-single/values.yaml#L26-L41

# These secrets should exist before the Helm is used to deploy this TimescaleDB.
# You can use generate_kustomization.sh to help in creating these secrets, or have
# a look at kustomize/example to see how you could install them.
secretNames:
  # This secret should contain environment variables that influence Patroni,
  # for example PATRONI_SUPERUSER_PASSWORD or PATRONI_REPLICATION_PASSWORD
  # https://patroni.readthedocs.io/en/latest/ENVIRONMENT.html#postgresql
  credentials: # defaults to RELEASE-credentials

  # This secret should be a Secret of type kubernetes.io/tls, containing
  # both a tls.key and a tls.crt
  certificate: # defaults to RELEASE-certificate

  # This secret should contain environment variables that influence pgBackRest,
  # for example, PGBACKREST_REPO1_S3_KEY or PGBACKREST_REPO1_S3_KEY_SECRET
  pgbackrest:  # defaults to RELEASE-pgbackrest

from helm-charts.

Firaenix avatar Firaenix commented on August 19, 2024

So I modified the script to take my nameOverride and namespace into account, is there some way I can validate that everything is all good?

from helm-charts.

Firaenix avatar Firaenix commented on August 19, 2024

I gave the upgrade a go, failed so I must have messed something up in the script.

image

from helm-charts.

feikesteenbergen avatar feikesteenbergen commented on August 19, 2024

Hmm, can you share some details about the pod? There shouldn't be any secrets in there,
but you may wish to look at the output before posting.

kubectl describe pod/<podname>

Could you please share that as plain text instead of a screenshot?

from helm-charts.

Firaenix avatar Firaenix commented on August 19, 2024

no probs, I rolled back to 0.5.4, will try the upgrade again and send you the output of describe

from helm-charts.

Firaenix avatar Firaenix commented on August 19, 2024
Name:         timescaledb-uptimesv-timescale-2
Namespace:    databases
Priority:     0
Node:         pool-tgrhprvvk-3fxch/10.130.61.118
Start Time:   Fri, 19 Jun 2020 18:06:09 +1000
Labels:       app=timescaledb-uptimesv-timescale
              cluster-name=timescaledb
              controller-revision-hash=timescaledb-uptimesv-timescale-7b96c9cf8
              release=timescaledb
              role=replica
              statefulset.kubernetes.io/pod-name=timescaledb-uptimesv-timescale-2
Annotations:  status:
                {"conn_url":"postgres://10.244.1.49:5432/postgres","api_url":"http://10.244.1.49:8008/patroni","state":"running","role":"replica","version...
Status:       Running
IP:           10.244.1.49
IPs:
  IP:           10.244.1.49
Controlled By:  StatefulSet/timescaledb-uptimesv-timescale
Init Containers:
  tstune:
    Container ID:  docker://a658b8d5690c83896adfcff6d03ebfb3149ce700559d1397e20a589662eec8a1
    Image:         timescaledev/timescaledb-ha:pg11-ts1.7
    Image ID:      docker-pullable://timescaledev/timescaledb-ha@sha256:3934896b7c8410da7e127667c7742fd7776f4f9cda51b1532c5f8866c5821cda
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      set -e
      [ $CPUS -eq 0 ]   && CPUS="${RESOURCES_CPU_LIMIT}"
      [ $MEMORY -eq 0 ] && MEMORY="${RESOURCES_MEMORY_LIMIT}"
      
      if [ -f "${PGDATA}/postgresql.base.conf" ] && ! grep "${INCLUDE_DIRECTIVE}" postgresql.base.conf -qxF; then
        echo "${INCLUDE_DIRECTIVE}" >> "${PGDATA}/postgresql.base.conf"
      fi
      
      touch "${TSTUNE_FILE}"
      timescaledb-tune -quiet -pg-version 11 -conf-path "${TSTUNE_FILE}" -cpus "${CPUS}" -memory "${MEMORY}MB" \
         -yes
      
      # If there is a dedicated WAL Volume, we want to set max_wal_size to 60% of that volume
      # If there isn't a dedicated WAL Volume, we set it to 20% of the data volume
      if [ "${RESOURCES_WAL_VOLUME}" = "0" ]; then
        WALMAX="${RESOURCES_DATA_VOLUME}"
        WALPERCENT=20
      else
        WALMAX="${RESOURCES_WAL_VOLUME}"
        WALPERCENT=60
      fi
      
      WALMAX=$(numfmt --from=auto ${WALMAX})
      
      # Wal segments are 16MB in size, in this way we get a "nice" number of the nearest
      # 16MB
      WALMAX=$(( $WALMAX / 100 * $WALPERCENT / 16777216 * 16 ))
      WALMIN=$(( $WALMAX / 2 ))
      
      echo "max_wal_size=${WALMAX}MB" >> "${TSTUNE_FILE}"
      echo "min_wal_size=${WALMIN}MB" >> "${TSTUNE_FILE}"
      
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 19 Jun 2020 18:07:11 +1000
      Finished:     Fri, 19 Jun 2020 18:07:11 +1000
    Ready:          True
    Restart Count:  0
    Environment:
      TSTUNE_FILE:             /var/run/postgresql/timescaledb.conf
      RESOURCES_WAL_VOLUME:    50Gi
      RESOURCES_DATA_VOLUME:   50Gi
      INCLUDE_DIRECTIVE:       include_if_exists = '/var/run/postgresql/timescaledb.conf'
      CPUS:                    0 (requests.cpu)
      MEMORY:                  0 (requests.memory)
      RESOURCES_CPU_LIMIT:     node allocatable (limits.cpu)
      RESOURCES_MEMORY_LIMIT:  node allocatable (limits.memory)
    Mounts:
      /var/run/postgresql from socket-directory (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from uptimesv-timescaledb-service-account-token-4xsxw (ro)
Containers:
  timescaledb:
    Container ID:  docker://7bae0ea4665e5ae3ed1e10fe54922591c3a7d36b3905b56d076c140bb013ae5e
    Image:         timescaledev/timescaledb-ha:pg11-ts1.7
    Image ID:      docker-pullable://timescaledev/timescaledb-ha@sha256:3934896b7c8410da7e127667c7742fd7776f4f9cda51b1532c5f8866c5821cda
    Ports:         8008/TCP, 5432/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /bin/bash
      -c
      
      install -o postgres -g postgres -d -m 0700 "/var/lib/postgresql/data" "/var/lib/postgresql/wal/pg_wal" || exit 1
      TABLESPACES=""
      for tablespace in ; do
        install -o postgres -g postgres -d -m 0700 "/var/lib/postgresql/tablespaces/${tablespace}/data"
      done
      
      # Environment variables can be read by regular users of PostgreSQL. Especially in a Kubernetes
      # context it is likely that some secrets are part of those variables.
      # To ensure we expose as little as possible to the underlying PostgreSQL instance, we have a list
      # of allowed environment variable patterns to retain.
      #
      # We need the KUBERNETES_ environment variables for the native Kubernetes support of Patroni to work.
      #
      # NB: Patroni will remove all PATRONI_.* environment variables before starting PostgreSQL
      
      # We store the current environment, as initscripts, callbacks, archive_commands etc. may require
      # to have the environment available to them
      set -o posix
      export -p > "${HOME}/.pod_environment"
      export -p | grep PGBACKREST > "${HOME}/.pgbackrest_environment"
      
      for UNKNOWNVAR in $(env | awk -F '=' '!/^(PATRONI_.*|HOME|PGDATA|PGHOST|LC_.*|LANG|PATH|KUBERNETES_SERVICE_.*)=/ {print $1}')
      do
          unset "${UNKNOWNVAR}"
      done
      
      touch /var/run/postgresql/timescaledb.conf
      
      echo "*:*:*:postgres:${PATRONI_SUPERUSER_PASSWORD}" >> ${HOME}/.pgpass
      chmod 0600 ${HOME}/.pgpass
      
      export PATRONI_POSTGRESQL_PGPASS="${HOME}/.pgpass.patroni"
      
      exec patroni /etc/timescaledb/patroni.yaml
      
    State:          Running
      Started:      Fri, 19 Jun 2020 18:07:14 +1000
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     1500m
      memory:  2500Mi
    Requests:
      cpu:      1
      memory:   2000Mi
    Readiness:  exec [pg_isready -h /var/run/postgresql] delay=5s timeout=5s period=30s #success=1 #failure=6
    Environment Variables from:
      timescaledb-credentials  Secret  Optional: false
      timescaledb-pgbackrest   Secret  Optional: true
    Environment:
      PATRONI_admin_OPTIONS:               createrole,createdb
      PATRONI_REPLICATION_USERNAME:        standby
      PATRONI_KUBERNETES_POD_IP:            (v1:status.podIP)
      PATRONI_POSTGRESQL_CONNECT_ADDRESS:  $(PATRONI_KUBERNETES_POD_IP):5432
      PATRONI_RESTAPI_CONNECT_ADDRESS:     $(PATRONI_KUBERNETES_POD_IP):8008
      PATRONI_NAME:                        timescaledb-uptimesv-timescale-2 (v1:metadata.name)
      PATRONI_POSTGRESQL_DATA_DIR:         /var/lib/postgresql/data
      PATRONI_KUBERNETES_NAMESPACE:        databases
      PATRONI_KUBERNETES_LABELS:           {app: timescaledb-uptimesv-timescale, cluster-name: timescaledb, release: timescaledb}
      PATRONI_SCOPE:                       timescaledb
      PGBACKREST_CONFIG:                   /etc/pgbackrest/pgbackrest.conf
      PGDATA:                              $(PATRONI_POSTGRESQL_DATA_DIR)
      PGHOST:                              /var/run/postgresql
    Mounts:
      /etc/certificate from certificate (ro)
      /etc/pgbackrest from pgbackrest (ro)
      /etc/timescaledb/patroni.yaml from patroni-config (ro,path="patroni.yaml")
      /etc/timescaledb/scripts from timescaledb-scripts (ro)
      /var/lib/postgresql from storage-volume (rw)
      /var/lib/postgresql/wal from wal-volume (rw)
      /var/run/postgresql from socket-directory (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from uptimesv-timescaledb-service-account-token-4xsxw (ro)
  pgbackrest:
    Container ID:  docker://4949f03478bf05a0fcc91d4c8fca05a5e266be288e8ef7219c8ffed16843352e
    Image:         timescaledev/timescaledb-ha:pg11-ts1.7
    Image ID:      docker-pullable://timescaledev/timescaledb-ha@sha256:3934896b7c8410da7e127667c7742fd7776f4f9cda51b1532c5f8866c5821cda
    Port:          8081/TCP
    Host Port:     0/TCP
    Command:
      /etc/timescaledb/scripts/pgbackrest_bootstrap.sh
    State:          Running
      Started:      Fri, 19 Jun 2020 18:07:18 +1000
    Ready:          True
    Restart Count:  0
    Environment Variables from:
      timescaledb-credentials  Secret  Optional: false
      timescaledb-pgbackrest   Secret  Optional: false
    Environment:
      PGHOST:             /var/run/postgresql
      PGBACKREST_STANZA:  poddb
      PGBACKREST_CONFIG:  /etc/pgbackrest/pgbackrest.conf
    Mounts:
      /etc/pgbackrest from pgbackrest (ro)
      /etc/timescaledb/scripts from timescaledb-scripts (ro)
      /var/lib/postgresql from storage-volume (rw)
      /var/lib/postgresql/wal from wal-volume (rw)
      /var/run/postgresql from socket-directory (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from uptimesv-timescaledb-service-account-token-4xsxw (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  wal-volume:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  wal-volume-timescaledb-uptimesv-timescale-2
    ReadOnly:   false
  storage-volume:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  storage-volume-timescaledb-uptimesv-timescale-2
    ReadOnly:   false
  socket-directory:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  patroni-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      timescaledb-uptimesv-timescale-patroni
    Optional:  false
  timescaledb-scripts:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      timescaledb-uptimesv-timescale-scripts
    Optional:  false
  pgbackrest:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      timescaledb-uptimesv-timescale-pgbackrest
    Optional:  false
  certificate:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  timescaledb-certificate
    Optional:    false
  uptimesv-timescaledb-service-account-token-4xsxw:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  uptimesv-timescaledb-service-account-token-4xsxw
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                  Age                From                           Message
  ----     ------                  ----               ----                           -------
  Normal   Scheduled               <unknown>          default-scheduler              Successfully assigned databases/timescaledb-uptimesv-timescale-2 to pool-tgrhprvvk-3fxch
  Normal   SuccessfulAttachVolume  89s                attachdetach-controller        AttachVolume.Attach succeeded for volume "pvc-b3730c83-0c80-424f-bd7d-1decb5ece58e"
  Warning  FailedMount             86s (x5 over 94s)  kubelet, pool-tgrhprvvk-3fxch  MountVolume.WaitForAttach failed for volume "pvc-7cdad8f3-ac00-41bf-ad3e-5e6c5ca0de66" : volume 08cadd5c-7215-11ea-9953-0a58ac14a251 has GET error for volume attachment csi-dc98f96a007c293322f68ee3a0136eba69ab659d436ae6248af77734a0cd87d9: volumeattachments.storage.k8s.io "csi-dc98f96a007c293322f68ee3a0136eba69ab659d436ae6248af77734a0cd87d9" is forbidden: User "system:node:pool-tgrhprvvk-3fxch" cannot get resource "volumeattachments" in API group "storage.k8s.io" at the cluster scope: no relationship found between node "pool-tgrhprvvk-3fxch" and this object
  Warning  FailedMount             78s                kubelet, pool-tgrhprvvk-3fxch  MountVolume.WaitForAttach failed for volume "pvc-7cdad8f3-ac00-41bf-ad3e-5e6c5ca0de66" : watch error:unknown (get volumeattachments.storage.k8s.io) for volume 08cadd5c-7215-11ea-9953-0a58ac14a251
  Normal   SuccessfulAttachVolume  67s                attachdetach-controller        AttachVolume.Attach succeeded for volume "pvc-7cdad8f3-ac00-41bf-ad3e-5e6c5ca0de66"
  Normal   Pulled                  50s                kubelet, pool-tgrhprvvk-3fxch  Container image "timescaledev/timescaledb-ha:pg11-ts1.7" already present on machine
  Normal   Created                 49s                kubelet, pool-tgrhprvvk-3fxch  Created container tstune
  Normal   Started                 49s                kubelet, pool-tgrhprvvk-3fxch  Started container tstune
  Normal   Pulling                 49s                kubelet, pool-tgrhprvvk-3fxch  Pulling image "timescaledev/timescaledb-ha:pg11-ts1.7"
  Normal   Pulled                  46s                kubelet, pool-tgrhprvvk-3fxch  Successfully pulled image "timescaledev/timescaledb-ha:pg11-ts1.7"
  Normal   Created                 46s                kubelet, pool-tgrhprvvk-3fxch  Created container timescaledb
  Normal   Started                 46s                kubelet, pool-tgrhprvvk-3fxch  Started container timescaledb
  Normal   Pulling                 46s                kubelet, pool-tgrhprvvk-3fxch  Pulling image "timescaledev/timescaledb-ha:pg11-ts1.7"
  Normal   Pulled                  43s                kubelet, pool-tgrhprvvk-3fxch  Successfully pulled image "timescaledev/timescaledb-ha:pg11-ts1.7"
  Normal   Created                 43s                kubelet, pool-tgrhprvvk-3fxch  Created container pgbackrest
  Normal   Started                 42s                kubelet, pool-tgrhprvvk-3fxch  Started container pgbackrest

from helm-charts.

Firaenix avatar Firaenix commented on August 19, 2024

I reckon I must have mucked up the script to the point where secrets were copied across incorrectly along with my pgbackrest settings, not sure about my tls stuff but I guess it would be safe to assume they were broken too

from helm-charts.

Firaenix avatar Firaenix commented on August 19, 2024

Any updates or insights?

from helm-charts.

feikesteenbergen avatar feikesteenbergen commented on August 19, 2024

What is the current state? Some (synchronous) troubleshooting may be useful, you can use https://slack.timescale.com/, where I'm normally available during European weekdays.

from helm-charts.

Firaenix avatar Firaenix commented on August 19, 2024

So its still not working, I'm just sitting on 0.5.4 at the moment, I need to identify which secrets I can delete so that it wont mess up my current configuration.

I will jump on the slack at some point, but I'm in the Australian Sydney timezone, so seems like it'll be asynchronous debugging for me. I could probably set aside some time after one or two of my workdays to debug with you.

from helm-charts.

Firaenix avatar Firaenix commented on August 19, 2024

We ended up needing to swap infrastructure providers anyway, so we provisioned the 0.6 version of the helm chart on our new cluster - everything running smoothly.

Although it doesn't fix the issue, I am no longer affected, I appreciate the help in any case.

from helm-charts.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.