Missing open ports in COS charms when running juju status about cos-lite-bundle HOT 13 OPEN

@rbarry82 can probably explain the ins and outs of this.

BTW, K8s itself does list the ports, for example alertmanager:

        _ports = SimpleNamespace(api=9093, ha=9094)
        # ...
        self.service_patcher = KubernetesServicePatch(
            self,
            [
                (f"{self.app.name}", self._ports.api, self._ports.api),
                (f"{self.app.name}-ha", self._ports.ha, self._ports.ha),
            ],
        )

$ k get services -A
NAMESPACE         NAME                     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
wlcm2             am                       ClusterIP   10.152.183.114   <none>        9093/TCP,9094/TCP        5m48s

from cos-lite-bundle.

@gustavosr98

I know this does not solve your issue, but I still have some comments that I think are worthwhile to post.

I did not use Traefik as a ingress controller in this Microk8s deployment

Traefik can be used in tandem with any other ingress controller on the deployed MicroK8s. It is, right now, the only ingress controller that fully supports the functionality needed by COS Lite.

other k8s charm services deployed on the same Microk8s

I would not recommend this topology for anything production or production-like. I totally get why this is tempting to keep the amount of moving parts as low as possible, but for COS Lite - or any observability solution really - to interfere as little as possible with your workloads, and more importantly: vice versa, we recommend you to always deploy it on an isolated MicroK8s cluster.

from cos-lite-bundle.

@sed-i Thank you that is good information to manually create the ingress resources.

Still, I am guessing Juju is be ignoring this information when running juju expose. Otherwise, It should create the ingress resources like it does with mattermost-k8s, discourse-k8s and others that show the ports on Juju.

from cos-lite-bundle.

Traefik can be used in tandem with any other ingress controller on the deployed MicroK8s. It is, right now, the only ingress controller that fully supports the functionality needed by COS Lite.

@simskij I started deploying Traefik just like the full bundle renders. However, I got Traefik charm stuck on waiting status with a message like "not obtaining a gateway ip". I assumed it was a conflict with the existing ingress controller from Microk8s ingress addon and did not dig much into it. If Traefik can co-exist with this plugin then that might be another issue I have to take a look back.

from cos-lite-bundle.

Traefik can be used in tandem with any other ingress controller on the deployed MicroK8s. It is, right now, the only ingress controller that fully supports the functionality needed by COS Lite.

@simskij I started deploying Traefik just like the full bundle renders. However, I got Traefik charm stuck on waiting status with a message like "not obtaining a gateway ip". I assumed it was a conflict with the existing ingress controller from Microk8s ingress addon and did not dig much into it. If Traefik can co-exist with this plugin then that might be another issue I have to take a look back.

Yeah that error message might not be the best tbh. What it means is thay you lack a source of external IPs, like MetalLB

from cos-lite-bundle.

I would not recommend this topology for anything production or production-like. I totally get why this is tempting to keep the amount of moving parts as low as possible, but for COS Lite - or any observability solution really - to interfere as little as possible with your workloads, and more importantly: vice versa, we recommend you to always deploy it on an isolated MicroK8s cluster.

@simskij Thank you this is valuable information to take into account for other deployments. I did not considered it for two reasons. One is that I have a really small deployment intended for demo purposes. And the other reason is I tried using COS to monitor some VM charmed workloads and hit into this caveat, which seems I would need to either use just one juju controller for the whole deployment or do some extra configuration when bootstraping the N controllers so that they can discover the pods network direction.

from cos-lite-bundle.

Traefik can definitely exist with the other ingress controller, assuming that your load balancer has more than one IP in the pool (since nginx will otherwise bind to 80/443, and Traefik won't be able to).

I've never really checked into whether juju expose works with OF/sidecar charms other than vague memories of hearing that it actually doesn't work as expected, but in any case, the default nginx ingress does "normal" k8s service-based-routing, which isn't appropriate for a lot of what's in sidecars right now for various reasons.

One controller for the whole deployment works, as long as that controller is publicly routable. The other configuration actually isn't about the pod's network direction, and more due to Juju not "knowing" what external address it should use for the controller, ala juju bootstrap microk8s coscontroller --config controller-service-type=external --config controller-external-name=mydnsname --config controller-external-ips=[x.x.x.x, y.y.y.y], so it defaults to handing you a k8s-internal, non-routable DNS name (and IP, probably) which can't be resolved from outside the cluster.

What is the actual "error" from the controller when you try to consume?

from cos-lite-bundle.

This is currently blocked pending a native way of opening ports using Juju/OF.

from cos-lite-bundle.

See canonical/operator#905

from cos-lite-bundle.

This could nominally be tested on a current version of Juju by using juju run ... to invoke open-port explicitly, to see if it actually percolates through the controller to be rendered when juju status interrogates it. At a glance, it looks like it might, but it may need a follow-up patch from Juju to resolve, even once the work is done in OF.

from cos-lite-bundle.

@benhoyt - do you have any suggestion as to how we could accomplish this for sidecar charms?

from cos-lite-bundle.

@simskij Yes, now that canonical/operator#905 is merged (and included in ops 2.1.0+) you should be able to use Unit.open_port() in the cos-lite charms. This will tell Juju to open those ports on the K8s end. I've just grepped the codebases for each of the cos-lite charms, and none of them are using open_port yet.

from cos-lite-bundle.

There are some related juju issues still open:

• https://bugs.launchpad.net/juju/+bug/2003782
• https://bugs.launchpad.net/juju/+bug/2003783

We should use open_port, but it may still not be visible in juju status.

from cos-lite-bundle.