nghttpx does not work with CNI about nghttpx-ingress-lb HOT 12 CLOSED

Could you tell us how to reproduce it? Kubernetes configurations, etc..

from nghttpx-ingress-lb.

I used the following coreos cloud configs to setup a kubernetes platform using the RKT container engine

[Unit]
Description=Kubelet via Hyperkube ACI
Wants=flanneld.service
Requires=k8s-assets.target
After=k8s-assets.target
[Service]
Environment=KUBELET_VERSION=v1.4.7_coreos.0
Environment="RKT_OPTS=--uuid-file-save=/var/run/kubelet-pod.uuid \
  --volume dns,kind=host,source=/etc/resolv.conf \
  --mount volume=dns,target=/etc/resolv.conf \
  {{ if eq .container_runtime "rkt" -}}
  --volume rkt,kind=host,source=/opt/bin/host-rkt \
  --mount volume=rkt,target=/usr/bin/rkt \
  --volume var-lib-rkt,kind=host,source=/var/lib/rkt \
  --mount volume=var-lib-rkt,target=/var/lib/rkt \
  --volume stage,kind=host,source=/tmp \
  --mount volume=stage,target=/tmp \
  {{ end -}}
  --volume var-log,kind=host,source=/var/log \
  --mount volume=var-log,target=/var/log"
ExecStartPre=/usr/bin/mkdir -p /etc/kubernetes/manifests
ExecStartPre=/usr/bin/mkdir -p /var/log/containers
ExecStartPre=/usr/bin/systemctl is-active flanneld.service
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/run/kubelet-pod.uuid
ExecStart=/usr/lib/coreos/kubelet-wrapper \
  --api-servers=http://127.0.0.1:8080 \
  --register-schedulable=true \
  --cni-conf-dir=/etc/kubernetes/cni/net.d \
  --network-plugin=cni \
  --container-runtime={{.container_runtime}} \
  --rkt-path=/usr/bin/rkt \
  --rkt-stage1-image=coreos.com/rkt/stage1-coreos \
  --allow-privileged=true \
  --pod-manifest-path=/etc/kubernetes/manifests \
  --hostname-override={{.domain_name}} \
  --cluster_dns={{.k8s_dns_service_ip}} \
  --cluster_domain=cluster.local
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/run/kubelet-pod.uuid
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target

from nghttpx-ingress-lb.

Perhaps this is an issue with kubernetes running under the CNI plugin.
When using the ingress controller with docker I get the following error:

E0115 22:39:42.744432       1 command.go:51] nghttpx error: exit status 255
E0115 22:39:43.741149       1 command.go:81] failed to execute killall [-HUP nghttpx]: nghttpx: no process found

Under the same configuration the contrib nginx controller runs fine.

from nghttpx-ingress-lb.

You still have this issue with docker, so this issue happens when CNI plugin is used, and not with rkt engine, right?

from nghttpx-ingress-lb.

yes I still have issue with docker when using CNI plug. So i will guess this is nothing to do with rkt

from nghttpx-ingress-lb.

Thank you. We have successful running record without CNI plugin. Will test it with CNI plugin, and see what's happening.

from nghttpx-ingress-lb.

Thank you. CNI does not look to stable for ingress controllers at the moment. It maybe that the container is now running with a different networking environment.

kubernetes/kubernetes#31307
kubernetes-retired/bootkube#247

from nghttpx-ingress-lb.

I heard that hostPort does not work with CNI currently.
I'm a bit behind the current progress of calico CNI, and recent updates destroys my older calico configuration. It requires some time to debug.
I'm interested in the way how nginx ingress controller works around this issue.

from nghttpx-ingress-lb.

This could be a workaround: kubernetes-retired/contrib#2135 (comment)

from nghttpx-ingress-lb.

I ran calico using Vagrant file from https://github.com/projectcalico/calico/tree/master/v2.0/getting-started/kubernetes/installation/vagrant
After adding "hostNetwork: true" to nghttpx ingress controller manifest file, and also changing livenessProbe port to 11249, and adding --healthz-port=11249 to nghttpx-ingress-controller arguments, it seems to me that nghttpx ingress controller is working fine.

from nghttpx-ingress-lb.

I set all of these values yesterday and was not so lucky. I will try again

from nghttpx-ingress-lb.

I have tested this in RKT without CNI and it works with the above changes of hostNetwork:true and the change of the healthz port. I will not be using CNI until some of the issues are resolved so I am happy with the work around. Thank you

from nghttpx-ingress-lb.