pblaas / nagoya Goto Github PK

When cloning the project and performing a terraform init you will fetch the latest openstack provider config. This currently seems to lead into issues.

After running the nagoya onliner change version in the provider config stanza in the k8s.tf before running the terraform init && terraform apply command.

provider "openstack" {
  user_name   = "username"
  tenant_name = "Your Tenant Name here"
  auth_url    = "https://identity.openstack.cloudvps.com:443/v3"
  version     = "<= 1.12.0"
}

Nagoya now supports anti-affinity by default in the deployment script.
Ideally this should be soft-anti-affinity however this does not seem to be supported just yet.

I'm tracking the issue here:
https://github.com/terraform-providers/terraform-provider-openstack/issues/118

Some kubelet flags currently used are deprecated and need to be removed.
Some flags need to be moved to the kubelet config file instead of invoking the flag directory on the kubelet.

Need to be removed:
--register-schedulable has been deprecated, will be removed in a future version
--allow-privileged has been deprecated, will be removed in a future version

Need to be moved to a new config file:
--pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag.
--cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag.
--cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag.
--client-ca-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag.
--anonymous-auth has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag.
--http-check-frequency has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag.
--node-status-update-frequency has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag.
--volume-stats-agg-period has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag.

The current setup of Nagoya doesn't provide any means for SSL certificate rotation.
Kubernetes does support this in current versions but Nagoya doesn't use this.

The current setup of Nagoya doesn't provide any means for SSL certificate rotation.
Kubernetes does support this in current versions but Nagoya doesn't use this.

The LBAAS K8S APISERVER loadbalancer ip gains the default security group with limited ingress and egress permissions.

Generation of the Terraform config file should include a proper security group config for the API loadbalancer on port 443.

I added a node with the following command:

./addnode.py --workerimageflavor 2012 192.168.3.x
terraform apply

The node added itself to the cluster, but the kube-proxy pod kept crashing.
I noticed it missed a value at the Commands section:

/hyperkube
proxy
--master=https://192.168.3.3
--cluster-cidr=10.244.0.0/16
--conntrack-max-per-core=0
--kubeconfig=/etc/kubernetes/master-kubeconfig.yaml
--v=

Which should be:

/hyperkube
proxy
--master=https://192.168.3.3
--cluster-cidr=10.244.0.0/16
--conntrack-max-per-core=0
--kubeconfig=/etc/kubernetes/master-kubeconfig.yaml
--v=1

It was missing the 1 at the --v variable.
I added the 1 at node_{ip}.yaml and .json file and did

terraform apply

again and then it worked.

Seems CoreOS stopped updating their hyperkube package on Quay.
https://quay.io/repository/coreos/hyperkube?tab=tags

coreos/bugs#2470

I will rewrite the config files to use grc.io.
https://console.cloud.google.com/gcr/images/google-containers/GLOBAL/hyperkube-amd64

WARNING: all flags other than --config, --write-config-to, and --cleanup are deprecated. Please begin using a config file ASAP.

flags should be placed in a config file due to warning.