add: jx create cluster eks about jx HOT 18 CLOSED

OK so the DOCKER_REGISTRY issue is all sorted now - so jx create cluster eks is pretty much there now - just need to fix the ingress IP address problem on Amazon - will try fix that ASAP and document it - then close this issue when its ready

from jx.

I’ve just merged a fix for Ingress - though it does require a DNS name so that you can setup a wildcard CNAME pointing to the ELB host. I’ve added automation of the Route 53 setup of the CNAME record if the user has a custom domain registered or transferred to route 53

from jx.

this looks like it could be handy https://eksctl.io/ as the CLI tool to actually spin up the EKS cluster. So the jx create cluster eks command could check if its on the path and if not install it (like we do for gcloud, az, kubectl etc) then use it to create the cluster?

from jx.

here's the AWS CLI for creating clusters: https://docs.aws.amazon.com/cli/latest/reference/eks/index.html

from jx.

Here's the terraform eks docs https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html

from jx.

Just FYI – I think that eksctl may end-up using Terraform under the hood one day, it's up for debate at the moment, there are certain advantages in using CloudFormation or leverage the machinery from kops.

As of AWS CLI, you still need to do a lot of stuff before and after aws eks create-cluster.

from jx.

we'll need insecure registries enabled too like this #1070 unless we configure ECR via Terraform too...

from jx.

I am happy to add insecure registeries to eksctl, and eventually through ECR into the mix.

In a way, I much rather add Jenkins X add-on, and make it enable insecure registeries or whatever other config is required - for the time being, an experimental flag could be added, so that we can all roll forward.

from jx.

@errordeveloper thanks! I just raised this issue eksctl-io/eksctl#92 as it'd be nice to be able to enable ECR on worker nodes via a CLI argument. Using ECR is a better option really on EKS than using insecure registries and a local docker registry

from jx.

just a heads up on progress; as of this PR #1198 we have an implementation of jx create cluster eks that uses eksctl to spin up the cluster then install Jenkins X (ensuring there's a storageclass first etc).

Am trying to get ECR working as well by default which needs a few more changes:

• add a policy to the worker nodes so that they can push and pull ECR (see eksctl-io/eksctl#92)
• default the DOCKER_REGISTRY to point to $accountId.dkr.ecr.$region.amazonaws.com
• for each project run the equivalent of aws ecr create-repository --repository-name org/app-name so that there is an ECR repository created for the project to push to

from jx.

once the new eksctl release is out and we do the next jenkins-x-platform release I'm hoping things will be mostly working.

Though one remaining EKS issue to fix is the load balancer; using the IP of the master node does seem to change every 12-24 hours so we need a better solution for ingress on EKS

from jx.

If you have an old eksctl binary locally and remove it, then run jx create cluster eks ... it should now spin up a cluster properly on EKS with the correct roles to publish to ECR, update your local ~/.kube/config to set the local context to point to the new cluster and install Jenkins X.

So very close now! There's still a pending issue that the DOCKER_REGISTRY environment variable doesn't get updated to default to ECR yet - I thought I'd fixed that one but apparently not yet. Then there's the ingress IP address changing every 12-24 hours. Hope to fix these very soon...

from jx.

from jx.

yeah sorry - I mean that eksctl will setup your ~/.kube/config for you - like creating a GKE cluster does. Works lovely!

from jx.

Am stuck on the ELB issue - I've raised #1256 to see if anyone with better AWS / ELB / Route 53 knowledge than I can help us figure out a way forward to support AWS / EKS nicely

from jx.

@jstrachan just tried this today - the wizard prompted me for the information, but the record failed to create in route 53. Not sure why, didnt' get any errors. I went to r53 and manually created the record, and that seemed to work!

from jx.

Is this complete? Since https://jenkins-x.io/commands/jx_create_cluster_eks/ appears to exist and be a thing now :-)

Might still be related issues but perhaps this can be closed and anything outstanding be submitted separately

from jx.

@Cervator agreed, thanks

from jx.