razee-io / razee Goto Github PK

Describe the bug
Watch-keeper never gets started in a new install because the YAML downloaded from the agent install endpoint uses the wrong apiGroup for the RemoteResource type. Provided yaml:

apiVersion: "kapitan.razee.io/v1alpha1"
kind: RemoteResource
metadata:
  name: watch-keeper-rr
  namespace: razee
spec:
  requests:
    - options:
        url: https://github.com/razee-io/watch-keeper/releases/latest/download/resource.yaml

Correct value would be

apiVersion: "deploy.razee.io/v1alpha1"
kind: RemoteResource
metadata:
  name: watch-keeper-rr
  namespace: razee
spec:
  requests:
    - options:
        url: https://github.com/razee-io/watch-keeper/releases/latest/download/resource.yaml

To Reproduce
Steps to reproduce the behavior:

1. Go to app.razee.io and register a github org
2. Click on Manage for that new org after authorization is done
3. Copy the agent install command line and apply on target cluster.
4. Note that watch-keeper pods aren't started and there's an error complaining about the RemoteResource type in the kapitan.razee.io/v1alpha1 type.

Expected behavior
Watch-keeper pods should show up and start running in my cluster.

Screenshots
n/a

Kubernetes Version (kubectl version):

• Client: 1.14.0
• Server: 1.14.0 IKS

Additional context
n/a

Create a new identity crd that will be able to assemble info about the cluster as well as custom info specified by the user to create a data section in the identity resource to be used by FeatureFlagSetLD and watch-keeper.

This will provide a consistent identity piece across all components.

What happpened

When trying to remove Razee from a cluster, I used kubectl delete -f https://github.com/razee-io/Kapitan-delta/releases/latest/download/resource.yaml to begin removing resources. After the command completes, the cluster-admin role binding has been removed from my cluster. The result is that my standard user IAM#[email protected] cannot access everything in the cluster per my IAM privs as account owner.

How I fixed the problem

Since I'm using IKS, I can re-export my kubeconfig files using ibmcloud ks cluster-config <mycluster> --export --admin and restore access to my cluster by applying the following YAML to the cluster.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: cluster-admin
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - '*'
- nonResourceURLs:
  - '*'
  verbs:
  - '*'

What I expected to happen

When using kubectl delete, I didn't expect to lose the built in cluster-admin ClusterRole that IKS provides me.

Describe the bug
http://<razeedash-api-lb_external_IP>:8081/api/install/cluster?orgKey=orgApiKey-<org_api_key> includes the razee namespace creation twice. So there would always be an error. Error from server (AlreadyExists): namespaces "razee" already exists

To Reproduce
Steps to reproduce the behavior:

1. GET http://<razeedash-api-lb_external_IP>:8081/api/install/cluster?orgKey=orgApiKey-<org_api_key>
2. In result, line 1-5 and line 176 - 182. Deplicate razee namespace creation
3. Run kubectl create -f http://<razeedash-api-lb_external_IP>:8081/api/install/cluster?orgKey=orgApiKey-<org_api_key> and get an error like Error from server (AlreadyExists): namespaces "razee" already exists

Expected behavior
No error message

Screenshots
If applicable, add screenshots to help explain your problem.

Kubernetes Version (kubectl version):

• Client: 1.14.9_1543
• Server: 1.14.9_1543

Additional context
Add any other context about the problem here.

Is your feature request related to a problem? Please describe.
For our use cases, when the Razee agent is installed on a cluster, we don't need the featureflagsetlid-controller, managedset-controller, or mustachetemplate-controller deployments. This is creating extra unneeded workload on clusters and making the setup time longer for each cluster where the agent is installed.

Describe the solution you'd like
A separate /api/install/cluster API or a way to configure the existing /api/install/cluster API such that it only deploys the required parts.

Describe alternatives you've considered
It looks like we could edit the razeedeploy-delta-resource-uris ConfigMap that comes back from the existing /api/install/cluster endpoint to remove the parts we don't need.

The section describing how to label resource only talks about individual resources, not how to label entire namespaces https://github.com/razee-io/Razee#step-2-visualize-deployment-information-in-razeedash. In addtion, the kubectl label command should be used instead of editing the YML file directly ( label namespace default 'razee/watch-resource=true' )

Granted the example docs do show a trailing / character after the root_url and razeedash_api_url it's not exactly clear that those really are required. Failure to include these final /'s will result in values like this in the Install Razee Agent field in the org management panel (note the missing slash between my domain and the start of the path):

kubectl create -f "https://razeedash-api.kubedclouds.usapi/install/cluster?orgKey=xxxxxxxxx"

If we were so inclined we could easily look for a trailing slash in the ConfigMap value and add one if it was omitted. (Or we could firm up the docs indicating that the trailing slash should be there).

Is your feature request related to a problem? Please describe.
We need to populate a list of namespaces for the cluster in our UI and need a mechanism to retrieve the list via Razee

Describe alternatives you've considered
Adding the watchkeeper label to all the namespaces

Right now each subscription can only map to a single channel. When building subscriptions, it would be ideal to support multiple channels to a single subscription. This way I can create a subscription for my development systems, that all have the same tags, to automatically install all components of my application which is currently delivered via 3 different channels.

From the query I can't tell if the cluster ip could be used or if you must provide the external load balancer address. For example

kubectl get svc
NAME               TYPE           CLUSTER-IP       EXTERNAL-IP                                                              PORT(S)          AGE
mongo              ClusterIP      None             <none>                                                                   27017/TCP        10m
razeedash-api-lb   LoadBalancer   172.31.189.127   a05346deebac511e9bee912da23a19d3-533233690.us-east-1.elb.amazonaws.com   8081:31724/TCP   9m56s
razeedash-lb       LoadBalancer   172.31.0.77      a0528bbfabac511e9bee912da23a19d3-125787880.us-east-1.elb.amazonaws.com   8080:32219/TCP   9m57s

Hi, I am using minikube on ubuntu 18.04 to try Razee now. When I was installing to step 5, the external-ip of both razeedash-lb and razeedash-api-lb are empty and appears to be when I use command like

kubectl get service razeedash-api-lb -n razee

I am pretty confused about what is going on here. For example, should I wait for a long time for the external ip address to appear? Thank you.

Describe the bug

1. According to your home page instructions found here https://razee.io/#get-razee

 # RazeeDash install 
kubectl apply -f https://github.com/razee-io/Kapitan-delta/releases/latest/download/resource.yaml 
kubectl apply -f https://github.com/razee-io/Razee/releases/latest/download/resource.yaml 

there is not any dash at all

[root@agentavtone-masterone kui]# kubectl get svc -n razee
NAME               TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
mongo              ClusterIP      None             <none>        27017/TCP        5m18s
razeedash-api-lb   LoadBalancer   10.111.139.146   <pending>     8081:30634/TCP   5m14s
razeedash-lb       LoadBalancer   10.99.115.67     <pending>     8080:30606/TCP   5m15s
[root@agentavtone-masterone kui]# curl http://9.46.74.217:30606/
curl: (7) Failed connect to 9.46.74.217:30606; Connection refused
[root@agentavtone-masterone kui]#
[root@agentavtone-masterone ~]# kubectl get -n razee po
NAME                                           READY   STATUS    RESTARTS   AGE                                  featureflagsetld-controller-68bf78798d-qvrzb   1/1     Running   0          9m8s
managedset-controller-776c4dcb9c-5sj47         1/1     Running   0          9m4s
mongo-5757d49748-c4kwc                         1/1     Running   0          9m29s                                mustachetemplate-controller-6c4f886d98-ksgnw   1/1     Running   0          9m7s
razeedeploy-delta-659c47ddb5-cgm5f             1/1     Running   1          9m46s
remoteresource-controller-6c45f98765-xblh7     1/1     Running   0          9m3s
remoteresources3-controller-655b759dc8-phjdw   1/1     Running   0          9m2s
[root@agentavtone-masterone ~]#
[root@agentavtone-masterone ~]# kubectl get -n razee po --show-labels | grep razeedash
[root@agentavtone-masterone ~]#

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Setup GHE
2. Input API Url as https://api.github.ibm.com and Github URL as https://github.ibm.com
3. Save configuration
4. Click Sign in button, it jumps to URL https://https//github.ibm.com/login/oauth/
   Expected behavior
   A clear and concise description of what you expected to happen.

And there is no approach to re-configure and correct configurations.

I'm running the following step:
kubectl apply -f https://github.com/razee-io/Razee/releases/latest/download/resource.yaml
and I get the following output:

Eduardos-MacBook-Air:razee edu$ k get po
NAME                                           READY   STATUS                       RESTARTS   AGE
featureflagsetld-controller-77c9b468c5-w4jch   1/1     Running                      0          10m
kapitan-delta-66bc5c48c4-f2jqf                 1/1     Running                      0          10m
managedset-controller-6f7d5865bf-sq59s         1/1     Running                      0          10m
mongo-56cc5595f7-f8nw7                         1/1     Running                      0          7m
mustachetemplate-controller-679b56f695-s2qsm   1/1     Running                      0          10m
razeedash-58bbf55d74-lp55x                     0/1     CreateContainerConfigError   0          6m
razeedash-api-86bcd7868-q7g5m                  1/1     Running                      0          6m
remoteresource-controller-868f4c4cf9-krmpx     1/1     Running                      0          10m
remoteresources3-controller-666cb88ddf-4gsjx   1/1     Running                      0          10m

Looking at the razeedash Pod, I see the following error:

Eduardos-MacBook-Air:razee edu$ k describe po razeedash-58bbf55d74-lp55x
...
Events:
  Type     Reason     Age                     From                     Message
  ----     ------     ----                    ----                     -------
  Normal   Scheduled  7m25s                   default-scheduler        Successfully assigned razee/razeedash-58bbf55d74-lp55x to 10.190.111.230
  Normal   Pulled     5m41s (x8 over 7m7s)    kubelet, 10.190.111.230  Successfully pulled image "quay.io/razee/razeedash:0.1.11"
  Warning  Failed     5m41s (x8 over 7m7s)    kubelet, 10.190.111.230  Error: configmaps "razeedash-config" not found
  Normal   Pulling    2m21s (x22 over 7m24s)  kubelet, 10.190.111.230  pulling image "quay.io/razee/razeedash:0.1.11"

How is the razeedash-config configmap supposed to be created?

Thanks!

This is identical issue to #47 , which was closed.

namespace/razee created
deployment.apps/remoteresource-controller created
customresourcedefinition.apiextensions.k8s.io/remoteresources.deploy.razee.io created
serviceaccount/razeedeploy-sa created
clusterrole.rbac.authorization.k8s.io/razeedeploy-admin-cr created
clusterrolebinding.rbac.authorization.k8s.io/razeedeploy-rb created
configmap/razeedeploy-delta-resource-uris created
deployment.apps/razeedeploy-delta created
configmap/watch-keeper-config created
secret/watch-keeper-secret created
Error from server (AlreadyExists): error when creating "blah": namespaces "razee" already exists
unable to recognize "blah": no matches for kind "RemoteResource" in version "kapitan.razee.io/v1alpha1"

The problem is kapitan.razee.io should be deploy.razee.io

Is your feature request related to a problem? Please describe.
Need the ability to remove a cluster from Razee programmatically in cases where customers no longer want their clusters to be registered with Razee.

Describe the solution you'd like
A RESTful API for deleting clusters in Razee.

Overview
Enhance the security of the all the razee.io workloads by configuring them to run as a non-root user.

Describe the solution you'd like
Most of the razee.io images are based on node:alpine image, which includes a node user.
These images should be build to use that user, and the Kubernetes workload configured to run as same.

Describe alternatives you've considered
Alternatively a new user id could be created, or the workloads could continue to run as root.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

There is a problem with deploying locally to minikube in that one of the docs says you need a cluster with at least two nodes. Also, minikube is not fully compatible with deployments to IKS. In deploying to our IKS cluster I hit an issue with having an ALB that requires an ingress.

Describe the solution you'd like
A clear and concise description of what you want to happen.

I think the instructions for installing razee locally should be using kind (kubernetes in docker: https://kind.sigs.k8s.io/docs/user/quick-start/). Kind is a certified knative implementation of kubernetes. It allows for multiple clusters and multiple nodes per cluster.

Using kind with the metallb product (see below) allows a seamless install of razeedash locally.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
minikube - not ok as it is limited

Additional context
Add any other context or screenshots about the feature request here.
One additional piece of information. Kind was not working as a solution because of the requirement of having load balancer services. Since load balancers require an external IP and that IP is allocated by the cloud provider there is a problem, kind is not running in a cloud. I found a product that gets around this by using the docker network as the external IP, the product is metallb (https://metallb.universe.tf/concepts/), this appears to be approved for use by the OSSC. Here is a guide on implementing: https://mauilion.dev/posts/kind-metallb/

• So I created a muti-node cluster using kind
• Then I installed metallb: kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.7.3/manifests/metallb.yaml
• Then I install razee based on the razee.io instructions

Hitting this error when running on OpenShift 4.1 on AS.

38s     Warning   CreatingLoadBalancerFailed   service/razeedash-lb                                Error creating load balancer (will retry): failed to ensure load balancer for service razee/razeedash-lb: unsupported load balancer affinity: ClientIP

The fix is to remove the affinity manually in the object svc/razeedash-lb

Hello,

I have an issue when I want push razee in my aws eks cluster
caused "exec: "sh/liveness.sh": permission denied": unknown
0s Normal Killing pod/mustachetemplate-controller-66478dd9dd-82fz4 Killing container with id docker://mustachetemplate-controller:Container failed liveness probe.. Container will be killed and recreated.
0s Normal Pulling pod/mustachetemplate-controller-66478dd9dd-82fz4 pulling image "quay.io/razee/mustachetemplate:0.0.3"
0s Normal Pulled pod/mustachetemplate-controller-66478dd9dd-82fz4 Successfully pulled image "quay.io/razee/mustachetemplate:0.0.3"
0s Normal Created pod/mustachetemplate-controller-66478dd9dd-82fz4 Created container
0s Normal Started pod/mustachetemplate-controller-66478dd9dd-82fz4 Started container
0s Warning FailedCreatePodSandBox pod/mongo-6cb8568cb7-7wjdg (combined from similar events): Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1f81dc0230d527c43c0f85cc907fff9511c1a70498168260ccfca062dc420230" network for pod "mongo-6cb8568cb7-7wjdg": NetworkPlugin cni failed to set up pod "mongo-6cb8568cb7-7wjdg_razee" network: add cmd: failed to assign an IP address to container
0s Warning BackOff pod/razeedash-api-57bdf94d59-zfss7 Back-off restarting failed container
0s Normal Pulling pod/razeedash-5d5c4dcf49-krzw5 pulling image "quay.io/razee/razeedash:0.1.0"
0s Normal EnsuringLoadBalancer service/razeedash-lb Ensuring load balancer
0s Warning CreatingLoadBalancerFailed service/razeedash-lb Error creating load balancer (will retry): failed to ensure load balancer for service razee/razeedash-lb: unsupported load balancer affinity: ClientIP
0s Warning Unhealthy pod/mustachetemplate-controller-66478dd9dd-82fz4 Liveness probe failed: OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: "sh/liveness.sh": permission denied": unknown
0s Normal SandboxChanged pod/mongo-6cb8568cb7-7wjdg Pod sandbox changed, it will be killed and re-created.
1s Normal Pulled pod/razeedash-5d5c4dcf49-krzw5 Successfully pulled image "quay.io/razee/razeedash:0.1.0"
0s Warning BackOff pod/razeedash-api-57bdf94d59-zfss7 Back-off restarting failed container
0s Normal EnsuringLoadBalancer service/razeedash-lb Ensuring load balancer
0s Warning CreatingLoadBalancerFailed service/razeedash-lb Error creating load balancer (will retry): failed to ensure load balancer for service razee/razeedash-lb: unsupported load balancer affinity: ClientIP

You have test the install on eks ?

Regards

Describe the bug
orgAdminKey is exposed on the GET & POST /v2/orgs endpoint

To Reproduce
Steps to reproduce the behavior:

1. Do a GET or POST to the /v2/orgs endpoint

For example, on a GET this response comes back:

{
    "_id": "70e9adfd-a4ac-4232-93f6-75a6ceb5e2fe",
    "name": "cfbtest",
    "orgKeys": [
        "orgApiKey-ddcf801b-ce63-45e7-ad41-b381240321c3"
    ],
    "orgAdminKey": "<REDACTED>", // orgAdminKey exposed here
    "created": "2020-01-14T02:24:18.976Z",
    "updated": "2020-01-14T02:24:18.976Z"
}

Expected behavior
orgAdminKey should never be exposed on any Razee API response.

Kubernetes Version (kubectl version):

• Client: 1.14.9_1543
• Server: 1.14.9_1543

Describe the bug
Stop using selfLink: selfLink is being deprecated around 1.20/1.21. we will need to either transition to creating the selfLink for the uses that we need it, or switching to a different attribute. I know selfLink is used in watch-keeper and composite-controller, need to see if it used anywhere else (maybe kube-class). see issue

[rcook@goku multicloud]$ kubectl get pods -n razee
NAME READY STATUS RESTARTS AGE
featureflagsetld-controller-6686c998dc-86z4h 1/1 Running 0 3m44s
kapitan-delta-5d4cc865f5-k2p57 1/1 Running 0 4m9s
managedset-controller-67f9899f46-vbzjv 1/1 Running 0 3m43s
mustachetemplate-controller-55684955b4-7d4kc 1/1 Running 0 3m42s
remoteresource-controller-6d79b48bb9-tspwf 1/1 Running 0 3m42s
remoteresources3-controller-79cdb96b4c-kw5fw 1/1 Running 0 3m41s
[rcook@goku multicloud]$ kubectl apply -f https://github.com/razee-io/Razee/releases/latest/download/resource.yaml
error: error validating "https://github.com/razee-io/Razee/releases/latest/download/resource.yaml": error validating data: [ValidationError(RemoteResource.spec.requests[0]): unknown field "options" in io.razee.kapitan.v1alpha1.RemoteResource.spec.requests, ValidationError(RemoteResource.spec.requests[1]): unknown field "options" in io.razee.kapitan.v1alpha1.RemoteResource.spec.requests]; if you choose to ignore these errors, turn validation off with --validate=false
[rcook@goku multicloud]$ kubectl version
Client Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.4+3569a06", GitCommit:"3569a06", GitTreeState:"clean", BuildDate:"2019-06-12T15:47:26Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.4+d4417a7", GitCommit:"d4417a7", GitTreeState:"clean", BuildDate:"2019-06-12T15:47:30Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

Current version of the repository:
commit 0f88fb5be03d2b0fda88e9eec999da357ca26f5d (HEAD -> master, origin/master, origin/HEAD)

Hello Razee team,

just to let you know that Kapitan as a name is already used by this project https://github.com/deepmind/kapitan (Google, DeepMind)
See also:

• https://kapitan.dev
• https://medium.com/kapitan-blog

Do you think you could rename your Kapitan component to avoid confusion? As both projects are addressing the same problems, I think it would be extremely confusing for users of both projects.

Regards
Alessandro, on behalf of the Kapitan team

• Update schemas: In kube 1.16 schemas are becoming mandatory and more strict. our current schemas aren't valid/structural, so we need to update all crd schemas to be ready for 1.16. See docs
• Stop using selfLink: selfLink is being deprecated around 1.20/1.21. we will need to either transition to creating the selfLink for the uses that we need it, or switching to a different attribute. I know selfLink is used in watch-keeper and composite-controller, need to see if it used anywhere else (maybe kube-class). see issue
• Enforcement: In 1.16, it appears kube has stopped sending ADDED every minute or so to watches. Need to investigate if this is true, and if so, how do we continue doing enforcement for our code.

• Watch-Keeper
• MustacheTemplate
• ManagedSet
• RemoteResource
• RemoteResourceS3
• FeatureFlagSetLD

Is your feature request related to a problem? Please describe.
Need a RESTful API to expose cluster, node, deployment, pod, and other resource info from the Razeedash database in a secure manner.

We will have a single instance of Razeedash set up which will have clusters from multiple clients registered (e.g. each clients cluster will have Watchkeeper installed and reporting back to the single Razeedash instance). We would like to display list of clusters, detailed cluster and node information, deployment, pod, and other resource information back to our clients in a Dashboard view.

Describe the solution you'd like
A RESTful API to expose, query, sort, and filter data from the Razeedash database including cluster, node, deployment, pod, and other resource info. For example, ability to query on deployment status and specifically what is failing is one of our use cases. We would also like the ability to query across one org's clusters for a deployment. If you need an exhaustive list of use cases, please let me know.

Data should not be exposed externally without auth in place.

Describe alternatives you've considered
We can query the Razeedash database directly or set up our own API shim, but we would prefer the Razee team to do that to isolate us from any changes you may make to the underlying database schema.

Additional context
Flows are documented here: https://ibm.box.com/s/2zkm8an33wi693feq0uv3eajl2k8r7c1 and have been discussed with Mike McKay.

If you would like separate Issues split out for this work, please let me know.

Is your feature request related to a problem? Please describe.
Need the ability to create a new org with a given ID via an API.

We will be supporting deployment of operators via Razee to client clusters and our solution will be supporting many clients. In order to isolate one clients clusters from another, we would like to be able to create one org within Razee per client. We would like to be able to specify the new org's ID.

Describe the solution you'd like
An API where we can pass in an org ID that we specify and any other info Razee requires, and it will create a new org with the given ID.

Describe alternatives you've considered
We can store a map of client account IDs to cluster IDs for a given client, and still accomplish our use cases, but it would make for a better integration if Razee could help keep track of this by using an org.

Additional context
Flows are documented here: https://ibm.box.com/s/2zkm8an33wi693feq0uv3eajl2k8r7c1 and have been discussed with Mike McKay.

Move internal pub/sub code to the razee-io versions for razeedash and razeeapi

When I was trying to install Razee in Kubernetes in Docker Desktop on Windows, I found pod featureflagsetld-controller failed with log

> [email protected] start /usr/src/app
> node src/index.js
{"name":"FeatureFlagSetLD","hostname":"featureflagsetld-controller-86798d5755-ck9j5","pid":22,"level":30,"msg":"Running FeatureFlagSetLDController.","time":"2019-06-04T12:44:21.813Z","v":0}
(node:22) [DEP0123] DeprecationWarning: Setting the TLS ServerName to an IP address is not permitted by RFC 6066. This will be ignored in a future version.
{"name":"FeatureFlagSetLD","hostname":"featureflagsetld-controller-86798d5755-ck9j5","pid":22,"level":50,"msg":"Unable to find KubeResourceMeta for kapitan.razee.io/v1alpha1: FeatureFlagSetLD","time":"2019-06-04T12:44:22.806Z","v":0}

I think the problem is probably caused by "Unable to find KubeResourceMeta for kapitan.razee.io/v1alpha1". This problem prevents me going on installing it. Is there any operation I missed?

Describe the bug
A clear and concise description of what the bug is.
When using automated deployment from a GHE repo, the yaml being deployed needs to be deployed to the cluster before automated deployment will work.

I have seen this several times. I install the remote resource and have a logs - f of the remotresource-controller pod. I watch the scheduler (it runs every 5 minutes and should be configurable, if it is I cannot find that configuration). I see a 404 error when the controller runs saying that the file designated in the RR is not found. I curl the same url with my token and it returns the file content. So I deploy the file manually and the next scheduled run says all is fine with that file - it exists and there are no changes. I update th efile and the update gets deployed by the RR
To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.
I would expect that any file in the rr would get deployed. I have added my rr itself to the rr, this way if I add files to that rr it should redeploy itself and pick up those new files
Screenshots
If applicable, add screenshots to help explain your problem.

Kubernetes Version (kubectl version):

• Client:
• Server:
  [jagaber@oc3715088365 razee-demo]$ kubectl version
  Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.0", GitCommit:"9e991415386e4cf155a24b1da15becaa390438d8", GitTreeState:"clean", BuildDate:"2020-03-25T14:58:59Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
  Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.7", GitCommit:"6c143d35bb11d74970e7bc0b6c45b6bfdffc0bd4", GitTreeState:"clean", BuildDate:"2020-01-14T00:28:37Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

Additional context
Add any other context about the problem here.

Is your feature request related to a problem? Please describe.
We implemented an add cluster callback API, but had to make it unauthenticated because Razee does not support a callback API that has authorization in place. We would like Razee to support auth on add cluster callback endpoints, because we are concerned about leaving our cluster callback endpoint open and exposed for attacks.

Describe the solution you'd like
Add an add cluster callback api key that can be specified in the razeedash-secret. Update Razee to pass this in as a header on each request to the add cluster callback API.

Describe alternatives you've considered
N/A

Describe the bug
On initial install, razeedash Deployment is failing. Logs from the remoteresource controller show:

"Error applying file to kubernetes. StatusCode: 422 url: https://github.com/razee-io/Razeedash/releases/latest/download/resource.yaml message: Deployment.apps \"razeedash\" is invalid: spec.template.metadata.labels: Invalid value: map[string]string{\"app\":\"razeedash\"}: `selector` does not match template `labels`

To Reproduce

1. Fresh IBM Kubernetes Service cluster (Kube 1.14.8_1536)
2. kubectl apply -f https://github.com/razee-io/Kapitan-delta/releases/latest/download/resource.yaml
3. kubectl apply -f https://github.com/razee-io/Razee/releases/latest/download/resource.yaml

Expected behavior
razeedash to be deployed

Kubernetes Version (kubectl version):

• Client: n/a
• Server: 1.14.8

Provide a way to hide resources i dont want to see, and a way to toggle them back to shown. ie. I dont want to see resources with kind: Event show up in the resources tab when i look at my cluster's list of resources.

My use-case is, I have a private GitHub.com repo, and I can use my personal-access-token in the Authorization header to read the content of a file in the private repo through the api. However I would like to put my access token in a secret resource, but it doesn't seem to be supported.

The documentation for RemoteResource doesn't mention any ability to use secretKeyRef in the headers - it would be good if that or some similar syntax could be supported.

That is, this works ok (when I insert my access token):

apiVersion: kapitan.razee.io/v1alpha1
kind: RemoteResource
metadata:
  name: hello-remote
spec:
  requests:
  - options:
      url: https://api.github.com/repos/maire-kehoe/test-razee-private/contents/helloConfigMap.yml?ref=master
      headers:
        Authorization: token <my-personal-access-token>
        Accept: application/vnd.github.VERSION.raw
        User-Agent: cluster-remoteresource-yml-github-agent

But this doesn't work:

apiVersion: v1
kind: Secret
metadata:
  name: hello-secret
type: kubernetes.io/basic-auth
stringData:
  password: token <my-personal-access-token>
---
apiVersion: kapitan.razee.io/v1alpha1
kind: RemoteResource
metadata:
  name: hello-remote
spec:
  requests:
  - options:
      url: https://api.github.com/repos/maire-kehoe/test-razee-private/contents/helloConfigMap.yml?ref=master
      headers:
        Authorization:
          valueFrom:
            secretKeyRef:
              name: hello-secret
              key: password
        Accept: application/vnd.github.VERSION.raw
        User-Agent: cluster-remoteresource-yml-github-agent

At present, the hello-remote resource is created, but the referenced config map is not created, and the kubectl describe rr hello-remote contains this status:

Status:
  Fatal:
    Status Code:  404
    Uri:          https://api.github.com/repos/maire-kehoe/test-razee-private/contents/helloConfigMap.yml?ref=master

that is, there's a 404 not found error because the Authorization value was not sent to the api.github.com (and they return 404 Not Found instead of 401 Unauthorized to avoid leaking file-existence info to unauthorized requests.)

Is your feature request related to a problem? Please describe.
Need a set of documentation on how to format remote resources in a way that Razee understands.

We have noticed instances where yamls that work with kubectl apply do not work with Razee so need to understand exactly what Razee expects, and if there are any suggested best practices on how to set them up (e.g. using List object instead of ---).

Describe alternatives you've considered
Trial and error

Additional context
Flows are documented here: https://ibm.box.com/s/2zkm8an33wi693feq0uv3eajl2k8r7c1 and have been discussed with Mike McKay.

I gave this a shot by provisioning a new IBM Cloud Databases for MongoDB instance and then tried to use the mongodb connection string provided:

mongodb://$USERNAME:$PASSWORD@c28c997f-c6eb-4e1d-a7df-325367ad9f11-0.0135ec03d5bf43b196433793c98e8bd5.databases.appdomain.cloud:31888,c28c997f-c6eb-4e1d-a7df-325367ad9f11-1.0135ec03d5bf43b196433793c98e8bd5.databases.appdomain.cloud:31888/ibmclouddb?authSource=admin&replicaSet=replset

But all I could get were connection errors from the mongodb connection lib:

razeedash-645c4d866b-l9bpz razeedash MongoNetworkError: connection 2 to c28c997f-c6eb-4e1d-a7df-325367ad9f11-0.0135ec03d5bf43b196433793c98e8bd5.databases.appdomain.cloud:31888 closed
razeedash-645c4d866b-l9bpz razeedash     at Socket.<anonymous> (/app/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/connection.js:276:9)
razeedash-645c4d866b-l9bpz razeedash     at Object.onceWrapper (events.js:315:30)
razeedash-645c4d866b-l9bpz razeedash     at emitOne (events.js:116:13)
razeedash-645c4d866b-l9bpz razeedash     at Socket.emit (events.js:211:7)
razeedash-645c4d866b-l9bpz razeedash     at TCP._handle.close [as _onclose] (net.js:561:12)

https://bxcs-slack-invite.mybluemix.net/
502 Bad Gateway: Registered endpoint failed to handle the request.

remove ns from kapitan components and let the razee build process wrap all necessary components in a managed set or have kapitan delta put in its ns. Then will need to fix the docs, the internal razee.io build process, and deploy steps.

I have a few clusters that have been rightly marked inactive now (after ~24 hours of not reporting in, it seems). When would I expect to see these clusters not reported on the Clusters view any longer and those associated Resources removed from the Resources view?

kubectl apply -f https://github.com/razee-io/Kapitan-delta/releases/latest/download/resource.yaml

kubectl get po -n razee                                                                    
NAME                                           READY   STATUS    RESTARTS   AGE
featureflagsetld-controller-5f7b86c5b9-5btnx   1/1     Running   0          7m44s
kapitan-delta-5d4cc865f5-wg89z                 1/1     Running   0          8m26s
managedset-controller-68c96974d6-mt7d7         1/1     Running   0          7m38s
mustachetemplate-controller-789b4d8ffd-dcsxw   1/1     Running   0          7m35s
remoteresource-controller-84744fc8fb-56r8q     1/1     Running   0          7m25s
remoteresources3-controller-675d868b57-bpk4b   1/1     Running   0          7m23s 

kubectl apply -f https://github.com/razee-io/Razee/releases/latest/download/resource.yaml
persistentvolume/mongo-pv-volume unchanged
persistentvolumeclaim/mongo-pv-claim unchanged
deployment.apps/mongo unchanged
service/mongo unchanged
secret/razeedash-secret unchanged
error: unable to recognize no matches for kind "RemoteResource" in version "kapitan.razee.io/v1alpha1"

Kubernetes version 1.3.7

Describe the bug
When running the Razee agent install on a cluster (e.g. kubectl create -f "http://169.45.231.109:8081/api/install/cluster?orgKey=orgApiKey-75dd7435-4567-4d5f-a1c9-a9735c55b1b4") it fails because RemoteResource CRD is not created yet.

The error is unable to recognize "http://169.63.135.198:8081/api/install/cluster?orgKey=orgApiKey-8ab80ecc-9c55-488c-b84e-35ad87dc7745": no matches for kind "RemoteResource" in version "deploy.razee.io/v1alpha1"

To Reproduce
Steps to reproduce the behavior:

1. Assuming Razee is set up run the "Razee Install Agent" command on a new ks cluster.

Expected behavior
Either one command that is successful or two separate commands if needed.

Kubernetes Version (kubectl version):

• Client: 1.14.9_1543
• Server: 1.14.9_1543

Is your feature request related to a problem? Please describe.
We are going to have customers installing the Razee agent on their clusters, and envision scenarios where they may no longer want the Razee agent components installed on their clusters anymore. Need a mechanism to cleanly remove everything installed by the "Install Razee Agent" command (e.g. kubectl create -f "http://169.45.231.109:8081/api/install/cluster?orgKey=orgApiKey-75dd7435-4567-4d5f-a1c9-a9735c55b1b4")

Describe alternatives you've considered
Tried kubectl delete -f "http://169.45.231.109:8081/api/install/cluster?orgKey=orgApiKey-75dd7435-4567-4d5f-a1c9-a9735c55b1b4" but it doesn't completely remove all installed components.

Is your feature request related to a problem? Please describe.
We need guidance or best practices on setting up a product ready instance of the Razeedash db and APIs.

Describe alternatives you've considered
Razee team hosts the instance for us.

Additional context
Flows are documented here: https://ibm.box.com/s/2zkm8an33wi693feq0uv3eajl2k8r7c1 and have been discussed with Mike McKay.

In OpenShift and on EKS I ran into the following situatations

RAZEEDASH_LB=$(kubectl get service razeedash-lb -n razee -o jsonpath="{.status.loadBalancer.ingress[*].ip}")
RAZEEDASH_API_LB=$(kubectl get service razeedash-api-lb -n razee -o jsonpath="{.status.loadBalancer.ingress[*].ip}")
kubectl create configmap razeedash-config -n razee \
  --from-literal=root_url=http://"${RAZEEDASH_LB}":8080/ \
  --from-literal=razeedash_api_url=http://"${RAZEEDASH_API_LB}":8081/```

This returned blank values as the load balancer svcs will be rather than an IP which was not returned within the json output

a255a6590936611e991111257d3fea92-714542047.us-east-1.elb.amazonaws.com

Also, for the deployments I had to remove the service load balancer parameter

    sessionAffinity: ClientIP

Hi Team,
I observe that if my source code url configured in the remoteResource contains an invalid kubernetes yaml, it permanently fails the remoteResource.
Even if the source code is fixed, RemoteResource remains in failed state.
ENV : IKS 1.13.7_1526

Is your feature request related to a problem? Please describe.
Need a notification from Razee whenever a cluster phones home to Razeedash for the first time, after the Watchkeeper agent has been installed.

Describe the solution you'd like
Webhook, event, or some type of notification to a REST API the first time a cluster connects back to Razeedash. The event should include the org ID, cluster ID, and cluster name associated with the cluster (if one was specified in a configmap).

Describe alternatives you've considered
We can poll Razee for clusters and check when the cluster in question is first listed there (assuming an API is implemented for this, or we could query the db directly). However, the ability to be notified would make for a better integration with Razee.

Additional context
Flows are documented here: https://ibm.box.com/s/2zkm8an33wi693feq0uv3eajl2k8r7c1 and have been discussed with Mike McKay.

Describe the bug
When using URL rewriting to route requests to Razee, watchkeeper fails to post to razeedash because the API base path is hardcoded to /api/v2 here https://github.com/razee-io/Razeedash-api/blob/1fe88efc21a2ba9e46aa8938fa9604828c0b7358/app/routes/install/index.js#L30.

In the watchkeeper logs we see “POST heartbeat to razeedash failed” and this is because it’s trying to post to https://<host>/api/v2/clusters/... instead of the route we are using for razeedash-api which is https://<host>/api/collector/v2/clusters/....

To Reproduce
Using Istio, we have a rewrite rule set up like this:

- match:
    - uri:
        prefix: /api/collector
    rewrite:
      uri: /api

So everything routed to https:///api/collector gets rerouted to https:///api

Expected behavior
We would like the API base path in https://github.com/razee-io/Razeedash-api/blob/1fe88efc21a2ba9e46aa8938fa9604828c0b7358/app/routes/install/index.js#L30 and everywhere else in the code to be configurable.