[Feature] Support dynamic refresh of watched namespaces about kuberay HOT 2 OPEN

@MortalHappiness Thanks for the proposal. Feel free to go ahead with working on this, I won't have any cycles to work on this in the next few months.

from kuberay.

I would like to work on this issue.

I also have some ideas about this issue:

• Instead of making this new ConfigMap only serve a single purpose of dynamically change the watched namespaces, I think it is better to have a flag like --configmap, which serves as all the operator configs that need to be constantly watched for changes. In this way, if we are trying to support dynamic change of other configs in the future, we don't need to introduce another CLI flag.
• Instead of polling the status of the configmap in a constant interval like the Flink operator does, we can watch it with kubebuilder just like we watch CRD. See https://book.kubebuilder.io/reference/watching-resources/externally-managed for details.
• Assume that the namespace of this configmap is the same as the namespace of the operator pod. If the operator is run outside of the cluster, assume the namespace of this configmap is default.
• If users set both watched namespaces in this configmap and in the operator CLI, the value in the configmap is ignored. This idea comes from that many CLI tools allows override config file values using the CLI arguments. For example, in helm you can have a values.yaml and override its value from the command line using --set.

from kuberay.