Containerize ts-bridge, remove GAE dependencies, and make it deployable in K8S about ts-bridge HOT 4 OPEN

The largest change that will be necessary to make ts-bridge run without App Engine is probably migrating metric records from App Engine Datastore API to Cloud Datastore (trivial) or another database (a bit harder). I would recommend Cloud Datastore, since the API is very similar. I suspect at some point this might become necessary even on App Engine, since newer App Engine Go runtimes no longer support proprietary App Engine APIs.

The second significant change is to make something trigger the regular sync (which is currently done by App Engine cron):

• if you are moving to a container, you can just use a periodic timer within the running process.
• if you are moving to GCF, you could use Cloud Scheduler API.

As part of this you might also want to implement some basic locking to prevent multiple sync operations from running concurrently (which can happen if you start multiple containers with the same configuration), since they will fight while trying to write data to Stackdriver. Since App Engine cron does not execute multiple concurrent operations, this has not been necessary on App Engine.

from ts-bridge.

@knyar @yanske1 - appreciate your thoughts on this.

from ts-bridge.

if you are moving to GCF, you could use Cloud Scheduler API.

Cloud Scheduler does not support VPC Service Controls as well. In case of K8S/GKE (which is a preferred product for a customer that is blocked at the moment) we could implement ts-bridge the following way:

• K8S deployment of size=1 that has single container pod that exposes / , /sync and /cleanup endpoints
• K8S ClusterIP service that exposes the deployment inside the cluster. To access index page(/) one can use kubectl port-forward , this allows to not expose index page to the outside world and do not have any external load balancers
• K8S CronJob that is scheduled every minute, it can be based on busybox or similar lightweight container, it just calls the /sync endpoint of ClusterIP service . As CronJobs have some limitations we should prevent multiple sync operations to run simultaneously.
• K8S CronJob that is scheduled every 12 hours and calls /cleanup enddpoint.

from ts-bridge.

#37 that @Temikus has pushed recently is the first step in addressing this issue.

from ts-bridge.