Comments (12)
Issue-Label Bot is automatically applying the label kind/feature
to this issue, with a confidence of 0.91. Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.
from tanka.
Can you draft a design doc first? This is one of the bigger features and I would like some additional discussion around it so we can make sure we get it on the first take right before we write anything to the users' clusters?
from tanka.
Hi, I like this idea!
We might not even need a CRD for this purpose, because Kubernetes labels should provide enough functionality to do what we need:
- On creation, Tanka injects a label into every object it creates (e.g.
tanka.dev/origin: default/loki@config-repo
- On apply / diff,
kubectl get -l
allows us to get a list of all previously created objects, which we can use to easily compute the objects to remote
It might make sense to leave this optional using a flag such as tk apply --tidy
, which would if supplied also include these objects as deleted in the diff
from tanka.
This is such a delightful simplification. I'll see if I can get it done in the next week or so.
from tanka.
I considered a design doc, but needed to get my hands dirty to prove it could work. Totally happy to rewrite the PR I just submitted based upon better ideas.
from tanka.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
from tanka.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
from tanka.
+1 on for the labels. Docker compose do it in the same way:
"Labels": {
"com.docker.compose.config-hash": "32e9d85cbe1711281c7a3b57905307c3a204af30a7b0e4d92f614ec9b307d3e6",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "dev",
"com.docker.compose.service": "dev",
"com.docker.compose.version": "1.16.1"
}
from tanka.
Using labels on resources is difficult as it requires checking for resources across every apiversion/kind that kube-api supports. With CRDs, this can become quite difficult.
I think a simple "state" file approach would work best here. eg, a simple configMap is used to list all of the resources created, with some simple metadata about each. The configmap should be named based on the environment name defined in spec.json metadata
eg
apiVersion: v1
kind: ConfigMap
metadata:
name: default
namespace: monitoring
data:
state.json: |
{
"apps/v1:Deployment:grafana": {
"timestamp": "2020-03-09T19:46:00+00:00",
"user": "woodsaj"
},
"v1:ConfigMap:grafana-dashboards": {
"timestamp": "2020-03-09T19:46:02+00:00",
"user": "woodsaj"
}
}
When new resources are deployed, they are added to the state file. Entries are only moved from the state file after the resources have been removed from k8s.
eg
When tk apply
is run, it fetches this configMap and loads the state. The new state.json is then created internally after generating all of the resources from jsonnet. Tanka can then inform the user of any resources that are defined in the current state, but not in the new state and so will therefore be pruned after the apply completes.
if tk apply -t
is used, the existing state will just be updated, adding new resources or simply updating the metadata for the specific resources being updated.
As an alternative to storing the state in configMaps, we could take the terraform approach and have pluggable support for state storage, eg S3/GCS. Though I would prefer to just start with the configMap approach, we can always add support for other storage options in future.
from tanka.
I worry the configmap option has a lot of edge cases that need to be considered when actually implementing. If it gets out of sync through asynchronous runs or inconsistent tk versions or manual changes it can be difficult to feel confident it is accurate any more. The labels are much simpler, and I like to hope the discovery problem is solvable.
I have a PoC for scanning a cluster for all resources of all types, including CRDs.
Repository here. It uses the discovery and dynamic apis to enumerate all types in your cluster, and scan them all with a label filter. It usually takes under 10 seconds to scan everything and list out objects.
So my proposed implementation would be something like:
- All resources managed by tanka get labeled with
tanka.dev/environment: $ENV
(I think we should do this regardless of pruning solution and will make another issue for that) - Add a
--prune
flag totk diff
andtk apply
that will scan the cluster for resources matching the above label and list them out and/or delete them.
It does require bringing in the client-go packages, which I hear we have been hesitant to do so far. I would also like to discuss that, but probably in another issue.
from tanka.
@captncraig some thoughts on this:
All resources managed by tanka get labeled with tanka.dev/environment: $ENV (I think we should do this regardless of pruning solution and will make another issue for that)
Yes!
It does require bringing in the client-go packages, which I hear we have been hesitant to do so far. I would also like to discuss that, but probably in another issue.
I don't think we need that. We already query kubectl api-resources
on every run anyways, so we know about all kinds active in the cluster. Also, it reports only latest versions of a given API, which allows us to check only those, thus saving us from doing advanced cross-version logic.
Add a --prune flag to tk diff and tk apply that will scan the cluster for resources matching the above label and list them out and/or delete them.
While my #131 did that, it became obvious that it has drawbacks, such as accidental deleting of resources when applying, because you forgot to rebase against master before.
Instead, I'd prefer to have this as a separate action tk prune
, which shows a prune-only diff before, much like a tk delete
would behave if we found the time to implement it, so it a very explicit action
from tanka.
I like an explicit command. tk prune --dry-run
and tk prune
would be very obvious with what you want to do. Perhaps if we merge in the labels, we can do more experimentation with how well kubectl alone can pick up changes.
from tanka.
Related Issues (20)
- Helm-Templating: if $.Capabilities... HOT 3
- container cannot start HOT 2
- Tanka init fail HOT 3
- Add TANKA_PAGER env var HOT 1
- Helm tool credentials management for private repo charts
- tk apply crashes by creating Role from helm chart HOT 3
- Manifests not sorted by .metadata.generateName HOT 1
- `tk fmt` is inconsistent in printing final newline
- Additional verbosity level for `tk fmt`: Only print changed files, not all files HOT 1
- tk using wrong kubeconfig context HOT 4
- Go version mismatch
- Keep comments in chartfile.yaml? HOT 2
- tk env list throws error if environments/envx/sub/main.jsonnet exists
- Tanka panics if non-string annotation values are passed HOT 1
- ingressFor HOT 3
- std.objectKeysValues fails tk lint HOT 1
- `tk apply|diff ...` panics with `panic: runtime error: slice bounds out of range [:200] with length 197` HOT 2
- Update website dependencies HOT 1
- It's not possible to install Kong through helm chart HOT 6
- Is there a way to generate documentation from code-comments? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tanka.