rancher / wrangler Goto Github PK

While working on adding a test for some unrelated pull request, I found that my code was panicking at some part.
Since I needed to use any object type for my test, I thought it was good to use to use a simple one from the Kubernetes core types, so I picked corev1.Service.

I turns out that the Conditions field for corev1.ServiceStatus uses metav1.Condition, whose fields include LastTransitionTime. This name is different from LastUpdatedTime, which is what wrangler's pkg/condition expects to set.
Beware that there is no call to IsValid() for the result of getFieldValue(value, "LastUpdateTime"), hence causing the panic when unconditionally trying to set its value.

On a separate note, this makes me wonder if the https://github.com/rancher/wrangler/tree/master/pkg/generated/controllers package is actually useful or it could be cleaned up.

Create a user cluster and delete, repeat the above, memory leak. #174

Running golangci-lint on master is generating the following issues:

pkg/objectset/objectset.go:116:8: Error return value of `o.err` is not checked (errcheck)
		o.err(errors.Wrapf(err, "failed to add %T", obj))
		     ^
pkg/objectset/objectset.go:122:8: Error return value of `o.err` is not checked (errcheck)
		o.err(errors.Wrapf(err, "failed to add %T", obj))
		     ^
pkg/apply/desiredset_process.go:213:8: Error return value of `o.err` is not checked (errcheck)
		o.err(err)
		     ^
pkg/relatedresource/changeset.go:86:15: Error return value is not checked (errcheck)
				runResolve(meta.GetNamespace(), meta.GetName(), ro)
				          ^
pkg/needacert/needacert.go:260:23: Error return value of `h.locker.Unlock` is not checked (errcheck)
	defer h.locker.Unlock(lockKey)
	                     ^
pkg/controller-gen/main.go:32:2: var `t` is unused (unused)
	t = true
	^
pkg/schemas/mappers/enum.go:15:2: field `field` is unused (unused)
	field string
	^
pkg/apply/desiredset_apply.go:92:11: S1012: should use `time.Since` instead of `time.Now().Sub` (gosimple)
		if d := time.Now().Sub(t); d.Seconds() > 1 {
		        ^
pkg/git/git.go:10:2: SA1019: "io/ioutil" has been deprecated since Go 1.19: As of Go 1.16, the same functionality is now provided by package io or package os, and those implementations should be preferred in new code. See the specific function documentation for details. (staticcheck)
	"io/ioutil"
	^
pkg/apply/desiredset_compare.go:7:2: SA1019: "io/ioutil" has been deprecated since Go 1.19: As of Go 1.16, the same functionality is now provided by package io or package os, and those implementations should be preferred in new code. See the specific function documentation for details. (staticcheck)
	"io/ioutil"

Could we get this using GitHub Actions to verify linting?

If a type is added and go generate is run, the auto generated files are not removed if the type is removed and go generate is run again.

Is it possible to override the generated UpdateFunc - current generated code produces:

                UpdateFunc: func(old, new interface{}) {
			newMeta, err := meta.Accessor(new)
			utilruntime.Must(err)
			oldMeta, err := meta.Accessor(old)
			utilruntime.Must(err)
			if newMeta.GetResourceVersion() == oldMeta.GetResourceVersion() {
				return
			}
			controller.handleObject(new)
		},

This is fine for controllers when the system is just made up of k8s objects. But what if we're making the attempt to sync external resources. For example I have a resource (outside of k8s) that exposes an api, that provides some controls, and describes the resources current state (such as a simple health check). This resource will creep away from the desired state do to "natural events". I would like to be able to auto remedy when the real world creeps away from the desired state described by the resource spec.

In the past I just used the template from the k8s sample controller. This provided granular control over the UpdateFunc. Maybe I'm using the wrong handler?

I was wondering what is the most native way to do this with wrangler, I'd like to avoid having to edit the generated code.

howdy folks,

built a project using wrangler very similar to github.com/rancher/wrangler-sample
i have a custom api, a controller, a codegen and clean func, etc.

when I run go generate I receive

FATA[0014] go modules copy failed: open ./.git/objects/04/f4b9bf3bd3e5a0229df44bc18d19d15ac0ac97: permission denied 

this somehow eliminates the content of all files in my .git directory (top level only).
the files themselves still exist but 0B empty

Followed README

cd $GOPATH/src/github.com/rancher/wrangler/sample
go generate
go build .
./sample

Go build failed with

sample git:(master) ✗ go build .
# github.com/rancher/wrangler/sample/pkg/generated/controllers/samplecontroller.k8s.io/v1alpha1
pkg/generated/controllers/samplecontroller.k8s.io/v1alpha1/foo.go:162:29: not enough arguments in call to c.controllerManager.Enqueue
    have (schema.GroupVersionKind, string, string)
    want (schema.GroupVersionKind, cache.SharedIndexInformer, string, string)
# github.com/rancher/wrangler/sample/pkg/generated/controllers/apps/v1
pkg/generated/controllers/apps/v1/deployment.go:162:29: not enough arguments in call to c.controllerManager.Enqueue
    have (schema.GroupVersionKind, string, string)
    want (schema.GroupVersionKind, cache.SharedIndexInformer, string, string)

Add scripts to /scripts that runs go generate and fails if there is a diff. Have drone run this script.

While working on the upgrade operator for k3os I've noticed that conditions are never updated and propagated even though the handler has mutated the status.

Currently wrangler only supports generation of a CRD with a single version.

The current release branches are based on rancher major.minor versions. Since the 1.0.0 release of wrangler there are revisions of wrangler (not released versions) used in rancher.

Three problems have arisen from this approach:

1. Dependabot and other tools can't detect updates to apply other projects as updates. This means that those using wrangler (including the many rancher repos) need to do the updates manually.
2. The version of wrangler being used across the many consumers (many of those being other rancher codebases) are inconsistent in the version used. It's often not the latest. This means some bugs are not fixed uniformly across the codebase.
3. Releasing wrangler now has no value or meaning. And, it has not happened since the 1.0.0 release.

This proposal is to do the following:

• Use release branches in the form release-vX.Y where X is the major version and Y is the minor version. There are two reasons for this scheme.
  • The rancher release branches are in the form release/vX.Y. Doing the wrangler version release branches with the same scheme would eventually cause conflicts.
  • This scheme is already in use within this repo (i.e. release-v0.8 already exists)
• All future releases would be cut from the release branch rather than master
• Once a release branch is created it's only for patch (i.e. bugfix releases). Those would first be merged into master and then cherry-picked into any release branches that should include it.
• The major/minor versions of wrangler to get continued work would be those that are in a supported release of rancher.

The git package within wrangler appears to only be used in two places and they are not using overlapping functionality. This package can be more easily maintained in those two separate places. So, this functionality should be documented as deprecated.

I would like my controller to only cache the secrets it controls.

I am thus only interested in a very small subset of the secret resources. Specifically the ones with specific owner references (a custom resource is the owner).

I would also be fine with caching the relevant namespaces or labels.

As part of a breaking change #194 that is required for kubernetes 1.24 support, I believe we should graduate this project from beta to stable.

Replace the this with a proper log library like logrus.

The readme says Wrangler is Beta but it has a 1.0.0 release. Assuming beta needs to be removed from the readme. Is that right?

For every generated resource in norman, there is also a generated mock interface which can easily be incorporated into unit tests. For example, the generated ConfigMap lister:

https://github.com/rancher/rancher/blob/ba269031d14456ffb19efabb826ea3d31147daf8/pkg/generated/norman/core/v1/zz_generated_config_map_controller.go#L60-L63

has a generated mock:

https://github.com/rancher/rancher/blob/ba269031d14456ffb19efabb826ea3d31147daf8/pkg/generated/norman/core/v1/fakes/zz_generated_config_map_mock.go#L48-L72

The equivalent resource in wrangler

has no corresponding mock. To use it in unit tests, the tests themselves need to create a mock. This can lead to either

1. no test is written because writing the mock is hard
2. the mock is created in a way that is inconsistent with other tests
3. the mock is duplicated across unit test files

There are some tests in wrangler. Can we get these run in CI? Say on every PR.

Note, running go test ./... fails today so the tests or functionality needs to be fixed as part of this.

Once Rancher is built on Go 1.20+ we should consider replacing the merr package with errors.Join from the standard library.

Similar to #210 but for lasso.

When attaching a debugger (eg. via delve) and hitting a breakpoint, wrangler processes are automatically killed after some seconds. This is due to the leader election mechanism.

The CATTLE_DEV_MODE environment variable allows to elongate the period, which makes it a good fit for debuggers. However, it also elongates the retry period, which means the initial leader election at process start will also take two hours, which is a bit impractical.

FMPOV CATTLE_DEV_MODE should make leader leases last longer, but not the retry period, so that process startup remains acceptably short.

CreateUnknownIfNotExists does not work for status objects. This is especially useful for the generated RegisterXStatusHandler functions, where it is idiomatic to only operate on the status field.

Application of nil DesiredSet does not delete existing resources. If I dry-run the apply and print the plan, it's clear that it's not finding the existing objects.

1. Run an Apply with an owner object and list of objects
2. Run an Apply with an owner and nil object list
3. The objects from step 1 are not deleted.

With wrangler 0.8.8:

DEBU[0005] DeleteManifest plan: apply.Plan{
  Create:objectset.ObjectKeyByGVK{schema.GroupVersionKind{Group:"helm.cattle.io", Version:"v1", Kind:"HelmChart"}:[]objectset.ObjectKey(nil)}, 
  Delete:objectset.ObjectKeyByGVK{schema.GroupVersionKind{Group:"helm.cattle.io", Version:"v1", Kind:"HelmChart"}:[]objectset.ObjectKey{objectset.ObjectKey{Name:"traefik", Namespace:"kube-system"}, objectset.ObjectKey{Name:"traefik-crd", Namespace:"kube-system"}}}, 
  Update:apply.PatchByGVK{}, 
  Objects:[]runtime.Object(nil)
}

With wrangler 0.8.10:

DEBU[0009] DeleteManifest plan: apply.Plan{
  Create:objectset.ObjectKeyByGVK{schema.GroupVersionKind{Group:"helm.cattle.io", Version:"v1", Kind:"HelmChart"}:[]objectset.ObjectKey(nil)}, 
  Delete:objectset.ObjectKeyByGVK{schema.GroupVersionKind{Group:"helm.cattle.io", Version:"v1", Kind:"HelmChart"}:[]objectset.ObjectKey{}}, 
  Update:apply.PatchByGVK{}, 
  Objects:[]runtime.Object(nil)
}

For some reason wrangler 0.8.10 isn't finding the existing objects to delete.