tektoncd / resolution Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
etcd publishes guidance on the approximate max amount of data it can handle given cluster size / hardware. https://etcd.io/docs/v3.5/op-guide/hardware/#example-hardware-configurations
Benchmark the numbers of supported resolved ResourceRequest
objects before cluster performance is negatively impacted and document here. We are specifically interested in the size of total data in the cluster so make sure to benchmark with many 1.5MB ResourceRequests
.
Add a method to the framework.Resolver
interface that lets a resolver specify the params it accepts, maybe as openapi? Then return that param spec to Pipelines somehow for it to use when validating taskRef.resource
and pipelineRef.resource
.
When Tekton Pipelines validates the resolver
and resource
fields of a pipelineRef
or taskRef
Pipelines' validating webhook should be able to check that those are the params that the resolver
of that type expects. e.g. the git
resolver expects params url
, commit
or branch
, path
. This should be something Pipelines can do before it submits those params in a ResourceRequest
so that errors can be very quickly bubbled back to the user.
Add the boilerplate and setup code necessary for e2e testing of the resolution project.
When new code is added it should be coverable with some kind of e2e testing.
Each resolver's docs should clearly indicate the type
label string that it responds to.
Right now we document the params
that each resolver expects but one oversight is that we don't document the type
. e.g. git
for git resolution, bundles
for bundle resolution, etc...
Right now any resolver based on pkg/resolver/framework
does not have an easy way to pull in admin configuration options. In any realistic production-ready resolver it's highly likely that some amount of configuration will be required:
framework.Resolver
interface has a hard-coded 30 second timeoutSo the feature request is to add a new optional interface for Resolvers to implement. Something like:
// ResolverConfigMap is an optional interface for a resolver to implement that allows it
// to register the name of a configmap that holds its admin settings.
type ResolverConfigMap interface {
// OnConfigMapUpdate is called whenever the watched configmap changes
OnConfigMapUpdated(ctx, v1.ConfigMap)
}
If a resolver implements this interface the framework would start monitoring the resolver's ConfigMap using knative's ConfigMap helpers. For inspiration on exact implementation see Tekton Pipeline's Config store codebase.
Alongside support in code it would also be useful to add:
timeout
setting to the git resolver and bundles resolverRight now each resolver includes an example ResolutionRequest
in its README that shows how it can be used. However, this is not very user-centred since users will most likely be coming to tekton resolution from tekton pipelines.
To better help users understand how resolution is used each resolver should document an example PipelineRun
that shows exactly how the resolver is used from Tekton Pipelines.
If a Pipeline references multiple Tasks that all utilize the same branch or tag then we need to make sure that all those references resolve to the same commit across that entire PipelineRun. Figure out a way to do this.
On large (mono)?repository, a git clone may take times a lot of time, if we need to just checkin a few files.
It would be nice if we could instead use Git providers (i.e: github/gitlab) APIs instead of checking out the full tree just to grab a few files.
We may along the way will be able to support private repository in gitresolver.
Large repository checking,
Setup whatever scripts, yamls, objects etc are needed for continuous integration to start on pull requests. See https://github.com/tektoncd/plumbing for the infra and https://github.com/tektoncd/results for a recent example of a project setting up CI.
Testing reconcilers requires quite a bit of setup code which we don't have yet for this project. This is an important short-term problem for three different classes of test:
Primarily concerned here with supporting developers of new resolvers by giving them a pain-free-as-possible way for them to exercise their resolver code quickly and easily.
Since a resolver is just a reconciler under the hood hopefully we can find a way to provide a Resolver Test Harness that is general purpose for anyone's resolver development.
The Tekton Pipelines unit test suite includes a plethora of setup code that performs all the necessary injection gubbins to support testing their Reconcilers. This might be a good first place to look for inspiration: https://github.com/tektoncd/pipeline/blob/main/test/controller.go
The version of knative.dev/pkg that tekton resolution is built on is now quite old. Upgrade to latest version.
Right now every ResolutionRequest
is timed out after 1 minute.
This feature request is to allow an admin to configure this global timeout to be longer or shorter depending on their cluster's needs. Individual resolvers can still set their own timeout for requests (so long as its less than the global one) but this feature is specifically for the global maximum enforced across all requests.
The reason for this global maximum is that it prevents "zombie" requests with a misconfigured resolution.tekton.dev/type
label from sticking around in incomplete state forever. It also means that a resolver crashing or hitting errors doesn't put a resolution request in permanent limbo.
Admins should be able to control the maximum amount of time any resolution request is allowed to take before being marked as "timed out" and failed, thereby also failing the PipelineRun depending on it.
There are essentially no docs for resolution currently. Introduce an initial set that covers:
Help end-users get started with tekton resolution, help developers attempting to write their own resolver, and capture as much background context as possible for the reasoning behind resolution's current state. I'm using https://documentation.divio.com/ as a guide to the different kinds of docs that it would be worth having initially.
Add a set of test helpers to resolution
that projects like Pipelines can use to easily stub / mock ResourceRequest
objects, Requester
and ResolvedResource
implementations, and errors.
Currently Pipelines maintains its own set of test helpers for stubbing resolution types and also code for mocking the base64 encoded response to a ResourceRequest
object: https://github.com/sbwsg/pipeline/blob/pipelineref-remote-res/pkg%2Freconciler%2Fpipelinerun%2Fpipelinerun_test.go#L8770-L8791
It would be great if Pipelines did not have to host this code and instead this kind of test support was available from resolution
test helpers. Once these helpers are written Pipelines should be updated to use them.
Nightly builds for resolution
Nightly builds enable installing the project from a known / signed version so we can be used in environments beyond dev - like the robocat and dogfooding cluster, or users who'd like to experiment with this.
Specifically I would like to do nightly installs on robocat like for other projects, to catch update issues, and I would like to run the service in dogfooding and start using it to refactor some of the logic used for nightly builds (today we rely on kustomize's ability to clone remote git repos on the fly).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.