operator-framework / api Goto Github PK
View Code? Open in Web Editor NEWContains the API definitions used by OLM and Marketplace
License: Apache License 2.0
Contains the API definitions used by OLM and Marketplace
License: Apache License 2.0
Is there a reason that we're not using the regex from semver?
here:
"([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?(?:\+[0-9A-Za-z-]+)?$"
there:
"^(0|[1-9]\d*).(0|[1-9]\d*).(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-])(?:.(?:0|[1-9]\d|\d*[a-zA-Z-][0-9a-zA-Z-]))))?(?:+([0-9a-zA-Z-]+(?:.[0-9a-zA-Z-]+)*))?$"
This one does not work for prerelease versions, right?
I realize that this PR is just moving this functionality, but I'm wondering if we need another PR to handle prerelease versioning in the checker. This question should in no way hold up this PR.
Originally posted by @grokspawn in #238 (comment)
While running operator-sdk bundle validate
it was seen that an invalid environment in the CSV is not validated by the validator. The operator-sdk issue is operator-framework/operator-sdk#5888
The code below is incorrect, environment should be name & value
not name & image
.
env:
- name: MY_IMAGE
image: docker.io/foo:bar
The way that the alm-examples are validated today does not verify that the json is correct: https://play.golang.org/p/gnaPaxtAjdk
this Issue is to request an API call be created so that we can get a bundle's contents from a bundle image, something similar to this:
func GetBundleFromImage(imageName string) (registry.Bundle, error)
then you could validate that bundle using a method added onto the Bundle such
as:
validationResults := myBundle.Validate()
also, ideally the Validate would not produce output to the logs but instead leave that to the caller to log from the results as they wish.
we also are in need of a helper function on a registry.Bundle that would let us get the list of CRs back from the CSV's alm-examples annotation, something akin to:
crList := myBundle.GetCRs()
we took a stab at creating such a function here: https://github.com/operator-framework/operator-sdk/pull/2890/files/98cfa697390d427c918470124839195d6d503a0c#diff-26ea1412bbd74151f01f483ebc6a42ef
I was looking for a description of the "Provider" field and went looking at the API sources but that field and others are undocumented:
api/pkg/operators/v1alpha1/clusterserviceversion_types.go
Lines 270 to 288 in a80624e
@varshaprasad96 Has pointed me to Operator Metadata section of the Building a Cluster Service Version (CSV) for the Operator Framework document.
Perhaps those descriptions should be copied to the API sources.
Currently, the make manifests
to regen go-bindata needs to be run manually and sometimes it is forgotten. This step should be either run automatically or verify and warn people in advance.
The Subscription
API (v1alpha1) is missing godoc descriptions for SubscriptionSpec
fields. Godoc descriptions for APIs can help clarify the behavior of OLM subscriptions.
The startingCSV
field in particular needs a descriptive comment. For example, can this be used to "pin" the version of a deployed operator if such version exists in the referenced catalog?
Bundles have a size limitation because their manifests are used to create a configMap, and the Kubernetes API does not allow configMaps larger than ~1MB. However, from OCP version 4.9, these values were increased to ~4MB because we are compressing them. (More info[0]).
The change to allow bigger bundles from OCP 4.9 only impacts the full bundle size amount. Any single manifest within the bundle such as the CRD will still make the bundle uninstallable if it exceeds the default file size limit on clusters (~1MB).
Currently, we check the total size compressed of the bundle: https://github.com/operator-framework/api/blob/master/pkg/manifests/bundleloader.go#L50-L77 and https://github.com/operator-framework/api/blob/master/pkg/validation/internal/bundle.go#L129-L151
This task is for us to improve the check to ensure that we will also raise an error for all files that has a size bigger than ~1MB.
This is a feature request to add a way to capture logs to the manifests.GetManifestsDir() function.
This would allow callers to capture the logrus output to process as they wish and independent of
the logger being used within the api.
Something perhaps like:
manifests.GetManifestsDir(dir string, logOutput io.Writer) (...)
See that we have https://pkg.go.dev/github.com/operator-framework/[email protected]/pkg/validation/, however, the validators available are their current options has not been properly documented.
That would be great to have a description of what is checked for each of them as all possible options available.
I was unable to find a license file associated with the code base. Is this intentional? If not, when/how will this be licensed?
Are you asking about community best practices, how to implement a specific feature, or about general context and help around the operator-sdk?
Opening this on behalf of a support issue that came in around the lack of description
elements for catalogsource.status (deploy/chart/crds/0000_50_olm_00-catalogsources.crd.yaml) especially w.r.t. catalogsource.status.configMapReference
What did you do?
installed olm and ran k explain catalogsource.status
What did you expect to see?
descriptive text blobs for areas indicated in 'what did you see instead' section indicated by <no description>
tags.
What did you see instead? Under which circumstances?
A clear and concise description of what you expected to happen (or insert a code snippet).
GROUP: operators.coreos.com
KIND: CatalogSource
VERSION: v1alpha1
FIELD: status <Object>
DESCRIPTION:
<empty>
FIELDS:
conditions <[]Object>
Represents the state of a CatalogSource. Note that Message and Reason
represent the original status information, which may be migrated to be
conditions based in the future. Any new features introduced will use
conditions.
configMapReference <Object>
<no description>
connectionState <Object>
<no description>
latestImageRegistryPoll <string>
The last time the CatalogSource image registry has been polled to ensure the
image is up-to-date
message <string>
A human readable message indicating details about why the CatalogSource is
in this condition.
reason <string>
Reason is the reason the CatalogSource was transitioned to its current
state.
registryService <Object>
<no description>
Environment
operator-lifecycle-manager version:
v0.26.0
Kubernetes version information:
Client Version: v1.29.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.27.3
Do we have a plan to add package manifest api into this repo?
Currently, it is in the OLM repo.
https://github.com/operator-framework/operator-lifecycle-manager/tree/master/pkg/package-server/apis/operators
The API documentaion for ClusterServiceVerion is incomplete for the following properties:
CSV properties:
.spec.customresourcedefinitions.owned[].resources[]
.spec.customresourcedefinitions.required[].resources[]
.spec.apiservicedefinitions.owned[].resources[]
.spec.apiservicedefinitions.required[].resources[]
These all share the same description:
APIResourceReference is a Kubernetes resource type used by a custom resource
I've searched through OpenShift, Kubernetes, and Operator Framework repos and documentation and cannot find any further reference to an APIResourceReference
schema. The schema consists of three string-type properties: kind
, name
, and version
. These properties have no description defined in the schema.
The resource[]
properties above are currently only consumed by the web console in order to list related resources on operand details pages, but we recently had a bug opened where this API was being misused. Adding some descriptions might be helpful in preventing future bugs or confusion.
We can add an additional check at https://github.com/operator-framework/api/blob/master/pkg/operators/v1alpha1/catalogsource_types.go#L287 to see if image has @sha
as I believe the polling to see the update is required only there is scope of image getting updated but an image referencing digest and not tag will not receive new updates via push.
Example:
$ operator-sdk bundle validate ./testdata/go/v3/memcached-operator/bundle/ --select-optional name=multiarch
WARN[0017] Warning: Value memcached-operator.v0.0.1: unable to inspect the image (quay.io/example/memcached-operator:v0.0.1) : docker pull quay.io/example/memcached-operator:v0.0.1 failed with error: (exit status 1) Error response from daemon: unauthorized: access to the requested resource is not authorized
WARN[0017] Warning: Value memcached-operator.v0.0.1: unable to inspect the image (gcr.io/kubebuilder/kube-rbac-proxy:v0.12.0) : docker manifest inspect gcr.io/kubebuilder/kube-rbac-proxy:v0.12.0 failed with error: (exit status 1) Get "https://gcr.io/v2/kubebuilder/kube-rbac-proxy/manifests/v0.12.0": unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication
WARN[0017] Warning: Value memcached-operator.v0.0.1: check if the CSV is missing the label (operatorframework.io/arch.<value>) for the Arch(s): ["amd64"]. Be aware that your Operator manager image ["quay.io/example/memcached-operator:v0.0.1"] provides this support. Thus, it is very likely that you want to provide it and if you support more than amd64 architectures, you MUST,use the required labels for all which are supported.Otherwise, your solution cannot be listed on the cluster for these architectures
WARN[0017] Warning: Value memcached-operator.v0.0.1: check if the CSV is missing the label (operatorframework.io/os.<value>) for the SO(s): ["linux"]. Be aware that your Operator manager image ["quay.io/example/memcached-operator:v0.0.1"] provides this support. Thus, it is very likely that you want to provide it and if you support more than linux SO you MUST,use the required labels for all which are supported.Otherwise, your solution cannot be listed on the cluster for these architectures
INFO[0017] All validation tests have completed successfully
PS.: It probably only occurs when we also are unable to inspect some image and we face errors
Currently, we are able to validate the bundle against scenarios to check if they are not ok to be published and then, we receive a list of errors and/or warnings. However, we need to grep by text to know the kind of errors and warnings that came out to do specific actions according to them.
I'd like to see the implementation of the api allowing us to define code for errors. In this way, we could check by these codes instead of grepping a text that can be easily changed/improved. One example of what is expected here is the HTTP implementation with code errors which allow us to easily check them.
This idea recently came up, and I figured the api repo is the best place to document org-wide annotations. There really should be some authoritative list of these annotations, which are currently (maybe) documented by the projects that use them, ex. the set of Operator SDK plugin names, helm-operator annotations, etc.
Suggested structure: docs/annotations/<repo or project or component>.md
with a list of each annotation and description:
- key: operatorframework.io/paused
key-format: <optional string for formatted keys, an alternative to "key">
values: ["true", "True"]
value-format: <optional string for formatted values, an alternative to "values">
owner: https://github.com/operator-framework/operator-lib <!-- some owner link -->
refs: ["predicate", "handler"] <!-- list of subpackages, files, etc -->
description: |
A long-form description of the annotation. Should be fairly all-encompassing if set.
description-link: <optional link to a description, an alternative to "description">
/kind documentation
/priority important-longterm
api/pkg/validation/internal/good_practices.go
Line 243 in 28c6773
This warning is always returned, even when you are not using the customresourcedefinitions
field. Example:
spec:
apiservicedefinitions: {}
customresourcedefinitions: {}
It's even in the docs where the show how to validate the bundle https://sdk.operatorframework.io/docs/advanced-topics/custom-bundle-validation/
Would be nice to be able to return INFO level Messages for the validators.
See that currently, we have not this option: https://github.com/operator-framework/api/blob/master/pkg/validation/errors/error.go
feature request...
if a user of this api wanted to specifically check a CSV for 'example annotations' it would be
nice to have a predefined sub-type for that specific validation check...
for example:
if err.Type == errors.CSVFileNotValid && err.SubType == errors.ExampleAnnotationsNotFound {
}
If we duplicate entries for the owned.crd then the error bellow will be faced:
ERRO[0000] Error: Value ceph.rook.io/v1, Kind=CephCluster: owned CRD "ceph.rook.io/v1, Kind=CephCluster" not found in bundle "ocs-operator.v4.8.0"
This issue is an RFE for we inform the users that the entry is duplicated instead
It was too hard to be found to solve the operator-framework/operator-sdk#4374
Description
ClusterServiceVersionValidator is not raising errors when the annotations are duplicated
The e.g scenario can be found in: https://github.com/redhat-openshift-ecosystem/community-operators-prod/pull/1121/files#diff-c0ad3966e15bf1757f16f02b664a3093ebf00f0e74956753ee48de223996268c
See that community is calling the bundle to validate and it does not fail when the CSV defined the annotations twice:
apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
annotations:
alm-examples: '[]'
capabilities: Basic Install
categories: Application Runtime, Monitoring, Security
certified: "false"
containerImage: quay.io/deployment-validation-operator/dv-operator:0.2.2
createdAt: 12/14/2021
description: The deployment validation operator
repository: https://github.com/app-sre/deployment-validation-operator
support: Best Effort
name: deployment-validation-operator.v0.2.2
annotations:
ignore-check.kube-linter.io/minimum-three-replicas: "This deployment uses 1 pod as currently replicating does not replicate metric data causing installation issues"
spec:
I would like to serialize a Subscription resource to YAML using code like this:
subscription := operatorsv1alpha1.Subscription{
TypeMeta: metav1.TypeMeta{
APIVersion: operatorsv1alpha1.SubscriptionCRDAPIVersion,
Kind: operatorsv1alpha1.SubscriptionKind,
},
ObjectMeta: metav1.ObjectMeta{
Namespace: namespaceName,
Name: pkg.Name,
},
Spec: &operatorsv1alpha1.SubscriptionSpec{
Package: pkg.Name,
Channel: channel.Name,
InstallPlanApproval: operatorsv1alpha1.Approval(subscribeFlags.Approval),
CatalogSource: pkg.Status.CatalogSource,
CatalogSourceNamespace: pkg.Status.CatalogSourceNamespace,
},
}
operatorsv1alpha1.AddToScheme(scheme.Scheme)
corev1.AddToScheme(scheme.Scheme)
serializer := json.NewSerializerWithOptions(
json.DefaultMetaFactory, scheme.Scheme, scheme.Scheme,
json.SerializerOptions{
Pretty: true,
Yaml: true,
Strict: true,
})
if err := serializer.Encode(&subscription, os.Stdout); err != nil {
return err
}
This works, except that operator-framework/api
defines a Subcription
resource like this:
type Subscription struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata"`
Spec *SubscriptionSpec `json:"spec"`
// +optional
Status SubscriptionStatus `json:"status"`
}
Which means that the serialized output always includes a status
element with a null status.lastUpdated
field. Submitting the serialized manifest fails with:
error: error validating "STDIN": error validating data: ValidationError(Subscription.status): missing required field "lastUpdated" in com.coreos.operators.v1alpha1.Subscription.status; if you choose to ignore these errors, turn validation off with --validate=false
Is there a canonical method for serializing these resources without including the status
field? Is there a reason that the Subscription
type isn't defined like:
type Subscription struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata"`
Spec *SubscriptionSpec `json:"spec"`
// +optional
Status *SubscriptionStatus `json:"status,omitempty"`
}
If the Status
field were nil-able, it would be easy to get the desired output.
#213 updated the way that tooling dependencies were pulled in after the root vendor directory had been removed, and the new mechanism relies on go get ...
functionality which will break in Go 1.18.
See the Go 1.18 release notes for more information.
Update the root module version from Go 1.17 to 1.18. The other major OLM repositories are already on Go 1.18.
Verify that all the operators' Deployment has resource requests set on every container. In cases where no resource requests have been supplied for CPU and memory the verification should fail, e.g;
...
# TODO(user): Configure the resources accordingly based on the project requirements.
# More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
resources:
requests:
cpu: 10m
memory: 64Mi
...
Kubernetes resource requests are required for ResourceQuota to work. Users would like to apply these quotas in namespaces where operators are deployed, to limit consumption on the cluster cores.
Create a new validator to check such as (ResourcesRequestsValidator).
c/c @dmesser
OLM requires the ClusterServiceVersion's spec.version
field to be valid semver (parsable with blang/semver.Parse()
). For example,
In opm:
olm.package
properties.In OLM:
See operator-framework/operator-registry#910
We should add a new validation in the CSV validator to ensure spec.version
is set to a string that can be parsed with semver.Parse
and metadata.annotations.skipRange
is set to a string that can be parsed with semver.ParseRange
.
Process:
Tried creating OperatorCondition CR on cluster, and fetching it using controller-runtime client API calls.
Steps followed:
// created a operator condition object
operatorCond = &api.OperatorCondition{
ObjectMeta: metav1.ObjectMeta{Name: "operator-condition", Namespace: ns},
Status: api.OperatorConditionStatus{
Conditions: []metav1.Condition{
{
Type: conditionFoo,
Status: metav1.ConditionTrue,
Reason: "foo",
Message: "The operator is in foo condition",
LastTransitionTime: transitionTime,
},
},
},
}
// Created a scheme, added the api to scheme since this is a third party api and passed in the client
var sch = runtime.NewScheme()
err := api.AddToScheme(sch)
Expect(err).To(BeNil())
cl, err := client.New(cfg, client.Options{Scheme: sch})
// Used the client to get the object from cluster
o := &api.OperatorCondition{}
err = cl.Get(ctx, *objKey, o)
The error appears to be: no matches for kind "OperatorCondition" in version "operators.coreos.com/v1"
I tried setting GVK in the operatorCond
object, specifying the APIVersion
in TypeMeta
field of the object and also registering the resource to the scheme builder manually. But nothing works. Am I missing something here?
validation that exists today in the operatorhub test suite only checks that the csv name is "name + a version" not "name + the version field"
api/pkg/validation/internal/operatorhub.go
Line 247 in a7ebc81
This means teams can release myoperator.v.1.0.0
with version: 1.0.1
which should not pass validation.
Description
See an example of the warns/issues raised by the validators: redhat-openshift-ecosystem/community-operators-pipeline#252 (comment)
The goal of this task is to ensure that all messages contain what is wrong/or could be improved with the suggested action to sort out the need.
Note that we can:
We're currently using the v3 module version of yq and v4 version has been out for a couple of years now.
This works with the v1alpha1 types that I have tried (ClusterServiceVersion, Subscription, CatalogSource) but not OperatorGroups. I've included a minimal example to trigger the panic but it also panics when the required fields in the OperatorGroup object are filled out.
For context, I am trying to use the v1 scheme to recognize an operator group yaml but create the object with a dynamic client which is why I want to convert from OperatorGroup to unstructured.
package main
import (
"fmt"
olmv1 "github.com/operator-framework/api/pkg/operators/v1"
olmv1scheme "github.com/operator-framework/api/pkg/operators/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
)
func main() {
var opGroup runtime.Object
opGroup = &olmv1.OperatorGroup{}
sch := runtime.NewScheme()
if err := olmv1scheme.AddToScheme(sch); err != nil {
fmt.Printf("Error adding scheme: %v", err)
}
unstr := unstructured.Unstructured{}
err := sch.Convert(opGroup, &unstr, nil)
if err != nil {
fmt.Printf("Error: %v",err)
} else {
fmt.Printf("Converted operator group to unstructured successfully")
}
}
panic: value method k8s.io/apimachinery/pkg/apis/meta/v1.Time.ToUnstructured called using nil *Time pointer
goroutine 1 [running]:
k8s.io/apimachinery/pkg/apis/meta/v1.(*Time).ToUnstructured(0x100000001?)
<autogenerated>:1 +0x47
sigs.k8s.io/structured-merge-diff/v4/value.TypeReflectCacheEntry.ToUnstructured({0x1, 0x0, 0x0, 0x0, 0x1, 0x0, 0x0, {0x0, 0x0, 0x0}}, ...)
/root/go/pkg/mod/sigs.k8s.io/structured-merge-diff/[email protected]/value/reflectcache.go:188 +0x6fd
k8s.io/apimachinery/pkg/runtime.toUnstructured({0xb06b80?, 0xc00019d020?, 0x2?}, {0xa60120?, 0xc0001f63f0?, 0x4?})
/root/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/converter.go:655 +0x125
k8s.io/apimachinery/pkg/runtime.structToUnstructured({0xab25e0?, 0xc00019d000?, 0x45bb25?}, {0xa60120?, 0xc0001f63a0?, 0x28a60?})
/root/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/converter.go:843 +0x7dd
k8s.io/apimachinery/pkg/runtime.toUnstructured({0xab25e0?, 0xc00019d000?, 0x3?}, {0xa60120?, 0xc0001f63a0?, 0x98?})
/root/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/converter.go:692 +0x85c
k8s.io/apimachinery/pkg/runtime.structToUnstructured({0xab2520?, 0xc00019cea0?, 0xc0001d2ff0?}, {0xa6c500?, 0xc00012c5f8?, 0x415945?})
/root/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/converter.go:843 +0x7dd
k8s.io/apimachinery/pkg/runtime.toUnstructured({0xab2520?, 0xc00019cea0?, 0xc00019cea0?}, {0xa6c500?, 0xc00012c5f8?, 0xc000151db8?})
/root/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/converter.go:692 +0x85c
k8s.io/apimachinery/pkg/runtime.(*unstructuredConverter).ToUnstructured(0x1023a50, {0xaf9a20?, 0xc00019cea0})
/root/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/converter.go:586 +0x3ba
k8s.io/apimachinery/pkg/runtime.(*Scheme).Convert(0xc0001a39d0, {0xaf9a20, 0xc00019cea0}, {0xb03720, 0xc00012c5f0}, {0x0?, 0x0})
/root/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/scheme.go:381 +0x2af
main.main()`
We were playing with deploying the gitlab.com/redhat gitlab-runner-operator. It does not specify resources in the rbac proxy in the csv. We tried overriding them with the Subscription object but it looks like the resources in the subscription don't have a way to reference a container name for which container to apply the resource config to when the csv references multiple containers. The resource requirements for the rbac proxy can be different then the requirements for the operator.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.