When I deploy GitLab app in KubeSphere Apps, KubeSphere complains the following error:

Error: 

error while running post render on files: error while running command 
/tmp/helm-operator/gitlab-project_gitlab-p7vtz0_8qjk9jm2924x5m/helm-post-render.sh.

error output: Error: map[string]interface {}(nil): yaml: unmarshal errors: line 19:
    mapping key "enable-vts-status" already defined at line 17 : exit status 1

That's because resource files rendered by helm contain multi key enable-vts-status.

I found that there are some related occurrences:

Suggestion

You can remove the dedicated enable-vts-status config in controller-configmap.yaml to solve this problem:

/kind bug
/good-first-issue

Current csi-qingcloud chart has resource request/limits for every container and it works for most small-size clusters (< 20 nodes). Now csi-qingcloud is widely used in public qingcloud and private clouds. Sometimes, user complains the csi-qingcloud is throttled or OOM killed due to lacks resources for some containers and we then tell them to scale up based on their report.

VPA(Vertical Pod Autoscaler) is a nice feature that we may use to automatically tune the resource request/limits based on some metrics. Please investigate if it's possible and what need to do.

refer:

• https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler)

can i do this?

helm install kubesphere \
  --set image.tag=v3.1.1 \

通过ks2.1.1平台导入官方2.0.0版本的 skywalking7.0.0 helm chart包到企业空间后，在提交部署时提示异常：

Internal Server Error
rpc error: code = Unknown desc = Object 'Kind' is missing in '# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # instance for template metric '

模板文件中的参数配置需要引用定义的变量，但变量未定义：
containerConfID: {{ .Values.sc.containerConfID }}

I deployed several apps from KubeSphere app store. It shows some error as follows.

I installer an HA K8s, then using the helm to installer qingcloud csi plugin. I found the controller is not HA. Is that problem?

/cc @min-zh

We are using our own monitoring stack. We want to avoid scenarios like when we reinstall Kubesphere, it should not bring back Kubesphere's Prometheus and conflict with our Prometheus deployment.

But when we choose monitoring type as external in the helm chart values
https://github.com/kubesphere/helm-charts/blob/master/src/main/ks-installer/values.yaml#L76 we are facing errors in the fresh Kubesphere deployment.

E0516 18:37:14.984475 1 notification_controller.go:113] get /, Kind= informer error, no matches for kind "Config" in version "notification.kubesphere.io/v2beta1" F0516 18:37:14.984537 1 server.go:339] unable to register controllers to the manager: no matches for kind "Config" in version "notification.kubesphere.io/v2beta1"

kubesphere controller manager goes into crashlooping status because of this error.
We need a reliable method to disable internal Prometheus so we can run external Prometheus without any issues.

We keep to receiving below report from artifactry.io, pls check if related URL is still valid? otherwise, it will be treated as invalid helm chart and take offline action from KubeSphere app store, thanks. @h0hmj

error scanning image harbor.cloud.netease.com/curve/curvefs:csi-v1.0.0: error running trivy on image harbor.cloud.netease.com/curve/curvefs:csi-v1.0.0: * GET https://harbor.cloud.netease.com/v2/: unexpected status code 403 Forbidden:

• redis
• mongodb
• rabbitmq
• nginx
• tomcat
• postgresql

When changing anything in the values file after the initial installation with helm the cc will not be updated when running helm install/upgrade again. This is because of the if-statement in the template file:

This blocks from managing the installation using helm override file for the configuration.

The KubeSphere v3.2.0 has been released, ks-installer‘s chart should also be updated to v3.2.x

https://github.com/metersphere/helm-chart

Kubesphere uninstall script has to be improved. https://github.com/kubesphere/ks-installer/blob/master/scripts/kubesphere-delete.sh

when I tried to run uninstall on Kubesphere host cluster, it is stuck in Kube-federation-system with terminating status. And I have to manually clean up by removing finalizers.

kubectl get namespaces

NAME STATUS AGE

add-ons Active 17d

apicentral Active 17d

default Active 17d

kube-federation-system Terminating 101m

This does not seem to happen in member cluster

Ideally helm uninstall should be able to remove all the resources deployed in Kubesphere. But it only deletes crd clusterconfiguration. So we have to run this script manually for uninstall.

2022.09.02 06:24:37 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
2022.09.02 06:24:37 INFO app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:39746]
2022.09.02 06:24:37 INFO app[][o.s.a.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch
2022.09.02 06:24:37 INFO app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
warning: no-jdk distributions that do not bundle a JDK are deprecated and will be removed in a future release
2022.09.02 06:24:40 INFO es[][o.e.n.Node] version[7.16.2], pid[31], build[default/tar/2b937c44140b6559905130a8650c64dbd0879cfb/2021-12-18T19:42:46.604893745Z], OS[Linux/3.10.0-957.el7.x86_64/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/11.0.13/11.0.13+8]
2022.09.02 06:24:40 INFO es[][o.e.n.Node] JVM home [/opt/java/openjdk]
2022.09.02 06:24:40 INFO es[][o.e.n.Node] JVM arguments [-XX:+UseG1GC, -Djava.io.tmpdir=/opt/sonarqube/temp, -XX:ErrorFile=../logs/es_hs_err_pid%p.log, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djna.tmpdir=/opt/sonarqube/temp, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=COMPAT, -Des.enforce.bootstrap.checks=true, -Xmx512m, -Xms512m, -XX:MaxDirectMemorySize=256m, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/opt/sonarqube/elasticsearch, -Des.path.conf=/opt/sonarqube/temp/conf/es, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=false]
2022.09.02 06:24:40 INFO es[][o.e.p.PluginsService] loaded module [analysis-common]
2022.09.02 06:24:40 INFO es[][o.e.p.PluginsService] loaded module [lang-painless]
2022.09.02 06:24:40 INFO es[][o.e.p.PluginsService] loaded module [parent-join]
2022.09.02 06:24:40 INFO es[][o.e.p.PluginsService] loaded module [reindex]
2022.09.02 06:24:40 INFO es[][o.e.p.PluginsService] loaded module [transport-netty4]
2022.09.02 06:24:40 INFO es[][o.e.p.PluginsService] no plugins loaded
2022.09.02 06:24:40 INFO es[][o.e.e.NodeEnvironment] using [1] data paths, mounts [[/opt/sonarqube/data (/dev/mapper/centos-root)]], net usable_space [438.9gb], net total_space [449.7gb], types [xfs]
2022.09.02 06:24:40 INFO es[][o.e.e.NodeEnvironment] heap size [512mb], compressed ordinary object pointers [true]
2022.09.02 06:24:41 ERROR es[][o.e.b.ElasticsearchUncaughtExceptionHandler] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: CorruptIndexException[checksum failed (hardware problem?) : expected=1ff992ec actual=38c0565 (resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="/opt/sonarqube/data/es7/nodes/0/_state/_7g.cfe")))];
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112) ~[elasticsearch-cli-7.16.2.jar:7.16.2]
at org.elasticsearch.cli.Command.main(Command.java:77) ~[elasticsearch-cli-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) ~[elasticsearch-7.16.2.jar:7.16.2]
Caused by: org.elasticsearch.ElasticsearchException: failed to bind service
at org.elasticsearch.node.Node.(Node.java:1090) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.node.Node.(Node.java:309) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:234) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) ~[elasticsearch-7.16.2.jar:7.16.2]
... 6 more
Caused by: org.apache.lucene.index.CorruptIndexException: checksum failed (hardware problem?) : expected=1ff992ec actual=38c0565 (resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="/opt/sonarqube/data/es7/nodes/0/_state/_7g.cfe")))
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:419) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:448) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
at org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.readEntries(Lucene50CompoundReader.java:125) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:55]
at org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:68) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:55]
at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:71) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:55]
at org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:101) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
at org.apache.lucene.index.SegmentReader.(SegmentReader.java:83) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:69) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:720) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:84) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:64) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
at org.elasticsearch.gateway.PersistedClusterStateService.nodeMetadata(PersistedClusterStateService.java:290) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.env.NodeEnvironment.loadNodeMetadata(NodeEnvironment.java:459) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:356) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.node.Node.(Node.java:427) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.node.Node.(Node.java:309) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:234) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) ~[elasticsearch-7.16.2.jar:7.16.2]
... 6 more
uncaught exception in thread [main]
ElasticsearchException[failed to bind service]; nested: CorruptIndexException[checksum failed (hardware problem?) : expected=1ff992ec actual=38c0565 (resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="/opt/sonarqube/data/es7/nodes/0/_state/_7g.cfe")))];
Likely root cause: org.apache.lucene.index.CorruptIndexException: checksum failed (hardware problem?) : expected=1ff992ec actual=38c0565 (resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="/opt/sonarqube/data/es7/nodes/0/_state/_7g.cfe")))
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:419)
at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:448)
at org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.readEntries(Lucene50CompoundReader.java:125)
at org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:68)
at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:71)
at org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:101)
at org.apache.lucene.index.SegmentReader.(SegmentReader.java:83)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:69)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:720)
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:84)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:64)
at org.elasticsearch.gateway.PersistedClusterStateService.nodeMetadata(PersistedClusterStateService.java:290)
at org.elasticsearch.env.NodeEnvironment.loadNodeMetadata(NodeEnvironment.java:459)
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:356)
at org.elasticsearch.node.Node.(Node.java:427)
at org.elasticsearch.node.Node.(Node.java:309)
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:234)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
at org.elasticsearch.cli.Command.main(Command.java:77)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)
For complete error details, refer to the log at /opt/sonarqube/logs/sonarqube.log
2022.09.02 06:24:41 WARN app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [es]: 1
2022.09.02 06:24:41 INFO app[][o.s.a.SchedulerImpl] Process[es] is stopped
2022.09.02 06:24:41 INFO app[][o.s.a.SchedulerImpl] SonarQube is stopped

We need to add the basic user guide for nine built-in APPs.

steps:

1. quota has been set in the enterprise space
2. deploy Tomcat application from the App Store ,and setting resource limits when deploying
3. the application deployment failure

Please see

• https://www.freebuf.com/vuls/303951.html [In Chinese] and
• kubesphere/ks-devops#354.

/kind bug
/priority high

Error: Service "redis-ck8cjy" is invalid: spec.clusterIPs[0]: Invalid value: "None": may not be set to 'None' for NodePort services

We need to verify if all charts in qingcloud and experimental meet the Helm technical requirements, it determines which upstream folder (Incubator or stable) we need to submit.

我把chart传到仓库里审核通过，在前台部署，所有pod都已经正常启动，但是应用仍处于创建中状态，最后会失败，所有pod又被删掉，其实服务是已经好了的，只是chart test未通过，求助如何解决此问题

auto build latest chart

Since SkyWalking Chart v3.0.0 is available, it supports Helm 3 and the latest version of SkyWalking, we need to upgrade the built-in SkyWalking chart to v3.0.0.

The current chart in this repo is based on Helm 2.

export this argument in values.yaml so user can config it

As attach

steps:

1. Successfully deploy chaos-mesh app from appstore
2. deploy chaos-mesh app again

problem:
Deployment failure

KubeSphere 3.3.0 has integrated OpenELB in its LoadBalancer configuration list, we need to move the OpenELB helm chart from /test to /stable which makes users could deploy it from KubeSphere App Store.

I find filter rewriteTag in fluentbit document https://github.com/fluent/fluent-operator/blob/master/docs/plugins/fluentbit/clusterfilter/rewritetag.md

But there is not rewriteTag in CRD filters.logging.kubesphere.io.

node-ds.yaml:

      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
      - key: dedicated
        operator: Exists
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
      - effect: NoSchedule
        key: node.cloudprovider.kubernetes.io/uninitialized
        value: "true"

need put them in values.yaml and allow user to change it.

@hlwanghl
I imported the repo to KubeSphere, and I try to deploy GitLab, it seems I need to set an email in the values.yaml?

Could you pls give guideline for this issue?

https://github.com/openelb/openelb/releases/tag/v0.5.1
But we are still using 0.4.4