Comments (9)
Ended up using the following. Setting to null as specified in the example didn't work for us.
Command to get valid userID: oc get project default -o yaml
elasticsearch:
enabled: true
replicas: 2
##################
securityContext:
runAsUser: {value from command}
podSecurityContext:
fsGroup: {value from command}
runAsUser: {value from command}
sysctlInitContainer:
enabled: false
from helm-charts.
Thank you so much for taking the time to test this out and writing up detailed notes. Adding support for other platforms (like Openshift) is planned for the future but we are currently only testing and supporting Google Kubernetes Engine (GKE).
Just a heads up, I have never used Openshift before so this will be my first time trying to figure this stuff out too! I don't currently have access to an Openshift cluster to help debug but in the future I hope to come back to this to get it properly working if you can't figure it out with my comments.
I couldn’t use Helm itself due to permissions issues related to: helm/helm#1918
Is the problem that you don't want to use tiller because of the permissions required? Or is it that you are unable to get it working on Openshift? A quick google makes it sound like helm should work just fine on openshift as long as you install it per namespace. They even have an official blog post about how to install it.
the initContainer tries to run as root.
There is a beta feature in Kubernetes 1.12 to set sysctls natively through Kubernetes. It looks like vm.*
would be considered "unsafe" and need to be allowed on a cluster level. Openshift also seems to support it though it still looks to be the alpha version. I can certainly see having the root init container being optional so that users can disable it and configure the needed sysctl values via annotations instead since I can imagine other users wanting to use this new feature too.
the fsGroup seems to have an invalid value. I am not sure what that should be. Deleting the fsGroup + securityContext does allow statefulset to be created, BUT, again, that’s probably there for a reason (haven’t looked into that in detail yet)
This is to make sure that the persistent volume is mounted with permissions that the Elasticsearch user can read. By default we set this to 1000 because that is the default uid. I quickly went over the Openshift persistent storage docs and it sounds like you need to manually set the fsGroup ID to something that is in your available range. So I think you can find this by running oc get project default -o yaml
but I would need to test it out myself.
the underlying filesystem used for the persistent volumes. My openshift cluster came with both NFS and glusterfs. tried both with same errors, but I do remember recommendations to NOT run elasticsearch on glusterfs a couple of years ago. Perhaps this still holds (and also goes for nfs…?)
Certainly not recommended to use NFS or Glusterfs. As far as I know it will work but it certainly won't perform very well. Another possibility is using local persistent volumes although it still seems to be in alpha on Openshift
from helm-charts.
Hi,
I got elasticsearch to run in openshift (okd v3.11. Only nonpersistent so far) by doing.
oc adm policy add-scc-to-user nonroot -z default
as cluster-admin and the following values (values-jbb.yaml).
persistence:
enabled: false
minimumMasterNodes: 1
antiAffinity: "soft"
podSecurityContext:
fsGroup: 1000
runAsUser: 1000
sysctlInitContainer:
enabled: false
clusterHealthCheckParams: "wait_for_status=green&timeout=30s"
I had to do:
sysctl vm.max_map_count=262144
on all nodes used by ES (could be automated by ansible (privileged init containers is a no-go).
Then:
helm -n jbtest template ./elasticsearch -f values-jb.yaml |oc apply -f -
master# curl elasticsearch-master.fooobar.svc.cluster.local:9200/_cat/nodes
10.131.0.11 8 82 4 0.11 0.09 0.14 dim - elasticsearch-master-1
10.129.0.11 11 96 4 0.12 0.11 0.22 dim - elasticsearch-master-0
10.129.0.12 9 96 4 0.12 0.11 0.22 dim * elasticsearch-master-2
master# curl elasticsearch-master.fooobar.svc.cluster.local:9200/_cat/health
1565958272 12:24:32 elasticsearch green 3 3 0 0 0 0 0 0 - 100.0%
oc get pods
NAME READY STATUS RESTARTS AGE
elasticsearch-master-0 1/1 Running 0 38m
elasticsearch-master-1 1/1 Running 0 38m
elasticsearch-master-2 1/1 Running 0 38m
jbtest-gvqqa-test 0/1 Error 0 38m
Don't know what the test container is for, but elasticsearch seems to work anyway.
from helm-charts.
Thanks for taking the time to test this out and adding in all of the info! It sounds like everything should be workable then assuming there are no weird requirements for persistent volumes in Openshift.
Just adding some comments about things you have changed:
minimumMasterNodes: 1
this is ignored for Elasticsearch 7 so you didn't need ot set this (unless you are using v6)
antiAffinity: "soft" # Is this because you
Did you have to add this because your test environment only has 1 node? Or was it some Openshift related reason?
podSecurityContext:
fsGroup: 1000
runAsUser: 1000
Only runAsUser
is not in the current defaults, we can certainly add it in though!
Assuming the changes you made are specifically for a testing environment it looks like Openshift should be working now by only disabling the sysctlInitContainer
which is expected. I found out recently that https://www.okd.io/minishift/ is a thing which I could use to test this out on.
from helm-charts.
from helm-charts.
I definitely think default uid as non-root should be default yes :-)
I have opened #259 to test that this works properly. If so it can go in as the new default.
Thanks for providing this helm chart by the way :-)
Thank you :)
from helm-charts.
I just added an Openshift example in #263. Could you try it out to see if it works for you? Openshift is still not supported (until we have some kind of automated testing) but this should at least be a good starting point to show what settings need to be tweaked.
from helm-charts.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
from helm-charts.
This issue has been automatically closed because it has not had recent activity since being marked as stale.
from helm-charts.
Related Issues (20)
- [Kibana] 8.5.1"message":"secrets \"kibana-kibana-es-token\" already exists" HOT 4
- Support file realm in the Elasticsearch spec
- readiness probe also always fails when security is disabled HOT 1
- Readiness probe is failing because of bad SSL HOT 1
- Kibana chart helm install failing with Error: INSTALLATION FAILED: failed pre-install: job failed: BackoffLimitExceeded HOT 5
- Logstash does not respect logstashJavaOpts
- Out of memory error both on
- Kibana chart fails when security disabled HOT 1
- Enabling security breaks ES endpoints.
- Kibana post delete can fail, leaving artifact behind. HOT 2
- Cannot enable Metricbeat modules HOT 1
- Elasticsearch 7.17.9 HOT 1
- [kibana] helm deployment with no values fails, kubeadm kubernetes deployment 1.25.3 HOT 1
- elasticsearch chart optimizes the discovery.seed_hosts Settings HOT 1
- How to add license on the helmchart
- Randomly receive list of indices with kubectl exec on single node installation.
- Kibana 8.5.1 Upgrade failed from 7.17.3- Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition
- UNABLE_TO_VERIFY_LEAF_SIGNATURE HOT 5
- [logstash] Statefulset annotations
- kibana cannot be installed, output self signed certificate in certificate chain HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from helm-charts.