Comments (17)
This sounds good to me; confd also, conveniently, supports environment variables as a backend.
We should standardize the configuration schema based on the work from @BugRoger 's ceph/config
, I'm guessing.
from ceph-container.
Great! confd should give us a little more flexibility. I'd started with direct consul-template integration but I'll see if I can switch it over to pure confd.
One issue I was dealing with was the initial config. @BugRoger's looks to take a good approach. Any thoughts though on how to handle large numbers of configuration options? We currently use ceph-ansible which has a good set of defaults but I hadn't found an clean way to get a large number of 'default' variables into the KV.
from ceph-container.
I'm not really familiar with confd but can we set a number of values in etcd and then iterate over them to build the ceph.conf? Or is it pure tempting with only static options?
Another solution might be to inject args, however not everything can be injected.
In which value are you interested in?
from ceph-container.
The two options I'd been pondering were bulk insertion into the KV before launch or setting a bunch of defaults as part of the entrypoint script. I think the first option should give a little more flexibility for a larger number of defaults but doesn't make it self contained.
For the initial untested experimentation I'd just added a function for adding defaults in the entrypoint (borrowed from the deis store) - https://github.com/AcalephStorage/ceph-docker/tree/wip-confd
from ceph-container.
@leseb Yes, confd does support iterators, at least with etcd and consul... I think we're out of luck for environment variables.
@hunter I wonder if we can have confd mutate the names of the keys, converting _
into spaces. That would provide a structured but somewhat free-form way of storing configuration values.
from ceph-container.
@Ulexus I was thinking of doing the same thing. I'd started with almost a straight copy/paste of the ceph-ansible ceph.conf but makes more sense to let the KV store provide the structure.
from ceph-container.
Interestingly, etcd does support spaces in keynames just fine, so we may not actually need to adapt it at all. I don't know what potential problems that opens up, though. It does sound a bit dangerous.
etcdctl set /ceph/osd/5/"osd use stale snap" true
etcdctl get /ceph/osd/5/"osd use stale snap" # returns true
works just fine
from ceph-container.
It seems that consul also does something similar (although needs to be url encoded)
curl -f http://localhost:8500/v1/kv/sample%20key
[{"CreateIndex":129368,"ModifyIndex":129368,"LockIndex":0,"Key":"sample key","Flags":0,"Value":"c2FtcGxlIHZhbHVl"}]
from ceph-container.
I'm sure the actual HTTP call has the key URL-encoded in etcd, as well. Can you check that confd works with keys-with-spaces?
from ceph-container.
Yep. Confirmed with confd and consul. I don't have an etcd install handy to test but my guess would be that it works in the same way.
from ceph-container.
Note to self: the deis guys are already using a confd-centered ceph system
from ceph-container.
I haven't had much time to look at any confd related work recently.
Something did jump out at me today though while I was looking at updating crushmaps. Has the basic Monitor KV store been considered for keeping track of config? Sage had mentioned it to me a while ago (http://permalink.gmane.org/gmane.comp.file-systems.ceph.devel/21950) and Calamari uses it for custom crushmap locations.
It may not be suitable for a complex config but perhaps for OSD/host/disk mapping?
from ceph-container.
This looks interesting, I'm just not sure, maybe we want something a bit more generic. However having something autonomous that doesn't require external softwares is also nice to have
from ceph-container.
A quick update since there was mention in #78. We've done a little work on abstracting out consul/etcd KV and using confd - https://github.com/AcalephStorage/ceph-docker/tree/wip-confd
It still needs some testing but should be close to working.
from ceph-container.
@hunter I just had a look at the branch, it looks really good. I'll be glad to see this being 'PRed' at some point. I haven't tried it out, what's missing? What needs further testing?
Thanks for the update.
from ceph-container.
We can probably close this now and open tickets for any specific issues around confd
from ceph-container.
Agreed
from ceph-container.
Related Issues (20)
- /opt/ceph-container/bin/osd_disk_prepare.sh: line 46: ceph-disk: command not found HOT 7
- Need fix for CVE-2022-21797 HOT 4
- Bootstrap process hangs up for hours HOT 2
- not found /var/lib/ceph/osd/ceph-2//keyring HOT 2
- dnf update in ceph v18 container image is failing HOT 2
- RocksDBStore - cannot set permissions: Operation not permitted HOT 2
- /usr/bin/ceph: stderr Error EIO: Module 'cephadm' has experienced an error and cannot handle commands: ContainerInspectInfo HOT 2
- add ceph-mgr-callhome to IBM downstream container HOT 2
- cephadm has failed ContainerInspectInfo HOT 2
- populate_kvstore error HOT 1
- rename and repurpose this repository HOT 19
- reef builds don't work HOT 12
- Question about osd directory HOT 2
- docker-compose setup dose not run as expected mds and osd HOT 3
- With new quay.io/ceph/ceph:v16 image, ceph-csi meet segfault error HOT 2
- ceph/demo container does not expose mon port 3300 HOT 2
- Instructions for getting the zabbix template to work with rook-ceph HOT 2
- smartctl could not scrape metrics from HPE Smart Array in HBA mode HOT 2
- support VERSION=8 for contrib/compose-rhcs.sh
- Include cephfs-shell HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ceph-container.