Comments (3)
@hookenz I run all my OSD containers by exporting the broad-scoped /var/lib/ceph/osd
directory (and I just updated the OSD documentation to describe this). They are not mounted as tmpfs in any way that I can see (each of the mounts shows as either ext4 or btrfs, on my systems, both inside and outside the OSD containers).
You are certainly correct that Docker does not (yet?) have the ability to change the volume attachments to any existing container, and I haven't actually seen much movement on that limitation.
I do want to change the bootstrap to be more intelligent about recognizing existing OSDs, and I think you've got a good idea, here.
I really like the idea of hot adds and removes of OSDs. That will require some thought, though. A real problem we have right now is the limitation of needing to know the OSD number in advance. One option is to decouple the mountpoint and directory names from the OSD ids, and just mount each OSD under /var/lib/ceph/osd as a random or UUID-based directory name. The script could then check to see if that directory contains an initialized OSD or create a new OSD for it.
I don't think a block device search and arbitrary container-side mounting overall is a "feasible" idea, since it breaks the container separation even more, requiring the container access to the host's device tree and probably privileged mode for mounting.... but I could be swayed in that direction by something compelling.
from ceph-container.
I kind of agree with you on the last point.
In the mean time, I've discovered a hot-add mechanism which may come in handy. Attach a volume to a container while it is running
Now I've written a script that can scan for block devices that are initialized with ceph-disk prepare.
If prepared that way, then the disks will be given the part type id 4fbd7e29-9d25-41b8-afd0-062c0ceff05d.
My script will mount accordingly. However, it mounts based on whoami file. As the filesystem isn't created yet, they don't mount.
So my solution is to create create another script which will wraps something along the lines of "docker run --privileged=true -v /etc/ceph:/etc/ceph -v /dev:/dev ceph/base ceph-disk prepare ..."
Then mount it by ID and add 'echo $id > /osd/id'
When ceph/osd docker image starts it'll run the ceph-osd --mkfs and --mkjournal steps to complete the process as normal.
I'm going to some more scripts to make this more seamless and help to get it running.
from ceph-container.
Closing this old issue; I believe this is archaic after the ceph-disk
stuff
from ceph-container.
Related Issues (20)
- I want ceph-container to support packaging private images HOT 2
- ceph uses normal users for ssh connections HOT 3
- Ceph container build failed with centos stream8 image HOT 3
- /opt/ceph-container/bin/osd_disk_prepare.sh: line 46: ceph-disk: command not found HOT 7
- Need fix for CVE-2022-21797 HOT 4
- Bootstrap process hangs up for hours HOT 2
- not found /var/lib/ceph/osd/ceph-2//keyring HOT 2
- dnf update in ceph v18 container image is failing HOT 2
- RocksDBStore - cannot set permissions: Operation not permitted HOT 2
- /usr/bin/ceph: stderr Error EIO: Module 'cephadm' has experienced an error and cannot handle commands: ContainerInspectInfo HOT 2
- add ceph-mgr-callhome to IBM downstream container HOT 2
- cephadm has failed ContainerInspectInfo HOT 2
- populate_kvstore error HOT 1
- rename and repurpose this repository HOT 19
- reef builds don't work HOT 12
- Question about osd directory HOT 2
- docker-compose setup dose not run as expected mds and osd HOT 3
- With new quay.io/ceph/ceph:v16 image, ceph-csi meet segfault error HOT 2
- ceph/demo container does not expose mon port 3300 HOT 2
- Instructions for getting the zabbix template to work with rook-ceph HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ceph-container.