beornf / linstor-docker-volume Goto Github PK
View Code? Open in Web Editor NEWLinstor volume plugin to manage DRBD volumes through Docker
License: GNU General Public License v2.0
Linstor volume plugin to manage DRBD volumes through Docker
License: GNU General Public License v2.0
So far diskless assignments get created "on demand" when needed. Our other plugins also delete diskless assignments when they are not needed anymore. It is not really a requirement for this plugin, but it is a good idea in general where we have like 10 hypervisor nodes where only one at a time accesses the data. There we really do not want to keep track of 9 unused diskless clients.
Hey there,
researching how we can get different docker containers to share the same linstor volume.
Initially, running the first container to use the linstor test volume works properly and the container has proper access to volume.
If we try to run a different container we get the following error.
[root@do2 linstor-docker-volume]# docker run -it -v test:/mnt/shared/ nginx sh
docker: Error response from daemon: error while mounting volume '': VolumeDriver.Mount: unable to get exclusive open on /dev/drbd1003.
ERRO[0000] error waiting for container: context canceled
Seems that this has been added in CSI plugin which unfortunately we can't yet use in Docker Swarm.
piraeusdatastore/linstor-csi#35
piraeusdatastore/linstor-csi@b9372af
piraeusdatastore/linstor-csi@3890953
However, adding changes of b9372aff07e7664608f6080619bcf2f5cf3c8436 to driver.go leads to another error
[root@do2 ~]# docker run -it -v test:/mnt/shared/ nginx sh
docker: Error response from daemon: error while mounting volume '/var/lib/docker/plugins/1d8eed5885c19913f0fcb283ac00efad1e79281d2c802f837a869fdc20c231b2/propagated-mount/test/data': VolumeDriver.Mount: mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t xfs /dev/drbd1003 /var/lib/docker-volumes/linstor/test
Output: mount: /var/lib/docker-volumes/linstor/test: /dev/drbd1003 already mounted on /var/lib/docker-volumes/linstor/test.
Any thoughts on how we could possibly tackle this would be much appreciated.
I'd like to have this plugin - which should become the default for LINBIT - as much of a drop in replacement for the old python based plugin as possible. So I would like it to have the same CLI options as defined here: https://github.com/LINBIT/linstor-docker-volume/blob/master/README
Test environment:
1.ARM64
2.root@node43:/home/horatio# cat /proc/version
Linux version 4.15.0-76-generic (buildd@bos02-arm64-060) (gcc version 7.4.0 (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1)) #86-Ubuntu SMP Fri Jan 17 17:25:58 UTC 2020
Docker plugin install information:
root@node43:/home/horatio# docker plugin install lade/linstor
Plugin "lade/linstor" is requesting the following privileges:
It looks like the LINSTOR socket problem.
Docker plugin list show:
root@node43:/home/horatio# docker plugin list
ID NAME DESCRIPTION ENABLED
21a145a416b2 lade/linstor:latest Linstor volume plugin for Docker false
I try to enable the plugin, it print the following information:
root@node43:/home/horatio# docker plugin enable 21a145a416b2
Error response from daemon: dial unix /run/docker/plugins/21a145a416b2f67311d44578dcf6496f91ec13dcae7c45fd62d4b9410f5b1ea1/linstor.sock: connect: no such file or directory
Configuration:
root@node43:/home/horatio# cat /etc/linstor/docker-volume.conf
[global]
controllers=192.168.1.10
Create volume test:
root@node43:/home/horatio# docker volume create -d lade/linstor:latest --opt fs=xfs --opt size=200M lsvol1
Error response from daemon: create lsvol1: error looking up volume plugin lade/linstor:latest: plugin lade/linstor:latest found but disabled
If anyone need more information,please let me know!
Thanks
In my environment I configured a highly-available LINSTOR controller as per the instructions here: https://linbit.com/drbd-user-guide/linstor-guide-1_0-en/#s-linstor_ha
I had attempted to configure this plugin with multiple, comma-separated, LINSTOR Controllers like so:
[global]
storagepool = lvm-thin
fs = xfs
size = 100MB
replicas = 2
controllers = linstor://192.168.221.60,linstor://192.168.221.61,linstor://192.168.221.62
This would result in the below error unless the controller was running on the first listed address (192.168.221.60).
root@linstor-0:~# docker volume create --opt fs=xfs --opt size=1G -d linbit/linstor-docker-volume test03
Error response from daemon: create test03: VolumeDriver.Create: Post "http://192.168.221.60:3370/v1/resource-definitions": dial tcp 192.168.221.60:3370: connect: connection refused
I was able to work around the issue fairly easily, by configuring drbd-reactor to use a virtual IP address and just set that within the docker-volume plugin configuration. Still, it would be nice if this wasn't necessary and the docker-volume plugin could "round-robin" attempts to different controllers much like the linstor-client can be configured to do.
Consider this a feature request. Thanks a bunch for the work on this plugin!
So far the plugin handles all the LINSTOR resources that it finds. This can be reproduced by manually creating LINSTOR/DRBD resources and then calling the plugin's List()
.
For our other plugins we usually handle this by:
It would be nice to reuse the same aux prop we have in the Python plugin: https://github.com/LINBIT/linstor-docker-volume
I will add some other feature requests. Let's do it like this: If you like to work on a topic, somehow "claim it" so that I know you work on it, otherwise add some "help wanted" tag and I will take it.
As the title says, wanted to ask if it's possible to create a filesystem upon volume creation or mounting?
Now, when creating a volume and try to mount it to a container, I get an error described below,
Output: mount: /var/lib/docker-volumes/linstor/test: wrong fs type, bad option, bad superblock on /dev/drbd1001, missing codepage or helper program, or other error.
Once i run mkfs.xfs /dev/drbd1001
then mounting is getting done properly.
Hi beornf,
I saw this repo because you sent a PR for the linstor-server. This is awesome! I'm the maintainer of the "official" linstor-docker-volume plugin, and I also co-maintain golinstor. So some comments/suggested improvements:
The "old" golinstor uses the linstor-client to do its actions. I saw that you use the "lade.io" repos to get the client. This version is very outdated and I guess it would not even work with a recent linstor controller. The old one used probuf + a custom protocol, recent client and server use a REST-API.
The good news is that golinstor now also provides a REST client. It is pretty "low level", but IMO the way forward (we already transitioned our CSI plugin to it). We might remove/or at least substantially change the high level abstraction of golinstor.
Do you want to work on the plugin together? Maybe we could eventually put it under some "LINBIT umbrella" and make this one the default. For historic reasons the official one started in Python, but I'd be all in to switch it to Go. Does any of that sound interesting? :-D.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.