ksperis / node-rados Goto Github PK
View Code? Open in Web Editor NEWCeph rados client for node.js
License: MIT License
Ceph rados client for node.js
License: MIT License
Hi, I got a docker/ceph cluster and it's "ceph.conf" is strange as below, and seems I can not connect to it through node-rados:
fsid = f9be0dfe-f839-4ebd-9c1b-ccabf96a1bef
mon initial members = mon_0
mon host = 192.168.23.130
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
Any suggestions?
Hi, thanks for you create this and its helpful!
I know there's function to manage pools in python. But seems not support here.
Would you please add manage pool functions?
Hi, I need a list function about specified pool, and I have added this function at my local for test. Could someone could help review it and let it be support function? Thanks in advance~
[at rados.h]
...
class Ioctx : public node::ObjectWrap {
...
static NAN_METHOD(aio_objects_list);
}
[at rados.cc]
...
NAN_METHOD(Ioctx::aio_objects_list) {
NanScope();
if (args.Length() < 1 ||
!args[0]->IsString()) {
return NanThrowError("Bad argument.");
}
Ioctx* obj = ObjectWrap::Unwrap(args.This());
if ( !obj->require_created() ) NanReturnNull();
rados_list_ctx_t h_ctx;
//Start listing objects in a pool.
int err = rados_objects_list_open(obj->ioctx, &h_ctx);
if (err < 0) {
return NanThrowError("open list failed.");
}
Local ret_list = NanNew();
uint32_t array_id = 0;
//Get the next object name and locator in the pool.
while(0 <= err) {
const char *obj_name;
err = rados_objects_list_next(h_ctx, &obj_name, NULL);
if (err == 0) {
ret_list->Set(array_id, NanNew(obj_name));
array_id++;
}
}
rados_objects_list_close(h_ctx);
if (err < 0 && err != -ENOENT) {
return NanThrowError("list_next failed.");
}
NanReturnValue(ret_list);
}
When calling aio_read() with a null size parameter, the module will call rados_stat() to get the object size and set size to the size of the object.
https://github.com/ksperis/node-rados/blob/master/rados.cc#L619-L630
This effectively turn the asynchronous aio_read call to be synchronous (that is the rados_stat() call is synchronous). Instead, the size parameter should be mandatory and an error should be returned, or the size could also default to DEFAULT_BUFFER_SIZE (defined in rados.h).
You can set nan to a nan version that works in package.json or maybe updating the package to use nan 2.0.0.
Here is the error:
npm install rados
/
[email protected] install /home/vagrant/tmp/node_modules/rados
node-gyp rebuild
make: Entering directory /home/vagrant/tmp/node_modules/rados/build' CXX(target) Release/obj.target/rados/module.o In file included from ../module.cc:3:0: ../rados.h:53:5: error: ‘NanCallback’ does not name a type NanCallback callback; ^ make: *** [Release/obj.target/rados/module.o] Error 1 make: Leaving directory
/home/vagrant/tmp/node_modules/rados/build'
gyp ERR! build error
gyp ERR! stack Error: make
failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/local/node/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:269:23)
gyp ERR! stack at ChildProcess.emit (events.js:110:17)
gyp ERR! stack at Process.ChildProcess._handle.onexit (child_process.js:1074:12)
gyp ERR! System Linux 3.13.0-59-generic
gyp ERR! command "node" "/usr/local/node/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/vagrant/tmp/node_modules/rados
gyp ERR! node -v v0.12.7
gyp ERR! node-gyp -v v2.0.1
gyp ERR! not ok
npm ERR! Linux 3.13.0-59-generic
npm ERR! argv "/usr/local/node/bin/node" "/usr/bin/npm" "install" "rados"
npm ERR! node v0.12.7
npm ERR! npm v2.11.3
npm ERR! code ELIFECYCLE
npm ERR! [email protected] install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script 'node-gyp rebuild'.
npm ERR! This is most likely a problem with the rados package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-gyp rebuild
npm ERR! You can get their info via:
npm ERR! npm owner ls rados
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR! /home/vagrant/tmp/npm-debug.log
any idea what may cause the following error?
> cluster = rados.Rados('ceph','client.admin','/etc/ceph/ceph.conf')
nodejs: /usr/include/nodejs/src/node_object_wrap.h:71: void node::ObjectWrap::Wrap(v8::Handle<v8::Object>): Assertion `handle->InternalFieldCount() > 0' failed.
nodejs v0.10.25
I have a user with the following auth entry in ceph:
client.libvirt
key: [secret]
caps: [mon] allow r
caps: [osd] allow class-read object_prefix rbd_children, allow rwx pool=libvirt-pool
Following this guide http://docs.ceph.com/docs/hammer/rbd/libvirt/
here is a sample code, that fails with this user
'use strict'
const rados = require('rados');
console.log('testing with admin account');
let cluster = new rados.Rados('ceph', 'client.admin', '/etc/ceph/ceph.conf');
let err = cluster.connect();
if (err !== 0) {
console.log(`Failed to connect with error ${err}`);
} else {
console.log(`fsid: ${cluster.get_fsid()}`);
}
console.log('testing with libvirt account');
cluster = new rados.Rados('ceph', 'client.libvirt', '/etc/ceph/ceph.libvirt.conf');
err = cluster.connect();
if (err !== 0) {
console.log(`Failed to connect with error ${err}`);
} else {
console.log(`fsid: ${cluster.get_fsid()}`);
}
The output is:
testing with admin account
fsid: a3bc6d05-198a-47c2-b3de-b19eb022e406
testing with libvirt account
Failed to connect with error 95
the same user using the rbd
tool works fine
# rbd -c /etc/ceph/ceph.libvirt.conf -k /etc/ceph/ceph.client.libvirt.keyring --id libvirt ls libvirt-pool
new-libvirt-image
#
ceph.libvirt.conf
mon_host = 192.168.1.32
keyring = ceph.client.libvirt.keyring
Hi @ksperis thank you very much for making this available in node.js! ;)
Would it be possible for you wrap / expose a node.js style createReadStream
and createWriteStream
around io_read()
and io_write()
?
This would make it easier to do something like:
bufferStream.pipe(ioctx.createWriteStream('obj-id'))
clone_range doesn't work in new node versions(+7). I see that function is there but not added to the module. So I added it but it throws error code 2. Also having something like clonedata (http://docs.ceph.com/docs/hammer/man/8/rados/) would be nice.
hello,
I'm interested in adding rados as a dependency to my project, but it's dependency list is significant due to node-gyp, which, as far as I know, isn't required. (see scrypt).
Could you please drop this dependency?
Hello,
Is there any plans to port this to the new node versions? This is the only librados port for node.
Thanks.
I noticed some aio_* functions receive the error code as the parameter to callback. On success, the error code is 0. This means the callback will receive the err parameter as 0. It's fairly common in Node to set the err parameter to null on success. Having the err parameter set to null is also very convenient when using the Node async module.
Instead of returning 0 to the callback on success, could we be returning null?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.