node-influx / node-influx Goto Github PK

View Code? Open in Web Editor NEW

863.0 21.0 178.0 4.31 MB

📈 The InfluxDB Client for Node.js and Browsers

Home Page: https://node-influx.github.io/

License: MIT License

JavaScript 6.82% TypeScript 93.18%

onwards influx influxdb-js-client timeseries javascript influxdb typescript hacktoberfest

node-influx's Introduction

InfluxDB v1.x JavaScript library

Node and browser library for InfluxDB v1.x and older. For v2, please use the official client.

Installation

For Node, simply:

$ npm install --save influx

For browsers, see the browser setup instructions.

Features

Simple API for nearly all Influx operations
Fully supported in Node and the browser
Performant, processing millions of rows/sec on modest hardware
Zero dependencies

Compatibility

Version 5.x.x is compatible with InfluxDB 1.x on Node 4 on onwards, and modern browsers

Version 4.x.x is compatible with InfluxDB 0.9.x - 0.13.x on Node 0.12 and onwards

Version 3.x.x is compatible with InfluxDB 0.8.x - 3.x will no longer have updates by core contributers, please consider upgrading.

Usage

node-influx's People

Contributors

Stargazers

Watchers

Forkers

nvdnkpr nichdiekuh bloglovin yiwang blacklightops lafikl amicus kenjitakahashi nstott dmcclure nmorsman pressla stariqmi ria-com spotistic omarahm luddd3 muzzley janza airtoxin rvennam mindnuts phagunbaya jmptrader cocophotos cainus mmalecki qiminglu hyusetiawan eliperelman gobwas leorossi rstets epignotti thefrontiergroup xapss joel-airspring pygmalios mulyoved selamanse tomhallam mfrye mcanthony rodrigoapereira kevinthorley hans-d samueltoh germanbluefox lpatiny behrad thinkdevcode mixer asseldonk robertsmedshammar tigerqiu712 sierrasoftworks jayengee venkates77 gitter-badger bovan hao8816 anhldbk andreylana evalok shaunwarman lapetitesouris lab11 heap-legacy strongpauly sickoscar josh-shaw-dev simison apollon77 roberttheprofessional yamadayuki farhan711 inetfuture apostopher togobr peaksnail creative-area therealliviux salehalbuga damoclark surajjha ashokaditya philjestin adrianliaw marshallrj admiralcoco michaelkovalchuk sidhantpanda kinnza endrit-b esco flafik flood-io avimar abdulajet rubycut

node-influx's Issues

getMeasurements Values from the array

Hello,

Was trying to fetch the values from the getMeasurements Function but i see undefined when i try retrieving them

client.getMeasurements(function(err,arrayMeasurements){
            console.log(arrayMeasurements[0].series)
            console.log(arrayMeasurements[0].series.values)
            console.log(arrayMeasurements[0].series.values.length)

            } )

Output

debug: [ { name: 'measurements',
columns: [ 'name' ],
values: [ [Object] ] } ]
debug: [Functions:values]
debug: 0

Db Values

name: server_info
tags: hostname=ip-10-20-30-00.ec2.internal, metric=memory, serverip=xx.xx.xx.xx
time                            value
----                            -----
2015-07-29T10:26:22.225Z        2101297152
2015-07-29T10:36:00.502Z        2101297152

getSeriesNames without databaseName if already defined on connection

Hi @bencevans

I think it would be a better design if databaseName is optional if i already defined it on connection, unless i overwrite it it'll use what i defined on connection.

I'l' submit a patch for this feature to explain what i mean better, Code is better than ranting :P

Strange problem in many write point calls

I'm running many writePoint calls with node-influx and many cases InfluxDB returns 500 and timeout. But, i try execute same query with request and InfluxDB returns successfully.

It seems to be something related with pool option in request.

Test Case: https://gist.github.com/rodrigoapereira/9a74cb1275d8606fdfcf
Can anyone help me?

Node Version: v0.10.37
Request Version: [email protected]
Node-Influx Version: [email protected]
InfluxDB Version: 0.9.4.2

Chunk writePoints and writeSeries data if it's too big

Right now it looks like there is a limit of 2MB on the amount of data one can send per write request. It would be nice if there was a flag to chunk the data so node-influx would end up sending multiple requests under 2MB if it had to.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Database undefined for Writepoint

Code

var influx = require('influx');
    client = influx({
         host:'localhost',
         port: 8086,
         database:'metrics'
    });

var points = [
                [{value:data.num_cores},{tag:'cores',tag:nodes.serverip,tag:hostname}],
                [{value:data.memory_capacity},{tag:'memory',tag:nodes.serverip,tag:hostname}],
            ]

            client.writePoint('metrics',points,function(err,dat){
                console.log(dat);
            })

Output

/opt/dockerstack-dashboard/node_modules/influx/index.js:304
if (!options.database) {
          ^
TypeError: Cannot read property 'database' of undefined
at InfluxDB.writeSeries (/opt/dockerstack-dashboard/node_modules/influx/index.js:304:15)
at InfluxDB.writePoint (/opt/dockerstack-dashboard/node_modules/influx/index.js:326:8)
at /opt/dockerstack-dashboard/api/controllers/ClusterController.js:166:24
at /opt/dockerstack-dashboard/node_modules/cadvisor/index.js:23:6
at ConcatStream.<anonymous> (/opt/dockerstack-dashboard/node_modules/concat-stream/index.js:36:43)
at ConcatStream.emit (events.js:129:20)
at finishMaybe (/opt/dockerstack-dashboard/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:460:14)
at afterWrite (/opt/dockerstack-dashboard/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:340:3)
at /opt/dockerstack-dashboard/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:327:9
at process._tickDomainCallback (node.js:381:11)

Tests for Influx 0.5 (RC1)

I think we should upgrade the Travis tests, influx 0.5 was released yesterday. Although the API remained the same, I guess it would make sence to test this module against 0.5 :-)

Reverse-Proxy Path Prefix

I keep my influxdb in a reverse-proxy'd subdirectory of my domain. This way, i can use its SSL Certificate without the need of one more certificate.

For this reason, my API does not look like this:

http://host/db/<dbname>...

but like this:

http://host/influx/db/<dbname>...
               ^-- this part is additional

Right now, i think, this is not supported. At least, i can't see, how.

WritePoint and WritePoints do not work with retention policies

If I use writepoint() in the following way

db.writePoints('mydb.my_retention_policy.temperature_1d', points, function(err, response) {

});

Expected:
Insert the points into 'temperature_1d' within 'mydb' using the retention policy 'my_retention_policy'.

Actual:
It creates a new measurement with the name 'mydb.my_retention_policy.temperature_1d' and inserts the points into this new measuremnt with the default retention policy.

select query builder

I'm thinking about writing a select-query-builder. The function would receive an object defining which fields to select from the series and how to aggregate them.

Although the influx SQL syntax is pretty straight forward, I would find it more useful to define my dynamically built queries in json notation. I imagine it to be similar to the mongo query syntax.

e.g.

SELECT MEDIAN(my_col), PERCENTILE(my_col,95) FROM my_series WHERE time > NOW()-2h GROUP BY time(1m);

could be become something like this:

var query = {
  select : [ {'median' : 'my_col'}, {'percentile' : ['my_col',95]}],
  where : [{ time : { 'gt' : 'now()-2h'}]
  group : [{time: '1m'}}
};
dbInflux.query('my_series',query,callback()}

This syntax is still far from perfect, but I hope you get the idea.

dbinflux.query() could be tweaked to accept either a raw SQL string or an object.

If it's not suitable for this module, it could also become a separate module which sits ontop of node-influx.

What do you think?

0 type is int and needs float

my script is writing float data to influx, but 0 values are always interpreted as an int, causing an error. whats going on here?

influx.writePoint('sites', 0.0, null, function(err, response) {
  if(err) { console.log(err) }
})

[Error: write failed: field type conflict: input field "value" on measurement "sites" is type int64, already exists as type float
]

unable to parse : invalid boolean

Hi,

During client.writePoint, field parameters did not insert double quotes surrounding string value, therefore, it was trying to parse the value as boolean.

eg:

/*
it will try to parse it as `activity, user=5 action=running`, which on InfluxDB version 0.9.1, will try to parse running value as boolean
*/
client.writePoint('activity', { action: 'running' }, {  user: 5 }, function(err, response){
 // handle response
});

is this expected behaviour? if no, i can submit a pull request.

Thanks

How to update tags of existing data points

I've been trying to update some existing data points using this API. The problem is that I always end up inserting new ones instead, probably because I'm unable to pass the timestamp correctly. What I'm doing is essentially

    var query="..."
    client.query(query, (err, data) ->
      data[0].forEach((event) ->
        client.writePoint(seriesName, { value: event.value, time: new Date(event.time) }, { testtag1: "boom" })
      )
    )

I also tried to pass the time from the query result event to writePoint but it doesn't seem to work either: I get unable to parse 'temperature,testtag3=boom value=12.7 2015-10-15T19:51:51.140565896Z': bad timestamp

Any idea how to make this work? Maybe writePoint should also accept the string value that I get from the query?

Error: 404 page not found in `err`

var client = require("influx")({
    host : 'localhost',
    username : 'root',
    password : 'root',
    database : 'test',
    port: 8086
});

client.writePoint("availability", [
    {
        time: new Date(),
        value: "1"
    },
    {
        time: new Date(),
        value: "2"
    }
], function(err, result) {
    console.log(err, result);
});

Gives [Error: 404 page not found] undefined on the command line

Using 8088 gives a socket hangup error.

Preserve default influxDB ordering

InfluxDB orders results descending, (as in queryRaw) however when parsingResults in query .sort changes default ordering to Ascending.
I propose disabling .sort by default, and/or defining a new query option for caller to set the ordering

Add https support

Should be able to communicate with an influxdb instance that requires ssl.

Support "Other options when querying data"

Add support to specify other options when querying data. Add an options object to query/rawQuery functions which urlencodes the data and append to request's querystring.

Suggested API...

client.query([database], query, [options], callback)

 database - Optional String
 query - String
 options - Object - declaire timestampFormat/chunkSize
           .timestampFormat = String (`epoch=[h,m,s,ms,u,ns]`)
           .chunkSize = Integer (`epoch=[h,m,s,ms,u,ns]`)
           .raw - Object
                 .* - adds to querystring

associated issues

39 Handle chunked query responses (epoch)

curl -G 'http://localhost:8086/query' --data-urlencode "db=mydb" --data-urlencode "epoch=s" --data-urlencode "q=SELECT value FROM cpu_load_short WHERE region='us-west'"
111 Query different timestamp formats (chunk_size)

curl -G 'http://localhost:8086/query' --data-urlencode "db=deluge" --data-urlencode "chunk_size=20000" --data-urlencode "q=SELECT * FROM liters"

docs

Add function for shard space configuration

We should add a function to configure the shard spaced introduced in the latest 0.8 release candidate.

QueryRaw a not existing database will crash node.

I made a new influxClient, to a database which does not exist. When I use the queryRaw function, it gives the following exception.

Somehow the body object is undefined. This results my node to crash, and not a simple way to catch it.

TypeError: Cannot read property 'results' of undefined at C:\Server\NodeJS\node_modules\influx\index.js:77:31
at InfluxRequest._parseCallback (C:\Server\NodeJS\node_modules\influx\lib\InfluxRequest.js:118:10)
at Request._callback (C:\Server\NodeJS\node_modules\influx\lib\InfluxRequest.js:107:10)
at Request.self.callback (C:\Server\NodeJS\node_modules\influx\node_modules\request\request.js:198:22)
at Request.emit (events.js:110:17)
at Request.<anonymous> (C:\Server\NodeJS\node_modules\influx\node_modules\request\request.js:1057:14)   
at Request.emit (events.js:129:20)
at IncomingMessage.<anonymous> (C:\Server\NodeJS\node_modules\influx\node_modules\request\request.js:1003:12)   
at IncomingMessage.emit (events.js:129:20)
at _stream_readable.js:908:16

Node v0.12.6 - Node-influx v4.0.1

Query different timestamp formats

I can't find any way of doing this with node-influx. Is it possible to implement it in the query and queryRaw methods?

Timestamp Format

Everything in InfluxDB is stored and reported in UTC. By default, timestamps are returned in RFC3339 UTC and have nanosecond precision, for example 2015-08-04T19:05:14.318570484Z. If you want timestamps in Unix epoch format include in your request the query string parameter epoch where epoch=[h,m,s,ms,u,ns]. For example, get epoch in seconds with:
curl -G 'http://localhost:8086/query' --data-urlencode "db=mydb" --data-urlencode "epoch=s" --data-urlencode "q=SELECT value FROM cpu_load_short WHERE region='us-west'"

https://influxdb.com/docs/v0.9/guides/querying_data.html#timestamp-format

query.time_precision not honored in writeSeries

It actually looks like all options.query is never used when the request is made to the server.

Add CONTRIBUTING.md and guidelines on how to peer review pull-requests etc.

If anyone's got suggestions on how to peer review pull-requests for such a project and any other suggestions which may help us move this project forward as in organisation and structure please do share!

Unfortunately I'm unable to dedicate as much time to this project as I'd like and believe @nichdiekuh too is in the same position. If the community were to peer review and comment +1's (although may be confused with people just +1'ing for the feature) on pull-requests, once others have reviewed it we could merge and get it published a lot quicker.

I've done a quick search and found these to be quite interesting but would love some community feedback too!

Fix for changed "list series" response

ce1f5fa#commitcomment-7252080

request Pool

I think you should increase the pool of requests, solve many operating request

Spaces or commas in the key are not escaped

According to https://github.com/influxdb/influxdb/blob/master/tsdb/README.md#key tag values must escape any spaces or commas using a backslash.

However they are not in the _createKeyTagString method

Changelog 1.x -> 2.x and Release

Include series name in multi-series query

For queries that span multiple series, I would like the results to include the series name. I'm not sure if there is a way to do this that exists today. Based on reading over _parseResults, this doesn't seem to be the case. I am able to easily modify _parseResults to achieve this result and would be happy to submit a PR if this is a reasonable request.

I am executing a query similar to the following:
SELECT LAST(value) FROM /.*/ WHERE sensorGroupId = 'some_id' GROUP BY sensorId

The results that are returned via node-influx are formatted as follows:

[ [ { time: '1970-01-01T00:00:00Z',
      last: 16,
      sensorId: 'a485a016-de0d-48bf-94ef-26063236f12a' },
    { time: '1970-01-01T00:00:00Z',
      last: 6,
      sensorId: 'e445de61-0238-484d-8a1f-5e3b88fc9d2b' },
....

As you can see, there is no series name returned as part of the results.

The results returned from the influxdb command line are as follows:

> SELECT LAST(value) FROM /.*/ WHERE sensorGroupId='some_id' group by sensorId
name: sensor.typeA.subtypeA
tags: sensorId=123abc
time            last
----            ----
1970-01-01T00:00:00Z    47

name: sensor.typeA.subtypeB
tags: sensorId=456def
time            last
----            ----
1970-01-01T00:00:00Z    46
............

I would like to see the following returned from influx-node:
The results that are returned via node-influx are formatted as follows:

[ [ { time: '1970-01-01T00:00:00Z',
      last: 16,
      sensorId: '123abc',
      name: 'sensor.typeA.subtypeA' },
    { time: '1970-01-01T00:00:00Z',
      last: 6,
      sensorId: '456def',
      name: 'sensor.typeA.subtypeA' },
.................

Select type of field

Hi!

First, thanks to all contributors for developing this great adapter.

I'm a bit confused about field types in InfluxDB 0.9.x:
Type of a given field is determined dynamically according to the type of the first value inserted.
Let's say I have a set of floats that can possibly have no decimal part (for example 1.0, that is 1 in javascript) and I want to store them. Depending on whether the first value I insert has a decimal part or not, the field's type will be Float or Integer, and only that type will be allowed for the next writes.
Using toFixed to have only floats will generate a string and force the field to have String type, which is not what I want.

Am I missing something? How would you enable storing any javascript Number instance in a given field of type Float?

Cheers

Batch write multiple points

Influx supports writing multiple points at once and we would probably get much better performance with such batching.

I think the easiest is to rewrite writePoint so that it can take an array of points. The code should not be much different.

Fix tests on Travis

Design a logo or identifier for the project

Test coverage for getSeriesNames

https://coveralls.io/files/127029733#L77

Writing strings or JSON Objects with the v0.9 client

Hi,

I did some testing with the new client and it appears that the writing of strings and JSON objects is pretty unintuitive. If I want to write a string, right now I have to write it as follows:
client.writePoint('seriesName', {value: ' "myString" '}, 'tagName: tag', cb);

If I want to write a JSON object, e.g.:
var myObject = {value: 'myString', key: 'value', key2: 'value2'};
client.writePoint('seriesName', myObject, 'tagName: tag', cb);
In this case I would have to manually add the " " to all of my strings inside the object. Now I would say that our client needs to add these " " automatically in case a value inside the second parameter is a string.

I think most users want to write a string as follows:
client.writePoint('seriesName', {value: 'myString'}, 'tagName: tag', cb);

I wanted to hear what you think about this, if you agree with me then I'll make a pull request. I was already able to fix this issue locally.

rename readPoints to query()

I suggest to rename readPoints() to query(), since you can use it for any kind of query - even DELETEs.

To maintain backwards compatibility, I'd keep a shorthand from readPoints() to query().

I'll write a fix if you agree @bencevans :)

writePoint callback(err, response) return 'null', 'undefined'

The callback from writePoint method return err=null and response=undefined

Single host in cluster configuration not working

I get an error from the API when I specify a single host object in the hosts array. node-influx uses a single host correctly when I use the standard flat configuration.

This Works

client = influx({
    host : 'localhost',
    port : 8086,
    protocol : 'http',
    username: 'root',
    password: 'root',
    database: 'site_dev'
});

client.getDatabaseNames(function(err, names) {
    if (err) {
        console.log(err);
    }
    console.log(names); // correctly returns my databases
});

This Does Not Work

client = influx({
    hosts: [{
        host: 'localhost',
        port: 8060,
        protocol: 'http'
    }],
    username: 'root',
    password: 'root',
    database: 'site_dev'
});

client.getDatabaseNames(function(err, names) {
    if (err) {
        console.log(err); // [Error: No host available]
    }
    console.log(names);
});

The first example works fine. The second responds with an error: [Error: No host available]

Response not sorted correct when running query()

Hi!

When I run a query(), I sometimes get the result not sorted by time in a correct way. The same query run through queryRaw() gives a correct result.

Example:

SELECT value FROM "metric.parameter1" where tenant = '1' AND asset='54a6606b25ff5606489c7e3e' AND source='hp' AND time > now() - 1d

Using query() the result is:

[[{
      "time": "2015-09-21T00:50:18.252Z",          <<< Wrong date, should probably be further down
      "value": 1
    },
    {
      "time": "2015-09-20T11:30:53.385Z",
      "value": 1
    },
    {
      "time": "2015-09-20T18:26:22.224Z",
      "value": 1
    },
    {
      "time": "2015-09-20T18:33:32.441Z",
      "value": 0
    },
    {
      "time": "2015-09-20T19:36:42.751Z",
      "value": 1
    },
    {
      "time": "2015-09-20T19:53:24.451Z",
      "value": 0
    },
    {
      "time": "2015-09-20T20:04:08.36Z",
      "value": 1
    },
    {
      "time": "2015-09-20T20:12:05.709Z",
      "value": 0
    },
    {
      "time": "2015-09-20T21:24:25.998Z",
      "value": 1
    },
    {
      "time": "2015-09-20T21:32:23.409Z",
      "value": 0
    },
    {
      "time": "2015-09-20T23:12:30.602Z",
      "value": 1
    },
    {
      "time": "2015-09-20T23:20:51.46Z",
      "value": 0
    },
    {
      "time": "2015-09-21T00:01:12.495Z",     <<< ???
      "value": 0
    },
    {
      "time": "2015-09-20T11:38:26.985Z",     <<< ???
      "value": 0
    },
    {
      "time": "2015-09-21T00:59:02.732Z",
      "value": 0
    },
    {
      "time": "2015-09-21T02:17:19.864Z",
      "value": 1
    },

....
]]

Using queryRaw() on the same query gives a correct result:

[
  {
    "series": [
      {
        "name": "metric.1A01",
        "columns": [
          "time",
          "value"
        ],
        "values": [
          [
            "2015-09-20T11:30:53.385Z",
            1
          ],
          [
            "2015-09-20T11:38:26.985Z",
            0
          ],
          [
            "2015-09-20T18:26:22.224Z",
            1
          ],
          [
            "2015-09-20T18:33:32.441Z",
            0
          ],
          [
            "2015-09-20T19:36:42.751Z",
            1
          ],
          [
            "2015-09-20T19:53:24.451Z",
            0
          ],
          [
            "2015-09-20T20:04:08.36Z",
            1
          ],
          [
            "2015-09-20T20:12:05.709Z",
            0
          ],
          [
            "2015-09-20T21:24:25.998Z",
            1
          ],
          [
            "2015-09-20T21:32:23.409Z",
            0
          ],
          [
            "2015-09-20T23:12:30.602Z",
            1
          ],
          [
            "2015-09-20T23:20:51.46Z",
            0
          ],
          [
            "2015-09-21T00:01:12.495Z",
            0
          ],
          [
            "2015-09-21T00:50:18.252Z",
            1
          ],
          [
            "2015-09-21T00:59:02.732Z",
            0
          ],
          [
            "2015-09-21T02:17:19.864Z",
            1
          ],
          [
            "2015-09-21T02:26:28.743Z",
            0
          ],
          [
            "2015-09-21T03:42:21.54Z",
            1
          ],
          [
            "2015-09-21T03:51:30.92Z",
            0
          ],
          [
            "2015-09-21T04:59:04.098Z",
            1
          ],
          [
            "2015-09-21T05:08:36.762Z",
            0
          ],
          [
            "2015-09-21T06:14:11.469Z",
            1
          ],
          [
            "2015-09-21T06:23:43.009Z",
            0
          ],
          [
            "2015-09-21T07:34:03.85Z",
            1
          ],
          [
            "2015-09-21T07:43:36.975Z",
            0
          ],
          [
            "2015-09-21T09:00:41.887Z",
            1
          ],
          [
            "2015-09-21T09:09:51.244Z",
            0
          ]
        ]
      }
    ]
  }
]

Regards
Lars-Ake

Escape series names

Hello! Can you add method for escaping series names? For example, for, that series contains " )

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

The callback in writeSeries should be optional

Hi,

Take the following function:

InfluxDB.prototype.writeSeries = function(series, options, callback) {..}

I should be able to call it like this:

dbInflux.writeSeries([cpuSeries, eventSeries]);

but then I do, I get the following stack trace:

/Users/me/Desktop/influxdb/node_modules/influx/index.js:159
  var query = options.query || {};
                     ^
TypeError: Cannot read property 'query' of undefined
    at InfluxDB.writeSeries (/Users/me/Desktop/influxdb/node_modules/influx/index.js:159:22)
    at Object.<anonymous> (/Users/me/Desktop/influxdb/app.js:54:10)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:906:3

The problem seems to be that writeSeries expects the callback (index.js:154):

if(typeof options === 'function') {
    callback = options;
    options  = {};
}

Unable to run the 'express response times' example

Hello,

When I run the example app 'express response times', I see the following error:

Ipanema:influxdb tito$ node server.js 

/Users/tito/Desktop/influxdb/node_modules/influx/index.js:29
  if ( !_.isArray(options.hosts) && 'string' === typeof options.host)
                         ^
TypeError: Cannot read property 'hosts' of undefined
    at new InfluxDB (/Users/tito/Desktop/influxdb/node_modules/influx/index.js:29:26)
    at Client (/Users/tito/Desktop/influxdb/node_modules/influx/index.js:279:46)
    at createClient (/Users/tito/Desktop/influxdb/node_modules/influx/index.js:281:10)
    at Object.<anonymous> (/Users/tito/Desktop/influxdb/server.js:12:20)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)

Thanks!

Line protocol

Should really use the line protocol for the newer versions of influx.
influxdata/influxdb#3607

I'm trying to send huge amounts of data very frequently using this library and it just seems to choke no matter how I batch it.

It works fine at lower levels of use, but as soon as I deploy it to production it chokes.

README Functions

I think it would make more sense to put the text e.g

Creates a new database - requires cluster admin privileges

above the code

createDatabase(databaseName, callback) { }

rather than below and in the context of the user doing the action so it would be...

Creates a new database - requires cluster admin privileges

createDatabase(databaseName, callback) { }

InfluxDB crash/bug report

I left InfluxDB running overnight collecting lots of statistics. The next morning it was stuck in the following crash loop. It does not look like the InfluxDB process ever quit, it just repeatedly threw http panics, rendering it inaccessible. Upon restarting InfluxDB, it complained about not having enough open file descriptors (Even though our limit was set to 1000 max open file descriptors). Once we increased that limit to 2000 and restarted InfluxDB again, it is working,

Having infinitely repeating internal crashes that render InfluxDB unusable is not desirable because it is hard to tell if/what is going wrong.

2014/07/30 10:43:26 http: panic serving 172.18.1.42:39119: write /opt/influxdb/shared/data/wal/log.36560005: bad file descriptor
goroutine 113160540 [running]:
net/http.func·009()
        /home/vagrant/bin/go/src/pkg/net/http/server.go:1093 +0xae
runtime.panic(0x87abc0, 0xc21478f660)
        /home/vagrant/bin/go/src/pkg/runtime/panic.c:248 +0x106
wal.(*WAL).AssignSequenceNumbersAndLog(0xc210071e00, 0xc2145d0680, 0x7f4074dcae48, 0xc21003cc40, 0x906700, ...)
        /home/vagrant/influxdb/src/wal/wal.go:404 +0x10a
cluster.(*ShardData).Write(0xc21003cc40, 0xc2145d0680, 0xc3e3651f60, 0x5c00000000000000)
        /home/vagrant/influxdb/src/cluster/shard.go:199 +0x96
coordinator.(*CoordinatorImpl).write(0xc2100fdc80, 0xc214846b26, 0xa, 0xc2145d0580, 0xf, ...)
        /home/vagrant/influxdb/src/coordinator/coordinator.go:726 +0x650
coordinator.(*CoordinatorImpl).CommitSeriesData(0xc2100fdc80, 0xc214846b26, 0xa, 0xc2145d0500, 0xf, ...)
        /home/vagrant/influxdb/src/coordinator/coordinator.go:691 +0x955
coordinator.(*CoordinatorImpl).WriteSeriesData(0xc2100fdc80, 0x7f4074db5650, 0xc210185a00, 0xc214846b26, 0xa, ...)
        /home/vagrant/influxdb/src/coordinator/coordinator.go:488 +0x277
api/http.func·005(0x7f4074db5650, 0xc210185a00, 0xc214846b33, 0x4, 0xc214846b3a)
        /home/vagrant/influxdb/src/api/http/api.go:371 +0x334
api/http.yieldUser(0x7f4074db5650, 0xc210185a00, 0x7f40660c9c18, 0xc214846b3a, 0x4, ...)
        /home/vagrant/influxdb/src/api/http/api.go:477 +0x3b
api/http.(*HttpServer).tryAsClusterAdmin(0xc2100c25a0, 0x7f4074db15c8, 0xc3e3a1c820, 0xc21058c680, 0x7f40660c9c18)
        /home/vagrant/influxdb/src/api/http/api.go:539 +0x34a
api/http.(*HttpServer).tryAsDbUserAndClusterAdmin(0xc2100c25a0, 0x7f4074db15c8, 0xc3e3a1c820, 0xc21058c680, 0x7f40660c9c18)
        /home/vagrant/influxdb/src/api/http/api.go:709 +0x268
api/http.(*HttpServer).writePoints(0xc2100c25a0, 0x7f4074db15c8, 0xc3e3a1c820, 0xc21058c680)
        /home/vagrant/influxdb/src/api/http/api.go:378 +0x1e5
api/http.*HttpServer.(api/http.writePoints)·fm(0x7f4074db15c8, 0xc3e3a1c820, 0xc21058c680)
        /home/vagrant/influxdb/src/api/http/api.go:106 +0x44
api/http.func·032(0x7f4074db15c8, 0xc3e3a1c820, 0xc21058c680)
        /home/vagrant/influxdb/src/api/http/cors.go:14 +0x1d0
net/http.HandlerFunc.ServeHTTP(0xc2101e33a0, 0x7f4074db15c8, 0xc3e3a1c820, 0xc21058c680)
        /home/vagrant/bin/go/src/pkg/net/http/server.go:1220 +0x40
github.com/bmizerany/pat.(*PatternServeMux).ServeHTTP(0xc21023ca20, 0x7f4074db15c8, 0xc3e3a1c820, 0xc21058c680)
/home/vagrant/influxdb/src/github.com/bmizerany/pat/mux.go:109 +0x1be
net/http.serverHandler.ServeHTTP(0xc210152640, 0x7f4074db15c8, 0xc3e3a1c820, 0xc21058c680)
        /home/vagrant/bin/go/src/pkg/net/http/server.go:1597 +0x16e
net/http.(*conn).serve(0xc210a45c80)
        /home/vagrant/bin/go/src/pkg/net/http/server.go:1167 +0x7b7
created by net/http.(*Server).Serve
        /home/vagrant/bin/go/src/pkg/net/http/server.go:1644 +0x28b
2014/07/30 10:43:30 http: panic serving 172.18.1.44:35200: write /opt/influxdb/shared/data/wal/log.36560005: bad file descriptor
goroutine 113160715 [running]:
net/http.func·009()
        /home/vagrant/bin/go/src/pkg/net/http/server.go:1093 +0xae
runtime.panic(0x87abc0, 0xc2b8ecd750)
        /home/vagrant/bin/go/src/pkg/runtime/panic.c:248 +0x106
wal.(*WAL).AssignSequenceNumbersAndLog(0xc210071e00, 0xc210d44f00, 0x7f4074dcae48, 0xc21003cc40, 0x906700, ...)
        /home/vagrant/influxdb/src/wal/wal.go:404 +0x10a
cluster.(*ShardData).Write(0xc21003cc40, 0xc210d44f00, 0xc2146216a0, 0xff00000000000000)
        /home/vagrant/influxdb/src/cluster/shard.go:199 +0x96
coordinator.(*CoordinatorImpl).write(0xc2100fdc80, 0xc2b90185c6, 0xa, 0xc210d44e80, 0xf, ...)
        /home/vagrant/influxdb/src/coordinator/coordinator.go:726 +0x650
coordinator.(*CoordinatorImpl).CommitSeriesData(0xc2100fdc80, 0xc2b90185c6, 0xa, 0xc210d44e00, 0xf, ...)
        /home/vagrant/influxdb/src/coordinator/coordinator.go:691 +0x955
coordinator.(*CoordinatorImpl).WriteSeriesData(0xc2100fdc80, 0x7f4074db5650, 0xc210185a00, 0xc2b90185c6, 0xa, ...)
        /home/vagrant/influxdb/src/coordinator/coordinator.go:488 +0x277
api/http.func·005(0x7f4074db5650, 0xc210185a00, 0xc2b90185d3, 0x4, 0xc2b90185da)
        /home/vagrant/influxdb/src/api/http/api.go:371 +0x334
api/http.yieldUser(0x7f4074db5650, 0xc210185a00, 0x7f4067777c18, 0xc2b90185da, 0x4, ...)
        /home/vagrant/influxdb/src/api/http/api.go:477 +0x3b
api/http.(*HttpServer).tryAsClusterAdmin(0xc2100c25a0, 0x7f4074db15c8, 0xc2b8fa1320, 0xc422315a90, 0x7f4067777c18)
        /home/vagrant/influxdb/src/api/http/api.go:539 +0x34a
api/http.(*HttpServer).tryAsDbUserAndClusterAdmin(0xc2100c25a0, 0x7f4074db15c8, 0xc2b8fa1320, 0xc422315a90, 0x7f4067777c18)
        /home/vagrant/influxdb/src/api/http/api.go:709 +0x268
api/http.(*HttpServer).writePoints(0xc2100c25a0, 0x7f4074db15c8, 0xc2b8fa1320, 0xc422315a90)
        /home/vagrant/influxdb/src/api/http/api.go:378 +0x1e5
api/http.*HttpServer.(api/http.writePoints)·fm(0x7f4074db15c8, 0xc2b8fa1320, 0xc422315a90)
        /home/vagrant/influxdb/src/api/http/api.go:106 +0x44
api/http.func·032(0x7f4074db15c8, 0xc2b8fa1320, 0xc422315a90)
        /home/vagrant/influxdb/src/api/http/cors.go:14 +0x1d0
net/http.HandlerFunc.ServeHTTP(0xc2101e33a0, 0x7f4074db15c8, 0xc2b8fa1320, 0xc422315a90)
        /home/vagrant/bin/go/src/pkg/net/http/server.go:1220 +0x40
github.com/bmizerany/pat.(*PatternServeMux).ServeHTTP(0xc21023ca20, 0x7f4074db15c8, 0xc2b8fa1320, 0xc422315a90)
/home/vagrant/influxdb/src/github.com/bmizerany/pat/mux.go:109 +0x1be
net/http.serverHandler.ServeHTTP(0xc210152640, 0x7f4074db15c8, 0xc2b8fa1320, 0xc422315a90)
        /home/vagrant/bin/go/src/pkg/net/http/server.go:1597 +0x16e
net/http.(*conn).serve(0xc210d44b80)
        /home/vagrant/bin/go/src/pkg/net/http/server.go:1167 +0x7b7
created by net/http.(*Server).Serve
        /home/vagrant/bin/go/src/pkg/net/http/server.go:1644 +0x28b

new configuration options (as object)

This is more of a reminder:

We should change the configuration of node-influx to accept an object of all configuration options, instead of one function argument per option.

Support write-consistency levels

Does this client code allow the user to select the write-consistency level?

https://github.com/influxdb/influxdb/blob/05d660a24de615ab2241063a70d418676ca6619b/cluster/points_writer.go#L25

Some people will want consistency other the default of ONE.

Handle chunked query responses

It would be nice to make a query and handle chunked responses.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Support for influxdb 0.9 [$215]

Will node-influx support influxdb 0.9? I think writePoint needs to support tags.

There is a $215 open bounty on this issue. Add to the bounty at Bountysource.

npm test fails for deleteDatabase

While updating the tests for the name change of readPoints() I noticed npm test fails on my machine for deleteDatabase() with "Error: timeout of 2000ms exceeded".

This is somehow related to the writeSeries() function though I cannot blame the function itself at all. It seems like after adding more and more tests for inserting points, influxDB takes a bit longer to delete the database.

I've written a debug script which does basically the same as the tests.js and measures the time between each step. As soon as I include writeSeries(), the deleteDatabase() call takes like 1000-1200ms, without writeSeries() it's like 400ms.

I'm not sure what to do.

"username cannot be empty" on createUser

The API probably changed
need to change request from
name: username,
password: password

to
username: username,
password: password

writeSeries allows for options but cant set database

Within index.js around line 191 sits this guy:

this.request.post({
    url: this.seriesUrl(this.options.database,query),
    headers: {
      'content-type': 'application/json'
...

The writeSeries, and by association, the other methods that internally end up calling this method, all accept options. Unfortunately, for clients that operate on many databases this line always takes the instances' options instead of the options passed in via the method.

node-influx / node-influx Goto Github PK

node-influx's Introduction

InfluxDB v1.x JavaScript library

Installation

Features

Compatibility

Usage

node-influx's People

Contributors

Stargazers

Watchers

Forkers

node-influx's Issues

39 Handle chunked query responses (epoch)

111 Query different timestamp formats (chunk_size)

Timestamp Format

This Works

This Does Not Work

Recommend Projects

Recommend Topics

Recommend Org