Comments (3)
Hi,
I'm glad that you've noticed it.
I've seen your module, but it seems that it's implementation is very specific to the company that sponsored it, eg. default field named value or input data precision hardcoded to nanoseconds.
On the other hand my module (as you've noticed) is not making any conversion nor validation of the data, putting the responsibility onto the users of it. As you are aware the InfluxDB is very sensitive to the format of the data (especially to the first write), and I've taken the position that the users know the best how they do want the data to be stored.
That's why my _to_line
function is extremely generic, simple and fast, and what is most important does exactly what the user asked for - creates the correctly formatted input to the /write
endpoint out of the hash reference.
And that was based on the fact that in vast majority of the cases the data that users will store in InfluxDB would not contain any special characters needed escaping nor quoting.
In other cases my module would happily accept the output of your data2line
function:
$cv = AE::cv;
$db->write(
database => 'mydb',
precision => 'n',
data => data2line('measurement', 42, { tag => 'foo'} ),
on_success => $cv,
on_error => sub {
$cv->croak("Failed to write data: @_");
}
);
$cv->recv;
I'll be glad to collaborate with you on extending the functionality of your module, as it seems your aim is to provide functions that automate the conversion of some complex data into line protocol expected by InfluxDB.
Thank you,
Alex
from anyevent-influxdb.
I've seen your module, but it seems that it's implementation is very specific to the company
that sponsored it, eg. default field named value or input data precision hardcoded to
nanoseconds.
Actually no, we just used the defaults suggested by the InfluxDB docs.
My goal was to make it very easy to write stats, without needing to know all the details of the LineProtocol (eg escaping, ordering of tags, what time precision to use, ..)
and I've taken the position that the users know the best how they do want the data to be stored
..
creates the correctly formatted input to the /write endpoint out of the hash reference.
which all asumes that the users know exactly how InfluxDB works internally. Which is a fair point, but I try to abstract the annoying part away for the user, so that there in only one source for bugs (my conversion code), and not unlimited sources (everywhere a user calls ->write)
And that was based on the fact that in vast majority of the cases the data that users will
store in InfluxDB would not contain any special characters needed escaping nor quoting.
Famous last words :-)
But in the end you are right that your module does not need to use InfluxDB::LineProtocol. as users can just call data2line themselves (as shown in your code sample)
You could just point out in the docs that when using an hash as input to write(), users MUST escape the values properly; or could use InfluxDB::LineProtocol to convert a hash to a line, and pass the line on to you're write(). That way your module stays lean and fast ( though IMO fast-and-wrongish as opposed to slowish-and-correct :-) ), but users of it still get educated on how to easily convert a hash to a proper line.
If you want I can prepare a doc patch.
And I still would love some feedback on my data2line() (missing features, bugs, pot. speed improvements..)
Greetings,
domm
from anyevent-influxdb.
The fact that there is no quoting nor escaping is already documented.
The use case for using asynchronous client library and dedicated time-series database is the speed - when lots of datapoints are being written, wasting CPU cycles for unnecessary work or transferring redundant data over network is not really acceptable.
But I'm also sure that there are also users (maybe even more) who would prefer hand-holding approach your module provides.
When I have a chance I'll look into your module in more details and make some suggestions if I find anything that needs improving without breaking the spirit of your module.
Thank you,
Alex
from anyevent-influxdb.
Related Issues (3)
- still active? HOT 5
- No "tags" key in selected data HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from anyevent-influxdb.