Git Product home page Git Product logo

clickhouse's Introduction

clickhouse

NodeJS client for ClickHouse. Send query over HTTP interface.

Install:

npm i clickhouse

Example:

const { ClickHouse } = require('clickhouse');

const clickhouse = new ClickHouse();

or with all options:

const clickhouse = new ClickHouse({
	url: 'http://localhost',
	port: 8123,
	debug: false,
	basicAuth: null,
	isUseGzip: false,
	format: "json", // "json" || "csv" || "tsv"
	raw: false,
	config: {
		session_id                              : 'session_id if neeed',
		session_timeout                         : 60,
		output_format_json_quote_64bit_integers : 0,
		enable_http_compression                 : 0,
		database                                : 'my_database_name',
	},
	
	// This object merge with request params (see request lib docs)
	reqParams: {
		...
	}
});

or change

basicAuth: null

to

basicAuth: {
	username: 'default',
	password: '',
},

Exec query:

const queries = [
	'DROP TABLE IF EXISTS session_temp',

	`CREATE TABLE session_temp (
		date Date,
		time DateTime,
		mark String,
		ips Array(UInt32),
		queries Nested (
			act String,
			id UInt32
		)
	)
	ENGINE=MergeTree(date, (mark, time), 8192)`,

	'OPTIMIZE TABLE ukit.loadstat PARTITION 201807 FINAL'
];

for(const query of queries) {
	const r = await clickhouse.query(query).toPromise();

	console.log(query, r);
}

Exec by callback way:

clickhouse.query(query).exec(function (err, rows) {
	...
});

Stream:

clickhouse.query(`SELECT number FROM system.numbers LIMIT 10`).stream()
	.on('data', function() {
		const stream = this;

		stream.pause();

		setTimeout(() => {
			stream.resume();
		}, 1000);
	})
	.on('error', err => {
		...
	})
	.on('end', () => {
		...
	});

or async stream:

// async iteration
for await (const row of clickhouse.query(sql).stream()) {
	console.log(row);
}

As promise:

const rows = await clickhouse.query(query).toPromise();

// use query with external data
const rows = await clickhouse.query('SELECT * AS count FROM temp_table', {
	external: [
		{
			name: 'temp_table',
			data: e._.range(0, rowCount).map(i => `str${i}`)
		},
	]
}).toPromise();

Set session:

clickhouse.sessionId = '...';
const r = await clickhouse.query(
	`CREATE TEMPORARY TABLE test_table
	(_id String, str String)
	ENGINE=Memory`
).toPromise();

In case your application requires specific sessions to manage specific data then you can send session_id with each query.

let mySessionId = 'some_randome_string';
const r = await clickhouse.query(
	`CREATE TEMPORARY TABLE test_table
	(_id String, str String)
	ENGINE=Memory`, {}, {sessionId: mySessionId}
).toPromise();

Insert stream:

const ws = clickhouse.insert('INSERT INTO session_temp').stream();
for(let i = 0; i <= 1000; i++) {
	await ws.writeRow(
		[
			e._.range(0, 50).map(
				j => `${i}:${i * 2}:${j}`
			).join('-')
		]
	);
}

//wait stream finish
const result = await ws.exec();

Pipe readable stream to writable stream (across transform):

const rs = clickhouse.query(query).stream();

const tf = new stream.Transform({
	objectMode : true,
	transform  : function (chunk, enc, cb) {
		cb(null, JSON.stringify(chunk) + '\n');
	}
});

clickhouse.sessionId = Date.now();
const ws = clickhouse.insert('INSERT INTO session_temp2').stream();

const result = await rs.pipe(tf).pipe(ws).exec();

Run Tests:

npm install
npm run test
# or
# node_modules/.bin/mocha --timeout 60000 --slow 5000 -f "SPECIFIC TEST NAME"

Changelogs:

  • 2020-11-02 (v2.4.1)
    • Merge list of PR
    • fix test with Base Auth check
  • 2020-11-02 (v2.2.0) Backward Incompatible Change
    • port from url more important than port from config
  • 2020-04-17 (v2.1.0)
    • Fix query with totals. For json formats work perfect, but for another - doesn't
  • 2019-02-13
    • Add compatibility with user and username options
  • 2019-02-07
    • Add TLS/SSL Protocol support
    • Add async iteration over SELECT

Links

clickhouse's People

Contributors

0xgeert avatar aiiejibcuhka avatar arman-oganesyan avatar baryshev avatar beglex avatar dependabot[bot] avatar ianmaddox avatar lexasa avatar masterodin avatar mhlm avatar nini-k avatar pavhov93 avatar rayvid avatar sarthaksahni avatar tagplus5 avatar tgarifullin avatar timonkk avatar umaxfun avatar vdemtcev avatar xboston avatar

Stargazers

 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.