Firebolt Node.js SDK

Installation

This library is published in the NPM registry and can be installed using any compatible package manager.

npm install firebolt-sdk --save

# For Yarn, use the command below.
yarn add firebolt-sdk

Using the library

import { Firebolt } from 'firebolt-sdk'

const firebolt = Firebolt();

const connection = await firebolt.connect({
  username: process.env.FIREBOLT_USERNAME,
  password: process.env.FIREBOLT_PASSWORD,
  database: process.env.FIREBOLT_DATABASE,
  engineName: process.env.FIREBOLT_ENGINE_NAME
});

const statement = await connection.execute("SELECT 1");

// fetch statement result
const { data, meta } = await statement.fetchResult();

// or stream result
const { data } = await statement.streamResult();

data.on("metadata", metadata => {
  console.log(metadata);
});

data.on("error", error => {
  console.log(error);
});

const rows = []

for await (const row of data) {
  rows.push(row);
}

console.log(rows)

About
Documentation
Usage
Recipes
- Streaming results
- Custom stream transformers

About

The Firebolt client for Node.js. firebolt-sdk provides common methods for quering Firebolt databases, fetching and streaming results, and engine management.

firebolt-sdk supports Node.js > v14.

Documentation

Usage

Create connection

const connection = await firebolt.connect(connectionOptions);

ConnectionOptions

type ConnectionOptions = {
  username: string;
  password: string;
  database: string;
  engineName?: string;
  engineEndpoint?: string;
};

Test connection

TODO: write motivation connection can be tested using:

const firebolt = Firebolt();
await firebolt.testConnection(connectionOptions)

which will perform authentication and simple select 1 query

Engine URL

Firebolt engine URLs use the following format:

<engine-name>.<account-name>.<region>.app.firebolt.io

For example: your-engine.your-account.us-east-1.app.firebolt.io. You can find and copy your engine endpoint name in the Firebolt web UI.

Execute Query

const statement = await connection.execute(query, executeQueryOptions);

ExecuteQueryOptions

export type ExecuteQueryOptions = {
  settings?: QuerySettings;
  response?: ResponseSettings;
};

QuerySettings

Parameter	Required	Default	Description
output_format		JSON_COMPACT	Specifies format of selected data

ResponseSettings

Parameter	Required	Default	Description
normalizeData		false
bigNumberAsString		false	hydrate BigNumber as String

Fetch result

const { data, meta, statistics } = await statement.fetchResult();

The Promise API is not recommended for SELECT queries with large result sets (greater than 10,000 rows). This is because it parses results synchronously, so will block the JS thread/event loop and may lead to memory leaks due to peak GC loads.

It is recommended to use LIMIT in your queries when using the Promise API.

Stream result

const { data } = await statement.streamResult();
const rows: unknown[] = [];

data.on("metadata", metadata => {
  console.log(metadata);
});

data.on("error", error => {
  console.log(error);
});

for await (const row of data) {
  rows.push(row);
}

Result hydration

firebolt-sdk maps SQL data types to their corresponding JavaScript equivalents. The mapping is described in the table below:

Category	SQL type	JavaScript type	Notes
Numeric	INT	Number	If value cannot be represented by JavaScript Number (determine using Number.isSafeInteger), BigNumber from "bignumber.js" is used
	INTEGER	Number
	BIGINT	Number
	LONG	Number
	FLOAT	Number
	DOUBLE	Number
String	VARCHAR	String
	TEXT	String
	STRING	String
Date & Time	DATE	Date

Engine management

Engines can be managed by using the resourceManager object.

const firebolt = Firebolt();
const enginesService = firebolt.resourceManager.engines

getById

Returns engine using engine ID and account ID.

const firebolt = Firebolt();
const engine = await firebolt.resourceManager.engines.getById(
  "c8a228ea-93df-4784-99f9-a99368518782",
  "a32b073b-e093-4880-8fd4-3b302b4cf221"
);

getByName

Returns engine using engine name.

const firebolt = Firebolt();
const engine = await firebolt.resourceManager.engines.getByName("engine_name")

Engine

Property	Type	Notes
`id`	`{engine_id: string; account_id: string}`
`name`	`string`
`endpoint`	`string`
`current_status_summary`	`string`

Start

Starts an engine.

const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engines.getByName("engine_name")
await engine.start()

Stop

Stops an engine.

const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engines.getByName("engine_name")
await engine.stop()

Restart

Restarts an engine.

const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engines.getByName("engine_name")
await engine.restart()

Recipes

Streaming results

The recommended way to consume query results is by using streams.

For convenience, statement.streamResult also returns meta: Promise<Meta[]> and statistics: Promise<Statistics>, which are wrappers over data.on('metadata') and data.on('statistics').

const firebolt = Firebolt();

const connection = await firebolt.connect(connectionParams);

const statement = await connection.execute("SELECT 1");

const {
  data,
  meta: metaPromise,
  statistics: statisticsPromise
} = await statement.streamResult();

const rows: unknown[] = [];

const meta = await metaPromise;

for await (const row of data) {
  rows.push(row);
}

const statistics = await statisticsPromise

console.log(meta);
console.log(statistics);
console.log(rows)

Custom stream transformers

To achieve seamless stream pipes to fs or stdout, you can use the Transform stream.

import stream,  { TransformCallback } from 'stream';

class SerializeRowStream extends stream.Transform {
  public constructor() {
    super({
      objectMode: true,
      transform(
        row: any,
        encoding: BufferEncoding,
        callback: TransformCallback
      ) {
        const transformed = JSON.stringify(row);
        this.push(transformed);
        this.push('\n')
        callback();
      }
    });
  }
}

const serializedStream = new SerializeRowStream()

const firebolt = Firebolt();
const connection = await firebolt.connect(connectionParams);
const statement = await connection.execute("select 1 union all select 2");

const { data } = await statement.streamResult();

serializedStream.pipe(serializedStream).pipe(process.stdout);

Or use rowParser that returns strings or Buffer:

const { data } = await statement.streamResult({
  rowParser: (row: string) => `${row}\n`
});

data.pipe(process.stdout);

Development process

Actions before

Setup env variables

cp .env.example .env

Execute tests

  npm test

License

Released under Apache License.

hightouchio / firebolt-node-sdk Goto Github PK

firebolt-node-sdk's Introduction