Git Product home page Git Product logo

calcite-pilosa's Introduction

Deprecation note:
Pilosa was rebranded to FeatureBase, which comes with its own SQL support.
Tools like Looker can solve the problem of joining tables from different sources.
This project is no longer actively maintained.

Calcite Pilosa Adapter

Plugin for Apache Calcite to query Pilosa distributed index using SQL.

General information

Pilosa is a high-performance distributed bitmap index. It has been successfully used in many data-intensive projects. One of the notable applications is the facts table in the analytical database. As the core of the analytical database, Pilosa solves the computational problem efficiently. Calcite Pilosa Adapter solves the problem of linking the Pilosa table with supplementary dimension tables from other databases, like Postgres, making Pilosa a powerful exploration tool for business intelligence.

How it works?

The Calcite Pilosa adapter works as a proxy. Clients connect with JDBC-compatible drivers and query the data with SQL, adapter translates queries into Pilosa Query language (PQL), and then translates results into JDBC ResultSet to send them back to the client.

Installation

<dependency>
  <groupId>com.alexrnv.calcite.adapter.pilosa</groupId>
  <artifactId>calcite-pilosa</artifactId>
  <version>0.0.1</version>
</dependency>

note the artifact is hosted in GitHub Packages

Usage

1.) Start from the configuration:

{
  "version": "1.0",
  "defaultSchema": "pilosa",
  "schemas": [
    {
      "name": "pilosa-cluster",
      "type": "custom",
      "factory": "com.alexrnv.calcite.adapter.pilosa.model.PilosaSchemaFactory",
      "operand": {
        "url": "http://localhost:10101"
      }
    }
  ]
} 

Provide your Pilosa server endpoint.

2.) The following code snippet starts a JDBC server inside your service.

LocalService service = new PilosaServiceFactory(modelFileUri).createLocalService();
HttpServer server = new PilosaHttpServerFactory(service, serverPort).createHttpServer();
server.start();
try {
    server.join();
} catch (InterruptedException e) {
    Thread.currentThread().interrupt();
} finally {
    server.stop();
}

Your service is now listening for JDBC connections.

3.) Connect from your favourite JDBC client.
Clients should use Avatica JDBC driver (mvn). Use version 1.14.0 or later.

jdbc:avatica:remote:url=http://<host>:<port>>/sql/v1

4.) Run your analysis with SQL

select 
count(distinct facts._id)
from 
pilosa.facts_table facts
join
postgres.dimension_table dimension
on dimension.X = facts.X
where 
dimension.Y = [value]

Please, check wiki for examples and available options.

calcite-pilosa's People

Contributors

alex-rnv avatar dependabot[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.