iq-scm / sybil Goto Github PK

Sybil is an append only analytics datastore with no up front table schema requirements; just log JSON records to a table and run queries. Written in Go, sybil is designed for fast full table scans of multi-dimensional data on one or more machines.

more documentation is available on the wiki and in the repo.

if sybil by itself is uninteresting (who wants to run command line queries, anyways?), sybil is a supported backend for snorkel

• Easy to setup and get data inside sybil - just pipe JSON on stdin to sybil
• Supports histograms (and percentiles), standard deviations and time series roll ups
• Runs fast full table queries (performance notes)
• Lower disk usage through per column compression schemes
• Serverless design with controlled memory usage
• Per table retention policies (specify max age and/or size of tables)
• Timestamp resistant query cache (optional) that avoids recomputation

• JOINS not supported
• No UPDATE operation on data - only writes

go get github.com/logv/sybil

• command line tour
• want to contribute?
• notes on performance
• implementation notes
• abadi survey of column stores