Git Product home page Git Product logo

lib-go-databox's Introduction

libDatabox

import "./"

Overview

Index

Package files

arbiterClient.go coreStoreClient.go coreStoreKV.go coreStoreTS.go coreStoreTSBlob.go databoxRequest.go databoxlog.go export.go helperFunction.go types.go

Constants

const (
    Equals            FilterType      = "equals"
    Contains          FilterType      = "contains"
    Sum               AggregationType = "sum"
    Count             AggregationType = "count"
    Min               AggregationType = "min"
    Max               AggregationType = "max"
    Mean              AggregationType = "mean"
    Median            AggregationType = "median"
    StandardDeviation AggregationType = "sd"
)

Allowed values for FilterType and AggregationFunction

const DefaultArbiterKeyPath = "/run/secrets/ARBITER_TOKEN"
const DefaultArbiterURI = "tcp://arbiter:4444"
const DefaultHTTPSCertPath = "/run/secrets/DATABOX.pem"

DefaultHTTPSCertPath is the defaut loaction where apps and drivers can find the https certivicats needed to offer a secure UI

const DefaultHTTPSRootCertPath = "/run/secrets/DATABOX_ROOT_CA"

DefaultHTTPSRootCertPath contins the Public key of this databoxes Root certificate needed to verify requests to other components (used in )

const DefaultStorePublicKeyPath = "/run/secrets/ZMQ_PUBLIC_KEY"

func ChkErr

func ChkErr(err error)

func ChkErrFatal

func ChkErrFatal(err error)

func Debug

func Debug(msg string)

func Err

func Err(msg string)

func GetHttpsCredentials

func GetHttpsCredentials() string

GetHttpsCredentials Returns a string containing the HTTPS credentials to pass to https server when offering an https server. These are read form /run/secrets/DATABOX.pem and are generated by the container-manger at run time.

func GetStoreURLFromDsHref

func GetStoreURLFromDsHref(href string) (string, error)

GetStoreURLFromDsHref extracts the base store url from the href provied in the hypercat descriptions.

func Info

func Info(msg string)

func NewDataboxHTTPsAPI

func NewDataboxHTTPsAPI() *http.Client

func NewDataboxHTTPsAPIWithPaths

func NewDataboxHTTPsAPIWithPaths(cmRootCaPath string) *http.Client

func Warn

func Warn(msg string)

type AggregationType

type AggregationType string

type ArbiterClient

type ArbiterClient struct {
    ArbiterToken string

    ZestC zest.ZestClient
    // contains filtered or unexported fields
}

func NewArbiterClient

func NewArbiterClient(arbiterTokenPath string, zmqPublicKeyPath string, arbiterZMQURI string) (*ArbiterClient, error)

NewArbiterClient returns an arbiter client for use by components that require conunication with the arbiter

func (*ArbiterClient) GetRootDataSourceCatalogue

func (arb *ArbiterClient) GetRootDataSourceCatalogue() (HypercatRoot, error)

GetRootDataSourceCatalogue is used by the container manager to access the Root hypercat catalogue

func (*ArbiterClient) GrantComponentPermission

func (arb *ArbiterClient) GrantComponentPermission()

func (*ArbiterClient) GrantContainerPermissions

func (arb *ArbiterClient) GrantContainerPermissions(permissions ContainerPermissions) error

GrantContainerPermissions allows the container manager to grant permissions to an app or driver on a registered store.

func (*ArbiterClient) InvalidateCache

func (arb *ArbiterClient) InvalidateCache(href string, method string)

InvalidateCache can be used to remove a token from the arbiterClient cache. This is done automatically if the token is rejected.

func (*ArbiterClient) RegesterDataboxComponent

func (arb *ArbiterClient) RegesterDataboxComponent(name string, tokenString string, databoxType DataboxType) error

RegesterDataboxComponent allows the container manager to register a new app, driver or store with the arbiter

func (*ArbiterClient) RemoveDataboxComponent

func (arb *ArbiterClient) RemoveDataboxComponent()

func (*ArbiterClient) RequestToken

func (arb *ArbiterClient) RequestToken(href string, method string) ([]byte, error)

RequestToken is used internally to request a token from the arbiter

func (*ArbiterClient) RevokeComponentPermission

func (arb *ArbiterClient) RevokeComponentPermission()

type ContainerManagerOptions

type ContainerManagerOptions struct {
    Version               string
    SwarmAdvertiseAddress string
    DefaultRegistryHost   string
    DefaultRegistry       string
    DefaultAppStore       string
    DefaultStoreImage     string
    ContainerManagerImage string
    CoreUIImage           string
    ArbiterImage          string
    CoreNetworkImage      string
    CoreNetworkRelayImage string
    AppServerImage        string
    ExportServiceImage    string
    EnableDebugLogging    bool
    ClearSLAs             bool
    OverridePasword       string
    Hostname              string
    InternalIPs           []string
    ExternalIP            string
    HostPath              string
    Arch                  string //current architecture used to chose the correct docker images "" for x86 or "arm64v8" for arm64v8 ;-)
}

ContainerManagerOptions is used to configure the Container Manager

type ContainerPermissions

type ContainerPermissions struct {
    Name    string   `json:"name"`
    Route   Route    `json:"route"`
    Caveats []string `json:"caveats"`
}

type CoreStoreClient

type CoreStoreClient struct {
    ZestC      zest.ZestClient
    Arbiter    *ArbiterClient
    ZEndpoint  string
    DEndpoint  string
    KVJSON     *KVStore
    KVText     *KVStore
    KVBin      *KVStore
    TSBlobJSON *TSBlobStore
    TSBlobText *TSBlobStore
    TSBlobBin  *TSBlobStore
    TSJSON     *TSStore
}

func NewCoreStoreClient

func NewCoreStoreClient(arbiterClient *ArbiterClient, zmqPublicKeyPath string, storeEndPoint string, enableLogging bool) *CoreStoreClient

func NewDefaultCoreStoreClient

func NewDefaultCoreStoreClient(storeEndPoint string) *CoreStoreClient

func (*CoreStoreClient) GetStoreDataSourceCatalogue

func (csc *CoreStoreClient) GetStoreDataSourceCatalogue(href string) (HypercatRoot, error)

func (*CoreStoreClient) RegisterDatasource

func (csc *CoreStoreClient) RegisterDatasource(metadata DataSourceMetadata) error

RegisterDatasource is used by apps and drivers to register datasource in stores they own.

type DataSource

type DataSource struct {
    Type          string       `json:"type"`
    Required      bool         `json:"required"`
    Name          string       `json:"name"`
    Clientid      string       `json:"clientid"`
    Granularities []string     `json:"granularities"`
    Hypercat      HypercatItem `json:"hypercat"`
}

type DataSourceMetadata

type DataSourceMetadata struct {
    Description    string
    ContentType    string
    Vendor         string
    DataSourceType string
    DataSourceID   string
    StoreType      StoreType
    IsActuator     bool
    Unit           string
    Location       string
}

func HypercatToDataSourceMetadata

func HypercatToDataSourceMetadata(hypercatDataSourceDescription string) (DataSourceMetadata, string, error)

HypercatToDataSourceMetadata is a helper function to convert the hypercat description of a datasource to a DataSourceMetadata instance Also returns the store url for this data source.

type DataboxType

type DataboxType string
const (
    DataboxTypeApp    DataboxType = "app"
    DataboxTypeDriver DataboxType = "driver"
    DataboxTypeStore  DataboxType = "store"
)

type ExportWhitelist

type ExportWhitelist struct {
    Url         string `json:"url"`
    Description string `json:"description"`
}

type ExternalWhitelist

type ExternalWhitelist struct {
    Urls        []string `json:"urls"`
    Description string   `json:"description"`
}

type Filter

type Filter struct {
    TagName    string
    FilterType FilterType
    Value      string
}

Filter types to hold the required data to apply the filtering functions of the structured json API

type FilterType

type FilterType string

type HypercatItem

type HypercatItem struct {
    ItemMetadata []interface{} `json:"item-metadata"`
    Href         string        `json:"href"`
}

type HypercatRoot

type HypercatRoot struct {
    CatalogueMetadata []RelValPair   `json:"catalogue-metadata"`
    Items             []HypercatItem `json:"items"`
}

type KVStore

type KVStore struct {
    // contains filtered or unexported fields
}

func (*KVStore) Delete

func (kvj *KVStore) Delete(dataSourceID string, key string) error

Delete deletes data under the key.

func (*KVStore) DeleteAll

func (kvj *KVStore) DeleteAll(dataSourceID string) error

DeleteAll deletes all keys and data from the datasource.

func (*KVStore) ListKeys

func (kvj *KVStore) ListKeys(dataSourceID string) ([]string, error)

ListKeys returns an array of key registed under the dataSourceID

func (*KVStore) Observe

func (kvj *KVStore) Observe(dataSourceID string) (<-chan ObserveResponse, error)

func (*KVStore) ObserveKey

func (kvj *KVStore) ObserveKey(dataSourceID string, key string) (<-chan ObserveResponse, error)

func (*KVStore) Read

func (kvj *KVStore) Read(dataSourceID string, key string) ([]byte, error)

Read will read the vale store at under tha key return data is a object of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (*KVStore) Write

func (kvj *KVStore) Write(dataSourceID string, key string, payload []byte) error

Write Write will add data to the key value data store.

type LogEntries

type LogEntries struct {
    Msg  string `json:"msg"`
    Type string `json:"type"`
}

type Logger

type Logger struct {
    Store *CoreStoreClient
}

func New

func New(store *CoreStoreClient, outputDebugLogs bool) (*Logger, error)

func (Logger) ChkErr

func (l Logger) ChkErr(err error)

func (Logger) Debug

func (l Logger) Debug(msg string)

func (Logger) Err

func (l Logger) Err(msg string)

func (Logger) GetLastNLogEntries

func (l Logger) GetLastNLogEntries(n int) Logs

func (Logger) GetLastNLogEntriesRaw

func (l Logger) GetLastNLogEntriesRaw(n int) []byte

func (Logger) Info

func (l Logger) Info(msg string)

func (Logger) Warn

func (l Logger) Warn(msg string)

type Logs

type Logs []LogEntries

type Macaroon

type Macaroon string

type Manifest

type Manifest struct {
    ManifestVersion      int                  `json:"manifest-version"` //
    Name                 string               `json:"name"`
    DataboxType          DataboxType          `json:"databox-type"`
    Version              string               `json:"version"`     //this is databox version e.g 0.3.1
    Description          string               `json:"description"` // free text description
    Author               string               `json:"author"`      //Tosh Brown <[email protected]>
    License              string               `json:"license"`     //Software licence
    Tags                 []string             `json:"tags"`        //search tags
    Homepage             string               `json:"homepage"`    //homepage url
    Repository           Repository           `json:"repository"`
    Packages             []Package            `json:"packages"`
    DataSources          []DataSource         `json:"datasources"`
    ExportWhitelists     []ExportWhitelist    `json:"export-whitelist"`
    ExternalWhitelist    []ExternalWhitelist  `json:"external-whitelist"`
    ResourceRequirements ResourceRequirements `json:"resource-requirements"`
    DisplayName          string               `json:"displayName"`
    StoreURL             string               `json:"storeUrl"`
}

type ObserveResponse

type ObserveResponse struct {
    TimestampMS  int64
    DataSourceID string
    Key          string
    Data         []byte
}

OBSERVE RESPONSE

type Package

type Package struct {
    Name        string   `json:"name"`
    Purpose     string   `json:"purpose"`
    Install     string   `json:"install"`
    Risks       string   `json:"risks"`
    Benefits    string   `json:"benefits"`
    DataSources []string `json:"datastores"`
}

type RelValPair

type RelValPair struct {
    Rel string `json:"rel"`
    Val string `json:"val"`
}

type RelValPairBool

type RelValPairBool struct {
    Rel string `json:"rel"`
    Val bool   `json:"val"`
}

type Repository

type Repository struct {
    Type string `json:"Type"`
    Url  string `json:"url"`
}

type ResourceRequirements

type ResourceRequirements struct {
    Store string `json:"store"`
}

type Route

type Route struct {
    Target string `json:"target"`
    Path   string `json:"path"`
    Method string `json:"method"`
}

type SLA

type SLA struct {
    ManifestVersion      int                  `json:"manifest-version"` //
    Name                 string               `json:"name"`             // container name  e.g core-store
    Image                string               `json:"image"`            //docker image tag e.g datboxsystems/core-store-amd64
    DataboxType          DataboxType          `json:"databox-type"`
    Version              string               `json:"version"`     //this is databox version e.g 0.3.1
    Description          string               `json:"description"` // free text description
    Author               string               `json:"author"`      //Tosh Brown <[email protected]>
    License              string               `json:"license"`     //Software licence
    Tags                 []string             `json:"tags"`        //search tags
    Homepage             string               `json:"homepage"`    //homepage url
    Repository           Repository           `json:"repository"`
    Packages             []Package            `json:"packages"`
    AllowedCombinations  []string             `json:"allowed-combinations"`
    Datasources          []DataSource         `json:"datasources"`
    ExportWhitelists     []ExportWhitelist    `json:"export-whitelist"`
    ExternalWhitelist    []ExternalWhitelist  `json:"external-whitelist"`
    ResourceRequirements ResourceRequirements `json:"resource-requirements"`
    DisplayName          string               `json:"displayName"`
    StoreURL             string               `json:"storeUrl"`
    Registry             string               `json:"registry"`
}

type StoreContentType

type StoreContentType string
const ContentTypeBINARY StoreContentType = "BINARY"
const ContentTypeJSON StoreContentType = "JSON"
const ContentTypeTEXT StoreContentType = "TEXT"

type StoreType

type StoreType string
const StoreTypeKV StoreType = "kv"
const StoreTypeTS StoreType = "ts"
const StoreTypeTSBlob StoreType = "ts/blob"

type TSBlobStore

type TSBlobStore struct {
    // contains filtered or unexported fields
}

func (*TSBlobStore) Earliest

func (tbs *TSBlobStore) Earliest(dataSourceID string) ([]byte, error)

Earliest will retrieve the first entry stored at the requested datasource ID return data is a byte array contingin of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (*TSBlobStore) FirstN

func (tbs *TSBlobStore) FirstN(dataSourceID string, n int) ([]byte, error)

FirstN will retrieve the first N entries stored at the requested datasource ID return data is a byte array contingin of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (*TSBlobStore) LastN

func (tbs *TSBlobStore) LastN(dataSourceID string, n int) ([]byte, error)

LastN will retrieve the last N entries stored at the requested datasource ID return data is a byte array contingin of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (*TSBlobStore) Latest

func (tbs *TSBlobStore) Latest(dataSourceID string) ([]byte, error)

TSBlobLatest will retrieve the last entry stored at the requested datasource ID return data is a byte array contingin of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (*TSBlobStore) Length

func (tbs *TSBlobStore) Length(dataSourceID string) (int, error)

TSBlobLength returns then number of items stored in the timeseries

func (*TSBlobStore) Observe

func (tbs *TSBlobStore) Observe(dataSourceID string) (<-chan ObserveResponse, error)

Observe allows you to get notifications when a new value is written by a driver the returned chan receives chan ObserveResponse the data value og which contins json of the form {"TimestampMS":213123123,"Json":byte[]}

func (*TSBlobStore) Range

func (tbs *TSBlobStore) Range(dataSourceID string, formTimeStamp int64, toTimeStamp int64) ([]byte, error)

Range will retrieve all entries between formTimeStamp and toTimeStamp timestamp in ms since unix epoch return data is a byte array contingin of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (*TSBlobStore) Since

func (tbs *TSBlobStore) Since(dataSourceID string, sinceTimeStamp int64) ([]byte, error)

Since will retrieve all entries since the requested timestamp (ms since unix epoch) return data is a byte array contingin of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (*TSBlobStore) Write

func (tbs *TSBlobStore) Write(dataSourceID string, payload []byte) error

Write will add data to the times series data store. Data will be time stamped at insertion (format ms since 1970)

func (*TSBlobStore) WriteAt

func (tbs *TSBlobStore) WriteAt(dataSourceID string, timstamp int64, payload []byte) error

WriteAt will add data to the times series data store. Data will be time stamped with the timstamp provided in the timstamp paramiter (format ms since 1970)

type TSStore

type TSStore struct {
    // contains filtered or unexported fields
}

func (TSStore) Earliest

func (tsc TSStore) Earliest(dataSourceID string) ([]byte, error)

Earliest will retrieve the first entry stored at the requested datasource ID return data is a JSON object of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (TSStore) FirstN

func (tsc TSStore) FirstN(dataSourceID string, n int, opt TimeSeriesQueryOptions) ([]byte, error)

FirstN will retrieve the first N entries stored at the requested datasource ID return data is an array of JSON objects of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (TSStore) LastN

func (tsc TSStore) LastN(dataSourceID string, n int, opt TimeSeriesQueryOptions) ([]byte, error)

LastN will retrieve the last N entries stored at the requested datasource ID return data is an array of JSON objects of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (TSStore) Latest

func (tsc TSStore) Latest(dataSourceID string) ([]byte, error)

Latest will retrieve the last entry stored at the requested datasource ID return data is a JSON object of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (TSStore) Length

func (tsc TSStore) Length(dataSourceID string) (int, error)

Length retruns the number of records stored for that dataSourceID

func (TSStore) Observe

func (tsc TSStore) Observe(dataSourceID string) (<-chan ObserveResponse, error)

func (TSStore) Range

func (tsc TSStore) Range(dataSourceID string, formTimeStamp int64, toTimeStamp int64, opt TimeSeriesQueryOptions) ([]byte, error)

Range will retrieve all entries between formTimeStamp and toTimeStamp timestamp in ms since unix epoch return data is a JSON object of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (TSStore) Since

func (tsc TSStore) Since(dataSourceID string, sinceTimeStamp int64, opt TimeSeriesQueryOptions) ([]byte, error)

Since will retrieve all entries since the requested timestamp (ms since unix epoch) return data is a JSON object of the format {"timestamp":213123123,"data":[data-written-by-driver]}

func (TSStore) Write

func (tsc TSStore) Write(dataSourceID string, payload []byte) error

Write will add data to the times series data store. Data will be time stamped at insertion (format ms since 1970)

func (TSStore) WriteAt

func (tsc TSStore) WriteAt(dataSourceID string, timstamp int64, payload []byte) error

WriteAt will add data to the times series data store. Data will be time stamped with the timstamp provided in the timstamp paramiter (format ms since 1970)

type TimeSeriesQueryOptions

type TimeSeriesQueryOptions struct {
    AggregationFunction AggregationType
    Filter              *Filter
}

TimeSeriesQueryOptions describes the query options for the structured json API

Development of databox was supported by the following funding

EP/N028260/1, Databox: Privacy-Aware Infrastructure for Managing Personal Data
EP/N028260/2, Databox: Privacy-Aware Infrastructure for Managing Personal Data
EP/N014243/1, Future Everyday Interaction with the Autonomous Internet of Things
EP/M001636/1, Privacy-by-Design: Building Accountability into the Internet of Things EP/M02315X/1, From Human Data to Personal Experience

lib-go-databox's People

Contributors

toshbrown avatar cgreenhalgh avatar

Watchers

James Cloos avatar avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.