Git Product home page Git Product logo

orbitdb's Introduction

OrbitDB

Matrix npm (scoped) node-current (scoped)

OrbitDB is a serverless, distributed, peer-to-peer database. OrbitDB uses IPFS as its data storage and Libp2p Pubsub to automatically sync databases with peers. It's an eventually consistent database that uses Merkle-CRDTs for conflict-free database writes and merges making OrbitDB an excellent choice for p2p and decentralized apps, blockchain applications and local-first web applications.

OrbitDB provides various types of databases for different data models and use cases:

  • events: an immutable (append-only) log with traversable history. Useful for "latest N" use cases or as a message queue.
  • documents: a document database to which JSON documents can be stored and indexed by a specified key. Useful for building search indices or version controlling documents and data.
  • keyvalue: a key-value database just like your favourite key-value database.
  • keyvalue-indexed: key-value data indexed in a Level key-value database.

All databases are implemented on top of OrbitDB's OpLog, an immutable, cryptographically verifiable, operation-based conflict-free replicated data structure (CRDT) for distributed systems. OpLog is formalized in the paper Merkle-CRDTs. You can also easily extend OrbitDB by implementing and using a custom data model benefitting from the same properties as the default data models provided by the underlying Merkle-CRDTs.

This is the Javascript implementation and it works both in Browsers and Node.js with support for Linux, OS X, and Windows.

A Go implementation is developed and maintained by the Berty project at berty/go-orbit-db.

Installation

Install OrbitDB and its dependencies:

npm install @orbitdb/core helia

Browser <script> tag

OrbitDB can be loaded in the browser using the distributed js file with the <script/> tag. OrbitDB is the global namespace and all external functions are available via this namespace:

<script>/path/to/orbitdb.min.js</script>

Quick Start

If you want to get up and running with OrbitDB quickly, install and follow the instructions in the @orbitdb/quickstart module.

Usage

If you're using @orbitdb/core to develop browser or Node.js applications, use it as a module with the javascript instance of IPFS.

import { createHelia } from 'helia'
import { createOrbitDB } from '@orbitdb/core'
import { gossipsub } from "@chainsafe/libp2p-gossipsub";
import { identify } from "@libp2p/identify";
import { createLibp2p } from 'libp2p'

const Libp2pOptions = {
  services: {
    pubsub: gossipsub({
      // neccessary to run a single peer
      allowPublishToZeroTopicPeers: true
    }),
    identify: identify()
  }
}

;(async function () {
  const libp2p = await createLibp2p({ ...Libp2pOptions })
  const ipfs = await createHelia({libp2p})
  const orbitdb = await createOrbitDB({ ipfs })

  // Create / Open a database. Defaults to db type "events".
  const db = await orbitdb.open("hello")
  
  const address = db.address
  console.log(address)
  // "/orbitdb/zdpuAkstgbTVGHQmMi5TC84auhJ8rL5qoaNEtXo2d5PHXs2To"
  // The above address can be used on another peer to open the same database

  // Listen for updates from peers
  db.events.on("update", async entry => {
    console.log(entry)
    const all = await db.all()
    console.log(all)
  })

  // Add an entry
  const hash = await db.add("world")
  console.log(hash)

  // Query
  for await (const record of db.iterator()) {
    console.log(record)
  }
  
  await db.close()
  await orbitdb.stop()
  await ipfs.stop()
})()

To configure your IPFS instance for persistency and Libp2p to connect to peers, see Creating a Helia instance and the Default Libp2p Configurations in @orbitdb/quickstart for examples.

Documentation

Use the Getting Started guide for an initial introduction to OrbitDB.

You can find more advanced topics in our docs covering:

API

See https://api.orbitdb.org for the full API documentation.

Development

Run Tests

npm run test

Build

npm run build

Benchmark

node benchmarks/orbitdb-events.js

See benchmarks/ for more benchmarks.

API

To build the API documentation, run:

npm run build:docs

Documentation is output to ./docs/api.

Other implementations

If you know of any other repos that ought to be included in this section, please open a PR and add them.

Contributing

Take a look at our organization-wide Contributing Guide. You'll find most of your questions answered there.

If you want to code but don't know where to start, check out the issues labelled "help wanted".

Sponsors

The development of OrbitDB has been sponsored by:

If you want to sponsor developers to work on OrbitDB, please donate to our OrbitDB Open Collective or reach out to @haadcode.

License

MIT © 2015-2023 Protocol Labs Inc., Haja Networks Oy, OrbitDB Community

orbitdb's People

Contributors

0xflotus avatar 1-byte-man avatar achingbrain avatar aphelionz avatar basraven avatar csdummi avatar dependabot[bot] avatar dignifiedquire avatar geolffreym avatar greenkeeperio-bot avatar haadcode avatar haydenyoung avatar koh-osug avatar mistakia avatar oed avatar omahs avatar phillmac avatar prabodhmeshram avatar ricardojmendez avatar richardlitt avatar saul-jb avatar shamb0t avatar silkroadnomad avatar spartanz51 avatar swedneck avatar tabcat avatar thiagodelgado111 avatar tyleryasaka avatar vasa-develop avatar zachferland avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

orbitdb's Issues

Cannot read property subscribe of undefined

I might vrery well have been wrong at some point,
When I'm trying to instantiate the Orbitdb document store, that error appear:

ipfs-pubsub.js:16 Uncaught TypeError: Cannot read property 'subscribe' of undefined(…)

I am using ipfs-api, in the browser.

Peergos vs orbit-db?

Not really an issue, just my curiosity (sorry if this is the wrong place to question this).
I would like to know what the main difference between peergos and orbit-db.

npm test fails for js-ipfs-api with TypeError

This is the relevant part of the test output, the same error is repeated 25 times for all js-ipfs-api tests but 2:

  108 passing (41s)
  25 failing

  1) Orbit Client with js-ipfs-api Add events adds an item to an empty channel:
     TypeError: callback is not a function
      at size (node_modules/ipld-dag-pb/src/dag-node.js:210:9)
      at util.serialize (node_modules/ipld-dag-pb/src/dag-node.js:183:7)
      at Object.exports.serialize.err [as serialize] (node_modules/ipld-dag-pb/src/util.js:55:3)
      at DAGNode.size (node_modules/ipld-dag-pb/src/dag-node.js:170:10)
      at multihash (node_modules/ipld-dag-pb/src/dag-node.js:198:12)
      at DAGNode.multihash (node_modules/ipld-dag-pb/src/dag-node.js:161:7)
      at DAGNode.toJSON (node_modules/ipld-dag-pb/src/dag-node.js:194:10)
      at ipfs.object.put.then (node_modules/ipfs-log/src/entry.js:37:26)
      at process._tickDomainCallback (internal/process/next_tick.js:129:7)

Looking at the problem, it seems that DAGNode.toJSON has a "nodeback" API since ipld-dag-pb 0.8.0, but toIpfsHash in entry.js of ipfs-log 1.5.2 still calls it synchronously without a callback.

Because of dependencies on the master branch of js-ipfs-api, it might be difficult to reproduce the same environment on your machine to reproduce this issue.

Trouble running install command with Windows 10 operating system

Currently trying to get the browser examples to work, but cannot get past the "npm install" command on my Windows 10 system. Below is the error output received. Any advice and feedback appreciated and thank you in advance.


C:\Users\Will\orbit-db>npm install

[email protected] install C:\Users\Will\orbit-db\node_modules\go-ipfs-dep
node src/bin.js $TARGET_OS

C:\Users\Will\orbit-db\node_modules\go-ipfs-dep\src\index.js:42
throw new Error('no binary available for os:' + goenv.GOOS)
^

Error: no binary available for os:windows
at checkPlatform (C:\Users\Will\orbit-db\node_modules\go-ipfs-dep\src\index.js:42:15)
at module.exports (C:\Users\Will\orbit-db\node_modules\go-ipfs-dep\src\index.js:11:3)
at Object. (C:\Users\Will\orbit-db\node_modules\go-ipfs-dep\src\bin.js:3:14)
at Module._compile (module.js:556:32)
at Object.Module._extensions..js (module.js:565:10)
at Module.load (module.js:473:32)
at tryModuleLoad (module.js:432:12)
at Function.Module._load (module.js:424:3)
at Module.runMain (module.js:590:10)
at run (bootstrap_node.js:394:7)
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.0.0 (node_modules\chokidar\node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
npm ERR! Windows_NT 10.0.10586
npm ERR! argv "C:\Program Files\nodejs\node.exe" "C:\Users\Will\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js" "install"
npm ERR! node v6.7.0
npm ERR! npm v3.10.8
npm ERR! code ELIFECYCLE

npm ERR! [email protected] install: node src/bin.js $TARGET_OS
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script 'node src/bin.js $TARGET_OS'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the go-ipfs-dep package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node src/bin.js $TARGET_OS
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs go-ipfs-dep
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls go-ipfs-dep
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! C:\Users\Will\orbit-db\npm-debug.log

.

Use username as seed for random in BackgroundAnimation's

Math.random function in BackgroundAnimation.js should use the the username as the random's seed (in LoginView).

BackgroundAnimation should take a seed from the passed properties:

<BackgroundAnimation seed='text'/>

This enables to tie the little orbiting planets to the username, ie. every time you type in your username, you see the same planets.

Error creating OrbitDB

I am getting this error now when creating an OrbitDB instance. I have updated to the latest version and re-installed npm modules after removing the directory. I am using the browser daemon and have tried both the IPFS signal server and a local signal server.

TypeError: this._daemon.load is not a function
    at _daemon.config.set (http://localhost:8100/build/main.js:211063:30)
    at http://localhost:8100/build/main.js:221933:9
    at IDBTransaction.tx.oncomplete (http://localhost:8100/build/main.js:149063:5)
    at IDBTransaction.n [as _oncomplete] (http://localhost:8100/build/polyfills.js:2:27340)
    at t.invokeTask (http://localhost:8100/build/polyfills.js:3:10284)
    at Object.onInvokeTask (http://localhost:8100/build/main.js:57608:37)
    at t.invokeTask (http://localhost:8100/build/polyfills.js:3:10220)
    at e.runTask (http://localhost:8100/build/polyfills.js:3:7637)
    at IDBTransaction.invoke (http://localhost:8100/build/polyfills.js:3:11397)

Examples Fail on pubsub

I've tried this both for js-ipfs example and the ipfs-api example. Is there something special I need to do to make this work like target a specific version of either? There does not appear to be a pubsub module on the ipfs object.

const IpfsApi = require('ipfs-api')
const OrbitDB = require('orbit-db')

const ipfs = IpfsApi('localhost', '5001')
const orbitdb = new OrbitDB(ipfs)

const db = orbitdb.eventlog("feed name")

db.add("hello world")
  .then(() => {
    const latest = db.iterator({ limit: 5 }).collect()
    console.log(JSON.stringify(latest, null, 2))
  })
ipfs-pubsub.js:16
      this._ipfs.pubsub.subscribe(hash, { discover: true }, (err, stream) => {
                       ^

TypeError: Cannot read property 'subscribe' of undefined
    at IPFSPubsub.subscribe (/home/victor/Workspace/src/github.com/vijayee/orbit-db-sandbox/node_modules/orbit-db/node_modules/orbit-db-pubsub/src/ipfs-pubsub.js:16:24)
    at OrbitDB._subscribe (/home/victor/Workspace/src/github.com/vijayee/orbit-db-sandbox/node_modules/orbit-db/src/OrbitDB.js:71:20)
    at OrbitDB._createStore (/home/victor/Workspace/src/github.com/vijayee/orbit-db-sandbox/node_modules/orbit-db/src/OrbitDB.js:62:17)
    at OrbitDB.eventlog (/home/victor/Workspace/src/github.com/vijayee/orbit-db-sandbox/node_modules/orbit-db/src/OrbitDB.js:30:17)
    at Object.<anonymous> (/home/victor/Workspace/src/github.com/vijayee/orbit-db-sandbox/index.js:7:20)
    at Module._compile (module.js:573:32)
    at Object.Module._extensions..js (module.js:582:10)
    at Module.load (module.js:490:32)
    at tryModuleLoad (module.js:449:12)
    at Function.Module._load (module.js:441:3)
    at Module.runMain (module.js:607:10)
    at run (bootstrap_node.js:420:7)
    at startup (bootstrap_node.js:139:9)
    at bootstrap_node.js:535:3

Is OrbitDB "live"?

When using the browser daemon and the ipfs dev signalling server, is the connection "live" to ipfs? I am not getting a "data" event from my databases, but when I change it to "write" the event gets thrown. This leads me to believe the data doesn't get written to ipfs. I also tried using a local ipfs node and setting the Addresses property in ipfs options and I still don't get the "data" event firing. Am I misunderstanding something?

Send actual objects to pubsub instead of the object hashes

The log is currently transported as a hash of the snapshot of the last 8 items of the log. We should change this so that the individual objects are sent to pubsub (not the hash, the actual object). This will speed up the transport significantly.

IPFS hash for orbit db record

Hello, I use new OrbitDB(ipfs).eventlog('MY_DB'); How I can obtain IPFS hash for my record or any other data structure which I can check directly by ipfs cat ... ?

Does the db sync work for browser?

I tried running the examples/browser/browser.html with the SignalServer changed to 0.0.0.0:9090 and started the js-libp2p-webrtc-star signaling server locally. But when I run it in two tabs then the data being pushed to eventlog doesn't seem to sync across both of them.

Am I missing something here? Does the data sync work in the browser as the node example seem to work fine.

P.S: I am new to IPFS and ordit-db so my knowledge is limited.

Possible types of databases in Orbit?

Can Orbit be used to make a NoSQL type database (MongoDB, etc). We want to build a distributed database system which will store individual people's id number and then store other data related to the number such as Name, age, etc.

Add ability to join to a network from a IPFS hash

Instead of
client.connect(host, port)

make it possible to connect to an IPFS hash:
client.connect(hash).

Upon calling connect(hash), the client will read the hash from IPFS. The returned data contains the pubsub addresses and the network name.

Entry point for the logic is OrbitDB._connect().

How to designate a canonical place for the database?

I'm trying to create a simple html-page where everyone who accessess it is using the same database, but I don't know how.

So, when I open the page with Firefox I get one local database (placed somewhere in ~/.mozilla?), which works well enough. But when I open it with Opera I get another database (with no documents I've added to the firefox-database), and when I open Chromium I get a third, unique database. And when place the file on ipfs and read it through http://localhost:8080/ipfs/hash I access on database and when I read it through https://gateway.ipfs.io/ipfs/hash I read from another database. That is, in one database I have [{_id: "jknfiunuöf", doc: "Hello World!"}] and in another I have [_id: "xajnkju2", "Mellon"}].

So, how can I make them all read from the same database?

I'm guessing I need to define the server and folder path when creating the IPFS-object, such as:

  const ipfs = new IPFS({
    IpfsDataDir: '/orbit-db-/examples/browser',
    SignalServer: 'star-signal.cloud.ipfs.team'
  })

But I cannot get this to work, when I try to add a doc to the database now it simple says it is listening to this address. What am I missing?

This is the code I run:

'use strict'
const IPFS = require('ipfs-daemon/src/ipfs-browser-daemon')
const OrbitDB = require('orbit-db')
var newEntry, putButton;
if (typeof window !== 'undefined'){
  newEntry = document.getElementById('new-entry')
  putButton = document.getElementById('add-to-docstore')
}



const showContentOfDatabase = (array) =>{
  if(!array) array = [];
  var contentInHtml = "";
  for(var x = 0; x < array.length; x++){
    contentInHtml += "<p>" + array[x].doc +"</p>"
  }
  if (typeof window !== 'undefined'){
    document.getElementById('database-entries').innerHTML = contentInHtml;
  }else{
    console.log(contentInHtml)
  }
}

const openAndEditDatabase = (obj) =>{
  const ipfs = new IPFS({
    IpfsDataDir: 'orbit-db',
    SignalServer: 'ws://ip4/127.0.0.1/tcp/4001'
  })
  ipfs.on('error', (e) => console.log(e))
  ipfs.on('ready', () => {
    const orbitdb = new OrbitDB(ipfs)
    const docstore = orbitdb.docstore('docstore');
    docstore.events.on('ready', () => {
      console.log(docstore.query( (doc) => true ))
      showContentOfDatabase(docstore.query( (doc) => true ));
      if(obj && obj.put) docstore.put(obj.put)
    })
    docstore.load()

    docstore.events.on('write', () => {
      console.log("on write event")
      showContentOfDatabase(docstore.query( (doc) => true ));
    })
  })
}

openAndEditDatabase();
const addEntry = () =>{
  openAndEditDatabase({put:{_id:randomId(),doc:newEntry.value}})
  newEntry.value = "";
}


if (typeof window !== 'undefined'){
  putButton.addEventListener('click', addEntry)
}

Running tests on Cloud9 workspace

I'm trying to run the test orbit-db/test/replicate.test.js and have restricted it to run with the IpfsNodeDaemon.

[IpfsNodeDaemon].forEach((IpfsDaemon) => {
instead of
[IpfsNativeDaemon].forEach((IpfsDaemon) => {

But it times out 'waiting for peers'.

All other tests pass and I am using Node 6.1.0 like on the passing CircleCi build.
I have tried running on the same image as CircleCi uses for the passing test, but the image is too big for my little Docker workspace, but the same tests fail on other Ubuntu images.

what am I missing?

$ npm test

  108 passing (2m)
  2 failing

  1) orbit-db replication two peers replicates database of 1 entry:
     Error: Timeout of 40000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
  

  2) orbit-db replication two peers replicates database of 100 entries:
     Error: Timeout of 40000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

Edited: for more information

map/reduce and design docs

I've been through the documentation several times and have been unable to find any references on how to query data from the DB using map/reduce functions.

PouchDB example:

// create a design doc
var ddoc = {
  _id: '_design/index',
  views: {
    index: {
      map: function mapFun(doc) {
        if (doc.title) {
          emit(doc.title);
        }
      }.toString()
    }
  }
}

// save the design doc
db.put(ddoc).catch(function (err) {
  if (err.name !== 'conflict') {
    throw err;
  }
  // ignore if doc already exists
}).then(function () {
  // find docs where title === 'Lisa Says'
  return db.query('index', {
    key: 'Lisa Says',
    include_docs: true
  });
}).then(function (result) {
  // handle result
}).catch(function (err) {
  console.log(err);
});

Is there any way to do this in orbit-db?

Does not persist across devices

I was experimenting with this cool db.

It seems that I am connected
e.g. console: "Swarm listening on /libp2p-webrtc-star/dns4/star-signal.cloud.ipfs.team/wss/ipfs/QmTcaqq1eP44LwTqc6eVfXDP9iy5aVzaSsdVpyeUxdaBEU"

I can run db.put commands, it's definitely persisted. If I restart my application and do a db.load waiting for event 'ready', I can then read those previously stored documents.

I was curious and also set up a "synced" event, which never gets called. Can this event take more than 5mins to be called? (I am talking about 3 rows with 2 fields as a test)

Because if I go to a different device and start my application, do a db.load waiting for event 'ready' and then query, I don't receive any results.

Not sure If I missed anything? My understanding was that this will "eventually" get synced/stored on IPFS and thus if I open e.g. orbitdb.docstore('orbit.jens.dummyServices') I shall receive it again from anywhere. (I do read content on IPFS is stored with no guarantee unless I pin it) But I didn't read anything about pinning this orbitdb?

I would be happy to get help on this question. Maybe I missed a crucial part that I couldn't find in the github documentation

Thanks,
Jens

Orbit-DB as PouchDB backend

Hi,

I'm a big fan of the PouchDB project for the development of offline first apps but somewhat dislike its fundamental dependence on a Server backend.

I was really psyched to find out about your project and its ties to IPFS but am wondering whether you ever considered the possibility to implement Orbit-DB as a PouchDB backend.
This would spare you large parts of the development effort going into the JS database frontend and open up Orbit-DB to a much larger audience.

Looking forward to some feedback!

README.md example not working?

$ node orbit_test.js
/Users/romanvg/dev/cgt-vcfsearch/node_modules/orbit-db-pubsub/src/ipfs-pubsub.js:15
      this._ipfs.pubsub.sub(hash, { discover: true }, (err, stream) => {
                       ^

TypeError: Cannot read property 'sub' of undefined
    at IPFSPubsub.subscribe (/Users/romanvg/dev/cgt-vcfsearch/node_modules/orbit-db-pubsub/src/ipfs-pubsub.js:15:24)
    at OrbitDB._subscribe (/Users/romanvg/dev/cgt-vcfsearch/node_modules/orbit-db/src/OrbitDB.js:66:20)
    at OrbitDB._createStore (/Users/romanvg/dev/cgt-vcfsearch/node_modules/orbit-db/src/OrbitDB.js:57:17)
    at OrbitDB.eventlog (/Users/romanvg/dev/cgt-vcfsearch/node_modules/orbit-db/src/OrbitDB.js:29:17)
    at Object.<anonymous> (/Users/romanvg/dev/cgt-vcfsearch/orbit_test.js:18:20)
    at Module._compile (module.js:556:32)
    at Object.Module._extensions..js (module.js:565:10)
    at Module.load (module.js:473:32)
    at tryModuleLoad (module.js:432:12)
    at Function.Module._load (module.js:424:3)
    at Module.runMain (module.js:590:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

The other console-based examples under orbit-db/examples seem to work, except the browser demo for which I'm getting connection refused to IPFS even when other demos do succeed connecting to it.

Document iterator options in Eventlog and Feed

We have the iterator() function in eventlog and feed to query the database. It's documented here: https://github.com/orbitdb/orbit-db/blob/master/API.md#eventlogname. The iterator() takes in an options object that can be used to filter the query. However, the different options are not documented!

The possible options are here https://github.com/orbitdb/orbit-db-eventstore/blob/master/src/EventStore.js#L47 in the code.

They are:

  • limit (how many results to return)
  • gt/gte (greater than, greater or equal)
  • lt/lte (lower than, lower or equal than)

Leaving this issue here as a reminder for myself but if anyone wants to do a small and easy contribution, this one is up for grabs!

Dependencies on moving targets

While trying to narrow down #141 I have realized that ipfs-daemon 0.0.3 depends on the github master branch of js-ipfs-api that has a dev dependency on ipfs-daemon 0.2.3

I understand that it might be uncomfortable to release every version on npm, but I suggest to at least tag those dependencies on github and change the package.json to something like the following:

dependencies {
  ...
  "ipfs-api": "https://github.com/haadcode/js-ipfs-api.git#v11.1.0",
  ...
}

This helps reproducibility of issues, so it might save some time for both people maintaining the code and people using the code to track down where some problems appeared first.

Reusing orbit-db instance or ipfs repo / data/blockstore between tabs on the same browser

I've been playing around with js-ipfs and orbit-db for a bit, and I came across a particular use-case that I can't get working easily. Forgive me if I seem terribly confused, this is quite new to me.

Everything is working as expected for my test app: In order to persist orbit-db instances locally I am using the same unchanging repo-name for my js-ipfs node. And after I found that orbit-db instances only sync after the first new message is received from any newly discovered nodes I've implemented a basic "introduction" routine for cold-syncing newly re/initialized db instances.

But what I really need for my use-case is for these orbit-db instances to not only sync across different browsers but also on the same browser when opened in different tabs, while also having persistent data.

Pubsub doesn't work if a js-ipfs repo-name is shared on the same browser. Not only can they not communicate between themselves, only one of these orbit-db instances sync with instances on other browsers. So right now I am resorting to offsetting my base repo-name with an index for every new js-ipfs node after the first for my one app. That way, it behaves like a "foreign" instance with a distinct repo id and all.

But doing so duplicates a lot of data on the IndexDB, especially the 'blocks'. A lot of data that I feel like shouldn't need to be duplicated as they're replicated across all of the orbit-db instances anyway even on the browser-local siblings tabs stored under the same domain in IDB, just a on a differently named IDB object.

Is there no way to reuse the existing repo from the first loaded tab on the same browser and same domain on every other newly loaded tabs while also being able to pubsub between them? If not the repo then can we not at least reuse the datastore or blockstore (whichever is relevant,) to the same effect?

I suspect that this may be more of a question for js-ipfs than orbit-db, but the use-case feels closer to orbit-db's concerns, so I decided to create this issue here first.

Separate FilePreview from File in the client

Currently the file preview functionality is in client/src/components/File.js (handleClick()). It's getting out of hand and we should refactor the file preview to its own React component.

npm test fails

I did the following steps:

git clone https://github.com/haadcode/orbit-db.git
cd orbit-db
npm install
npm run build
npm test

Which spits out the following errors:

Orbit Client with js-ipfs-api
1) "before all" hook
(node:23742) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): TypeError: Cannot read property 'stopDaemon' of undefined
(node:23742) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
2) "after all" hook

106 passing (22s)
2 failing

  1. Orbit Client with js-ipfs-api "before all" hook:
    Error: ENOENT: no such file or directory, open '/tmp/orbit-db-tests/config'
    at Object.fs.openSync (fs.js:557:18)
    at Object.fs.readFileSync (fs.js:467:33)
    at parseConfig (node_modules/ipfsd-ctl/src/node.js:47:21)
    at Node.startDaemon (node_modules/ipfsd-ctl/src/node.js:127:5)
    at ipfsDaemon.init (node_modules/ipfs-daemon/ipfs-daemon.js:89:22)
    at configureNode (node_modules/ipfsd-ctl/src/node.js:99:11)
    at end (node_modules/run-series/index.js:8:15)
    at done (node_modules/run-series/index.js:11:10)
    at Stream.each (node_modules/run-series/index.js:16:43)
    at Socket.handleEnd (node_modules/duplexer/index.js:81:21)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickDomainCallback (internal/process/next_tick.js:122:9)

  2. Orbit Client with js-ipfs-api "after all" hook:
    Error: timeout of 20000ms exceeded. Ensure the done() callback is being called in this test.

npm ERR! Test failed. See above for more details.

Currently trying to run this from Bash for Windows 10

IIIF Annotation Demo for Vatican

@aeschylus and I are scheduled to do a demonstration of image annotation using IPFS at the upcoming IIIF conference in June: http://iiif.io/event/2017/vatican/ This will entail Drew adding annotations in Mirador, and them appearing in the Universal Viewer in real time.

We were wondering if it would be possible to have a chat with @dignifiedquire and @haadcode to discuss setting up a minimal example of this using Orbitdb. Perhaps a simple text field whose contents update on a localhost server for both of us as we edit? We can then build out the rest on top of that.

cc @flyingzumwalt

Trouble Running Browser example

Wanted to play around with orbit-db for a project but was having trouble getting the browser example working (though the Node.js example did work). I'm using Mac OS X with Node v6.4.0 and npm version 3.10.8. Would appreciate any assistance or feedback about what the problem may be! Thanks a ton

Here's the output when I try to execute npm run build:examples after npm install did not error:

> orbit-db@0.13.2 build:examples /Users/employee/Documents/gilbert/orbit-db
> webpack --config conf/webpack.example.config.js

Hash: fc0cf69b5c90511c787d
Version: webpack 2.1.0-beta.25
Time: 17644ms
                       Asset    Size  Chunks             Chunk Names
./examples/browser/bundle.js  282 kB       0  [emitted]  main
    + 151 hidden modules

ERROR in ./~/exports-loader?IpfsApi!./~/ipfs-api/dist/index.js
Module build failed: SyntaxError: Deleting local variable in strict mode (26454:4)

  26452 |   deps = (typeof ids === 'string') ? factory.slice(2) : ids.slice(2);
  26453 |   if(nodeJS) {
> 26454 |     delete define;
        |     ^
  26455 |     return tmpDefine.apply(null, Array.prototype.slice.call(arguments, 0));
  26456 |   }
  26457 |   define = tmpDefine;

 @ ./examples/browser/index.js 3:14-63

ERROR in ./~/orbit-db-eventstore/src/EventStore.js
Module not found: Error: Can't resolve 'lodash.slice' in '/Users/employee/Documents/gilbert/orbit-db/node_modules/orbit-db-eventstore/src'
 @ ./~/orbit-db-eventstore/src/EventStore.js 37:12-35
 @ ./src/OrbitDB.js
 @ ./examples/browser/index.js

ERROR in ./~/orbit-db-eventstore/src/EventStore.js
Module not found: Error: Can't resolve 'lodash.findindex' in '/Users/employee/Documents/gilbert/orbit-db/node_modules/orbit-db-eventstore/src'
 @ ./~/orbit-db-eventstore/src/EventStore.js 39:16-43
 @ ./src/OrbitDB.js
 @ ./examples/browser/index.js

ERROR in ./~/orbit-db-counterstore/src/CounterIndex.js
Module not found: Error: Can't resolve 'crdts/src/G-Counter' in '/Users/employee/Documents/gilbert/orbit-db/node_modules/orbit-db-counterstore/src'
 @ ./~/orbit-db-counterstore/src/CounterIndex.js 13:14-44
 @ ./~/orbit-db-counterstore/src/CounterStore.js
 @ ./src/OrbitDB.js
 @ ./examples/browser/index.js

ERROR in (webpack)/~/node-libs-browser/~/buffer/index.js
Module not found: Error: Can't resolve 'base64-js' in '/Users/employee/Documents/gilbert/orbit-db/node_modules/webpack/node_modules/node-libs-browser/node_modules/buffer'
 @ (webpack)/~/node-libs-browser/~/buffer/index.js 11:13-33
 @ ./~/ipfs-log/src/log.js
 @ ./~/orbit-db-store/src/Store.js
 @ ./~/orbit-db-eventstore/src/EventStore.js
 @ ./src/OrbitDB.js
 @ ./examples/browser/index.js

ERROR in (webpack)/~/node-libs-browser/~/buffer/index.js
Module not found: Error: Can't resolve 'ieee754' in '/Users/employee/Documents/gilbert/orbit-db/node_modules/webpack/node_modules/node-libs-browser/node_modules/buffer'
 @ (webpack)/~/node-libs-browser/~/buffer/index.js 12:14-32
 @ ./~/ipfs-log/src/log.js
 @ ./~/orbit-db-store/src/Store.js
 @ ./~/orbit-db-eventstore/src/EventStore.js
 @ ./src/OrbitDB.js
 @ ./examples/browser/index.js

ERROR in (webpack)/~/node-libs-browser/~/timers-browserify/main.js
Module not found: Error: Can't resolve 'process/browser.js' in '/Users/employee/Documents/gilbert/orbit-db/node_modules/webpack/node_modules/node-libs-browser/node_modules/timers-browserify'
 @ (webpack)/~/node-libs-browser/~/timers-browserify/main.js 1:15-44
 @ ./~/bluebird/js/browser/bluebird.js
 @ ./~/ipfs-log/src/log.js
 @ ./~/orbit-db-store/src/Store.js
 @ ./~/orbit-db-eventstore/src/EventStore.js
 @ ./src/OrbitDB.js
 @ ./examples/browser/index.js

npm ERR! Darwin 14.5.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "build:examples"
npm ERR! node v6.4.0
npm ERR! npm  v3.10.8
npm ERR! code ELIFECYCLE
npm ERR! orbit-db@0.13.2 build:examples: `webpack --config conf/webpack.example.config.js`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the orbit-db@0.13.2 build:examples script 'webpack --config conf/webpack.example.config.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the orbit-db package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     webpack --config conf/webpack.example.config.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs orbit-db
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls orbit-db
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/employee/Documents/gilbert/orbit-db/npm-debug.log

npm install don't work

Just did as described in README:

git clone....
cd orbit-db
npm install

Got a bunch of errors:

$ npm install
npm ERR! git rev-list -n1 feat/floodsub-rebase: fatal: ambiguous argument 'feat/floodsub-rebase': unknown revision or path not in the working tree.
npm ERR! git rev-list -n1 feat/floodsub-rebase: Use '--' to separate paths from revisions, like this:
npm ERR! git rev-list -n1 feat/floodsub-rebase: 'git <command> [<revision>...] -- [<file>...]'
npm ERR! git rev-list -n1 feat/floodsub-rebase: 
npm ERR! git rev-list -n1 feat/floodsub-rebase: fatal: ambiguous argument 'feat/floodsub-rebase': unknown revision or path not in the working tree.
npm ERR! git rev-list -n1 feat/floodsub-rebase: Use '--' to separate paths from revisions, like this:
npm ERR! git rev-list -n1 feat/floodsub-rebase: 'git <command> [<revision>...] -- [<file>...]'
npm ERR! git rev-list -n1 feat/floodsub-rebase: 
npm ERR! git clone --template=/home/user/.npm/_git-remotes/_templates --mirror [email protected]:ipfs/js-ipfs-api.git /home/user/.npm/_git-remotes/git-github-com-ipfs-js-ipfs-api-git-feat-floodsub-rebase-784cdc1e: Cloning into bare repository '/home/user/.npm/_git-remotes/git-github-com-ipfs-js-ipfs-api-git-feat-floodsub-rebase-784cdc1e'...
npm ERR! git clone --template=/home/user/.npm/_git-remotes/_templates --mirror [email protected]:ipfs/js-ipfs-api.git /home/user/.npm/_git-remotes/git-github-com-ipfs-js-ipfs-api-git-feat-floodsub-rebase-784cdc1e: Permission denied (publickey).
npm ERR! git clone --template=/home/user/.npm/_git-remotes/_templates --mirror [email protected]:ipfs/js-ipfs-api.git /home/user/.npm/_git-remotes/git-github-com-ipfs-js-ipfs-api-git-feat-floodsub-rebase-784cdc1e: fatal: Could not read from remote repository.
npm ERR! git clone --template=/home/user/.npm/_git-remotes/_templates --mirror [email protected]:ipfs/js-ipfs-api.git /home/user/.npm/_git-remotes/git-github-com-ipfs-js-ipfs-api-git-feat-floodsub-rebase-784cdc1e: 
npm ERR! git clone --template=/home/user/.npm/_git-remotes/_templates --mirror [email protected]:ipfs/js-ipfs-api.git /home/user/.npm/_git-remotes/git-github-com-ipfs-js-ipfs-api-git-feat-floodsub-rebase-784cdc1e: Please make sure you have the correct access rights
npm ERR! git clone --template=/home/user/.npm/_git-remotes/_templates --mirror [email protected]:ipfs/js-ipfs-api.git /home/user/.npm/_git-remotes/git-github-com-ipfs-js-ipfs-api-git-feat-floodsub-rebase-784cdc1e: and the repository exists.
npm ERR! Linux 4.8.0-45-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! node v6.10.1
npm ERR! npm  v3.10.10
npm ERR! code 128

npm ERR! Command failed: git clone --template=/home/user/.npm/_git-remotes/_templates --mirror [email protected]:ipfs/js-ipfs-api.git /home/user/.npm/_git-remotes/git-github-com-ipfs-js-ipfs-api-git-feat-floodsub-rebase-784cdc1e
npm ERR! Cloning into bare repository '/home/user/.npm/_git-remotes/git-github-com-ipfs-js-ipfs-api-git-feat-floodsub-rebase-784cdc1e'...
npm ERR! Permission denied (publickey).
npm ERR! fatal: Could not read from remote repository.
npm ERR! 
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR! 
npm ERR! 
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /home/user/Projects/orbit-db/npm-debug.log

Create docstore errors

I am using jsipfs + orbit-db

I tried to create a db as outlined:

const IPFS = require('ipfs')
const OrbitDB = require('orbit-db')

const ipfs = new IPFS()
const orbitdb = new OrbitDB(ipfs)
const docstore = orbitdb.docstore('db name')

This gave me an error - it said it was offline and a daemon needed to be started. So I used:

ipfs.on('ready', (e) => {
	const docstore = orbitdb.docstore('db name')
})

Now I get an error :

error: Unhandled Rejection at: Promise  Promise {                                                                                       
  <rejected> TypeError: Cannot read property 'listenerCount' of undefined                                                               
    at subscribe (/home/nmeadmin/cpxplane/node_modules/ipfs/src/core/components/pubsub.js:90:11)                                        
    at Promise (/home/nmeadmin/cpxplane/node_modules/ipfs/src/core/components/pubsub.js:23:11)                                          
    at Object.subscribe (/home/nmeadmin/cpxplane/node_modules/ipfs/src/core/components/pubsub.js:22:16)                                 
    at IPFSPubsub.subscribe (/home/nmeadmin/cpxplane/node_modules/orbit-db-pubsub/src/ipfs-pubsub.js:23:27)                             
    at OrbitDB._createStore (/home/nmeadmin/cpxplane/node_modules/orbit-db/src/OrbitDB.js:71:20)                                        
    at OrbitDB.docstore (/home/nmeadmin/cpxplane/node_modules/orbit-db/src/OrbitDB.js:41:17)                                            
    at IPFS.ipfs.on (/home/nmeadmin/cpxplane/src/ipfs/index.js:14:24)                                                                   
    at emitNone (events.js:86:13)                                                                                                       
    at IPFS.emit (events.js:188:7)                                                                                                      
    at done (/home/nmeadmin/cpxplane/node_modules/ipfs/src/core/boot.js:61:10) } TypeError: Cannot read property 'listenerCount' of unde
fined                                                                                                                                   
    at subscribe (/home/nmeadmin/cpxplane/node_modules/ipfs/src/core/components/pubsub.js:90:11)                                        
    at Promise (/home/nmeadmin/cpxplane/node_modules/ipfs/src/core/components/pubsub.js:23:11)                                          
    at Object.subscribe (/home/nmeadmin/cpxplane/node_modules/ipfs/src/core/components/pubsub.js:22:16)                                 
    at IPFSPubsub.subscribe (/home/nmeadmin/cpxplane/node_modules/orbit-db-pubsub/src/ipfs-pubsub.js:23:27)                             
    at OrbitDB._createStore (/home/nmeadmin/cpxplane/node_modules/orbit-db/src/OrbitDB.js:71:20)                                        
    at OrbitDB.docstore (/home/nmeadmin/cpxplane/node_modules/orbit-db/src/OrbitDB.js:41:17)                                            
    at IPFS.ipfs.on (/home/nmeadmin/cpxplane/src/ipfs/index.js:14:24)                                                                   
    at emitNone (events.js:86:13)                                                                                                       
    at IPFS.emit (events.js:188:7)                                                                                                      
    at done (/home/nmeadmin/cpxplane/node_modules/ipfs/src/core/boot.js:61:10)           

libstdc++ error

How can i install on centos 6?
When i try start app, i get Error: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by /home/admin/ipfs-node/node_modules/wrtc/build/wrtc/v0.0.61/Release/node-v48-linux-x64/wrtc.node)

The feed example returns 'undefined'

Using NodeJs v7.7.3

** My code **

const IPFS = require('ipfs-daemon/src/ipfs-node-daemon')
const OrbitDB = require('orbit-db')

const ipfs = new IPFS()

ipfs.on('error', (e) => console.error(e))
ipfs.on('ready', (e) => {
	const orbitdb = new OrbitDB(ipfs)
	const feed = orbitdb.feed('haad posts')
	feed.add({ title: 'Hello', content: 'World' })
	  .then(() => {
	    const posts = feed.iterator().collect()
	    posts.forEach((post) => console.log(post.title + '\n', post.content))    
	    // Hello 
	    // World  
	  })
})

output

Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/192.168.20.244/tcp/4001
undefined
 undefined

Both eventlog and documentstore return data. Yet the feed implementation doesn't.

Security & Permissions

This project looks really interesting, but I've been asking myself following things that I didn't see explained in the docs.

  • When I create a database with name global would other people that have access to IPFS also see the same data as I do in that global database?
  • How do I protect my data being overridden in a key store? If the database would be shared over IPFS then it would be hard to prevent that I imagine.

Hope there's someone who can answer these questions, or forward me to answers to similar questions.

Support write-permissioned databases

The goal here is to add write-permissioned databases to OrbitDB. It'll mean that we can have per-user databases or define a set of keys that can write to a given database.

Make orbit-server (redis) dependency optional

Great work on this package. I'm wondering about the redis dependency, though - theoretically, this could work e.g. with leveldb, correct? And, in that case, we would not need to run an orbit-server instance?

Ideas for demos for orbit-db

orbit-db starts to be in a shape that it's usable on various platforms and for various use cases. In order to show what orbit-db can do, we should have a bunch of cool demos that display orbit-db's capabilities and features.

One of them is Orbit, but I would like to see more and more varied ones. Another demo I made at some point (the code might not run anymore) is a twitter-like feed app https://github.com/haadcode/planet-express, but it was a hack and I haven't had the time to continue.

I would like to reach out to the community to source ideas as to what we could build to demo orbit-db?

What would you like to see? What do you propose to showcase the technology in a cool way? These could be either Node.js or in the browser.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.