Git Product home page Git Product logo

couchdb-2-elasticsearch's Introduction

An Elasticsearch CouchDB River replacement

Elasticsearch rivers have been deprecated, which makes one think about alternatives. The recommended method is using Logstash input plugins, but that feels a bit too heavy for my needs.

There's a node.js based solution which allows to perform one-off imports from CouchDB and continuous sync as well. The little tool is available as couch2elastic4sync and in its minimal usage only needs the source database url and the target index.

If that's enough for your needs, just head over to it. If you want to run it in a Docker container, feel free to use this repo, or directly run the image gesellix/river. A mapper has been added to make the load and sync actions ignore CouchDB design documents. See below for other options.

Usage

Sync a database from the last checkpoint (or from now), then stop:

docker run -e couch2elastic4sync_database=http://couchdb:5984/db-name -e couch2elastic4sync_elasticsearch=http://elastic:9200/index/type gesellix/river 

Continuously sync a database from the last checkpoint (or from now):

docker run -e couch2elastic4sync_database=http://couchdb:5984/db-name -e couch2elastic4sync_elasticsearch=http://elastic:9200/index/type -e couch2elastic4sync_end_on_catchup=false gesellix/river 

Index a complete database, without continuous sync:

docker run -e couch2elastic4sync_database=http://couchdb:5984/db-name -e couch2elastic4sync_elasticsearch=http://elastic:9200/index/type gesellix/river load 

Caution: the last example also indexes the design documents. To make them being ignored, just add the provided mapper like this:

 docker run -e couch2elastic4sync_database=http://couchdb:5984/db-name -e couch2elastic4sync_elasticsearch=http://elastic:9200/index/type -e couch2elastic4sync_mapper=drop_design_docs gesellix/river load

couchdb-2-elasticsearch's People

Contributors

gesellix avatar

Stargazers

Sam Lin avatar  avatar Julio Betta avatar micaksica avatar  avatar Danilo Oliveira avatar AhmeD HegazY avatar Ryan Ramage avatar

Watchers

hgc2002 avatar  avatar James Cloos avatar  avatar  avatar

couchdb-2-elasticsearch's Issues

ECONNREFUSED with CouchDb2 on HighSierra

Hi!
Thank you for this VERY useful tool.
I tried to migrate my couchdb to elastic but when i run

sudo docker run -e couch2elastic4sync_database=http://localhost:5984/jobs -e couch2elastic4sync_elasticsearch=http://localhost:9200/index/type -e couch2elastic4sync_mapper=drop_design_docs gesellix/river load

the script crashes with:

> couch2elastic4sync load

events.js:141
      throw er; // Unhandled 'error' event
      ^

Error: connect ECONNREFUSED 127.0.0.1:5984
    at Object.exports._errnoException (util.js:870:11)
    at exports._exceptionWithHostPort (util.js:893:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1063:14)

npm ERR! Linux 4.9.49-moby
npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "run" "load"
npm ERR! node v4.3.0
npm ERR! npm  v2.14.12
npm ERR! code ELIFECYCLE
npm ERR! [email protected] load: `couch2elastic4sync load`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] load script 'couch2elastic4sync load'.

CouchDb si reachable via curl or browser. Do you have an idea what could cause the issue please?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.