Git Product home page Git Product logo

kiji-bento's Introduction

(c) Copyright 2013 WibiData, Inc.

Kiji BentoBox ${project.version}

The Kiji BentoBox is an SDK for developing Big Data Applications with the Kiji framework. It includes a complete set of Kiji framework modules, with compatible versions of each.

This kiji-bento release (${project.version}) includes:

  • kiji-schema ${kiji-schema.version}: Included at the top-level of the distribution, kiji-schema provides a simple Java API and command-line tools for storing and managing typed data in HBase.
  • kiji-mapreduce ${kiji-mapreduce.version}: Included in the lib directory, kiji-mapreduce provides a simple Java API and command-line tools for using Hadoop MapReduce to import and process data in Kiji tables.
  • kiji-mapreduce-lib ${kiji-mapreduce-lib.version}: Included in the lib directory, kiji-mapreduce-lib is a Java library of utilities for writing MapReduce jobs on Kiji tables, as well as ready to use producers, gatherers, and importers.
  • bento-cluster ${bento-cluster.version}: Located in the cluster directory, bento-cluster allows users to run HDFS, MapReduce, and HBase clusters easily on the local machine.
  • kiji-schema-shell ${kiji-schema-shell.version}: Included in the schema-shell directory, kiji-schema-shell provides a layout definition language for use with kiji-schema.
  • kiji-phonebook ${kiji-phonebook.version}: Included in the examples/phonebook directory, kiji-phonebook is an example standalone application (with source code) that stores, processes, and retrieves data from a Kiji instance.
  • kiji-music ${kiji-music.version}: Included in the examples/music directory, kiji-music is an example of loading the listening history of users of a music service into a Kiji table, and then generating new music recommendations.
  • API documentation is made available in the docs directory.

Installation

Untar kiji-bento

Untar your kiji-bento distribution into a location convenient for you. We suggest your home directory. In the future we'll call the path to your kiji-bento distribution $KIJI_HOME.

Configure your system to use kiji-bento.

kiji-bento includes a script that will configure your environment to use the HDFS, MapReduce and HBase clusters managed by the included bento-cluster distribution, as well as kiji and kiji-schema-shell. To configure your environment, run:

source $KIJI_HOME/bin/kiji-env.sh

Add this line to your ~/.bashrc file to ensure your environment is always configured for kiji-bento.

kiji-env.sh configures your environment to use the Hadoop ecosystem distributions included with bento-cluster, as well as the clusters we'll soon use bento-cluster to start. It also adds the hadoop, hbase, bento, kiji and kiji-schema-shell scripts to your $PATH, making them available as top-level commands.

Start bento-cluster.

Now that your system is configured, you can use the bento script to start your local HDFS, MapReduce, and HBase clusters.

bento start

Use kiji

With the clusters started, you can run Hadoop ecosystem tools like kiji. For example, to list your home directory in HDFS, run:

hadoop fs -ls

Or, to install the default kiji instance in HBase, run:

kiji install

The hadoop, hbase, kiji and kiji-schema-shell scripts are available for use. You can also use other Hadoop ecosystem tools like hive or pig and they will use the local clusters managed by bento-cluster when run in an environment configured with kiji-env.sh.

Stop bento-cluster.

When you're ready to call it a day, you can stop bento-cluster by running:

bento stop

This will shutdown the HBase, MapReduce, and HDFS clusters managed by bento-cluster.

Quickstart

To continue using Kiji, consult the online quickstart guide.

Upgrade Server Check-in

Kiji BentoBox will periodically check in with an upgrade server to determine if there are any upgrades available for your distribution. If upgrades are available, the kiji script that comes with BentoBox will periodically remind you (once a day) of available upgrades. BentoBox sends anonymized information when checking in with the upgrade server.

To disable checking in with the upgrade server, write a file named disable-checkin to the conf directory that is part of the BentoBox distribution. Using the touch command is sufficient. For example:

touch $KIJI_HOME/conf/disable-checkin

will disable check in with the upgrade server.

kiji-bento's People

Contributors

robotoer avatar

Watchers

James Cloos avatar 郭芸 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.