prographer / hazelcast-jet Goto Github PK

Hazelcast Jet is a distributed computing platform built for high-performance stream processing and fast batch processing. It embeds Hazelcast In Memory Data Grid (IMDG) to provide a lightweight package of a processor and a scalable in-memory storage.

Visit jet.hazelcast.org to learn more about the architecture and use-cases.

• Low latency and distributed general data processing framework with high throughput.
• Highly parallel and distributed stream and batch processing of data.
• Distributed java.util.stream API support for Hazelcast data structures such as IMap and IList.
• Connectors allowing high-velocity ingestion of data from Apache Kafka, HDFS, Hazelcast IMDG, sockets and local data files (such as logs or CSVs)
• API for custom connectors
• Distributed implementations of java.util.{Queue, Set, List, Map} data structures highly optimized to be used for the processing
• Dynamic node discovery for both on-premise and cloud deployments.
• Virtualization support and resource management via Docker, YARN and Mesos

See the Getting Started Guide.

See Hazelcast Jet Code Samples for some examples.

See the Jet Reference Manual.

See Jet Architecture.

See the write up of our high performance secret source.

Download from jet.hazelcast.org.

Use Maven snippet:

<dependency>
    <groupId>com.hazelcast.jet</groupId>
    <artifactId>hazelcast-jet</artifactId>
    <version>${hazelcast.jet.version}</version>
</dependency>
<repository>
    <id>cloudbees-release-repository</id>
    <url>https://repository-hazelcast-l337.forge.cloudbees.com/release/</url>
</repository>

Maven snippet:

<dependency>
    <groupId>com.hazelcast.jet</groupId>
    <artifactId>hazelcast-jet</artifactId>
    <version>${hazelcast.jet.version}</version>
</dependency>
<repository>
    <id>cloudbees-snapshot-repository</id>
    <url>https://repository-hazelcast-l337.forge.cloudbees.com/snapshot/</url>
    <releases>
        <enabled>false</enabled>
    </releases>
    <snapshots>
        <enabled>true</enabled>
    </snapshots>
</repository>

Pull latest from repo git pull origin master and use Maven install (or package) to build mvn clean install.

Type mvn test to run unit and integration tests.

In each Pull Request, we do static analyzing on the changes. Run the following commands locally to check if your contribution is checkstyle and findbugs compatible.

mvn clean compile -P findbugs

mvn clean validate -P checkstyle

We encourage pull requests and process them promptly.

To contribute:

• see Developing with Git for our Git process
• complete the Hazelcast Contributor Agreement

For an enhancement or larger feature, create a GitHub issue first to discuss.

Please join the mail group if you are interested in using or developing Hazelcast.

http://groups.google.com/group/hazelcast-jet

Hazelcast Jet is available under the Apache 2 License. Please see the Licensing section for more information.

Copyright (c) 2008-2017, Hazelcast, Inc. All Rights Reserved.

Visit www.hazelcast.com for more info.