spread0x / milan Goto Github PK

Milan is a data-oriented programming language and runtime infrastructure.

The Milan language is a DSL embedded in Scala. The output is an intermediate language that can be compiled to run on different target platforms. Currently there exists a single compiler that produces Flink applications.

The Milan language is similar in look and feel to other JVM-based streaming frameworks like Spark Streaming, Flink, or Kafka Streams. The main differences are that Milan uses higher-level constructs to build streaming applications, and that Milan applications are not tied to a specific runtime infrastructure.

Examples of the Milan language and how to build Milan applications using Maven and sbt are available in the samples.

Some of the language features are described in the docs.

Currently Milan has a single compiler, based on Apache Flink. It generates a Scala file that contains a Flink program that executes the Milan application.

Examples of how to integrate the compiler into a Maven or sbt build are available in the the samples.

Milan depends on the Flink Kinesis connector, which is not currently available on Maven central. You will need to build it yourself and install it into your local maven repo. The latest documentation on using the connector is available at https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kinesis.html .

1. Clone the Flink repository from https://github.com/apache/flink.
2. Checkout the version of Flink used by Milan: git checkout release-1.9
3. Build and install the snapshots for this release: mvn clean install -Punsafe-mapr-repo -Pinclude-kinesis -DskipTests
4. Any time the Flink version that Milan uses is changed you will need to repeat this process using the appropriate Flink release branch.

Please complete the steps in Environment Setup above before building for the first time.

mvn clean package will compile, run tests, and create jars of the Milan packages.

mvn clean install will compile, run tests, package, and install the snapshot version into your local maven repository. You can then use these from other projects.

This project is licensed under the Apache-2.0 License.