pndaproject / example-applications Goto Github PK

In examples that are using kafka (eg: spark-streaming) should pom.xml be updated to have 0.10.0.1 version of kafka clients? I believe that is what is being used in platform.

Ran into this issue trying to build spark-streaming application.

Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.6:single (make-assembly) on project spark-streaming-to-hbase-example-app: Execution make-assembly of goal org.apache.maven.plugins:maven-assembly-plugin:2.6:single failed: group id '1385651864' is too big ( > 2097151 ). Use STAR or POSIX extensions to overcome this limit -> [Help 1]

Fix seems to adding tarLongFileMode in appropriate pom.xml file - app-package/pom.xml in this case.

    ```
<configuration>
           <tarLongFileMode>posix</tarLongFileMode>
           <descriptors>
               <descriptor>src/main/assembly/assembly.xml</descriptor>
           </descriptors>
           <appendAssemblyId>false</appendAssemblyId>
        </configuration>

User guide is required for usage of Spark Magic.

Steps regarding usage of sparkmagic in spark kernel will be helpful in order to use various predefined magics. Details regarding livy connectivity to interact with spark can be added in steps which will help in creating and managing sessions.