Git Product home page Git Product logo

Comments (11)

mmalohlava avatar mmalohlava commented on May 24, 2024

@tcfuji make sure that you have example files available fro docker environment.

from sparkling-water.

tcfuji avatar tcfuji commented on May 24, 2024

@mmalohlava Do you mean I need to download the examples from github?

Right now there is only:

ls examples/scripts/
Meetup20141217.md  dlDemo.script

from sparkling-water.

mmalohlava avatar mmalohlava commented on May 24, 2024

@tcfuji which version of Sparkling Water are you using?

from sparkling-water.

tcfuji avatar tcfuji commented on May 24, 2024

I cloned it from github about 6 hours ago. I'm using the Dockerfile that
was included in that clone.

On Tue, Oct 20, 2015 at 6:12 PM Michal Malohlava [email protected]
wrote:

@tcfuji https://github.com/tcfuji which version of Sparkling Water are
you using?


Reply to this email directly or view it on GitHub
#19 (comment)
.

from sparkling-water.

mmalohlava avatar mmalohlava commented on May 24, 2024

ahh,my fault, it is using old Sparkling Water version:

RUN curl -s http://d3kbcqa49mib13.cloudfront.net/spark-1.2.0-bin-cdh4.tgz | tar -xz -C /opt && \
    ln -s /opt/spark-1.2.0-bin-cdh4 /opt/spark && \
    mkdir /opt/spark/work && \
    chmod 0777 /opt/spark/work

# Install Sparkling water latest version
RUN curl -s http://h2o-release.s3.amazonaws.com/sparkling-water/master/75/sparkling-water-0.2.10-75.zip --output sw.zip && \
  unzip sw.zip -d /opt/ && \
  ln -s /opt/sparkling-water-0.2.10-75 /opt/sparkling-water && \

If you edit it and replace spark by, for example:
http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.4.tgz

and sparkling water by:
http://h2o-release.s3.amazonaws.com/sparkling-water/rel-1.5/2/sparkling-water-1.5.2.zip

Please free to make PR.

from sparkling-water.

tcfuji avatar tcfuji commented on May 24, 2024

Updating the versions seems to have introduced memory allocation problems.
When running docker run -i -t --rm sparkling-test-base bin/run-example.sh:

---------
  Using example                  : org.apache.spark.examples.h2o.AirlinesWithWeatherDemo2
  Using master    (MASTER)       : local-cluster[3,2,1024]
  Deploy mode     (DEPLOY_MODE)  : client
  Driver memory   (DRIVER_MEMORY): 1G
  H2O JVM options (H2O_SYS_OPS)  :
---------
Spark Command: /usr/lib/jvm/java-7-oracle/jre/bin/java -cp /opt/spark/conf/:/opt/spark/lib/spark-assembly-1.5.1-hadoop2.4.0.jar:/opt/spark/lib/datanucleus-core-3.2.10.jar:/opt/spark/lib/datanucleus-rdbms-3.2.9.jar:/opt/spark/lib/datanucleus-api-jdo-3.2.6.jar -Xms1G -Xmx1G org.apache.spark.deploy.SparkSubmit --master local-cluster[3,2,1024] --deploy-mode client --conf spark.driver.memory=1G --conf spark.driver.extraJavaOptions= --class org.apache.spark.examples.h2o.AirlinesWithWeatherDemo2 /opt/sparkling-water-1.5.2/assembly/build/libs/sparkling-water-assembly-1.5.2-all.jar
========================================
15/10/21 02:35:39 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/10/21 02:35:44 WARN MetricsSystem: Using default name DAGScheduler for source because spark.app.id is not set.
15/10/21 02:35:45 WARN H2OContext: Increasing 'spark.locality.wait' to value 30000
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007fa71956d000, 12288, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory.

When running docker run -i -t --rm sparkling-test-base bin/sparkling-shell:

-----
  Spark master (MASTER)     : local-cluster[3,2,2048]
  Spark home   (SPARK_HOME) : /opt/spark
  H2O build version         : 3.2.0.5 (slater)
  Spark build version       : 1.5.0
----

Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000740000000, 2147483648, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 2147483648 bytes for committing reserved memory.

from sparkling-water.

tcfuji avatar tcfuji commented on May 24, 2024

@mmalohlava Should I make a PR anyway?

from sparkling-water.

mmalohlava avatar mmalohlava commented on May 24, 2024

Yes please 👍

from sparkling-water.

srisatish avatar srisatish commented on May 24, 2024

yes, please make a PR.

my guess is that the native mem oom might mean new flags to manage the
offheap DirectByteBuffers that might be needed in 1.5

thanks, sri

On Fri, Oct 23, 2015 at 1:00 PM, Ted [email protected] wrote:

@mmalohlava https://github.com/mmalohlava Should I make a PR anyway?


Reply to this email directly or view it on GitHub
#19 (comment)
.

culture . code . customer
c o m m u n i t y_, _h2o*.ai, *
+1-408.316.8192

from sparkling-water.

tcfuji avatar tcfuji commented on May 24, 2024

Done

from sparkling-water.

jakubhava avatar jakubhava commented on May 24, 2024

Hi tcfuji,
thanks for creating PR and sorry for so late response. We have actually created gradle task which creates docker file and determines latest version of sparkling-water automatically. It's gonna be in master and hopefully in a new sparkling-water release soon.

Kuba

from sparkling-water.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.