Comments (11)
@tcfuji make sure that you have example files available fro docker environment.
from sparkling-water.
@mmalohlava Do you mean I need to download the examples from github?
Right now there is only:
ls examples/scripts/
Meetup20141217.md dlDemo.script
from sparkling-water.
@tcfuji which version of Sparkling Water are you using?
from sparkling-water.
I cloned it from github about 6 hours ago. I'm using the Dockerfile that
was included in that clone.
On Tue, Oct 20, 2015 at 6:12 PM Michal Malohlava [email protected]
wrote:
@tcfuji https://github.com/tcfuji which version of Sparkling Water are
you using?—
Reply to this email directly or view it on GitHub
#19 (comment)
.
from sparkling-water.
ahh,my fault, it is using old Sparkling Water version:
RUN curl -s http://d3kbcqa49mib13.cloudfront.net/spark-1.2.0-bin-cdh4.tgz | tar -xz -C /opt && \
ln -s /opt/spark-1.2.0-bin-cdh4 /opt/spark && \
mkdir /opt/spark/work && \
chmod 0777 /opt/spark/work
# Install Sparkling water latest version
RUN curl -s http://h2o-release.s3.amazonaws.com/sparkling-water/master/75/sparkling-water-0.2.10-75.zip --output sw.zip && \
unzip sw.zip -d /opt/ && \
ln -s /opt/sparkling-water-0.2.10-75 /opt/sparkling-water && \
If you edit it and replace spark by, for example:
http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.4.tgz
and sparkling water by:
http://h2o-release.s3.amazonaws.com/sparkling-water/rel-1.5/2/sparkling-water-1.5.2.zip
Please free to make PR.
from sparkling-water.
Updating the versions seems to have introduced memory allocation problems.
When running docker run -i -t --rm sparkling-test-base bin/run-example.sh
:
---------
Using example : org.apache.spark.examples.h2o.AirlinesWithWeatherDemo2
Using master (MASTER) : local-cluster[3,2,1024]
Deploy mode (DEPLOY_MODE) : client
Driver memory (DRIVER_MEMORY): 1G
H2O JVM options (H2O_SYS_OPS) :
---------
Spark Command: /usr/lib/jvm/java-7-oracle/jre/bin/java -cp /opt/spark/conf/:/opt/spark/lib/spark-assembly-1.5.1-hadoop2.4.0.jar:/opt/spark/lib/datanucleus-core-3.2.10.jar:/opt/spark/lib/datanucleus-rdbms-3.2.9.jar:/opt/spark/lib/datanucleus-api-jdo-3.2.6.jar -Xms1G -Xmx1G org.apache.spark.deploy.SparkSubmit --master local-cluster[3,2,1024] --deploy-mode client --conf spark.driver.memory=1G --conf spark.driver.extraJavaOptions= --class org.apache.spark.examples.h2o.AirlinesWithWeatherDemo2 /opt/sparkling-water-1.5.2/assembly/build/libs/sparkling-water-assembly-1.5.2-all.jar
========================================
15/10/21 02:35:39 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/10/21 02:35:44 WARN MetricsSystem: Using default name DAGScheduler for source because spark.app.id is not set.
15/10/21 02:35:45 WARN H2OContext: Increasing 'spark.locality.wait' to value 30000
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007fa71956d000, 12288, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory.
When running docker run -i -t --rm sparkling-test-base bin/sparkling-shell
:
-----
Spark master (MASTER) : local-cluster[3,2,2048]
Spark home (SPARK_HOME) : /opt/spark
H2O build version : 3.2.0.5 (slater)
Spark build version : 1.5.0
----
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000740000000, 2147483648, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 2147483648 bytes for committing reserved memory.
from sparkling-water.
@mmalohlava Should I make a PR anyway?
from sparkling-water.
Yes please 👍
from sparkling-water.
yes, please make a PR.
my guess is that the native mem oom might mean new flags to manage the
offheap DirectByteBuffers that might be needed in 1.5
thanks, sri
On Fri, Oct 23, 2015 at 1:00 PM, Ted [email protected] wrote:
@mmalohlava https://github.com/mmalohlava Should I make a PR anyway?
—
Reply to this email directly or view it on GitHub
#19 (comment)
.
culture . code . customer
c o m m u n i t y_, _h2o*.ai, *
+1-408.316.8192
from sparkling-water.
Done
from sparkling-water.
Hi tcfuji,
thanks for creating PR and sorry for so late response. We have actually created gradle task which creates docker file and determines latest version of sparkling-water automatically. It's gonna be in master and hopefully in a new sparkling-water release soon.
Kuba
from sparkling-water.
Related Issues (20)
- New issue template
- Avoid CVE-2019-10086 by upgrading MOJO2 lib
- Upgrade Spark in Testing Images to 3.4.1 and 3.2.4
- Migrate source of release notes from Jira to GitHub issues
- Upgrade to H2O 3.42.0.2
- CVE exposed via `ai.h2o:[email protected] › no.priv.garshol.duke:[email protected] ` HOT 2
- Get weights and biases from trained model
- Add Documentation Link for Spark 3.4 to Python Package Description
- Upgrade H2O to 3.42.0.3
- Upgrade Spark 3.4 docker base img to Spark 3.4.1 HOT 1
- Upgrade H2O to 3.42.0.4
- Fix Spark 3.4 Docker image building
- Add Spark 3.5 support
- Upgrade H2O to 3.44.0.1
- Add support for Python 3.10
- Upgrade Jetty to 9.4.53.v20231009
- Add support for Python 3.11
- Add support for Java 21 HOT 1
- Unable to save a Mojo Model using Java HOT 2
- Nexus upload problems
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sparkling-water.