Trouble creating h2o_context - R about sparkling-water HOT 8 CLOSED

Hi @acorso522 ,

can you please provide log with more details?

btw: I expect you call library(rsparkling) before calling spark_connect.

Thx,
michal

from sparkling-water.

Hi @acorso522 , @tarqui

thank you for bug report! It was fixed in master - by the following commit:
abd1bbd

Can you pleas try to upgrade your Rsparkling library: devtools::install_github("h2oai/sparkling-water", ref = "master", subdir = "/r/rsparkling", force=TRUE)?

Thank you!
Michal

from sparkling-water.

Sorry, one more update:

You should setup version of Sparkling Water package:
options(rsparkling.sparklingwater.version = "2.0.0")

from sparkling-water.

Hi @mmalohlava ,

I went back and disconnected from my local spark instance and then reconnected, such that the connection was initiated after running library(rsparkling), and it appears to work now. Wasn't aware of how linear this process needed to be.

Thanks!

from sparkling-water.

Hi,

I have the same problem. Running this code

Sys.setenv("SPARK_HOME"="/usr/local/spark")
library(sparklyr)
library(dplyr)
sc <- spark_connect(master = "local")

library(rsparkling)
library(h2o)
mtcars_tbl <- copy_to(sc, mtcars, "mtcars", overwrite = TRUE)

partitions <- mtcars_tbl %>%
  filter(hp >= 100) %>%
  mutate(cyl8 = cyl == 8) %>%
  sdf_partition(training = 0.5, test = 0.5, seed = 1099)

training <- as_h2o_frame(sc, partitions$training)

I got the same error:

Error: failed to invoke spark command
16/11/09 22:55:23 INFO SparkSqlParser: Parsing command: SELECT *
16/11/09 22:55:23 ERROR getOrCreate on org.apache.spark.h2o.H2OContext failed

My session info:

> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.10

locale:
 [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_GB.UTF-8        LC_COLLATE=en_GB.UTF-8    
 [5] LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_GB.UTF-8   
 [7] LC_PAPER=en_GB.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rsparkling_0.1.0 h2o_3.10.0.8     statmod_1.4.26   dplyr_0.5.0     
[5] sparklyr_0.4.23 

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.7     withr_1.0.2     digest_0.6.10   rprojroot_1.1  
 [5] assertthat_0.1  bitops_1.0-6    R6_2.2.0        jsonlite_1.1   
 [9] DBI_0.5-1       backports_1.0.4 magrittr_1.5    lazyeval_0.2.0 
[13] config_0.2      tools_3.3.2     readr_1.0.0     RCurl_1.95-4.8 
[17] parallel_3.3.2  yaml_2.1.13     tibble_1.2   

The only difference with the example code in http://spark.rstudio.com/h2o.html is the version. I am using h2o_3.10.0.8, while in the link uses h2o_3.10.0.6

I do not know where I can find the log for more information

Regards

from sparkling-water.

Thank you @mmalohlava

I got the same error,

I am using:

> spark_version(sc) [1] ‘2.0.1’

If you could tell me where can I see log, I will attach them

from sparkling-water.

Sorry my awkwardness. Where should I setup that option in my code?

from sparkling-water.

@tarqui you need to setup option before calling spark_connect, at the current moment it is recommended way to pass extra information to sparklyr package which handle spark and attaching custom java packages, like sparkling water. If "2.0.0" would not work, try "2.0.1". Feel free to reopen issue if that wouldn't help you.

from sparkling-water.