installation problem about deep-learning-with-r-2nd-edition-code HOT 13 CLOSED

Seems I finally succeeded.
The trick was to select in RStudio options:
Python / Select / [automatically activate virtual environment ] ./virtualenvs/r-reticulate/bin/python3.9

Don't ask me why since I was expecting a miniconda envrt

At least

library(tensorflow)
library(keras)
mnist <- dataset_mnist()

is now downloading the data

from deep-learning-with-r-2nd-edition-code.

Are you on an M1 mac? If yes, then presently tensorflow::install_tensorflow() is only compatible with miniconda python, which you can install with reticulate::install_miniconda(). Additionally, you must be running an Arm native build of R, not the x86 build running under Rosetta.
Note the macOS tab section on https://tensorflow.rstudio.com/install/local_gpu:

You can test if you have an Arm build of R like this:

R.version$arch

If this returns "aarch64", you’re using the correct build of R. If instead it returns "x86_64", then you need to install a different build of R. You can install an Arm build of R by navigating to https://cloud.r-project.org/bin/macosx/ and selecting the appropriate package (e.g., “R-4.2.1-arm64.pkg”) or using an installation manager like rig and running rig add release.

This sequence of commands should work on an M1 mac:

reticulate::miniconda_uninstall() # start with a blank slate
reticulate::install_miniconda()
keras::install_keras()

from deep-learning-with-r-2nd-edition-code.

Thanks for answering.
I'm on an Intel Mac. I have a big GPU still, a AMD Radeon Pro Vega 48.
Does this question mean that you can't use tensorflow if you don't have a Nvidia card ?
I've seen conflicting info by people claiming they can tweak Cuda apps for Mac.
Anyway, this is not my concern here as I 'm using it for learning purposes, and am not looking for speed, and could accept to fall for CPU use.
I was wondering if I had rather a problem with my python version or a conflict with the tensorflow version

Here are the details of
sessionInfo()

R version 4.2.2 (2022-10-31)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Monterey 12.6.3

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] keras_2.11.0.9000 tensorflow_2.11.0 reticulate_1.28-9000

loaded via a namespace (and not attached):
[1] Rcpp_1.0.10 here_1.0.1 lattice_0.20-45 png_0.1-8 rprojroot_2.0.3 zeallot_0.1.0 withr_2.5.0 grid_4.2.2 R6_2.5.1
[10] jsonlite_1.8.4 magrittr_2.0.3 tfruns_1.5.1 cli_3.6.0 rlang_1.0.6 remotes_2.4.2 rstudioapi_0.14 whisker_0.4.1 Matrix_1.5-3
[19] generics_0.1.3 tools_4.2.2 compiler_4.2.2 base64enc_0.1-3

While

library(tensorflow)
tf$config$list_physical_devices("GPU")

gets

Error: Valid installation of TensorFlow not found.

Python environments searched for 'tensorflow' package:
/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11

Python exception encountered:
Traceback (most recent call last):
File "/Library/Frameworks/R.framework/Versions/4.2/Resources/library/reticulate/python/rpytools/loader.py", line 119, in _find_and_load_hook
return _run_hook(name, _hook)
^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/R.framework/Versions/4.2/Resources/library/reticulate/python/rpytools/loader.py", line 93, in _run_hook
module = hook()
^^^^^^
File "/Library/Frameworks/R.framework/Versions/4.2/Resources/library/reticulate/python/rpytools/loader.py", line 117, in _hook
return find_and_load(name, import)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'tensorflow'

I'm not familiar with the concept of python environments, so maybe the problem is here
conda_list()

      name                                                           python

1 base /Users/hseverac/Library/r-miniconda/bin/python
2 r-reticulate /Users/hseverac/Library/r-miniconda/envs/r-reticulate/bin/python

and
virtualenv_list()

[1] "r-reticulate"

from deep-learning-with-r-2nd-edition-code.

As a followup, as I believed the problem was with environments, I erased ~/.virtualenvs and /Users/hseverac/Library/r-miniconda then relaunched the installation

install.packages("remotes")
remotes::install_github(sprintf("rstudio/%s", c("reticulate", "tensorflow", "keras")))
reticulate::miniconda_uninstall() # start with a blank slate
reticulate::install_miniconda()
keras::install_keras()

Which proceeded successfully.

However I stilll get the same error

mnist <- dataset_mnist()
Error: Valid installation of TensorFlow not found.

If I try to force the environment I get this cryptic error message

use_condaenv("r-reticulate")

ERROR: The requested version of Python ('/Users/hseverac/Library/r-miniconda/envs/r-reticulate/bin/python') cannot be used, as another
version of Python ('/usr/local/bin/python3') has already been initialized. Please restart the R session if you need to attach reticulate
to a different version of Python.
Error in use_python(python, required = required) :
failed to initialize requested version of Python

I'm sorry to flood the thread with my Python problems, but I bought "Deep learning with R", and I m not supposed to know Python configuration astuces.

from deep-learning-with-r-2nd-edition-code.

I'm on an Intel Mac. I have a big GPU still, a AMD Radeon Pro Vega 48.

This is going to be a very uphill battle to get a GPU to work on this machine.

• TensorFlow hasn't supported GPUs on Intel Macs, officially or unofficially for years at this time.
• AMD GPUs are not officially supported by the TensorFlow project. I'm aware there is an external project attempting to add support to TensorFlow for AMD GPU's through ROCm, but last I looked, it's incomplete and not easy to configure.

Everything should work on your machine for CPU only tasks.
Note, your version of Python installed is too recent, the latest Tensorflow supports only up to Python 3.10, not 3.11.

If you want TensorFlow to use a GPU, unfortunately your best option is to install Linux and get an Nvidia GPU.

Note, all these issues are orthogonal to whether you're using TensorFlow from R or Python, you would encounter the same issues if you're working in Python.

from deep-learning-with-r-2nd-edition-code.

Thanks for answering me on the GPU part.
But as said, my main problem is that I bought a book advertised as Deep learning for R and I would like a minimal set of instruction guaranteeing I can use the example provided in the book on my machine (even on a CPU) without having to learn how to set up Python.
If Tensorflow can't work with Python 3.11, there should be someone providing the necessary configuration setup allowing it to work.
So far I spent maybe 5 hours using 2 sets of your recommandations :
here,
here

and it is not working.
Python is working on my machine, the packages are loaded so there is something wrong in your configuration process specific to tensorfow, and I would appreciate you help me to solve it.

sessionInfo()
R version 4.2.2 (2022-10-31)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Monterey 12.6.3
Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] reticulate_1.28-9000 keras_2.11.0.9000 tensorflow_2.11.0.9000

from deep-learning-with-r-2nd-edition-code.

ERROR: The requested version of Python ('/Users/hseverac/Library/r-miniconda/envs/r-reticulate/bin/python') cannot be used, as another
version of Python ('/usr/local/bin/python3') has already been initialized. Please restart the R session if you need to attach reticulate

This indicates that reticulate was already bound to a python interperter when use_condaenv() was called.

• Are you in a fresh R session? (Cmd/Ctrl + Shift + F10 in the RStudio IDE to start a fresh session)
• If this happens in a fresh R session, then you need to see what's causing reticulate to start up. Some likely candidates:
  • You're using R through an interface that is built on reticulate (e.g., radian)
  • You have something in initialization files somewhere, ~/.Rprofile, .Rprofile, or similar.

from deep-learning-with-r-2nd-edition-code.

Thanks for helping.

I've been using a fresh R session all along.
There is nothing in my .RProfile linked to Python.
The problem seems to be that I'm using Python 3.11 per default and I don't know how to change it.

library(reticulate)
use_python("/usr/local/bin/python3.9")

provides an error

ERROR: The requested version of Python ('/usr/local/bin/python3.9') cannot be used, as another version of Python
('/usr/local/bin/python3') has already been initialized. Please restart the R session if you need to attach reticulate to a different
version of Python.
Error in use_python("/usr/local/bin/python3.9") :
failed to initialize requested version of Python

from deep-learning-with-r-2nd-edition-code.

We're progressing. I remembered you could choose the python interpretor in RStudio options.
Which is now /usr/local/bin/python3.9.16

Now I get a new error

keras::install_keras()
Error in get(Info[i, 1], envir = env) :
lazy-load database '/Library/Frameworks/R.framework/Versions/4.2/Resources/library/tensorflow/R/tensorflow.rdb' is corrupt
In addition: Warning message:
In get(Info[i, 1], envir = env) : internal error -3 in R_decompress1

from deep-learning-with-r-2nd-edition-code.

ERROR: The requested version of Python ('/usr/local/bin/python3.9') cannot be used, as another version of Python
('/usr/local/bin/python3') has already been initialized. Please restart the R session if you need to attach reticulate to a different
version of Python.

This is indicating that something is still causing reticulate to initialize python before you interact with it. Are you activating the python repl in the RStudio IDE maybe? Some debugigng steps:

• Try outside the IDE, see if you get the same error
• Try launching R with a --vanilla flag

lazy-load database '/Library/Frameworks/R.framework/Versions/4.2/Resources/library/tensorflow/R/tensorflow.rdb' is corrupt

This happens if installation of the R package failed for some reason, most often due to being interrupted or due to file-locking contention. You can try to call remotes::install_github() again and that usually clears it up.

from deep-learning-with-r-2nd-edition-code.

I forced remote reinstallation through

install.packages("remotes")
remotes::install_github(sprintf("rstudio/%s", c("reticulate", "tensorflow", "keras")),force = TRUE)
reticulate::miniconda_uninstall() # start with a blank slate
reticulate::install_miniconda()
keras::install_keras()

which went OK
...except for the last line

keras::install_keras()
Error: .onLoad failed in loadNamespace() for 'tensorflow', details:
call: NULL
error: ModuleNotFoundError: No module named 'tensorflow'

and sessionInfo() does not give any package attached , but [22] reticulate loaded via a namespace

R version 4.2.2 (2022-10-31)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Monterey 12.6.3

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

loaded via a namespace (and not attached):
[1] Rcpp_1.0.10 compiler_4.2.2 prettyunits_1.1.1 base64enc_0.1-3 remotes_2.4.2 tools_4.2.2 zeallot_0.1.0
[8] pkgbuild_1.4.0 jsonlite_1.8.4 lattice_0.20-45 png_0.1-8 rlang_1.0.6 Matrix_1.5-3 cli_3.6.0
[15] rstudioapi_0.14 curl_5.0.0 withr_2.5.0 desc_1.4.2 generics_0.1.3 rprojroot_2.0.3 grid_4.2.2
[22] reticulate_1.28-9000 here_1.0.1 R6_2.5.1 processx_3.8.0 callr_3.7.3 magrittr_2.0.3 whisker_0.4.1
[29] ps_1.7.2 tfruns_1.5.1 crayon_1.5.2

from deep-learning-with-r-2nd-edition-code.

Hey, I have the same problem. Can you please tell me exactly what you did when you mentioning this:

"The trick was to select in RStudio options:
Python / Select / [automatically activate virtual environment ] ./virtualenvs/r-reticulate/bin/python3.9"

from deep-learning-with-r-2nd-edition-code.

Hi, the installation instruction (and underlying code) on https://rstudio.tensorflow.org/install were updated recently. To install, on all platforms, this should now be sufficient:

install.packages("remotes")
remotes::install_github("rstudio/keras")
reticulate::install_python()
keras::install_keras()

from deep-learning-with-r-2nd-edition-code.