coatless-mac / r-macos-rtools Goto Github PK
View Code? Open in Web Editor NEWScripts to build an **unofficial** Rtools-esq installer for the macOS R toolchain
Scripts to build an **unofficial** Rtools-esq installer for the macOS R toolchain
As of R 3.4.* , the state of the toolchain changes with every release. The main component being changed is the compiler used to build the R CRAN binaries.
In particular, we have:
The gfortran
binaries are largely stable:
Going forward, we'll likely need to:
gfortran
installer used; andI get the same set of messages when I attempt to update the move package or install terra from GitHub (https://github.com/rspatial/terra) so it seems like there is a path problem somewhere but I can't tell where or how to fix.
Here's the start of the error messages.
- installing source package ‘move’ ...
** package ‘move’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include' -I/usr/local/include -fPIC -Wall -g -O2 -c bgbVar.c -o bgbVar.o
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include' -I/usr/local/include -fPIC -Wall -g -O2 -c bgbVarRcpp.cpp -o bgbVarRcpp.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include' -I/usr/local/include -fPIC -Wall -g -O2 -c bgb_bbmm.c -o bgb_bbmm.o
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include' -I/usr/local/include -fPIC -Wall -g -O2 -c emd.cpp -o emd.o
In file included from emd.cpp:1:
In file included from /Library/Frameworks/R.framework/Resources/include/R.h:50:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cmath:304:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/math.h:760:41: error: no member named 'fabsf' in the global namespace; did you mean 'labs'?
abs(float __lcpp_x) _NOEXCEPT {return ::fabsf(__lcpp_x);}
~~^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdlib.h:148:7: note: 'labs' declared here
long labs(long) __pure2;
^
In file included from emd.cpp:1:
In file included from /Library/Frameworks/R.framework/Resources/include/R.h:50:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cmath:304:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/math.h:764:42: error: no member named 'fabs' in the global namespace
abs(double __lcpp_x) _NOEXCEPT {return ::fabs(__lcpp_x);}
I guess the installer has to be updated for Mac OS Mojave.
It gets sucked in Running package scripts for 5+ min and then fails the installation.
It just suggest to contact the developer. Could you help me?
Hi,
I tried downloading the R toolchain for mac OS high sierra and it failed. I have the updated version of RStudio. Not sure why it won't work.
Thanks,
Hannah
everytime I try to install it through the installer for mac catalina I have "installation fail: installer encountered an error ... contact the software manufacturer ..)
Hello! I was unsuccessfully attempting to install rstanarm on Mac OSX 10.15.3, and I was instructed to use this toolchain to fix the issue I was encountering.
Unfortunately, after installing the C++ toolchain and attempting to install rstanarm in a fresh session, compilation failed reading the math.h header:
In file included from /usr/local/clang7/include/c++/v1/cmath:305:
/usr/local/clang7/include/c++/v1/math.h:301:15: fatal error: 'math.h' file not found
#include_next <math.h>
^~~~~~~~1 error generated.
make: *** [stan_files/bernoulli.o] Error 1
rm stan_files/bernoulli.cc
ERROR: compilation failed for package ‘rstanarm’
- removing ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rstanarm’
- restoring previous
‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rstanarm’
As the error suggests, there is a file at /usr/local/clang7/include/c++/v1/math.h that attempts to include another file called math.h on line 301. Does anyone have any idea how or why math.h might fail to read another file called math.h?
The goal here is to provide a second level of security related to #8 by checking the downloaded packages are signed.
Consider:
pkgutil --check-signature $PKG_INSTALLER_NAME
Package "macos-rtools.pkg":
Status: signed by a certificate trusted by Mac OS X
Certificate Chain:
1. Developer ID Installer: James Balamuta (LNVWJ4S7DC)
SHA1 fingerprint: 4B CF 66 88 56 AC 73 E8 70 14 5C 2C 7E BA E2 67 90 3C 85 BF
-----------------------------------------------------------------------------
2. Developer ID Certification Authority
SHA1 fingerprint: 3B 16 6C 3B 7D C4 B7 51 C9 FE 2A FA B9 13 56 41 E3 88 E1 86
-----------------------------------------------------------------------------
3. Apple Root CA
SHA1 fingerprint: 61 1E 5B 66 2C 59 3A 08 FF 58 D1 4A E2 24 52 D1 98 DF 6C 60
If status is not "signed by a certificate trusted by Mac OS X", then an exit is triggered.
Thus, we likely will want to verify the Developer ID (full name + ID):
1. Developer ID Installer: James Balamuta (LNVWJ4S7DC)
1 % 2 % 3% 4 % 5 % 6 % 7
The following extract out the name and the ID
PKG_AUTHOR=`pkgutil --check-signature $PKG_INSTALLER_NAME | awk /'Developer ID Installer/{print $5" "$6}'`
PKG_DEV_ID=`pkgutil --check-signature $PKG_INSTALLER_NAME | awk /'Developer ID Installer/{print $7}' | tr -d '()'`
In Simon's case, we have:
pkgutil --check-signature ~/Downloads/clang-6.0.0.pkg
Package "clang-6.0.0.pkg":
Status: signed by a certificate trusted by Mac OS X
Certificate Chain:
1. Developer ID Installer: Simon Urbanek (VZLD955F6P)
SHA1 fingerprint: 7B 6B 81 12 E6 26 8C 16 F8 D4 0F 94 E4 3E 62 69 2E 92 22 81
-----------------------------------------------------------------------------
2. Developer ID Certification Authority
SHA1 fingerprint: 3B 16 6C 3B 7D C4 B7 51 C9 FE 2A FA B9 13 56 41 E3 88 E1 86
-----------------------------------------------------------------------------
3. Apple Root CA
SHA1 fingerprint: 61 1E 5B 66 2C 59 3A 08 FF 58 D1 4A E2 24 52 D1 98 DF 6C 60
Alternatively, we probably should verify just the SHA1 fingerprint
if pkgutil --check-signature $PKG_INSTALLER_NAME | grep 'SHA1 fingerprint: 4B CF 66 88 56 AC 73 E8 70 14 5C 2C 7E BA E2 67 90 3C 85 BF'; then
echo 0
else
exit 1
fi
go.illinois.edu/
to generate a short linkREADME.md
The installation failed.
The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.
From stan-dev/rstan#716 (comment),
We need to verify write ability of ~/.R/Makevars
and ~/.Renviron
.
We'll likely need to change it from root
back to $USER
.
Address robustness with hardening
Consider adding a "bundled" offline installer by merging existing .pkg
builds
Details on improving the installer build process can be found inside of the 2019 MacAdmins conference talk by Rich Trouton.
Installer Package Scripting: Making your deployments easier, one ! at a time – Rich Trouton: https://www.youtube.com/watch?v=fIkh7RzrjqY
Slides for the talk are at:
https://bpb-us-e1.wpmucdn.com/sites.psu.edu/dist/4/24696/files/2019/07/psumac2019-345-Installer-Package-Scripting-Making-your-deployments-easier-one-at-a-time.pdf
And a cached copy can be found here:
psumac2019-345-Installer-Package-Scripting-Making-your-deployments-easier-one-at-a-time.pdf
Sent a tweet out to him as well:
Thanks a lot for creating this tool!
I found it necessary to add /usr/local/gfortran/bin
to the PATH in .Renviron
in order to compile packages containing fortran code. Before adding that to the path I got
make[1]: gfortran: No such file or directory
This was on macOS Catalina version 10.15.2 with a fresh R-devel 4.0.0 while attempting to install package ucminf
from CRAN.
It is entirely possible that I'm just missing something as I'm in over my head here, but thought I'd mention it.
Cheers
Rune
Thanks for the great installer. I just wondered for which macOS versions it works.
We recently upgraded a machine in our lab to macOS Monterey, and since then compiling R packages does no longer work. Is this installer supposed to work under Monterey? Or is it independent of the macOS version?
Thanks again,
Tim
clang6
installer doesn't automatically set the path for users... 😡
case $SHELL in
*/zsh)
shell_type="zsh"
;;
*/bash)
shell_type="bash"
;;
*)
exit 1 # error'd
esac
zsh
exists, then write to ~/.zshrc
otherwise, write to ~/.bash_profile or
/etc/path.d/clang6`/etc/path.d/
is preserved during a system upgrade compared to /etc/path
.In the ~/.bash_profile
or ~/.zshrc
, append to the end:
if [ -d "/usr/local/clang6/bin" ] ; then
export PATH="/usr/local/clang6/bin:$PATH"
fi
Consider including Java:
# Headless download
# curl ...
# This URL seems unstable.
# Then do a hdi mount/install/eject
# Run javareconf
R CMD javareconf
I can't install on my mac: Apple can't check for malicious software.
Thanks for writing this easy to use installer!
I upgraded R from 3.5.2 to 3.6.1 and used it to update a couple packages requiring clang and gfortran (data.table 1.12.4 and KernSmooth 2.23.16, respectively). However, by default the path to the gfortran installation was not added to the .Renviron file, and thus I was still getting the error "gfortran not found".
Adding it to the .Renviron file fixed this. It may be helpful to include writing this path to the .Renviron file as a default along with clang.
Hi,
I got it opened!
But the installation failed. I am on macOS Big Sur 11.0.1.
I took a screen shot:
Originally posted by @paprika920 in #39 (comment)
My MAC tells me it cannot open the package as it is unable to scan it for malicious software. It says the package needs to be updated.
From #23, I think we might have to notarize the installer app.
Details:
I may not have been paying attention while the installer was running and hence missed a notification or comment about this, but it tripped me up for a bit.
Specifically, I somehow missed that my old .Renviron was moved to .Renviron.bck and a new file created. It broke some other stuff that I wasn't expecting and it took me a while to piece together that this is what had happened.
With such an easy to use (and appreciated!) tool like this with everything packaged up into a macOS installer, it might make sense to be more explicit about what's going to happen to the contents of their .Renviron file. Again, it's possible I missed it because I was just clicking through the installer, but that might be the sort of user behavior you're going to see with a tool like this.
Thanks again for building this!
No documentation given for failure.
Per Simon's e-mail on R-SIG-MAC, we're going to implement a check that verifies the md5 hash of the file.
Sample function:
#1 Path to File
#2 md5 Hash to Check Against
check_md5_hash() {
# Obtain file hash
FILE_HASH=$(md5 $1 | awk '{print $4}')
# MD5 (path/to/file.pkg) = c29700c4e7b2914073ef7e741eb105bc
# 1 % 2 %3% 4
# Check against hash
if [ "$FILE_HASH" = "$2" ]; then
echo 0
else
exit 1
fi
}
Hard coded hashes:
2bec4fbae8d9caf6499c941ef9b87eae
457b59a6453069cf72dee9a4f9bf1b0a
1849cea667bb714c5c04a8565a9fe231
Not much info given on failed page
The installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.
I have already Xcode, Clang and Gfortran installed, could that be the problem?
Thank you
Need to show the installer isn't timing out just requiring more time...
Download and install the clang6.pkg
via:
curl -O https://cran.r-project.org/bin/macosx/tools/clang-6.0.0.pkg
sudo installer -pkg clang-6.0.0.pkg -target /
rm -rf clang-6.0.0.pkg
double clicking the installer pkg results in this message getting displayed:
“macos-rtools-3.2.2.pkg” can’t be opened because Apple cannot check it for malicious software.
I'm running macOS Catalina, 10.15.3
Probably should investigate testing the installation routine via the new GitHub Actions.
Here's the program from the coatless prof
#include <RcppArmadillo.h>
// [[Rcpp::depends(RcppArmadillo)]]
// [[Rcpp::export]]
void hello_world() {
Rcpp::Rcout << "Hello World!" << std::endl;
}
// After compile, this function will be immediately called using
// the below snippet and results will be sent to the R console.
/*** R
hello_world()
*/
Here's the beginning of the error messages
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppArmadillo/include" -I"/Users/gcn/Documents/workspace/ISIMIPData/temp" -I/usr/local/include -fPIC -Wall -g -O2 -c helloworld.cpp -o helloworld.o
In file included from helloworld.cpp:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppArmadillo/include/RcppArmadillo.h:31:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppArmadillo/include/RcppArmadilloForward.h:26:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include/RcppCommon.h:29:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include/Rcpp/r/headers.h:67:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include/Rcpp/platform/compiler.h:100:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cmath:304:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/math.h:760:41: error: no member named 'fabsf' in the global namespace
abs(float __lcpp_x) _NOEXCEPT {return ::fabsf(__lcpp_x);}
~~^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/math.h:764:42: error: no member named 'fabs' in the global namespace; did you mean 'abs'?
abs(double __lcpp_x) _NOEXCEPT {return ::fabs(__lcpp_x);}
~~^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/math.h:760:1: note: 'abs' declared here
abs(float __lcpp_x) _NOEXCEPT {return ::fabsf(__lcpp_x);}
^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/math.h:768:47: error: no member named 'fabsl' in the global namespace
abs(long double __lcpp_x) _NOEXCEPT {return ::fabsl(__lcpp_x);}
And the final error message is
make: *** [helloworld.o] Error 1
Error in Rcpp::sourceCpp("temp/helloworld.cpp") :
Error 1 occurred building shared library.
From @jimhester on rOpensci slack:
Why do you modify
~/.R/Makevars
rather than the site levelR_HOME/etc/Makevars.site
?
Makevars.site
value.README.md
about this?Incorporate in the headless Xcode CLI install for Mojave.
sudo installer -pkg \
/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg \
-target /
Avoid messing with Makevars
via:
sudo touch ~/.R/Makevars && sudo echo 'CPPFLAGS="-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include"' >> ~/.R/Makevars
c.f. https://twitter.com/axiomsofxyz/status/1045463113863180290, https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#macOS
From Dealing with Catalina and Dealing with Catalina II, we need to modify how the compiler is setup to ease the C++ exception issue.
@bgoodri seems to have found a good option here:
CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
Note, the -nostdinc++
is disabling standard directorys, c.f. https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html
-nostdinc
Do not search the standard system directories for header files. Only the directories explicitly specified with -I, -iquote, -isystem, and/or -idirafter options (and the directory of the current file, if appropriate) are searched.
-nostdinc++
Do not search for header files in the C++-specific standard directories, but do still search the other standard directories. (This option is used when building the C++ library.)
Poking softwareupdate
on macOS 10.14 gives:
* Command Line Tools (macOS Mojave version 10.14) for Xcode-10.3
# Command Line Tools (macOS Mojave version 10.14) for Xcode (10.3), 199140K [recommended]
With macOS Catalina 10.15, the output changed to:
* Label: Command Line Tools for Xcode-11.3
Title: Command Line Tools for Xcode, Version: 11.3, Size: 224878K, Recommended: YES,
We initially attempted to fix this by splitting on OS:
I think the issue here is we're using head
instead of tail
.
Elsewhere around the macOS community, homebrew
approaches the headless install with:
clt_label_command = "/usr/sbin/softwareupdate -l | " \
"grep -B 1 -E 'Command Line Tools' | " \
"awk -F'*' '/^ *\\*/ {print $2}' | " \
"sed -e 's/^ *Label: //' -e 's/^ *//' | " \
"sort -V | " \
"tail -n1"
clt_label = `#{clt_label_command}`.chomp
This is limited to macOS 10.13 and above.
Related to #16
From Stan user jroon
on their discourse setup:
Catalina 10.15.2
R 3.5.1
Does not adhere to:
# clang: start
SHLIB_CXXLDFLAGS+=-Wl,-rpath,${R_HOME}/lib ${R_HOME}/lib/libc++abi.1.dylib
SHLIB_CXX14LDFLAGS+=-Wl,-rpath,${R_HOME}/lib ${R_HOME}/lib/libc++abi.1.dylib
# clang: end
https://discourse.mc-stan.org/t/dealing-with-catalina-ii/11802/93
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.