Build native application using GraalVM about renjin HOT 12 CLOSED

Here's a shell script I used when attempting to build a native version of KeenWrite. Uncomment one of the last two lines, depending on whether GraalVM or BellSoft's NIK (recommended) is used to build.

#!/usr/bin/env bash

cd /tmp
git clone https://github.com/DaveJarvis/keenwrite
cd keenwrite
mkdir -p src/main/resources/META-INF/native-image
export LD_LIBRARY_PATH="$(pwd)/nik/lib:$LD_LIBRARY_PATH"

# Gradle and Java must be installed and available via the PATH.
gradle jar

wget "https://download.bell-sw.com/vm/22.3.0/bellsoft-liberica-vm-full-openjdk17.0.5+8-22.3.0+2-linux-amd64.tar.gz"
tar xf *gz
rm *gz
mv bell* nik

# Run the app and use many options.
java \
  -agentlib:native-image-agent=config-output-dir=src/main/resources/META-INF/native-image \
  --add-opens=javafx.controls/javafx.scene.control=ALL-UNNAMED \
  --add-opens=javafx.controls/javafx.scene.control.skin=ALL-UNNAMED \
  --add-opens=javafx.graphics/javafx.scene.text=ALL-UNNAMED \
  --add-opens=javafx.graphics/com.sun.javafx.css=ALL-UNNAMED \
  --add-opens=javafx.graphics/com.sun.javafx.text=ALL-UNNAMED \
  --add-exports=javafx.base/com.sun.javafx.event=ALL-UNNAMED \
  --add-exports=javafx.graphics/com.sun.javafx.application=ALL-UNNAMED \
  --add-exports=javafx.graphics/com.sun.javafx.geom=ALL-UNNAMED \
  --add-exports=javafx.graphics/com.sun.javafx.text=ALL-UNNAMED \
  --add-exports=javafx.graphics/com.sun.javafx.scene=ALL-UNNAMED \
  --add-exports=javafx.graphics/com.sun.javafx.scene.text=ALL-UNNAMED \
  --add-exports=javafx.graphics/com.sun.javafx.scene.traversal=ALL-UNNAMED \
  -jar build/libs/keenwrite.jar $@

# Fails due to missing reflective class lookup.
./nik/bin/native-image \
  --add-modules=javafx.controls,javafx.swing \
  --verbose \
  -H:+ReportExceptionStackTraces \
  --no-fallback \
  --report-unsupported-elements-at-runtime \
  -Djava.awt.headless=false \
  -cp src/main/resources/META-INF/native-image \
  -jar build/libs/keenwrite.jar

# ./graalvm/bin/gu install native-image
# ./graalvm/bin/native-image \

from renjin.

Which version of renjin are you using? Master and latest versions are built on Java 8. There is a Java 11 branch that might be more successful in attempting this.

from renjin.

From https://github.com/DaveJarvis/keenwrite/blob/master/build.gradle:

  // R
  implementation 'org.renjin:renjin-script-engine:3.5-beta76'
  implementation 'org.renjin.cran:rjson:0.2.15-renjin-21'

The application uses Java 19 and JavaFX 19.

Is there a more recent version? (Note: I tried building Renjin, but didn't want to pollute my dev environment with a different GCC version. It'd be great if the dependencies on a particular version could be eliminated.)

from renjin.

Interesting problem!

The offending class is created by Renjin's C/Fortran to JVM bytecode compiler. One of the recurring challenges has been that this compiler can generate very long methods. Sometimes this because the C or Fortran method is very long. I believe this is the guilty subroutine here:

https://github.com/wch/r-source/blob/431d589d479358a11ace2a0b7443fb519bf0ce05/src/library/stats/src/portsrc.f#L3995

Long functions are not a problem for machine code, but* many tools like OpenJDK's JIT or possibly the native imaging tools simply won't accept this kind of complexity becuase the optimization algorithms they are using have exponential (or worse) running time.

One solution we came up with this is to run the compiler's output through Soot, a JVM-bytecode optimizer. TBH I'm not sure if this is running in the beta3.5 branch.

from renjin.

You could also try removing this file from the source branch and rebuilding. (or excluding it from the jar file).

This might to lead to failures at runtime if you use these specific functions from the stats package, but should be ok if you're not using this particular part of the stats package.

from renjin.

should be ok if you're not using this particular part of the stats package.

KeenWrite has 10,000 downloads and no telemetry. I don't see a way for me to know whether that part is being used. I'm wondering:

• What would it take to get a version of Renjin released that doesn't have this issue (preferably with no loss in functionality)?
• Do you have build instructions that show how to build Renjin using GCC (4.x?) such that it doesn't install into /usr/local (i.e., non-polluting)?
• Is it possible to verify whether Soot was run on renjin-3.5-beta76? (I suspect it wasn't because if it was, the error shouldn't have appeared? I'm a little unclear on this point.)
• What/Where are the step-by-step instructions that show how to run the compiler's output through Soot?

I tried building Renjin a little while ago and found the steps to be a little convoluted without much in the way of documentation. Also, I really don't want to install an older version of GCC into the system. Installing GCC into $HOME/dev/renjin/gcc would be fine, but I couldn't see an easy way to do so.

(Aside, there seems to be a lot of duplication with respect to the GCC version number, which may make upgrading to a newer GCC version a bit of a chore.)

from renjin.

Set up a virtual environment e.g. using Vagrant. GCC 4.2 is installed there saving your environment from "pollution". Instructions can be found here: https://github.com/bedatadriven/renjin/blob/master/BUILDING.md

I had to make a few changes to get soot to work in Java 11 (j11-beta1 branch). It is integrated into the build i.e. runs automatically upon build, no ned to do anythig.

from renjin.

Thanks @perNyfelt . I'm not confident that the instructions will work on my machine (Arch Linux), nor that they are exhaustive enough for me to successfully create a build, nor do I want to install another virtual environment on my machine.

What would it take to get a version of Renjin released that doesn't have this issue (preferably with no loss in functionality)?

Ideally, I could change the following line to a new version:

implementation 'org.renjin:renjin-script-engine:3.5-beta76'

Thoughts?

from renjin.

from renjin.

I doubt there is any easy fix here.

Darn. This blocks giving native binaries to users for all platforms. I'll take a look at switching to FastR. Thanks for investigating.

from renjin.

Let me know what you find, I'm curious now. FastR is part of the GraalVM project, so it should be possible to compile R code to a native image. But it sounds like what you you're interested in is dynamically interpreting R code at runtime as your users include R expressions in text documents, at runtime. That seems by nature something very dynamic and not at all amenable to ahead of time compilation.

from renjin.

• https://medium.com/graalvm/faster-r-with-fastr-4b8db0e0dceb
• https://github.com/graalvm/examples/blob/master/r_java_embedding/src/main/java/com/oracle/truffle/r/fastrembedding/FastREmbeddingDemo.java

Context ctx = Context.newBuilder("R").allowAllAccess(true).build();
ctx.eval("R", "sum").execute(new int[] {1,2,3});

It looks like dynamic execution is possible.

from renjin.