Coordinated Restore at Checkpoint is an OpenJDK feature that provides a fast start and immediate performance for Java applications.

A Java application and JVM are started from an image in a warmed-up form. The image is created from a running Java instance at arbitrary point of time ("checkpoint"). The start from the image ("restore") continues from the point when checkpoint was made.

The restore in general is faster than initialization. After the restore, Java runtime performance is also on-par with the one at the checkpoint. So, after proper warm-up before the checkpoint, restored Java instance is able to deliver the best runtime characteristics immediately.

Coordinated Restore undisruptively introduces new before-checkpoint and after-restore phases in Java application lifecycle. In contrast with uncoordinated checkpoint/restore, coordination allows restored Java applications to behave differently. For example, it is possible to react on changes in execution environment that happened since checkpoint was done.

CRaC implementation creates the checkpoint only if the whole Java instance state can be stored in the image. Resources like open files or sockets are cannot, so it is required to release them when checkpoint is made. CRaC emits notifications for an application to prepare for the checkpoint and return to operating state after restore.

Coordinated Restore is not tied to a particular checkpoint/restore implementation and will able to use existing ones (CRIU, docker checkpoint/restore) and ones yet to be developed.

• Results
• JDK
• User's flow
• Programmer's flow
  • API
    • jdk.crac
    • javax.crac
    • org.crac
• Examples
  • Tomcat / Sprint Boot
  • Quarkus
  • Micronaut
  • AWS Lambda
• Implemenation details

CRaC support was implemented in a few frameworks with next results. The source code can be found in the Examples section.

host$ docker build -t full-bench -f Dockerfile.full-bench utils
host$ docker run -it --privileged -v $HOME:$HOME -v $PWD:$PWD -w $PWD full-bench
cont# JDK=<path/to/jdk> bash ./utils/full-bench.sh collect
...
cont# exit
host$ bash ./utils/full-bench.sh parse
host$ cp *.data docs
host$ make -C docs

OpenJDK CRaC Project is developed in https://github.com/openjdk/crac.

Latest release can be found in https://crac.github.io/openjdk-builds/. The JDK archive should be extracted with sudo:

$ sudo tar zxf <jdk>.tar.gz

CRaC deployment scheme reflects the need to collect data required for Java application initialization and warm-up.

1. a Java application (or container) is deployed in the canary environment
   • the app processes canary requests that triggers class loading and JIT compilation
2. the running application is checkpointed by some mean
   • this creates the image of the JVM and application; the image is considered as a part of a new deployment bundle
3. the Java application with the image are deployed in the production environment
   • the restored Java process uses loaded classes from and JIT code from the immediately

WARNING: next is a proposal phase and is subject to change

Please refer to examples sections or step-by-step guide to get an application with CRaC support. The rest of the section is written for the spring-boot example.

For the first, Java command line parameter -XX:CRaCCheckpointTo=PATH defines a path to store the image and also allows the java instance to be checkpointed. By the current implementation, the image is a directory with image files. The directory will be created if it does not exist, but no parent directories are created.

export JAVA_HOME=./jdk
$JAVA_HOME/bin/java -XX:CRaCCheckpointTo=cr -jar target/spring-boot-0.0.1-SNAPSHOT.jar

For the second, in another console: supply canary worload ...

$ curl localhost:8080
Greetings from Spring Boot!

... and make a checkpoint by a jcmd command

$ jcmd target/spring-boot-0.0.1-SNAPSHOT.jar JDK.checkpoint
1563568:
Command executed successfully

Due to current jcmd implementation, success is always reported in jcmd output, problems are reported in the console of the application.

Another option to make the checkpoint is to invoke the jdk.crac.Core.checkpointRestore() method (see API). More options are possible in the future.

For the third, restore the cr image by -XX:CRaCRestoreFrom=PATH option

$JAVA_HOME/bin/java -XX:CRaCRestoreFrom=cr

Programs may need to be adjusted for use with Coordinated Restore at Checkpoint.

A step-by-step guide provides information on how to implement the CRaC support in the code.

Another option is to use an existing framework with CRaC support. There are a few frameworks available, an application need to be configured to use some of them. Possible configuration changes for applications are below.

• spring-boot
• quarkus
• micronaut

The CRaC API is not a part of Java SE specification. We hope that eventually it will be there, until then there are different packages that can be used.

• javadoc

This is the API that is implemented in the CRaC JDK.

Please refer to org.crac if you are looking to add CRaC support to a code that should also work on a regular JDK/JRE.

The package is a mirror of jdk.crac except the package name. It is available in javax-crac branch of CRaC JDK and in javax-crac release builds.

This is the API that will be proposed to inclussion into Java SE specification. Until then, the use of the package is discuraged.

The package is provided by org.crac compatibility library.

The org.crac is designed to provide smooth CRaC adoption. Users of the library can build against and use CRaC API on Java runtimes with jdk.crac, javax.crac, or without any implementation.

• In compile-time, org.crac package totally mirrors jdk.crac and javax.crac.
• In runtime, org.crac uses reflection to detect CRaC implementation. If the one is available, all requests to org.crac are passed to the implementation. Otherwise, requests are forwarded to a dummy implementation.

The dummy implementation allows an application to run but not to use CRaC:

• resources can be registered for notification,
• checkpoint request fails with an exception.

CRaC support in a framework allows small if any modification to applications using it. Proof-of concept CRaC support was implemented in a few third-party frameworks and libraries.

Source code links are below. Builds can be found in Maven Central under io.github.crac artifact-id.

• Tomcat with CRaC provides several modules
  • Maven Central
  • Changes are:
    • for Tomcat Embed (used by spring-boot-example below):
      • java/org/apache/tomcat/util/net/AbstractEndpoint.java
      • res/maven/tomcat-embed-core.pom
    • for JDBC Pool library:
      • modules/jdbc-pool/
    • for standalone Tomcat (Catalina):
      • java/org/apache/catalina/
      • java/org/apache/juli/
      • bin/, conf/
    • for general build infrastructure:
      • build.xml, build.properties.default
• Example-spring-boot is a sample Spring Boot applicaton using CRaC Tomcat
  • Build script changes
  • CI runs the application on CRaC

• Quarkus with CRaC
  • GitHub Packages
  • Changes are:
    • for managing vertx-http: extensions/vertx-http/
• Example-quarkus is a sample app on Quarkus with CRaC
  • Build script changes
  • CI

• Micronaut with CRaC
  • Maven Central
  • Changes are:
    • for managing Netty: http-server-netty/
    • for publishing artifacts: gradle.properties, gradle/*
• Example-micronaut is a sample app on Micronaut with CRaC
  • Build script changes
  • CI

• AWS Libs with CRaC
  • Maven Central
  • Changes to coordinate the connection with the service
• Example-lambda is a sample Java serverless function on CRaC
  • Changes to the official example
    • pom.xml, src/**.java -- the code
    • the rest is helpers for deployment in as the container function

Current OpenJDK implementation is based on using the CRIU project to create the image.

CRIU hosts a few changes made to improve CRaC usability.