Git Product home page Git Product logo

allocation-instrumenter's Introduction

The Allocation Instrumenter is a Java agent written using the java.lang.instrument API and ASM. Each allocation in your Java program is instrumented; a user-defined callback is invoked on each allocation.

How to get it

The latest release is available from Maven Central as:

<dependency>
  <groupId>com.google.code.java-allocation-instrumenter</groupId>
  <artifactId>java-allocation-instrumenter</artifactId>
  <version>3.3.4</version>
</dependency>

Basic usage

In order to write your own allocation tracking code, you have to implement the Sampler interface and pass an instance of that to AllocationRecorder.addSampler():

AllocationRecorder.addSampler(new Sampler() {
  public void sampleAllocation(int count, String desc, Object newObj, long size) {
    System.out.println("I just allocated the object " + newObj +
      " of type " + desc + " whose size is " + size);
    if (count != -1) { System.out.println("It's an array of size " + count); }
  }
});

You can also use the allocation instrumenter to instrument constructors of particular classes. You do this by instantiating a ConstructorCallback and passing it to ConstructorInstrumenter.instrumentClass():

try {
  ConstructorInstrumenter.instrumentClass(
      Thread.class, new ConstructorCallback<Thread>() {
        @Override public void sample(Thread t) {
          System.out.println("Instantiating a thread");
        }
      });
} catch (UnmodifiableClassException e) {
  System.out.println("Class cannot be modified");
}

For more information on how to get or use the allocation instrumenter, see Getting Started.

allocation-instrumenter's People

Contributors

cgdecker avatar cpovirk avatar cushon avatar inder123 avatar jhmanson avatar remkop avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

allocation-instrumenter's Issues

JDK11 Compilation Error

Hello,

I just wanted to compile the tool (mvn package) with JDK11 and mvn 3.6.1. It seems that it always has the compilation errors:

[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.680 s
[INFO] Finished at: 2019-12-25T21:13:34-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project java-allocation-instrumenter: Compilation failure: Compilation failure:
[ERROR] /home/bli11/allocation-instrumenter/target/generated-sources/java/com/google/monitoring/runtime/instrumentation/AllocationInstrumenterBootstrap.java:[7,24] cannot find symbol
[ERROR] symbol: class Generated
[ERROR] location: package javax.annotation
[ERROR] /home/bli11/allocation-instrumenter/target/generated-sources/java/com/google/monitoring/runtime/instrumentation/AllocationInstrumenterBootstrap.java:[33,2] cannot find symbol
[ERROR] symbol: class Generated
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

I have added some dependencies in pom.xml, but it doesn't work? Any suggestions? Thanks a lot!

Filter/limit instrumentation by package or class name?

Is there an option to limit instrumentation by package/class name (incl. wildcard)? Or to exclude packages which cause problems? Our program does not start if we use Google Guice and Eureka Discovery Client together.

1) Error injecting constructor, java.lang.RuntimeException: Failed to initialize DiscoveryClient!
  at com.netflix.discovery.DiscoveryClient.<init>(DiscoveryClient.java:299)
  at com.netflix.discovery.DiscoveryClient.class(DiscoveryClient.java:111)
  while locating com.netflix.discovery.DiscoveryClient

The only difference is the instrumenter JAR added as agent option to java:

-javaagent:/testdir/java-allocation-instrumenter-3.2.0.jar

Support arbitrarily named agent files.

The build currently requires the agent to be called the name it was given at build time. This results in awkward naming like -javaagent:agent-SNAPSHOT-3.0.jar, which isn't fun if you have to change your command line invocation whenever we upgrade version numbers.

NullPointerException in other shutdown hooks

We are working to make Log4j 2 garbage-free and we are using the Allocation Instrumenter in our unit tests to verify that we don't allocate objects during steady state logging.

We see our tests randomly fail with this exception:

Exception in thread "Thread-0" java.lang.NullPointerException
 at com.google.monitoring.runtime.instrumentation.AllocationRecorder.getObjectSize(AllocationRecorder.java:200)
 at com.google.monitoring.runtime.instrumentation.AllocationRecorder.recordAllocation(AllocationRecorder.java:244)
 at java.util.Hashtable.getEnumeration(Hashtable.java:665)
 at java.util.Hashtable.keys(Hashtable.java:329)
 at java.util.logging.LogManager$LoggerContext.getLoggerNames(LogManager.java:688)
 at java.util.logging.LogManager.reset(LogManager.java:1135)
 at java.util.logging.LogManager$Cleaner.run(LogManager.java:248)

I believe this is a bug in AllocationRecorder:

The NullPointerException above occurs when the (java.util.logging) LogManager$Cleaner shutdown hook allocates a new object and the instrumented bytecode invokes the recordAllocation method.

The AllocationRecorder also has a shutdown hook which sets its instrumentation field to null.
These shutdown hook threads can run at the same time, so if the instrumentation field is set to null after the null check (line 236) but before it is used in getObjectSize (line 200), we get the above exception.

One solution would be to introduce a local Instrumentation variable, assign the value of the static instrumentation field to the local variable, and do the null check on that local variable. If non-null, call the getObjectSize method with the local Instrumentation variable as one of the parameters. The getObjectSize method would use the passed Instrumentation parameter and should not access the static field.

allocation-instrumenter breaks JPMS compliant builds

The allocation-instrumenter uses the shade plugin to include some guava classes into its jar. Any project - such as Log4j 2 - that includes this tool as a test dependency will fail to build if they have other dependencies that require guava and have a module-info.java in their jar. The errors all resemble

[ERROR] error: module com.lmax.disruptor reads package com.google.thirdparty.publicsuffix from both guava and java.allocation.instrumenter
[ERROR] error: module org.jctools.core reads package com.google.thirdparty.publicsuffix from both guava and java.allocation.instrumenter
[ERROR] error: module org.yaml.snakeyaml reads package com.google.thirdparty.publicsuffix from both guava and java.allocation.instrumenter

Class loading breakage: Will not be able to instrument JDK classes

I did everything as described on your Getting Started page.

$ uname -a
Linux sweden 3.5.7-03050732-generic #201403120653 SMP Wed Mar 12 10:54:56 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
$ javac -version
javac 1.8.0_45
$ java -version
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

$ javac -classpath allocation-jdk8.jar Test.java
$ java -javaagent:allocation-jdk8.jar Test
Class loading breakage: Will not be able to instrument JDK classes

I downloaded the latest java-allocation-instrumenter-3.0.jar and renamed to allocation-jdk8.jar.

Interestingly enough, this was working very fine with the previous allocation.jar for JDK7. But now I need to make it work for JDK8 as well.

I am lost here, can you give me some help.

Thanks!

Capture the location where the new object is created

Hello,

May I know is there any way to capture the location where the new object is created with the memory allocation sampler (e.g. print the stack trace from a throwable in the constructor of new object)? It will be useful for further investigate.

Thanks!

Support JVM 21

Any plans to support Java 21?

Version 3.3.3 fails with:
Unsupported class file major version 65

This would probably require migrating to Asm 9.5

Failing to instrument classes due to needing ASM9 in Java stdlib

The upgrade of ASM to 9.2 in order to support JDK 17 does "work" - but, due to this library still specifying ASM7 when loading the Java Agent, some warnings happen:

WARNING: Failed to instrument class.
java.lang.UnsupportedOperationException: PermittedSubclasses requires ASM9
	at com.google.monitoring.runtime.instrumentation.asm.ClassVisitor.visitPermittedSubclass(ClassVisitor.java:281)
	at com.google.monitoring.runtime.instrumentation.asm.ClassReader.accept(ClassReader.java:706)
	at com.google.monitoring.runtime.instrumentation.asm.ClassReader.accept(ClassReader.java:424)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.instrument(AllocationInstrumenter.java:200)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.instrument(AllocationInstrumenter.java:220)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.transform(AllocationInstrumenter.java:174)
	at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
	at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
	at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
	at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
	at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:169)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.bootstrap(AllocationInstrumenter.java:157)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.premain(AllocationInstrumenter.java:109)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenterBootstrap.premain(AllocationInstrumenterBootstrap.java:48)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:491)
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:503)

Doing some quick debugging, this specific stack trace is when visiting java.lang.reflect.Constructor, which extends from a sealed class java.lang.reflect.Executable. What are the implications of this warning - is it just that non-instrumented classes will not show up as calls to the allocation sampler callback (i.e. all record classes and permitted subclasses will not show up)? And what are the blockers to using ASM8/9 over 7 - is there significant work involved in supporting additional class visitors, or could it be a simple version bump (not used ASM before)?

Dynamic load issue

Hi,

I'm using Allocation-instrumenter for some running Java process. Based on my understanding, the current allocation-instrumenter doesn't support dynamic load, because it only contains the premain method (please correct me if I'm wrong here).

To attach allocation-instrumenter to a running process, I created an agentmain method in AllocationInstrumenter.java. Now I can see that the Allocation-instrumenter's java agent has been attached to a running process successfully. However, the recordAllocation method won't be invoked during allocation. I'm wondering that does asm support this dynamic way to monitor allocations?

Thank you!

Support for jdk13?

It seems that the allocation instrumenter cannot work on jdk13. Can I know the reason? Like, some ASM APIs used in the allocation instrumenter aren't supported by jdk13 yet? Thank you!

Instrumenting constructors fails with VerificationError when interacting with classes that lack stack map frames

The Java Allocation Instrumenter passes the ClassWriter.COMPUTE_MAXS flag to the ASM ClassWriter during constructor instrumentation. According to the ASM Javadocs:

for classes whose version is Opcodes.V1_7 of more, this option requires valid stack map frames. The maximum stack size is then computed from these frames, and from the bytecode instructions in between. If stack map frames are not present or must be recomputed, used COMPUTE_FRAMES instead.

Other Java agents (such as the JaCoCo code coverage agent) may use ASM to write modified class files without computing stack map frames by passing 0 as a flag to the ClassWriter object. This flag is mapped to MethodWriter.COMPUTE_NOTHING which means that neither stack map frames nor the maximum stack size are computed. The interaction between agents which modify bytecode and do not recompute stack map frames and the Allocation Instrumenter leads to VerifyErrors when the JVM attempts to verify the twice-modified classes.

To improve the resiliency of the Allocation Instrumenter, we recommend passing the ClassWriter.COMPUTE_FRAMES flag to the ASM ClassWriter in place of the ClassWriter.COMPUTE_MAXS flag.

I have a minimal reproducible test case which has been used to verify that making the change suggested above does resolve the issue; I can provide you with it if necessary. In parallel, I am also opening an issue with the JaCoCo project to adjust the flags that they pass to their ClassWriter. However, in theory any bytecode-modifying agent could elect not to write stack map frames so it seems most effectively to address the issue here in the Allocation Instrumenter.

I don't understand how to integrate it in a standard Gradle-based project

Here is my configuration in build.gradle file:

plugins {
    id 'java'
}

group 'com.company.test'
version '1.0.0-SNAPSHOT'

sourceCompatibility = '11'

repositories {
    mavenCentral()
}

dependencies {
    implementation 'com.google.code.java-allocation-instrumenter:java-allocation-instrumenter:3.2.0'
    testCompile group: 'junit', name: 'junit', version: '4.12'
}

My code is the same as the example:

package com.company.test;

import com.google.monitoring.runtime.instrumentation.AllocationRecorder;

public final class App {
    public static void main(final String[] arg) {
        AllocationRecorder.addSampler((count, desc, newObj, size) -> {
            System.out.printf(
                    "I just allocated the object %s of type %s whose size is %d%n",
                    newObj,
                    desc,
                    size
            );
            if (count != -1) {
                System.out.println("It's an array of size " + count);
            }
        });
        for (int i = 0; i < 10; i++) {
            new String("foo");
        }
    }
}

But I don't understand what is the path/to/allocation.jar in this example:

% javac -classpath path/to/allocation.jar Test.java
% java -javaagent:path/to/allocation.jar Test

And what's the proper way to compile my application with Gradle and run it with the javaagent?

Thanks for help.

String created through java makeConcat not reported

Based on findings from bazelbuild/bazel#14890 (comment) looks like some string concatenations are not reported.

makeConcat was introduced in java 9 - see https://openjdk.java.net/jeps/280 and implementation in java 11 in
http://hg.openjdk.java.net/jdk/jdk11/file/jdk-11+28/src/java.base/share/classes/java/lang/invoke/StringConcatFactory.java#l1483

Possibly UNSAFE.allocateUninitializedArray calls done from
http://hg.openjdk.java.net/jdk/jdk11/file/jdk-11+28/src/java.base/share/classes/java/lang/invoke/StringConcatFactory.java#l1633 could be instrumented - but not sure how it will play with JVM machinery.

Current workaround seems to be -Djava.lang.invoke.stringConcat=MH_SB_SIZED that forces to use concat implementation that instrumenter is able to catch.

Support for JDK 11?

We are getting errors on JDK 11:

WARNING: Failed to instrument class.
java.lang.IllegalArgumentException
	at com.google.monitoring.runtime.instrumentation.asm.ClassReader.<init>(ClassReader.java:160)
	at com.google.monitoring.runtime.instrumentation.asm.ClassReader.<init>(ClassReader.java:143)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.instrument(AllocationInstrumenter.java:177)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.instrument(AllocationInstrumenter.java:210)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.transform(AllocationInstrumenter.java:157)
	at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:246)
	at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
	at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
	at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
	at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:167)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.bootstrap(AllocationInstrumenter.java:141)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.premain(AllocationInstrumenter.java:117)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenterBootstrap.premain(AllocationInstrumenterBootstrap.java:48)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
Oct 21, 2018 6:49:27 PM com.google.monitoring.runtime.instrumentation.AllocationInstrumenter instrument

RoaringBitmap/RoaringBitmap#277 (comment)

Will the instrumenter detect allocations made via JNI?

Hi,

Apologies in advance, this is a question rather than an issue - but I couldn't find a forum etc to ask the question, so here goes.

The instrumenter will instrument java code to identify allocations and invoke the appropriate callback.

However, if a java object is allocated from within JNI code - will this also be identified and the sampler callback invoked?

Thanks,
--Jatinder

Escape analysis

Am I correct in thinking that the instrumenter reports both on-heap and on-stack allocations? It seems that it has to, given that it's (in part) instrumenting calls to constructors — which should take place irrespective of the kind of allocation.

I ask in part because a bunch of comments in the code mention heap allocation but not on-stack allocation.

Is there a way to modify the instrumenter to only report on-heap allocations, which are the ones that are relevant to garbage collection pressure? (I think the answer is no, unfortunately.) I know one can use the -XX:+PrintEscapeAnalysis JVM flag but parsing the output appears rather tedious.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.