anamorphosee / stacktrace-decoroutinator Goto Github PK

I'm seeing an exception when trying to use this neat lib in Jars packaged by Spring Boot's Gradle Plugin.

Exception in thread "main" java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
Caused by: java.lang.NullPointerException: classBodyStream must not be null
	at dev.reformator.stacktracedecoroutinator.jvmagentcommon.Utils_jvm_agent_commonKt.addDecoroutinatorClassFileTransformers(utils-jvm-agent-common.kt:46)
	at dev.reformator.stacktracedecoroutinator.runtime.DecoroutinatorRuntime.load(runtime-jvm.kt:26)
	at com.example.demo.DemoApplicationKt.main(DemoApplication.kt:11)
	... 8 more

Minimum steps to reproduce:

1. Go to https://start.spring.io/ and generate an all-Kotlin, Gradle project using no dependencies
   
2. Download it, add Decoroutinator
3. Build the jar with ./gradlew build or ./gradlew bootJar
4. Run it with java -jar build/libs/demo-0.0.1-SNAPSHOT.jar

You can find an example project here: https://github.com/napstr/stacktrace-decoroutinator-spring-boot-npe

Running the project from my IDE, or via ./gradlew bootRun works fine.

Spring Boot does some classloader shenaningans when packaging the Jar, maybe that's the cause?

Great library!

Have you considered creating no-op version of it so we can do

debugImplementation("realVersion") 
releaseImplementation("noopVersion")

It's pretty standard practice on android to avoid bringing unnecessary libraries to release build

minSdk = 26 covers 80% of the devices according to play market stats. While it might work for some projects, it will not work for others. Is there a specific reason why it has to be 26? Looking at the code, I guess it can be easily lowered to something like 21 which would make this library easier to use.

Currently you have to add a pesky

    <uses-sdk
        android:minSdkVersion="24"
        tools:overrideLibrary="dev.reformator.stacktracedecoroutinator"
        />

in order to use this library in android project which needs lower minSdk

It's not really convenient that you bring the default application impl with your android library. It's very rare for real-life project to don't have a chosen way of init already, usually via it's own Application class or via jetpack app-startup.

Not only it's unnecessary and mostly useless piece of code, it also forces usage of tricks then this library is used in your manifest - adding obscure tools:replace="android:name" in app AndroidManifest.xml

This is the exception:
Caused by: java.lang.IllegalStateException: different file names for class [our-namespace.InterestProvider$createFlow$$inlined$transform$1]: [InterestProvider.kt] and [Emitters.kt]
at dev.reformator.stacktracedecoroutinator.jvmagentcommon.DecoroutinatorJvmAgentStacktraceMethodHandleRegistry.getStacktraceMethodHandles$lambda-5(methodHandleRegistry-jvm-agent-common.kt:31)

It seems that Asm(?) has assigned "Emitters.kt" as the fileName of this particular ClassNode. As a result ClassNode.transform (classTransformer.kt) calls buildMarkerAnnotation with "Emitters.kt" as the filename, while clearly the filename should be InterestProvider.kt. The class "InterestProvider$createFlow$$inlined$transform$1" is generated inside InterestProvider.class.
When decompiling InterestProvider.class I can see that the DebugMetaData information of "InterestProvider$createFlow$$inlined$transform$1" has the right filename: "InterestProvider.kt".

Adding "val filename: String" to DebugMetaDataInfo and replacing sourceFile in this line (classTransformer.kt):

visibleAnnotations.add(buildMarkerAnnotation(sourceFile, suspendFuncName2LineNumbers))

with

getDebugMetadataInfo()?.filename ?: sourceFile

results in Decoroutinator not complaining anymore when instrumenting our codebase.

Unfortunately I was not able to create a reproduction scenario and could not determine why Asm comes up with "Emitters.kt" as the source filename.

On Android API 26 got exception below. Seems it somehow related to the way SD replaces implementation of class BaseContinuationImpl at runtime.

java.lang.IncompatibleClassChangeError: The method 'java.lang.Class java.lang.Object.getClass()' was expected to be of type interface but instead was found to be of type virtual (declaration of 'dev.reformator.stacktracedecoroutinator.common.DecoroutinatorContinuationStacktraceElementRegistryImpl' appears in /data/app/dev.reformator.stacktracedecoroutinator.test-FP9cAHXGzdvvqHD-T2lQ1w==/base.apk:classes6.dex)
	at dev.reformator.stacktracedecoroutinator.common.DecoroutinatorContinuationStacktraceElementRegistryImpl.lambda$getStacktraceElements$2(DecoroutinatorContinuationStacktraceElementRegistryImpl.java:66)
	at dev.reformator.stacktracedecoroutinator.common.DecoroutinatorContinuationStacktraceElementRegistryImpl$$ExternalSyntheticLambda1.apply(Unknown Source:4)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:194)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:235)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:501)
	at dev.reformator.stacktracedecoroutinator.common.DecoroutinatorContinuationStacktraceElementRegistryImpl.getStacktraceElements(DecoroutinatorContinuationStacktraceElementRegistryImpl.java:79)
	at dev.reformator.stacktracedecoroutinator.stdlib.StdlibKt.decoroutinatorResumeWith(stdlib.kt:28)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(continuation-stdlib.kt:18)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
	at kotlinx.coroutines.JavaTool.EventLoopProcessNextEvent(JavaTool.java:78)
	at dev.reformator.stacktracedecoroutinator.performancetest.Performance_test_androidKt.BlockingCoroutineJoinBlocking(performance-test-android.kt:332)
	at kotlinx.coroutines.JavaTool.createAndJoinBlockingCoroutine(JavaTool.java:53)
	at dev.reformator.stacktracedecoroutinator.performancetest.Performance_test_androidKt.runBlocking(performance-test-android.kt:306)
	at dev.reformator.stacktracedecoroutinator.performancetest.Performance_test_androidKt.runBlocking$default(performance-test-android.kt:281)
	at dev.reformator.stacktracedecoroutinator.performancetest.PerformanceTest.resumeWithDepth(performance-test-android.kt:277)
	at dev.reformator.stacktracedecoroutinator.performancetest.PerformanceTest.depth10(performance-test-android.kt:231)
	at java.lang.reflect.Method.invoke(Native Method)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at androidx.test.ext.junit.runners.AndroidJUnit4.run(AndroidJUnit4.java:162)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
	at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:444)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2074)

Thank you so much for building this library, getting usable stack traces from my coroutines is huge.

I am noticing that if I instrument my unit tests with your agent, I'm seeing a pretty significant drop in my Jacoco code coverage reports, effectively I'm losing coverage for all my suspend functions.

I've created a sample project based on your example in the readme, and I'm able to confirm that when enabling your agent, I lose coverage for suspend methods.

If you do the following, you can generate the report for yourself:

1. cd into my unzipped example
2. run ./gradlew test
3. open the jacoco report at build/reports/jacoco/test/html/demo/index.html

From this report you will see that there is no coverage for the suspend method. However, If you comment out the the following line in build.gradle, and re-run ./gradlew test

test {
    useJUnitPlatform()
    jvmArgs = [
//        "-javaagent:${configurations.agent.asPath}"
    ]
    finalizedBy jacocoTestReport
}

You'll get a report that includes coverage for this suspend method.

Could you please help me figure out how to preserve code coverage while using this agent?

When using Jacoco, methods are instrumented to call $jacocoInit first. As a consequence the first instructions of suspend methods are not ALOAD (25) and INSTANCEOF (193) anymore. The method MethodNode.getDebugMetadataInfo() in classTransformer.kt assumes this.

The following code searches for two consecutive ALOAD/INSTANCEOF instructions with the same conditions as previously.
It replaces the second part of the body starting with "val firstInstructions: ".

return instructions.asSequence().filter {
        val next = it.next
        it is VarInsnNode && it.opcode == Opcodes.ALOAD && it.`var` == continuationIndex
                && next != null && next is TypeInsnNode && next.opcode == Opcodes.INSTANCEOF
    }
        .firstOrNull()?.let {
            val continuationClassName = (it.next as TypeInsnNode).desc.replace('/', '.')
            decoroutinatorJvmAgentRegistry.metadataInfoResolver.getDebugMetadataInfo(continuationClassName)
        }

Does this library work with JVM 11, 17, etc?
For example, with corretto-17.0.3 and stacktrace-decoroutinator-jvm:2.2.1 I get

Exception in thread "main" java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.findLoadedClass(java.lang.String) accessible: module java.base does not "opens java.lang" to unnamed module @1c20c684
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199)
	at java.base/java.lang.reflect.Method.setAccessible(Method.java:193)
	at dev.reformator.stacktracedecoroutinator.utils.Utils_jvmKt.getClassIfLoaded(utils-jvm.kt:11)
	at dev.reformator.stacktracedecoroutinator.runtime.DecoroutinatorRuntime.getState(runtime-jvm.kt:42)
	at dev.reformator.stacktracedecoroutinator.runtime.DecoroutinatorRuntime.load(runtime-jvm.kt:52)
	at dev.reformator.stacktracedecoroutinator.runtime.DecoroutinatorRuntime.load$default(runtime-jvm.kt:51)
	at MainKt.main(Main.kt:14)
	at MainKt.main(Main.kt)

when I launch the example from README. With 1.8 it works fine.

JVM agent API allows to modify the bytecode before loading classes.
SD can use it instead of generating extra classes at runtime.

I am calling DecoroutinatorRuntime.load() at the top of my app's onCreate but it seems to be causing the app to crash
I've attached a screenshot of the full stack trace.

Are there plans to provide MP version of this? Spicifically for the native target.

It would be better to exclude the instrumented class kotlin.coroutines.jvm.internal.BaseContinuationImpl from the classpath. And load it on demand (in DecoroutinatorRuntime.load() method).

That will allow to completely eliminate the effect of SD if it's required.

Hey @Anamorphosee, I'm running into this crash when trying to use SD on Android:

Caused by: java.lang.NullPointerException: it must not be null
        at dev.reformator.stacktracedecoroutinator.runtime.DecoroutinatorRuntime.load(runtime-android.kt:38)
        at dev.reformator.stacktracedecoroutinator.runtime.DecoroutinatorRuntime.load$default(runtime-android.kt:24)

The android artifact isn't an AAR so I wasn't able to step into the sources to debug what exactly is null. I was testing on Android 13.