google / ksp Goto Github PK

View Code? Open in Web Editor NEW

2.6K 51.0 245.0 22.48 MB

Kotlin Symbol Processing API

Home Page: https://github.com/google/ksp

License: Apache License 2.0

Kotlin 99.52% Java 0.21% Shell 0.28%

ksp's Issues

exception in ksp compilation

I've hit an exception in compiler but cannot reproduce in a sample.

org.jetbrains.kotlin.utils.KotlinExceptionWithAttachments: Exception while analyzing expression at (339,28) in /home/yboyar/src/androidx-master-dev/frameworks/support/room/integration-tests/kotlintestapp/src/androidTest/java/androidx/room/integration/kotlintestapp/dao/BooksDao.kt

full stacktrace

It happens when I try to resolve the return type of a function actual code:

@Transaction
fun getDefaultBook() = getBook("DEFAULT_ID")

@Query("SELECT * FROM book WHERE bookId = :bookId")
fun getBook(bookId: String): Book

It happens when I call funDeclaration.returnType?.resolve(). Here, funDeclaration is the KSFunctionDeclaration of the getDefaultBook method.

I can help reproduce within androidx but unfortunately my efforts to isolate it didn't work :/.

Support retrieving jvm signatures of functions, constructors, and fields

This is supported in metadata APIs and useful as reference at times. We use this in Moshi to deduce its runtime constructor signature for dynamically invoking a defaults constructor, if any.

Resolution for local declarations

Currently, the implementation doesn't support resolving references to local declarations.

Got some directions from JB:
https://jetbrains.slack.com/archives/CHLKDT4Q6/p1582675225005300

Approach 1. Resolve the body of the inner most, non-local function and look up the elements. This is sub-optimal as it resolves everything in the body.

Approach 2. On demand build the scope hierarchy should be the optimal way, although it doesn't help in the worst case described below.

Performance note:
Because all local variables are modeled as local properties, the total number of accessible elements are much more than kapt. Resolving them in TestProcessor would be slowed down considerably (~2x slower). If only local function and local classes are resolved, the slow down is ~10% in tachiyomi.

compileOnly dependencies are not visible to ksp

In annotation processing, a common pattern is to use compileOnly to put annotations artifacts on the classpath for the processor. With ksp, I've noticed I always have to put these as implementation or api dependencies.

Repro PR: ZacSweers/MoshiX#25

Change the autoServiceAnnotations dep in /moshi-sealed-codegen/build.gradle to compileOnly and run a simple build, the processor will fail because it cannot find the AutoService class in its processor (AutoServiceSymbolProcessor.kt).

$ gw clean :moshi-sealed-codegen:build

> Task :moshi-sealed-codegen:compileTestKotlin FAILED
e: java.lang.IllegalStateException: @AutoService type not found on the classpath.
        at dev.zacsweers.auto.service.ksp.AutoServiceSymbolProcessor.process(AutoServiceSymbolProcessor.kt:65)
        at org.jetbrains.kotlin.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:76)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:114)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:91)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:560)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:83)
        at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:115)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:551)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:178)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:164)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:51)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:86)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:44)
        at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:105)

Changes in symbol processor does not invalidate compilation

Seems like dependencies in the ksp classpath do not affect the task invalidation for the project compilation.

e.g. if you have
:app, :processor modules in a project where processor is a ksp configuration dependency

// app's gradle file

ksp(project(":processor"))

Changes in the processor project do not invalidate the compilation for app unless i invoke clean.

To reproduce,

compile the whole project

./gradlew :app:build --info

make changes in the processor

./gradlew :app:build --info

processor will be recompiled but app is not recompiled.

Any should be added to superTypes implicitly

class A {}

Any should be a super type of such a class.

trying to resolve an annotation's type from a Java class throws NPE

// full exception at the bottom

When I have a class like this:

@Entity
public class JavaEntity {
    @PrimaryKey
    public Long id;

    public String bookId;
}

When JavaEntity is obtained as a KsAnnotated, calling it.annotations.first().resolve() throws NPE.

To reproduce:

checkout androidx
cherry-pick revision 7 from https://android-review.googlesource.com/c/platform/frameworks/support/+/1351340
cherry-pick revision 13 from https://android-review.googlesource.com/c/platform/frameworks/support/+/1350690
cd frameworks/support
./gradlew :room:i-t:room-t-kotlin:clean :room:i-t:room-t-kotlin:assembleAndroidTest

I think this is a KI but couldn't find the bug hence filing. I can try to create an isolated repro if it helps.

e: java.lang.NullPointerException
        at org.jetbrains.kotlin.ksp.symbol.impl.java.KSClassDeclarationJavaImpl.asType(KSClassDeclarationJavaImpl.kt:115)
        at org.jetbrains.kotlin.ksp.symbol.impl.java.KSAnnotationJavaImpl$annotationType$2.invoke(KSAnnotationJavaImpl.kt:23)
        at org.jetbrains.kotlin.ksp.symbol.impl.java.KSAnnotationJavaImpl$annotationType$2.invoke(KSAnnotationJavaImpl.kt:13)
        at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
        at org.jetbrains.kotlin.ksp.symbol.impl.java.KSAnnotationJavaImpl.getAnnotationType(KSAnnotationJavaImpl.kt)
        at androidx.room.compiler.ksp.KspUtilKt.hasAnnotation(KspUtil.kt:16)
        at androidx.room.compiler.ksp.KspElement.hasAnnotation(KspElement.kt:33)
        at androidx.room.compiler.usp.UElement$DefaultImpls.hasAnyOf(UElement.kt:17)
        at androidx.room.compiler.ksp.KspElement.hasAnyOf(KspElement.kt:10)
        at androidx.room.processor.EntityProcessorKt.EntityProcessor(EntityProcessor.kt:102)
        at androidx.room.processor.EntityProcessorKt.EntityProcessor$default(EntityProcessor.kt:99)
        at androidx.room.processor.DatabaseProcessor.processEntities(DatabaseProcessor.kt:282)
        at androidx.room.processor.DatabaseProcessor.doProcess(DatabaseProcessor.kt:62)
        at androidx.room.processor.DatabaseProcessor.process(DatabaseProcessor.kt:53)
        at androidx.room.RoomKspProcessor.process(RoomKspProcessor.kt:50)
        at org.jetbrains.kotlin.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:63)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:114)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:91)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:560)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:83)
        at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:115)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:551)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:178)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:164)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:51)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:86)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:44)
        at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:105)
        at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:83)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execAndOutputXml(CLICompiler.kt:50)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.jetbrains.kotlin.compilerRunner.GradleKotlinCompilerWork.compileInProcessImpl(GradleKotlinCompilerWork.kt:350)
        at org.jetbrains.kotlin.compilerRunner.GradleKotlinCompilerWork.access$compileInProcessImpl(GradleKotlinCompilerWork.kt:66)
        at org.jetbrains.kotlin.compilerRunner.GradleKotlinCompilerWork$compileInProcess$future$1.call(GradleKotlinCompilerWork.kt:318)
        at org.jetbrains.kotlin.compilerRunner.GradleKotlinCompilerWork$compileInProcess$future$1.call(GradleKotlinCompilerWork.kt:66)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

API to check if a function has default implementation

As far as I can see, there is no way to check if a KSFunctionDeclaration has a default implementation.

As Room generates Java code, when creating the implementation for a Dao interface, we have to generate the implementation that delegates to the default implementation so this is necessary for room.

Even if Room was generating kotlin code, we would still need this functionality as we have to print an error if a function in dao is not annotated properly. For that case, interface methods with default implementations are OK.

e.g.

@Dao
interface MyDao {
    @Query("select * from foo")
    fun goodQuery():List<Foo>

   fun okMethod() {
       
   }

   @Transaction
   fun transactionMethod() {
        // room will override this to be called inside a database transaction
   }

   // method without implementation nor any room annotations so we need to throw an error
   fun badMethod()
    
}

Caching for KSTypeReferenceJavaImpl is not working correctly

Reproduce:

class A {
public List<? extends Set> extendsSetFun() {}

public List<? extends List> extendsListFun() {}

}

Two functions' return type will be same KSTypeReferenceJavaImpl, despite difference in type argument. My guess is PsiType didn't calculate hash based on type argument.

Command line tool

Add overrides function to KSPropertyDeclaration

KSFunctionDeclaration has an overrides function to check if the function overrides another function. Properties can override too so it makes sense to have the same function. ex:

interface Foo {
    val foo: String
}

class Bar : Foo {
    override val foo: String = ""
}

KSP sample project (20200716) failed to build on Windows

Relavent log:

PS C:\Users\jiaxiang\Downloads\playground-1.4-M1-dev-experimental-20200716> .\gradlew.bat build --stacktrace

> Task :workload:compileKotlin FAILED
e: C:\Users\jiaxiang\Downloads\playground-1.4-M1-dev-experimental-20200716\workload\src\main\java\com\example\A.kt: (3, 8): Unresolved reference: HELLO
e: C:\Users\jiaxiang\Downloads\playground-1.4-M1-dev-experimental-20200716\workload\src\main\java\com\example\A.kt: (6, 17): Unresolved reference: HELLO
e: C:\Users\jiaxiang\Downloads\playground-1.4-M1-dev-experimental-20200716\workload\src\main\java\com\example\A.kt: (9, 19): Unresolved reference: AClassBuilder

FAILURE: Build failed with an exception.

It looks that generated source is not included in normal build.

createNewFile always requires an extension

This isn't always the case, especially if writing resources like config files for service loaders. Putting in an empty string results in a trailing period.

Unable to get the typealias after resolving a reference

When resolving a reference to a type alias, the underlying type is returned instead of the alias. This makes type checking much easier. However, the type alias is skipped and cannot be obtained. Something like KSType.alias: KSTypeAlias? is needed.

Address cases where containing files isn't available

Support incremental processing

Passing processor options

Need to support passing processor options from Gradle and command line.

Helpers around implicit elements

Here are examples by Evan Tatarka:

class Foo --> primaryConstructor is null
class Bar() -> primaryConstructor is not null

Another example is that properties of interfaces are implicitly abstract.

Currently, KSP models program after its grammar structure and does not expose those implicit elements. We need helpers to unify the behaviors of explicit and implicit elements.

Implement ClassifierReference.qualifier

Debugging support

This is less of a feature request and more just sharing this in case anyone else is interested.

In general, I'd recommend writing up an example of debugging/testing story, perhaps a demo or contributing support to @tschuchortdev's https://github.com/tschuchortdev/kotlin-compile-testing project.

In the meantime, I've found success attaching the debugger manually from the Gradle invocation.

Example: ./gradlew :sample:build --no-daemon -Dorg.gradle.debug=true -Pkotlin.compiler.execution.strategy=in-process. Then connect via remote debugger in the IDE and you can hit your processor's breakpoints.

FunctionKind.TOP_LEVEL for top level functions

Currently, top level functions are of kind FunctionKind.STATIC. This can be confusing. Let's use FunctionKind.TOP_LEVEL and leave STATIC for java static functions.

Annotation with `Class[]` member always returns an empty list

I've been trying to write a prototype of AutoService via KSP, and notice that the value field (a Class array) always returns an empty list when I try to query the annotation value in KSP

Repro branch: https://github.com/ZacSweers/moshi-sealed/tree/z/ksp

Under AutoServiceKsProcessor.kt, there's a println with a comment where it finds an empty list

Investigate local scope for annotation entry

See TODO in KSAnnotationImpl.

Model Java modifiers

Java modifiers have different semantics to Kotlin modifiers. It's better to have a separate set of modifiers for Java. android/kotlin-for-ksp#9 depends on this.

Add Windows test configuration

Simple string member is always null in `20200716`

This appears to be a regression from 20200626 to 20200716

Repro case in this PR: ZacSweers/MoshiX#25. Run ./gradlew :sample:compileKotlin

In KspUtil.kt, there is a helper getMember() function that gets an annotation member's value by name. That PR uses this from MoshiSealedSymbolProcessor to look up the value of a generator member. It does find the member, but now its string value is always null.

internal inline fun <reified T> KSAnnotation.getMember(name: String): T {
  val matchingArg = arguments.find { it.name?.asString() == name }
      ?: error("No member name found for '$name'. All arguments: ${arguments.map { it.name?.asString() }}")

  // NOTE fails here
  return matchingArg.value as? T ?: error("No value found for $name. Was ${matchingArg.value}")
}

e: java.lang.IllegalStateException: No value found for generator. Was null
	at dev.zacsweers.moshisealed.codegen.MoshiSealedSymbolProcessor.process(MoshiSealedSymbolProcessor.kt:179)
	at org.jetbrains.kotlin.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:76)
	at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:114)
	at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:91)

Multiple rounds

Need to discuss with upstream before proceeding.

Introduce Error type to allow forward reference

Add helper for equivlent of Types.asMemeberOf

When writing an annotation processor you can use Types.asMemeberOf() to resolve the concrete type of a method based on the enclosing class. Ex if you had:

interface GenericInterface<T> {
  bar(): T
}

abstract class Foo : GenericInterface<String> {
}

you could get that bar() in the context of Foo returns String.

Model init blocks

Although they do not carry symbol information, local declarations inside them do.

Annotation values that are default are not visible in KSP

When reading a Java annotation with default value, its value is not visible in the arguments list.

public @interface JavaAnnotation {
    String debug();
    String withDefaultValue()  default "";
}

If you annotate an element with JavaAnnotation and don't specify a withDefaultValue property, KSP processor threats it as not existing such that arguments map in KSAnnotation will only include debug field.

Files are always generated to `src/main` even if they are not source files

I'm testing KSP partially with an AutoService implementation of it, which involves generating service files under META-INF/services/.... Due to the hardcoded src/main path, I have to work around it by manually including it via sourceSets

sourceSets {
  main {
    resources {
      srcDir(file("build/generated/ksp/src/main"))
    }
  }
}

Feature request: Logging APIs

Similar to how annotation processing has Messager APIs, this would make communicating information to the user easy.

KSName doesn't offer a deterministic naming API

KSName isn't ideal for handling non-trivial class names, such as heavily nested types. KotlinPoet's ClassName.bestGuess() API is used in the Glide example project, but that API should really only be used as a last resort.

In Kotlin's internals and metadata, it has a pattern for its names that is deterministic (e.g. "org/foo/bar/Baz.Nested"). It would be ideal if this name was exposed or otherwise surfaced a more concrete API.

Here's an implementation we have in KotlinPoet for piecing together a name from it: https://github.com/square/kotlinpoet/blob/06aad8e024aaa453477d420ad65cfe4af9e84f3a/kotlinpoet-metadata-specs/src/main/kotlin/com/squareup/kotlinpoet/metadata/specs/internal/ClassInspectorUtil.kt#L171-L209

Example of where KSName currently falls short:

Given this name: dev.zacsweers.moshisealed.sample.test.MessageTest.MessageWithNullDefault
The result of getQualifier is dev.zacsweers.moshisealed.sample.test.MessageTest 🤔

interface Foo {
  val bar: Int get() = 1
}

and

interface Foo {
   val bar: Int
}

Adding isAbstract to KSPropertyDeclaration as well should work.

If I have a java class like this:

public class TestClass {
  protected void test() {}
}

And try to check if test overrides Any.equals, it throws IllegalStateException.

// processor
        override fun process(resolver: Resolver) {
            val testClass = resolver.getClassDeclarationByName(
                resolver.getKSNameFromString("TestClass")
            )!!
            val any = resolver.getClassDeclarationByName(
                resolver.getKSNameFromString("kotlin.Any")
            )!!
            val equalsMethod = any.getAllFunctions().first {
                it.simpleName.asString() == "equals"
            }
            val testMethod = testClass.getDeclaredFunctions().first {
                it.simpleName.asString() == "test"
            }
            check(testMethod.overrides(equalsMethod) == false) { // << throws
                "should not override"
            }
        }

e: java.lang.IllegalStateException: unhandled visibility: protected/*protected and package*/
	at org.jetbrains.kotlin.ksp.symbol.impl.UtilsKt.toKSModifiers(utils.kt:122)
	at org.jetbrains.kotlin.ksp.symbol.impl.binary.KSFunctionDeclarationDescriptorImpl$modifiers$2.invoke(KSFunctionDeclarationDescriptorImpl.kt:101)
	at org.jetbrains.kotlin.ksp.symbol.impl.binary.KSFunctionDeclarationDescriptorImpl$modifiers$2.invoke(KSFunctionDeclarationDescriptorImpl.kt:22)
	at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
	at org.jetbrains.kotlin.ksp.symbol.impl.binary.KSFunctionDeclarationDescriptorImpl.getModifiers(KSFunctionDeclarationDescriptorImpl.kt)
	at org.jetbrains.kotlin.ksp.symbol.impl.binary.KSFunctionDeclarationDescriptorImpl.overrides(KSFunctionDeclarationDescriptorImpl.kt:47)

Model Java source files

How would I use "Kotlin Symbol Processing API" from maven

The example showed gradle as build tool, how would you use it with maven?

google / ksp Goto Github PK

ksp's Issues

Recommend Projects

Recommend Topics

Recommend Org