I have added --debug-query=ast in the CLI to dump tree-sitter output.

from ast-grep.

Hi! Thanks for the report! I don't have time to give it a look at the moment but I will give it back as soon as possible!
Thank you for the patience!

from ast-grep.

Thanks 👍 Having a cli debug output will be very helpful.
I will implement it in near future

from ast-grep.

thanks for the suggestion. But a rule YAML usually has many patterns and displaying them in one command is not straightforward. For example, there may be multiple patterns but only one pattern is interested. Using the playground has a better debugging experience.

from ast-grep.

The issue is

@Serializable data class $$$

is not parsed as class constructor in tree-sitter-kotlin 0.3.6

See the official parser page https://fwcd.dev/tree-sitter-kotlin/

However, ast-grep playground is using older kotlin parser so it somehow works.

UPDATE: I have updated the tree-sitter-kotlin wasm on playground

from ast-grep.

In the case of tree-sitter parser cannot parse your pattern, using relational + kind rule is easier.
See this example

I also tested it on command line it works across different parser versions.

from ast-grep.

@HerringtonDarkholme Thank you for your prompt response! I was able to extract the class parameters as intended with the rules you suggested.

The issue is

@Serializable data class $$$

is not parsed as class constructor in tree-sitter-kotlin 0.3.6

However, I am really sorry, but I feel that this explanation does not accurately describe the cause. Because the CLI does not show any parsing errors for pattern, and it seems natural that tree-sitter Kotlin cannot parse code with meta variables as it is.

Upon further reflection after receiving the response, I realized that I could make the reproducible code much smaller. In the example below, the relational rule is no longer used.

Still, the situation remains the same: no match in the CLI, but a match in the playground. Is there any other possible cause?

⏯ Playground Link

Playground link with relevant code

💻 Code

Code (Unchanged from the original):

package com.example

import kotlinx.serialization.Serializable
import java.util.UUID

@Serializable
data class Annotated(
    val a: Int,
    val b: UUID,
)

data class NotAnnotated(
    val a: Int,
    val b: UUID,
)

Pattern:

id: serializable-class
language: kotlin
rule:
  pattern: |
    @Serializable
    data class $$$

🙁 Actual behavior

The CLI does not match.

🙂 Expected behavior

The CLI should match with the annotated class as the playground does.

from ast-grep.

I have updated the title of the issue. My main concern is the difference between CLI and playground. Because such a difference would make rule development more difficult.

If the difference is due to differences in library versions, it would be nice if either of the following could be accomplished.

• Both the CLI and the playground show the version of the ast-grep and depending libraries including the tree-sitter so that a user can notice the difference of the versions.
• The CLI provides something like a debug mode that shows AST structure of the pattern and the input code so that a user can notice the difference of the parsed structure.

from ast-grep.

One more thing is to note is the behavior of treesitter.
At most time web and native tree sitter behave the same. However, when the input is not valid, which is commonly seen in pattern, web and native will behave differently due to tree sitter heuristics.

TS will try to recover from parsing error and the recovery is based on the input length. Sadly, web and native tree sitter have different string encoding so their length are different.

This makes different parsing results

from ast-grep.

That is a fascinating explanation for why tree sitter has different behavior in a browser vs natively.
Here's an example of tree sitter behaving differently for strings of different lengths:

from ast-grep.

@HerringtonDarkholme @StevenLove Thank you for clarifying the difference between web and native. I understand that the parsing result depends on the length of the input.

I am interested in the different encodings that affect the length of the input. Personally, if there is an option to use the same encoding in the CLI and playground, I would prefer to use that, even if it slows down the execution. This is because I often refer to the playground results when developing ast-grep rules, and I consider the consistency of the results to be important.

from ast-grep.

Related documentation https://ast-grep.github.io/reference/cli/run.html#debug-query-format

from ast-grep.

@HerringtonDarkholme Thank you for adding the debugging feature so quickly! It will be useful for debugging patterns.

If possible, the same flag for the sg scan command would also make it easier to debug when we write the pattern in a yaml file.

from ast-grep.