Git Product home page Git Product logo

regression-tool's Introduction

Checkstyle is a tool for checking Java source code for adherence to a Code Standard or set of validation rules (best practices).

The latest release version can be found at GitHub releases or at Maven repo.

Each-commit builds of maven artifacts can be found at Maven Snapshot repository.

Documentation is available in HTML format, see https://checkstyle.org/checks.html .

Build instructions and Contribution

Build instructions

Setup IDE for development

Explanation on how to create your own module

Verification of code quality

Sending Pull Request

Report Issue

Continuous integration and Quality reports

See our CIs statuses.

Quality reports: https://checkstyle.org/project-reports.html

JavaScript, CSS and Java source file analysis on Codacy:

Feedback/Support

Please send any feedback to https://groups.google.com/forum/?hl=en#!forum/checkstyle

Questions and Answers from community:

Bugs and Feature requests (not the questions): https://github.com/checkstyle/checkstyle/issues

Support/Sponsor checkstyle

If you want to speed up fixing of issue and want to encourage somebody in internet to resolve any issue:

Licensing

This software is licensed under the terms in the file named "LICENSE" in this directory.

The software uses the ANTLR package (https://www.antlr.org/). Its license terms are in the file named "RIGHTS.antlr" in this directory.

This product includes software developed by The Apache Software Foundation (https://www.apache.org/).

The software uses the Logging and Beanutils packages from the Apache Commons project (https://commons.apache.org/). The license terms of these packages are in the file named "LICENSE.apache20" in this directory.

The software uses the Google Guava Libraries (https://github.com/google/guava/). The license terms of these packages are in the file named "LICENSE.apache20" in this directory.

The software uses the Picocli Library (https://github.com/remkop/picocli/). Its license terms are in the file named "LICENSE.apache20" in this directory.

regression-tool's People

Contributors

dependabot[bot] avatar luolc avatar nrmancuso avatar rdiachenko avatar rnveach avatar romani avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

regression-tool's Issues

Full verify test fails on checkstyle

From mvn clean verify:

[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.github.checkstyle:regression-tool:jar:1.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found duplicate declaration of plugin org.apache.maven.plugins:maven-checkstyle-plugin @ line 155, column 21
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]                                                                       
[INFO] ------------------------------------------------------------------------
[INFO] Building regression-tool 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ regression-tool ---
[INFO] Deleting /home/ricky/opensource/regression-tool/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ regression-tool ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ regression-tool ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 19 source files to /home/ricky/opensource/regression-tool/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ regression-tool ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 5 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ regression-tool ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 9 source files to /home/ricky/opensource/regression-tool/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ regression-tool ---
[INFO] Surefire report directory: /home/ricky/opensource/regression-tool/target/surefire-reports

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.github.checkstyle.regression.module.ModuleInfoCollectorTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.27 sec - in com.github.checkstyle.regression.module.ModuleInfoCollectorTest
Running com.github.checkstyle.regression.module.ModuleUtilsTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - in com.github.checkstyle.regression.module.ModuleUtilsTest
Running com.github.checkstyle.regression.report.ReportGeneratorTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in com.github.checkstyle.regression.report.ReportGeneratorTest
Running com.github.checkstyle.regression.configuration.ConfigGeneratorTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.936 sec - in com.github.checkstyle.regression.configuration.ConfigGeneratorTest
Running com.github.checkstyle.regression.internal.CommitValidationTest
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.156 sec - in com.github.checkstyle.regression.internal.CommitValidationTest
Running com.github.checkstyle.regression.extract.ExtractInfoProcessorTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.254 sec - in com.github.checkstyle.regression.extract.ExtractInfoProcessorTest
Running com.github.checkstyle.regression.git.DiffParserTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.569 sec - in com.github.checkstyle.regression.git.DiffParserTest

Results :

Tests run: 32, Failures: 0, Errors: 0, Skipped: 0

[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ regression-tool ---
[INFO] Building jar: /home/ricky/opensource/regression-tool/target/regression-tool-1.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default) @ regression-tool ---
[INFO] Building jar: /home/ricky/opensource/regression-tool/target/regression-tool-1.0-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (check) @ regression-tool ---
[INFO] There are 62 errors reported by Checkstyle 7.8.1 with https://raw.githubusercontent.com/checkstyle/checkstyle/checkstyle-7.8.1/config/checkstyle_checks.xml ruleset.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableGitChange.java:[0] (javadoc) JavadocPackage: Missing package-info.java file.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableGitChange.java:[1] (header) Header: Line does not match expected header line of '////////////////////////////////////////////////////////////////////////////////'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableGitChange.java:[1] (header) RegexpHeader: Line does not match expected header line of '^/{80}$'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableGitChange.java:[6] (imports) CustomImportOrder: Import statement for 'java.util.List' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableGitChange.java:[6] (imports) ImportOrder: Wrong order for 'java.util.List' import.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableGitChange.java:[7] (imports) ImportOrder: 'javax.annotation.Generated' should be separated from previous imports.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableGitChange.java:[7] (imports) CustomImportOrder: Import statement for 'javax.annotation.Generated' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableGitChange.java:[11] (javadoc) JavadocParagraph: <p> tag should be placed immediately before the first word, with no space after.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableGitChange.java:[11] (javadoc) JavadocParagraph: <p> tag should be preceded with an empty line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableProperty.java:[1] (header) Header: Line does not match expected header line of '////////////////////////////////////////////////////////////////////////////////'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableProperty.java:[1] (header) RegexpHeader: Line does not match expected header line of '^/{80}$'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableProperty.java:[6] (imports) CustomImportOrder: Import statement for 'java.util.List' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableProperty.java:[6] (imports) ImportOrder: Wrong order for 'java.util.List' import.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableProperty.java:[7] (imports) ImportOrder: 'javax.annotation.Generated' should be separated from previous imports.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableProperty.java:[7] (imports) CustomImportOrder: Import statement for 'javax.annotation.Generated' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableProperty.java:[11] (javadoc) JavadocParagraph: <p> tag should be placed immediately before the first word, with no space after.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableProperty.java:[11] (javadoc) JavadocParagraph: <p> tag should be preceded with an empty line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/GsonAdaptersModuleExtractInfo.java:[1] (header) Header: Line does not match expected header line of '////////////////////////////////////////////////////////////////////////////////'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/GsonAdaptersModuleExtractInfo.java:[1] (header) RegexpHeader: Line does not match expected header line of '^/{80}$'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/GsonAdaptersModuleExtractInfo.java:[3] (imports) AvoidStarImport: Using the '.*' form of import should be avoided - com.google.gson.*.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/GsonAdaptersModuleExtractInfo.java:[4] (imports) AvoidStarImport: Using the '.*' form of import should be avoided - com.google.gson.reflect.*.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/GsonAdaptersModuleExtractInfo.java:[5] (imports) AvoidStarImport: Using the '.*' form of import should be avoided - com.google.gson.stream.*.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/GsonAdaptersModuleExtractInfo.java:[6] (imports) CustomImportOrder: Import statement for 'java.io.IOException' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/GsonAdaptersModuleExtractInfo.java:[6] (imports) ImportOrder: Wrong order for 'java.io.IOException' import.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/GsonAdaptersModuleExtractInfo.java:[7] (imports) ImportOrder: 'javax.annotation.Generated' should be separated from previous imports.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/GsonAdaptersModuleExtractInfo.java:[7] (imports) CustomImportOrder: Import statement for 'javax.annotation.Generated' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/GsonAdaptersModuleExtractInfo.java:[10] (sizes) LineLength: Line is longer than 100 characters (found 114).
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ModifiableModuleInfo.java:[1] (header) Header: Line does not match expected header line of '////////////////////////////////////////////////////////////////////////////////'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ModifiableModuleInfo.java:[1] (header) RegexpHeader: Line does not match expected header line of '^/{80}$'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ModifiableModuleInfo.java:[6] (imports) CustomImportOrder: Import statement for 'java.util.ArrayList' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ModifiableModuleInfo.java:[6] (imports) ImportOrder: Wrong order for 'java.util.ArrayList' import.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ModifiableModuleInfo.java:[7] (imports) CustomImportOrder: Import statement for 'java.util.List' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ModifiableModuleInfo.java:[8] (imports) ImportOrder: 'javax.annotation.Generated' should be separated from previous imports.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ModifiableModuleInfo.java:[8] (imports) CustomImportOrder: Import statement for 'javax.annotation.Generated' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ModifiableModuleInfo.java:[12] (javadoc) JavadocParagraph: <p> tag should be preceded with an empty line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ModifiableModuleInfo.java:[14] (javadoc) JavadocParagraph: <p> tag should be preceded with an empty line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleExtractInfo.java:[1] (header) Header: Line does not match expected header line of '////////////////////////////////////////////////////////////////////////////////'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleExtractInfo.java:[1] (header) RegexpHeader: Line does not match expected header line of '^/{80}$'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleExtractInfo.java:[6] (imports) CustomImportOrder: Import statement for 'java.util.List' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleExtractInfo.java:[6] (imports) ImportOrder: Wrong order for 'java.util.List' import.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleExtractInfo.java:[7] (imports) ImportOrder: 'javax.annotation.Generated' should be separated from previous imports.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleExtractInfo.java:[7] (imports) CustomImportOrder: Import statement for 'javax.annotation.Generated' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleExtractInfo.java:[11] (javadoc) JavadocParagraph: <p> tag should be placed immediately before the first word, with no space after.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleExtractInfo.java:[11] (javadoc) JavadocParagraph: <p> tag should be preceded with an empty line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleInfo.java:[1] (header) Header: Line does not match expected header line of '////////////////////////////////////////////////////////////////////////////////'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleInfo.java:[1] (header) RegexpHeader: Line does not match expected header line of '^/{80}$'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleInfo.java:[7] (imports) CustomImportOrder: Import statement for 'java.util.List' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleInfo.java:[7] (imports) ImportOrder: Wrong order for 'java.util.List' import.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleInfo.java:[8] (imports) ImportOrder: 'javax.annotation.Generated' should be separated from previous imports.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleInfo.java:[8] (imports) CustomImportOrder: Import statement for 'javax.annotation.Generated' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleInfo.java:[12] (javadoc) JavadocParagraph: <p> tag should be placed immediately before the first word, with no space after.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/data/ImmutableModuleInfo.java:[12] (javadoc) JavadocParagraph: <p> tag should be preceded with an empty line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/ImmutableArguments.java:[0] (javadoc) JavadocPackage: Missing package-info.java file.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/ImmutableArguments.java:[1] (header) Header: Line does not match expected header line of '////////////////////////////////////////////////////////////////////////////////'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/ImmutableArguments.java:[1] (header) RegexpHeader: Line does not match expected header line of '^/{80}$'.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/ImmutableArguments.java:[8] (imports) CustomImportOrder: Import statement for 'java.util.List' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/ImmutableArguments.java:[8] (imports) ImportOrder: Wrong order for 'java.util.List' import.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/ImmutableArguments.java:[9] (imports) CustomImportOrder: Import statement for 'java.util.Optional' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/ImmutableArguments.java:[10] (imports) ImportOrder: 'javax.annotation.Generated' should be separated from previous imports.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/ImmutableArguments.java:[10] (imports) CustomImportOrder: Import statement for 'javax.annotation.Generated' is in the wrong order. Should be in the 'STANDARD_JAVA_PACKAGE' group, expecting not assigned imports on this line.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/ImmutableArguments.java:[14] (javadoc) JavadocParagraph: <p> tag should be placed immediately before the first word, with no space after.
[ERROR] target/generated-sources/annotations/com/github/checkstyle/regression/ImmutableArguments.java:[14] (javadoc) JavadocParagraph: <p> tag should be preceded with an empty line.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.082 s
[INFO] Finished at: 2017-07-31T13:13:39-04:00
[INFO] Final Memory: 34M/348M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (check) on project regression-tool: You have 62 Checkstyle violations. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Utility Regression

We have many utilities that our modules rely on.
Example: https://github.com/checkstyle/checkstyle/tree/master/src/main/java/com/puppycrawl/tools/checkstyle/utils

If we make a change to them, we need to run regression on the modules that rely on them.

First we need to identify which methods in the utilities were changed.
Then we need to identify which modules are affected. We can do this by looking at import statements and then looking for those methods to be called in the module.
We have to beware of utilities that call other utilities as we need to be able to trace all of them back to their modules that use them.

I think anything that is not a module, grammar/antlr, or test should be considered a possible utility.

This is blocked by #5 .

Travis failue at checkstyle_regression_no_exception.sh "cannot get Checkstyle version from"

https://travis-ci.org/checkstyle/regression-tool/jobs/591464250#L35700

onfig generated at /home/travis/build/checkstyle/regression-tool/config-test-branch-20190930121503.xml
Installing Checkstyle artifact (master) into local Maven repository ...
Switched to branch 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)
MSG:[maven-release-plugin] prepare release checkstyle-8.25
SHA-1:2f0c2f488a0d302bafe808e08ef47f691163c53b
Caught: groovy.lang.GroovyRuntimeException: Error: cannot get Checkstyle version from /home/travis/build/checkstyle/regression-tool/checkstyle/pom.xml!
groovy.lang.GroovyRuntimeException: Error: cannot get Checkstyle version from /home/travis/build/checkstyle/regression-tool/checkstyle/pom.xml!
  at diff.getCheckstyleVersionFromPomXml(diff.groovy:174)
  at diff$getCheckstyleVersionFromPomXml$8.callCurrent(Unknown Source)
  at diff.generateCheckstyleReport(diff.groovy:184)
  at diff$generateCheckstyleReport$5.callCurrent(Unknown Source)
  at diff.run(diff.groovy:26)
Exception in thread "main" java.lang.IllegalStateException: an error occurred when running diff.groovy
  at com.github.checkstyle.regression.report.ReportGenerator.generate(ReportGenerator.java:61)
  at com.github.checkstyle.regression.Main.runRegression(Main.java:190)
  at com.github.checkstyle.regression.Main.main(Main.java:111)
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE module
  PUBLIC "-//Checkstyle//DTD Checkstyle Configuration 1.3//EN" "https://checkstyle.org/dtds/configuration_1_3.dtd">
<module name="Checker">
   <property name="charset" value="UTF-8"/>
   <!-- do not change severity to 'error', as that will hide errors caused by exceptions -->
   <property name="severity" value="warning"/>
   <!-- haltOnException is required for exception fixes and reporting of all exceptions -->
   <property name="haltOnException" value="false"/>
   <!-- BeforeExecutionFileFilters is required for sources of java9 -->
   <module name="BeforeExecutionExclusionFileFilter">
      <property name="fileNamePattern" value="module\-info\.java$"/>
   </module>
   <!-- Contents below are generated by regression-tool. -->
   <module name="TreeWalker"/>
</module>

Module Component

Taken from #2
We need to identify the type of file to restrict to only modules (for now). In the future, we also need to identify if the module has new/removed properties as they will need to be included in the regression.

Codecov: Main

We need to add test to appease the codecov for class Main

ConfigGenerator: generate module parent node dynamically instead of hardcoding

Taken from #43 (comment)

The TreeWalker node is hardcoding currently. We should generate these nodes dynamically. For each module, we should get its parent module via its parent field, and then search the parent module in the module extract info map. If there is no such parent, then throw an exception and stop the process, otherwise generate every node we need dynamically in this way.

Non-Default Module Testing

We need to go beyond default module testing and dive changing property values.
Reason can be seen at checkstyle/checkstyle#3656 (comment) .

We can already identify properties, but how are going to decide what values to set? Booleans are easy since they only have 2 values, but what about regular expression strings or files.
Should we hardcode some possible values and randomize the rest? Can we scan tests for property values?

How will testing multiple versions of the same module be handled for reporting as we need to know the results for the specific version of the module being checked.
If we build multiple configurations for multiple module versions, regression runtime will take longer.

We can request changes to make this faster/better in main checkstyle repo if it is reasonable.

This is blocked by default module testing.

DiffParser.parse should close/free resources on exceptions

public static List<GitChange> parse(String repositoryPath, String branchName)
throws IOException, GitAPIException {
final File gitDir = new File(repositoryPath, ".git");
final Repository repository = new FileRepositoryBuilder().setGitDir(gitDir)
.readEnvironment().findGitDir().build();
final Git git = new Git(repository);
final AbstractTreeIterator featureTreeParser =
prepareTreeParser(repository, Constants.R_HEADS + branchName);
final AbstractTreeIterator masterTreeParser =
prepareTreeParser(repository, Constants.R_HEADS + "master");
final List<GitChange> returnValue = git.diff()
.setOldTree(masterTreeParser)
.setNewTree(featureTreeParser)
.call()
.stream()
.map(DiffParser::convertDiffEntryToGitChange)
.collect(Collectors.toList());
git.close();
repository.close();
return returnValue;
}

private static AbstractTreeIterator prepareTreeParser(Repository repository, String ref)
throws IOException {
final Ref head = repository.exactRef(ref);
final RevWalk walk = new RevWalk(repository);
final RevCommit commit = walk.parseCommit(head.getObjectId());
final RevTree tree = walk.parseTree(commit.getTree().getId());
final CanonicalTreeParser treeParser = new CanonicalTreeParser();
final ObjectReader reader = repository.newObjectReader();
treeParser.reset(reader, tree.getId());
reader.close();
walk.dispose();
walk.close();
return treeParser;
}

We should always close resources, even if an exception occurs, to free up any locked resources.
Code should be changed to try-finally (not try-with-resource).
Powermock can be used to make exceptions happen and appease code coverage if required.

Parent Class Testing

Taken from #29 (comment) ,

If AbstractSuperCheck is modified, we need to determine that SuperCloneCheck and SuperFinalizeCheck need regression.

If we see a modification for a parent class of a module, than we need to run regression on all modules that extend that class.
It is possible that a change to parent class will not require changes to the module itself, but we should still see some new test(s) for it.

Should we extend this testing to interfaces or only classes?
Should we exclude some classes that almost every module implements like AbstractCheck and such?

This will be blocked by Issue #6 .

Report: using GroovyShell instead of running CLI directly

Taken from #48 (comment)

We are now using CLI way to invoke diff.groovy and it is better to do this by a Groovy invoker library, like GroovyShell.

An example of GroovyShell: https://github.com/Luolc/regression-tool/blob/dd02aa9019a7c02bafb028dcf7ec85e635256aa2/src/main/java/com/github/checkstyle/regression/report/ReportGenerator.java#L49.

More information about invoking Groovy script: http://groovy-lang.org/integrating.html

The reason we don't use GroovyShell now is stated at #48 (comment):

I am impossible to change the working directory of the invoker. There are many relative path used in the script, and we would have problems when invoking it outside its parent folder. Even in CLI, we are not able to run it outside.

Example of relative path in diff.groovy: https://github.com/checkstyle/contribution/blob/master/checkstyle-tester/diff.groovy#L183

The way to get the path of the script: https://stackoverflow.com/questions/1163093/how-do-you-get-the-path-of-the-running-script-in-groovy?answertab=votes#tab-top

We need to update diff.groovy if we want to use GroovyShell way.

Recognize difference between comments and code changes

Example commit: checkstyle/checkstyle@2a56547

Lots of no-inspection comments were added, only a few actual code changes but regression was still run on all modules in commit.

Generated configuration:

  <module name="TreeWalker">
    <module name="IndentationCheck"/>
    <module name="CyclomaticComplexityCheck"/>
  </module>
  <module name="UniquePropertiesCheck"/>

Only CyclomaticComplexityCheck had actual code changes and needs regression.

handle the tracking of deleted changes

Taken from #22 (review)

The newPath of deleted file is always /dev/null as mentioned at https://github.com/eclipse/jgit/blob/005e5feb4ecd08c4e4d141a38b9e7942accb3212/org.eclipse.jgit/src/org/eclipse/jgit/diff/DiffEntry.java#L373.
These could also reflected in our test https://github.com/Luolc/regression-tool/blob/c5d69e8a780fa84e7b414641f263e01e18a11f45/src/test/java/com/github/checkstyle/regression/git/DiffParserTest.java#L87.

In the early discussion I want to keep "src/test" and deleted files mainly because I thought it is better to keep the possibility of getting all the changes, and leave a filter interface to let the real strategy code do decide which type of changes to use.

But I am re-thinking to drop deleted files now. If we would treat "ADD" and "MODIFY" the same way, then there is no need to distinguish the ChangeType. And "DELETE" is for sure useless as the file is gone.

Fix diff problems when parsing a branch behind master

Taken from #22 (comment)
If the PR branch hasn't rebased on the latest commits of master, the GitChange list would contain the files that changed in master but haven't been merged into PR branch yet. And these are not expected.

As mentioned at #18 (comment), git diff master...<branch> could work, but there is no directly implementation of it in JGit.
Now I find a synonymous command:

$ git diff $(git merge-base master <branch>)..<branch>

So we just need to implement git merge-base master <branch> and run the common diff command.
There is no directly merge-base command in JGit either, but it seems not hard to implement it by some workaround. Ref: https://stackoverflow.com/questions/26434452/how-is-a-merge-base-done-in-jgit

Upgrade Checkstyle version and fix Regression

Based on CI run at #87 , Checkstyle has been modified too much that our regression isn't working anymore.

https://travis-ci.org/checkstyle/regression-tool/jobs/302059902

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/travis/build/checkstyle/regression-tool/checkstyle/src/test/java/com/puppycrawl/tools/checkstyle/ExtractInfoGeneratorTest.java:[34,48] cannot find symbol
symbol: class CheckUtil
location: package com.puppycrawl.tools.checkstyle.internal
[ERROR] /home/travis/build/checkstyle/regression-tool/checkstyle/src/test/java/com/puppycrawl/tools/checkstyle/ExtractInfoGeneratorTest.java:[49,56] cannot find symbol
symbol: variable CheckUtil
location: class com.puppycrawl.tools.checkstyle.ExtractInfoGeneratorTest
[INFO] 2 errors

Update checkstyle's regression configuration

We need to update checkstyle's standard regression configuration as it may not be testing everything.
See checkstyle/checkstyle#3656 .

We should add a new mode that builds a configuration file on all modules, not just ones that changed in git.
It needs to make a distinction between Normal modules + Java modules and Javadoc modules because Javadocs are slower.

These are the files that need to be updated:
https://github.com/checkstyle/contribution/blob/master/checkstyle-tester/checks-nonjavadoc-error.xml
https://github.com/checkstyle/contribution/blob/master/checkstyle-tester/checks-only-javadoc-error.xml

This is blocked by #5 .

DiffParser: ignore chmod change

Taken from #22 (comment)

If the only change is a chmod(no changes to the contents of the file), we should ignore the change altogether.

After this if fixed, the UT of only chmod change in #22 should update.

We will need a new test for this: chmod plus file change.

Double maven package fails with error in 'com.sun.tools.javac'

If I run mvn package -Passembly on a clean project, it passes fine.
However, when I run it a second time after the first without cleaning target, I get this error:

[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.github.checkstyle:regression-tool:jar:1.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found duplicate declaration of plugin org.apache.maven.plugins:maven-checkstyle-plugin @ line 156, column 21
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]                                                                       
[INFO] ------------------------------------------------------------------------
[INFO] Building regression-tool 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ regression-tool ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ regression-tool ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 26 source files to /home/ricky/opensource/regression-tool/target/classes
An exception has occurred in the compiler (1.8.0_131). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) after checking the Bug Database (http://bugs.java.com) for duplicates. Include your program and the following diagnostic in your report. Thank you.
java.lang.IllegalStateException: endPosTable already set
    at com.sun.tools.javac.util.DiagnosticSource.setEndPosTable(DiagnosticSource.java:136)
    at com.sun.tools.javac.util.Log.setEndPosTable(Log.java:350)
    at com.sun.tools.javac.main.JavaCompiler.parse(JavaCompiler.java:667)
    at com.sun.tools.javac.main.JavaCompiler.parseFiles(JavaCompiler.java:950)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.<init>(JavacProcessingEnvironment.java:892)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.next(JavacProcessingEnvironment.java:921)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1187)
    at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1170)
    at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:856)
    at com.sun.tools.javac.main.Main.compile(Main.java:523)
    at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
    at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
    at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:125)
    at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:169)
    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:823)
    at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:129)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] An unknown compilation problem occurred
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.655 s
[INFO] Finished at: 2017-07-31T13:44:00-04:00
[INFO] Final Memory: 27M/177M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project regression-tool: Compilation failure
[ERROR] An unknown compilation problem occurred
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

This is inside a linux VM.

Apache Maven 3.3.9
Maven home: /usr/share/maven
Java version: 1.8.0_131, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "4.8.0-59-generic", arch: "amd64", family: "unix"

Report Component

Taken from #2
We will need report component, which runs the configuration(s) and pushes the reports to a server. We should try to re-use the groovy script in contribution repo.

Add more to CI

We need to add code coverage. This is a priority.

We should split checkstyle and unit tests and code coverage to their own travis items so we get parallel running.
We should add sevntu checks.
We should add findbugs and pmd.

We need to add commit validation test from main repo.

Configuration Component

Taken from #2
configuration should be given only the information it needs to create the configuration(s). This would be the list of modules and the specified properties.

Extract Component - Generate Module Extract list

Taken from discussions in #29 ,

We need to generate a list of modules, properties, and hierarchy from checkstyle PR for our analysis and to identify full regression model needed.

This the list of information we need right now:

-- Module Information

Package
Class Name
Class hierarchy: Can contain multiple classes. If AbstractSuperCheck is modified, we need to determine that SuperCloneCheck and SuperFinalizeCheck need regression. Must contain full class path.
Interfaces: Same reason as class heirarchy.
Parent Module: Checker or TreeWalker
List of properties

-- Property Information

Name
Type
Default Value
Acceptable Tokens (only for Java and Javadoc checks)

Based on discussions at #19 (comment) and in issue:

As of right now, this probably has to be loaded as a file from a separate maven process.

invoke some simple program in our PR copy of Checkstyle through maven and have it generate the results for us that we need. It will pass the information back to our regression-tool's JVM by a file. We don't need the actual class in memory, we just need it's meta-information.

So we don't have to inject gson dependency into Checkstyle too, I think we should just write out file by hand.
It is ok if we can't cover external maven command by coverage or UT for now.

This is blocked by #19 .

Refine DiffParserTest

Taken from #17 (review)

  • We should include all types of differences in git.
    New file, removed file, renamed file, moved file, chmod change (linux only).
  • We should also add a test where PR branch is behind master. This happens quite a bit when people don't stay up to date with changes in master.
    We should still correctly identify PR branch commits in this scenario and only get it's differences.
  • We also need a test where PR branch has multiple commits on master.

Regression on code changes and message changes

Identified in PR checkstyle/checkstyle#4903 ,

If a PR makes changes to Check logic and changes to a existing message, normal regression will display too many differences as all will be centered around change in message and it will be impossible/hard to discern if any where from code changes.

In this situation, we should force check's message to a specific message for both base and patch runs so only the differences in the code is examined.
Example of configuration for the PR mentioned:

<?xml version="1.0"?>
<!DOCTYPE module PUBLIC
          "-//Puppy Crawl//DTD Check Configuration 1.3//EN"
          "http://www.puppycrawl.com/dtds/configuration_1_3.dtd">

<module name="Checker">
    <property name="charset" value="UTF-8"/>
    <property name="severity" value="warning"/>
    <property name="haltOnException" value="false"/>

    <module name="TreeWalker">
        <module name="SummaryJavadoc">
            <property name="forbiddenSummaryFragments" value="^@return the *|^This method returns |^A [{]@code [a-zA-Z0-9]+[}]( is a )"/>
            <message key="summary.first.sentence" value="First sentence of Javadoc is missing an ending period."/>
        </module>
    </module>
</module>

Notice the message is only for summary.first.sentence as that was what was modified in the PR.

Main

We need a main class that users will use to access and run 'regression-tool'.

It needs to take in custom parameters, validate the parameters, and use those values to execute the tool.
If any parameters are wrongly entered, we need to give an informative error message and exit with non-zero code.
If an exception happens in any component, we should rethrow it all the way up to the Main class and display it to the user.

What kind of parameters will we accept? Which are required, which can be optional, and what will be the default values of the optional ones?
We will need a type of help menu, to display to users the list of parameters and their brief descriptions and anything else they need to know to use them correctly.

Blocked by Issue #19, #20, #21 .

git: add line changes detection

We need to update GitChange. Add a field to store the line numbers of the changed code.
And update DiffParser to let it be able to detect line changes.

Message Testing

If a module changes, we will always do regression on it.

Assuming a module is not changed, but a message of the module is (spelling/grammar fixes), do we need to run regression on the module still?
Does it need to be a full regression, or can we minimize what is tested somehow?

Checkstyle repo already does UT on message, so it should be verified that way if change works. So do we need to do anything more?

This is blocked by #6 (currently) and should be done dead last as it is still not clear if we need to do anything for this.

Git component

Taken from #1 ,

We need to analyze git repository for changes in new commits in PR branch.

Notes:

  • We most ignore deleted files. Deleted checks and such can never be checked for changes as they are gone. Moved/Renamed files should stay to make sure there is no issue using them.
  • We must only grab files from src/main. Any changes in tests/it should be ignored.

Should we also grab the contents of the changes? Will they be needed?

Restricting files further by type should be done elsewhere as we will have to look at class hierarchy and such.

NoException Checkstyle Project Regression

We need to consider running the snapshot version of the jar against the real Checkstyle project as a final test to make sure any new changes work without throwing an exception.

The question is what commit should we run against? Only the latest one or the top X? Should we create our own commit to test some basic avenues?

Reports shouldn't be uploaded, regardless.

This is completely separate from the junits.

This is blocked by default module testing.

These regression(s) will also showcase to Checkstyle project how to use our utility and incorporate it into their process.

Module Extract File

At the request of @romani ,
This is broken off from #19 (comment) to discuss producing and maintaining a physical file for regression-tool to use to determine what are modules, properties, etc.
This can also be used to discuss who should own this file creator, Checkstyle or regression-tool.

What items we need:

-- Module Information

Package
Class Name
Class hierarchy: Can contain multiple classes. If AbstractSuperCheck is modified, we need to determine that SuperCloneCheck and SuperFinalizeCheck need regression. Must contain full class path.
Interfaces: Same reason as class heirarchy.
Parent Module: Checker or TreeWalker
List of properties
List of messages

-- Property Information

Name
Type
Default Value
Acceptable Tokens (only for Java and Javadoc checks)

-- Message Information

Key name

Layout of XML file:

    <module>
        <package></package>
        <name></name>
        <parent></parent>
        <heirarchy>
            <class></class>
            <class></class>
        </heirarchy>
        <interfaces>
            <class></class>
            <class></class>
        </interfaces>
        <properties>
            <property>
                <name></name>
                <type></type>
                <acceptableTokens></acceptableTokens>
                <acceptableJavadocTokens></acceptableJavadocTokens>
                <default></default>
            </property>
        </properties>
        <messages>
            <message></message>
        </messages>
    </module>
</modules>

Example Check, FinalLocalVariableCheck: (http://checkstyle.sourceforge.net/config_coding.html#FinalLocalVariable)

<modules>
    <module>
        <package>com.puppycrawl.tools.checkstyle.checks.coding</package>
        <name>FinalLocalVariableCheck</name>
        <parent>TreeWalker</parent>
        <heirarchy>
            <class>com.puppycrawl.tools.checkstyle.api.AbstractCheck</class>
            <class>com.puppycrawl.tools.checkstyle.api.AbstractViolationReporter</class>
            <class>com.puppycrawl.tools.checkstyle.api.AutomaticBean</class>
        </heirarchy>
        <interfaces>
            <class>com.puppycrawl.tools.checkstyle.api.Configurable</class>
            <class>com.puppycrawl.tools.checkstyle.api.Contextualizable</class>
        </interfaces>
        <properties>
            <property>
                <name>validateEnhancedForLoopVariable</name>
                <type>boolean</type>
                <default>false</default>
            </property>
            <property>
                <name>tokens</name>
                <type>tokens</type>
                <acceptableTokens>VARIABLE_DEF, PARAMETER_DEF</acceptableTokens>
                <default>VARIABLE_DEF</default>
            </property>
        </properties>
        <messages>
            <message>final.variable</message>
        </messages>
    </module>
</modules>

@Luolc Let me know if you agree or think we need anything else.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.