oss-review-toolkit / ort Goto Github PK
View Code? Open in Web Editor NEWA suite of tools to automate software compliance checks.
Home Page: https://oss-review-toolkit.org
License: Apache License 2.0
A suite of tools to automate software compliance checks.
Home Page: https://oss-review-toolkit.org
License: Apache License 2.0
Currently the Tooling API model classes only have fields for errors. Also add fields for warnings, because not everything reported there should be an error, for example the usage of flat dir repositories (see #251).
See https://buckbuild.com/ https://buck.build/ as used e.g. by https://github.com/SeleniumHQ/selenium/tree/9a65d4cb3ab44d5d1c77532aece8517d7678388d/java and older versions of Gerrit (newer versions have moved to Bazel).
PyPI recently moved from pypi.python.org to pypi.org. When querying the API for package metadata, we should use this domain instead of relying on working HTTP redirects.
Add support for the CocoaPods dependency manager to the analyzer module.
Since this is an open Source compliance toolkit, I would love to see the best practises for a good OSS project realized in this project.
Some of the best practises are:
The FSFE hast set up a Project with best practizes relevant for smart license compliance support, the proejct is called reuse. Please see https://reuse.software/
No file of the current release carries a standarde license haeder, there is no license information in the files at all. The same applies to the copyright information, except the README.md file.
Regarding the README.md file, please remove the statement "See the LICENSE file in the root of this project for license details." because no license scanner can "auto conclude" such a statement.
The file oss-review-toolkit-preview/analyzer/src/funTest/assets/projects/synthetic/mixed/NPMNestedMaven/package.json
{
"name": "npmnestedmaven",
"version": "1.0.0",
"description": "A dummy NPM project with a nested Maven project",
"scripts": {
"test": "echo "Error: no test specified" && exit 1"
},
"author": "Sebastian Schuberth",
"license": "ISC",
"repository": {
"type": "git",
"url": "https://github.com/heremaps/oss-review-toolkit.git"
}
}
Carries a license information which doea not fit. Even in case of test files you should care for license hygiene
Currently, we support SBT by running sbt makePom
and then using our Maven-backend on the generated POMs. While that works fairly well, there are edge-cases where the results are incomplete e.g. for some inter-module dependencies or dynamically generated projects. Ideas to improve this include using Coursier directly, or maybe yet better the Build Server Protocol for Scala.
A VCS repository from which releases are created is preferrable to the location of a source artifact ("sdist" in Python lingo). There is a rarely used attribute in setup.py
where this information can be recorded. Make sure this information is captured, if it is present.
If the VCS of a package can not be used, for example because it is undefined, invalid, or requires authorization, fall back to using the source artifact to get the source code.
The Python Packaging Guide specifies what characters can be used in the name
field in setup.py. For some illegal characters (e.g. spaces), setuptools does not throw an error, but replaces them with dashes during installation.
The analyzer uses pydep
, which hooks setup()
and captures it's keyword arguments verbatim. Therefore it fails when searching for the original name in the output of pipdeptree
.
Hi,
I can't compile the current master branch and get an error on Task :analyzer:compileKotlin
I have :
$ dpkg -l
ii openjdk-8-jdk:amd64 8u162-b12-1 amd64 OpenJDK Development Kit (JDK)
JAVA_HOME
environment variable is set.$ echo $JAVA_HOME
/usr/lib/jvm/java-8-openjdk-amd64
git clone --recurse-submodules https://github.com/heremaps/oss-review-toolkit.git
cd oss-review-toolkit
./gradlew installDist
Successfull build
FAILURE: Build failed with an exception.
What went wrong:
Execution failed for task ':analyzer:compileKotlin'.
(More details below)
Did I miss somethig ?
Thanks,
Camille
Task :analyzer:compileKotlin
file or directory '/home/camille/Devel/oss-review-toolkit/analyzer/src/main/java', not found
Build cache key for task ':analyzer:compileKotlin' is 4b3ff08342112e8deffd6c1f2d4a5f2b
Task ':analyzer:compileKotlin' is not up-to-date because:
Task has failed previously.
All input files are considered out-of-date for incremental task ':analyzer:compileKotlin'.
file or directory '/home/camille/Devel/oss-review-toolkit/analyzer/src/main/java', not found
file or directory '/home/camille/Devel/oss-review-toolkit/analyzer/src/main/java', not found
Using Kotlin incremental compilation
Options for KOTLIN DAEMON: IncrementalCompilationOptions(super=CompilationOptions(compilerMode=INCREMENTAL_COMPILER, targetPlatform=JVM, reportCategories=[0], reportSeverity=2, requestedCompilationResults=[0]), areFileChangesKnown=false, modifiedFiles=null, deletedFiles=null, workingDir=/home/camille/Devel/oss-review-toolkit/analyzer/build/kotlin/compileKotlin, customCacheVersionFileName='gradle-format-version.txt', customCacheVersion=4, resultDifferenceFile=/home/camille/Devel/oss-review-toolkit/analyzer/build/kotlin/compileKotlin/build-history.bin, friendDifferenceFile=null, usePreciseJavaTracking=truelocalStateDirs=[/home/camille/Devel/oss-review-toolkit/analyzer/build/classes/kotlin/main])
e: /home/camille/Devel/oss-review-toolkit/analyzer/src/main/kotlin/managers/Bundler.kt: (144, 13): No value passed for parameter 'definitionFilePath'
Task :analyzer:compileKotlin FAILED
[KOTLIN] deleting /home/camille/Devel/oss-review-toolkit/analyzer/build/classes/kotlin/main on error
[KOTLIN] deleting /home/camille/Devel/oss-review-toolkit/analyzer/build/classes/kotlin/main on error
:analyzer:compileKotlin (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 5.363 secs.
FAILURE: Build failed with an exception.
Compilation error. See log for more details
BUILD FAILED in 7s
17 actionable tasks: 1 executed, 16 up-to-date
There are broken links to the subprojects analyzer, graph, downloader, and scanner in the README file. Also, the link to the license file isn't correct.
Add support for the Conan dependency manager to the analyzer module.
Currently the analyzer creates one YAML file per defintion file. These files different files can be hard to handle, and also can contain a lot of duplication in the package list.
Add an option to the analyzer to create a merged YAML file that contains the results for all definition files. The structure should be:
Add support for the PHP Composer package manger to the analyer module.
For re-using already known scan results, ORT can use a remote cache. Currently only Artifactory is supported but we would like to add AWS S3 support as it more commonly used and cheaper to set up than Artifactory instance.
See also: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-s3.html
Here's what I did.
./gradlew sync
: worked./gradlew check
: failed
hg
could not be found; should skip these tests./scanner/src/funTest/assets/scanners/scancode-toolkit/
could not be found; should just work.Turns out that for that last issue, pulling git submodules gets one step further, so that should be added to the instructions.
But the test case still fails; the subsequent step that looks for configure
in the same directory now works. I'm wondering why you're looking for configure
if not to build binaries in that directory, and if so, why they're not built.
Lastly, gradlew check
at some point just hangs, or seems to hang, with no indication what's going on. I killed it after about 15 minutes run-time.
At this point I'd abandon the project unless I had immediate need for it. I'm not sure how non-technical folk just interested in a compliance tool would react :)
Add support for Hex packages (projects written in Elixir and Erlang) to the ORT analyzer module.
Hex is a package manager for the BEAM ecosystem, any language that compiles to run on the BEAM VM, such as Elixir and Erlang, can be used to build Hex packages.... Many build tools support Hex packages including Mix for Elixir projects, rebar and erlang.mk for Erlang projects.
However, the use of the build tools is not as clear-cut as this, see also Requirements 1 below. In general, a Hex package can use the following build tools:
- Mix:
- The presence of mix.exs and/or mix.lock may indicate that the projext build tool is Mix
- rebar:
- The presence of rebar.config, rebar.lock, and/or src/.app.src file makes it likely the project build tool is rebar3.
- We could use the above files for the globsForDefinitionFiles
However, the use of the build tools is not as clean-cut as this. For example, the project ssl_verify_fun makes use of both Mix and rebar3.
- Mix:
- Licenses can be declared in the mix.exs see https://github.com/semlabs/kudos/blob/c36f23760992793962fd90aabf0630ccbd6e7530/mix.exs#L41
- Is this a standard? Mix docs do not provide an answer.
- rebar3:
- Licenses can be declared in [src/.app.src file](https://hex.pm/docs/rebar3_publish#adding-metadata-to-code-clas (and Mix/rebar3) sinlinesrcltmyappgtappsrccode)
Options:
rebar3:
- use
rebar3 deps
orrebar3 pkgs
, but note there is no machine-readable output:→ rebar3 deps ... cowboy* (package) recon* (git source) erlware_commons (locked git source) getopt* (locked git source) providers (locked hg source) relx (locked git source)Erlang have scopes
→ rebar3 tree ... |- bootstrap-0.0.2 (git repo) |- dirmon-0.1.0 (project app)any |- file_monitor-0.1 (git repo) |- peeranha-0.1.0 (git repo) | |- gproc-git (git repo) | |- interclock-0.1.2 (git repo) | | |- bitcask-1.7.0 (git repo) | | | |- lager-2.1.1 (hex package) | | | | |- goldrush-0.1.6 (hex package) | | |- itc-1.0.0 (git repo) | |- merklet-1.0.0 (git repo) |- recon-2.2.2 (git repo) on which to test Mix/rebar3 support |- uuid-1.5.0 (git repo) | |- quickrand-1.5.0 (git repo)
Documentation:
- Mix:
- install and use licensir -- lists all licenses (Mix projects only)
- (or) write a script (in Erlang?) to loop over definition files and use Mix.Dep API (see https://github.com/semlabs/kudos/blob/master/lib/kudos.ex#L83-L100), but generate JSON
- https://repo.hex.pm???
- How can ORT's downloader download sources for an Erlang package?
- Further investigation for Mix required: https://hexdocs.pm/mix/Mix.SCM.html.
- Are scopes supported/recognised in Mix/rebar3?
- rebar3 has the concept of profiles (default, native, prod, test) which we possibly could use to do this.
Example Hex projects:
- Mix:
- https://github.com/jshmrtn/iona -- mix.exs names dependencies and package license
- https://github.com/unnawut/licensir -- mix.exs names dependencies
- rebar3:
- https://github.com/certifi/erlang-certifi -- no dependencies in rebar.config, licenses in src/certifi.app.src
- https://github.com/benoitc/erlang-idna
- rebar3 and erlang.mk:
- https://hex.pm/packages/mimerl -- rebar.config does not list dependencies, they are fetched by erlang.mk
- Both Mix and rebar3:
- https://hex.pm/packages/ssl_verify_fun -- neither mix.exs nor rebar.config list any dependencies
Lack of file makes it difficult to determine how ORT maintainers like to see contributions.
In general I've been advocating building everything from source, so that we can be sure that we are working with known components (and extra points for verifying bit-for-bit reproducibility).
If we trust the package managers there are at least a couple of risks:
Add functional test to the downloader module to verify its functionality.
See http://darcs.net/, which seems to be popular in the Haskell community.
Consider redesigning the package identifier
as a "puurl" as described at nexB/scancode-toolkit#805 (comment).
In the Git downloader there is some code to refine the vcs revision that is fetched (e.g., looking up the package version as a tag). That gets the right source but the revision fetched is never recorded in the package data as far as I can tell. This info is key to get out and recorded somewhere.
Add support for the Swift dependency manager to the analyzer module.
https://github.com/apple/swift-package-manager
swift package show-dependencies --format json
is the command to fetch dependencies according to https://www.cocoawithlove.com/blog/package-manager-fetch.html
Related materials:
Add support for the Cargo dependency manager to the analyzer module.
https://github.com/rust-lang/cargo
https://doc.rust-lang.org/cargo/
Analyzer checks yarn's version using yarn --version
while yarn expects yarn version
.
In order to build OSGi bundles
http://bndtools.org/
can be used.
Not sure if you could scan bnd.bnd files to at least list possible dependencies as I assume analyzing its dependencies is not that easy as one can import raw JARs and bundle them in your target JAR.
See https://ossindex.sonatype.org/ and related resources:
Add support for the LiD license scanner to the scanner module.
https://github.com/codeauroraforum/lid
(previously) https://source.codeaurora.org/external/qostg/lid/about/
Packages have a packageManager
(e.g, npm, maven, ...). Since a given type of package could come from many different places, the Package
should also talk about a provider
. The provider
should not be the URL of the repository, rather the notional name of the repository (e.g., npmjs.org, github.com). This allows the repos to move and change their URL structure without affecting the identity of the data stored in ORT.
Add support for the BitBake (Yocto) dependency manager to the analyzer module.
Links:
BitBake manual: https://www.yoctoproject.org/docs/1.6/bitbake-user-manual/bitbake-user-manual.html
Useful Bitbake commands: https://community.nxp.com/docs/DOC-94953
Bitbake repos: https://git.yoctoproject.org/
Projects starting with poky-* and meta-* likely contain bitbake files
Add support for Godep to the analyzer module.
Add support for the NuGet dependency manager to the analyzer module.
Add support for Mercurial repositories to the downloader module.
Add support for CVS repositories to the downloader module.
The current Python/pip implementation does not try to determine which Python version to use for the project being analyzed and instead always uses Python 2.7.
There are ways to specify supported Python versions in both setup.py and requirements.txt files (as well as tox.ini and other files commonly found in Python projects). The analyzer could use that information to decide which version of Python/pip to invoke for gathering information about dependencies.
Add a parameter to the scanner module to select which scopes will be scanned.
As another simple example on how to integrate a scanner, adding support for Licensee would be nice.
Add support for the Carthage dependency manager to the analyzer module.
Add support for the peerDependencies scope to NPM. Be aware that the specific version of the peer dependency will only be resolved in the context of a project using the library.
npm-install-peers can be used to install peer dependencies, see this relevant ticket:
spatie/npm-install-peers#13
See https://bazel.build/ as used by newer versions of Gerrit .
As another simple example on how to integrate a scanner, adding support for LicenseFinder would be nice.
I'll just dump the log here.
licensee
in particular appears to come from a submodule, but is not built.$ ./gradlew check
Starting a Gradle Daemon (subsequent builds will be faster)
Parallel execution with configuration on demand is an incubating feature.
> Task :utils-test:detektCheck
. :utils:detektCheck
> Task :graph:detektCheck
. :analyzer:detektCheck
> Task :utils-test:detektCheck
1 kotlin files were analyzed.
Ruleset: comments
Ruleset: complexity
Ruleset: empty-blocks
Ruleset: exceptions
Ruleset: performance
Ruleset: potential-bugs
Ruleset: style
Complexity Report:
- 27 lines of code (loc)
- 4 source lines of code (sloc)
- 2 logical lines of code (lloc)
- 18 comment lines of code (cloc)
- 0 McCabe complexity (mcc)
- 0 number of total code smells
- 450 % comment source ratio
- 0 mcc per 1000 lloc
- 0 code smells per 1000 lloc
Project Statistics:
- number of properties: 0
- number of functions: 0
- number of classes: 0
- number of packages: 1
- number of kt files: 1
detekt finished in 1958 ms.
> Task :graph:detektCheck
1 kotlin files were analyzed.
Ruleset: comments
Ruleset: complexity
Ruleset: empty-blocks
Ruleset: exceptions
Ruleset: performance
Ruleset: potential-bugs
Ruleset: style
Complexity Report:
- 160 lines of code (loc)
- 104 source lines of code (sloc)
- 74 logical lines of code (lloc)
- 27 comment lines of code (cloc)
- 13 McCabe complexity (mcc)
- 0 number of total code smells
- 25 % comment source ratio
- 175 mcc per 1000 lloc
- 0 code smells per 1000 lloc
Project Statistics:
- number of properties: 16
- number of functions: 3
- number of classes: 0
- number of packages: 1
- number of kt files: 1
detekt finished in 2393 ms.
> Task :downloader:detektCheck
.......
> Task :model:detektCheck
.................
17 kotlin files were analyzed.
Ruleset: comments
Ruleset: complexity
Ruleset: empty-blocks
Ruleset: exceptions
Ruleset: performance
Ruleset: potential-bugs
Ruleset: style
Complexity Report:
- 1705 lines of code (loc)
- 848 source lines of code (sloc)
- 650 logical lines of code (lloc)
- 648 comment lines of code (cloc)
- 63 McCabe complexity (mcc)
- 0 number of total code smells
- 76 % comment source ratio
- 96 mcc per 1000 lloc
- 0 code smells per 1000 lloc
Project Statistics:
- number of properties: 64
- number of functions: 28
- number of classes: 31
- number of packages: 1
- number of kt files: 17
detekt finished in 2868 ms.
> Task :analyzer:detektCheck
........
> Task :downloader:detektCheck
..............
21 kotlin files were analyzed.
Ruleset: comments
Ruleset: complexity
Ruleset: empty-blocks
Ruleset: exceptions
Ruleset: performance
Ruleset: potential-bugs
Ruleset: style
Complexity Report:
- 3313 lines of code (loc)
- 2189 source lines of code (sloc)
- 1553 logical lines of code (lloc)
- 588 comment lines of code (cloc)
- 277 McCabe complexity (mcc)
- 0 number of total code smells
- 26 % comment source ratio
- 178 mcc per 1000 lloc
- 0 code smells per 1000 lloc
Project Statistics:
- number of properties: 295
- number of functions: 93
- number of classes: 29
- number of packages: 2
- number of kt files: 21
detekt finished in 3201 ms.
> Task :utils:detektCheck
........
8 kotlin files were analyzed.
> Task :scanner:detektCheck
........oader:compileKotlin
> Task :utils:detektCheck
Ruleset: comments
Ruleset: complexity
Ruleset: empty-blocks
Ruleset: exceptions
Ruleset: performance
Ruleset: potential-bugs
Ruleset: style
Complexity Report:
- 1414 lines of code (loc)
- 851 source lines of code (sloc)
- 594 logical lines of code (lloc)
- 349 comment lines of code (cloc)
- 128 McCabe complexity (mcc)
- 0 number of total code smells
- 41 % comment source ratio
- 215 mcc per 1000 lloc
- 0 code smells per 1000 lloc
Project Statistics:
- number of properties: 124
- number of functions: 36
- number of classes: 4
- number of packages: 2
- number of kt files: 8
detekt finished in 2778 ms.
> Task :analyzer:detektCheck
........................
32 kotlin files were analyzed.
Ruleset: comments
Ruleset: complexity
Ruleset: empty-blocks
Ruleset: exceptions
Ruleset: performance
Ruleset: potential-bugs
Ruleset: style
Complexity Report:
- 4263 lines of code (loc)
- 2651 source lines of code (sloc)
- 1585 logical lines of code (lloc)
- 902 comment lines of code (cloc)
- 358 McCabe complexity (mcc)
- 0 number of total code smells
- 34 % comment source ratio
- 225 mcc per 1000 lloc
- 0 code smells per 1000 lloc
Project Statistics:
- number of properties: 460
- number of functions: 116
- number of classes: 36
- number of packages: 6
- number of kt files: 32
detekt finished in 3429 ms.
> Task :scanner:detektCheck
.....
13 kotlin files were analyzed.
Ruleset: comments
Ruleset: complexity
Ruleset: empty-blocks
Ruleset: exceptions
Ruleset: performance
Ruleset: potential-bugs
Ruleset: style
Complexity Report:
- 1657 lines of code (loc)
- 1022 source lines of code (sloc)
- 730 logical lines of code (lloc)
- 356 comment lines of code (cloc)
- 169 McCabe complexity (mcc)
- 0 number of total code smells
- 34 % comment source ratio
- 231 mcc per 1000 lloc
- 0 code smells per 1000 lloc
Project Statistics:
- number of properties: 145
- number of functions: 45
- number of classes: 16
- number of packages: 2
- number of kt files: 13
detekt finished in 3997 ms.
> Task :downloader:test
com.here.ort.downloader.vcs.GitTest > GitTest.Detected Git version is not empty STARTED
com.here.ort.downloader.vcs.GitTest > GitTest.Detected Git version is not empty PASSED
com.here.ort.downloader.vcs.GitTest > GitTest.Git detects non-working-trees STARTED
com.here.ort.downloader.vcs.GitTest > GitTest.Git detects non-working-trees PASSED
com.here.ort.downloader.vcs.GitTest > GitTest.Git correctly detects URLs to remote repositories STARTED
> Task :analyzer:test
com.here.ort.analyzer.MergedResultsBuilderTest > MergedResultsBuilderTest.MergedResultsBuilder merges results from all files STARTED
com.here.ort.analyzer.MergedResultsBuilderTest > MergedResultsBuilderTest.MergedResultsBuilder merges results from all files PASSED
com.here.ort.analyzer.YamlFilePackageCurationProviderTest > YamlFilePackageCurationProviderTest.Provider can read YAML file STARTED
com.here.ort.analyzer.YamlFilePackageCurationProviderTest > YamlFilePackageCurationProviderTest.Provider can read YAML file PASSED
com.here.ort.analyzer.YamlFilePackageCurationProviderTest > YamlFilePackageCurationProviderTest.Provider returns only matching curations STARTED
com.here.ort.analyzer.YamlFilePackageCurationProviderTest > YamlFilePackageCurationProviderTest.Provider returns only matching curations PASSED
com.here.ort.utils.NpmTest > expandShortcutURL.should do nothing for empty URLs STARTED
com.here.ort.utils.NpmTest > expandShortcutURL.should do nothing for empty URLs PASSED
com.here.ort.utils.NpmTest > expandShortcutURL.should properly handle NPM shortcut URLs STARTED
com.here.ort.utils.NpmTest > expandShortcutURL.should properly handle NPM shortcut URLs PASSED
com.here.ort.util.PackageManagerTest > findManagedFiles.should find all managed files STARTED
com.here.ort.util.PackageManagerTest > findManagedFiles.should find all managed files PASSED
com.here.ort.util.PackageManagerTest > findManagedFiles.should find only files for active package managers STARTED
com.here.ort.util.PackageManagerTest > findManagedFiles.should find only files for active package managers PASSED
com.here.ort.util.PackageManagerTest > findManagedFiles.should find no files if no package managers are active STARTED
com.here.ort.util.PackageManagerTest > findManagedFiles.should find no files if no package managers are active PASSED
com.here.ort.util.PackageManagerTest > findManagedFiles.should fail if the provided file is not a directory STARTED
com.here.ort.util.PackageManagerTest > findManagedFiles.should fail if the provided file is not a directory PASSED
> Task :downloader:test
com.here.ort.downloader.vcs.GitTest > GitTest.Git correctly detects URLs to remote repositories PASSED
com.here.ort.downloader.vcs.GitTest > GitTest.Detected Git working tree information is correct STARTED
com.here.ort.downloader.vcs.GitTest > GitTest.Detected Git working tree information is correct PASSED
com.here.ort.downloader.vcs.GitTest > GitTest.Git correctly lists remote tags STARTED
com.here.ort.downloader.vcs.GitTest > GitTest.Git correctly lists remote tags PASSED
> Task :scanner:funTest
com.here.ort.scanner.HttpCacheTest > HttpCacheTest.HTTP GET returns what was PUT STARTED
com.here.ort.scanner.HttpCacheTest > HttpCacheTest.HTTP GET returns what was PUT PASSED
com.here.ort.scanner.ScanPathTest > ScanPathTest.BoyterLc recognizes our own LICENSE STARTED
> Task :downloader:test
com.here.ort.downloader.vcs.CvsTest > CvsTest.Detected CVS version is not empty STARTED
com.here.ort.downloader.vcs.CvsTest > CvsTest.Detected CVS version is not empty SKIPPED
com.here.ort.downloader.vcs.CvsTest > CvsTest.CVS detects non-working-trees STARTED
com.here.ort.downloader.vcs.CvsTest > CvsTest.CVS detects non-working-trees SKIPPED
com.here.ort.downloader.vcs.CvsTest > CvsTest.CVS correctly detects URLs to remote repositories STARTED
com.here.ort.downloader.vcs.CvsTest > CvsTest.CVS correctly detects URLs to remote repositories SKIPPED
com.here.ort.downloader.vcs.CvsTest > CvsTest.Detected CVS working tree information is correct STARTED
com.here.ort.downloader.vcs.CvsTest > CvsTest.Detected CVS working tree information is correct SKIPPED
com.here.ort.downloader.vcs.CvsTest > CvsTest.CVS correctly lists remote tags STARTED
com.here.ort.downloader.vcs.CvsTest > CvsTest.CVS correctly lists remote tags SKIPPED
com.here.ort.downloader.MercurialTest > MercurialTest.Detected Mercurial version is not empty STARTED
com.here.ort.downloader.MercurialTest > MercurialTest.Detected Mercurial version is not empty SKIPPED
com.here.ort.downloader.MercurialTest > MercurialTest.Mercurial detects non-working-trees STARTED
com.here.ort.downloader.MercurialTest > MercurialTest.Mercurial detects non-working-trees SKIPPED
com.here.ort.downloader.MercurialTest > MercurialTest.Mercurial correctly detects URLs to remote repositories STARTED
com.here.ort.downloader.MercurialTest > MercurialTest.Mercurial correctly detects URLs to remote repositories SKIPPED
com.here.ort.downloader.MercurialTest > MercurialTest.Detected Mercurial working tree information is correct STARTED
com.here.ort.downloader.MercurialTest > MercurialTest.Detected Mercurial working tree information is correct SKIPPED
com.here.ort.downloader.MercurialTest > MercurialTest.Mercurial correctly lists remote tags STARTED
com.here.ort.downloader.MercurialTest > MercurialTest.Mercurial correctly lists remote tags SKIPPED
com.here.ort.downloader.SubversionTest > SubversionTest.Detected Subversion version is not empty STARTED
com.here.ort.downloader.SubversionTest > SubversionTest.Detected Subversion version is not empty PASSED
com.here.ort.downloader.SubversionTest > SubversionTest.Subversion detects non-working-trees STARTED
com.here.ort.downloader.SubversionTest > SubversionTest.Subversion detects non-working-trees PASSED
com.here.ort.downloader.SubversionTest > SubversionTest.Subversion correctly detects URLs to remote repositories STARTED
> Task :analyzer:funTest
com.here.ort.analyzer.BabelTest > Babel dependencies.should be correctly analyzed STARTED
com.here.ort.analyzer.BabelTest > Babel dependencies.should be correctly analyzed FAILED
java.io.IOException: Cannot run program "npm": error=2, No such file or directory
Caused by:
java.io.IOException: error=2, No such file or directory
com.here.ort.analyzer.MainTest > MainTest.Activating only Gradle works STARTED
> Task :scanner:funTest
com.here.ort.scanner.ScanPathTest > ScanPathTest.BoyterLc recognizes our own LICENSE PASSED
com.here.ort.scanner.ScanPathTest > ScanPathTest.Licensee recognizes our own LICENSE STARTED
com.here.ort.scanner.ScanPathTest > ScanPathTest.Licensee recognizes our own LICENSE FAILED
java.io.IOException: Cannot run program "licensee" (in directory ".."): error=2, No such file or directory
Caused by:
java.io.IOException: error=2, No such file or directory
com.here.ort.scanner.ScanPathTest > ScanPathTest.ScanCode recognizes our own LICENSE STARTED
> Task :downloader:test
com.here.ort.downloader.SubversionTest > SubversionTest.Subversion correctly detects URLs to remote repositories PASSED
com.here.ort.downloader.SubversionTest > SubversionTest.Detected Subversion working tree information is correct STARTED
com.here.ort.downloader.SubversionTest > SubversionTest.Detected Subversion working tree information is correct PASSED
com.here.ort.downloader.SubversionTest > SubversionTest.Subversion correctly lists remote tags STARTED
> Task :analyzer:funTest
com.here.ort.analyzer.MainTest > MainTest.Activating only Gradle works PASSED
com.here.ort.analyzer.MainTest > MainTest.Activating only NPM works STARTED
com.here.ort.analyzer.MainTest > MainTest.Activating only NPM works FAILED
java.io.IOException: Cannot run program "npm": error=2, No such file or directory
Caused by:
java.io.IOException: error=2, No such file or directory
com.here.ort.analyzer.MainTest > MainTest.Merging into single results file creates correct output STARTED
com.here.ort.analyzer.MainTest > MainTest.Merging into single results file creates correct output PASSED
com.here.ort.analyzer.MainTest > MainTest.Package curation data file is applied correctly STARTED
com.here.ort.analyzer.MainTest > MainTest.Package curation data file is applied correctly PASSED
com.here.ort.analyzer.NpmTest > NPM should.resolve shrinkwrap dependencies correctly STARTED
com.here.ort.analyzer.NpmTest > NPM should.resolve shrinkwrap dependencies correctly FAILED
java.io.IOException: Cannot run program "npm": error=2, No such file or directory
Caused by:
java.io.IOException: error=2, No such file or directory
com.here.ort.analyzer.NpmTest > NPM should.resolve package-lock dependencies correctly STARTED
com.here.ort.analyzer.NpmTest > NPM should.resolve package-lock dependencies correctly FAILED
java.io.IOException: Cannot run program "npm": error=2, No such file or directory
Caused by:
java.io.IOException: error=2, No such file or directory
com.here.ort.analyzer.NpmTest > NPM should.show error if no lockfile is present STARTED
com.here.ort.analyzer.NpmTest > NPM should.show error if no lockfile is present FAILED
java.io.IOException: Cannot run program "npm": error=2, No such file or directory
Caused by:
java.io.IOException: error=2, No such file or directory
com.here.ort.analyzer.NpmTest > NPM should.show error if multiple lockfiles are present STARTED
com.here.ort.analyzer.NpmTest > NPM should.show error if multiple lockfiles are present FAILED
java.io.IOException: Cannot run program "npm": error=2, No such file or directory
Caused by:
java.io.IOException: error=2, No such file or directory
com.here.ort.analyzer.NpmTest > NPM should.resolve dependencies even if the node_modules directory already exists STARTED
com.here.ort.analyzer.NpmTest > NPM should.resolve dependencies even if the node_modules directory already exists FAILED
java.io.IOException: Cannot run program "npm": error=2, No such file or directory
Caused by:
java.io.IOException: error=2, No such file or directory
com.here.ort.analyzer.NpmTest > yarn should.resolve dependencies correctly STARTED
com.here.ort.analyzer.NpmTest > yarn should.resolve dependencies correctly FAILED
java.io.IOException: Cannot run program "npm": error=2, No such file or directory
Caused by:
java.io.IOException: error=2, No such file or directory
com.here.ort.analyzer.PipTest > setup py dependencies.should be resolved correctly for spdx-tools-python STARTED
> Task :downloader:test
com.here.ort.downloader.SubversionTest > SubversionTest.Subversion correctly lists remote tags PASSED
com.here.ort.downloader.VersionControlSystemTest > For an absolute working directory, getPathToRoot .should work if given absolute paths STARTED
com.here.ort.downloader.VersionControlSystemTest > For an absolute working directory, getPathToRoot .should work if given absolute paths PASSED
com.here.ort.downloader.VersionControlSystemTest > For an absolute working directory, getPathToRoot .should work if given relative paths STARTED
com.here.ort.downloader.VersionControlSystemTest > For an absolute working directory, getPathToRoot .should work if given relative paths PASSED
com.here.ort.downloader.VersionControlSystemTest > For a relative working directory, getPathToRoot .should work if given absolute paths STARTED
com.here.ort.downloader.VersionControlSystemTest > For a relative working directory, getPathToRoot .should work if given absolute paths PASSED
com.here.ort.downloader.VersionControlSystemTest > For a relative working directory, getPathToRoot .should work if given relative paths STARTED
com.here.ort.downloader.VersionControlSystemTest > For a relative working directory, getPathToRoot .should work if given relative paths PASSED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for Bitbucket.should not modify URLs without a path STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for Bitbucket.should not modify URLs without a path SKIPPED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for Bitbucket.should split tree URLs STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for Bitbucket.should split tree URLs PASSED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for Bitbucket.should split blob URLs STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for Bitbucket.should split blob URLs PASSED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitHub.should not modify URLs without a path STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitHub.should not modify URLs without a path PASSED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitHub.should not fail for a user called blob or a project called tree STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitHub.should not fail for a user called blob or a project called tree PASSED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitHub.should split tree URLs STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitHub.should split tree URLs PASSED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitHub.should split blob URLs STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitHub.should split blob URLs PASSED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitHub.should split extra path components STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitHub.should split extra path components PASSED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitLab.should not modify URLs without a path STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitLab.should not modify URLs without a path PASSED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitLab.should split tree URLs STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitLab.should split tree URLs PASSED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitLab.should split blob URLs STARTED
com.here.ort.downloader.VersionControlSystemTest > splitUrl for GitLab.should split blob URLs PASSED
> Task :downloader:funTest
com.here.ort.downloader.BabelTest > BabelTest.Babel packages should be correctly downloaded STARTED
> Task :analyzer:funTest
com.here.ort.analyzer.PipTest > setup py dependencies.should be resolved correctly for spdx-tools-python PASSED
com.here.ort.analyzer.PipTest > requirements txt dependencies.should be resolved correctly for example-python-flask STARTED
> Task :downloader:funTest
com.here.ort.downloader.BabelTest > BabelTest.Babel packages should be correctly downloaded PASSED
com.here.ort.downloader.DirectoryTest > DirectoryTest.Creates directories for Gradle submodules STARTED
com.here.ort.downloader.DirectoryTest > DirectoryTest.Creates directories for Gradle submodules PASSED
com.here.ort.downloader.DownloaderTest > DownloaderTest.Downloads and unpacks JAR source package STARTED
com.here.ort.downloader.DownloaderTest > DownloaderTest.Downloads and unpacks JAR source package PASSED
com.here.ort.downloader.DownloaderTest > DownloaderTest.Download of JAR source package fails when hash is incorrect STARTED
com.here.ort.downloader.DownloaderTest > DownloaderTest.Download of JAR source package fails when hash is incorrect PASSED
com.here.ort.downloader.DownloaderTest > DownloaderTest.Falls back to downloading source package when download from VCS fails STARTED
com.here.ort.downloader.DownloaderTest > DownloaderTest.Falls back to downloading source package when download from VCS fails PASSED
com.here.ort.downloader.DownloaderTest > DownloaderTest.Can download source artifact from SourceForce STARTED
com.here.ort.downloader.DownloaderTest > DownloaderTest.Can download source artifact from SourceForce PASSED
com.here.ort.downloader.BeanUtilsTest > BeanUtilsTest.BeanUtils SVN tag should be correctly downloaded STARTED
> Task :analyzer:funTest
com.here.ort.analyzer.PipTest > requirements txt dependencies.should be resolved correctly for example-python-flask PASSED
com.here.ort.analyzer.SbtTest > Dependencies of the.external 'directories' project should be detected correctly STARTED
com.here.ort.analyzer.SbtTest > Dependencies of the.external 'directories' project should be detected correctly FAILED
java.io.IOException: Cannot run program "sbt" (in directory "src/funTest/assets/projects/external/directories"): error=2, No such file or directory
Caused by:
java.io.IOException: error=2, No such file or directory
com.here.ort.analyzer.GradleTest > GradleTest.Root project dependencies are detected correctly STARTED
com.here.ort.analyzer.GradleTest > GradleTest.Root project dependencies are detected correctly PASSED
com.here.ort.analyzer.GradleTest > GradleTest.Project dependencies are detected correctly STARTED
com.here.ort.analyzer.GradleTest > GradleTest.Project dependencies are detected correctly PASSED
com.here.ort.analyzer.GradleTest > GradleTest.External dependencies are detected correctly STARTED
com.here.ort.analyzer.GradleTest > GradleTest.External dependencies are detected correctly PASSED
com.here.ort.analyzer.GradleTest > GradleTest.Unresolved dependencies are detected correctly STARTED
com.here.ort.analyzer.GradleTest > GradleTest.Unresolved dependencies are detected correctly PASSED
com.here.ort.analyzer.GradleTest > GradleTest.Fails nicely for Gradle version < 3.3 STARTED
> Task :downloader:funTest
com.here.ort.downloader.BeanUtilsTest > BeanUtilsTest.BeanUtils SVN tag should be correctly downloaded PASSED
com.here.ort.downloader.vcs.GitDownloadTest > GitDownloadTest.Git can download a given revision STARTED
com.here.ort.downloader.vcs.GitDownloadTest > GitDownloadTest.Git can download a given revision PASSED
com.here.ort.downloader.vcs.GitDownloadTest > GitDownloadTest.Git can download only a single path STARTED
com.here.ort.downloader.vcs.GitDownloadTest > GitDownloadTest.Git can download only a single path PASSED
com.here.ort.downloader.vcs.GitDownloadTest > GitDownloadTest.Git can download based on a version STARTED
com.here.ort.downloader.vcs.GitDownloadTest > GitDownloadTest.Git can download based on a version PASSED
com.here.ort.downloader.vcs.GitDownloadTest > GitDownloadTest.Git can download only a single path based on a version STARTED
com.here.ort.downloader.vcs.GitDownloadTest > GitDownloadTest.Git can download only a single path based on a version PASSED
com.here.ort.downloader.vcs.CvsDownloadTest > CvsDownloadTest.CVS can download a given revision STARTED
com.here.ort.downloader.vcs.CvsDownloadTest > CvsDownloadTest.CVS can download a given revision SKIPPED
com.here.ort.downloader.vcs.CvsDownloadTest > CvsDownloadTest.CVS can download only a single path STARTED
com.here.ort.downloader.vcs.CvsDownloadTest > CvsDownloadTest.CVS can download only a single path SKIPPED
com.here.ort.downloader.vcs.CvsDownloadTest > CvsDownloadTest.CVS can download based on a version STARTED
com.here.ort.downloader.vcs.CvsDownloadTest > CvsDownloadTest.CVS can download based on a version SKIPPED
com.here.ort.downloader.vcs.CvsDownloadTest > CvsDownloadTest.CVS can download only a single path based on a version STARTED
com.here.ort.downloader.vcs.CvsDownloadTest > CvsDownloadTest.CVS can download only a single path based on a version SKIPPED
com.here.ort.downloader.vcs.MercurialDownloadTest > MercurialDownloadTest.Mercurial can download a given revision STARTED
com.here.ort.downloader.vcs.MercurialDownloadTest > MercurialDownloadTest.Mercurial can download a given revision SKIPPED
com.here.ort.downloader.vcs.MercurialDownloadTest > MercurialDownloadTest.Mercurial can download only a single path STARTED
com.here.ort.downloader.vcs.MercurialDownloadTest > MercurialDownloadTest.Mercurial can download only a single path SKIPPED
com.here.ort.downloader.vcs.MercurialDownloadTest > MercurialDownloadTest.Mercurial can download based on a version STARTED
com.here.ort.downloader.vcs.MercurialDownloadTest > MercurialDownloadTest.Mercurial can download based on a version SKIPPED
com.here.ort.downloader.vcs.MercurialDownloadTest > MercurialDownloadTest.Mercurial can download only a single path based on a version STARTED
com.here.ort.downloader.vcs.MercurialDownloadTest > MercurialDownloadTest.Mercurial can download only a single path based on a version SKIPPED
com.here.ort.downloader.vcs.SubversionDownloadTest > SubversionDownloadTest.Subversion can download a given revision STARTED
> Task :analyzer:funTest
com.here.ort.analyzer.GradleTest > GradleTest.Fails nicely for Gradle version < 3.3 PASSED
com.here.ort.analyzer.GradleTest > GradleTest.Is compatible with Gradle >= 3.3 STARTED
com.here.ort.analyzer.GradleTest > GradleTest.Is compatible with Gradle >= 3.3 SKIPPED
com.here.ort.analyzer.MavenTest > MavenTest.jgnash parent dependencies are detected correctly STARTED
com.here.ort.analyzer.MavenTest > MavenTest.jgnash parent dependencies are detected correctly PASSED
com.here.ort.analyzer.MavenTest > MavenTest.jgnash-core dependencies are detected correctly STARTED
> Task :scanner:funTest
com.here.ort.scanner.ScanPathTest > ScanPathTest.ScanCode recognizes our own LICENSE PASSED
> Task :downloader:funTest
com.here.ort.downloader.vcs.SubversionDownloadTest > SubversionDownloadTest.Subversion can download a given revision FAILED
com.here.ort.downloader.DownloadException: Subversion failed to download from URL 'https://svn.code.sf.net/p/sendmessage/code'.
Caused by:
java.io.IOException: Running 'svn checkout https://svn.code.sf.net/p/sendmessage/code --depth empty .' in directory '/tmp/tmp2273889995595760709.tmp' failed with exit code 1:
svn: E170013: Unable to connect to a repository at URL 'https://svn.code.sf.net/p/sendmessage/code'
svn: E175002: Unexpected HTTP status 504 'Gateway Time-out' on '/p/sendmessage/code'
com.here.ort.downloader.vcs.SubversionDownloadTest > SubversionDownloadTest.Subversion can download only a single path STARTED
^C
Currently the pip package manager will be called with the first file it recognizes, and requirements.txt takes precedence over setup.py. That means the analyzer might not catch any of the metadata contained in setup.py. The implementation should be changed to always take setup.py into account.
Add support for the Bower package manager to the analyzer module.
Add support for the optionalDependencies scope to NPM. Make sure that the report contains a warning in case an optional dependency could not be installed.
Here is the output from analyzer
:
The following package managers are activated:
Gradle, Maven, SBT, NPM, PIP
Scanning project path:
/app
NPM projects found in:
oss-attribution-builder/package.json
17:46:30.994 [main] INFO com.here.ort.util.UtilsKt$log$1 - Running 'npm --version'...
17:46:30.999 [main] DEBUG com.here.ort.util.UtilsKt$log$1 - Keeping temporary files:
17:46:31.000 [main] DEBUG com.here.ort.util.UtilsKt$log$1 - /tmp/npm6011353000025587699.stdout
17:46:31.000 [main] DEBUG com.here.ort.util.UtilsKt$log$1 - /tmp/npm6832287346807849534.stderr
17:46:31.242 [main] INFO com.here.ort.util.UtilsKt$log$1 - Running 'yarn --version'...
17:46:31.242 [main] DEBUG com.here.ort.util.UtilsKt$log$1 - Keeping temporary files:
17:46:31.243 [main] DEBUG com.here.ort.util.UtilsKt$log$1 - /tmp/yarn2289262920421819612.stdout
17:46:31.243 [main] DEBUG com.here.ort.util.UtilsKt$log$1 - /tmp/yarn792803821624333627.stderr
Exception in thread "main" com.vdurmont.semver4j.SemverException: Invalid version (no patch version): 0.27
at com.vdurmont.semver4j.Semver.<init>(Semver.java:81)
at com.vdurmont.semver4j.Semver.<init>(Semver.java:20)
at com.vdurmont.semver4j.Requirement.isSatisfiedBy(Requirement.java:515)
at com.here.ort.util.ProcessCaptureKt.checkCommandVersion(ProcessCapture.kt:117)
at com.here.ort.util.ProcessCaptureKt.checkCommandVersion$default(ProcessCapture.kt:114)
at com.here.ort.analyzer.managers.NPM.prepareResolution(NPM.kt:116)
at com.here.ort.analyzer.PackageManager.resolveDependencies(PackageManager.kt:73)
at com.here.ort.analyzer.Main$main$5.accept(Main.kt:247)
at com.here.ort.analyzer.Main$main$5.accept(Main.kt:52)
at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
at com.here.ort.analyzer.Main.main(Main.kt:245)
If I run yarn --version
here is the output:
0.27
Add support for the Bundler dependency manager to the analyzer module.
Add support for Subversion repositories to the downloader module.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.