Git Product home page Git Product logo

jarviz's Introduction

Jarviz

Jarviz Build Status

Interdependencies between software modules are very difficult problems to solve. When your organization grows, these interdependencies multiply, potentially leading to low-cohesion and high-coupling among software modules. This is where Jarviz can help to understand these intricate dependencies down to the field level.

Jarviz is a dependency analysis tool for Java applications. Since non-private methods in Java classes can be accessed by other classes, method calls can be intertwined creating manifold couplings among each other. Jarviz deeply analyzes Java bytecode to reveal these couplings in a user-friendly format. It scans binary artifacts using a custom classloader and generates a complete graph of dependency couplings between methods across multiple artifacts. Jarviz consists of three modules: a Java library to scan dependencies in binary artifacts, a Node.js module to generate a visual graph and a command-line-interface to run it easility in a shell.

ℹ️ Learn more about Jarviz on the Expedia Group Technology blog

Jarviz consists of 3 components:

Jarviz Library (jarviz-lib)

This Java library scans the Java bytecode of binary artifacts using a custom classloader and generates the dependency coupling data as a JSON Lines (.jsonl) file. Currently only JAR and WAR artifact formats are supported. To find the dependency couplings, Jarviz analyzes the opcodes using ASM bytecode analysis framework.

Jarviz Graph Tool (jarviz-graph)

Jarviz Graph is a Node application to read the dependency coupling data and generate the dependency graph as an HTML file. The dependency coupling data, provided as a JSON Lines (.jsonl) input file, is generated from the Jarviz Java library.

Jarviz CLI Tool (jarviz-cli)

Jarviz CLI is a command-line tool designed for *nix systems to perform dependency analysis for Java applications. Internally it uses both the Jarviz Java library and Jarviz graph tool to provide a useful command-line interface to the user.

Quick Start

  • Prerequisite: Verify that java, maven, node and npm are installed in the system.
    • Java and Maven are required to download Java libraries.
    • Node and NPM are required to download Node modules.
  • Checkout the project from GitHub and change the directory to jarviz-cli module.
  • Run ./jarviz graph -f samples/filter.json -a samples/artifacts.json
  • Open the generated HTML file in the browser.

Samples

Sample Dependency Graph

Sample Coupling Data

{
  "appSetName": "FooPortfolio",
  "applicationName": "MyApp",
  "artifactFileName": "foo-product-1.2.1.jar",
  "artifactId": "foo-product",
  "artifactGroup": "com.foo.bar",
  "artifactVersion": "1.2.1",
  "sourceClass": "foo.bar.MyClass",
  "sourceMethod": "doFirstTask",
  "targetClass": "foo.bar.YourClass",
  "targetMethod": "getProductId"
}
...

Dependency Couplings

What is a coupling?

A coupling is a dependency between two methods. Let's say method hello of class com.xyz.Alpha calls the method world of class com.xyz.Beta. Then there is a dependency coupling between those two methods and Jarviz represents that coupling this way:

Source Class Source Method Target Class Target Method
com.xyz.Alpha hello com.xyz.Beta world

Why field level dependencies?

There are many tools capable of generating dependency graphs either down to the class or library level. This is useful information to have when you are supporting an application. It is also very important to know the dependencies down to the field level access. Jarviz provides a way to generate dependency coupling data down to field level (assuming that the internal state of the fields are accessed via method calls). Jarviz will analyze deeply inside the methods, including lambda functions to extract the dependency data.

Sample Couplings

In the Java source code shown below, the class com.mycompany.MySource calls methods from two classes namely com.xyz.foo.Alpha and com.xyz.bar.Beta.

package com.xyz.foo;
public class Alpha {

    private final int val;

    public Alpha(final int val) {
        this.val = val;
    }

    public int getVal() {
        return val;
    }
}
package com.xyz.bar;
public class Beta {

    public int getBetaIntVal(final String value) {
        return Integer.parseInt(value);
    }

    public long getBetaLongVal(final String value) {
        return Long.parseLong(value);
    }

    public Alpha getBetaAlphaVal(final String value) {
        return new Alpha(Integer.parseInt(value));
    }
}
package com.mycompany;
public class MySource {

    private static int MY_STATIC_VAR = new Alpha(1).getVal();

    private int myInstanceAlphaValue = new Alpha(2).getVal();

    public int callBeta1() {
        return new Beta().getBetaIntVal("10");
    }

    public int callBeta2() {
        return new Beta().getBetaIntVal("20");
    }

    public int callAlpha1() {
        return new Alpha(10).getVal();
    }

    public int callAlphaBeta() {
        return new Beta().getBetaAlphaVal("30").getVal();
    }

    public long callLambda(final List<String> values) {
        return values.stream()
                     .filter(a -> !a.isEmpty())
                     .map(a -> new Beta().getBetaLongVal(a))
                     .count();
    }
}

If com.mycompany.MySource class is analyzed with Jarviz for the usage of any classes in the com.xyz package, it will find the following dependency couplings:

Source Class Source Method Target Class Target Method
com.mycompany.MySource <clinit> com.xyz.foo.Alpha <init>
com.mycompany.MySource <clinit> com.xyz.foo.Alpha getVal
com.mycompany.MySource <init> com.xyz.foo.Alpha <init>
com.mycompany.MySource <init> com.xyz.foo.Alpha getVal
com.mycompany.MySource callBeta1 com.xyz.bar.Beta <init>
com.mycompany.MySource callBeta1 com.xyz.bar.Beta getBetaIntVal
com.mycompany.MySource callBeta2 com.xyz.bar.Beta <init>
com.mycompany.MySource callBeta2 com.xyz.bar.Beta getBetaIntVal
com.mycompany.MySource callAlpha1 com.xyz.foo.Alpha <init>
com.mycompany.MySource callAlpha1 com.xyz.foo.Alpha getVal
com.mycompany.MySource callAlphaBeta com.xyz.foo.Alpha getVal
com.mycompany.MySource callAlphaBeta com.xyz.bar.Beta <init>
com.mycompany.MySource callAlphaBeta com.xyz.bar.Beta getBetaAlphaVal
com.mycompany.MySource lambda$callLambda$1 com.xyz.bar.Beta <init>
com.mycompany.MySource lambda$callLambda$1 com.xyz.bar.Beta getBetaLongVal

See the full sample file: sample_jarviz_result.jsonl

Development

Initial Setup

Clone the Jarviz project locally from the GitHub repo, and verify it builds properly.

Jarviz Library

$ mvn clean install

Jarviz Graph Tool

$ npm install
$ npm run build:example

Jarviz CLI Tool

$ ./jarviz

How to Contribute

If you are enthusiastic about contributing to Jarviz development, send a pull request. Please follow the guidance in CONTRIBUTING.md.

Contact

This project is an open source software by Expedia Group and maintained by ExpediaGroup/teams/jarviz-committers. For support discussions please join Jarviz Google group.

Legal

This project is available under the Apache 2.0 License.

Copyright 2019 Expedia, Inc.

jarviz's People

Contributors

dependabot[bot] avatar hadisfr avatar lasanthak avatar nath-abhishek avatar rcbrown avatar renovate[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

jarviz's Issues

To generate the HTML graph

Library Version
What version are you using?

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior. Please provide:

  • Jarviz version
  • Java version
  • Node/NPM version

Expected behavior
A clear and concise description of what you expected to happen.

While running the below command: the following error raises:

npx "jarviz-graph" -i ../jarviz-cli/jarviz-results-20230403-205312/*.jsonl -o /target_dir

Could not find any files match *.jsonl at path: /Users/user/Downloads/jarviz-jarviz-parent-0.1.7/jarviz-cli/jarviz-results-20230403-205312/jarviz-dependency-data.jsonl

JsonParseException: Unrecognized token 'PK'

Library Version
master branch

Describe the bug
checkout, installed the lib, switching to cli dir and executing:

./jarviz analyze -a ~/dev/mylib.jar -f samples/filter.json

Resulted in

Java: Running /home/daniel/.jarviz/jarviz_0.1.5.jar
Analyze failed: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'PK': was expecting ('true', 'false' or 'null')
 at [Source: (StringReader); line: 1, column: 5]
	com.vrbo.jarviz.util.JsonUtils.fromJsonString(JsonUtils.java:78)
	com.vrbo.jarviz.AnalyzeCommand.main(AnalyzeCommand.java:145)

To Reproduce
Steps to reproduce the behavior. Please provide:

  • Jarviz version
    see above
  • Java version
    OpenJDK Runtime Environment Temurin-17.0.3+7 (build 17.0.3+7)
  • Node/NPM version
    not relevant

Expected behavior
generates some output file

Running jarviz cli under windows does not work due to command invocation

Describe the bug
When I try to execute the quickstart guide under windows (via the mingw git bash) the .\jarviz produces the following error:

java.io.IOException: Cannot run program "mvn": CreateProcess error=2, The system cannot find the file specified

To Reproduce
Just follow the quickstart guide under windows.

  • Jarviz version: 0.1.7
  • Java version: 1.8.0

Expected behavior
It should just run and produce the HTML file.

Fix
This error is due to the window enviroment.
I can fix the issue by

  • replacing "mvn ... in com.vrbo.jarviz.service.MavenArtifactDiscoveryService.java with "mvn.cmd ... and then
  • recompiling and installing locally.

This appears to be a known issue with maven:

Maybe you could use maven-embedder or maven-invoker. It would be nice if there were at least a code switch that selects the correct command depending on the operating system.

Handle applications with no dependencies or unversioned dependencies

Hello, I am working on a dependency analysis project on our software repositories and this involves scraping pom.xmls from each of our application repos.

However some repositories have no dependencies, and other projects have dependencies with no version.

I worked around dependencies with no version declared by using version LATEST even though this is depreciated in Maven 3. However I cannot get Jarviz to graph my large appSet because some of the applications use no external dependencies.

Ideally I would see these applications as a single node on the resulting graph just with no dependencies branching from them. However if I could simply exclude them that would work too as it's unfeasible to manually edit this artifacts.json given it's size.

I feel this would be a good use case for adding exclusions in a filter file. Is this possible using the existing filter files?

`npm install` error.

Library Version

Describe the bug
Within folder jarviz-jarviz-parent-0.1.6/jarviz-graph, run command 'npm install`, failed with error:

ERROR in ./node_modules/3d-force-graph/dist/3d-force-graph.mjs 142:16-28
Can't import the named export 'AmbientLight' from non EcmaScript module (only default export is available)
@ ./node_modules/react-force-graph/dist/react-force-graph.mjs
@ ./lib/client/graph.js
@ ./lib/client/index.js

ERROR in ./node_modules/three-render-objects/dist/three-render-objects.mjs 145:12-20
Can't import the named export 'BackSide' from non EcmaScript module (only default export is available)
@ ./node_modules/3d-force-graph/dist/3d-force-graph.mjs
@ ./node_modules/react-force-graph/dist/react-force-graph.mjs
@ ./lib/client/graph.js
@ ./lib/client/index.js

ERROR in ./node_modules/three-forcegraph/dist/three-forcegraph.mjs 411:8-12
......

ERROR in ./node_modules/aframe-extras/src/loaders/fbx-model.js
Module not found: Error: Can't resolve 'three/addons/loaders/FBXLoader.js' in '/webapp/seaward/jarviz-jarviz-parent-0.1.6/jarviz-graph/node_modules/aframe-extras/src/loaders'
@ ./node_modules/aframe-extras/src/loaders/fbx-model.js 1:0-62 2:18-27
@ ./node_modules/aframe-extras/src/loaders/index.js
@ ./node_modules/aframe-extras/index.js
@ ./node_modules/3d-force-graph-vr/dist/3d-force-graph-vr.mjs
@ ./node_modules/react-force-graph/dist/react-force-graph.mjs
@ ./lib/client/graph.js
@ ./lib/client/index.js
Child html-webpack-plugin for "jarviz-graph.html":
1 asset
Entrypoint undefined = jarviz-graph.html
[0] ./node_modules/html-webpack-plugin/lib/loader.js!./lib/client/index.html 1.03 KiB {0} [built]
[2] (webpack)/buildin/global.js 472 bytes {0} [built]
[3] (webpack)/buildin/module.js 497 bytes {0} [built]
+ 1 hidden module
npm ERR! code 2
npm ERR! path /webapp/seaward/jarviz-jarviz-parent-0.1.6/jarviz-graph
npm ERR! command failed
npm ERR! command sh -c -- npm run clean && npm run build

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2024-03-28T09_22_32_055Z-debug-0.log

To Reproduce
Jarviz 0.1.7 & 0.1.6
openjdk version "11.0.20" 2023-07-18 LTS
Apache Maven 3.6.3 (Red Hat 3.6.3-15)
nodejs v16.20.2
npm 8.19.4

Expected behavior
npm install should succeeded.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • Update dependency commons-cli:commons-cli to v1.7.0
  • Update dependency org.apache.maven.plugins:maven-checkstyle-plugin to v3.3.1
  • Update dependency org.apache.maven.plugins:maven-compiler-plugin to v3.13.0
  • Update dependency org.apache.maven.plugins:maven-jar-plugin to v3.4.1
  • Update dependency org.apache.maven.plugins:maven-javadoc-plugin to v3.6.3
  • Update dependency org.apache.maven.plugins:maven-jxr-plugin to v3.3.2
  • Update dependency org.apache.maven.plugins:maven-resources-plugin to v3.3.1
  • Update dependency org.apache.maven.plugins:maven-source-plugin to v3.3.1
  • Update dependency org.assertj:assertj-core to v3.25.3
  • Update dependency org.ow2.asm:asm to v9.7
  • Update glassfish.hk2.version to v3.1.0 (org.glassfish.hk2:hk2-locator, org.glassfish.hk2:hk2-api)
  • Update actions/cache action to v4
  • Update actions/checkout action to v4
  • Update actions/setup-java action to v4
  • Update actions/setup-node action to v4
  • Update dependency babel-loader to v9
  • Update dependency chalk to v5
  • Update dependency com.puppycrawl.tools:checkstyle to v10
  • Update dependency commander to v12
  • Update dependency css-loader to v7
  • Update dependency cssnano to v7
  • Update dependency d3-scale to v4
  • Update dependency d3-scale-chromatic to v3
  • Update dependency eslint to v9
  • Update dependency eslint-loader to v4
  • Update dependency eslint-plugin-prettier to v5
  • Update dependency file-loader to v6
  • Update dependency glob to v10
  • Update dependency html-webpack-plugin to v5
  • Update dependency mini-css-extract-plugin to v2
  • Update dependency mkdirp to v3
  • Update dependency org.apache.maven.plugins:maven-deploy-plugin to v3
  • Update dependency org.apache.maven.plugins:maven-failsafe-plugin to v3
  • Update dependency org.apache.maven.plugins:maven-gpg-plugin to v3
  • Update dependency org.apache.maven.plugins:maven-install-plugin to v3
  • Update dependency org.apache.maven.plugins:maven-release-plugin to v3
  • Update dependency org.apache.maven.plugins:maven-surefire-plugin to v3
  • Update dependency prettier to v3
  • Update dependency pretty-quick to v4
  • Update dependency rimraf to v5
  • Update dependency style-loader to v4
  • Update dependency url-loader to v4
  • Update dependency webpack to v5
  • Update dependency webpack-cli to v5
  • Update dependency webpack-dev-server to v5
  • Update dependency webpack-merge to v5
  • Update github/codeql-action action to v3
  • Update react monorepo to v18 (major) (react, react-dom)
  • Update slf4j monorepo to v2 (major) (org.slf4j:slf4j-api, org.slf4j:log4j-over-slf4j)
  • 🔐 Create all rate-limited PRs at once 🔐

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/workflows/build.yml
  • actions/checkout v2
  • actions/setup-java v1
  • actions/cache v2
  • actions/checkout v2
  • actions/setup-node v2
  • actions/cache v2
.github/workflows/codeql-analysis.yml
  • actions/checkout v2
  • github/codeql-action v1
  • github/codeql-action v1
  • github/codeql-action v1
.github/workflows/release.yml
  • actions/checkout v2
  • actions/setup-java v1
  • actions/cache v2
  • actions/checkout v2
  • actions/setup-node v2
  • actions/cache v2
maven
jarviz-lib/pom.xml
  • com.fasterxml.jackson:jackson-bom 2.9.8
  • ch.qos.logback:logback-classic 1.2.11
  • commons-cli:commons-cli 1.5.0
  • org.ow2.asm:asm 9.1
  • com.google.code.findbugs:jsr305 3.0.2
  • com.google.errorprone:error_prone_annotations 2.3.2
  • com.google.guava:guava 30.0-jre
  • javax.inject:javax.inject 1
  • org.glassfish.hk2:hk2-api 3.0.3
  • org.glassfish.hk2:hk2-locator 3.0.3
  • org.immutables:value 2.9.0
  • org.slf4j:log4j-over-slf4j 1.7.36
  • org.slf4j:slf4j-api 1.7.36
  • junit:junit 4.13.2
  • org.assertj:assertj-core 3.9.1
  • com.puppycrawl.tools:checkstyle 8.29
pom.xml
  • org.apache.maven.plugins:maven-clean-plugin 3.1.0
  • org.apache.maven.plugins:maven-project-info-reports-plugin 3.0.0
  • org.apache.maven.plugins:maven-resources-plugin 3.2.0
  • org.apache.maven.plugins:maven-compiler-plugin 3.10.1
  • org.apache.maven.plugins:maven-surefire-plugin 2.22.2
  • org.apache.maven.plugins:maven-failsafe-plugin 2.22.2
  • org.apache.maven.plugins:maven-jar-plugin 3.2.2
  • org.apache.maven.plugins:maven-source-plugin 3.2.1
  • org.apache.maven.plugins:maven-javadoc-plugin 3.4.0
  • org.apache.maven.plugins:maven-shade-plugin 3.2.1
  • org.apache.maven.plugins:maven-install-plugin 2.5.2
  • org.apache.maven.plugins:maven-release-plugin 2.5.3
  • org.apache.maven.plugins:maven-checkstyle-plugin 3.1.2
  • org.apache.maven.plugins:maven-jxr-plugin 3.2.0
  • org.apache.maven.plugins:maven-dependency-plugin 3.1.1
  • org.apache.maven.plugins:maven-deploy-plugin 2.8.2
  • org.sonatype.plugins:nexus-staging-maven-plugin 1.6.13
  • org.apache.maven.plugins:maven-gpg-plugin 1.6
npm
jarviz-graph/package.json
  • @babel/core ^7.5.5
  • @babel/plugin-proposal-class-properties ^7.5.5
  • @babel/plugin-proposal-object-rest-spread ^7.5.5
  • @babel/plugin-transform-destructuring ^7.5.0
  • @babel/preset-env ^7.5.5
  • @babel/preset-react ^7.0.0
  • alphabetize-object-keys ^3.0.0
  • babel-eslint ^10.0.3
  • babel-loader ^8.0.6
  • chalk ^2.4.2
  • commander ^3.0.0
  • copy-dir ^1.2.0
  • css-loader ^3.2.0
  • cssnano ^4.1.10
  • d3-scale ^3.0.0
  • d3-scale-chromatic ^1.3.3
  • eslint ^6.2.2
  • eslint-loader ^3.0.0
  • eslint-plugin-prettier ^3.1.0
  • eslint-plugin-react ^7.14.3
  • file-loader ^4.2.0
  • glob ^7.1.4
  • html-webpack-plugin ^3.2.0
  • js-beautify ^1.10.0
  • memory-fs ^0.4.1
  • mini-css-extract-plugin ^0.8.0
  • mkdirp ^0.5.1
  • prettier ^1.18.2
  • pretty-quick ^1.11.1
  • react ^16.9.0
  • react-dom ^16.9.0
  • react-force-graph ^1.24.1
  • rimraf ^2.6.3
  • style-loader ^1.0.0
  • system-bell-webpack-plugin ^1.0.0
  • url-loader ^2.1.0
  • webpack ^4.39.2
  • webpack-cli ^3.3.7
  • webpack-dev-server ^3.8.0
  • webpack-merge ^4.2.1
  • html-webpack-inline-source-plugin 0.0.10
  • node >=10.0.0
  • npm >=6.0.0

  • Check this box to trigger a request for Renovate to run again on this repository

No html file

Just downloaded this.
Followed the readme instructions (ie - ./jarviz graph -f samples/filter.json -a samples/artifacts.json).

There is no HTML file generated.
jarviz exits with a 0, no errors are shown.

Please advise.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.