ghaffarian / progex Goto Github PK
View Code? Open in Web Editor NEWPROGEX (Program Graph Extractor); a cross platform tool for extracting graphical program representations from software source code
License: Apache License 2.0
PROGEX (Program Graph Extractor); a cross platform tool for extracting graphical program representations from software source code
License: Apache License 2.0
The code in the file JavaClassExtractor looks for path "\lib\src.zip" which is not available.
java.nio.file.NoSuchFileException: lib\src.zip
java.nio.file.NoSuchFileException: lib\src.zip
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
at java.base/sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:198)
at java.base/java.nio.file.Files.readAttributes(Files.java:1842)
at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1160)
at java.base/java.util.zip.ZipFile$CleanableResource.(ZipFile.java:719)
at java.base/java.util.zip.ZipFile.(ZipFile.java:239)
at java.base/java.util.zip.ZipFile.(ZipFile.java:169)
at java.base/java.util.zip.ZipFile.(ZipFile.java:140)
at ghaffarian.progex.java.JavaClassExtractor.extractJavaLangInfo(JavaClassExtractor.java:52)
at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:72)
at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:46)
at ghaffarian.progex.graphs.cfg.ICFGBuilder.buildForAll(ICFGBuilder.java:24)
at ghaffarian.progex.Execution.execute(Execution.java:219)
at ghaffarian.progex.Main.main(Main.java:36)
Process finished with exit code 0
Hi,
When I try to generate AST for large files, PROGEX give me the following error:
Exception in thread "main" java.lang.NullPointerException
at java.base/java.lang.StringBuilder.(StringBuilder.java:124)
I have increased the memory. But still getting the error.
Thanks.
The directory which is being scanned by progex does not have a reference to "src.zip" so I'm not sure how to chase the bug from a user perspective. Attached is a progex.log file showing the bug occurring. Aside from -info, it also happens with -cfg and -icfg. This code is great work! Thank you for posting it!
progex.log
Hello,
When I try to execute ICFG I have this problem:
java.io.FileNotFoundException: lib/src.zip (Aucun fichier ou dossier de ce type)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:225)
at java.util.zip.ZipFile.(ZipFile.java:155)
at java.util.zip.ZipFile.(ZipFile.java:126)
at ghaffarian.progex.java.JavaClassExtractor.extractJavaLangInfo(JavaClassExtractor.java:52)
at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:72)
at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:46)
at ghaffarian.progex.graphs.cfg.ICFGBuilder.buildForAll(ICFGBuilder.java:24)
at ghaffarian.progex.Execution.execute(Execution.java:226)
at ghaffarian.progex.Main.main(Main.java:36)
What does that mean ?
I use this command line:
java -jar "/home/progex-3.2.0/progex.jar" -outdir /home/progex-3.2.0/output/synapse_1.0 -icfg -lang java -format dot /home/progex-3.2.0/projets_source/synapse_1.0
Hi ghaffarian,
thx for your briliiant tool progex, but it seems not supoort parsing for lambda expression and method reference yet.
is there any plan for adding support for these new features?
When running PROGEX's ICFG analysis on its own source code, the following exception is thrown:
Exception in thread "main" java.lang.NullPointerException
at progex.java.JavaICFGBuilder$ICFGVisitor.classInstantiate(JavaICFGBuilder.java:1079)
at progex.java.JavaICFGBuilder$ICFGVisitor.findClassbyName(JavaICFGBuilder.java:993)
at progex.java.JavaICFGBuilder$ICFGVisitor.findVariableType(JavaICFGBuilder.java:969)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprPrimary(JavaICFGBuilder.java:627)
...
More precisely, the exception is raised while analysing line 32 of progex.java.JavaClass.java
, which is:
fields = new ArrayList<>();
The source of the issue is at line 1079 of JavaICFGGuilder.java
inside the classInstantiate(...)
method declaration, which is:
String[] genericParam = params.split(",");
At this point a NullPointerException
is raised, because params
is null
. This is most probably due to not handling the case where the generic type parameter is empty (e.g. new ArrayList<>();
).
Hi,
I wonder what other input languages are supported by this tool, apart from java, and how one may call them when using the application.
Regards
Hooman
Hello,
Does the "mvn install" works well to build the source code ?
Personaly, I have problem with th none-logger and graphs dependencies ...
ICFG crashes with a NullPointerException
given the following test case:
Test3.txt
Note that other analyses (such as CFG and PDG) run successfully on the same test case.
Here is the stack trace:
Exception in thread "main" java.lang.NullPointerException
at progex.java.JavaICFGBuilder$ICFGVisitor.findMethodbyName(JavaICFGBuilder.java:858)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprPrimary(JavaICFGBuilder.java:587)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprPrimary(JavaICFGBuilder.java:144)
at progex.java.parser.JavaParser$ExprPrimaryContext.accept(JavaParser.java:7443)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprDotID(JavaICFGBuilder.java:600)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprDotID(JavaICFGBuilder.java:144)
at progex.java.parser.JavaParser$ExprDotIDContext.accept(JavaParser.java:7617)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprMethodInvocation(JavaICFGBuilder.java:693)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitExprMethodInvocation(JavaICFGBuilder.java:144)
at progex.java.parser.JavaParser$ExprMethodInvocationContext.accept(JavaParser.java:7257)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitVariableInitializer(JavaBaseVisitor.java:265)
at progex.java.parser.JavaParser$VariableInitializerContext.accept(JavaParser.java:2948)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitVariableDeclarator(JavaBaseVisitor.java:251)
at progex.java.parser.JavaParser$VariableDeclaratorContext.accept(JavaParser.java:2829)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitVariableDeclarators(JavaBaseVisitor.java:244)
at progex.java.parser.JavaParser$VariableDeclaratorsContext.accept(JavaParser.java:2765)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitLocalVariableDeclaration(JavaICFGBuilder.java:468)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitLocalVariableDeclaration(JavaICFGBuilder.java:144)
at progex.java.parser.JavaParser$LocalVariableDeclarationContext.accept(JavaParser.java:5322)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitLocalVariableDeclarationStatement(JavaBaseVisitor.java:496)
at progex.java.parser.JavaParser$LocalVariableDeclarationStatementContext.accept(JavaParser.java:5267)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitBlockStatement(JavaBaseVisitor.java:489)
at progex.java.parser.JavaParser$BlockStatementContext.accept(JavaParser.java:5203)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitBlock(JavaICFGBuilder.java:251)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitBlock(JavaICFGBuilder.java:144)
at progex.java.parser.JavaParser$BlockContext.accept(JavaParser.java:5136)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitTryStatement(JavaICFGBuilder.java:413)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitTryStatement(JavaICFGBuilder.java:144)
at progex.java.parser.JavaParser$TryStatementContext.accept(JavaParser.java:5438)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitBlockStatement(JavaBaseVisitor.java:489)
at progex.java.parser.JavaParser$BlockStatementContext.accept(JavaParser.java:5203)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitBlock(JavaICFGBuilder.java:251)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitBlock(JavaICFGBuilder.java:144)
at progex.java.parser.JavaParser$BlockContext.accept(JavaParser.java:5136)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitMethodBody(JavaBaseVisitor.java:356)
at progex.java.parser.JavaParser$MethodBodyContext.accept(JavaParser.java:3924)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitMethodDeclaration(JavaICFGBuilder.java:505)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitMethodDeclaration(JavaICFGBuilder.java:144)
at progex.java.parser.JavaParser$MethodDeclarationContext.accept(JavaParser.java:1913)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitMemberDeclaration(JavaBaseVisitor.java:160)
at progex.java.parser.JavaParser$MemberDeclarationContext.accept(JavaParser.java:1797)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitClassBodyDeclaration(JavaBaseVisitor.java:153)
at progex.java.parser.JavaParser$ClassBodyDeclarationContext.accept(JavaParser.java:1681)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitClassBody(JavaBaseVisitor.java:139)
at progex.java.parser.JavaParser$ClassBodyContext.accept(JavaParser.java:1547)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitClassDeclaration(JavaICFGBuilder.java:231)
at progex.java.JavaICFGBuilder$ICFGVisitor.visitClassDeclaration(JavaICFGBuilder.java:144)
at progex.java.parser.JavaParser$ClassDeclarationContext.accept(JavaParser.java:811)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitTypeDeclaration(JavaBaseVisitor.java:41)
at progex.java.parser.JavaParser$TypeDeclarationContext.accept(JavaParser.java:452)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
at progex.java.parser.JavaBaseVisitor.visitCompilationUnit(JavaBaseVisitor.java:20)
at progex.java.parser.JavaParser$CompilationUnitContext.accept(JavaParser.java:214)
at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
at progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:82)
at progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:46)
at progex.graphs.cfg.ICFGBuilder.buildForAll(ICFGBuilder.java:24)
at progex.Execution.execute(Execution.java:211)
at progex.Main.main(Main.java:32)
Hi, I find another error when export json.
for instance,
{
"id": 6,
"line": 14,
"label": "if (base.substring(i, i + remove.length()).toLowerCase().equals(
remove.toLowerCase()))"
},
and
{
"id": 4,
"line": 10,
"label": "if (str.length() >= 2 &&
(str.charAt(0) == 'g' && str.charAt(1) != 'g' ||
str.charAt(str.length()-1) == 'g' &&
str.charAt(str.length()-2) != 'g'))"
},
All conditions in same IF STATEMENT
are successfully in same node, however, it is not recognized as one string. Therefore, when I try to read that json, it is failed with JSONDecodeError
. Can you fix it?
Before all, thank you for your excellent work ,it is very helpful for me.
But when I use command line for generating ICFG, some errors was reported as below:
java.io.FileNotFoundException: lib\src.zip
java.io.FileNotFoundException: lib\src.zip
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(Unknown Source)
at java.util.zip.ZipFile.<init>(Unknown Source)
at java.util.zip.ZipFile.<init>(Unknown Source)
at ghaffarian.progex.java.JavaClassExtractor.extractJavaLangInfo(JavaClassExtractor.java:52)
at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:72)
at ghaffarian.progex.java.JavaICFGBuilder.buildForAll(JavaICFGBuilder.java:46)
at ghaffarian.progex.graphs.cfg.ICFGBuilder.buildForAll(ICFGBuilder.java:24)
at ghaffarian.progex.Execution.execute(Execution.java:219)
at ghaffarian.progex.Main.main(Main.java:36)
So how to deal with it?
Thank you in advance for your help.
Lin
Hi,
Thank you for your work.
May I ask why the generated data dependence graph contains control flow edges?
Thanks for your time.
I'm trying to use progex, I'm having trouble following your usage guide, would you mind explaining how to setup everything in more details?
how do i get the progex.jar
for java -jar PROGEX.jar /path/to/program/src
While processing progex.java.JavaDDGBuilder.java
from PROGEX's own source code, the following exception is thrown during control-dependency analysis (JavaCDGBuilder
), after processing line 620: debug("METHOD DEF INFO NOT FOUND!");
of the JavaDDGBuilder.java
file:
Exception in thread "main" java.lang.IllegalArgumentException: no such vertex in graph: 0: ELSE
at org.jgrapht.graph.AbstractGraph.assertVertexExist(AbstractGraph.java:132)
at org.jgrapht.graph.AbstractBaseGraph.addEdge(AbstractBaseGraph.java:178)
at progex.java.JavaCDGBuilder$ControlDependencyVisitor.checkBuildFollowRegion(JavaCDGBuilder.java:670)
at progex.java.JavaCDGBuilder$ControlDependencyVisitor.addNodeEdge(JavaCDGBuilder.java:642)
at progex.java.JavaCDGBuilder$ControlDependencyVisitor.visitStatementExpression(JavaCDGBuilder.java:175)
at progex.java.JavaCDGBuilder$ControlDependencyVisitor.visitStatementExpression(JavaCDGBuilder.java:51)
...
The issue is most likely related to the else if (...)
statement at line 585; which may not be processed correctly by JavaCDGBuilder
.
The input Java file is attached below:
JavaDDGBuilder.java.txt
Hello,
Before all, congratulations for your source code and your presentation, it is very very clean and structured. I executed and generated the DOT files. That's excellent !!
I saw that it's possible to generate separately files AST / CFG / PDG... but how to make a merge of the 3? It's possible for me to complete your source code in this goal ? I would like to reproduce a single global graph merged like the Code property Graph defined in this document http://user.informatik.uni-goettingen.de/~krieck/docs/2014-ieeesp.pdf
Thank you in advance for your help
I export a json not a dot, and it generates a json file successfully.
However, when I try to read the json file using below codes, it shows an error JSONDecodeError: Expecting value
with open("Test1-CFG.json", 'r') as f:
data = json.load(f)
I try to fix it. When I remove each comma in the list of nodes and edges, I can read the json file correctly.
From
"nodes": [
{
"id": "n1",
"line": "4",
"code": "void main(String[] args)"
},
{
"id": "n10",
"line": "14",
"code": "System.out.println(\"End\")"
},
],
"edges": [
{
"id": "e1",
"source": "n1",
"target": "n2",
"label": ""
},
{
"id": "e2",
"source": "n2",
"target": "n3",
to
"nodes": [
{
"id": "n1",
"line": "4",
"code": "void main(String[] args)"
},
{
"id": "n10",
"line": "14",
"code": "System.out.println(\"End\")"
}
],
"edges": [
{
"id": "e1",
"source": "n1",
"target": "n2",
"label": ""
},
{
"id": "e2",
"source": "n2",
"target": "n3",
Can you fix this in the code?
I think it is maybe in nodes and edges
If it is a last node or a last edge, it does not have to write ,(comma)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.