This repository includes the code and experimental data in our paper entitled "A Novel Neural Source Code Representation based on Abstract Syntax Tree" that will be presented at ICSE'2019. It can be used to encode code fragments into supervised vectors for various source code related tasks. We have applied our neural source code representation to two common tasks: source code classification and code clone detection. It is also expected to be helpful in more tasks.

• python 3.6
  
• pandas 0.20.3
  
• gensim 3.5.0
  
• scikit-learn 0.19.1
  
• pytorch 0.3.1
  
• pycparser 2.18
  
• javalang 0.11.0
  
• RAM 16GB or more
• GPU with CUDA support is also needed
• BATCH_SIZE should be configured based on the GPU memory size

Install all the dependent packages via pip:

$ pip install pandas==0.20.3 gensim==3.5.0 scikit-learn==0.19.1 pycparser==2.18 javalang==0.11.0

Install pytorch 0.3.1 with cuda:

$ pip install https://download.pytorch.org/whl/cu80/torch-0.3.1-cp36-cp36m-linux_x86_64.whl

The version of pytorch 0.3.1 is mandatory. Higher versions may lead to errors for our exisiting code, and we will improve our code in the future. See https://pytorch.org/get-started/previous-versions/ for more options according to your CUDA version.

1. cd astnn
2. run python pipeline.py to generate preprocessed data.
3. run python train.py for training and evaluation

1. cd clone
2. run python pipeline.py --lang c or python pipeline.py --lang java to generate preprocessed data for the two datasets.
3. run python train.py --lang c to train on OJClone, python train.py --lang java on BigCLoneBench respectively.