[representation] iS2S: Structure 2 Sequence Code Embeddings about ds4se HOT 6 OPEN

For Sam:

• Run Code2Vec and checkout the architecture
• Read the three papers and discuss the results
• Set-up ds4se environment

from ds4se.

Updated work plan:

• Check and analyze literature review in structural models indeep learning
• Set up DS4SE environment
• Test out the original code2vec
• Using code2vec as a library to generate embedding vectors from CodeSearchNet-Java
• Use RNNs to build an autoencoder to compress the dimension of embedding vectors
• Move our autoencoder to use code2vec encoder
• Later, to use transformers decoder
• Evaluate our model for clone detection
• Test our autoencoder for interpretability by actively trans-forming the data and evaluate the output
• If time permits, extend the work to traceability and codegeneration

from ds4se.

Meeting note 2021-03-18:

• Use the Keras autoencoder as a template to train against CodeSearchNet-Java.
• Use google/sentencepiece for input tokenization.
• Add an embedding layer after it.
• Use multiclass cross entropy for loss function (binary doesn't work).
• Don't deploy code2vec yet. It doesn't fit.

References:

• https://blog.keras.io/building-autoencoders-in-keras.html
• https://www.tensorflow.org/tutorials/text/nmt_with_attention

from ds4se.

Meeting note 2021-03-25:

• Ignore excessive length snippets
• Prepare for "design of the case studies"
• Refine previous sections

from ds4se.

Update 2021-04-01:

• Finally got the autoencoder training running.
• The network is shrunk in its dimension to deal with OOM situations.
• Not seeing significant loss reduction.
• Need help connecting to GPU inside Docker. (Someone has eaten the VRAM.)
• Need some help in "design of the case studies", if possible, a guideline in text version.
  

from ds4se.

Meeting note 2021-04-02:

Tasks:

1. Complete sampling
2. How to incorporate code2vec

Sampling (2 types):

1. focus on encoder, obtain the middle vectors (<- for now)
2. create random noise, how do they generate code

Case of studies:

The experiments we are going to run.

• Checking the clones (clone library provided)
• Test the encoder & decoder of GRU
• The other case: encoder is code2vec, decoder is GRU

from ds4se.