wm-semeru / ds4se Goto Github PK

There are pathing errors in the simulate mongodb method in 1.0_exp.i. These need to be fixed in order to use this as a basis for testing exploratory.

Running list of dependencies that were not installed using requirements.txt
0.0
tokenizers

1.1
fastprogress - in ds4se.mgmnt.prep.bpe (?)

6.2
Lizard
Tree_sitter

3.3
Gensim
Prg

Have the repository properly test the project with each push/pull request

Testing team needs to flag all of the tests created in the desc section of the nbs and then add those flags to the settings.ini file in the proper format. Before this issue can be submitted as complete the tests must be invokable through the ds4se format of running tests.

Formalizing existing and creating new test cases for methods in desc.stats file

Create a program in python that when run will run nbdev tests on our select nbdev files and will print out to the User a stub summary of the process.

Create Branches For Project 2:

SE_Proj2: Main branch of project too, interacted with by everyone
SE_Proj2_Testing: Branch used by Yangchen and Alex
SE_Proj2_Refactor: Branch used by Robert and Will
SE_Proj2_Facade: BRanch used by Charles and Daniel

These separte branches are meant to prevent unnecessary collisions in merging and pushing allowing domains to merge with each other when needed but also keep certain changes isolated until the group can confirm them.

Doc2vec and Word2vec are available

Please use and integrate CodeBERT to the baseline in a separate component:
https://github.com/microsoft/CodeBERT

A minor return error in Facade file that needs correcting. Test cases should be reorganized.

Test shannon dit and data frame methods

Create Assertion Tests for the Facade created by the Facade team. These proto asserts are barebones assertions that will need to be updated as the Facade changes.

Certain files have become corrupted resulting in problems throughout the repo. Fix these files and push to main.

We use DIT as our core library for entropy metrics.
https://dit.readthedocs.io/en/latest/
https://github.com/dit/dit

Please focus on Copy Mutual Information: https://dit.readthedocs.io/en/latest/measures/divergences/copy_mutual_information.html

architecture, installation, deployment, and usage

Import Comet Library Functionality to process the mutual information metric
(this is a test issue)

Create a colab notebook presenting a tutorial of how to use DS4SE to analyze a system probabilistically. Use the refactored information theory and statistical components.

Use refactored code to implement functions in facade to return actual calculated value

3.0_mining.ir.model
3.0_mining.unsupervised.traceability.ida
3.1_mining.ir.i
3.1_mining.unsupervised.traceability.eda - stuff in here, maybe need to export?

• TraceLinkValue function for word2vec and doc2vec functionality
• NumDoc
• VocabSize
• AverageToken
• Vocab
• VocabShared
• SharedVocabSize
• MutualInformation
• CrossEntropy

Please use and integrate CodeBERT to the baseline in a separate component:
https://github.com/microsoft/CodeBERT

Make test cases for methods exported in 1.1_exp.info

Notebook should present a tutorial of how to use the traceability facades.
Maybe use Libest

Phase II is aiming at filling the gaps to have a fully functional T-Miner (beta) version. To have a stable version, we need to adopt new SE methodologies that work specifically for data science and machine learning. Such methodologies involve other frameworks such as DVC, nbdev, and TFX. This phase is composed of the following activities:

T-Miner

• T-Miner Interoperability and Deployment. We must guarantee that T-miner is communicating with the DS4SE library, Jenkins, and a SecureReqNet deployed version.
• T-Miner Navigation. We must guarantee that the proposed navigation is functional and stable. Important use cases: information recovery (traceability) and information analysis (entropy). The tool should retrieve, create, update, and delete traceability results.
• Causal Inference View. We require to implement a causal inference view for T-miner. CI should be consumed from DS4SE. However, no modules in DS4SE have been fully developed. This is a whole bach-end solution to update our previous COMET solution.

DS4SE

• Data repository integration. We have been employing DVC for data versioning. However, our projects are not fully integrated. We require to centralize in a single remote all the SE-Related data. Our current architecture allows one remote per git-project, which generates data redundancies.
• Data Science/ML Continues Integration. We want to adopt Continuous Machine Learning or CML. The main goal of CML is to keep all our experiments and models under control. Similar to TFX, DVC has its own pipeline solution here.
• Migrating Unsupervised Traceability Models into CML-DVC. All our unsupervised models will be shaped as an ML pipeline for further enhancement and development.

Create tag for facade tests. Wait to implemented into settings.ini until confirming the process by which tests are used and generated.

Edit the settings.ini to incorporate the flaggin system used by testing.