We have a small collection of features within our training data that are strings, see below for the features.
How will we handle these features? For now, I disabled them, see here.

These features might be interesting for some refactoring types, especially the "Rename" refactorings types, thus it would be good if we could use them.

Most ways to handle strings as features for machine-learning are not applicable in our case, because:

1. One-Hot-Encoding/ Categorical Variables: descriptors are difficult to map into categories that still contain the properties we are interested in
2. Convert to number: naively converting a name to a byte array, will probably not yield to good results, as the entire data is very messy and the length are very different.
3. Extract properties: we could extract properties from the names that we deem potentially relevant, e.g. number of characters, number of special characters, etc.

Features

Method Level:

• fullMethodName
• shortMethodName

Field Level:

• fieldName

Variable Level:

• variableName

I have the impression that the early stop mechanism is not really working. We need to double check as it would save lots of processing time.

In the new version of the code, we should set stratify=y in the train_test_split_call

We have many outdated, probably unused files in the ml src folder, e.g. results_parsing/. We should remove them, in order to keep the project more maintainable and easier to understand.

We are currently, using tensorflow-gpu which is no longer the suggested version of tensorflow to use for gpu support.
This creates issues with the Travis CI pipeline and might create further issues in the future.

Right now, we only collect performance metrics (e.g., precision, recall, accuracy).

We need to collect some examples for future qualitative analysis. In other words, for each of the models we build, a collection of [method_id, expected_prediction, model_prediction].

This way we can later look at code examples of false positives, false negatives, etc.

I suppose all these changes will be:

• _single_run_model should receive X_train, X_test, y_train, and y_test (which will be implemented in refactoring-ai/predicting-refactoring-ml#36), we should pass X_test_id.
• _single_run_model then returns, besides the performance metrics, a dataframe as suggested above.
• This should be printed to the logs in a way that becomes easy to parse later. Suggestion: "PRED,refactoring,model,id_element,expected_prediction,predicted_value". "PRED" is just a prefix that is easy to be find by grep.

I'm using method as an example, but it can also be a class or a variable or a field, i.e., everything we predict.

The Readme is not up-to-date.

Update the Readme to the latest developments.

Linked to #10

We should write more assertions to be 100% sure that all the transformations happened as expected.

We have already some assertions to make sure that, e.g., after balancing, the dataset has 50%-50% in both classes. What other assertions should we add?

Suggestion:

• Number of features at the end is the one expected
• We have no duplicated data (to avoid some SQL query returning the same IDs more than once)
• ... ?

pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT processmetrics.authorOwnership, processmetrics.bugFixCount, processmetrics.qtyMajorAuthors, processmetrics.qtyMinorAuthors, processmetrics.qtyOfAuthors, processmetrics.qtyOfCommits, processmetrics.refactoringsInvolved FROM stablecommit INNER JOIN commitmetadata ON stablecommit.commitmetadata_id = commitmetadata.id INNER JOIN processmetrics ON stablecommit.processmetrics_id = processmetrics.id WHERE stablecommit.level = 0 AND stablecommit.project_id in (select id from project where datasetName = "github") order by commitmetadata.commitDate': 1146 (42S02): Table 'refactoringdb.commitmetadata' doesn't exist

I think this is caused by case sensitivity between windows and Linux.
https://stackoverflow.com/questions/6134006/are-table-names-in-mysql-case-sensitive?rq=1
On Windows tables are case insensitive as seen by the above post. However in Linux they are not causing the python scripts to fail since the tables are camel cased while in the python program they are all lower case

Enable us to configure different feature reduction strategies (e.g., variance-based, etc). Enable also the possibility of "no feature reduction".

See how we implemented the different balancing strategies, and follow the same code pattern.

I'm curious to see what happens if we don't balance the data during training. Do we improve the number of FPs? (I am seeing this trend in other works I'm doing...)

We want to investigate the results of the classifier. Therefore, we want to have a look at the original files.

For each result of a classifier store:

• CommitID
• Affected File/Class/Method/Variable
• Repository ID

Little news and refactory repo has been archived, have you moved to something else ? Is there a fork ?