myroslavarm / bachelor_thesis Goto Github PK

1. Benchmarking in the implementation chapter
2. Examples of code (how the models / sorting strategies were implemented)

To shortly wrap up everything that was explained.

Find 10 cases and test them on all three sorting strategies and compare the accuracy of results.

• evaluation #4
• write people to confirtm about the Dr title
• add more code snippets #3
• finish the Conclusion chapter
• either write or delete Dedication
• correct links on the title page
• taking everything through Word and fix typos if any
• rewrite what's needed based on feedback
• read the Thesis book & see if there's anything i can use
• change all 'we' to 'I'
• include Julien in the acknowledgements
• consistency in n-gram/N-gram
• consistency in state-of-the-art
• replace all you with one
• mention somewhere why we didn't implement 3 or 4-grams

qualitative

• example 1 : collection vs col -- does it give a better result?
• example 2 : what is the token that it takes here?
• example 3 : fix formulation without the 'only'
• example 4 : fix token type to tokens
• example 6 : check 'word before' if we are at the beginning
• example 7 : check why it suggests so many wrong things (maybe)

quantitative

• check manual results & do a write-up on why bigram performs worse
• i.e. does it deal worse with parantheses, brackets, etc.

general

• in the conclusion chapter in future work describe what i would do with the qualitative or quantitative evaluation if i had more time

so there are several ideas but the main suspicions are:

• when training, some problems involving punctuation and delimiters, such as ; ] ) etc.

• when training, essentially tokenisation problems in the ngram model, because in a lot of cases something like parantheses might be missed and so the bigram dependency will be trained differently (think something = 1 ifTrue: the completion itself is wrong because it's for an integer, but also when we train the ngram sequence is also number-based, and then during the actual evaluation when we are comparing the actual code that was typed in and calculating the accuracy it will be way off because our typing/completion is different from the end result)

• another thing is with error nodes, because we might be getting the wrong ast, and hence getting wrong placeholder stuff for the actual code at the end, so again training/actual code discrepancy