zacbi / cs224n-2019-solutions Goto Github PK

Nice work!
The version of implementation can reach 22+ BLUE score. However, my implementation have only 0.16+ BLUE score on test dataset. Comparing with your work, I found changing the concatenation torch.cat((Y_t, o_pre), dim=1) to torch.cat((o_pre, Y_t), dim=1) can only reach 0.16+ BLUE score.

B.W.T. chaging the concatenating order between dec_hidden and a_t in step function also resulting in bad BLUE score in test dataset.

Would you like share your ideas why concatenating Y_t and o_pre in such way?

Thank you!

2(b) : A sentence containing n words will be parsed in how many steps ?

I think it should need 2*n steps. Because steps include SHIFT.

Hey, I was googling github 224n to compare my solutions to others and stumbled on Your repo. While browsing I noticed that you have an error that is not catched by sanity code in assigment1 - Question 1.2: Implement compute_co_occurrence_matrix

M = np.where(M > 0, 1, 0)

This will turn all numbers greater than 1 into 1 while co-occurrence matrix as stated in the assignment itself (and in literature) is the COUNT of co-occurrence of words. Unfortunately the example provided in sanity check code by authors is badly designed as no co-occurrence is greater than 1 in those sentences therefore Your code passes. Here is the quote from assigment definition of co-occurence matrix:

[...] We build a co-occurrence matrix $M$, which is a symmetric word-by-word matrix in which $M_{ij}$ is the number of times $w_j$ appears inside $w_i$'s window.

Removing this line solve half of the problem as now You are assigning co-ocurence twice:

M[outside_idx, center_idx] += 1
M[center_idx, outside_idx] += 1

Note that You are iterating over all words in a document so when assigning values for word2 to co-occur with word1 and word3 (and vice-versa) in next iteration you will assign values for word3 to co-occur with word2 and word4 (and vice-versa) therefore assigning word2<->word3 twice in both direction

All you need is one assignment and the symmetric nature of the problem will fill the symmetric co-occurrence in later iteration.

Hello! Thanks for your repo!
I found in a4 written part (c), the BLEU score may be wrong.
In (i)
Regarding c1: p1=0.6, p2=0.5; c2: p1=0.8,p2=0.5.
In (ii)
c1: p1=0.6, p2=0.5; c2: p1=0.4, p2=0.25
I have tested with nltk.translate.bleu_score.sentence_bleu.

Best
Cheng

Hi,
I am doing the CS224n course by following the YouTube videos. In the Stanford course page, they have only listed up to assignment 4. For assignment 5, only "Assignment 5" is released is written. So I am assuming they would have provided the assignment in some different medium. So if it's okay with you, can you please share the questions for assignment 5? It will be a great help.
Thanks in advance.

I think the code "gradOutsideVecs[negSampleWordIndices] += np.outer((z-1),centerWordVec)*(-1)", at line 147 of word2vec.py, assigment 2, is not right.
Because as the annotation says, "Note: The same word may be negatively sampled multiple times. For example if an outside word is sampled twice, you shall have to double count the gradient with respect to this word."
I write testing codes, gradOutsideVecs[negSampleWordIndices] could not double the gradient, it only add one time(the last one).

Iam a bit confused about why we should delta[outsideWordIdx] -= 1, any hint? thx ( lam kinda new to this stuff)

In a4, we use docopt to parse our options and arguments, but if you try sh run.sh test now, you will recognize that the usage run.py decode [options] MODEL_PATH TEST_SOURCE_FILE TEST_TARGET_FILE OUTPUT_FILE doenst work.

Actually, in script run.sh, our command is python run.py decode model.bin ./en_es_data/test.es ./en_es_data/test.en outputs/test_outputs.txt --cuda and the option --cuda wont get parsing because you have not defined in your correspond usage, so we have to add an option beyond this decode usage like this:
run.py decode [options] MODEL_PATH TEST_SOURCE_FILE TEST_TARGET_FILE OUTPUT_FILE [--cuda].
Btw the options ahead just make trouble for us and if it exits we cant get the whole usage parsed, i suggest remove it and final version is as below:
run.py decode MODEL_PATH TEST_SOURCE_FILE TEST_TARGET_FILE OUTPUT_FILE [--cuda].

On an other hand, the first train usage also cant recognize the option --cuda just like the wildcard option [options] fails, so i have to modified the source code to do training, it's really annoying.

Maybe these bugs just occured after some other bugs of docopt fixed, for convience of some rookies, you can test your code and modified some content to make it work, thanks a lot!