Still can't get the data ): about is13 HOT 10 CLOSED

Anyone?

from is13.

@lucas0 I will solve that later but in the meantime you can split the data yourself in 3 splits, train - valid - test.

from is13.

(if @mesnilgr, don't mind of course), i gave myself some time to update the code a bit.
@lucas0 you may reuse this if you like, i'll put the links of the forked repo below.

the method atisfold in the load.py can accept any pickle file with the same format:
also a method was added to create train-validation-test splits, so you don't need the pickle files for that.

i updated the elman-forward example to load specific file (which should be the pickle file) from command line argument and here you it sends it to the atisfold method.

a script also to create the pickle file for any labeled IOB data.

so you have everything now you are ready to go.

ps: jordan-forward.py and atisfull method in load.py file are not updated yet.

from is13.

@mesnilgr , On a different note, i have noticed in the pickle file the named entity information train_ne in the elman-forward.py are not used. am i missing something ?

from is13.

@hadyelsahar thanks! feel free to send me a pull request. train_ne are not used you're right.

from is13.

Guys, thanks once again for all the attention and help presented.

I've cloned your repo @hadyelsahar, and I guess I'm closer than ever to run this example. This is what I'm getting:

Lucass-MacBook-Pro:examples lucaslourenco$ python elman-forward.py -i atis.pkl
Traceback (most recent call last):
  File "elman-forward.py", line 36, in <module>
    train_set, valid_set, test_set, dic = load.atisfold(args.input)
  File "/Users/lucaslourenco/Developer/OpenIE/RNN/RNN-w2v/elman-jordan-fwd-rnn/examples/load.py", line 35, in atisfold
    data, dicts = cPickle.load(f)
ValueError: too many values to unpack

what is the next move?

I was hopping to learn more about the slot filling problem with the exemple provided by @mesnilgr , but my real task is to build an LSTM to solve the same problem and compare the results. I suspect that I would only reuse the index2word and index2label functions to build an LSTM for the same database, right?

My question is, in order to build an LSTM for the same problem and the same database would be easier to modify elman-forward.py code or create my own from scratch?

from is13.

The code that i have written works only if you use the script to build your pickle file.
As the pickle file will contain only dataset in one variable instead of having train-test splits.

You are expected to update it yourself if you would like, you can write a script to convert the old pickle file and merge the Train and Test sets into one.
or download the Atis Dataset tagged in IOB and use/update the Script to create the Pickle file.

from is13.

@mesnilgr I've realized test_set and train_set are tuples containing each 3 other tuples of the exactly same length, 893 and 4978, as described in the deeplearning tutorial. I wonder why there are 3 sets of each? Are they the same arrays of indexes?

@hadyelsahar I'm trying to understand you generating script in order to update atis.pkl but it's hard for me, as a begginer to python. Would you please send me your updated .pkl from the atis.pkl?

Thanks once again, guys.

from is13.

Another question: In the chapter 12 and in the tutorial (http://deeplearning.net/tutorial/rnnslu.html) there is this example of "index2word" and "index2label". I would like to know those are the inverse dictionaries of "word2idx" and "label2idx", respectively. Thanks!

from is13.

everything should be fine by now.

from is13.