mesnilgr / is13 Goto Github PK
View Code? Open in Web Editor NEWRNN for Spoken Language Understanding
RNN for Spoken Language Understanding
as mentioned (on http://deeplearning.net/tutorial/rnnslu.html):
Running experiments on ATIS using this repository will run one epoch in less than 40 seconds
this must be for nh=100, right?
for nh=200 (at least for my pc) it takes about 3 times longer than for nh=100
--is that what it is supposed to be?
--how can we speed it up?
thanks in advance
I wondering where to download an ATIS dataset including the intent labels?
The 5 folds available at
http://lisaweb.iro.umontreal.ca/transfert/lisa/users/mesnilgr/atis/
does not include the intent labels, and the dropbox link seems to be no longer active.
I would be very grateful for any hint of how to obtain the ATIS dataset.
Hi,
I would like to re-use the learned word embeddings (self.emb) in other experiments. It looks difficult to me to know what each word embedding (stored embeddings.npy) in the matrix corresponds to (words2idx dictionary). In addition there is an additional "word" which is used for padding.
My question is: how can I align the words with learned embeddings?
Thanks
Hi
I am trying to use your code for event detection as a sequence labelling task. I looked at the atis dataset and tried to create one in same format but I get this error while train
ing. Could you please take a look at the pickle file attached. It just has one sentence in all training, test and validation sets for easier debugging.
I would really appreciate your help on this. I have been trying to get this done for quite sometime.
Traceback (most recent call last):
File "/home/ddua/workspace/RNN/examples/elman-forward.py", line 74, in
rnn.train(word_batch, label_last_word, s['clr'])
File "/usr/local/lib/python2.7/dist-packages/Theano-0.7.0-py2.7.egg/theano/compile/function_module.py", line 615, in call
storage_map=self.fn.storage_map)
File "/usr/local/lib/python2.7/dist-packages/Theano-0.7.0-py2.7.egg/theano/compile/function_module.py", line 604, in call
outputs = self.fn()
IndexError: index out of bounds
Apply node that caused the error: Subtensor{int64, int64, int64}(forall_inplace,cpu,scan_fn}.1, Constant{-1}, Constant{0}, ScalarFromTensor.0)
Inputs types: [TensorType(float64, (False, True, False)), Scalar(int64), Scalar(int64), Scalar(int64)]
Inputs shapes: [(1, 1, 4), (), (), ()]
Inputs strides: [(32, 32, 8), (), (), ()]
Inputs values: [array([[[ 0.30284221, 0.21770984, 0.31160879, 0.16783916]]]), -1, 0, 5]
P.S. Please remove .png from the extension of pkl file. I couldn't attach a .pkl file in github.
Thanks a lot,
Dheeru Dua
Hello!
Very nice work.
Can u give a valid link for this dataset? Thank u very much!
README.md - bug
git clone [email protected]:mesnilgr/is13.git
cd is13
python examples/elman-forward.py
error
Traceback (most recent call last):
File "examples/elman-forward.py", line 9, in
from is13.rnn.elman import model
ImportError: No module named is13.rnn.elman
git clone [email protected]:mesnilgr/is13.git
python is13/examples/elman-forward.py
is no error
I'm not sure if data related issues are within scope, but the second example from the training dataset seems incorrect to me. The output looks like
WORD LABEL
what O
is O
the O
arrival B-flight_time
time I-flight_time
in O
san B-fromloc.city_name
francisco I-fromloc.city_name
for O
the O
DIGITDIGITDIGIT B-depart_time.time
am I-depart_time.time
flight O
leaving O
washington B-fromloc.city_name
Shouldn't san francisco
be toloc.city_name
?
Hello,
it would be really useful to know what are the elements on the .pkl.
There are 3 "test_set", 3 "train_set" and 3 dictionaries inside "dicts":
I know they are simple questions, and I'm sorry if I'm being inconvenient with all those questions, but please be patient with me, I'm new on deeplearning. Thank you.
I've seen different issue reports from other people but they couldn't help me. I'm using windows now and no matter how I try, it seems I can't fix this, even with the path added to my environment variables. I've tried adding the is13 path in most different ways without success. I always end up getting "no module named is13.data" when try to run any of the two examples. Can someone help me? Thanks.
/home/ubgpu/github/is13:/usr/local/lib/python2.7/dist-packages
ubgpu@ubgpu:~/github/is13$
ubgpu@ubgpu:/github/is13$ python examples/elman-forward.py/github/is13$
Traceback (most recent call last):
File "examples/elman-forward.py", line 8, in
from is13.data import load
ImportError: No module named is13.data
ubgpu@ubgpu:
hi,@mesnilgr. i want to run the code:
import theano, numpy
from theano import tensor as T
# nv :: size of our vocabulary
# de :: dimension of the embedding space
# cs :: context window size
nv, de, cs = 1000, 50, 5
emb = theano.shared(0.2 * numpy.random.uniform(-1.0, 1.0, \
(nv+1, de)).astype(theano.config.floatX)) # add one for PADDING at the end
idxs = T.imatrix() # as many columns as words in the context window and as many lines as words in the sentence
x = self.emb[idxs].reshape((idxs.shape[0], de*cs))
but errors happened in emb[idxs].i do not understand why emb[idxs] is correct?i think emb is a matrix,idxs is also a matrix.So why?
I would appreciate it if you could give me some tips.
thanks.
To examine the running of the code step-by-step, I copy-pasted all the elman-keras code in an iPython Notebook (with all the dependencies).
The weights were initialized normally after model.compile(..), but after running the training section, all the weights turned into 'nan'.
Need some help to understand the anomaly.
Thanks!
In OSX, after finishing training epoch 0 I get this error:
OSError: [Errno 2] No such file or directory: 'conlleval.pl'
petrux@PC:~$ git clone [email protected]:mesnilgr/is13.git
Cloning into 'is13'...
remote: Counting objects: 94, done.
remote: Total 94 (delta 0), reused 0 (delta 0), pack-reused 94
Receiving objects: 100% (94/94), 19.61 KiB | 0 bytes/s, done.
Resolving deltas: 100% (38/38), done.
Checking connectivity... done.
petrux@PC:~$ cd is13/
petrux@PC:~/is13$ ls
data examples __init__.py metrics README.md rnn utils
petrux@PC:~/is13$ python examples/jordan-forward.py
Traceback (most recent call last):
File "examples/jordan-forward.py", line 8, in <module>
from is13.data import load
ImportError: No module named is13.data
Maybe there is something wrong with some __init__.py
file? HTH.
EDIT: the problem was that the package was not in the PYTHONPATH
. My bad. Sorry!
no acces to data on http://www-etud.iro.umontreal.ca/~mesnilgr/ anymore
no possible download of http://www-etud.iro.umontreal.ca/~mesnilgr/atis/atis.fold3.pkl.gz
and the perl evaluation script
cc @lucas0
In W7 It starts to learn on epoch 0, but then:
chmod('conlleval.pl', stat.S_IRWXU) # give the execute permissions
any hints? Thanks!
Hi,
I'm running your code rnnslu.py for ATIS (the the 5 folds of data you provide us with)
(DeepLearningTutorials/code/rnnslu.py)
I expected to get F1=94.98% as mentioned in the paper:
'Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding'
(table 3: Elman/Single,context(9))
Instead, the best performance with this code is F1=94.058% for cont_win=9 and emb_dim=100.
Is that what I'm suppose to get?
Thank you in advance!
Hello,
I'm trying to use your pre-processing methods to feed my LSTM RNN, and I cannot find a way to use the method for creating the Word Embeddings matrixes:
import theano, numpy
from theano import tensor as T
# nv :: size of our vocabulary
# de :: dimension of the embedding space
# cs :: context window size
nv, de, cs = 1000, 50, 5
embeddings = theano.shared(0.2 * numpy.random.uniform(-1.0, 1.0, \
(nv+1, de)).astype(theano.config.floatX)) # add one for PADDING at the end
idxs = T.imatrix() # as many columns as words in the context window and as many lines as words in the sentence
x = self.emb[idxs].reshape((idxs.shape[0], de*cs))
I can do the context window, but the code provided for the Word Embedding generation gives me this error:
NameError: name 'self' is not defined
I've tried to put the code in another .py file but is of no use, I'm sure it has a simple solution but I can't figure it out. Would you please give me a hand in this?
Thank you really much for offering your code, i appreciate it
(it's really helpful in order to understand rnns in slu - ive just started my master about them)
there are a couple of issues i've already dealt with in order to run rnnslu.py
I downloaded DeepLearningTutorials-master
and in code/rnnslu.py i replaced atisfold() and load() and added download_dropbox()
and finally i got the right data but now the error is
<<IndexError: index 557 is out of bounds for size 534>>
it is from function train() when self.sentence_train(words, labels, learning_rate)
as i see voc_size= 533 but here its trying to find the word id 557
SO: is it possible to upload sth that works without any errors?
(i tried to fix it myself, but then sth else came up and i thought maybe you could help)
thx in advance
Dir Sir or Madam,
The following errors occurred while running your code "is13/examples/elman-keras.py". Could you help me with that?
Thank you.
Traceback (most recent call last):
File "is13/examples/elman-keras.py", line 77, in
model.train_on_batch(X, Y)
File "/home/kevin/.local/lib/python2.7/site-packages/keras/models.py", line 1048, in train_on_batch
class_weight=class_weight)
File "/home/kevin/.local/lib/python2.7/site-packages/keras/engine/training.py", line 1838, in train_on_batch
self._make_train_function()
File "/home/kevin/.local/lib/python2.7/site-packages/keras/engine/training.py", line 997, in _make_train_function
**self._function_kwargs)
File "/home/kevin/.local/lib/python2.7/site-packages/keras/backend/theano_backend.py", line 1233, in function
return Function(inputs, outputs, updates=updates, **kwargs)
File "/home/kevin/.local/lib/python2.7/site-packages/keras/backend/theano_backend.py", line 1219, in init
**kwargs)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/compile/function.py", line 317, in function
output_keys=output_keys)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/compile/pfunc.py", line 486, in pfunc
output_keys=output_keys)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/compile/function_module.py", line 1841, in orig_function
fn = m.create(defaults)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/compile/function_module.py", line 1715, in create
input_storage=input_storage_lists, storage_map=storage_map)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/link.py", line 699, in make_thunk
storage_map=storage_map)[:3]
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/vm.py", line 1084, in make_all
impl=impl))
File "/home/kevin/.local/lib/python2.7/site-packages/theano/scan_module/scan_op.py", line 866, in make_thunk
on_unused_input='ignore')
File "/home/kevin/.local/lib/python2.7/site-packages/theano/compile/function.py", line 317, in function
output_keys=output_keys)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/compile/pfunc.py", line 486, in pfunc
output_keys=output_keys)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/compile/function_module.py", line 1841, in orig_function
fn = m.create(defaults)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/compile/function_module.py", line 1715, in create
input_storage=input_storage_lists, storage_map=storage_map)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/link.py", line 699, in make_thunk
storage_map=storage_map)[:3]
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/vm.py", line 1084, in make_all
impl=impl))
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/op.py", line 955, in make_thunk
no_recycling)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/op.py", line 858, in make_c_thunk
output_storage=node_output_storage)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/cc.py", line 1217, in make_thunk
keep_lock=keep_lock)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/cc.py", line 1157, in compile
keep_lock=keep_lock)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/cc.py", line 1620, in cthunk_factory
key=key, lnk=self, keep_lock=keep_lock)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/cmodule.py", line 1174, in module_from_key
module = lnk.compile_cmodule(location)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/cc.py", line 1523, in compile_cmodule
preargs=preargs)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/cmodule.py", line 2368, in compile_str
return dlimport(lib_filename)
File "/home/kevin/.local/lib/python2.7/site-packages/theano/gof/cmodule.py", line 302, in dlimport
rval = import(module_name, {}, {}, [module_name])
ImportError: ('The following error happened while compiling the node', forall_inplace,cpu,scan_fn}(Shape_i{1}.0, Subtensor{int64:int64:int8}.0, IncSubtensor{InplaceSet;:int64:}.0, Shape_i{1}.0, simple_rnn_1/kernel, simple_rnn_1/recurrent_kernel, InplaceDimShuffle{x,0}.0), '\n', 'The following error happened while compiling the node', Dot22(<TensorType(float32, matrix)>, simple_rnn_1/kernel_copy), '\n', '/home/kevin/.theano/compiledir_Linux-3.8--generic-x86_64-with-Ubuntu-12.04-precise-x86_64-2.7.3-64/tmpRc1mHB/b8e34ea4363b3812bd111a731cb835f81ccc84235ec10367467e6198f7abc278.so: undefined symbol: _gfortran_st_write_done', '[Dot22(<TensorType(float32, matrix)>, simple_rnn_1/kernel_copy)]')
Apologies for what is more a theoretical question than a code issue, but I am new to RNNs and am using this code to learn.
Why is the cost function based on the negative log likelihood of the last word in the sentence, rather than the mean of the NLLs of the entire sentence? I see this in line 53 of rnn/elman.py for examples.
Thanks in advance for the help as I learn.
i wanted to apply CRF over the RNN output of this repo following http://research.microsoft.com/apps/pubs/default.aspx?id=210167 . I need to replace the softmax function with the crf classsification .But finding difficult to implement in the recurrence function. Any help is appreciated.
Thanks
Can anyone provide a link for dataset download. The one given does not contain the file.
@mesnilgr @petrux @vierja @stray-leone
I have been reading the rnn code and I got an question in rnn/elman.py:
38--> def recurrence(x_t, h_tm1):
h_t = T.nnet.sigmoid(T.dot(x_t, self.Wx) + T.dot(h_tm1, self.Wh) + self.bh)
s_t = T.nnet.softmax(T.dot(h_t, self.W) + self.b)
return [h_t, s_t]
[h, s], _ = theano.scan(fn=recurrence, \
sequences=x, outputs_info=[self.h0, None], \
n_steps=x.shape[0])
p_y_given_x_lastword = s[-1,0,:]
It defines a simple recurrent network and my question is what's the component of s_t?
When we want to get p_y_given_x_lastword we use s[-1,0,:], "-1" means the lase element and ":" means
all, so why we use "0"? Because I think x_t and s_t is 1-d vector so why use index 0?
Could you solve this problem for me? Thank you very much.
Unable to run conlleval.pl even after downloading to code directory.
Errors :
File "C:\AppData\Local\Continuum\anaconda3\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
OSError: [WinError 193] %1 is not a valid Win32 application
File "C:\AppData\Local\Continuum\anaconda3\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
Ok, changed the download URL, and deleted the '.gz' from the atis.pkl files, but I'm still getting
File "/Users/lucaslourenco/Developer/is13-master/data/load.py", line 33, in atisfold
train_set, valid_set, test_set, dicts = cPickle.load(f)
IOError: Not a gzipped file
what method should I use instead of cPickle.load(f)?
def load(filename):
if not isfile(filename):
download('https://www.dropbox.com/s/3lxl9jsbw0j7h8a/'+filename)
f = gzip.open(filename,'rb')
return f
def atisfull():
f = load(PREFIX + 'atis.pkl')
train_set, test_set, dicts = cPickle.load(f)
return train_set, test_set, dicts
def atisfold(fold):
assert fold in range(5)
f = load(PREFIX + 'atis.fold'+str(fold)+'.pkl')
train_set, valid_set, test_set, dicts = cPickle.load(f)
return train_set, valid_set, test_set, dicts
thats how load.py is right now, there's something else I should change? Thank you for the alternative and the support @hadyelsahar and @mesnilgr!
README.md -
The two lines of code to run the examples do not work if you do not add your actual path into the PYTHONPATH variable.
git clone [email protected]:mesnilgr/is13.git
python is13/examples/elman-forward.py
Will throw the error:
Traceback (most recent call last):
File "./is13/examples/elman-forward.py", line 8, in <module>
from is13.data import load
ImportError: No module named is13.data
By default Python do not add your actual path in sys.path and you need to add it explicitly.
In case of Bash I solved the problem by running:
git clone [email protected]:mesnilgr/is13.git
export PYTHONPATH=${PYTHONPATH}:`pwd -P`
python is13/examples/elman-forward.py
This line adds your current path without following symbolic links.
I know that in every Shell and Operating System the solution can be a different procedure, for that reason, I think you could just indicate that it is necessary to add your actual path in the variable PYTHONPATH.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.