yscacaca / deepsense Goto Github PK

Why is CONV_MERGE_LEN = 8?

The shape of sensor_conv_in is (64, 20, 2, 4, 64). So shouldn't CONV_MERGE_LEN be 4 so that it matches matches the features from each sensor?

Here you have put WIDE = 20 as the length of a sequence .

You also use tf.nn.dynamic in order to work with different length sequences .

In order to make that happen you have a tensor - length that calculate the length of the sequence in each batch.

But as my understanding all the input examples have fixed size . My problem is in the real world scenario we can get time steps with only zeros . Then the number of time steps can be changed. That means we can't say always this number is fixed WIDE*FEATURE_DIM .

What should I do ?

Hi I have modified this code to run on multiple GPUs referencing Cifar 10 multi GPU example . But then I am not sure how to do batch normalization in distributed training . Specially

Calculating moving averages for use in the inference . How to do this ?

In this line for the function layers.batch_norm you have used

updates_collections=None

So what if we use updates_collections=tf.GraphKeys.UPDATE_OPS

In order to work this we need to explicitly update the GraphKeys class rel event to moving averages .

  update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
  with tf.control_dependencies(update_ops):
    train_op = optimizer.minimize(loss)

Also the TF document says ,

One can set updates_collections=None to force the updates in place, but that can have a speed penalty, especially in distributed settings.

https://github.com/shamanez/DeepSense/blob/master/deepSense_HHAR_tf.py#L97

Hi,

Thank you very much for sharing your code. I downloaded your package and I tried to run it in Ubuntu 16.04 tf-1.1. I have received an error when the function tf.train.shuffle_batch is executed within the function input_line. The output error is:

Caused by op u'shuffle_batch', defined at:
File "deepSense_HHAR_tf.py", line 215, in
batch_feature, batch_label = input_pipeline(csvFileList, BATCH_SIZE)
File "deepSense_HHAR_tf.py", line 64, in input_pipeline
min_after_dequeue=min_after_dequeue)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 1214, in shuffle_batch
name=name)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 784, in _shuffle_batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 458, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 1328, in _queue_dequeue_many_v2
timeout_ms=timeout_ms, name=name)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op
op_def=op_def)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1228, in init
self._traceback = _extract_stack()

OutOfRangeError (see above for traceback): RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 8)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

Suggestions?

Thanking in advance.

Gabriel.

In this line what I have understood is you represent the label as an one hot encoding . Any way in the csv file where can I find the label?

When computing loss with the optimizer object does tensorflow add regularization loss automatically for all the parameters ?

Hi, I'm working on the cat tracking problem with DeepSense, you said that

1. one GPS measurement -> one time interval.
2. one sample -> a journey with speed from 0 to 0.

I think the number of time intervals(i.e "T" in your paper) is different in different samples, and my question is how to fix this problem, to pad 0 in pre-processing or slove it in the model?

I download the data, unzip it in ./ and run deepSense_HHAR_tf.py, and got this error:

Traceback (most recent call last):
  File "deepSense_HHAR_tf.py", line 257, in <module>
    _, lossV, _trainY, _predict = sess.run([discOptimizer, loss, batch_label, predict])
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 997, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
    target_list, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 10)
	 [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
	 [[Node: shuffle_batch/_19477 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_2574_shuffle_batch", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

Caused by op u'shuffle_batch', defined at:
  File "deepSense_HHAR_tf.py", line 209, in <module>
    batch_feature, batch_label = input_pipeline(csvFileList, BATCH_SIZE)
  File "deepSense_HHAR_tf.py", line 59, in input_pipeline
    min_after_dequeue=min_after_dequeue)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/input.py", line 1217, in shuffle_batch
    name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/input.py", line 788, in _shuffle_batch
    dequeued = queue.dequeue_many(batch_size, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/data_flow_ops.py", line 457, in dequeue_many
    self._queue_ref, n=n, component_types=self._dtypes, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 946, in _queue_dequeue_many_v2
    timeout_ms=timeout_ms, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1269, in __init__
    self._traceback = _extract_stack()

OutOfRangeError (see above for traceback): RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 10)
	 [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
	 [[Node: shuffle_batch/_19477 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_2574_shuffle_batch", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

The "current size" changes randomly for different run. If set shuffle_sample as False, it also gives an error but with RandomShuffleQueue changed to FIFOQueue.

Any recommendation to solve it? Thanks!

In this line you do the convolution2D operation with acceleration inputs(acc_inputs) . The size has mentioned as follows ,

(BATCH_SIZE, WIDE, FEATURE_DIM, CHANNEL=1)

So I have this problem . Batch_Size is for number of examples in a mini batch . But here you have to do the convolution operation for acceleration input of each time step . In the acc_inputs do we have each time step inputs . And can we do the convolution in parallel way ?

Hi,thank you for your release.I run your code in computer successfully.I deploy the *.pb on android,but I meet the issue like this:No OpKernel was registered to support Op 'All' with these attrs. Registered devices: [CPU], Registered kernels: .
I find the 'All' op used in the deepSense/rnn/All layer.
Could you give me some advices to fix the bug?
Thank you very much!

I am using your network for my research and wanted to evaluate your network on android.
The Tensorflow version you used to generate the frozen graph is TF 1.4 according to the documentation.
I trained and generated a pb file but I am still confused how to run it on android. Currently Tensorflow has TF Lite. I tried to convert the pb file to tflite but I get errors of compatibility.
Tensorflow Lite was not available yet on TF 1.4.
How did you run the model on android without using TensorFlow lite?
(I have limited knowledge about android apps and java)

Hi,

We are trying to understand the preprocessing and we ran into a couple of questions.
In the preprocessed dataset, for each file (let's say eval_1.csv), we have 2406 values in one row. We know that the last 6 elements compose the one hot representation of ground truth. Can you elaborate on how the remaining 2400 points are generated.

Thanks for your time,

Hi,

I've read your code for sep_HHAR_data and compared with the paper. I found that you set the fftSpan=5 (correspond to 0.5 s) while the paper take 0.25 s. So, can you confirm your code?

Thanks Yao!

Nice work!

Would you release the training/eval codes for the CarTrack task?

Thanks!

In this line . It even says ,

updates_collections=None

What is actually meant by this ?

In this This line.

These things mainly control how to extract a batch from the que. So assigning them incorrectly can cause errors. So can you please explain ?

Hello,
Thanks for your work. Recently I have read the DeepSense and SADeepSense and want to run the code on my computer, but I haven't found the SADeepSense code. I'd appreciate it if you could share the SADeepSense code.
Best wishes!

anyone can run the code without problem? I am using tensorflow 1.8.0 with python 3.5 but keep getting problem to run the code, currently I am stuck with plot.py for y_vals.

The error occur mention about unhashable type : 'dict_keys'

I hope someone can help me if you have successfully run the code

I tried train this model with data provided in readme, I find the training procedure is slow. It already has taken more than 2 hours to just iterate about 700 times.
I watch the usage of gpu and cpu, gpu usage is about 10%~20%, sometimes even low to 0%~3%, cpu usage is about 50%, sometimes drop to near 0.
CPU usage diagram is as below

My laptop is with intel i7 6700k and gtx 1050ti. I run the code in pycharm with python 2.7 and tensorflow 1.3.

Hi @yscacaca ,

Thank you for sharing your source code.
I tried to run your source code but faced the below error:

OutOfRangeError (see above for traceback): RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 14

I tried to change tensorflow version 10 times between 1.1 to 1.10 with python 2.7 and it didn't solve the problem.
I've read the previous issues on this problem and didn't find a proper solution.
Is it possible to share the packages versions here? like python, tensorflow, ...
Or share all requirement packages versions as a yml file?

It gives me the error on the below line:
example_batch, label_batch = tf.train.shuffle_batch( [example, label], batch_size=batch_size, num_threads=16, capacity=capacity, min_after_dequeue=min_after_dequeue)

Why this is bit slow . What is the reason ? Do you think it's because of the #5 issue the input/output stream ?

In the keras there's TimeDistributed wrapper where we can use to do convolution for each time step. So I think by feeding the tensor with Batch Size , Sequence Length , .. etc we can have the same property. Am I right ?

I noticed that the data you shared is different from the original data, can you share your code that for data processing? Thanks for your sharing!!!

Hi, I want to learn about DeepSense using my own gait data collection but for gait recognition purpose not for activity recognition, recorded with 50Hz in a csv files, with 150 participants, the format is timestamp followed by 3 axis acc and 3 axis gyro. How I should preprocess the data to become like your eval or train dataset?

because my sampling rate is fix to get 50 samples per seconds, should I still perform the time interpolation?

I would really appreciate it if you have some explanation about your dataset. Thank you

1. Is this representing frequency domain data for each sensor .
2. How to find the label. Which is like what is the task related to each example ?
3. How to find the different time duration in each example

In your paper you write that you first apply a 2d convolution to X^(k) and then two 1d convolution. Finally you write that you flatten the matrix to stack the vectors of each sensor.
When I look at your code on Github, I see that you don't flatten the matrix after the individual convolution and use a 3d convolution layer .

Question:
Is there an important reason why you didn't flatten the matrix and use a 3d convolution for the matrix X^(3)?

Hi,I am very new to Deep Learning .If anyone has implemented the code for GAIT ANALYSIS and a brief explanation of the code implemented ,that would be great. Also the code for preprocessing the data would also help.

Thanks

I would like to test the neural network in an Android app, by importing the frozen and optimized model with TF's TensorFlowInferenceInterface. I have edited the deepSense_HHAR_tf.py source to export the evaluation model in PB and PBTXT formats, and exported a checkpoint using tf.train.Saver. Then, I applied TF's freeze_graph tool.

In the Android application I can successfully load the model and feed the input parsed from the CSV files, but trying to run it yields the following exception:

No OpKernel was registered to support Op 'FIFOQueueV2' with these attrs. Registered devices: [CPU], Registered kernels: no registered kernels

This is probably due to Android's TensorFlow library not including the specified Op.

I have then applied TF's optimize_for_inference tool to the frozen graph, and imported the output in the Android app, but with this approach an exception is immediately thrown upon loading the model:

NodeDef expected inputs '' do not match 1 inputs specified; Op<name=Const; signature= -> output:dtype; attr=value:tensor; attr=dtype:type>; NodeDef: deepSense_1/rnn/while/add/y = Const[dtype=DT_INT32, value=Tensor<type: int32 shape: [] values: 1>](deepSense_1/rnn/while/Switch:1)

Do you have some advice on exporting the model for Android, based on your experience with the Nexus 5?