axon-research / c3d-keras Goto Github PK
View Code? Open in Web Editor NEWC3D for Keras + TensorFlow
License: Other
C3D for Keras + TensorFlow
License: Other
I just use the convert_caffe_model.py to convert the c3d model finetuned on UCF101, I have changed the the last fc layer of c3d.py, is that all right? After the conversion, I found the accuracy is very low...I am not sure if it's right to do so..Thank you !
Sorry for creating a new issue here (totally unrelated) but there was no "issues" tab on chuckcho/c3d-tensorflow2 repository. It would be great if you can enable it.
Thank you very much.
Hello,I can't find the sports1M_weights_tf.json in you models, could u please tell me how to get this json? Thank you!!!
I'm getting the error: No such file or directory: './models/sports1M_weights_tf.json'. I would think that the get_weights_and_mean script would be getting the weights. Clearly, it's not. But it's getting a file called "conv3d_deepnetA_sport1m_iter_1900000". It's not clear what I'm supposed to do with this, and when I do try to load this file, I get the error " Unable to open file (File signature not found)", probably because it's not a HDF5 file or it has been corrupted.
Is there supposed to be a weights file that is supposed to be downloaded when you run ./do_everything.sh
Can you please upload its trained model especial its weights on repo or send me on my personal email? I am on Windows and installing all these things are the heck process for me.
It seems that the C3D team have changed the folder structure of their Git repo. When getting the caffe.proto file in do_everything.sh, line 13 needs to be:
wget -N https://raw.githubusercontent.com/facebook/C3D/master/C3D-v1.0/src/caffe/proto/caffe.proto
I was following the steps mentioned but getting this error.
I was wondering if there is any work done towards being able to replicate the training. I am currently trying to adapt the network to 8 frames instead of 16, for which (as far as I know) the network needs to be trained from scratch.
Traceback (most recent call last):
File "test_model.py", line 225, in <module>
main()
File "test_model.py", line 171, in main
X = vid[start_frame:(start_frame + 16), :, :, :]
IndexError: too many indices for array
I have already checked the closed issue with this exact same problem. My opencv installation is correct since I already use it in other projects. Any ideas?
Hello,
When i run the convert_caffe_model.py, I have a problem.
[libprotobuf ERROR google/protobuf/io/coded_stream.cc:207] A protocol message was rejected because it was too big (more than 67108864 bytes). To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
Traceback (most recent call last):
File "convert_caffe_model.py", line 121, in
main()
File "convert_caffe_model.py", line 67, in main
p.ParseFromString(open(caffe_model_filename, 'rb').read())
google.protobuf.message.DecodeError: Error parsing message
please, give me some hints, thank you very much!
I am using C3D for a different task, so I was wondering if the mean_cube
was calculated from UCF101 dataset.
Please set "image_dim_ordering" in ${KERASCONF} to be "tf"
When I try to run bash do_everything.sh I get above error. Can anyone explain how to address this error
Thanks
Regards
Did you export model and weights of Sport1M for Keras and Tensorflow like this post? May I download it for pre-trained my networks? Thank you so much
When running the script, I end-up getting the following error:
Traceback (most recent call last):
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 671, in _call_cpp_shape_fn_impl
input_tensors_as_shapes, status)
File "/home/dani/Software/anaconda3/lib/python3.6/contextlib.py", line 89, in __exit__
next(self.gen)
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Negative dimension size caused by subtracting 2 from 1 for 'pool3/MaxPool3D' (op: 'MaxPool3D') with input shapes:
[?,1,4,28,256].
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "convert_caffe_model.py", line 121, in <module>
main()
File "convert_caffe_model.py", line 51, in main
model = c3d_model.get_model(summary=True, backend=dim_ordering)
File "/home/dani/Git/c3d-keras/c3d_model.py", line 36, in get_model
border_mode='valid', name='pool3'))
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/keras/models.py", line 455, in add
output_tensor = layer(self.outputs[0])
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/keras/engine/topology.py", line 545, in __call__
output = self.call(inputs, **kwargs)
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/keras/layers/pooling.py", line 323, in call
data_format=self.data_format)
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/keras/layers/pooling.py", line 381, in _pooling_function
padding, data_format, pool_mode='max')
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 3055, in pool3d
x = tf.nn.max_pool3d(x, pool_size, strides, padding=padding)
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1625, in max_pool3d
strides=strides, padding=padding, name=name)
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2329, in create_op
set_shapes_for_outputs(ret)
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1717, in set_shapes_for_outputs
shapes = shape_func(op)
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1667, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn
debug_python_shape_fn, require_shape_fn)
File "/home/dani/Software/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 676, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Negative dimension size caused by subtracting 2 from 1 for 'pool3/MaxPool3D' (op: 'MaxPool3D') with input shapes: [?,1,4,28,256].
download_test_video.sh: line 4: youtube-dl: command not found
Using TensorFlow backend.
[Info] image_dim_order (from default ~/.keras/keras.json)=<function image_dim_ordering at 0x7f5a8ad332f0>
[Info] Using backend=<function image_dim_ordering at 0x7f5a8ad332f0>
[Info] Reading model architecture...
Traceback (most recent call last):
File "test_model.py", line 225, in <module>
main()
File "test_model.py", line 136, in main
model = model_from_json(open(model_json_filename, 'r').read())
FileNotFoundError: [Errno 2] No such file or directory: './models/sports1M_weights_tf.json'
I understand that there are certain problems inherent of using python3, but what about the rest? youtube-dl
should be added as dependency. Where can I download sports1M_weights_tf.json
?
Thanks,
Dani
Hi,
Many thanks for this repo.
I ran the code, but got stuck due to ffmpeg inconsistencies which renders cv2.VideoCapture()
unusable.
To get a workaround, I bursted frames from the video using:
ffmpeg -i dM06AMFLsrc.mp4 -vf scale=171:128 -r 25 -f image2 'vid_frames/%04d.jpg'
Then read it in the code using scipy.ndimage module:
def load_video_clip():
print("[Info] Loading a sample video...")
path_to_vid = "/home/vid_frames/"
vid = []
for frame_num in range(2000, 2016):
frame = scipy.ndimage.imread(path_to_vid + str(frame_num) + ".jpg")
vid.append(frame)
return np.array(vid, dtype=np.float32)
The point is that the bursted video frames looks same enough as in https://gist.github.com/albertomontesg/d8b21a179c1e6cca0480ebdf292c34d2.
But, the output I get is:
Position of maximum probability: 359
Maximum probability: 0.13669
Corresponding label: volleyball
Top 5 probabilities and labels:
volleyball: 0.13669
gaelic football: 0.08436
inline speed skating: 0.07865
roller derby: 0.06737
hurdles: 0.06567
What in your opinion might be causing this ?
Thanks again.
Hi @chuckcho
I have successfully converted C3D v1.0 model to keras(with theano backend) follow https://gist.github.com/albertomontesg/d8b21a179c1e6cca0480ebdf292c34d2
For C3D v1.1,
First, I extracted caffe weights to h5 file following the codes which were used to convert ResNet50 to keras
https://github.com/MoyanZitto/keras-scripts/blob/master/utils/caffe_converter.py (tensorflow backend, I am also using tf backend now).
Second, load weight using
def load_weights(model, weights_path):
"""
This function load the pretrained weights to the model
"""
f = h5py.File(weights_path, 'r')
for layer in model.layers:
if layer.name[:3] == 'res':
layer.set_weights([f[layer.name]['weights'][:].transpose((2,3,4,1,0)), f[layer.name]['bias'][:]])
elif layer.name[:2] == 'bn':
scale_name = 'scale'+layer.name[2:]
weights = []
weights.append(f[scale_name]['weights'][:])
weights.append(f[scale_name]['bias'][:])
weights.append(f[layer.name]['weights'][:])
weights.append(f[layer.name]['bias'][:])
layer.set_weights(weights)
model.get_layer('conv1').set_weights([f['conv1']['weights'][:].transpose((2,3,4,1,0)), f['conv1']['bias'][:]])
model.get_layer('fc8-new').set_weights([f['fc8-new']['weights'][:].T, f['fc8-new']['bias'][:]])
return model
model = get_resC3D_k2()#keras model architecture
model = load_weights(model, 'c3d_resnet50-1.h5')
But keras model using the converted weights always output one class whatever the input is.
Can you give me some suggestions?
If I use theano backend with your code it computes different features in fc6 layer, which are not accurate. Is there anyway I can use your code with Theano backend or are C3D weights ported in theano? Please help me out.
First, thanks a lot for providing this tensorflow implementation.
I successfully run it and got the same probabilities as given in the script.
However I compared on the snippet used in
https://github.com/Lasagne/Recipes/blob/master/examples/Video%20features%20with%20C3D.ipynb
and get quite different probabilities compared to the caffe reference implementation.
This code:
wiffle ball: 30.72%
knife throwing: 22.49%
boomerang: 5.09%
croquet: 4.96%
disc golf: 3.22%
wallball: 3.10%
rounders: 2.98%
archery: 1.96%
skeet shooting: 1.91%
bocce: 1.87%
Caffe:
wiffle ball: 29.91%
knife throwing: 13.11%
croquet: 11.27%
disc golf: 5.29%
kickball: 5.18%
rounders: 4.48%
bocce: 3.53%
dodgeball: 2.27%
boomerang: 1.71%
tee ball: 1.39%
did you ever verify the output against the caffe implementation?
Any idea what the reason for this might be?
Hello Sir,
FileNotFoundError: [Errno 2] No such file or directory: 'models/sports1M_weights_tf.json'
I can't find those two files !!
Hi! Thanks so much for doing this work, its already saved me a great deal of time since most of my stuff is built in Keras :)!
I'm attempting to fine tune this model for a completely different task, predicting steering angles from dash camera videos. I have the data: driving sequence images and steering angles with each frame. 40k total frames. Though, I'm having some issues and wanted to be sure it doesn't have anything to do with any limitation related to the model/weights your scripts output.
Please note this is my first time fine tuning anything! I read a couple of blogs and such to get an idea but please be critical!
I decided to split the work into two main steps.
Here was my plan of action:
The code:
json_file = open(orignal_model_path, 'r')
loaded_model_json = json_file.read()
json_file.close()
model = model_from_json(loaded_model_json)
model.load_weights(orginal_weights_path)
model.summary()
model.pop()
for layer in model.layers:
layer.trainable = False
model.add(Dense(1, activation='tanh', name='cutom'))
The output I got from this step was already problematic. It was only predicting angles between 10 and 12 degrees. You can check out some of my logs here: http://imgur.com/a/zKTwY
Next:
6. Unfreeze all layers and load model/weights from step 5.
7. Train model again (50 epochs). For my optimizer I used SGD with a lr(0.0001).
The output I got from this layer actually just kept on outputting a single number for every single frames. Which means something went terribly wrong.
Any feedback on where you think I'm going wrong would be awesome!
Could you publish the pretrained model on UCF-101 in keras?Thank you!
Hello! Have you fixed the issue: Activation outputs begin to differ from conv4a layer for Theano and TensorFlow.
file "test_model.py",line 169, in main
X = vid[start_frame:(start_frame + 16), :, :, :,]
IndexError:too many indices for array
hi, i run the code "python test_model.py" ,then get this. please help me
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.