gowthamgottimukkala / i3d_feature_extraction_resnet Goto Github PK

Hi @GowthamGottimukkala,

Thanks for this repo. Is there any reason that you're using the baseline I3D by default? Shouldn't the non-local version give slightly better results?

Hi @GowthamGottimukkala,
I observed that the ID3 features extracted using the code provided in this repository, and the ones provided by the authors of RTFM for the ShanghaiTech dataset (here) are quite different.
I believe the following code should run fine-

from resnet import i3_res50
from extract_features import run 

pretrainedpath = "pretrained/i3d_r50_kinetics.pth"
frequency = 16
batch_size = 20
sample_mode = 'oversample'
frames_path = "frames/01_0015"

i3d = i3_res50(400, pretrainedpath)
i3d.cuda()
i3d.train(False)

features = run(i3d, frequency, frames_path, batch_size, sample_mode)

frames_path is the path to the directory containing the frames. But the features extracted do not match at all with the ones that were used by the authors of RTFM. Please correct me if I am making a mistake.

Thank you for the great work! After reading your source codes, I realize that you used the function
oversample_data to perform 10 crop operations without doing other processing. However, in the original RTFM paper/repo, the author mentioned about the group normalize step. Would you please provide some clues about how I can adapt your code to implement the group normalize? Thank you.

Hi,

Thanks for sharing the code. I have encountered an error while running the code using the sample command python3 main.py --datasetpath=samplevideos/ --outputpath=output. My setup is python=3.7, pytorch=1.10, cuda=10.2.

I got the following message. Could you please tell me how to fix it? Thank you very much.

Received Pretrained model..
Generating for samplevideos/Abuse001_x264.mp4
Preprocessing done..
batchsize 20
Traceback (most recent call last):
  File "main.py", line 49, in <module>
    generate(args.datasetpath, str(args.outputpath), args.pretrainedpath, args.frequency, args.batch_size, args.sample_mode)    
  File "main.py", line 34, in generate
    features = run(i3d, frequency, temppath, batch_size, sample_mode)
  File "extract_features.py", line 97, in run
    temp = forward_batch(batch_data_ten_crop[i])
  File "extract_features.py", line 66, in forward_batch
    features = i3d(inp)
  File "/miniconda3/envs/cv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "resnet.py", line 189, in forward
    feat = self.forward_single(batch['frames'])
  File "resnet.py", line 181, in forward_single
    x = self.layer3(x)
  File "/miniconda3/envs/cv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/miniconda3/envs/cv/lib/python3.7/site-packages/torch/nn/modules/container.py", line 141, in forward
    input = module(input)
  File "/miniconda3/envs/cv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "resnet.py", line 63, in forward
    out = self.conv1(x)
  File "/miniconda3/envs/cv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/miniconda3/envs/cv/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 590, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/miniconda3/envs/cv/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 586, in _conv_forward
    input, weight, bias, self.stride, self.padding, self.dilation, self.groups
RuntimeError: CUDA error: unknown error
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

I use the sample video getting a .npy file which has the shape (170, 10, 2048)
Can you tell me the meaning of each dimension？
Thank you very much

Thank you for your code! I ran the code very smoothly, but it did not achieve the expected effect after the run of RTFM. The result of the SH dataset was only less than 70%, but the features of the RTFM author can meet the expectations. My reproduction steps are as follows:

1. Use your code to extract i3D features
2. Run Make_list_SH.py in RTFM
   3.Train and test RTFM

Are there any steps I miss? thank you for your time

wget https://dl.fbaipublicfiles.com/video-nonlocal/i3d_nonlocal_32x2_IN_pretrain_400k.pkl -P pretrained/
Giving me this error. How to fix it?

Invoke-WebRequest : Parameter cannot be processed because the parameter name 'P' is ambiguous. Possible matches include: -Proxy
-ProxyCredential -ProxyUseDefaultCredentials -PassThru -PipelineVariable.
At line:1 char:91

• ... .com/video-nonlocal/i3d_baseline_32x2_IN_pretrain_400k.pkl -P pretrai ...

•                                                            ~~
  

  • CategoryInfo : InvalidArgument: (:) [Invoke-WebRequest], ParameterBindingException
  • FullyQualifiedErrorId : AmbiguousParameter,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

Thank you so much for creating this useful repo!

After I follow the instructions in the Usage section but the output I get in /temp/ is the frames of the video.
Please tell me how to get a .npy file and shape (10, n/16, 2048) like in this paper Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning.

Looking forward to your help

for each video, why it saves I3D feature numpy file of dimension n/16 * 10 * 2048 where n is the no.of frames in the video?

Thanks for your excellent work. I wonder that I3D has RGB and Flow pathway but it seems that you only user RGB image to extract I3D feature. So, if I use your code, I can only get RGB feature of I3D? Am I right?

hello, how to extract the optical flow I3d feature? Can i3d_baseline_32x2_IN_pretrain_400k.pkl be used directly after extract optical flow images or is there have other optical flow model? thank you

Hi,
Nice work on feature extraction
I am facing a issue while running the below line of code:
python -m utils.convert_weights pretrained/i3d_baseline_32x2_IN_pretrain_400k.pkl pretrained/i3d_r50_kinetics.pth
It is giving out error: ModuleNotFoundError: No module named 'models'
I followed the following lines of code
! git clone https://github.com/GowthamGottimukkala/I3D_Feature_Extraction_resnet.git
!wget https://dl.fbaipublicfiles.com/video-nonlocal/i3d_baseline_32x2_IN_pretrain_400k.pkl -P pretrained/
cd I3D_Feature_Extraction_resnet
!python -m utils.convert_weights pretrained/i3d_baseline_32x2_IN_pretrain_400k.pkl pretrained/i3d_r50_kinetics.pth
can you help me with this issue?