datamllab / autovideo Goto Github PK

works with torch==1.9.0 , torchvision==0.10.0 because torchvision has deprecated Scale in favour of Resize but d3m does not support it yet, so need to downgrade to torchvision<0.12.0 for this repo to work.

when i ran

"from autovideo import extract_frames"

I get following error

"ImportError: cannot import name 'extract_frames' from 'autovideo' (/Volumes/Disk-Data/pose estimation/autovideo-main/autovideo/init.py)"

Hi, in your demo video and your paper you mentioned this nice and fancy ui based on Orange. But I cannot seem to find it in the code. Can you point it to me? Also how do I bring up the ui after I install it? Thank you.

Hi,

I'm trying to benchmark the hmdb51 and ucf101 datasets with the pertained weights available on Google Drive. I'm unfamiliar with axolotl library and am a little confused on how to populate fitted_pipeline['runtime'] if I don't try fitting using example/fit.py. Do you have any suggestions on how to accomplish this?

Thank you,
Rohita

d3m.exceptions.StepFailedError: Step 7 for pipeline c43355b7-0e87-499f-a9f2-defc56b6713a failed

I have trained this model using fit.py on your given dataset and saved weights in the weights directory than I run produce.py these two files run smoothly.
But when I try to run recognize.py it gives me this exception.

I am trying to run the given example of hmbd6 but getting error :

Traceback (most recent call last):
  File "examples/fit.py", line 56, in <module>
    run(args)
  File "examples/fit.py", line 20, in run
    from autovideo.utils import set_log_path, logger
  File "/content/autovideo/autovideo/__init__.py", line 4, in <module>
    from .utils import build_pipeline, fit, produce, fit_produce, produce_by_path, compute_accuracy_with_preds
  File "/content/autovideo/autovideo/utils/__init__.py", line 2, in <module>
    from .axolotl_utils import *
  File "/content/autovideo/autovideo/utils/axolotl_utils.py", line 12, in <module>
    from axolotl.backend.simple import SimpleRunner
  File "/usr/local/lib/python3.7/dist-packages/axolotl/backend/simple.py", line 5, in <module>
    from d3m import runtime as runtime_module
  File "/usr/local/lib/python3.7/dist-packages/d3m/runtime.py", line 23, in <module>
    from d3m.contrib import pipelines as contrib_pipelines
  File "/usr/local/lib/python3.7/dist-packages/d3m/contrib/pipelines/__init__.py", line 13, in <module>
    assert os.path.exists(NO_SPLIT_TABULAR_SPLIT_PIPELINE_PATH)
AssertionError

Running on Google colab.
Code :

!git clone https://github.com/datamllab/autovideo.git

%cd autovideo
!pip3 install -e .

!gdown --id 1nLTjp6l6UucXEy8_eOM5Zj4Q1m79OhmT
!unzip hmdb6.zip -d datasets

!python3 examples/fit.py --alg tsn --data_dir datasets/hmdb6/ --gpu "cuda"

How to resolve it?

How can we read a RTSP LINK and get prediction in this repo. i checked the code but I think RTSP link is not handling
or how can Modify the code in your action recognition repository to accept frames as input instead of a complete video file.

Minimum size of dataset is 4, I have the following hack in produce_by_path that works.

# minimum size is 4
dataset = {
    'd3mIndex': [0,1,2,3],
    'video': [video_name,video_name,video_name,video_name],
    'label': [0,0,0,0]
}

Hi,

First of all, I'd like to congratulate about this repo, we've found this very useful. While training TSM, we've discovered that the parameter is_shift is by default false. Also, the import there cannot be resolved since the original make_temporal_shift code is not integrated into this repo.

Without is_shift enabled, does that mean that we're using a vanilla 2D Resnet50 and averaging the output of every input image in the sequence? Am I missing anything? The original contribution of TSM was this special temporal shift in the internal feature maps of any 2D CNN model.

Thanks in advance.

This issue occurs with python 3.8, typing==3.7.4.3

is this project open-source?
Or are there any documents?

currently, it is working with 0.5 (Confidence Threshold) for both "normal" and "object-falling" actions. I wanted to make changes in my confidence. So how can I change the Confidence Threshold?

Hi all!

I'm running into some problems with generating fitted pipelines for the different algorithms available. So I was trying to run the following command:

python3 examples/fit.py --alg tsn --pretrained --gpu 0,1 --data_dir datasets/hmdb6/ --log_path logs/tsn.txt --save_path fittted_timelines/TSN/

And I got the following output.

--> Running on the GPU

Initializing TSN with base model: resnet50.
TSN Configurations:
input_modality: RGB
num_segments: 3
new_length: 1
consensus_module: avg
dropout_ratio: 0.8

Downloading: "https://download.pytorch.org/models/resnet50-0676ba61.pth" to /home/myuser/.cache/torch/hub/checkpoints/resnet50-0676ba61.pth
100%|##########| 97.8M/97.8M [00:02<00:00, 40.4MB/s]
Downloading: "https://open-mmlab.s3.ap-northeast-2.amazonaws.com/mmaction/models/kinetics400/tsn2d_kinetics400_rgb_r50_seg3_f1s1-b702e12f.pth" to /home/myuser/.cache/torch/hub/checkpoints/tsn2d_kinetics400_rgb_r50_seg3_f1s1-b702e12f.pth
Traceback (most recent call last):
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/d3m/runtime.py", line 1008, in _do_run_step
self._run_step(step)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/d3m/runtime.py", line 998, in _run_step
self._run_primitive(step)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/d3m/runtime.py", line 873, in _run_primitive
multi_call_result = self._call_primitive_method(primitive.fit_multi_produce, fit_multi_produce_arguments)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/d3m/runtime.py", line 974, in _call_primitive_method
raise error
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/d3m/runtime.py", line 970, in _call_primitive_method
result = method(**arguments)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/d3m/primitive_interfaces/base.py", line 532, in fit_multi_produce
return self._fit_multi_produce(produce_methods=produce_methods, timeout=timeout, iterations=iterations, inputs=inputs, outputs=outputs)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/d3m/primitive_interfaces/base.py", line 559, in _fit_multi_produce
fit_result = self.fit(timeout=timeout, iterations=iterations)
File "/home/myuser/autovideo/autovideo/base/supervised_base.py", line 54, in fit
self._init_model(pretrained = self.hyperparams['load_pretrained'])
File "/home/myuser/autovideo/autovideo/recognition/tsn_primitive.py", line 206, in _init_model
model_data = load_state_dict_from_url(pretrained_url)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/torch/hub.py", line 553, in load_state_dict_from_url
download_url_to_file(url, cached_file, hash_prefix, progress=progress)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/torch/hub.py", line 419, in download_url_to_file
u = urlopen(req)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/urllib/request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/urllib/request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "examples/fit.py", line 61, in
run(args)
File "examples/fit.py", line 49, in run
pipeline=pipeline)
File "/home/myuser/autovideo/autovideo/utils/axolotl_utils.py", line 55, in fit
raise pipeline_result.error
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/d3m/runtime.py", line 1039, in _run
self._do_run()
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/d3m/runtime.py", line 1025, in _do_run
self._do_run_step(step)
File "/home/myuser/anaconda3/envs/autovideo/lib/python3.6/site-packages/d3m/runtime.py", line 1017, in _do_run_step
) from error
d3m.exceptions.StepFailedError: Step 5 for pipeline e61792eb-f54b-44ae-931c-f0f965c5e9de failed.

As you can see, I'm having problems with an Access Denied to the .pth files hosted at Amazon Cloud. Do you have any ideas on how to fix this?

I had a problem trying to load the function "extract_frames" with from autovideo import extract_frames
In another issue I saw that the function is deprecated. My question is, is the guide available in https://towardsdatascience.com/autovideo-an-automated-video-action-recognition-system-43198beff99d no longer valid?

What is the most current example?

I followed all the process of installation mentioned but when i ran python3 examples/fit.py i got following erro

r