Now We have uploaded all the model and data to google drive and you can download to reproduce our result.
Also, we provide all the inference scores corresponding to each model in folder "inference_testphase" so that you can run inf_rgb.py and inf_rgbd.py to perform model fusion and get our result directly.
If you want to know more detail about train, please refer to Train_schedule.md
For Slowonly
Step 1: Installation for mmaction
cd to root path of our mmaction and perform the installation following the instruction on the mmaction
It is crucial to config the third_party (i.e. decord and dense_flow,etc.)
Note that we make some change in loading the data (i.e. detect the bounding box)
Step 2: Get the processed data for slowonly
Note that the data is almost 20G! We use the dense_flowmmaction to extract frames and optical flow from both RGB data and depth data
# cd to root path of slowfast
# 1. get slowonly_addvalset_addtestsetv1_detect_depth_epoch60.npy
bash inference_scripts/slowonly_detect_depth.sh
then we will get the inference score "slowonly_addvalset_addtestsetv1_detect_depth_epoch60.npy" and prediction result in folder "val_result"
Note that the inference score is used to perform the final model fusion
We take 1 hour with 3 GTX1080TI to perform this inference
# 2. get slowonly_addvalset_addtestsetv1_detect_RGB_epoch92.npy
bash inference_scripts/slowonly_detect_RGB.sh
# 3. get slowonly_addvalset_addtestsetv1_depth_epoch81.npy
bash inference_scripts/slowonly_depth.sh
# 4. get slowonly_addvelset_addtestsetv1_lr_0.01_cropratio0.08_epoch87.npy
bash inference_scripts/slowonly_RGB.sh
# 5. get slowonly_addvalset_addtestsetv1_input256_inference288_epoch98.npy
bash inference_scripts/slowonly_RGB_input256_inference288.sh
We provide these five inference score with slowonly model in folder "inference_testphase" for final model fusion
For TSM
Step 1: Download the model for TSM
cd to root path of our temproal-shift-module and download the model for TSM : TSM_model
# cd to root path of slowfast
# 1. get TSM_addvalset_addtestsetv1_RGB_finetune_Epoch10.npy
First,modify line 107 of ops/dataset_config.py to the absolute path to mmaction/data/AUTSL/test/rawframes_align
Then,modify line 5 of scripts_testphase/test_tsm_AUTSL_rgb_8f.sh to the absolute path to mmaction/data/AUTSL/test/test_RGB_pse93.csv
Then,run bash scripts_testphase/test_tsm_AUTSL_rgb_8f.sh
Finally,we get the inference score and prediction result in the folder "val_result"
# 2. get TSM_addvalset_addtestsetv1_RGBflow_finetune_Epoch10.npy
First,modify line 117 of ops/dataset_config.py to the absolute path to mmaction/data/AUTSL/test/optical_flow
Then,modify line 5 of scripts_testphase/test_tsm_AUTSL_rgb_8f.sh to the absolute path to mmaction/data/AUTSL/test/test_RGB_pse93.csv
Then,run bash scripts_testphase/test_tsm_AUTSL_rgbflow_8f.sh
Finally,we get the inference score and prediction result in the folder "val_result"
# 3. get TSM_addvalset_addtestsetv1_depth_flow_Epoch10
First,modify line 117 of ops/dataset_config.py to the absolute path to mmaction/data/AUTSL/test/optical_flow
Then,modify line 5 of scripts_testphase/test_tsm_AUTSL_rgb_8f.sh to the absolute path to mmaction/data/AUTSL/test/test_depth_pse93.csv
Then,run scripts_testphase/test_tsm_AUTSL_depthflow_8f.sh
Finally,we get the inference score and prediction result in the folder "val_result"
We provide these three inference score with TSM model in folder "inference_testphase" for final model fusion
For SlowFast
Step 1: Install SlowFast
Follow the install instruction on the github of the slowfast
.
Step 2: Get Human Segmentation Data of the Test Set