This repo holds codes of the paper: AdaBrowse: Adaptive Video Browser for Efficient Continuous Sign Language Recognition.(ACMMM 2023) [paper]

This repo is based on VAC (ICCV 2021). Many thanks for their great work!

• This project is implemented in Pytorch (>1.8). Thus please install Pytorch first.

• ctcdecode==0.4 [parlance/ctcdecode]，for beam search decode.

• sclite [kaldi-asr/kaldi], install kaldi tool to get sclite for evaluation.

• SeanNaren/warp-ctc for ctc supervision.

We now implement our AdaBrowse with three resolution candidates: {96×96, 160×160, 224×224}, and three subsequence lengths: {1/4, 1/2, 1.0}.

You can choose any one of following datasets to verify the effectiveness of AdaBrowse.

1. Download the RWTH-PHOENIX-Weather 2014 Dataset [download link]. Our experiments based on phoenix-2014.v3.tar.gz.

1. Download the RWTH-PHOENIX-Weather 2014 Dataset [download link]

1. Request the CSL Dataset from this website [download link]

1. Request the CSL-Daily Dataset from this website [download link]

Due to some practical reasons for system deployment, we only provide the weights of stage one and now don't release the weights of stage two for AdaBrowse. One can train the model from stage one to verify the effectiveness of AdaBrowse.

First, follow the instructions of Stage_one to prepare the weights for resolutions of {96×96, 160×160, 224×224}, or directly use the weights provided by us.

Second, follow the instructions of Stage_two to train AdaBrowse.