🚀: We update video_chat by instruction tuning for image & video chatting now! Old version of video_chat moved to video_chat_with_chatGPT. We release instruction data at InternVideo

中文 README 及 中文交流群

Currently, Ask-Anything is a simple yet interesting tool for chatting with video. Our team is trying to build a smart and robust chatbot that can understand video.

We are also working on a updated version, stay tuned! ⭐️

• 2023/05/11 End-to-end VideoChat

  • VideoChat: Instruction tuning for image & video chatting.

• 2023/04/25 Watch videos longer than one minute with chatGPT

  • VideoChat LongVideo: Incorporating langchain and whisper into VideoChat.

• 2023/04/21 Chat with MOSS

  • VideoChat with MOSS: Explicit communication with MOSS.

• 2023/04/20: Chat with StableLM

  • VideoChat with StableLM: Explicit communication with StableLM.

• 2023/04/19: Code release & Online Demo

  • VideoChat with ChatGPT: Explicit communication with ChatGPT. Sensitive with time. demo is avaliable!
  • MiniGPT-4 for video: Implicit communication with Vicuna. Not sensitive with time. (Simple extension of MiniGPT-4, which will be improved in the future.)

• ChatGPT
• StableLM
• MiniGPT-4
• MOSS

If you find this project useful in your research, please consider cite:

@article{2023videochat,
  title={VideoChat: Chat-Centric Video Understanding},
  author={KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao},
  journal={arXiv preprint arXiv:2305.06355},
  year={2023}
}

Our team constantly studies general video understanding and long-term video reasoning:

• Strong video foundation model.
• Video-text dataset and video reasoning benchmark.
• Video-language system with LLMs.
• Artificial Intelligence Generated Content (AIGC) for Video.
• ...

We are hiring researchers, engineers and interns in General Vision Group, Shanghai AI Lab. If you are interested in working with us, please contact Yi Wang ([email protected]).