yxxxb / voco-llama Goto Github PK

Your work is fascinating! Could you please tell me which files are the main modifications? Thank you!

Hi, authors. I wonder how to present the efficiency via inference time.

I get the following error when I follow the installation instructions:

$ pip install -e .

Obtaining file:///projects/vision/azhang42/experiments/VoCo-LLaMA
ERROR: file:///projects/vision/azhang42/experiments/VoCo-LLaMA does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

Does this project have the same requirements as llava?

Hi, thanks for your awesome work.

I am the author of Video-MME and would like to invite you to evaluate your VoCo framework on Video-MME, which might be better to show your advantages on long videos compared with short videos such as MSRVTT and ActivityNetQA.

By the way, we are also really interested in whether the compression would influence the temporal understanding, our benchmarks VITATECS (ECCV 24, https://github.com/lscpku/VITATECS) and TempCompass (https://huggingface.co/spaces/lyx97/TempCompass) are also welcome for your participation.

Hello! I followed the instructions to train VoCO-LLaMA, and I get the following loss curve.

Is it normal to have spikes where the loss drops to around 0.3 roughly every 20 steps? Thanks!

Wonderful job!
I've also been thinking about video token compression recently. Your idea is so interesting.
I want to know when checkpoint will be released.
At the same time, I also want to know the results on other benchmarks, such as mvbench, videomme.

Hi,

Really liked the paper + idea. Congrats!

Great job!
Orr

Hello! I am looking to train the model on 1 VoCo token. Therefore, I set voco_num to 1 in llava_arch.py and the number of inserted voco tokens to 1 in train_compress.py. I was wondering if I should use the same hyperparamters that were used to train with the default of 2 voco tokens. Thanks for any help in advance.

Dear authors,
first of all congrats for your idea and paper!!

I have a question about the code. I see here

thanks

Hello, Thank you for your work! I think it has a great novelty.

I have some questions about checkpoint!
Are there no checkpoints for VoCo-LLaMA?
I only have 1 GPU(L40), and just want to check inference or evaluation. But I cannot find voco-llama checkpoints in this repository.

if there is plan to upload the checkpoints, when would you upload them?

Thank you

Great Idea! I was wondering if the final model weights would be provided. Also, how long did training take?

Congratulations on a Great work ! I did not go over every line of code but could you please help to point out where did you implement the KL divergence training objective for your model in your code. Thank you!!!