Running on macos / mps about tracking-anything-with-deva HOT 5 OPEN

I don't have a Mac device and cannot test it but I suppose you can replace all the .cuda() and cuda.amp calls with mps equivalent.

from tracking-anything-with-deva.

If you don't have a Mac that is a good reason 👍 .
I tried replacing the cuda specific things, also there are some datatypes that aren't supported on mps like int64... and some functions that fallback to cpu. I got as far as it running but with an error from MPSGraphUtilities.mm scrolling by. After 10 minutes, the progress bar showed 2 images had been completed, then it quit with an error, and no output images. It looks like it is using up more than the 64GB of unified memory, I'm not sure I can fix it further without understanding each part that's being used.

from tracking-anything-with-deva.

If you try the example video, i.e.,

python demo/demo_with_text.py --chunk_size 4 \
--img_path ./example/vipseg/images/12_1mWNahzcsAc \ 
--amp --temporal_setting semionline \
--size 480 \
--output ./example/output --prompt person.hat.horse

, it is quite unlikely that it uses more than a few GB of memory.

If it processed two frames without any output images, it sounds like the error occurred during consensus. You can test using the online mode (consensus disabled).

from tracking-anything-with-deva.

I was running the other one - demo_automatic.py
The demo_with_text has a different error from GroundingDINO, I'll have a look into that tomorrow.

from tracking-anything-with-deva.

Ref: hkchengrex/Cutie#14

from tracking-anything-with-deva.