VRAM needed about tracking-anything-with-deva HOT 3 CLOSED

Memory consumption comes from two parts: 1) a constant cost used by the image model and the propagation model (scale with chunk size and resolution), and 2) a linear cost w.r.t. to the number of objects (and resolution) in memory.

It seems like that (2) is giving you problems. Can you try reducing max_num_objects?
Not sure if you have already, but specifying --amp also helps a lot.

from tracking-anything-with-deva.

When using the max_num_objects argument some of the output frames end up black and I get the output:

Tracking-Anything-with-DEVA/deva/inference/segment_merging.py:118: UserWarning: Number of objects exceeded maximum (--max_num_objects); discarding new objects
  warnings.warn(
Tracking-Anything-with-DEVA/deva/inference/inference_core.py:69: RuntimeWarning: Empty object mask!
  warnings.warn('Empty object mask!', RuntimeWarning)
Tracking-Anything-with-DEVA/deva/inference/inference_core.py:95: RuntimeWarning: Trying to segment without any memory!
  warnings.warn('Trying to segment without any memory!', RuntimeWarning)

Can this situation be imporved?

I was able to make it work tho with --SAM_NUM_POINTS_PER_SIDE 32 --SAM_NUM_POINTS_PER_BATCH 32 and 2k input images.

from tracking-anything-with-deva.

This looks weird. Can you provide more info? Like when does it become blank? Can you post the output video?

from tracking-anything-with-deva.