image encoder image size is too big ,can reduce 1024 to 640/480 for acceleration? about nanosam HOT 2 OPEN

Hi @Mediumcore ,

This would require distilling a new model, for this you may be able to follow these steps.

Disclaimer: I haven't tested these, so let me know if you run into issues.

Step 1 - Register a new model

Register a new model, and ensure that it outputs features of shape 256x64x64 for your desired input resolution. For example, for an input of size 512x512, your model must have an output stride of 8.

You can register the model similar to here:

You could try registering a new model with a different stride, and setting the student size to a lower resolution (ie: 512x512).

Step 2 - Train the distilled model

Next, you'll need to train the model on unlabeled images. Follow the training instructions in the README, but set the "student_size" parameter to the desired size (512).

Step 3 - Evaluate the distilled model

Follow the evaluation instructions in the README to compare the accuracy for small / medium / large objects.

As a note: It's worth noting that distillation only applies to the image encoder. It's worth benchmarking the mask decoder to see if this is worth it, as the image encoding speed is approaching the decoding speed and may no longer be a performance bottleneck.

Hope this helps. If we end up releasing a lower resolution model I will update this thread, but we have no current plans at the moment.
John

from nanosam.

Understood，thank you very much for reply

from nanosam.