Dear zihan,
Thank you for your impressive work!
I have been following your work and I have successfully run the code with the setting 'model_type="LViT"', which does not require a pretrained model. However, I am facing some challenges in understanding how to obtain the pretained model for the setting 'model_type="LViT"'.
As discussed in Section 2.1 of your instructions:
![image](https://private-user-images.githubusercontent.com/57900441/289254115-8233f05b-6d2a-4206-ba59-d3ec54fd68ea.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM1ODE1MTIsIm5iZiI6MTcyMzU4MTIxMiwicGF0aCI6Ii81NzkwMDQ0MS8yODkyNTQxMTUtODIzM2YwNWItNmQyYS00MjA2LWJhNTktZDNlYzU0ZmQ2OGVhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODEzVDIwMzMzMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWJmOTExNDRhYTZiMWQ2OGJlYzQ0NDQyYTg4NGVjYmY1M2UzNGQ0MDRmMjk1ZmI4OTkzZDRkZTkwMDZiNjA5NWUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.gUbxJYkKoKNonUz4IZokui5c7aCAFnUNSw3LhSldCxE)
It appears that a U-Net model might be a prerequistite for the LViT model. Could you please clarifiy if this is the case? If so, does this imply I need to train a U-Net model first before proceeding with 'LViT_pretrain'? Also, in such a scenario, should I change the model type and write the corresponding code here?
![image](https://private-user-images.githubusercontent.com/57900441/289255693-b83dd626-e7c7-48d9-83c0-c3c833223b98.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM1ODE1MTIsIm5iZiI6MTcyMzU4MTIxMiwicGF0aCI6Ii81NzkwMDQ0MS8yODkyNTU2OTMtYjgzZGQ2MjYtZTdjNy00OGQ5LTgzYzAtYzNjODMzMjIzYjk4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODEzVDIwMzMzMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTQ2ZmM0ZDQ4OGEwZDA5NzNhNTZlNDNkMTA5MjI3NzFhMTEyNzVmMzQxMzk1YWUzZmU1YThjYmU2NzIwMzcxODQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.K2M31da1XT1dSzJpD4rTdRudHLXjGNQn-VhZYgQ0XDY)
Furthermore, I am also curious about how to load the pretrained U-Net model once it is obtained. Is this U-Net model directly applicable to LViT_pretrain, or are there additional steps or modifications required?
Your guidance on these matters would be greatly appreciated, as it would greatly assist me in understanding and utilizing your work more effectively.
Thank you for your time and consideration. I am looking forward to your valuable insights.
Best regards,
Pengyu Zhao