corentin-ryr / multimedeval Goto Github PK

Hello, thanks for your excellent work! I'd love to try your work.

However, I can't access kaggle or Huggingface from my server because of network problems. I can only download the dataset locally first.
How do I load the dataset locally instead of downloading it from the network? Do I need to process this dataset? For example, extract the answers in a certain format. Question equivalence?

If possible, can you give a process and method that can be referenced?
Thank you very much for your help!

Sorry to bother you again! But I have some questions about the format of the input and output data as follows:

Is this the image path? Read it as PIL.image(path)?
and

What is the json format of answers? Can you give me a specific example?
I am very sorry to bother you all the time, and I would really appreciate your reply.

Add ability to have stratified results when possible:

• diverse demographics
• types of diseases
• sources of medical images

Add ability to also take care of: fairness metrics such as balance, equality, and disparity. (Refer: https://arxiv.org/pdf/2310.09507.pdf)

Do you have any plans to evaluate more multimodal medical models? E.g. BiomedCLIP, PubMedCLIP, etc.?

Hello, thanks for your excellent work!

I'm currently running VQA_RAD and PATH-VQA.
Despite metrics have finished calculated, the progress bar fails to update.

Could you please advise on how to configure it for updates?