Comments (5)
Hello,
If I understand correctly, you can manually copy files to your server but you can't download from the internet. MultiMedEval will skip downloading from the internet if the datasets are already present on the specified path. Here are suggestions to get the files locally for Kaggle and for Huggingface.
Manual download for Kaggle:
The Kaggle API that I use simply downloads and unzips the dataset, you can do the same manually by going to the page of the dataset.
Once you have the zip file you can add it to your server and unzip it. You can then specify the correct path in the SetupParams
.
Manual download for Huggingface
For Huggingface, it is not as straightforward because the API does some processing on the data. I would suggest loading the datasets by running MultiMedEval locally (with the following code for example) and then uploading the folder where the datasets are stored (in this case data
) to your server. You can then specify the appropriate path on your server and HF will skip the network call.
from multimedeval import MultiMedEval, SetupParams
engine = MultiMedEval()
setupParams = SetupParams(MedQA_dir="data/")
tasksReady = engine.setup(setupParams=setupParams)
Note that this solution would also work for the Kaggle dataset.
I hope this helps.
from multimedeval.
Thank you very much for your timely reply!
But I also have a question.
Do you mean to initialize the local path here ?
which I screenshot from the .SetupParams.
from multimedeval.
Yes, in SetupParams
, you can specify a path to download each dataset. We chose to implement it this way because some of the datasets are very large and needed to be downloaded in a different location on our own system.
However, you can also specify the same path for all the datasets you want to use (e.g. data
) and it will work fine.
If you don't specify some of the paths, those datasets will be skipped during the setup.
from multimedeval.
Roger that! Thank you very much for your help!
Besides, I am working on multimodal medical treatment, and I hope to cooperate with you if I have the opportunity. A reliable, effective and universal evaluation framework is what I need most now!
Finally, congratulations on such an amazing job!
from multimedeval.
Thank you very much, I'm glad you appreciate it.
Don't hesitate to contact me via email :)
from multimedeval.
Related Issues (5)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from multimedeval.