Can't use the dataset :( about iclabel-dataset HOT 11 CLOSED

I have the same =(

from iclabel-dataset.

I'm sorry for having missed this for so long. The problem is that the automatic file download fails and then it can't find the file. I'll see if I can fix the problem. In the meantime, you can download the files manually.

The block of code containing file urls:

        self.base_url_download = 'https://labeling.ucsd.edu/download/'
        self.feature_train_zip_url = self.base_url_download + 'features.zip'
        self.feature_train_urls = [
            self.base_url_download + 'features_0D1D2D.mat',
            self.base_url_download + 'features_PSD_med_var_kurt.mat',
            self.base_url_download + 'features_AutoCorr.mat',
            self.base_url_download + 'features_ICAChanlocs.mat',
            self.base_url_download + 'features_MI.mat',
        ]
        self.label_train_urls = [
            self.base_url_download + 'ICLabels_experts.pkl',
            self.base_url_download + 'ICLabels_onlyluca.pkl',
        ]
        self.feature_test_url = self.base_url_download + 'features_testset_full.mat'
        self.label_test_url = self.base_url_download + 'ICLabels_test.pkl'
        self.db_url = self.base_url_download + 'anonymized_database.sqlite'
        self.cls_url = self.base_url_download + 'other_classifiers.mat'

from iclabel-dataset.

This is actually 2 different problems.

1. ICLabels_experts.pkl should be ICLabels_expert.pkl
2. Downloading features_0D1D2D.mat stops before the file is complete.

from iclabel-dataset.

Item (1) has been fixed, but I'm still having trouble with (2). My attempt at fixing it was to recreate the zip archive as a multi-disk zip where each file is no more than 1 GB. Unfortunately python's zipfile library does not support multi-disk zips. I'm looking into using libarchive instead but I've run out of time for now.

from iclabel-dataset.

I believe this is fixed. I confirmed the download now works but I can't actually load the dataset on my personal computer due to lack of RAM. If there are any problems, feel free to reopen this issue.

from iclabel-dataset.

@lucapton Thank you for sharing the valuable resource! I tried to download the data, but had a similar error:

Loading full dataset...
Traceback (most recent call last):
  File "e:\GoogleDriveBB\Program\ICLabel-Train\loading_data.py", line 4, in <module>
    icldata = icl.load_semi_supervised()
  File "e:\GoogleDriveBB\Program\ICLabel-Train\icldata.py", line 1231, in load_semi_supervised
    icl = self.load_data()
  File "e:\GoogleDriveBB\Program\ICLabel-Train\icldata.py", line 969, in load_data
    with h5py.File(join(self.datapath, 'features', 'features_0D1D2D.mat'), 'r') as f:
  File "C:\ProgramData\Anaconda3\lib\site-packages\h5py\_hl\files.py", line 406, in __init__
    fid = make_fid(name, mode, userblock_size,
  File "C:\ProgramData\Anaconda3\lib\site-packages\h5py\_hl\files.py", line 173, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py\h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = 'features\features_0D1D2D.mat', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

I also tried to download manually from the link https://labeling.ucsd.edu/download, but it seems that this page does not exist... I tried in different browsers and got the same page as below:

from iclabel-dataset.

@datalw
You need to call download_trainset_features in order to download the files that it says are missing (I've updated the readme to show show that). If you want to download them manually, you have to use the link for the exact files:
https://labeling.ucsd.edu/download/features_0D1D2D.mat
https://labeling.ucsd.edu/download/features_PSD_med_var_kurt.mat
https://labeling.ucsd.edu/download/features_AutoCorr.mat
https://labeling.ucsd.edu/download/features_ICAChanlocs.mat
https://labeling.ucsd.edu/download/features_MI.mat

What I find curious though is that if you don't have the files you should have hit an assertion error at line 967 of iclabel.py. Any idea why it didn't?

from iclabel-dataset.

@datalw
You need to call download_trainset_features in order to download the files that it says are missing (I've updated the readme to show show that). If you want to download them manually, you have to use the link for the exact files:
https://labeling.ucsd.edu/download/features_0D1D2D.mat
https://labeling.ucsd.edu/download/features_PSD_med_var_kurt.mat
https://labeling.ucsd.edu/download/features_AutoCorr.mat
https://labeling.ucsd.edu/download/features_ICAChanlocs.mat
https://labeling.ucsd.edu/download/features_MI.mat

What I find curious though is that if you don't have the files you should have hit an assertion error at line 967 of iclabel.py. Any idea why it didn't?

@lucapton Thanks a lot! Right now both ways work - downloading via the three-line codes and with the links above : )
I have checked why it did not work, here is what I have found:

As you see, in the line 1708 I printed data_type, it is a string instead of a list as shown in the terminal. That's why val takes only one letter t, which cannot find its compatible case in the if-elif cases ; )

from iclabel-dataset.

Thanks for finding that! Glad it works for you.

from iclabel-dataset.

So I looked into it and I should say that the code works as-is in Python 2.7 but not in 3.x which it appears you're using. I just want to provide a warning that I wrote this all in 2.7 and can't guarantee anything for python 3. I realize that was a poor choice on my part but it's just a fact not. That said, if you run into anymore issue, let me know and I'll try to fix them. I'm about to push a change that makes this specific piece of the code work for both.

from iclabel-dataset.

So I looked into it and I should say that the code works as-is in Python 2.7 but not in 3.x which it appears you're using. I just want to provide a warning that I wrote this all in 2.7 and can't guarantee anything for python 3. I realize that was a poor choice on my part but it's just a fact not. That said, if you run into anymore issue, let me know and I'll try to fix them. I'm about to push a change that makes this specific piece of the code work for both.

Thanks for the note! There was no big incompatibility, as I run the dataset codes. What I had to change were only a few lines of print.

from iclabel-dataset.