Git Product home page Git Product logo

deepsleepnet's Introduction

DeepSleepNet

A deep learning model for automatic sleep stage scoring based on raw, single-channel EEG.

We have published a more efficient deep learning model, named TinySleepNet, which is much smaller and can achieve a better scoring performance.

TinySleepNet: An Efficient Deep Learning Model for Sleep Stage Scoring based on Raw Single-Channel EEG by Akara Supratak and Yike Guo from The Faculty of ICT, Mahidol University and Imperial College London respectively. [paper][github]

Code for the model in the paper DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG by Akara Supratak, Hao Dong, Chao Wu, Yike Guo from Data Science Institute, Imperial College London.

This work has been accepted for publication in IEEE Transactions on Neural Systems and Rehabilitation Engineering.

You can also find our accepted version before the publication in arXiv.

The architecture of DeepSleepNet: DeepSleepNet Note: Fs is the sampling rate of the input EEG signals

This figure illustrates one interpretable LSTM cell from the model, which learn to keep track when each subject is awake (i.e., in W stage): Sleep Onset Cell

Environment

The following setup has been used to reproduce this work:

  • Ubuntu 18.04 / Windows 10 1903 x64
  • CUDA toolkit 10.0 and CuDNN v7.6.4
  • Python 3.5.4 x64
  • tensorflow-gpu (1.15.2)
  • matplotlib (1.5.3)
  • scikit-learn (0.19.1)
  • scipy (1.4.1)
  • numpy (1.18.2)
  • pandas (0.25.3)
  • mne (0.20.0)
  • tensorlayer (optional)
  • MongoDB (optional)
  • eAE (optional)

Prepare dataset

We evaluated our DeepSleepNet with MASS and Sleep-EDF dataset.

For the MASS dataset, you have to request for a permission to access their dataset. For the Sleep-EDF dataset, you can run the following scripts to download SC subjects.

cd data
chmod +x download_physionet.sh
./download_physionet.sh

Then run the following script to extract specified EEG channels and their corresponding sleep stages.

python prepare_physionet.py --data_dir data --output_dir data/eeg_fpz_cz --select_ch 'EEG Fpz-Cz'
python prepare_physionet.py --data_dir data --output_dir data/eeg_pz_oz --select_ch 'EEG Pz-Oz'

Training a model

Run this script to train a DeepSleepNet model for the first fold of the 20-fold cross-validation.

python train.py --data_dir data/eeg_fpz_cz --output_dir output --n_folds 20 --fold_idx 0 --pretrain_epochs 100 --finetune_epochs 200 --resume False

You need to train a DeepSleep model for every fold (i.e., fold_idx=0...19) before you can evaluate the performance. You can use the following script to run batch training

chmod +x batch_train.sh
./batch_train.sh data/eeg_fpz_cz/ output 20 0 19 0

Scoring sleep stages

Run this script to determine the sleep stages for the withheld subject for each cross-validation fold.

python predict.py --data_dir data/eeg_fpz_cz --model_dir output --output_dir output

The output will be stored in numpy files.

Get a summary

Run this script to show a summary of the performance of our DeepSleepNet compared with the state-of-the-art hand-engineering approaches. The performance metrics are overall accuracy, per-class F1-score, and macro F1-score.

python summary.py --data_dir output

Submit the job to the eAE cluster equipped with TensorLayer

  1. Setup an eAE cluster (follows the instruction in this link)
  2. Setup a MongoDB
  3. Change location of MongoDB in deepsleep/trainer.py
  4. Modify submit_eAE.py
  5. Run python submit_eAE.py

ToDo

  • Release a version that does not depend on MongoDB and Tensorlayer (easier to install, but could take longer time for training).

Citation

If you find this useful, please cite our work as follows:

@article{Supratak2017,
    title = {DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG},
    author = {Supratak, Akara and Dong, Hao and Wu, Chao and Guo, Yike},
    journal = {IEEE Transactions on Neural Systems and Rehabilitation Engineering},
    year = {2017},
    month = {Nov},
    volume = {25}, 
    number = {11}, 
    pages = {1998-2008}, 
    doi = {10.1109/TNSRE.2017.2721116}, 
    ISSN = {1534-4320}, 
}

Licence

  • For academic and non-commercial use only
  • Apache License 2.0

deepsleepnet's People

Contributors

akaraspt avatar genaris avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

deepsleepnet's Issues

UnicodeDecodeError: 'gbk' codec can't decode byte 0xe4 in position 2050: illegal multibyte sequence

I get the following when I run the code:

Traceback (most recent call last):
<_io.TextIOWrapper name='.\data\physionet_sleep\SC4001E0-PSG.edf' mode='r' encoding='cp936'>
File "D:/report/deepsleepnet-master/prepare_physionet.py", line 229, in
main()
File "D:/report/deepsleepnet-master/prepare_physionet.py", line 112, in main
reader_raw.read_header()
File "D:\report\deepsleepnet-master\dhedfreader.py", line 90, in read_header
self.header = h = edf_header(self.file)
File "D:\report\deepsleepnet-master\dhedfreader.py", line 46, in edf_header
assert f.read(8) == '0 '
UnicodeDecodeError: 'gbk' codec can't decode byte 0xe4 in position 2050: illegal multibyte sequence

Resource exhausted

Hi Mr.Akaraspt
I met some error and I don't know how to solve.
Emmm log is as fallow:

Stats:
Limit: 1836384256
InUse: 1819745536
MaxInUse: 1819745536
NumAllocs: 5133326
MaxAllocSize: 628932608
W tensorflow/core/common_runtime/bfc_allocator.cc:274] ****************************************************************************************************
W tensorflow/core/common_runtime/bfc_allocator.cc:275] Ran out of memory trying to allocate 7.69MiB. See logs for memory state.

I use GTX1050 to ran it.And my memory is 2G.Maybe it's small.
But I'm sorry to bother you. And can you help me solve this problem?Thank you for reading this.: ) 

loss_value is NaN while training

Hi. During training the error occurs in _run_epoch of module trainer.py: "Model diverged with loss = NaN". What could it be due to?

raw.to_data_frame() error

Hello, I got the following error on PC in Windows 10.

python prepare_physionet.py --data_dir data --output_dir data/eeg_fpz_cz --select_ch "EEG Fpz-Cz"
Extracting EDF parameters from data\SC4001E0-PSG.edf...
EDF file detected
Setting channel info structure...
Creating raw.info structure...
Reading 0 ... 7949999 = 0.000 ... 79499.990 secs...
Traceback (most recent call last):
File "prepare_physionet.py", line 218, in
main()
File "prepare_physionet.py", line 95, in main
raw_ch_df = raw.to_data_frame(scale_time=100.0)[select_ch]
TypeError: to_data_frame() got an unexpected keyword argument 'scale_time'

Problems in prepare_physionet

/Users/apple/.conda/envs/EEG_classification/bin/python /Users/apple/PycharmProjects/deepsleepnet/prepare_physionet.py
Traceback (most recent call last):
File "/Users/apple/PycharmProjects/deepsleepnet/prepare_physionet.py", line 217, in
main()
File "/Users/apple/PycharmProjects/deepsleepnet/prepare_physionet.py", line 72, in main
os.makedirs(args.output_dir)
File "/Users/apple/.conda/envs/EEG_classification/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Users/apple/.conda/envs/EEG_classification/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Users/apple/.conda/envs/EEG_classification/lib/python3.7/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/data'

Process finished with exit code 1

Here is the problem I met. I tried many ways like sodu but it does not work.

download_physionet.sh missing

Hi, I think the following script is missing in your repo 'download_physionet.sh'

Could you add it ? Thanks in advance

bidirectional_dynamic_rnn error

I changed bidirectional_rnn in model.py to bidirectional_dynamic_rnn as otherwise it does not recognize it. After that I got the following error: ValueError: Shape (3072, 10) must have rank at least 3. Do you know what I should change? I have been stuck here for a while.

More details:
Finish pre-training
Traceback (most recent call last):
File "train.py", line 95, in
tf.app.run()
File "C:\Users\Wen\AppData\Local\conda\conda\envs\tensorflow\lib\site-packages\tensorflow\python\platform\app.py", line 126, in run
sys.exit(main(argv))
File "train.py", line 90, in main
n_epochs=FLAGS.finetune_epochs
File "train.py", line 72, in finetune
resume=FLAGS.resume
File "C:\Users\Wen\CNNs\deepsleepnet-master\deepsleep\trainer.py", line 595, in finetune
train_net.init_ops()
File "C:\Users\Wen\CNNs\deepsleepnet-master\deepsleep\model.py", line 422, in init_ops
network = self.build_model(input_var=self.input_var)
File "C:\Users\Wen\CNNs\deepsleepnet-master\deepsleep\model.py", line 372, in build_model
initial_state_bw=self.bw_initial_state
File "C:\Users\Wen\AppData\Local\conda\conda\envs\tensorflow\lib\site-packages\tensorflow\python\ops\rnn.py", line 412, in bidirectional_dynamic_rnn
time_major=time_major, scope=fw_scope)
File "C:\Users\Wen\AppData\Local\conda\conda\envs\tensorflow\lib\site-packages\tensorflow\python\ops\rnn.py", line 627, in dynamic_rnn
dtype=dtype)
File "C:\Users\Wen\AppData\Local\conda\conda\envs\tensorflow\lib\site-packages\tensorflow\python\ops\rnn.py", line 690, in dynamic_rnn_loop
for input
in flat_input)
File "C:\Users\Wen\AppData\Local\conda\conda\envs\tensorflow\lib\site-packages\tensorflow\python\ops\rnn.py", line 690, in
for input
in flat_input)
File "C:\Users\Wen\AppData\Local\conda\conda\envs\tensorflow\lib\site-packages\tensorflow\python\framework\tensor_shape.py", line 762, in with_rank_at_least
raise ValueError("Shape %s must have rank at least %d" % (self, rank))
ValueError: Shape (3072, 10) must have rank at least 3

trainer.py

Dear Professor,
There is an error on line 674 of the trainer.py: tmp_tensor = tf.compat.v1.get_default_graph().get_tensor_by_name(k)
When I debug, I find that the shape of tmp_tensor is empty(),it makes the whole pre-training value unable to be loaded.
I am looking forward to your reply as soon as possible

ZeroDivisionError: float modulo

Hi, Akaraspt. Thanks for sharing your script.

I got this error while running the following script.
python prepare_physionet.py --data_dir data --output_dir data/eeg_fpz_cz --select_ch 'EEG Fpz-Cz'.
Error info:
Traceback (most recent call last):
File "", line 1, in
File "/home/irving/anaconda3/envs/py27/lib/python2.7/site-packages/numpy/lib/shape_base.py", line 498, in split
if N % sections:
ZeroDivisionError: float modulo

As you saw, the error was self-understanding. After debugging, I have traced the reason for this error is happening from line 168 of prepare_physionet.py, which the variable n_trims was assigned to 0. I was understanding the previous code's function is finding useful data with labels. However, I am still confused about this error. Do you have any idea the reason for this part? I am really appreciative of it.

Best wishes,
Irving

Model diverged while predicting

Hello. After successfull training, I'm trying to predict stages using the command given (python predict.py --data_dir data/eeg_fpz_cz --model_dir output --output_dir output) using my directories of course. But it returns the following error:

Traceback (most recent call last):
  File "predict.py", line 705, in <module>
    tf.compat.v1.app.run()
  File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow_core\python\platform\app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\absl\app.py", line 312, in run
    _run_main(main, args)
  File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\absl\app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "predict.py", line 700, in main
    n_subjects_per_fold=n_subjects_per_fold
  File "predict.py", line 651, in predict
    subject_idx=subject_idx
  File "predict.py", line 561, in custom_run_epoch
    "Model diverged with loss = NaN"
AssertionError: Model diverged with loss = NaN

It seems like your model is trying to perform some more finetuning or something but I just want to use the saved trained model, forward pass and get the predicted labels. Is it possible in this simple way?

If my collection point changes, does your code work?

hi ,I come back again.:)
I'm sorry to bother you . I just want to know can I use your code if my collection point is changed ?
Just like your DS is fpz-cz I will change it to fp1-f4 . Does your code can work?
Thanks for your reading .:)
And I also have a question . I'm too lazy to try .Can you tell me if I use fpz-cz to train model but I use Pz-Oz'data to predict ?I guess that's not going to work, right ?

Error when extracting data

hey, i just have an assignment from my class about deeplearning, i choose your topic for my assignment. I have done the way in the readme file, but there is an error when extracting the EEG data.
I use google colab for simulation your code.

!python2 drive/deepsleepnet-master/prepare_physionet.py --data_dir drive/deepsleepnet-master/data --output_dir drive/deepsleepnet-master/data/eeg_fpz_cz --select_ch 'EEG Fpz-Cz' !python2 drive/deepsleepnet-master/prepare_physionet.py --data_dir drive/deepsleepnet-master/data --output_dir drive/deepsleepnet-master/data/eeg_pz_oz --select_ch 'EEG Pz-Oz'

the error is
Extracting EDF parameters from drive/deepsleepnet-master/data/SC4001E0-PSG.edf...
EDF file detected
Setting channel info structure...
Creating raw.info structure...
Reading 0 ... 7949999 = 0.000 ... 79499.990 secs...
Traceback (most recent call last):
File "drive/deepsleepnet-master/prepare_physionet.py", line 217, in
main()
File "drive/deepsleepnet-master/prepare_physionet.py", line 95, in main
raw_ch_df = raw.to_data_frame(scale_time=100)[select_ch]
TypeError: to_data_frame() got an unexpected keyword argument 'scale_time'
Extracting EDF parameters from drive/deepsleepnet-master/data/SC4001E0-PSG.edf...
EDF file detected
Setting channel info structure...
Creating raw.info structure...
Reading 0 ... 7949999 = 0.000 ... 79499.990 secs...
Traceback (most recent call last):
File "drive/deepsleepnet-master/prepare_physionet.py", line 217, in
main()
File "drive/deepsleepnet-master/prepare_physionet.py", line 95, in main
raw_ch_df = raw.to_data_frame(scale_time=100)[select_ch]
TypeError: to_data_frame() got an unexpected keyword argument 'scale_time

when i removed the scale time, the code can working but another error like this happen.
..........................
Include onset:50550.0, duration:480.0, label:4 (Sleep stage R)
Include onset:51030.0, duration:33030.0, label:0 (Sleep stage W)
Remove onset:84060.0, duration:2340.0, label:5 (Sleep stage ?)
before remove unwanted: (675000,)
after remove unwanted: (675000,)
before intersect label: (675000,)
after intersect label: (675000,)
before remove extra labels: (675000,), (2802,)
after remove extra labels: (0,), (0,)
Traceback (most recent call last):
File "drive/deepsleepnet-master/prepare_physionet.py", line 217, in
main()
File "drive/deepsleepnet-master/prepare_physionet.py", line 183, in main
x = np.asarray(np.split(raw_ch, n_epochs)).astype(np.float32)
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/shape_base.py", line 557, in split
if N % sections:
ZeroDivisionError: float modulo

can you explain why?
sorry im new in AI or deeplearning topics :)
thank you.

can I use data which fs=512?

I'm sorry to bother you again. As you can see can I use fs=512 data for direct staging? If I use your code and don't make any changes.Thanks for your reading in advance.: )

How to install tensorflow-gpu (0.12.1) in python 2.7?

Hi akaraspt.I have some promble.Can you help me?My problem is error when I install tensorflow-gpu (0.12.1) in python 2.7.13 Error is as follow.

Could not find a version that satisfies the requirement tensorflow==0.12.1 (from versions: )
No matching distribution found for tensorflow==0.12.1
My command is pip install --upgrade tensorflow==0.12.1 .And tensorflow-gpu (0.12.1) is also error.

I can't understand why you can use it? Did you installed it by pip?
And I hava another question.Is CUDA toolkit 8.0 and CuDNN v5 necessary?

Train result differently when run from train_bash.sh with running directly form the train.py

when run the train.py directly
the train result in fold0 (for example) directory tree is
--fold0

--deepfeaturenet
--train_summary (fold)
--perf_fold0.npz
--checkpoint
--params_fold0.npz
--perf_fold0.npz
.........and so on
image

--deepsleepnet
--train_summary (fold)
--perf_fold0.npz
--checkpoint
--params_fold0.npz
--perf_fold0.npz
.........and so on

when use the argument:./batch_train.sh data/eeg_fpz_cz/ output 20 0 19 0
the train result in fold0 directory tree is
--fold0
image
--deepfeaturenet (only has one fold)
--train_summary
--perf_fold0.npz

The difference is when the bash files result lack deepsleepnet fold and perf_fold0.npz
I just wonder why the result is different when use bash files. Could you help me?

Problem in prepare_physionet.py

Hello akaraspt,

Thank you for your contribution. I got the following error when trying to run the code:

Traceback (most recent call last):
File "prepare_physionet.py", line 15, in
from mne import Epochs, pick_types, find_events
File "L:\Programs\anaconda3\envs\env_py2\lib\site-packages\mne_init_.py", line 22, in
from .utils import (set_log_level, set_log_file, verbose, set_config,
File "L:\Programs\anaconda3\envs\env_py2\lib\site-packages\mne\utils_init_.py", line 23, in
from .fetching import _fetch_file, _url_to_local_path
File "L:\Programs\anaconda3\envs\env_py2\lib\site-packages\mne\utils\fetching.py", line 11, in
from urllib import parse, request
ImportError: cannot import name parse

It seems the dependency library is out of date? Is there any way to counter the problem?
Thank you so much.

Value error in optimize.py of the deepsleep folder after complete pre-training.

Hi Akara,
I successfully updated the full code of deepsleepnet to tensorflow 1.5. There happens to be an error in optimization after the completion of convolutional pre-training.
To exactly point out where the error is, I printed the optimizing parameters as shown in the screenshot attached. The list train_vars2 happens to be empty which is why the opt.applygradients(gvs) is giving us ValueError: No variables provided. in the second run of that loop.
optimize

Here is the traceback.

Traceback (most recent call last):
File "train.py", line 95, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 124, in run
_sys.exit(main(argv))
File "train.py", line 90, in main
n_epochs=FLAGS.finetune_epochs
File "train.py", line 72, in finetune
resume=FLAGS.resume
File "/home/DeepSleepNet/deepsleepnet/deepsleep/trainer.py", line 626, in finetune
clip_value=10.0
File "/home/DeepSleepNet/deepsleepnet/deepsleep/optimize.py", line 60, in adam_clipping_list_lr
apply_gradient_op = opt.apply_gradients(gvs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 493, in apply_gradients
raise ValueError("No variables provided.")
ValueError: No variables provided.

Please help me out in resolving this error.

Best Regards,
Abhay.

A wondering detail in your repository at the step of preparing dataset.

hi

when I prepared the dataset by your instruction, but something confused me.

cd data chmod +x download_physionet.sh ./download_physionet.sh

error info:

--2018-07-25 06:48:52-- https://www.physionet.oirg/physiobank/database/sleep-edfx/SC4001E0-PSG.edf Resolving www.physionet.org (www.physionet.org)... 128.30.30.88 Connecting to www.physionet.org (www.physionet.org)|128.30.30.88|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2018-07-25 06:49:07 ERROR 404: Not Found.

I checked the file 'download_physionet.sh'.

wang@wang:~/github/deepsleepnet-master/data$ head download_physionet.sh wget https://www.physionet.org/physiobank/database/sleep-edfx/SC4001E0-PSG.edf wget https://www.physionet.org/physiobank/database/sleep-edfx/SC4001E0-PSG.edf.hyp wget https://www.physionet.org/physiobank/database/sleep-edfx/SC4001EC-Hypnogram.edf

I open the first link : https://www.physionet.osrg/physiobank/database/sleep-edfx/SC4001E0-PSG.edf
It cannot be found, result as below:

The requested URL /physiobank/database/sleep-edfx/SC4001E0-PSG.edf was not found on this server.

So I just compared "the error link" with "the open source link":

No.1: wget https://www.physionet.org/physiobank/database/sleep-edfx/sleep-cassette/SC4001E0-PSG.edf
No.2: wget https://www.physionet.org/physiobank/database/sleep-edfx/SC4001E0-PSG.edf

No.1 worked, because I can download it matually even but No.2 can't.

So, my question is that if the link should be refreshed? and how about modifing your repository, refresh your file 'download_physionet.sh', into:

No.1: wget https://www.physionet.org/physiobank/database/sleep-edfx/sleep-cassette/xxx.edf
No.2: wget https://www.physionet.org/physiobank/database/sleep-edfx/xxx.edf

Just add "sleep-cassette/" after "sleep-edfx/".

It would be grateful if it would be responsed.

Start pre-training

What can I do about the problems I ran into running the train. py file?
File "C:\Users\admin\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\core\shape_base.py", line 283, in vstack
return _nx.concatenate(arrs, 0)
File "<array_function internals>", line 6, in concatenate
ValueError: need at least one array to concatenate

Question on the code

I have a question on the code 'predict.py'.

Why did you use custom_bidirectional_rnn in the 'predict.py'? What is difference between the custom_bidirectional_rnn and the bidirectional_rnn that is used in train?

If you give some comments, it will be very helpful to me.

Thank you for reading my question.

dataset

Dear akaraspt:
Hello,I am working on sleep stages。I have made some improvement on your code .But i can't get
the MASSdataset.Can you give me some advice?
Best regards!
Jianji Zhang

error in reading edf files

Dear all,
an error occured while reading edf files as below:

dhedfreader.py", line 32, in annotation_to_list
return unicode(annotation, 'utf-8').split('\x14') if annotation else []
NameError: name 'unicode' is not defined

as I searched this error based on that the code is on python 2 whereas I use python 3.
please provide python 3 version of the code.
thanks

About the Fs

I noticed that your flow chart of paper , the size of convolution kernel is defined as Fs/2 , but the model.py 77 Line was written that the real convolution kernel is 50. The SAMPLING_RATE = 256 in sleep_stage.py. Is the actual sampling rate 100 or 256?
I will be looking forward to your reply. :)

Killed during prepare_physionet.py

Hi all, when I am running prepare_physionet.py, I meet the following error:

=======================================

Extracting edf Parameters from data/SC4042E0-PSG.edf...
EDF file detected
Setting channel info structure...
Created Raw.info structure...
Reading 0 ... 8375999 = 0.000 ... 83759.990 secs...
Ready.
Killed

I am using the exact environment as you do, plus Azure VM D8S_V3. Do you have the similar issue before? Thanks

bidirectional_rnn Error

Hi, the version of my tensorlfow is 1.2.0 . when I run deepsleepnet, there is a error:
File "/home/zjm/deepsleepnet/deepsleep/model.py", line 340, in build_model
outputs, fw_state, bw_state = tf.nn.bidirectional_rnn(
AttributeError: 'module' object has no attribute 'bidirectional_rnn'

when I change tf.nn.bidirectional_rnn into tf.nn.bidirectional_dynamic_rnn, the error is:
File "/home/zjm/deepsleepnet/deepsleep/model.py", line 395, in init_ops
network = self.build_model(input_var=self.input_var)
File "/home/zjm/deepsleepnet/deepsleep/model.py", line 347, in build_model
initial_state_bw=self.bw_initial_state
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/rnn.py", line 375, in bidirectional_dynamic_rnn
time_major=time_major, scope=fw_scope)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/rnn.py", line 574, in dynamic_rnn
dtype=dtype)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/rnn.py", line 637, in dynamic_rnn_loop
for input
in flat_input)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/rnn.py", line 637, in
for input_ in flat_input)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 649, in with_rank_at_least
raise ValueError("Shape %s must have rank at least %d" % (self, rank))
ValueError: Shape (3072, 10) must have rank at least 3.

How can I fix bidirectional_rnn Error ?
thanks

Running this code on Colab

Hi
I was just wondering that owing to limitations of my local machine, can i run this code on colab ? and see the results

Thanks

predict.py

HI,professor.When I run predict.py, it shows that there is no definition of _infer_state_dtype(). Where does this function come from

Problems to get hypnogram automatically scored by DeepSleepNet

Hi, @genaris @akaraspt. Thanks for sharing your script.

But now, I can't find hypnogram automatically scored by DeepSleepNet when I run the whole code. However, you showed it in your literature like this.
image

Did you add this drawing function to your code?Do you have any idea for this part? I am really appreciative of it.
Best wishes,

pymongo.errors.ServerSelectionTimeoutError: 146.169.33.34:27020: timed out

Hi, when i run train.py, there is an error, how can i connect the TensorDB ?
File "train.py", line 8, in
from deepsleep.trainer import DeepFeatureNetTrainer, DeepSleepNetTrainer
File "/home/zjm/deepsleepnet/deepsleep/trainer.py", line 24, in
db = TensorDB(ip='146.169.33.34', port=27020, db_name='DeepSleepNet', user_name='tensorlayer', password='Tensorlayer123', studyID='1')
File "/home/zjm/deepsleepnet/tensorlayer/db.py", line 89, in init
self.db.authenticate(user_name, password)
File "/usr/local/lib/python2.7/dist-packages/pymongo/database.py", line 1048, in authenticate
connect=True)
File "/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 500, in _cache_credentials
writable_preferred_server_selector)
File "/usr/local/lib/python2.7/dist-packages/pymongo/topology.py", line 214, in select_server
address))
File "/usr/local/lib/python2.7/dist-packages/pymongo/topology.py", line 189, in select_servers
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: 146.169.33.34:27020: timed out

how to extract many channel ?

Hi,
if we want to extract many channel, what should we do ?
python prepare_physionet.py --data_dir data --output_dir data/eeg_fpz_cz --select_ch 'EEG Fpz-Cz EOG horizontal' is wrong

issue with setting the scaling_time parameter in the to_data_frame()

when trying to set the scaling_time parameter in the to_data_frame(), the following error is encountered:

TypeError: to_data_frame() got an unexpected keyword argument 'scaling_time'

The documentation here https://mne.tools/stable/generated/mne.io.Raw.html#mne.io.Raw.to_data_frame does'nt mention the scaling_time parameter in the signature, but the github page mentions the parameter.

Is there a way of setting the scaling_time parameter?

Problems in Prepare_physionet

/Users/apple/.conda/envs/EEG_classification/bin/python /Users/apple/PycharmProjects/deepsleepnet/prepare_physionet.py
Traceback (most recent call last):
File "/Users/apple/PycharmProjects/deepsleepnet/prepare_physionet.py", line 217, in
main()
File "/Users/apple/PycharmProjects/deepsleepnet/prepare_physionet.py", line 72, in main
os.makedirs(args.output_dir)
File "/Users/apple/.conda/envs/EEG_classification/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Users/apple/.conda/envs/EEG_classification/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Users/apple/.conda/envs/EEG_classification/lib/python3.7/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/data'

Process finished with exit code 1

Here is the problem that I met and I tried many ways like sodu but it did not work.

Is ubuntu necessary?

Thank you so much for posting this repository. I have found it very helpful in further understanding deep learning. I have been trying to run your program. Is it possible to do so without downloading ubuntu?

Can't read headers

When I try to read the headers for the PSG file
I keep getting this error :
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 2110: character maps to <undefined>

Question on the trained model for Prediction

Hi, I am a researcher studing CNN and your code is very helpful to me, thank you!

However, I have one question.

If you give me a short comment, also it will very helpful to me.

When I reproduce the prediction accuracy, it was slightly lower (1.5 %) than that in the paper.

I think that the reason is the modification of original code by me to run the code in my computer.

Or, is any additional processing needed to reproduce the accuracy?

For example, the code saves the trained model after pre-determined epochs and uses the saved modle

for prediction.

However, the model can be saved in a certain epoch by monitoring the validation accuracy.

Is the accuracy in the paper calculated by this or other similar process?

Thank you for reading this.

Error in prepare_physionet

I have downloaded the EEG data using the script provided, when I run the command:
python prepare_physionet.py --data_dir data --output_dir data/eeg_fpz_cz --select_ch 'EEG Fpz-Cz' it returns an error: ZeroDivisionError: float modulo


I tried to debug, and figuired that raw.ch is equal to 0 at the end. I think its because when it trimmes the tail, it trimmes the whole thing.
Could you please help me understand what is wrong? thank you

How do I process individual EEG recordings?

Hi akaraspt. I got the model running all the way through summary.py. You've done awesome work. However, how do I run the model on my own data? I can convert my data to npz format but how do I feed it to the model?

Is there a pre-trained model in the files?

I am currently referring this project for one of my ML classes and I heard it takes about one week to train the model, and I am still working on getting the MASS dataset. Was the trained model generated and saved somewhere in the files or I have to train it every time? It would be awesome if there is a pre-trained model, then I can maybe skip the training part to save some time for the predicting part.

Thanks in advance!

Dependencies Details

Dear Akara,
Awesome work you have here.

Having gone through all the posted issue, really appreciate if you can add the detail version of the following dependencies.

mne
matplotlib
scikit-learn
scipy
%%%%%%%
Ubuntu 16.04
CUDA toolkit 8.0 and CuDNN v5
Python 2.7
tensorflow-gpu (0.12.1)
numpy (1.11.1)
pandas (0.18.1)

I notice many report were mne-related issue (e.g., here, here)

Thanks in advance

Error in download_physionet.sh and prepare_physionet.sh

Hi,

I get the following error if I write the command ./download_physionet.sh

./download_physionet.sh: line 1: wget: command not found
./download_physionet.sh: line 2: wget: command not found
...

and get also an error at the command python prepare_physionet.py --data_dir data --output_dir data/eeg_fpz_cz --select_ch 'EEG Fpz-Cz'

Traceback (most recent call last):
  File "prepare_physionet.py", line 13, in <module>
    import pandas as pd
ImportError: No module named pandas

What can be the Problem? Where is my mistake?

Thanks in advance.

Overfitting problem

Why is the accuracy high on the training set but extremely low on the test set and all the data is from sleep-EDF

deepsleepnet virtual environment

Hi Akara
I am looking forward to using your amazing toolbox to stage some sleep!
I was wondering, do you have a virtual version of your whole environment for download to use with the toolbox?
Big ask i know, just hoping.
Thank you
zita

predict.py

Hello @akaraspt

while running predict.py file after training the data. I am getting the following data . can you kindly guide me on this 👍

2

Thanks in advance

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.