catalystneuro / ecogvis Goto Github PK

Time series visualizer and data processing tools for Electrocorticography (ECoG) signals stored in NWB files, for Python.

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

neuroscience electrocorticography gui python3 open-source nwb pynwb neurodata-without-borders python

ecogvis's Introduction

ecogVIS

Timeseries visualizer and data processing tools for Electrocorticography (ECoG) signals stored in NWB files, for Python.

A collaboration with the Chang Lab.

Installation

From PyPI

$ pip install ecogVIS

From source

First download the ecogVIS source code from this repo.

$ git clone https://github.com/catalystneuro/ecogVIS.git
$ cd ecogVIS

If you're installing into an active environment:

$ pip install -e . -r requirements.txt

You can also create a fresh environment with ecogVIS installed:

$ conda env create -f make_env.yml

Troubleshooting

If you're using macOS 11+ (e.g. Big Sur or Monterey), you may experience errors related to OpenGL. Using Python 3.8 should solve these, and you can use the file make_env_py38.yml to create the correct environment.
If you're using macOS 11+ (e.g. Big Sur or Monterey), you may also experience errors with PyQt freezing and not opening the GUI window. Try setting the following environment variable (which can be added to your ~/.bashrc or ~/.bash_profile):
```
$ export QT_MAC_WANTS_LAYER=1
```

Use

After activating the correct environment, ecogVIS can be started from the terminal:

$ ecogvis

You can also directly pass a file to be opened:

$ ecogvis --source 'filename.nwb'

ecogVIS can be imported and run from python scripts. If the file does not exist (or if you provide an empty string ''), you'll be prompted to choose a file from a dialog.

from ecogvis import main
import os

fpath = os.path.join('path_to', 'file.nwb')
main(fpath)

Features

ecogVIS makes it intuitive and simple to viualize and process ECoG signals. It currently features:

Navigation

Seamless visual navigation through long signals from large arrays of electrodes, by mouse-dragging visualization window, control buttons, value fields and keyboard keys.

Annotations

Add, delete, save and load annotations for meaningful comments anywhere in the visualization.

Intervals

Add, delete, save, load and create custom intervals types to mark specific points in time, with simple click-drag-release mouse movements.

Bad Channels

Mark and un-mark bad channels. Choices are saved in the electrodes group of the current NWB file.

Signal preprocessing

Preprocessing of raw voltage signals, including user-defined Downsampling, CAR and Notch filtering. The resulting processed signals are stored as an LFP object, in the processing group of the current NWB file.

Events detection

Automatic detection of events in audio recordings for Consonant-Vowel tasks. The audio data should be stored in the NWB file in the following way:

Speaker audio - As a TimeSeries object, named 'Speaker CV', in the stimulus group.
Microphone audio - As a TimeSeries object, named 'Microphone CV', in the acquisition group.

The resulting detected intervals, 'TimeIntervals_mic' and 'TimeIntervals_speaker', are saved as TimeIntervals objects in the intervals group of the current NWB file and can be used later for ERP analysis. A preview allows for testing of the detection parameters before running it for the whole duration of the audio signals.

High Gamma

Estimation of high gamma analytic amplitude, with the average of user-defined specific bands. The results are saved as a TimeSeries object, named 'high_gamma', in the processing group of the current or of a new NWB file.

Event-Related Potentials

Grid visualization of high gamma ERP calculated in reference to:

Stimulus (speaker) or response (microphone) time intervals
Onset or offset points

The grid items are coloured to mark specific cortical areas and can be rotated to correspond anatomically to them. Emphasis can be given to specific areas of interest and double-clicking an item allows for fast inspection of the single electrode's ERP in detail.

Periodograms

Grid visualization of Periodograms:

Raw and preprocessed data
FFT and Welch methods
Individual devices

Spectral Decomposition

Analytic signal amplitude estimation by Hilbert transform of user-defined frequency bands.

WARNING: This function will manipulate an array of size (nSamples, nChannels, nBands), which might be in the order of gigabytes and demand a large memory to operate and is likely to produce a large file.

Plus

Select electrodes from specific brain areas
Easy moving between block files for the same subject

ecogvis's People

Contributors

Stargazers

Watchers

Forkers

jgmakin bouchardlab stebick nki-ecog armin12 emilyps14 tianshizhilei98305 peternsteinmetz lauragwilliams yangxinfeng jersaal nikhil-garg jessierliu 5l1v3r1 eegkit

ecogvis's Issues

Quick move between block files

@jrliu95
I figured this could be an interesting option, to have a quick move between blocks of the same subject. This takes away space, though, that could possibly be taken by more important action buttons for you guys. Let me know what you think.

improve comments

Add/Del multiple Bad Channels

let user add/del multiple bad channels in one go

Unable to create link when trying to save preprocessed data?

In trying to preprocess a raw file, I'm getting this error when it tries to write to the nwb file. I've done processing on this block before (I made a copy of the original raw NWB file). Do you know what may be causing this? It was able to do downsampling, CAR, and notch filtering before this error came up.

RuntimeError                              Traceback (most recent call last)
~/ChangLab/projects/ecogVIS/ecogvis/functions/subDialogs.py in run(self)
    434                         mode=self.mode,
    435                         config=self.config,
--> 436                         new_file=self.new_fname)
    437 
    438 

~/ChangLab/projects/ecogVIS/ecogvis/signal_processing/processing_data.py in processing_data(path, subject, blocks, mode, config, new_file)
     26 
     27         if mode=='preprocess':
---> 28             preprocess_raw_data(block_path, config=config)
     29         elif mode=='decomposition':
     30             spectral_decomposition(block_path, bands_vals=config)

~/ChangLab/projects/ecogVIS/ecogvis/signal_processing/processing_data.py in preprocess_raw_data(block_path, config)
    298 
    299             # Write LFP to NWB file
--> 300             io.write(nwb)
    301             print('LFP saved in '+block_path)
    302 

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    386                         raise_from(ExceptionType(msg), None)
    387 
--> 388                 return func(self, **parsed['args'])
    389         else:
    390             def func_call(*args, **kwargs):

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py in write(self, **kwargs)
    217 
    218         cache_spec = popargs('cache_spec', kwargs)
--> 219         call_docval_func(super(HDF5IO, self).write, kwargs)
    220         if cache_spec:
    221             ref = self.__file.attrs.get(SPEC_LOC_ATTR)

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in call_docval_func(func, kwargs)
    279 def call_docval_func(func, kwargs):
    280     fargs, fkwargs = fmt_docval_args(func, kwargs)
--> 281     return func(*fargs, **fkwargs)
    282 
    283 

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    386                         raise_from(ExceptionType(msg), None)
    387 
--> 388                 return func(self, **parsed['args'])
    389         else:
    390             def func_call(*args, **kwargs):

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/io.py in write(self, **kwargs)
     40         container = popargs('container', kwargs)
     41         f_builder = self.__manager.build(container, source=self.__source)
---> 42         self.write_builder(f_builder, **kwargs)
     43 
     44     @abstractmethod

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    386                         raise_from(ExceptionType(msg), None)
    387 
--> 388                 return func(self, **parsed['args'])
    389         else:
    390             def func_call(*args, **kwargs):

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py in write_builder(self, **kwargs)
    444         f_builder, link_data = getargs('builder', 'link_data', kwargs)
    445         for name, gbldr in f_builder.groups.items():
--> 446             self.write_group(self.__file, gbldr)
    447         for name, dbldr in f_builder.datasets.items():
    448             self.write_dataset(self.__file, dbldr, link_data)

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    386                         raise_from(ExceptionType(msg), None)
    387 
--> 388                 return func(self, **parsed['args'])
    389         else:
    390             def func_call(*args, **kwargs):

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py in write_group(self, **kwargs)
    597         if links:
    598             for link_name, sub_builder in links.items():
--> 599                 self.write_link(group, sub_builder)
    600         attributes = builder.attributes
    601         self.set_attributes(group, attributes)

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    386                         raise_from(ExceptionType(msg), None)
    387 
--> 388                 return func(self, **parsed['args'])
    389         else:
    390             def func_call(*args, **kwargs):

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py in write_link(self, **kwargs)
    634             msg = 'cannot create external link to %s' % path
    635             raise ValueError(msg)
--> 636         parent[name] = link_obj
    637         builder.written = True
    638         return link_obj

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/h5py/_hl/group.py in __setitem__(self, name, obj)
    390             with phil:
    391                 self.id.links.create_external(name, fn, self._e(obj.path),
--> 392                                               lcpl=lcpl, lapl=self._lapl)
    393 
    394     @with_phil

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5l.pyx in h5py.h5l.LinkProxy.create_external()

RuntimeError: Unable to create link (name already exists)

Preprocess button with options

In the "Preprocess" button can you add a box to be able to specify what number channel CAR to use?

Additionally, if the preprocessed data already exists, is there a way to display in this box what those preprocessing options had been set to? For example, printing out there that the preprocessed data exists and then saying what sampling rate, notch filter, and CAR were used? I guess these would have to be written into the file somewhere. Let me know what you think

load HTK files

The ecogVIS should be able to load data from HTK, and save into NWB, which will ease the transition to ecogVIS and NWB.

This will require us to load data without all of the important meta-data. We will also build a feature enabling users to enter in missing meta-data via a GUI or a yaml file.
We should be able to elegantly handle missing meta-data when doing visualization. For instance, the grid ERP viewer should check if there is anatomical information in the electrodes table and if not it should display the grid without an anatomical color scheme.
Handle missing electrodes with “null” electrode value

file not closed on GUI exit

plotERP should elegantly handle when electrode meta-data is missing

import preprocessing code from external library

Add Bad Channel -> cancel results in error

High Gamma - reorganize functions

Preprocess data: this will add the LFP if not already in the file. It will prompt a box to come up with the option to change some of the parameters (listed below), but shows the defaults:

CAR: whether or not to do a CAR, and how many channels
Which notch filters to apply (default 60 Hz and harmonics)
What frequency to downsample to. The default is to downsample the preprocessed data to 400 Hz sampling rate
If the person doesn't want to change the defaults, should just press "Ok" or some affirmative button.

Calculate high gamma: this will run the spectral decomposition and calculate the high gamma. It will prompt a box to come up with the option to change some of the parameters:

What frequency of high gamma (default 70-150Hz, but some people may want to go up to 200Hz) to use.
Save High gamma on the current file or new one

Way to make the cp_objs dictionary customizable?

In ecogvis.signal_processing.processing_data.make_new_nwb, the objectives dictionary is predefined. I don't have any epochs information in the files I've been making, so when it tries to copy it over, it obviously gets an error. Is there some way to make this customizable in the kwargs of the function? I call this function directly in a script to batch preprocess files without the GUI. Let me know if there's some other way I can get around this.

https://github.com/ben-dichter-consulting/ecogVIS/blob/297f199f1259c2e62784a13db1aeba534849a9cc/ecogvis/signal_processing/processing_data.py#L56

This is a short excerpt from my code to show you how I'm calling the functions.

if export_high_gamma:
    # If exporting high gamma to a new file, create that new file.
    new_file = os.path.join(general_path, '{}_B{}_hg.nwb'.format(subject, block))
    make_new_nwb(old_file=block_path, new_file=new_file)
    print('Made new NWB file for high gamma data.', flush=True)
else:
    # Otherwise, provide an empty string to save the high gamma to
    # the existing file.
    new_file = ''

if not file_hg:
    # Compute high gamma data. Save to separate file if specified.
    high_gamma_estimation(block_path,
                                  bands_vals=config['hg_bands_vals'],
                                  new_file=new_file)

ecogVIS main plot, axes ticks not properly aligned

x- and y-axis ticks do not appear to scale/align appropriately

Running ecogvis from Jupyter notebook, on a Windows 10 machine

Bad channel color

The darker colors for the general channels are great, but when a bad channel is added, the red is quite faint. Can we make this darker as well?

Cannot save bad channels

Originally had the issue that I couldn't open a nwb file without a "bad" channel column, but it looks like that has been addressed in #85 , so I installed the newest version of ecogVIS using the "setup.py" file. Is there a way of saving bad channels without having an existing bad channels column?

Update to ndx_ecog

I think in the merge, the updates to ndx_ecog somehow got lost. The only changes needed are in the requirements.txt, setup.py, make_env.yml, ecogvis/functions/nwb_copy_file.py, and ecogvis/functions/subFunctions.py.

Since nothing functionally should've changed, shouldn't be difficult to update.

Audio Event Detection GUI doesn't reflect jgmakin/ecogVIS updates

PR #76 was supposed to merge in changes from the changlab fork, but (at least) the Audio Event Detection GUI does not seem to have the design updates.

Cannot run ecogvis from Jupyter notebook

Adding username to intervals/annotation/badchannels

I see that there's now a username field in the CSV file that's saved, but where would that be specified? Perhaps just a dialog box when asking to save to an external file?

Would it be simple to add a user to the badchannels selection as well?

Ability to add meta-data that would be missing from other forms of data

initial dialog

The initial dialog is confusing. It asks for the session name and then says "e.g. your_name". Why are you asking for the session name? Can't you get that from the file? And it looks like you are asking for my name (Ben).

Update readme?

Super small thing, but would you mind updating the readme to reflect the new repo ownership (for cloning/installation instructions)? I've given out the link to people in lab so if they install it (which may or may not happen), just want to make sure they don't install the wrong version.

https://github.com/bendichter/ecogVIS.git >> https://github.com/ben-dichter-consulting/ecogVIS.git

ERP plot - saving the image

Clicking the "Save image" button didn't seem to do anything, can you have it open up a dialog box where the user can select where to save it to?

ERP plot - selecting channels

Didn't seem to do anything when I selected a few channels. Can you make it emphasize the selected channels in some way?

ERP - Change grid orientation

User control buttons for:

90 degrees clockwise
90 degrees counterclockwise
Transpose

rebrand

I'd like to rebrand this to be more general, to be used for extracellular electrophysiology more broadly

rename. Any ideas?
interface with SpikeInterface to provide access to spike sorters. Maybe via spikely

Electrodes as attr in high gamma TimeSeries

When trying to plot the ERPs, I got the following attribute error:

AttributeError                            Traceback (most recent call last)
~/ChangLab/projects/ecogVIS/ecogvis/ecogvis.py in event_related_potential(self)
    546         #If all tries passed, open ERP
    547         if open_dialog:
--> 548             w = ERPDialog(parent=self)
    549 
    550 

~/ChangLab/projects/ecogVIS/ecogvis/functions/subDialogs.py in __init__(self, parent)
   1546         self.source = self.parent.model.nwb.processing['ecephys'].data_interfaces['high_gamma'].data
   1547         self.fs = self.parent.model.nwb.processing['ecephys'].data_interfaces['high_gamma'].rate
-> 1548         self.electrodes = self.parent.model.nwb.processing['ecephys'].data_interfaces['high_gamma'].electrodes
   1549         self.speaker_start_times = self.parent.model.nwb.intervals['TimeIntervals_speaker']['start_time'].data[:]
   1550         self.speaker_stop_times = self.parent.model.nwb.intervals['TimeIntervals_speaker']['stop_time'].data[:]

AttributeError: 'TimeSeries' object has no attribute 'electrodes'

Is high_gamma supposed to have electrodes as an attribute? Couldn't it use the electrodes in nwb_file.electrodes?

be able to separate raw and processed data

track elegant copying

The pynwb dev team is working on a more elegant way to save portions of an NWB file. See hdmf-dev/hdmf#326

when closing, as if you want to save bad time intervals

Saving on NWB files - two sets of files

Two sets of files in different folders: one with Raw ElectricalSeries & Preprocessed DecompositionSeries, and one with High gamma ElectricalSeries. Both files would have the same behavioral info (time intervals if we get that saved into the file, trial info, electrode info). We could copy the general subject file into both folders maybe to make the linking easier. The benefit to this is 1) if people want to copy files to their local computer, they don't need to copy all the raw/decomp as well and 2) there's enough people that only look at the high gamma, so it's worth replicating the current setup (a raw folder and a high gamma folder for each subject)

Error in writing the Preprocessed data

Was trying to re-preprocess the data (on a new file with only the raw in it), and got the following error when it was trying to write the processed data:

TypeError                                 Traceback (most recent call last)
~/ChangLab/projects/ecogVIS/ecogvis/functions/subDialogs.py in run(self)
    433                         mode=self.mode,
    434                         config=self.config,
--> 435                         new_file=self.new_fname)
    436 
    437 

~/ChangLab/projects/ecogVIS/ecogvis/signal_processing/processing_data.py in processing_data(path, subject, blocks, mode, config, new_file)
     26 
     27         if mode=='preprocess':
---> 28             preprocess_raw_data(block_path, config=config)
     29         elif mode=='decomposition':
     30             spectral_decomposition(block_path, bands_vals=config)

~/ChangLab/projects/ecogVIS/ecogvis/signal_processing/processing_data.py in preprocess_raw_data(block_path, config)
    294 
    295             # Write LFP to NWB file
--> 296             io.write(nwb)
    297             print('LFP saved in '+block_path)
    298 

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    412                         raise_from(ExceptionType(msg), None)
    413 
--> 414                 return func(self, **parsed['args'])
    415         else:
    416             def func_call(*args, **kwargs):

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/hdf5/h5tools.py in write(self, **kwargs)
    217 
    218         cache_spec = popargs('cache_spec', kwargs)
--> 219         call_docval_func(super(HDF5IO, self).write, kwargs)
    220         if cache_spec:
    221             ref = self.__file.attrs.get(SPEC_LOC_ATTR)

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in call_docval_func(func, kwargs)
    305 def call_docval_func(func, kwargs):
    306     fargs, fkwargs = fmt_docval_args(func, kwargs)
--> 307     return func(*fargs, **fkwargs)
    308 
    309 

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    412                         raise_from(ExceptionType(msg), None)
    413 
--> 414                 return func(self, **parsed['args'])
    415         else:
    416             def func_call(*args, **kwargs):

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/backends/io.py in write(self, **kwargs)
     39     def write(self, **kwargs):
     40         container = popargs('container', kwargs)
---> 41         f_builder = self.__manager.build(container, source=self.__source)
     42         self.write_builder(f_builder, **kwargs)
     43 

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    412                         raise_from(ExceptionType(msg), None)
    413 
--> 414                 return func(self, **parsed['args'])
    415         else:
    416             def func_call(*args, **kwargs):

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/build/map.py in build(self, **kwargs)
    168                 # TODO: if Datasets attributes are allowed to be modified, we need to
    169                 # figure out how to handle that starting here.
--> 170                 result = self.__type_map.build(container, self, builder=result, source=source, spec_ext=spec_ext)
    171         return result
    172 

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    412                         raise_from(ExceptionType(msg), None)
    413 
--> 414                 return func(self, **parsed['args'])
    415         else:
    416             def func_call(*args, **kwargs):

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/build/map.py in build(self, **kwargs)
   1658         builder.set_attribute('namespace', namespace)
   1659         builder.set_attribute(attr_map.spec.type_key(), data_type)
-> 1660         builder.set_attribute(attr_map.spec.id_key(), container.object_id)
   1661         return builder
   1662 

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    410                     if parse_err:
    411                         msg = ', '.join(parse_err)
--> 412                         raise_from(ExceptionType(msg), None)
    413 
    414                 return func(self, **parsed['args'])

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/six.py in raise_from(value, from_value)

TypeError: incorrect type for 'value' (got 'NoneType', expected 'NoneType')

Environment installation quits when trying to install nomkl

When trying to install the environment fresh, I run into this issue. I've tried on both my normal setup and my personal computer (fresh install of Anaconda and the repo), both Macs, with the same issue.

It seems to download nomkl fine, but has trouble installing it.

WARNING: The conda.compat module is deprecated and will be removed in a future release.
Collecting package metadata: done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.6.11
  latest version: 4.7.10

Please update conda by running

    $ conda update -n base -c defaults conda



Downloading and Extracting Packages
jupyter_core-4.5.0   | 48 KB     | ############################################################################################ | 100% 
libopenblas-0.3.6    | 8.4 MB    | ############################################################################################ | 100% 
pip-19.1.1           | 1.8 MB    | ############################################################################################ | 100% 
sqlite-3.29.0        | 2.4 MB    | ############################################################################################ | 100% 
kiwisolver-1.1.0     | 60 KB     | ############################################################################################ | 100% 
numpy-base-1.16.4    | 4.2 MB    | ############################################################################################ | 100% 
notebook-6.0.0       | 6.0 MB    | ############################################################################################ | 100% 
matplotlib-3.1.0     | 6.6 MB    | ############################################################################################ | 100% 
pyparsing-2.4.0      | 58 KB     | ############################################################################################ | 100% 
ipywidgets-7.5.0     | 107 KB    | ############################################################################################ | 100% 
pygments-2.4.2       | 664 KB    | ############################################################################################ | 100% 
pexpect-4.7.0        | 82 KB     | ############################################################################################ | 100% 
libpng-1.6.37        | 325 KB    | ############################################################################################ | 100% 
scikit-learn-0.21.2  | 5.9 MB    | ############################################################################################ | 100% 
llvm-openmp-4.0.1    | 454 KB    | ############################################################################################ | 100% 
defusedxml-0.6.0     | 23 KB     | ############################################################################################ | 100% 
libblas-3.8.0        | 6 KB      | ############################################################################################ | 100% 
libcblas-3.8.0       | 6 KB      | ############################################################################################ | 100% 
joblib-0.13.2        | 367 KB    | ############################################################################################ | 100% 
wheel-0.33.4         | 39 KB     | ############################################################################################ | 100% 
terminado-0.8.2      | 22 KB     | ############################################################################################ | 100% 
scipy-1.3.0          | 16.0 MB   | ############################################################################################ | 100% 
pytz-2019.1          | 236 KB    | ############################################################################################ | 100% 
widgetsnbextension-3 | 1.8 MB    | ############################################################################################ | 100% 
prometheus_client-0. | 42 KB     | ############################################################################################ | 100% 
openblas-0.3.6       | 9.2 MB    | ############################################################################################ | 100% 
jupyter_client-5.3.1 | 69 KB     | ############################################################################################ | 100% 
numpy-1.16.4         | 48 KB     | ############################################################################################ | 100% 
setuptools-41.0.1    | 635 KB    | ############################################################################################ | 100% 
liblapack-3.8.0      | 6 KB      | ############################################################################################ | 100% 
liblapacke-3.8.0     | 6 KB      | ############################################################################################ | 100% 
ipython-7.6.1        | 1.1 MB    | ############################################################################################ | 100% 
openssl-1.1.1c       | 3.4 MB    | ############################################################################################ | 100% 
nomkl-3.0            | 48 KB     | ############################################################################################ | 100% 
jinja2-2.10.1        | 188 KB    | ############################################################################################ | 100% 
ipykernel-5.1.1      | 162 KB    | ############################################################################################ | 100% 
pyopengl-3.1.1a1     | 1.3 MB    | ############################################################################################ | 100% 
tornado-6.0.3        | 644 KB    | ############################################################################################ | 100% 
parso-0.5.0          | 67 KB     | ############################################################################################ | 100% 
ca-certificates-2019 | 133 KB    | ############################################################################################ | 100% 
numexpr-2.6.9        | 129 KB    | ############################################################################################ | 100% 
qtconsole-4.5.2      | 92 KB     | ############################################################################################ | 100% 
blas-2.10            | 7 KB      | ############################################################################################ | 100% 
certifi-2019.6.16    | 154 KB    | ############################################################################################ | 100% 
nbconvert-5.5.0      | 381 KB    | ############################################################################################ | 100% 
pyqtgraph-0.10.0     | 996 KB    | ############################################################################################ | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Ran pip subprocess with arguments:
['/Users/jessieliu/anaconda3/envs/ecog_vis/bin/python', '-m', 'pip', 'install', '-r', '/Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt']
Pip subprocess output:
Collecting git+https://github.com/NeurodataWithoutBorders/pynwb.git (from -r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 1))
  Cloning https://github.com/NeurodataWithoutBorders/pynwb.git to /private/var/folders/h1/tpxkkr8s0db4_j57q_1wst1r0000gn/T/pip-req-build-dbw30pk1
Obtaining file:///Users/jessieliu/ChangLab/projects/ecogVIS (from -r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 4))
Collecting git+https://github.com/hdmf-dev/hdmf.git (from -r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 5))
  Cloning https://github.com/hdmf-dev/hdmf.git to /private/var/folders/h1/tpxkkr8s0db4_j57q_1wst1r0000gn/T/pip-req-build-8fed529l
Collecting git+https://github.com/bendichter/ndx-spectrum.git (from -r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 6))
  Cloning https://github.com/bendichter/ndx-spectrum.git to /private/var/folders/h1/tpxkkr8s0db4_j57q_1wst1r0000gn/T/pip-req-build-ahmtud7u
Collecting PyQt5 (from -r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/32/0f/0ba7fb39e89e331c3d9182f1a69421fa5461c232a85e0b55aea66b99e7ff/PyQt5-5.13.0-5.13.0-cp35.cp36.cp37.cp38-abi3-macosx_10_6_intel.whl (37.4MB)
Collecting nwbext_ecog==0.7.2 (from -r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/7f/45/67ae64875ec3125c511c59cb7afd9737bf5265991918a49c658c76a88425/nwbext_ecog-0.7.2-py2.py3-none-any.whl
Collecting chardet (from pynwb==1.0.3.post0.dev1->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
Requirement already satisfied: h5py in /Users/jessieliu/anaconda3/envs/ecog_vis/lib/python3.7/site-packages (from pynwb==1.0.3.post0.dev1->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 1)) (2.9.0)
Collecting hdmf (from pynwb==1.0.3.post0.dev1->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/a7/7d/c6d27a52bce1f6b86162f76b790d3a5bbed98d397f8f9b4fdf2aa566e0f0/hdmf-1.0.5-py2.py3-none-any.whl (80kB)
Requirement already satisfied: numpy in /Users/jessieliu/anaconda3/envs/ecog_vis/lib/python3.7/site-packages (from pynwb==1.0.3.post0.dev1->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 1)) (1.16.4)
Requirement already satisfied: pandas in /Users/jessieliu/anaconda3/envs/ecog_vis/lib/python3.7/site-packages (from pynwb==1.0.3.post0.dev1->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 1)) (0.24.2)
Requirement already satisfied: python-dateutil in /Users/jessieliu/anaconda3/envs/ecog_vis/lib/python3.7/site-packages (from pynwb==1.0.3.post0.dev1->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 1)) (2.8.0)
Collecting ruamel.yaml (from pynwb==1.0.3.post0.dev1->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/a6/e0/e65db3b59ad417113b50040bd083e489309343f7653f4ce75ec52518ad56/ruamel.yaml-0.15.100-cp37-cp37m-macosx_10_9_x86_64.whl (254kB)
Requirement already satisfied: six in /Users/jessieliu/anaconda3/envs/ecog_vis/lib/python3.7/site-packages (from pynwb==1.0.3.post0.dev1->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 1)) (1.12.0)
Requirement already satisfied: matplotlib in /Users/jessieliu/anaconda3/envs/ecog_vis/lib/python3.7/site-packages (from ecogvis==1.0.0->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 4)) (3.1.0)
Requirement already satisfied: cycler in /Users/jessieliu/anaconda3/envs/ecog_vis/lib/python3.7/site-packages (from ecogvis==1.0.0->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 4)) (0.10.0)
Requirement already satisfied: scipy in /Users/jessieliu/anaconda3/envs/ecog_vis/lib/python3.7/site-packages (from ecogvis==1.0.0->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 4)) (1.3.0)
Requirement already satisfied: pyqtgraph in /Users/jessieliu/anaconda3/envs/ecog_vis/lib/python3.7/site-packages (from ecogvis==1.0.0->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 4)) (0.10.0)
Requirement already satisfied: pyopengl in /Users/jessieliu/anaconda3/envs/ecog_vis/lib/python3.7/site-packages (from ecogvis==1.0.0->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 4)) (3.1.1a1)
Collecting nomkl (from ecogvis==1.0.0->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 4))

Pip subprocess error:
  Running command git clone -q https://github.com/NeurodataWithoutBorders/pynwb.git /private/var/folders/h1/tpxkkr8s0db4_j57q_1wst1r0000gn/T/pip-req-build-dbw30pk1
  Running command git clone -q https://github.com/hdmf-dev/hdmf.git /private/var/folders/h1/tpxkkr8s0db4_j57q_1wst1r0000gn/T/pip-req-build-8fed529l
  Running command git clone -q https://github.com/bendichter/ndx-spectrum.git /private/var/folders/h1/tpxkkr8s0db4_j57q_1wst1r0000gn/T/pip-req-build-ahmtud7u
  ERROR: Could not find a version that satisfies the requirement nomkl (from ecogvis==1.0.0->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 4)) (from versions: none)
ERROR: No matching distribution found for nomkl (from ecogvis==1.0.0->-r /Users/jessieliu/ChangLab/projects/ecogVIS/condaenv.u0ysyeiw.requirements.txt (line 4))

#
# To activate this environment, use
#
#     $ conda activate ecog_vis
#
# To deactivate an active environment, use
#
#     $ conda deactivate

support bipolar scheme

CV event detection?

I forget if we discussed this over email, but is event detection for the CV (consonant-vowel) task on the agenda? I think this would just be based on the microphone and/or speaker channels. I think that might be one of the last main functions to replicate the old version of the GUI.

There was a way that people were previously evaluating when stop/starts were (I heard it was a bunch of thresholding, etc.), and heard from Matt that it wasn't exactly ideal. I'd be happy to explore some better options to implement. Let me know what you think!

ERP plot y-axis

I was comparing the ERP plots made by ecogVIS and those made by the Matlab GUI, and just a small stylistic thing. Could you make it so that you can see the y-scale? They're usually global max scaled, so just putting the y-axis ticks on one plot should be fine

PERIODOGRAM - grid plots

Axes not displaying correctly on external monitor

The error below occurs after selecting the file 'CP3Rt_2019_07_15_16_51_45__MR_9.nwb' upon start up of ecogvis. This file is shared via Box (ChangLabData_ShareWithCatalystNeuro/CP3)

No 'preprocessed' data in 'processing' group.
No 'high_gamma' data in 'processing' group.
C:\Anaconda\envs\ecogVIS_CatalystNeuro\lib\site-packages\hdmf\spec\namespace.py:470: UserWarning: ignoring namespace 'hdmf-common' because it already exists
  warn("ignoring namespace '%s' because it already exists" % ns['name'])
C:\Anaconda\envs\ecogVIS_CatalystNeuro\lib\site-packages\hdmf\spec\namespace.py:470: UserWarning: ignoring namespace 'core' because it already exists
  warn("ignoring namespace '%s' because it already exists" % ns['name'])
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-8-1fba33462e73> in <module>
----> 1 main('')

C:\Anaconda\envs\ecogVIS_CatalystNeuro\lib\site-packages\ecogvis\ecogvis.py in main(filename)
    969     if app is None:
    970         app = QApplication(sys.argv)  # instantiate a QtGui (holder for the app)
--> 971     ex = Application(filename=filename)
    972     sys.exit(app.exec_())
    973 

C:\Anaconda\envs\ecogVIS_CatalystNeuro\lib\site-packages\ecogvis\ecogvis.py in __init__(self, filename, parent)
     70 
     71         # Run the main function
---> 72         self.model = TimeSeriesPlotter(self)
     73 
     74     def closeEvent(self, event):

C:\Anaconda\envs\ecogVIS_CatalystNeuro\lib\site-packages\ecogvis\functions\subFunctions.py in __init__(self, par)
     83 
     84         self.current_rect = []
---> 85         self.badChannels = np.where( self.nwb.electrodes['bad'][:] )[0].tolist()
     86 
     87         # Load invalid intervals from NWB file

C:\Anaconda\envs\ecogVIS_CatalystNeuro\lib\site-packages\hdmf\common\table.py in __getitem__(self, key)
    502                 return self.__indices[key]
    503             else:
--> 504                 raise KeyError(key)
    505         else:
    506             # index by int, list, or slice --> return pandas Dataframe consisting of one or more rows

KeyError: 'bad'

Cannot specify NWB file at command line

Minor issue: when attempting to open the GUI at the command line by providing a file path string, the user is still prompted to choose a file. Works fine once you select a file in the GUI.

For specificity, here is the exact command:
python ecog_vis.py '/path/to/data/EC125_B22.nwb'

ERP plot - stimulus vs response, switching causes error

On the ERP plot, I see there's buttons for stimulus and response. What exactly do these correspond to? For CVs, I would assume that stimulus corresponds to the listening portion (from the speaker channel) and that response corresponds to the speaking portion (from the mic channel).

When I tried to click between "stimulus" and "response" I often got this error in my ipython session. I don't think it successfully changes the plot either, only when switching between onset and offset does the plot change.

If I'm on "onset" and trying to switch between stimulus and response, the shapes it can't broadcast are (0) to (800). Once I switch to "offset" the shapes it can't broadcast (as in the above image) are (420) to (800). It's always at line 1731 in subDialogs.

Creating separate high gamma file, why include previous ECoG data?

I don't think it did this before (not sure though), but when making a new file, why does it copy other processing modules? This happens to copy the preprocessed data and the full decomposition, when we'd like to just copy over (mostly) everything that isn't ECoG, and then save the new high gamma data to it.

https://github.com/ben-dichter-consulting/ecogVIS/blob/9ce2a4bc7be14bedfb48f7b0231773da7dd8fa32/ecogvis/signal_processing/processing_data.py#L73

High Gamma Button

Now the high gamma button works for me.

Can you make the default name option for saving to a separate file be 'ECX_BY_hg.nwb'?

I also like how there's the option to add more bands. Can you make it the option to add both a custom band (which is what there is now), but also to add a band from the typical Chang Lab bands? Like, the preprocessing should be done on bands up to 200Hz. Some people in the lab use 70-200 Hz for high gamma so it'd be super nice to be able to just select (maybe like checkboxes) those extra couple bands above 150Hz.

Periodogram: spectrum definition issue

Getting this error when it's trying to calculate the power spectra information. Tried on a few files to make sure, doesn't seem like an error with the files but with ndx_spectrum maybe?

~/ChangLab/projects/ecogVIS/ecogvis/functions/subDialogs.py in run(self)
    200     def run(self):
    201         psd_estimate(src_file=self.src_file,
--> 202                      type=self.type)
    203 
    204 

~/ChangLab/projects/ecogVIS/ecogvis/signal_processing/periodogram.py in psd_estimate(src_file, type)
     61                                          power=PY_welch,
     62                                          source_timeseries=data_obj,
---> 63                                          electrodes=elecs_region)
     64 
     65         spectrum_module_fft = Spectrum(name='Spectrum_fft_'+type,

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/hdmf/utils.py in func_call(*args, **kwargs)
    420                     if parse_err:
    421                         msg = ', '.join(parse_err)
--> 422                         raise_from(ExceptionType(msg), None)
    423 
    424                 return func(self, **parsed['args'])

/anaconda3/envs/ecog_vis/lib/python3.7/site-packages/six.py in raise_from(value, from_value)

TypeError: unrecognized argument: 'electrodes'

Audio Event Detection "Run Test" button crashes ecogVIS app on click

Traceback (most recent call last):
File ".../ecogvis/functions/audio_event_detection.py", line 294, in run
direction='both'
TypeError: detect_events() got an unexpected keyword argument 'threshold'
Abort trap: 6

Indentation error in subDialogs

There's an indentation error in the check_hg_exists function in subDialogs.py. It's unclear what the else is referring to, could you fix this?

https://github.com/luiztauffer/ecogVIS/blob/aa85e6088fdb06bbdf4b325d4684d9c08ed9fc8a/ecogvis/functions/subDialogs.py#L485

Thanks!

Copying of changlab_to_nwb code into this repo

@luiztauffer I see that on the htk_loader branch, some of the code that I gave you guys is copied in here. Some of that might be confusing because a lot of it is outdated or is specific to our private changlab_to_nwb repo (so it's code that won't even run).

For example, ecogvis.functions.tdt_to_nwb.chang2nwb hosts a lot of outdated code but also the updated electrodes table function. I think it would be best to remove everything that is not being used and create a specific file to contain functions that are being used like elecs_to_electrode_table.

Reorganization caused slight break? Unable to start GUI

I think the recent reorganization of some of the code has caused a break. So there is now an ecogvis folder inside of ecogVis and the Python script to call is inside the ecogvis folder. When I try to call that script, I get the error:

Traceback (most recent call last):
  File "ecogvis/ecogvis.py", line 15, in <module>
    from ecogvis.functions.subFunctions import ecogVIS
  File "/Users/jessieliu/ChangLab/projects/ecogVIS/ecogvis/ecogvis.py", line 15, in <module>
    from ecogvis.functions.subFunctions import ecogVIS
ModuleNotFoundError: No module named 'ecogvis.functions'; 'ecogvis' is not a package

I completely removed my previous environment and deleted the repo, and re-installed everything and ran into this issue.

Proper channels ordering from HTK

Wav11.htk
...
Wav19.htk
Wav110.htk
...
Wav164.htk
Wav21.htk
...
Wav29.htk
Wav210.htk
...
Wav264.htk

Power vs Absolute value in spectral decomp of data

I just noticed that the spectral decomposition is being saved as the bandpower (V**2/Hz), but only the absolute value is being taken (not being squared). Is that still giving out the bandpower or just absolute value of the analytic signal? Maybe @bendichter needs to weigh in on this, or I can ask around.

https://github.com/luiztauffer/ecogVIS/blob/aa85e6088fdb06bbdf4b325d4684d9c08ed9fc8a/ecogvis/signal_processing/processing_data.py#L348

Auto-width of the sidebar cutting off some button labels

When I open the ERP plot, sometimes the button labels get cut off a little on the sidebar. Is there a way to auto-width the sidebar or something?