Git Product home page Git Product logo

miv-simulator's People

Contributors

austine4 avatar dependabot[bot] avatar frthjf avatar gauravu2 avatar iraikov avatar skim0119 avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

miv-simulator's Issues

Improve env config validation

We currently compute and validate configuration during env construction which has the downside that job misconfiguration may be detected late. We should provide a config schema that allows for stronger validation ahead of the job submission to prevent bogus executions.

Error msg in using `SC_nrn` model without synapse creation.

We need an improved error message when the single compartment model is used without synapse connections

image

Note

  • "It should fail with a more descriptive message, but that error should be generated by explicit validation checks at the beginning of distribute_poisson_synapses and distribute_uniform_synapses"

[Bug] MPI processes hanging after the job is done.

Description

I'm seeing some hanging processes even after the jobs are cleared. They don't occupy any memory or compute resources and they don't seem to affect other runs, so I'm setting the priority to be low.

top:
image

Environment

  • Local machine
  • Linux (Ubuntu) 21
  • Docker image

Improve support for electrode geometry

We currently only have a basic way to specify the locations of the simulated neurons. In particular, we have no easy way to describe the placement of the MEA electrodes that stimulate individual neurons. The goal of this project is to extend the simulator geometry module to improve the configuration and visualization of stimulating electrodes.

Resources

  • pydantic documentation; we use pydantic models to implement strongly-typed specifications of the simulator configuration
  • h5py and neuroh5 to write and read the H5 coordinate files used by the MiV simulator
  • 3D visualization packages like mayavi, or good old matplotlib
  • RBF package that is used for geometry calculation

Subgoals

  • Design appropriate pydantic models to configure and validate electrode geometries (see MEA package for inspiration). In its most basic form, this could be X,Y, Z coordinates plus a radius. Use plots to visualize and verify
  • Implement a function that takes an electrode config model from above and computes the neurons it 'touches' using the coordinates from the H5
  • Write documentation explaining the developed approach

To get started

Launch the the docker image via docker-compose up and open the JupyterLab environment.

The example code below shows how to configure and call relevant APIs to generate and visualize neural geometries. Read the relevant method source code and the miv_simulator.config pydantic models to get started.

1. Creating the neural H5 file
from miv_simulator.simulator import create_neural_h5

cell_distributions = {
  "STIM": {"SO": 0, "SP": 1000, "SR": 0, "SLM": 0},
  "PYR": {"SO": 0, "SP": 80000, "SR": 0, "SLM": 0},
  "PVBC": {"SO": 352, "SP": 1034, "SR": 88, "SLM": 0},
  "OLM": {"SO": 438, "SP": 0, "SR": 0, "SLM": 0},
}

create_neural_h5(
  output_filepath='./example.h5',
  cell_distributions=cell_distributions,
  synapses={
      'PYR': {'STIM': True, 'PYR': True, "PVBC": True, "OLM": True},
      'PVBC': {'PYR': True, 'STIM': True, 'PVBC': True, 'OLM': True},
      'OLM': {'PYR': True, 'PVBC': True}
  }
)
2. Generate coordinates from geometry configuration
from miv_simulator.simulator import generate_network_architecture

layer_extents={
  "SO": [[0, 0, 0], [4000, 4000, 100]],
  "SP": [[0, 0, 100], [4000, 4000, 150]],
  "SR": [[0, 0, 150], [4000, 4000, 350]],
  "SLM": [[0, 0, 350], [4000, 4000, 450]],
}

rotation=[0, 0, 0]

generate_network_architecture(
  output_filepath="example.h5",
  cell_distributions=cell_distributions,
  layer_extents=layer_extents,
  rotation=rotation,
  cell_constraints={"PC": {"SP": [100, 120]}, "PVBC": {"SR": [150, 200]}},
  output_namespace= "Generated Coordinates",
  geometry_filepath=None,
  populations=(),
  resolution= (3, 3, 3),
  alpha_radius=2500.0,
  nodeiter=10,
  dispersion_delta=0.1,
  snap_delta=0.01,
  h5_types_filepath=None,
  io_size=-1,
  chunk_size=1000,
  value_chunk_size=1000,
)
3. Visualize generated geometry
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from miv_simulator.geometry.geometry import get_total_extents
from neuroh5.io import read_cell_attributes

# options
populations = ('STIM',)
mayavi = False # set to True for 3D
subvol = False
subpopulation = -1
scale = 25.0

print("Reading coordinates...")

(extent_u, extent_v, extent_l) = get_total_extents(layer_extents)

pop_min_extent = None
pop_max_extent = None

xcoords = []
ycoords = []
zcoords = []
cmap = cm.get_cmap("Dark2")
cmap_range = np.linspace(0, 1, num=len(populations))

colors = []
for pop_id, population in enumerate(populations):
  coords = read_cell_attributes(
      'example.h5', population, namespace="Generated Coordinates"
  )

  count = 0
  cxcoords = []
  cycoords = []
  czcoords = []
  for k, v in coords:
      count += 1
      cxcoords.append(v["X Coordinate"][0])
      cycoords.append(v["Y Coordinate"][0])
      czcoords.append(v["Z Coordinate"][0])
  if subpopulation > -1 and count > subpopulation:
      ridxs = np.random.choice(
          np.arange(len(cxcoords)), replace=False, size=subpopulation
      )
      cxcoords = list(np.asarray(cxcoords)[ridxs])
      cycoords = list(np.asarray(cycoords)[ridxs])
      czcoords = list(np.asarray(czcoords)[ridxs])

  colors += [cmap(cmap_range[pop_id]) for _ in range(len(cxcoords))]
  xcoords += cxcoords
  ycoords += cycoords
  zcoords += czcoords
  
  print(f"Read {count} coordinates...")

  pop_distribution = cell_distributions[population]
  pop_layers = []
  for layer in pop_distribution:
      num_layer = pop_distribution[layer]
      if num_layer > 0:
          pop_layers.append(layer)

          if pop_min_extent is None:
              pop_min_extent = np.asarray(layer_extents[layer][0])
          else:
              pop_min_extent = np.minimum(
                  pop_min_extent, np.asarray(layer_extents[layer][0])
              )

          if pop_max_extent is None:
              pop_max_extent = np.asarray(layer_extents[layer][1])
          else:
              pop_max_extent = np.maximum(
                  pop_min_extent, np.asarray(layer_extents[layer][1])
              )

pts = np.concatenate(
  (
      np.asarray(xcoords).reshape(-1, 1),
      np.asarray(ycoords).reshape(-1, 1),
      np.asarray(zcoords).reshape(-1, 1),
  ),
  axis=1,
)

if mayavi:
  from mayavi import mlab
else:
  import matplotlib.pyplot as plt

print("Plotting coordinates...")
if mayavi:
  mlab.points3d(*pts.T, color=(1, 1, 0), scale_factor=scale)
else:
  fig = plt.figure()
  ax = fig.add_subplot(111, projection="3d")
  ax.scatter(*pts.T, c=colors, s=int(scale))

print("Constructing volume...")
from miv_simulator.volume import make_network_volume

if subvol:
  subvol = make_network_volume(
      (pop_min_extent[0], pop_max_extent[0]),
      (pop_min_extent[1], pop_max_extent[1]),
      (pop_min_extent[2], pop_max_extent[2]),
      resolution=[3, 3, 3],
      rotate=rotation,
  )
else:
  vol = make_network_volume(
      (extent_u[0], extent_u[1]),
      (extent_v[0], extent_v[1]),
      (extent_l[0], extent_l[1]),
      resolution=[3, 3, 3],
      rotate=rotation,
  )

print("Plotting volume...")

if subvol:
  if mayavi:
      subvol.mplot_surface(color=(0, 0.4, 0), opacity=0.33)
  else:
      subvol.mplot_surface(color="k", alpha=0.33, figax=[fig, ax])
else:
  if mayavi:
      vol.mplot_surface(color=(0, 1, 0), opacity=0.33)
  else:
      vol.mplot_surface(color="k", alpha=0.33, figax=[fig, ax])
if mayavi:
  mlab.show()
else:
  ax.view_init(-90, 0)
  plt.show()

Untitled

Allow overrides for default mechanisms and morphology

Like with the YAML configuration, there should be a way to progressively override the default mechanism and morphology without duplication all of the configuration files. For example, a user might change the cat.mod mechanism in their code base, while still falling back on the default mechanism in all other cases.

Proposed implementation:

  • Copy defaults into tmp directory within venv, merge in user files to potentially override defaults
  • Compile dll and return location of tmp directory to be used instead of user directory
  • Compute a directory hash of user directory (likely using a suitable subprocess call) to avoid unnecessary recompilation

Bug: default mechanism path

@click.option(
"--mechanisms-path",
"-m",
required=False,
type=click.Path(exists=True, dir_okay=True, file_okay=False),
)

There is a bug in this part of the code. If user don't provide -m, --mechanisms, the rest of the code seems like the default path is set to be "./mechanisms":

But currently, click passes None and causes error downstream.

Naming conflict: generate_input_spike_trains

There are three incidences of the name generate_input_spike_trains, and we should probably refactor them for the documentation. Also, it causes some conflicts here.

  • File src/scripts/generate_input_spike_trains.py (executable)
  • File src/miv_simulator/simulator/_generate_input_spike_trains.py (backend of executable)
    • Includes method generate_input_spike_trains
  • File src/miv_simulator/stimulus.py
    • Includes method generate_input_spike_trains

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.