Failure to split: component starvation about moksm HOT 4 CLOSED

This issue is most likely not a bug but a problem with the data. Component starvation means that one of the components had less data points assigned to it than necessary to estimate the covariance matrix. This usually happens once you start overfitting (obviously not the case here) or if there are outliers in the dataset.

There are two things you can try to diagnose the problem:

1. Plot the peak-to-peak amplitudes or first PC versus time to check if there are obvious outliers. If that's the case, try removing them manually and re-run the algorithm.

2. Run MoKsm with verbose=true and maybe have it plot the data after every few iterations (insert a call to plot() in the EM iteration) to see what's happening. Most likely one of the two components converges towards a small number of data points and gets a weird shape.

from moksm.

I've tried what you suggested - removing outliers and also plotting during EM iteration. However, even when using only the feature dimension with greatest separation between two clusters, I observed that two cluster means converge towards each other with one cluster eventually dropping out. I played around with the parameter settings, but again this doesn't seem to help.

I have created test-case at eywalker/moksm debug branch eywalker/moksm@eb282e9 - it will be great if you can take a look at it by running through the testMoKsm script.

from moksm.

I think the problem is the scale of the data. It has to be in muV, but it seems your data is on a different scale (judging from the feature vs. time plot in the image above). In this case the CovRidge and DriftRate parameters (and possibly others, which are sensitive to the scale of the data) need to be scaled down accordingly.

from moksm.

Scaling was indeed the problem - scaling up the data by a factor of 100 did the trick and now the data clusters correctly using the parameter settings I have configured previously. It looks like that for some reason this particular data-set breaks assumptions made in the gain adjustment inside the SpikeSortingHelper. I'll work on correcting the issue there. Thanks!

from moksm.