system is computationally singular about slingshot HOT 4 CLOSED

Hi @galicae ,

Thank you for the detailed report! I have not yet been able to reproduce the error, but I have a few thoughts. The quickest workaround might be to use the diagonal (rather than full) covariance structure. To do this, you will need to pass this function to getLineages as the dist.fun argument.

Additionally (and I'm just guessing here), it looks like the coordinates you're passing in from the diffusion map are all quite small, between -0.04 and 0.04. So the computational issues may be related to handling very small numbers. If this is the case, you might be able to solve this by multiplying the coordinates by a large constant (100 to 1000?).

Also, did you mean to attach the file sim98_slingshot_destiny_log_k1? In your code, you import sim98_destiny_log_k1 and it looks like that is a diffusion map object (as opposed to a SlingshotDataSet).

from slingshot.

Hey Kelly,

Wow that was fast :) thanks for taking the time!

I think there is merit in the "multiply by 100/1000" guess/solution, but it doesn't explain why adding noise to the diffusion components solves the problem. You are right about the file - I misclicked D: it was supposed to be `sim98_destiny_log_k" (no "1"s though), the diffusion map on which coordinates I want to calculate the lineage. I am going to update the original post.

Also, thanks for the pointer to use the diagonal instead of the full covariance. Seems cleaner than the random noise idea. Will try that too.

cheers,
Niko

EDIT: updated!

from slingshot.

Hey Niko,

Thanks! I'm now able to reproduce the error and I think the problem may have to do with some of the clusters being highly linear. I tried to invert the sums of all pairwise combinations of cluster covariance matrices and found that certain clusters were more "problematic" than others and most of these clusters were highly linear. For example, here are all ten dimensions of Cluster 4:

This could explain why adding some Gaussian noise solves the problem (and multiplying by a large constant does not).

I'm not super familiar with Mclust, but I think you might be able to avoid this by either decreasing the range of possible clusters (I got 47, which seems like a lot, but it's also not unreasonable for 9500 cells) or by adjusting the modelNames parameter, which affects cluster shape.

Best,
Kelly

from slingshot.

Thanks a lot for your time! Your suggestions more or less answer my question. Would be happy to revisit if you decide to include clustering in future slingshot versions 😉

best,
Niko

from slingshot.