fjug / moma Goto Github PK

It would be useful to check how well MoMA works when the phase contrast acquisition is changed: although the image intensity is normalized, a change in the dynamic range will affect thresholding / segment generation...

Obviously linked to #20 which would provide a general solution to this as well (by allowing users to customize moma to their input images)

Right now it seems that the way MoMA was run (i.e command-line argument and parameter file settings) are not displayed in the results file (i.e. neither in the exported cell stats nor in the user curation file). As a general rule, bioinformatics software that I wrote always starts the results file by simply pasting the command line and parameters that were used to obtain those results.
Might be helpful to add this.

What is the current way to set the length of the GC that is analysed?
I guess it requires to use a parameter file, not so clear which one yet (probably highlighting the need for #4)

• allow to configure export options

certainly worth a page on the wiki

• how to contribute?
• coding style?
• ???

pruned lineages are not propagated after the previously last frame of the range when hitting <–all

UBBTB should e.g. become 2BBTB, if the cell in question is a descendent of the 2nd cell in the GL at movie onset.

following the skype meeting today, Florian proposed to develop a plugin extension architecture to MoMA (requires ≈2.5 days).
would be useful for e.g.:

• refined length estimation
• refined fluo estimation
• intracellular object tracking

Have hyperstackreg work only on a subset of the picture (typically a ROI located on the numbers). We think this would fix small drift in long datasets.

This issue is two-fold:

• the way to change which assignments are displayed should be explained more explicitly:
  • double click for dialog (already mentioned in shortcuts)
  • click and drag… this is still mysterious to me (btw what is delta??)
• in theory, I expect that all assignments between each pair of segmentation hypotheses should be considered (even between hypothesis tat aren't currently in use); hence with very permissive costs limits (e.g. -3e5; +3e5) I expect to display them. not only I don't see any hypothesis between inactive segments, bu some between active segments even keep being hidden. I don't know whether I misunderstand something or whether this is a bug.

very low priority!

because cells about to leave a growth lane will always be problematic (partly cut, blurred, etc) and hence discarded from the data analysis, it would be very useful to discard them in MoMA already. This probably means giving a special status to segments that touch the exit (at least pruning them, even better would be to exclude them from the optimization).

At the moment, I give myself a visual threshold and don't fix issues closer to the exit (sometimes pruning the corresponding lineage to be sure that the error cannot be propagated). However, this has 2 limitations:

• in some datasets, a large fraction of the errors occur on cells touching the exit (or the one below), I guess the optimisation would work best if all segments touching the exit would be discarded
• the visual curation would be easier if segments and assignments of the cell touching the exit (and hence shrinking with time) would not be displayed.

I hope this request makes sense. Curious to read whether you've considered (and hence discarded something similar during the development)…

Best, Thomas

since mmpreprocess is built around imageJ, would it be easily possible to refactor it a plugin rather than a standalone executable? what I have in mind would be a plugin that would be given one image (or one frame of an hyperstack) and would create one roi per growth channel (ideally named filename_GLxx)?
Then it would be straightforward to crop and save one dataset per GL…

In my opinion, it would be much more user friendly because it would keep all the preprocessing in Fiji. It might also help preserving image metadata during the preprocessing (#3). Other issues (e.g. #14) would be addressed directly by the user.

in order to run Erik's postprocessing on phase contrast data, all the space between segments is currently attached to the neighbouring cells.

this is visually disturbing and curation would be easier if the cells length was shown more accurately

• might be addressed by generating better segment hypotheses
• might be circumvented by post processing the segments length (using Erik's algorithm) before displaying them

it's only when trying to reproduce this bug that I realised that it is possible to set the optimisation range before hitting optimize for the first time…

here is the step-by-step to the bug:

• load a dataset
• hit optimize
• then set the range to e.g. 150 frames
• select one frame in the first 150 frames and hit <–all

an ILP infeasible error is raised, normally easily solved by hitting optimize a second time…

Hi,

I sometimes encounter segments that do exist but can't be selected. Usually I simply prune the whole sub lineage but it "kills" the GL when it's the bottom cell… See pictures below from 20160526_pos0_GL09 (I'll share this dataset as well).

Is there any equivalent to the assignment to changing the costs of displayed assignments but for segments?

In the case I describe the corresponding "mapping" assignment is usually very difficult to select (even when releasing cost constraints). I suspect that this comes from the fact that shift-click is hiding all assignment below the mouse, which is great to clear things up at first but makes it sometimes very difficult to pick one (when it's perfectly overlapped by smaller ones, yes shit happens!). A simple work around would be yet another short cut to mask visible assignments one by one below the mouse.

Best,
Thomas

this has come over and over in emails but never made it as an issue (probably because not required for the first method paper and not a minor request…)

because MoMA produces a set of nested segmentations hypotheses and then handle them as a graph, it should in principle be possible to generate this segmentations hypotheses from other types of images such as fluorescence (uniform cytoplasm tagging) or more sensitive phase reconstructed images (such as correlation images).

Importantly, this should be designed as far as possible such that users can support new image types by themselves (by defining a config file and e.g. training a classifier)…

This should also help improving the following:

• more precise segment length estimation
• reduce the need for manual curation (because e.g. fluorescence images are supposed to be less error-prone)

So far this is done by post processing scripts generated in Basel.

• Need to see what these steps are and how they could be incorporated.
• Evaluate post processing plug-in concept versus re-implementation of these two scripts.

would be very handy because several datasets are usually analysed in parallel to minimise waiting time.

I'm working with a ubuntu 16.04 LTS 64 bit.
I installed MoMA according to the fjug/MoMA page instructions. Everything went well until I reach the step of data processing (following the Quick user guide instructions, on the MoMA_prepoc_example file) : after selecting the growth channel dataset to analyze in the dialog opened by MoMA, an error message was displayed :

Gurobi wasn't already installed on my computer, so I installed it according to the instructions of the Gurobi website. The problem persisted, so I created the directories that Java is expecting when I run it (/.../Fiji/Fiji.app/lib/linux64 and /.../Fiji/Fiji.app/mm/linux64) and copied the required files from the gurobi files in those directories (content of the /opt/gurobi652/linux64/lib/ file and simply gurobi.jar respectively).

I was then able to run MoMA properly on the example samples.
I don't know if this can be useful, but this is one problem that can be encountered during the installation of MoMA, and that may come from the absence of anterior Gurobi installation.

Best regards,

Theo

I followed all the instructions to get MoMA and the preprocessing working, but I'm stuck with a bug that I'm not able to solve. When I run the preprocessing using the moma_preprocess script on the official dataset in the folder called "MoMA_preproc_example", I get the following error message:

Caused by: java.lang.IllegalArgumentException: ERROR File list corrupt. Time could not be extracted for file /Users/guillaume/Desktop/PostdocBasel/MoMA_test/MoMA_preproc_example/20150624-lac-2-MMStack-Pos0-preproc_t0001_c0001.tif.
at com.jug.mmpreprocess.MMDataSource.isInDataRange(MMDataSource.java:82)
at com.jug.mmpreprocess.MMDataSource.(MMDataSource.java:47)
at com.jug.mmpreprocess.MMPreprocess.main(MMPreprocess.java:52)
... 5 more

I reproduced the bug with other datasets with other names, but I can't find a solution. The file name parser seems to have a problem...

anywhere, but visible...

Hi!

I have been trying to install and use MoMA on Fiji (fresh install, up-to-date). After installation via ImageJ's updater as described in the installation guide, when I want to run MoMA for the first time (Plugins > MoMA > MoMA application processing a folder/image/current), instead of the pop-up window asking for a Gurobi license I get the following exception:

(Fiji Is Just) ImageJ 2.0.0-rc-69/1.52i; Java 1.8.0_66 [64-bit]; Linux 4.17.19-100.fc27.x86_64; 150MB of 20667MB (<1%)
 
java.lang.NoClassDefFoundError: gurobi/GRBException
    at com.jug.fijiplugins.MotherMachineDefaultPipelinePlugin.run(MotherMachineDefaultPipelinePlugin.java:40)
    at ij.IJ.runUserPlugIn(IJ.java:228)
    at ij.IJ.runPlugIn(IJ.java:192)
    at ij.Executer.runCommand(Executer.java:137)
    at ij.Executer.run(Executer.java:66)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: gurobi.GRBException
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 6 more

I have tried this both on Linux (Fedora 27) and MacOSX and I get the same exception. I have tried installing Gurobi separately and then launching MoMA but to no avail. There are a couple (minor) things that could be updated also in the install guide on the wiki.

Thanks!

I figured out that when a custom property file is used (with -p), then the values in the default mm.properties sitting in the MoMA directory are updated to the values of the custom files.
This is very counterintuitive and affects which parameters are used for image analysis at the next run without custom property file…

I imagine that this comes from the fact that MoMA saves things like its window position.
It should be saved back to the custom file imho, or saving of analysis-related parameters and gut-related parameters would have to be done in separate files.

Connected to #11: in practice one always want to set a reasonable optim range at start.
Would be nice to have it as a parameter…

when trying the fiji version of MoMA on our centOS server (where we usually run it from the command line), I ran into the problem that my gurobi license was not detected despite GRB_LICENSE_FILE being set correctly (but the license file is not gurobi.lic in the home directory).

Would be nice to use this variable if the default file is not found before raising an error.

when using -c 2 with a dataset for which 3 channels exist, the following error occurs (while the expected behaviour would be to load channels 1 and 2 only and to ignore images named _c0003.tif):

Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.IllegalArgumentException: ERROR    File list corrupt. Attempt to load file /scratch/11477251.1.long.q/20151218_switch8h_pos6/20151218-switch8h-1-MMStack-Pos6-preproc_t0001_c0003.tif as channel 1.
    at com.jug.mmpreprocess.MMDataFrame.sanityChecks(MMDataFrame.java:82)
    at com.jug.mmpreprocess.MMDataFrame.<init>(MMDataFrame.java:59)
    at com.jug.mmpreprocess.MMDataSource.<init>(MMDataSource.java:66)
    at com.jug.mmpreprocess.MMPreprocess.main(MMPreprocess.java:52)
    ... 5 more

I copy/paste some email discussions for the records of what is still missing in the wiki:

• explain moma_preprocess requirements regarding input file names
• explain how to launch MoMA
• a version history page would really not be superfluous as well as a way to know which version (ideally which commit) is running
• upload sample datasets using LFS
• (distribute jars attached to releases rather than committed to the wiki). alternatively, hide the release tab on MoMA's homepage because it will be confusing (or delete the 2 existing releases)

• because scrolling the timeline is really the primary type of action, the timeline should be the default active element after most/every interaction (e.g. clicking a segment/assignment, hitting <-all, etc)
• add a keyboard shortcut to 'all correct before' (a would be obvious although it's taken)
• among currently available shortcuts, should we keep the following:
  • r Reset optimization (tracking) very risky despite the validation dialog
  • a Show Cell Counting Panel * I never use this panel… should I? should it be moved to some other view*
• configure export options, cf #5
• set a default optimisation range in the options (to be used at startup)

Thanks,
Thomas

Hello,

We are trying to make a more extensive use of pruning to detect some cell lysis events…
Doing so, I realised that the end_type variable of a pruned lineage is set by what happens downstream of pruning (one of exit / div / eod). I think that the general behaviour should be to set the end_type to "pruned".

Thanks!

would it be possible to use a score for each segment/assignment to identify a subset of the lineage that has a very high probability to be correct. If the fraction of the data extracted is large enough (and without too frequent breaks along branches), we could skip the systematic curation.

• already curated data could be used to check whether there is a clear cutoff based on simple scoring metrics
• this is more likely to work with more reliable input data such as fluorescence