montpellierressourcesimagerie / microscope-metrics Goto Github PK

There is a place for a comment as an output but there is no way to give a comment when running an analysis

It seems that isort conflicts with blacked in the way we format multiple imports. I'd rather have blacked so for the moment I deactivate isort

FutureWarning: multichannel is a deprecated argument name for gaussian. It will be removed in version 1.0. Please use channel_axis instead.

FutureWarning: indices argument is deprecated and will be removed in version 0.20. To avoid this warning, please do not use the indices argument. Please see peak_local_max documentation for more details.
indices true will make the function return the mask for the maxima. Unfortunately we do not use the default bahaviour.

FutureWarning: The 'mad' method is deprecated and will be removed in a future version. To compute the same result, you may do (df - df.mean()).abs().mean().

Scientists and microscope facility engineers need to be able to easily change metrology workflows and manage pipelines. Integrating microscope-metrics with a workflow manager can offer the possibillity to modify pipelines in a modular way. Using such a framework also permits adding analysis and ML features.

Possible solutions:

• KNIME
• Kedro
• Ranku
• BIAFLOWS
• Snakemake
• Galaxy

Things to keep in mind:

• open source development
• use in the imaging community
• integration with existing tools

It might be easier to work with xarrays in stead of numpy arrays.

We need to find a proper way to serialise the output measurements;
Must be compatible with numpy arrays.

We need to provide type hinting for, at least, all the exposed functions.
We will have to go through the code and verify it.

This is shooting a warning

Currently the input analysis parameters are not reflected in the log. This should be the case.
We should probably run a number of predefined tasks before running the analysis:

• validate requirements
• log input parameters

We would like a separate library for plotting the output (and input) data that is provided by microscope-metrics.
We could call it something like microscope-metrics-viz/plots/graphs/gallery.

It has been discussed that we would need a Dashboard application to visualize the data. Let's use this issue to list and discuss the options. Links ar etowards the galleries of the different solutions.

• Jupyter notebooks. Not really a dashboard app but very interesting to provide a quick experimental dashboard
• Dash https://dash.gallery/Portal/
• H2O wave https://h2o.ai/platform/ai-cloud/make/h2o-wave/
• Streamlit (very easy) https://streamlit.io/

More options?

It has been suggested that we will need a workflow manager to bind together the different components. Let's use this issue to list and discuss the different options.

• Knime https://www.knime.com/
• Kedro https://kedro.org/
• Biaflows https://biaflows-sandbox.neubias.org/
• Galaxy https://galaxyproject.org/
• Renku https://renkulab.io/
• SnakeMake https://snakemake.readthedocs.io/en/stable/

More suggestions?

Here are the main steps of the Field Illumination algorithm.
1- Image is split into individual channel images.
2- (opt) Each channel is blurred (Gaussian blur, sigma 2.0) [mind this influences the next step as it may remove saturated noise pixels]
3- Saturation is measured
4- A fia (field illumination array) class is created for each channel. This arrays stores coordinates
5- ImageStatistics are computed for each channel.

• xCenterOfMass and yCenterOfMass are used and stored as xCenterOfMass and yCenterOfMass
• min is used (stored as min). Using the min value, the pixel (starting from (0,0) and scanning lines after lines) that has the min value is considered as the minimum pixel. Coordinates are stored as xMin and yMin.
• max is used (stored as max. Using the max value, the pixel (starting from (0,0) and scanning lines after lines) that has the max value is considered as the minimum pixel. Coordinates are stored as xMax and yMax. Mind that if several pixels have the min or max value, only the first one will be considered as min and max pixels.
  6- (opt) Using the max value, a threshold is set (derived from the bins of the normalized intensity image). For instance if bins are set to 10, a threshold 0.9max-max is set (for each channel using respective max values).
  7- ImageStatistics of the thresholded image are computed and xCenterOfMass and yCenterOfMass are used for xCentThresholdZone and yCentThresholdZone. If the (“use last bin as max reference zone”) option is not chosen, then a max-max threshold is used.
  8- A normalized intensity image is computed using the bins (each pixel which intensity is “in a bin” are given the same “bin” value).
  Uniformity is calculated (min/max * 100.0).
  9- Intensity of remarkable pixels are measured (corners, edges middles).
  10- FieldUniformity is calculated as = 100.0 - 100.0 * standard deviation of (intensities of remarkable pixels) / max;
  centeringAccuracy is calculated as
  100-1002/√(〖image^' s width〗^2+〖image^' s height〗^2 )√(〖(xCentThresholdZone-(image^' s width)/2)〗^2+〖(yCentThresholdZone-(ima〖ge〗^' s height)/2)〗^2 )
  If the (“use last bin as max reference zone”) was not chosen, centeringAccuracy is calculated as
  100-1002/√(〖image^' s width〗^2+〖image^' s height〗^2 )*√((xMax-(image^' s width)/2)^2+(yMax-(ima〖ge〗^' s height)/2)^2 )

11- Profile images are generated by drawing an intensity/coordinates plot along the remarkable lines.

The color of ROIS fill and strokes is implemented as a tuple. Pydantic's Color type would be way more flexible.

The different analysis have to do a number of validations that are going to be generic to any analysis, such as if all the required parameters are in place.
We can generalize those within the run function and make this one call a private _run that is override in the analysis subclass

We need some images to test the analysis of field illumination

Analysis date and time is not provided as output.

Microscope-metrics does not currently raise error exceptions. At the moment the module solely logs activity. Implementing error handling also entails identifying possible errors and their meaning to users.

An example of a possible error in this project is a saturation error when the saturation present on an image does not permit deducting viable metrics.