pyxem / kikuchipy Goto Github PK
View Code? Open in Web Editor NEWToolbox for analysis of electron backscatter diffraction (EBSD) patterns
Home Page: https://kikuchipy.org
License: GNU General Public License v3.0
Toolbox for analysis of electron backscatter diffraction (EBSD) patterns
Home Page: https://kikuchipy.org
License: GNU General Public License v3.0
Describe the bug
The signal._lazy
attribute is not set to False
when calling signal.compute()
and reading signal data into memory. Although the class is changed from LazyEBSD
to EBSD
, the EBSD
init()
function is not called, as is done by e.g. pyXem here... Since the metadata isn't any different between the two classes, I think just setting signal._lazy = False
in compute()
is OK.
To reproduce/expected behaviour
s = kp.load('/some/data.h5', lazy=True)
s.compute()
print(s._lazy) # Should print False, will still print True
Useful e.g. when deciding with which adjacent patterns to average each pattern, to avoid averaging across a grain boundary, as per the discussion in #13. A pattern difference map in itself is rich in information, and is comparable to e.g. a kernel average misorientation map or grain boundary map.
The 'pattern difference' procedure as explained by Wright et al. (https://doi.org/10.1016/j.ultramic.2014.10.002):
This is similar to obtaining the normalized cross-correlation (NCC) coeffienct or normalized dot product (NDP). We should have general implementations of both. Should implement them together since they are mainly used for the same puproses. Implementations should be so that they can be used in multiple applications in KikuchiPy. See #116 and #117 for example uses.
To do:
EBSD method to calculate similarity between each pattern in a scan to a dictionary (preferably with a mask). Might be faster to do this as EMsoft does it, i.e. construct 2D matrices e.g. of 1024 experimental and simulated patterns, and multiplying these to collect dot products, continuously sorting them for each experimental pattern. Should perhaps start with a naive implementation an go from there.
Should be careful not to duplicate work done in pyXem: https://github.com/pyxem/pyxem/blob/master/pyxem/utils/indexation_utils.py#L44
Similarity metrics we should support:
This will be slow, however, let's start with the correct, naive implementation first and improve from there.
Have to wait for #15.
To do:
Functionality:
When performing adaptive histogram equalisation, the frequency in each histogram intensity bin should be approximately the same, as in Marquardt et al. (2017), dx.doi.org/10.2138/am-2017-6062, and not like scikit-image's histogram equalization example. If desired, we could also support this functionality at a later time.
Describe the bug
Images in the documentation get stretched (ugly!) when the viewport width is reduced, e.g. reducing browser width or in upright mode on phones. Pretty sure this is due to the use of Sphinx' scale
:
.. figure:: _static/image/background_correction/dynamic_correction.jpg
:align: center
:scale: 50%
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Width's should be fixed, e.g. as in HyperSpy's documentation (http://hyperspy.org/hyperspy-doc/current/user_guide/signal2d.html).
Screenshots
These were acquired with the same browser width...
After discussion with Jarle Hjelen, related to #12.
The pattern intensity on boundaries are usually higher than within grains because of pattern overlap. We could use this to get a binary grain boundary map by:
Some thoughts/caveats:
When performing lazy decomposition it should be possible to only save the learning results. However, when I performed a lazy decomposition on a 15 GB float32
(I have 16 GB RAM) dataset using IncrementalPCA
and tried this, a MemoryError
was raised. The learning results were around only 600 MB...
Have to look into HyperSpy's s.learning_results.save()
to see where unecessary stuff was written to memory.
Is your feature request related to a problem? Please describe.
EMsoft's wealth of utility functions in Fortran-90 are available via Python wrappers. See https://github.com/EMsoft-org/EMsoft/tree/develop/Source/pyEMsoft and https://github.com/EMsoft-org/EMsoft/wiki/python-examples. We should start to take advantage of this.
This issue is opened to facilitate a discussion of (or just my thoughts on) pyEMsoft and how we can use it.
Describe the solution you'd like
We could already start using the routines available in crystal.f90
(https://github.com/EMsoft-org/EMsoft/blob/develop/Source/pyEMsoft/docs/pyEMsoft.rst) for creating phases and including them in the metadata.
Describe alternatives you've considered
It might be benefitial to rely on pyEMsoft instead of diffpy.structure (as pyXem does) or pymatgen for crystal structures and phases, since users then only need to be familiar with one way of doing things.
Additional context
Another benefit of starting to use pyEMsoft now is that it will hopefully be easier to use other parts of EMsoft's functionality when they become available via Python wrappers.
I've started using Marc DeGraef's EMsoft to simulate dynamical EBSPs with a plan to use EMsoft's dictionary indexing to index EBSPs. As far as I can tell, all output files from EMsoft are in a HDF5 file format, so it is easy to visualise in Python, see e.g. the nice master patterns from the EMsoft wiki. DeGraef mentions in the the latest ReadMe.md:
Some of our developers told us they have been working on python wrappers for EMsoft !!! If they are willing to make these available to us, then we will make sure they become part of one of the next releases.
But our use can be complementary to theirs.
Initial ideas:
I think classes for the different data (master pattern, distribution of BSEs on detector after Monte Carlo simulations, simulated dynamic EBSPs) is the best approach. Will look into this after a successful dictionary indexing.
Oh boy, we cannot wait too long before writing tests for it all. Shouldn't be too much work if we just look at what pyXem and HyperSpy does.
get_decomposition_model()
for large datasets (perhaps lazily?)map()
atm)Let's update this list as we go along.
I'm working on the task(s) in bold.
Describe the bug
Dask throws this warning when iterating over the data matrix using IncrementalPCA.partial_fit(X[start:end])
FutureWarning: The `numpy.may_share_memory` function is not implemented by
Dask array. You may want to use the da.map_blocks function or something similar
to silence this warning. Your code may stop working in a future release.
So, we should update the IncrementalPCA object by calling da.map_blocks
on appropriate chunks. We loose the tqdm progressbar, but can use Dask's own instead. Should put some description for the progressbar, since two will pop up from getting factors and then loadings.
To Reproduce
Perform decomposition on lazy signal with s.decomposition(algorithm='IPCA', output_dimension=1)
.
We should create a reader/writer plugin for the standard .h5ebsd HDF5 format reported by Jackson et al.. According to EMsoft's dictionary indexing manual this format is used by them and also EDAX/TSL. It would be good to think of this when implementing readers for varias EMsoft data #25.
Describe the bug
The HyperSpy extension YAML file hyperspy_extension.yaml
is not included when installing from the source distribution on PyPI for v0.1.1. Thus, the following won't work:
>>> import hyperspy.api as hs
>>> import numpy as np
>>> s = hs.signals.Signal2D(np.zeros((10, 10, 10, 10)))
>>> s.set_signal_type('EBSD')
To Reproduce
>>> import kikuchipy as kp
Failed to load hyperspy extension from kikuchipy. Please report this issue to the
kikuchipy developers
Expected behavior
The extension file should be included in site-packages/kikuchipy/
when installing with pip
.
Additional context
With this fixed, hopefully, tests launched from the PR to the conda-forge/staged-recipies
repo will pass (conda-forge/staged-recipes#10523).
Make a simple Jupyter notebook showcasing the functionality available in KikuchiPy:
I noticed some new errors when loading a hdf5 dataset with hs and then converting it to kp.EBSD signal. I also removed dead pixels before proceeding to the background removal, and did not do it inplace, so the error might lie there too.
s = hs.load('Pattern.hdf5', lazy = False)
s = kp.EBSD(s)
deadpixels = s.find_deadpixels(mask=mask, pattern_number=30, to_plot=True)
s_rd = s.remove_deadpixels(deadpixels=deadpixels, inplace=False)
For completion, this is what the metadata looks like:
s_rd.metadata
Out[11]:
├── General
│ └── title =
└── Signal
├── binned = False
└── signal_type = electron_backscatter_diffraction
s_rd.original_metadata
Out[12]:
└── Acquisition_instrument
└── SEM
└── Detector
└── EBSD
├── deadpixels = array([[ 6, 156],
[ 28, 4],
[ 28, 150],
[ 38, 20],
... , 130],
[180, 107],
[193, 26],
[222, 91]], dtype=int64)
├── deadpixels_corrected = True
└── deadvalue = average
s_rd.remove_background(static=True, dynamic=True)
line 196, in remove_background
bg = os.path.join(omd.General.original_filepath, bg_fname)
AttributeError: 'DictionaryTreeBrowser' object has no attribute 'General'
s_rd.remove_background(static=True, dynamic=True, bg = 'Background acquisition pattern.bmp')
line 207, in remove_background
if omd.deadpixels_corrected and omd.deadpixels and omd.deadvalue:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Would be nice to have functions to assess this for at least decomposition()
and get_decomposition_model_write()
.
dask.array
does not have a tofile()
function as numpy
has, so calling
for pattern in signal._iterate_signal():
pattern.flatten().tofile(f)
as is done in file_writer()
in nordif.py
does not work. Calling compute()
on each pattern before calling tofile()
is not feasible.
A NORDIF .dat file is one long byte string, so using HyperSpy's block_iterator()
can only be used if chunks comply with this requirement (chunks with the full horizontal row of patterns, however many rows can be in the same chunk [I think]).
It might be that dask.array.store
can write to a binary file? Will look into this.
Is your feature request related to a problem? Please describe.
Dictionary indexing with EMsoft requires knowledge of an average pattern centre (xpc, ypc, zpc) for the data set (see e.g. https://link.springer.com/article/10.1007/s40192-019-00137-4). If two patterns with the same Kikuchi bands are acquired with a known increase in the specimen-scintillator distance between the two, (xpc, ypc) can be determined following an approach by Hjelen et al. (http://dx.doi.org/10.1155/TSM.20.29 and https://doi.org/10.1016/0739-6260(91)90128-M).
Describe the solution you'd like
Ideally, a confidence/error in the choice of (xpc, ypc) should be provided.
Is your feature request related to a problem? Please describe.
A convenience method to bin data along the navigation axes, i.e. removing e.g. every other pattern along x and y, would be nice to have when e.g. a data set is large.
Describe the solution you'd like
A method called perhaps thin()
or trim()
should accept parameters called perhaps xstep
and ystep
stating which patterns to remove (e.g. xstep=2
removes every other pattern from the x-direction). The distance between each pattern must be the same for all patterns in each direction. Step sizes in axes_manager
must of course be updated.
Describe alternatives you've considered
This should work
import numpy as np
original = np.arange(25).reshape(5, 5)
xstep = 2
ystep = 2
thinned = original[::ystep, ::xstep]
I am not sure what is the most cost effective way in terms of memory and speed.
Is your feature request related to a problem? Please describe.
We should register KikuchiPy as a HyperSpy extension following their user guide (http://hyperspy.org/hyperspy-doc/current/dev_guide/writing_extensions.html#registering-extensions) and e.g. how pyXem does it (https://github.com/pyxem/pyxem/blob/master/pyxem/hyperspy_extension.yaml).
Describe the solution you'd like
Adding a hyperspy_extension.yaml file in kikuchipy/kikuchipy listing all new HyperSpy objects and add entry_points={'hyperspy.extensions': 'your_package_name = your_package_name'}
in setup.py
.
Additional context
Doing this might raise awareness of KikuchiPy, both so that it is used more and so to avoid duplication of efforts.
Is your feature request related to a problem? Please describe.
So far, writing to the NORDIF binary format just creates a Pattern.dat
binary file. However, all relevant files created by the NORDIF acquisition software should be created upon calling save()
.
Describe the solution you'd like
What also should be created upon save()
are:
Setting.txt
with relevant information from metadataBackground acquisition.bmp
. Unless the user supplies this themselves, we should, in prioritised order: (1) use the one in signal metadata (bin if necessary), (2) create it the average of all patterns with s.mean()
.[(x1, y1), (x2, y2), ...]
to save()
. These should be bmp
files.Area ROI.bmp
. This then would typically be a virtual backscatter electron image. We can be passed to save()
, but we should have a default if not supplied.Describe alternatives you've considered
A program that updates the "Number of samples" etc. is available here: https://github.com/hwagit/nordif2hdf5/blob/master/update_nordif_setting_file.py
Before dask version 2.8.1
(see their changelog), chunks
was a required parameter (now the default is chunks='auto'
). However, chunks
is not passed internally in static_background_correction()
, leading to TypeError
.
Solution: Set dask >= 2.8.1
in setup.py
to fix this.
>>> import kikuchipy as kp
>>> s = kp.load('/some/data/with/background/Pattern.dat')
>>> s.static_background_correction()
static_background_correction(self, operation, relative, static_bg)
404 ebsd_node = kp.util.io.metadata_nodes(sem=False)
405 static_bg = da.from_array(
--> 406 md.get_item(ebsd_node + ".static_background")
407 )
408 except AttributeError:
TypeError: from_array() missing 1 required positional argument: 'chunks'
Would be nice to be able to install KikuchiPy without having to install Visual Studio C++ compilers on Windows... As far as I know, HyperSpy doesn't need it, although pyXem does.
Problem: When calling ElectronBackscatterDiffraction(s)
on a LazySignal
loaded with hs.load(file, lazy=True)
, the signal is cast to Signal2D
and thus written to memory.
pyXem has its own load function, will look into loading signals as they do.
Is your feature request related to a problem? Please describe.
We need a branching model which eases releasing new features and patches.
Describe the solution you'd like
After considering HyperSpy's three RELEASE_next_major/minor/patch
branches, and the branching model of pyXem/scikit-image/etc. with a master
and version branches, I like a version of the latter: https://dev-cafe.github.io/branching-model/. Everything is nicely explained in that link, with a link at the start to semver.org. In short, from the link above:
Branch semantics
1. master
collects changes towards the next major release.
2. release/X.Y
branches collect changes towards the next minor release.
3. release/X.Y.Z
branches collect changes towards the next patch release.
4. New features are directed either towards master
or towards release/X.Y
branches.
5. Patch release branches release/X.Y.Z
never receive new features, they only receive bugfixes.
Bugfixes
1. Bugfixes can be directed either towards release/X.Y.Z
, or release/X.Y
, or master
, depending on the intent.
2. Bugfixes directed towards release/X.Y.Z
require to bump the patch number, signalled by creating a new tag.
3. Important bugfixes in a given release/X.Y.Z
can, if necessary, be ported to release/X.Y
and further to master
by merging.
4. Important bugfixes in master
can, if necessary, be ported to release/X.Y
and release/X.Y.Z
by cherry picking.
The above link should be included in the contributing guide, along with explanations of which branch to branch off of for features/bugfixes for contributors.
HyperSpy makes it possible to mask nav/signal space using its block_iterator
for lazy signals. I did not use this in my implementation of IncrementalPCA
... This should of course be possible. Will look into this sometime in the future.
We should use logging to track what our code does. Some example usages from HyperSpy:
def normalize_poissonian_noise(self, navigation_mask=None,
signal_mask=None):
[...]
_logger.info(
"Scaling the data to normalize the (presumably)"
" Poissonian noise")
def file_reader(filename, endianess='<', mmap_mode=None,
lazy=False, **kwds):
_logger.debug("Reading blockfile: %s" % filename)
[...]
try:
header['Note'] = note.decode("latin1").strip("\x00")
except BaseException:
# Not sure about the encoding so, if it fails, we carry on
_logger.warning(
"Reading the Note metadata of this file failed. "
"You can help improving "
"HyperSpy by reporting the issue in "
"https://github.com/hyperspy/hyperspy")
_logger.debug("File header: " + str(header))
I think this tutorial is a good place to start to both understand logging and use it correctly: https://realpython.com/python-logging/.
The benefit of having a robust radon transform of patterns is clear. If not for detecting bands (I cannot so far see the benefit of developing or implementing an indexing algorithm in KikuchiPy), than perhaps for background removal like (Schwarzer and Sukkau, 2012: https://www.sciencedirect.com/science/article/pii/S0167865511004260)? Virtual imaging in the radon space (@tinabe's idea) might also prove useful.
The work should build upon the work already done in the radon-transform
branch.
Is your feature request related to a problem? Please describe.
Should have a reader/writer of patterns stored in Oxford Instruments' HDF5 file specification (https://github.com/oinanoanalysis/h5oina), H5OINA, based on the h5ebsd format.
Describe the solution you'd like
We should first create a reader following the above linked specification. If any changes are made to the specification in the future, these should be incorporated as appropriate. A writer might not be strictly necessary, until other softwares appart from Oxford's implement a similar reader.
Describe alternatives you've considered
The reader/writer should be implemented as part of the existing h5ebsd
reader, which already reads Bruker Nano's and EDAX TSL's HDF5 file formats. It would be best to get a hold of a H5OINA file to ensure that the reader is correct.
The NORDIF file reader doesn't treat correctly the case where no Setting.txt file is provided but the user inputs scan size and pattern size.
Static background correction with operation='divide'
clips intensities since patterns are cast to int16
instead of float16
.
>>> import kikuchipy as kp
>>> import numpy as np
>>> s = kp.load('some/data/with/background_pattern')
>>> s.static_background_correction(operation='divide')
>>> s.plot() # Rubbish, clipped intensities
Not relevant.
Should of course cast to float16
or float32
when dividing... All use of data types should be managed generally, allowing the user to input patterns in whatever data type they want. Simplest solution would be to cast to float64
, however, this might give memory problems unless Dask is used properly.
Is your feature request related to a problem? Please describe.
Maps acquired at high stage tilt and slow scan speed (most scans?) can suffer from tilt and drift distortions.
Describe the solution you'd like
Loading reference images with a feature recognised in a virtual image or orientation map derived from the distorted scan... And do as Tong and Britton explains in their paper.
Describe alternatives you've considered
They only correct orientation maps afterward... If a distortion field is obtained, could this be applied in some way to experimental patterns, e.g. using averaging to some extent?
Additional context
Source is the above referenced paper: https://arxiv.org/pdf/1909.00347.pdf
We need a reader/writer for patterns stored as images in a directory, with names explaining where in the scan they are situated, like "x0y0.jpg", "x0y86.jpg" etc. if the step size is 86 nm. The only necessary user input should be a filename prefix/postfix, a step size and the image extension/format. All experimental parameters etc. must of course be added to the signal after reading.
Describe the bug
scan_size
is overwritten in the NORDIF file_reader()
method.
To Reproduce
import kikuchipy as kp
s = kp.load('/path/to/Pattern.dat', scan_size=(3, 3))
print(s.shape) # Will not return (3, 3) patterns if other info is in Setting.txt file
Expected behavior
Patterns in NORDIF Pattern.dat file should be reshaped into a (3, 3) navigation dimension, even though the Setting.txt file says something else.
So s.remove_background()
calls kikuchipy.utils.expt_utils.rescale_pattern_intensity
which stretches the contrast in patterns 'locally' (each pattern to [0, 255]) or 'globally' (min. and max. intensity in pattern stack to [0, 255]). The latter maintains relative intensities.
I was initially hesitant to also apply so-called adaptive histogram equalization (http://scikit-image.org/docs/dev/auto_examples/color_exposure/plot_equalize.html), which equalizes intensities within tiles/windows in each pattern to [0, 255], because:
scikit-image
(docs: http://scikit-image.org/docs/dev/api/skimage.exposure.html#skimage.exposure.equalize_adapthist, source: https://github.com/scikit-image/scikit-image/blob/master/skimage/exposure/_adapthist.py) calls rescale_intensity
, however patterns are already rescaled in rescale_pattern_intensity
! This does nothing and adds computation time.However, I see that De Graef and co-workers (https://doi.org/10.2138/am-2017-6062) use adaptive histogram equalization to great effect... Ergo, this is useful. I see three possibilities:
map()
and we don't implement anythingscikit-image
to do just what we need and nothing mores.remove_background()
, so only equalize_adapthist
is called by us (which again calls rescale_intensity
)?Is your feature request related to a problem? Please describe.
Detector pixel size is needed for dictionary indexing with EMsoft (see e.g. https://link.springer.com/article/10.1007/s40192-019-00137-4).
Describe the solution you'd like
Convenience function to measure the detector pixel size from features with a known distance in an "EBSD pattern" (image). This can e.g. be a mm grid.
Describe alternatives you've considered
It can be argued that this might fall outside of what problems KikuchiPy's should solve, however using HyperSpy has some nice measurement tools etc. to do this... I myself have a notebook (https://nbviewer.jupyter.org/github/hwagit/detector-pixel-size-ebsd-camera/blob/master/detector_pixel_size.ipynb) to determine pixel size "manually" from an image of a mm grid, but it would be nice to have this more automated an reproducible via KikuchiPy.
After performing decomposition on an EBSD signal, the signal type changes to Signal2D. Then the methods for EBSD signals are not available. If changing the signal to EBSD after the decomposition, the learning results are not found, and methods requiring learining results cannot be used. This is an issue, since then it is not possible to utilize e.g. the classify_decomposition_components() method.
s
<EBSD, title: , dimensions: (10, 10|240, 240)>
s.decomposition(algorithm='svd', output_dimension=10)
s
<Signal2D, title: , dimensions: (10, 10|240, 240)>
s = kp.EBSD(s)
components = s.classify_decomposition_components()
<ValueError: No learning results were found.>
Performing background corrections using one full static/dynamic background pattern on patterns acquired with detectors with unbalanced exposure in the two CCD chip halves can have undesirable effects, as discussed by @wiai in the docs of aloe/xcdskd
(https://xcdskd.readthedocs.io/en/latest/pattern_processing/kikuchi_pattern_processing.html#Examples-of-Detector-Effects).
@tmcaul has a solution for the dynamic background pattern in https://github.com/tmcaul/ebspy here: https://github.com/tmcaul/ebspy/blob/master/ebspy/ebspy.py#L185. He has generously said we can implement his solution, which was first implemented in AstroEBSD
(https://github.com/benjaminbritton/AstroEBSD/blob/b9a080648d7cda8977803a78c6749f56992b134d/bin/EBSP_BGCor.m#L145). Both ebspy
and AstroEBSD
should be cited in the docstring and user guide.
How we will do it for the static correction, I haven't thought about yet.
Describe the bug
Calling s = kp.load('/path/to/Pattern.hdf5')
raises:
Traceback (most recent call last):
[...]
s = kp.load(data)
File "/home/hakon/kode/kikuchipy/kikuchipy/io.py", line 159, in load
for filename in filenames]
File "/home/hakon/kode/kikuchipy/kikuchipy/io.py", line 159, in <listcomp>
for filename in filenames]
File "/home/hakon/kode/kikuchipy/kikuchipy/io.py", line 194, in load_single_file
return load_with_reader(filename=filename, reader=reader, **kwargs)
File "/home/hakon/kode/kikuchipy/kikuchipy/io.py", line 231, in load_with_reader
objects.append(dict2signal(signal_dict, lazy=lazy, axis=axis))
File "/home/hakon/kode/kikuchipy/kikuchipy/io.py", line 294, in dict2signal
lazy=lazy)(**signal_dict)
File "/home/hakon/kode/kikuchipy/kikuchipy/io.py", line 382, in assign_signal_subclass
and s._signal_type == ''][0]
IndexError: list index out of range
since none of our classes have a blank _signal_type
attribute, hence the list we try to get index 0 of is empty. Because the .hdf5 file has no metadata explaining if it is a RadonTransform
or an EBSD
data set, and both of these classes have the same _signal_dimension
and _signal_type
, we cannot distinguish between them in assign_signal_subclass()
.
To Reproduce
import kikuchipy as kp
kp.load('/path/to/Pattern.hdf5') # With no metadata
Suggested solution
signal_type
to load()
like so: kp.load('/path/to/Pattern.hdf5', signal_type='electron_backscatter_diffraction')
(or signal_type='EBSD'
, as EBSD
will be available as a _alias_signal_types
when I push the hotfix commit for this problem). Or signal_type='RadonTransform'
if applicable.signal_type='EBSD'
as the default when loading .hdf5 files without necessary metadata. In HyperSpy, the default is Signal1D
or Signal2D
.Additional context
This is no problem when loading from a .hdf5 or .hspy file with metadata with the signal_type
parameter.
Patterns stored in EDAX and Bruker's h5ebsd files are not chunked (at least the ones I've come across), thus when trying to read lazily, this error message is produced:
Traceback (most recent call last):
File "/home/hakon/miniconda3/envs/kp-test/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2712, in safe_execfile
self.compile if shell_futures else None)
File "/home/hakon/miniconda3/envs/kp-test/lib/python3.7/site-packages/IPython/utils/py3compat.py", line 168, in execfile
exec(compiler(f.read(), fname, 'exec'), glob, loc)
File "/home/hakon/.PyCharmCE2019.3/config/scratches/kp_test.py", line 18, in <module>
s = kp.load(data, lazy=True)
File "/home/hakon/kode/kikuchipy/kikuchipy/io/_io.py", line 77, in load
scan_dicts = reader.file_reader(filename, lazy=lazy, **kwargs)
File "/home/hakon/kode/kikuchipy/kikuchipy/io/plugins/h5ebsd.py", line 139, in file_reader
scan_dict_list.append(h5ebsd2signaldict(scan, man, ver, lazy=lazy))
File "/home/hakon/kode/kikuchipy/kikuchipy/io/plugins/h5ebsd.py", line 315, in h5ebsd2signaldict
data = da.from_array(data_dset, chunks=data_dset.chunks)
File "/home/hakon/miniconda3/envs/kp-test/lib/python3.7/site-packages/dask/array/core.py", line 2725, in from_array
chunks, x.shape, dtype=x.dtype, previous_chunks=previous_chunks
File "/home/hakon/miniconda3/envs/kp-test/lib/python3.7/site-packages/dask/array/core.py", line 2402, in normalize_chunks
raise ValueError(CHUNKS_NONE_ERROR_MESSAGE)
ValueError: You must specify a chunks= keyword argument.
This specifies the chunksize of your array blocks.
The relevant lines are (as evident from the traceback) in h5ebsd.py
:
if lazy:
data = da.from_array(data_dset, chunks=data_dset.chunks)
>>> import kikuchipy as kp
>>> s = kp.load('/some/edax/or/bruker/h5ebsd/file', lazy=True)
[...] # See above error message
ValueError: You must specify a chunks= keyword argument.
This specifies the chunksize of your array blocks.
Not relevant.
Should of course assign chunks to data sets if they don't have them already...
In addition to fix this, tests reading lazily from EDAX and Bruker h5ebsd files must be added.
Another way of reducing noise in patterns than creating model patterns from decomposition factors is by simply averaging a pattern with the adjacent patterns. This can be done by e.g. using scipy.ndimage.filters.generic_filter
and passing numpy.mean
as the function to apply to each pattern. There are (at least) two problems to address:
map()
since we have to also access the adjacent patternsA s.average_patterns(num_neighbours=1)
function would be nice!
Using GitHub Pages https://pages.github.com/.
Contents:
Create functions (or a dedicated VFSD class) to streamline formation of VFSD images. Desired functionality:
Focus should be on maximising the contrast in each image, by e.g. using histogram equalization (http://scikit-image.org/docs/dev/auto_examples/color_exposure/plot_equalize.html). The latter is related to #14.
Is your feature request related to a problem? Please describe.
Would be nice to be able to reader patterns from Bruker's composite file (.bcf) format.
Describe the solution you'd like
An IO plugin like for the NORDIF and h5ebsd Bruker and KikuchiPy formats. At least the patterns, but should also read the metadata.
Describe alternatives you've considered
HyperSpy already has a reader for Bruker's .bcf file format containing EDS data, should look into how the file is read there. Relevant links:
Should support Python 3.8 when the conda-forge
Python 3.8 migration is done.
Test in builds on Travis CI.
EBSD method to calculate similarity between each pattern in a scan (preferably with mask, most likely this feature will be implemented later) and its neighbours. Which neighbours should preferably be up to the user to decide, should start with 1st, 2nd etc. in either a cross or square. This will be similar to EMsofts average dot product (ADP) map (their implementation https://github.com/EMsoft-org/EMsoft/blob/7762e1961508fe3e71d4702620764ceb98a78b9e/Source/DictionaryIndexing/EMgetADP.f90#L360). The coefficient map can then be used e.g. to look at (how nice!), or use as input to a pattern averaging method so the similarity can be used as a weight in averaging.
Similarity metrics we should support:
This will be slow, however, let's start with the correct, naive implementation first and improve from there.
Have to wait for #15.
To do:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.