Package structure about lumispy HOT 12 CLOSED

I completely agree. I will migrate the load_hypcard() function to the new io_plugins structure, when I have some time.
EDIT: Added to Projects

from lumispy.

@jlaehne I have looked into the Hyperspy way of loading files and it seems unclear to me on how to modify the load function.

As I understand, the io.py file (which contains the generic hs.load() function reads through all the io_plugins/*.py files looking for the right extension. If it finds it, then it runs the .file_reader() function specific to the manufacturer.

The issue I have is: how can I "inherit" the io functions (eg. hs.load) from Hyperspy, adding the specific luminescence providers? The only idea I have is to create a lum.load() function which searches for the luminescence specific file types first, and then if it doesn't find them, it runs the hs.load() function.

Is this the right way to go?
If so, I will write the code then :-)

from lumispy.

I am not so sure, because the IO Plugins of Hyperspy will become an external package in the future.

@ericpre, @francisco-dlp: Is there a predefined way if a hyperspy extension wants to define additional IO routines? Is there a way to inherit, should we define them upstream so that they will become part of the new IO package in the future?

from lumispy.

I have managed to find a way for now.
I basically inherit the hs.io.load() function as lum.io.load() but the only difference is that I redefine the hs.io.load_single_file() method for a lumispy specific one which first checks for io_plugins files in lumispy, and if it doesn't find any, then it runs the normal hs.io.load_single_file() from hyperspy.

For now it does the job perfectly, supporting any new file type in the lumispy/io_plugins but also supporting all the hyperspy io_plugins too.

I will pull request today.

from lumispy.

I think that it is preferable not to hack hyperspy's load function à la pyxem. Instead just contribute the IO plugin to hyperspy. As mentioned above, the plan is to split the io plugins into a separate package, RosettaScIO, see hyperspy/hyperspy#2174 . If that was not possible, could you explain why? Alternatively we could consider enabling the addition of IO plugins using the extensions mechanism. However, we decided in the past not to do this in order to motivate developers to create IO plugins that are not tight to a particular software ecosystem (hyperspy in this case) so that they can be easily used by other packages. So, to change our mind in this regard we'll need some good arguments.

from lumispy.

For now I have managed to "monkey patch" the load function as a short term solution to this issue.

However, this restructuring makes the process of transporting this code to hyperspy easier, as it is following the same structure as hyperspy. I will attempt so in the near future.

@francisco-dlp The main issue I am facing is that it doesn't seem clear to me what exact dictionary the load_single_file() needs to return to the load_with_reader() hyperspy function. Besides, this load_with_reader() function calls the dict2signal() function, which I assume it can only create signal classes defined in Hyperspy... so eventually I would need to "path" one of these functions to do that in lumispy.

from lumispy.

@jordiferrero #20 looks like a very sensible approach to me to implement the functionality here and then port it upstream to contribute it to hyperspy (thus RosettaScIO in the future).

Well, LumiSpy is registering its signals with HyperSpy (http://hyperspy.org/hyperspy-doc/current/dev_guide/writing_extensions.html), so when both are loaded, the HyperSpy functions should be able to access the LumiSpy signals

from lumispy.

I think that the simplest approach is to implement it in hyperspy branch straight away and pull request it as soon as you have a prototype. You can always work with hyperspy in development mode until the code is released. In this way you'll avoid dealing with the complexity of hacking hyperspy's load function and you won't force your users to amend theirs scripts once you move the code to hyperspy/RosettaScIO. We will probably release v1.6 in the next few weeks, so this is a good time to contribute it if you care about it making into a release soon.

from lumispy.

@jlaehne @francisco-dlp OK. That all makes sense. I will try to contribute to Hyperspy branch then, as an extension of io_plugins.
However, I still have this issue before proceeding: it doesn't seem clear to me what exact dictionary the load_single_file() needs to pass to the load_with_reader() hyperspy function.

Right now, I have my personal loading function for the Attolight system (to be added in the io_plugins) which returns a CLSEMSpectrum object (not a dictionary), with its axis, metadata and parameters. If I pass this Lumispy class as the load_with_reader() parameter, I run into issues.

from lumispy.

That's right, the reader must not import any of hyperspy's functions. Instead it returns a dictionary. That's not documented, but you can get a good idea of the specification by looking for example at the code of the ripple plugin. Then add it to this list and it'll be done.

from lumispy.

Unfortunately, there are very limited documentation on writing io plugins but there are many examples which are more or less complicated/sophisticated so it is definitively worth looking at them.

from lumispy.

Following from the discussion, I have created a PR in the main Hyperspy package for the next minor release hyperspy/hyperspy#2380. The loading of the data works perfectly, yet there is an issue with importing the external lumispy classes... which I don't know how to solve.

from lumispy.