Git Product home page Git Product logo

pysap's Introduction

CI Codecov Python36 Python37 Python38 Python39 PyPi Doc Docker

PySAP

PySAP (Python Sparse data Analysis Package) is a Python module for sparse data analysis that offers:

  • A common API for astronomical and neuroimaging datasets.
  • Access to Sparse2D executables with both wrappers and bindings.
  • Access to application specific plug-ins.
  • A graphical user interface to play with the provided functions.

This package is the result of the COSMIC project, which is a collaboration between the CEA Neurospin UNATI and CEA CosmoStat laboratories.

Important links

Dependencies

PySAP will automatically install all of the required dependencies, however issues may arise on some operating systems. If you encounter any problems please ensure that you have all of the following dependencies installed before opening a new issue.

  1. PySAP requires that the COSMIC package ModOpt be installed.
  1. PySAP also requires the installation of the following third party software packages:
  • scipy [>=1.5.4]
  • numpy [>=1.19.5]
  • matplotlib [>=3.3.4]
  • astropy [>=4.1]
  • nibabel [>=3.2.1]
  • pyqtgraph [>=0.11.1]
  • progressbar2 [>=3.53.1]
  • scikit-learn [>=0.24.1]
  • pybind11 [==2.6.2]
  • pyqt5 [==5.15.4]
  • PyWavelets [>=1.1.1]

Plug-Ins

PySAP currently supports the following plug-ins:

Installation

The installation of PySAP has been extensively tested on Ubuntu and macOS, however we cannot guarantee it will work on every operating system. A Docker image is available (see below) for those unable to install PySAP directly.

If you encounter any installation issues be sure to go through the following steps before opening a new issue:

  1. Check that that all the dependencies listed above have been correctly installed.
  2. Read through all of the documentation provided, including the troubleshooting suggestions.
  3. Check if your problem has already been addressed in a previous issue.

Further instructions are available here.

From PyPi

To install PySAP simply run:

$ pip install python-pysap

Depending on your Python setup you may need to provide the --user option.

$ pip install --user python-pysap

Locally

To build PySAP locally, clone the repository:

$ git clone https://github.com/CEA-COSMIC/pysap.git

and run:

$ pip install .

or:

$ python setup.py install

As before, use the --user option if needed.

Custom Installation

The following options can be passed when running python setup.py install:

  • --noplugins : Install PySAP without any plug-ins
  • --only=<PLUG-IN NAME> : Install PySAP with only the specified plug-in names (comma separated)
  • --nosparse2d : Install PySAP without building Sparse2D

For example, to install PySAP with only the Etomo plug-in and without Sparse2D you would run the following.

$ python setup.py install --nosparse2d --only=pysap-etomo

Note that these options can also be invoked when installing with pip using the --install-option="<OPTION>" option.

$ pip install . --install-option="--noplugins"

However, this will disable the use of wheels and make take significantly longer to build all of the dependencies. Therefore, when installing PySAP this way it is recommended to pre-install all the required dependencies or use the Conda environment provided.

Conda Environment

A a conda environment.yml file is provided to facilitate the installation of the required PySAP dependencies along with some optional dependencies that provide additional features. To build the environment run:

$ conda env create -f environment.yml

Then to activate the environment run:

$ conda activate pysap

Finally, install PySAP following the instructions above.

Docker Image

A PySAP Docker image is available via DockerHub that includes the latest stable version of PySAP pre-installed. To install the image run:

$ docker pull ceacosmic/pysap

A Jupyter notebook can be launched using the Docker image as a backend and with access to the users current working directory as follows:

$ docker run -p 8888:8888 -v ${PWD}:/home ceacosmic/pysap

macOS

Help with installation on macOS is available here.

Linux

Please refer to the PyQtGraph homepage for issues regarding the installation of pyqtgraph.

Windows

Help with installation on Windows on this Gist.

Contributing

If you want to contribute to pySAP, be sure to review the contribution guidelines and follow to the code of conduct.

Citation

If you use PySAP in a scientific publication, we would appreciate citations to the following paper: PySAP: Python Sparse Data Analysis Package for multidisciplinary image processing, S. Farrens et al., Astronomy and Computing 32, 2020

The bibtex citation is the following: k:

@Article{farrens2020pysap,
  title={{PySAP: Python Sparse Data Analysis Package for multidisciplinary image processing}},
  author={Farrens, S and Grigis, A and El Gueddari, L and Ramzi, Z and Chaithya, GR and Starck, S and Sarthou, B and Cherkaoui, H and Ciuciu, P and Starck, J-L},
  journal={Astronomy and Computing},
  volume={32},
  pages={100402},
  year={2020},
  publisher={Elsevier}
}

pysap's People

Contributors

agrigis avatar bensarthou avatar chaithyagr avatar hanaecarrie avatar hcherkaoui avatar jeremiedecock avatar jstarck avatar lelgueddari avatar paquiteau avatar philouc avatar sfarrens avatar starcksophie avatar sylvainlan avatar zaccharieramzi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pysap's Issues

[INSTALLATION ERROR]

System setup
OS: [e.g] Linux Ubuntu 18.04 LTS
Python version: [e.g.] v3.6.8
Python environment (if any):

Describe the bug
Problem on the binding of sparse2d with pybind11 at the installation step.
pysparse.cpp:17:16: error: expected constructor, destructor, or type conversion before ‘(’ token
PYBIND11_MODULE(pysparse, module)

To Reproduce
• pip3 install modopt
• export CC=/usr/bin/gcc
• export CXX=/usr/bin/g++
• git clone https://github.com/CEA-COSMIC/pysap.git
• sudo apt install pybind11-dev
• python3 setup.py develop –user (failed up to now)

Screenshots
See attached

Are you planning to submit a Pull Request?

  • Yes
  • No
    Capture d’écran de 2019-08-05 11-04-30

scapy version download error for python3

I am trying to install pysap for python3.6 directly through pip install for my anaconda python

When trying to pip install pysap, I get the following error message

Collecting pysap
Using cached pysap-0.1.14.zip
Collecting scapy==2.3.3 (from pysap)
Using cached scapy-2.3.3.tgz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-j1kx12h6/scapy/setup.py", line 36
os.chmod(fname, 0755)
^
SyntaxError: invalid token

I checked with python2, this syntax is valid but for python3 it should be
os.chmod(fname, 0o755)

For python3 we should be installing scapy3 I guess.

Wrong size of kspace_data when using NFFT and p_MRI

I tried to use the NFFT on p MRI reconstruction I got this issue :

  File "p_MRI_noncartesian_reconstruction.py", line 88, in <module>
    S=Smaps)
  File "/home/le251198/ProximalAlgorithm/Condat-Vu/pisap/pisap/plugins/mri/parallel_mri/gradient.py", line 206, in __init__
    S)
  File "/home/le251198/ProximalAlgorithm/Condat-Vu/pisap/pisap/plugins/mri/parallel_mri/gradient.py", line 123, in __init__
    self.get_spec_rad(extra_factor=1.1)
  File "/home/le251198/.local/lib/python2.7/site-packages/modopt/math/matrix.py", line 321, in get_spec_rad
    x_new = self._operator(x_old) / x_old_norm
  File "/home/le251198/.local/lib/python2.7/site-packages/modopt/opt/gradient.py", line 194, in trans_op_op
    return self.trans_op(self.op(data))
  File "/home/le251198/.local/lib/python2.7/site-packages/modopt/base/wrappers.py", line 49, in wrapper
    return func(*args)
  File "/home/le251198/ProximalAlgorithm/Condat-Vu/pisap/pisap/plugins/mri/parallel_mri/gradient.py", line 146, in _synth_op_method
    rsl[:, :, l] = self.fourier_op.op(self.S[:, :, l] * img)
IndexError: too many indices for array

This is due to the fact that the shape of the result of the fourier_op.op depends on the fourier operator that we are using it (can be the same size of the image in the cartesian case or a huge vector for the NFFT case)

https://github.com/neurospin/pisap/blob/795f37849b8459c682f366e05f154491fa211020/pisap/plugins/mri/parallel_mri/gradient.py#L145

This has to be fixed in all the parallel_mri/gradient.py file

__setitem__ to enable operations on coefficients

Allow user to alter coefficients in a specific band/scale.
e.g.

wavelet=WaveletTransformBase.REGISTRY['HaarWaveletTransform'](nb_scale=5,verbose=1)
wavelet.data=<<myimageindirectspace>>
wavelet.analysis()
wavelet[2,2]*=0  # put band 2 of scale 2 to 0.

PySAP can't find plugin modules

@AGrigis unfortunately it looks like PySAP is still have issues finding the plugins. I get the following after building the current master.

python -c "from pysap.plugins import mri"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: cannot import name 'mri'

Checking the installation, pip freeze clearly shows

pysap-astro==0.0.0
pysap-mri==0.0.0

Also, oddly, running (inside PluginsMetaImportHook.find_module)

print(importlib.util.find_spec(self.sub_name, path))

works without any issues.

Wrong wavelet adjoint operator

The adjoint operator of the wavelet transform operator is not correct in all cases. Namely, when using pywt wavelets, we have redundant wavelet transforms (currently hardcoded to be so because of #79 ). In this case, wavelets are not orthogonal and therefore the adjoint is not the inverse anymore.

For the case of isap it is also wrong but it might be because of accuracy issues cf #53 .

This paper by Folberth and Becker shows how to compute the wavelet tranform adjoint. The code is available in Matlab. The operator wextend might become available in pywt under the name pad after this issue is taken care of. It is however more or less already available.

Also remember that we will have to compute 2 adjoints: one for the decomposition and one for the reconstruction (since the inverse is not the adjoint of the decomposition, there is no reason that the decomposition is going to be the adjoint of the inverse). The article only studies the synthesis case, that is the adjoint of the reconstruction operator.

Finally, we will need some unit tests to test that the adjoint property is indeed verified.

/!\ Important: this probably affects badly the code base. Indeed, people probably used the adj_op function in lieu of the reconstruction. This would need to be reviewed.

[BUG] Move away from using pip.main for pybind11 installation

System setup
Affected everywhere

Describe the bug
Currently we use:

pysap/setup.py

Lines 24 to 29 in ddd9f9f

try:
from pip._internal.main import main as pip_main
except ImportError:
import warnings
warnings.warn("pybind11 install using needs pip to 19.3 or above. This will be an error in future")
from pip._internal import main as pip_main

This is messy and also uses pip internal functions which is not really recommended.

In python 3.7, I had failure in trying to install pybind11 while installing pysap:
Requirement already satisfied: six in /home/chaithyagr/tempenv/lib/python3.7/site-packages (from progressbar2->pysap-mri==0.1.1) (1.13.0) running build running build_py running build_ext Collecting pybind11>=2.3.0 Using cached https://files.pythonhosted.org/packages/4b/4d/ae1c4d8e8b139afa9682054dd42df3b0e3b5c1731287933021b9fd7e9cc4/pybind11-2.4.3-py2.py3-none-any.whl ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/tmp/pip-req-tracker-ex87fkn0/42ab732b1adc582cccd635603a388aa64fefe2d0a660b08a758d98bb'

This seems to be a bizarre issue which can be avoided if we use subproccess call to pip and then install pybind11.

To Reproduce
Create a new python 3.7 environment and try to install by cloning to repository and using python setup.py install

Module and lines involved
setup.py

Are you planning to submit a Pull Request?

  • Yes
  • No

Travis builds failed because of pycodestyle warnings

The travis builds don't pass anymore after the latest update of pycodestyle

The first error is just a missing 'r' in the pysap logo.
The second one is a raised warning when there is linebreak after a binary operator (https://pypi.python.org/pypi/pycodestyle), I think that the release was meant to have this warning ignored by default but I guess that because we specify which errors we ignore we have to update the list of ignored errors / warnings.

I made a PR for solving this issue.

Install errors

Hi everyone,

I just finished installing pisap on my device after some problems solved thanks to @LElgueddari and @AGrigis .

I will make a list of those errors so that it can be checked in the future :

  • NFFT is needed, and furthermore the 3.2.4 version seems more suitable. In the newer versions a specific file (nfft3util.h) is removed from the include folder and raises errors when installing pynfft with pip
  • Sparse2D is needed also on its own. When installing pisap, I guess that a pruned version of sparse2d is added but some binaries are missing, like mr_transform. Therefore the dependencies of Sparse2D are also compulsory (e.g. cfitsio).
  • In sf_tools/signal/optimisation.py there is an error if line 463 is not commented when using CONDAT
  • I still have a deprecation warning when loading pisap concerning pyfits, but I guess it's just a dependency issue
  • Obviously all the libraries installed previously have to be properly linked in the PATH.

Maybe all of those errors are obvious for you but as @LElgueddari advised me to, I supposed you'd be interested in my experience as a new user.

[NEW FEATURE] Support for Wavelet Operator for direct use with ModOpt

Is your feature request related to a problem? Please describe.
Currently pysap-mri implements a wavelet class that can be used with ModOpt for optimization.
https://github.com/CEA-COSMIC/pysap-mri/blob/master/mri/reconstruct/linear.py#L24
We notice that as we develop any other plugins in future, we may need this class repeatedly as Wavelet operator is important and comes up repeatedly for imaging applications with structured sparsity.

Describe the solution you'd like
To prevent each plugin having a redundant implementation, we could have PySAP (or ModOpt: The location is still debatable) to have a general wavelet class that implements the Wavelet such that it can be used with ModOpt directly out of the box

Are you planning to submit a Pull Request?

  • Yes, Well it is just a basic class movement from pysap-mri to pysap, te base class at pysap-mri is pretty general at the moment.
  • No

Wrong name pywavelet in 3D

from pysap.numerics.utils import generate_operators, one can initialize a 3D linear operator with one of the pyWavelet wavelet like Db4, but cannot use it in 3D getting this error:
ValueError: This wavelet can only be applied on 2D square images
But when we take the from pysap.plugins.mri.reconstruct_3D.linear import pyWavelet3 the names are different pyWavelet3("Db4") is now unrecognised however pyWavelet3("db4") is fine.
Inconsistency between wavelet names in pywt and pysap.pywt wavelets.

[BUG] Travis

Always testing against Python3.7 on Travis using Conda.

installation problems on windows

Hi,
I'm trying to install pisap under Windows. I've tried several approaches that all fail at the cmake compilation stage.

  • Python 3.6: compilation fails (cfits and boost)
  • Python 2.7: compilation fails (cfits and boost)
    This is the part of the log that seems relevant: (identical results were obtained for both Python versions)
 "C:\Users\FL232714\Desktop\pisap-master\build\temp.win-amd64-3.6\Release\ALL_BUILD.vcxproj" (cible par défaut) (1) ->
"C:\Users\FL232714\Desktop\pisap-master\build\temp.win-amd64-3.6\Release\cfitsio.vcxproj" (cible par défaut) (5) ->
(CustomBuild cible) ->
  C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(171,5): error MSB6006: Arrêt de
"cmd.exe" avec le code 9009. [C:\Users\FL232714\Desktop\pisap-master\build\temp.win-amd64-3.6\Release\cfitsio.vcxproj
]


"C:\Users\FL232714\Desktop\pisap-master\build\temp.win-amd64-3.6\Release\ALL_BUILD.vcxproj" (cible par défaut) (1) ->
"C:\Users\FL232714\Desktop\pisap-master\build\temp.win-amd64-3.6\Release\Boost.vcxproj" (cible par défaut) (4) ->
  C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(171,5): error MSB6006: Arrêt de
"cmd.exe" avec le code 9009. [C:\Users\FL232714\Desktop\pisap-master\build\temp.win-amd64-3.6\Release\Boost.vcxproj]

I've noticed that you've also setup Travis builds (https://github.com/neurospin/pisap/blob/master/.travis.yml) and realized that the installation on Travis is not python setup.py install but manual installation of dependencies with pip. So I'm assuming it's not necessary to follow the cmake build steps to get it working.

Do you have any hints on how I can manage to get pisap working, based on the above?

Thanks for your help,

Florian

Wrong reference to pypipe

pypipe is defined as an extra dependency in https://github.com/CEA-COSMIC/pysap/blob/master/pysap/info.py#L87

pypipe corresponds to a completely different package on PyPI (https://pypi.org/project/pypipe/) and thus pip install a wrong dependency with pysap.

Also, there are several Python packages named pypipe on internet and there is no reference to the right one in pysap documentation. I guess the right package is this one: https://github.com/AGrigis/pypipe (it should be mentionned somewhere in pysap documentation).

[BUG] Unaccessible Second Generation of Starlet

System setup
OS: macOS v10.14.5
Python version: v3.7.3
Python environment : conda v4.7.10

Describe the bug
In the C++ code, the boolean ModifiedATWT allows the user to switch from the First Generation Starlet to the Second one. However in the associated Python code, he cannot get the filters of the Second Generation because it seems that he does have access to this boolean variable.

Module and lines involved
The missing boolean variable should be an argument of get_mr_filters() in ModOpt/modoptt/signal/wavelet.py. And it can be found in Sparse2D/MR_Filter.cc and its role is explained in the comment of line 851.

Are you planning to submit a Pull Request?

  • Yes
  • No

SAMPLE_DATE_FILES outdated

in data.py:
get_sample_data need proper urls from SAMPLE_DATE_FILES which is outdated as it points on local file-system for now..

Psitive doesn't exist

In pysap/numerics/reconstruct.py Positivity is imported but not used and instead Positive is used a typo that breaks the code

Multi-coil NUFFT interface

Hello,

I am the author of PyNUFFT. I have almost finished your suggested ideas which are very helpful. Actually, the improved version is beyond my original expectation, which I would acknowledge your contribution to the PyNUFFT repository.

Now I am facing a choice about the multi-coil version, which can be the following two different choices (I have both transforms):

1. Multiple-images to multiple data, and multiple data to multiple images. For example, (256,256, Nc) image -> (65536, Nc) data, or (65536, Nc) data -> (256,256, Nc) image.

2. Single-image to multiple data, multiple data to single-image. For example, (256,256) image -> (65536, Nc) data, or (65536, Nc) data -> (256,256) image. However, this involves sensitivities and a final combination is a must.

My personal preference is 1 because it is a simple transformation without coil sensitivities. However, 2 is more common in MRI community but it introduces the extra complexity.

Just want to listen to your opinion so I would formulate the code as the most natural way.

Regars,

Jyh-Miin

[BUG] [INSTALLATION ERROR][TRAVIS] Install fails

System setup
OS: [e.g] macOS v10.14.1
Python version: All
Python environment (if any): [e.g.] conda v4.5.11 / Venv

Describe the bug
A python setup.py install is broken with at build_ext task:

running build_ext
Traceback (most recent call last):
  File "setup.py", line 198, in <module>
    "test": HybridTestCommand
  File "/home/chaithyagr/Envinronments/testenv/lib/python3.5/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/chaithyagr/Envinronments/testenv/lib/python3.5/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/home/chaithyagr/Envinronments/testenv/lib/python3.5/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/chaithyagr/Envinronments/testenv/lib/python3.5/site-packages/setuptools/command/bdist_egg.py", line 172, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/chaithyagr/Envinronments/testenv/lib/python3.5/site-packages/setuptools/command/bdist_egg.py", line 158, in call_command
    self.run_command(cmdname)
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/chaithyagr/Envinronments/testenv/lib/python3.5/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/usr/lib/python3.5/distutils/command/install_lib.py", line 109, in build
    self.run_command('build_ext')
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 90, in run
    self._preinstall(preinstall_list)
  File "setup.py", line 74, in _preinstall
    pip_main(['install'] + options + package_list)
TypeError: 'module' object is not callable

To Reproduce
On a new conda or venv try:
python setup.py install
or
pip install git+https://github.com/CEA-COSMIC/pysap

Expected behavior
For installation to run smoothly

Module and lines involved
This is something recent that popped up just yesterday, note that pip 19.3 was recently released (14th October). This is causing errors while installing pybind11 at line 74. This needs to be changed

pysap/setup.py

Lines 67 to 74 in ffc5ee9

def _preinstall(package_list, options=[]):
""" Pre-install PyPi packages before running cmake.
"""
if not isinstance(package_list, list) or not isinstance(options, list):
raise TypeError('preinstall inputs must be of type list.')
pip_main(['install'] + options + package_list)

Are you planning to submit a Pull Request?

  • Yes
  • No

Istransform attribute

Add a way to print in which space the transform currently is.

e.g.

wl=WaveletTransformBase.REGISTRY[-mywavelet-]
wl.data=image
wl.istransform()

===> return FALSE

wl.analysis()
wl.istransform()

===> return TRUE

[Enhance]: Transformation speed

Is your feature request related to a problem? Please describe.
For Wavelets with pywt backend, the complex numbers are supported while the current implementation is performing twice the same decomposition (real and imag part) this could be removed for pywt backend making the transforms twice faster

Describe the solution you'd like
Checking the backend of the transforms and to leave complex value numbers for ISAP transforms

This holds for both analysis and synthesis

Are you planning to submit a Pull Request?

  • Yes
  • No

DisplayTransform method

A simple method to display all currently available transforms.
(instead of doing ....REGISTRY.keys() to print them)

Fail to compile PySAP dependencies on MacOSX using Anaconda and MacPorts

Here are the commands I use to compile and install PySAP on MacOSX using Anaconda and MacPorts
(the environment.yml is available there: https://raw.githubusercontent.com/jeremiedecock/pysap/master/environment.yml )

# Install cmake and GCC7 with MacPorts (in /opt/local)
sudo port install cmake gcc7

# Make the conda virtual environment and activate it
conda env create -f environment.yml
source activate pysap

# Set the compiler environment variable (to use the right MacPorts version of GCC)
export CC="/opt/local/bin/gcc-mp-7"
export CXX="/opt/local/bin/g++-mp-7"

# Set the Python headers directory (here in the Anaconda pysap virtual environment)
export CPLUS_INCLUDE_PATH=~/anaconda3/envs/pysap/include/python3.6m

# Build and install
python3 setup.py build
python3 setup.py install

It can compile but it cannot link.
I also tried with Python2.7 and I’ve got the same errors.
I'll try with homebrew instead of macports.

No way to have control over the padding in the Wavelet operator

Right now it is not possible when initializing a Wavelet operator to change the padding.

  • In the case of pywt, it is rather simple, we just need to make visible the kwarg "mode" (right now hardcoded to "symmetric" in pysap/extensions/transform.py).

  • In the case of isap, it is a bit more complicated since we need to make the argument of sparse2d visible in the binding (currently it is not).

Wrong fourier adjoint operator

The ajdoint operator of the fourier operator is not correct. We need to normalize by the square root of the dimension of the data (or if we choose to use the numpy package -more complete- rather than the scipy one, we can just use the "norm" kwarg).

To see this, run the following experiment:

import numpy as np

from pysap.numerics.fourier import FFT2
from pysap.numerics.utils import convert_mask_to_locations

dim_1d = 512
dim = (dim_1d, dim_1d)

mask = np.ones(dim)
samples = convert_mask_to_locations(mask)

fourier_op = FFT2(samples=samples, shape=dim)

def fft(x):
    return fourier_op.op(x)

def ifft(x):
    return fourier_op.adj_op(x)

fft_example = np.random.rand(*dim) + 1j * np.random.rand(*dim)  # x
im_example = np.random.rand(*dim)  # y
print("Adjoint:")
x_Tadj_y = np.dot(fft_example.flatten(), np.conjugate(fft(im_example).flatten()))
T_x_y = np.dot(ifft(fft_example).flatten(), np.conjugate(im_example.flatten()))
print(np.allclose(x_Tadj_y, T_x_y))

I guess the easiest solution is to switch to numpy and use the "norm" kwarg.

We also need one (or more) unit test(s) to make sure it doesn't happen. (Not sure whether the one above can be used because of the randomness but sthg similar should do).

I haven't yet checked for NFFT2 but it's probably the same situation. If it is, we need to correct it as well (can probably be done in one PR).

/!\ Important: I am not sure how this affects all the existing code base (namely whether all the examples would still work, whether in some places the correction has already been made manually).

[Installation issue]: python fails to build in a virtualenv due to pybind

I tried to install pysap in a virtualenv and it could not find pybind.h. Here is the error that I get:

  /tmp/pip-req-build-wcrcu_4f/sparse2d/python/pysparse.cpp:10:31: fatal error: pybind11/pybind11.h: No such file or directory
  compilation terminated.
  CMakeFiles/pysparse.dir/build.make:62: recipe for target 'CMakeFiles/pysparse.dir/pysparse.cpp.o' failed
  make[2]: *** [CMakeFiles/pysparse.dir/pysparse.cpp.o] Error 1
  CMakeFiles/Makefile2:142: recipe for target 'CMakeFiles/pysparse.dir/all' failed
  make[1]: *** [CMakeFiles/pysparse.dir/all] Error 2
  Makefile:83: recipe for target 'all' failed
  make: *** [all] Error 2```

Enhance transformation speed and accuracy

In sparse2d/python/NumPyArrayData.h, we can see that we do not use the buffer during the data type conversion (speed issue), and that we cast the data to float (accuracy issue).

[BUG] Clean up Travis to test for library installations

Describe the bug
This issue could as well be tagged as enhancement, but I am marking it as a bug as it is causing us to miss many other important issues with installations. (like #114 )

In the current state of travis, we install all the libraries specifically:

pysap/.travis.yml

Lines 43 to 60 in 275811d

- pip install pybind11
- pip install cython
- pip install git+https://github.com/ghisvail/pyNFFT.git
- pip install nose
- pip install nose-exclude
- pip install coverage
- pip install coveralls
- pip install pycodestyle
- pip install scikit-learn
- pip install nibabel
- pip install progressbar2
- pip install astropy
- pip install PyWavelets
- pip install matplotlib
- pip install modopt
- pip install pysap-mri
- pip install pysap-astro
- pip install -b $TRAVIS_BUILD_DIR/build -t $TRAVIS_BUILD_DIR/install --no-clean --no-deps .

Ideally we would want a much cleaner and minimalistic travis configuration to get things just working.

Are you planning to submit a Pull Request?

  • Yes
  • No - Not anytime soon, but eventually, yes

[BUG] Parallel usage of Wavelets results in errors

System setup
OS: [e.g] macOS v10.14.1
Python version: [e.g.] v3.6.7
Python environment (if any): [e.g.] conda v4.5.11

Describe the bug
While using Pysap with Parallel in joblib to carry out Wavelet Transforms for various cases, We face issues. The nb_band_per_scale variable is not being populated.

To Reproduce

coeffs, coeffs_shape = \
                zip(*Parallel(n_jobs=self.n_cpu)
                    (delayed(self._op)
                    (data[i], self.transform[i])
                    for i in numpy.arange(self.num_channels)))

with _op function defined as :

def _op(self, data, transform):
        if isinstance(data, numpy.ndarray):
            data = pysap.Image(data=data)
        transform.data = data
        transform.analysis()
        coeffs, coeffs_shape = flatten(transform.analysis_data)
        return coeffs, coeffs_shape

Expected behavior
We expect the adjoint operation to work. But we get random errors, especially that nb_band_per_scale is None.

Module and lines involved
I see that when n_cpu=1, things work smoothly, the issue is when we extend it to have more cores.

Are you planning to submit a Pull Request?

  • Yes ---> If I get some fix that is, but this is at a lower priority for me
  • No

Too long transform names ?

the keys in the REGISTRY dict are a bit too long.
==> Sam? Jean-Luc ?

for reference:

['PyramidalWaveletTransformInFourierSpaceAlgo1',
 'DecompositionOnScalingFunction',
 'PyramidalWaveletTransformInFourierSpaceAlgo2',
 'MallatWaveletTransform79Filters',
 'OnLine44AndOnColumn53',
 'IsotropicAndCompactSupportWaveletInFourierSpace',
 'PyramidalLinearWaveletTransform',
 'PyramidalBsplineWaveletTransform',
 'UndecimatedDiadicWaveletTransform',
 'PyramidalLaplacian',
 'NonOrthogonalUndecimatedTransform',
 'UndecimatedBiOrthogonalTransform',
 'MorphologicalMedianTransform',
 'PyramidalMedianTransform',
 'MixedHalfPyramidalWTAndMedianMethod',
 'UndecimatedHaarTransformATrousAlgorithm',
 'MorphologicalPyramidalMinmaxTransform',
 'FeauveauWaveletTransformWithoutUndersampling',
 'WaveletTransformInFourierSpace',
 'OnLine53AndOnColumn44',
 'MixedWTAndPMTMethod',
 'BsplineWaveletTransformATrousAlgorithm',
 'MeyerWaveletsCompactInFourierSpace',
 'MorphologicalMinmaxTransform',
 'WaveletTransformViaLiftingScheme',
 'LinearWaveletTransformATrousAlgorithm',
 'HalfPyramidalTransform',
 'HaarWaveletTransform']

Installation details

The README has not been updated with the update to modopt

Also to fix the deprecation warning associated to pyfits it should be removed by using astropy instead

Installation problems on Ubuntu (17.10)

Here is a list of solutions I have been using to solve problems I have encountered while installing pISAP on my machine (Ubuntu 17.10).

  • Install CMake package using sudo apt install cmake for the python setup.py develop --user command to run properly.

  • PyQtGraph requires one of PyQt4, PyQt5 or PySide. Solved using sudo apt install python-qt4.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.