Cant find cuda driver:
However, I do have pytorch working in this virtualenv.

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-244-a2d06b630338> in <module>()
----> 1 import scatwave

/home/rafael/code/pyscatwave/scatwave/__init__.py in <module>()
      6 
      7 
----> 8 from .scattering import Scattering
      9 from . import utils

/home/rafael/code/pyscatwave/scatwave/scattering.py in <module>()
      8 import warnings
      9 import torch
---> 10 from .utils import cdgmm, Modulus, Periodize, Fft
     11 from .filters_bank import filters_bank
     12 from torch.legacy.nn import SpatialReflectionPadding as pad_function

/home/rafael/code/pyscatwave/scatwave/utils.py in <module>()
      6 
      7 import torch
----> 8 from skcuda import cublas, cufft
      9 from pynvrtc.compiler import Program
     10 import numpy as np

/home/rafael/.virtualenvs/cvndev/local/lib/python2.7/site-packages/skcuda/cublas.py in <module>()
     20 from string import Template
     21 
---> 22 from . import cuda
     23 from . import utils
     24 

/home/rafael/.virtualenvs/cvndev/local/lib/python2.7/site-packages/skcuda/cuda.py in <module>()
      8 
      9 from .cudart import *
---> 10 from .cudadrv import *

/home/rafael/.virtualenvs/cvndev/local/lib/python2.7/site-packages/skcuda/cudadrv.py in <module>()
     30         break
     31 if _libcuda == None:
---> 32     raise OSError('CUDA driver library not found')
     33 
     34 # Exceptions corresponding to various CUDA driver errors:

OSError: CUDA driver library not found

Hi,

I don't know if this the right place to ask but I was wondering if you have information regarding the number of FLOPs performed by the scattering network.

I am running tests on a model that has 22GFLOPs (using inception as the feature extractor + a few other big convolutions). I replaced the whole feature extractor by the scattering network with J=3 and input size of 368x368 (same as the original model).

My results show that even if the GFLOPs count of the network wen from 22 to 5 in the hybrid network,
inference time is 200 times slower using the Scattering network.

Is this an expected result?
Thank you,

In Mallat's work, there are implementations for the 1-D and 2-D Scattering Networks, for working with audio and images respectively. Does your code implement at all the 1-D Scattering Transform? And if not, have you or another one of Mallat's students have implemented the 1-D Scattering Transform in python or C++, or is the only implementation available Mallat's Matlab version? (Here I'm assuming that your 2-D Scattering implementation is NOT suitable for audio analysis by setting one of the dimensions to 1, please correct me if I'm wrong)

Hi Edouardo,

Thanks for the library!

I was trying to run your mnist.py demo on a AWS gpu node, I got a trivial error.
The suggested fix (not tested) is below. Probably just a versioning issue with
PyTorch.

Do you have a way of visualizing the results of your demo? Or perhaps the scattering
network it learned?

Thanks
Ben

Error:
.
.
.
File "/home/ec2-user/pyscatwave/scatwave/scattering.py", line 82, in pad
output.narrow(4, 0, 1).copy(out_)
RuntimeError: The expanded size of the tensor (1) must match the existing size (36) at non-singleton dimension 4

Change:

line 78 scattering.py added squeeze
output.narrow(output.ndimension()-1, 0, 1).squeeze(4).copy_(input)

and

line 82 of scattering.py added squeeze
output.narrow(4, 0, 1).squeeze(4).copy_(out_)

I write a code to call the encode_processed function of the reconstruction_exp.py, but it have a little problem, I want to scattering a image use this function, and then call the decode function of the reconstruction_exp.py to recovery this picture. I don't know if I can do that. And I can't find test code in your project would you please give me a test code about how to call these two function? Thanks very much.

Hi!
I tried using the lib and got some weird unpredictable behaviour. I just created a matrix A, computed the scattering transform of A and 1.01*A, and printed the ratio between the two outputs.
`
A = torch.FloatTensor(np.ones((1,1,4,4)))

A[0,0,0,0]=3

scat = Scattering(M=4, N=4, J=2).cpu()

res1 = scat(A).numpy().copy().squeeze()

res2 = scat(1.01*A).numpy().copy().squeeze()

print(res2/res1)
`

When I repeat this code, which as far as I understand should always give the same result, I get totally different matrices. Sometimes with an order of magnitude of 1, sometimes 10^{-3}, 10^{25}, 10^{20} etc.
I checked and there is no 'rand' function in your code, what the hell is it doing? ^^

I'm curious how you might reverse the scatter transform? For example, perform the scatter operation on an image, then do the inverse and get the image back. Is that possible?

Hi,
Good work with this technique! I think its use could be expanded beyond simple feed-forward classifiers. In particular, I think it could be useful in generative neural networks.

Is there a means to calculate a gradient through this such that this module could be used as an intermediate part of a larger computation graph?

Best wishes,
Michael

All the drivers/modules have been installed and verified.
It throws an 'cublasNotInitialized' error when I run scat(x).
Though, I can run skcuda.cublas.cublasCreate() in python.

Not quite an issue but a question I have on the code: where is the gaussian smoothing implemented? My guess is the Phi filter, but that doesn't look like a gaussian filter. What am I missing?

First, congratulation for this very interesting work.
I would be interested in tested hybrid networks for 1D specific signals. When do you plan the new version you mention, including "1D-2D-3D fast, optimized, differentiable Scattering Transform" ?

Calling of scat = Scattering(M=3200, N=3200, J=2) is taking too much time. Can it become faster?

I tried to install pyscatwave, but do not have CUDA (so I cannot install skcuda).
I can run python setup.py install without issue, but when I run import scatwave I get the following

---------------------------------------------------------------
ImportError                   Traceback (most recent call last)
<ipython-input-2-a2d06b630338> in <module>()
----> 1 import scatwave

~/anaconda/lib/python3.5/site-packages/scatwave-0.0.1-py3.5.egg/scatwave/__init__.py in <module>()
      6 
      7 
----> 8 from .scattering import Scattering
      9 from . import utils

~/anaconda/lib/python3.5/site-packages/scatwave-0.0.1-py3.5.egg/scatwave/scattering.py in <module>()
      8 import warnings
      9 import torch
---> 10 from .utils import cdgmm, Modulus, Periodize, Fft
     11 from .filters_bank import filters_bank
     12 from torch.legacy.nn import SpatialReflectionPadding as pad_function

~/anaconda/lib/python3.5/site-packages/scatwave-0.0.1-py3.5.egg/scatwave/utils.py in <module>()
      6 
      7 import torch
----> 8 from skcuda import cublas, cufft
      9 from pynvrtc.compiler import Program
     10 import numpy as np

ImportError: No module named 'skcuda'

Is it possible to simply remove this dependency and still run pyscatwave? Or would it be significantly more complicated than that?

I see you based your code off of "https://github.com/pytorch/tnt/blob/master/example/mnist.py".

You forgot to reset the "classerr" meter. So when you call "engine.test", the accuracy that is printed out is the average classification accuracy of testing and training, not just testing.

Thanks for releasing that code !

I have not analyzed the scattering implementation thoroughly so this may not be a well-informed question but would it be feasible to write a pure pytorch (or tensorflow or theano for that matter) implementation using only the library's primitives ?

If it is, have you tried such an implementation and was the performance satisfying?

Is there any easy hack to use your code for reconstructing the input image ?

Please provide details about how to calculate K, the output dimension of the output of scattering.

Waiting for your new version update!

Hello,

Thanks a lot for your generous sharing.

I want to use your proposed structure in some adversarial training framework.

My problem is that:

When batch size is 64, the previous data has the size of 64332*32

For J=1, after scattering, the matrix size is 64391616.
For J=2, after scattering, the matrix size is 6431788.
For J=3, after scattering, the matrix size is 6432544.
For J=4, after scattering, the matrix size is 6433322.

Looks like the tensor-shape for third dimesion is 1+J*8, the fourth and fifth dimension is 32/2^j.

I am not sure, is there a inverse operation API that I can restore the matrix after scattering to the original matrix, e.g., Tensor with size 64332*32.

I am sorry for double submitting issues.

However, I am not sure which repository would be in fix.

Thanks & regards!
Momo

I was curious to see what would happen if one could backprop through the Scattering transform (in the context of GANs for instance) so I ended up re-implementing your code in pure TF.

Whenever I used or modified your work, I added a link to your repo. Is this enough to respect the clauses of the BSD license or should I add an explicit copyright notice ?

My eventual Hybrid GANs were not very successful unfortunately...