Git Product home page Git Product logo

torch_lfilter's Introduction

torch_lfilter

Bring low pass filtering to PyTorch!

This pytorch extension offers a PyTorch alternative for Scipy's lfilter - with gradient tracking.

CPU tensors only (efficiently...)

Although it's certainly the goal to implement an efficient CUDA lfilter in C++, for now only the CPU version is implemented in C++. That said, the implementation is reasonably fast and doing the filtering on the CPU might be a viable option. Moreover, the pure-python implementation works on all devices.

Installation

The library can be installed with pip:

pip install torch_lfilter

Please note that no pre-built wheels exist. This means that pip will attempt to install the library from source. Make sure you have the necessary dependencies installed for your OS.

Dependencies

Linux

On Linux, having PyTorch installed is often enough to be able install the library (along with the typical developer tools for your distribution). Run the following inside a conda environment:

conda install pytorch -c pytorch
pip install torch_lfilter

Windows

On Windows, the installation process is a bit more involved as typically the build dependencies are not installed. To install those, download Visual Studio Community 2017 from here. During installation, go to Workloads and select the following workloads:

  • Desktop development with C++
  • Python development

Then go to Individual Components and select the following additional items:

  • C++/CLI support
  • VC++ 2015.3 v14.00 (v140) toolset for desktop

Then, download Microsoft Visual C++ Redistributable from here.

After these installation steps, run the following commands inside a x64 Native Tools Command Prompt for VS 2017, after activating your conda environment:

set DISTUTILS_USE_SDK=1
conda install pytorch -c pytorch
pip install torch_lfilter

License

© Floris Laporte 2020, GPLv3

torch_lfilter's People

Contributors

flaport avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

rnwatanabe muhammedakyuzlu

torch_lfilter's Issues

error while installing

$ pip install torch-lfilter
Defaulting to user installation because normal site-packages is not writeable
Collecting torch-lfilter
Using cached torch_lfilter-0.0.3.tar.gz (4.1 kB)
Preparing metadata (setup.py) ... done
Building wheels for collected packages: torch-lfilter
Building wheel for torch-lfilter (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python3.8 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-fz87kws2/torch-lfilter_c26077d327b34bb286edb959429c7ca6/setup.py'"'"'; file='"'"'/tmp/pip-install-fz87kws2/torch-lfilter_c26077d327b34bb286edb959429c7ca6/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-jk7bf_bc
cwd: /tmp/pip-install-fz87kws2/torch-lfilter_c26077d327b34bb286edb959429c7ca6/
Complete output (34 lines):
/tmp/pip-install-fz87kws2/torch-lfilter_c26077d327b34bb286edb959429c7ca6/torch_lfilter.py:33: UserWarning: no efficient C++ lfilter implementation for cpu-tensors found. falling back to a (much slower) pure python implementation.

Maybe something went wrong during the compilation of torch_lfilter? Please check out the installation instructions at https://github.com/flaport/torch_lfilter.
warnings.warn(
running bdist_wheel
/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:352: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
warnings.warn(msg.format('we could not find ninja.'))
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-38
copying torch_lfilter.py -> build/lib.linux-x86_64-cpython-38
running build_ext
/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:294: UserWarning:

                             !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (g++ 4.8.5) may be ABI-incompatible with PyTorch!
Please use a compiler that is ABI-compatible with GCC 5.0 and above.
See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.

See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6
for instructions on how to install GCC 5 or higher.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                            !! WARNING !!

warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler))

building 'torch_lfilter_cpp' extension
creating build/temp.linux-x86_64-cpython-38
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/TH -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/include/python3.8 -c torch_lfilter.cpp -o build/temp.linux-x86_64-cpython-38/torch_lfilter.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=torch_lfilter_cpp -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
gcc: error: unrecognized command line option ‘-std=c++14’
error: command '/usr/bin/gcc' failed with exit code 1

ERROR: Failed building wheel for torch-lfilter
Running setup.py clean for torch-lfilter
Failed to build torch-lfilter
Installing collected packages: torch-lfilter
Running setup.py install for torch-lfilter ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python3.8 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-fz87kws2/torch-lfilter_c26077d327b34bb286edb959429c7ca6/setup.py'"'"'; file='"'"'/tmp/pip-install-fz87kws2/torch-lfilter_c26077d327b34bb286edb959429c7ca6/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-3ykcbmet/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/shrutibhilare/.local/include/python3.8/torch-lfilter
cwd: /tmp/pip-install-fz87kws2/torch-lfilter_c26077d327b34bb286edb959429c7ca6/
Complete output (36 lines):
/tmp/pip-install-fz87kws2/torch-lfilter_c26077d327b34bb286edb959429c7ca6/torch_lfilter.py:33: UserWarning: no efficient C++ lfilter implementation for cpu-tensors found. falling back to a (much slower) pure python implementation.

Maybe something went wrong during the compilation of torch_lfilter? Please check out the installation instructions at https://github.com/flaport/torch_lfilter.
  warnings.warn(
running install
/home/shrutibhilare/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-38
copying torch_lfilter.py -> build/lib.linux-x86_64-cpython-38
running build_ext
/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:352: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
  warnings.warn(msg.format('we could not find ninja.'))
/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:294: UserWarning:

                               !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (g++ 4.8.5) may be ABI-incompatible with PyTorch!
Please use a compiler that is ABI-compatible with GCC 5.0 and above.
See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.

See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6
for instructions on how to install GCC 5 or higher.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                              !! WARNING !!

  warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler))
building 'torch_lfilter_cpp' extension
creating build/temp.linux-x86_64-cpython-38
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/TH -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/include/python3.8 -c torch_lfilter.cpp -o build/temp.linux-x86_64-cpython-38/torch_lfilter.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=torch_lfilter_cpp -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
gcc: error: unrecognized command line option ‘-std=c++14’
error: command '/usr/bin/gcc' failed with exit code 1
----------------------------------------

ERROR: Command errored out with exit status 1: /usr/local/bin/python3.8 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-fz87kws2/torch-lfilter_c26077d327b34bb286edb959429c7ca6/setup.py'"'"'; file='"'"'/tmp/pip-install-fz87kws2/torch-lfilter_c26077d327b34bb286edb959429c7ca6/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-3ykcbmet/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/shrutibhilare/.local/include/python3.8/torch-lfilter Check the logs for full command output.

Error while installing torch-lfilter

pip install torch-lfilter
Defaulting to user installation because normal site-packages is not writeable
Collecting torch-lfilter
Using cached torch_lfilter-0.0.3.tar.gz (4.1 kB)
Preparing metadata (setup.py) ... done
Building wheels for collected packages: torch-lfilter
Building wheel for torch-lfilter (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [34 lines of output]
/tmp/pip-install-bdbj55hq/torch-lfilter_d2b3ef8627bb49dcaf58ecc60af4262b/torch_lfilter.py:33: UserWarning: no efficient C++ lfilter implementation for cpu-tensors found. falling back to a (much slower) pure python implementation.

  Maybe something went wrong during the compilation of torch_lfilter? Please check out the installation instructions at https://github.com/flaport/torch_lfilter.
    warnings.warn(
  running bdist_wheel
  /home/shrutibhilare/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:352: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-cpython-38
  copying torch_lfilter.py -> build/lib.linux-x86_64-cpython-38
  running build_ext
  /home/shrutibhilare/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:294: UserWarning:
  
                                 !! WARNING !!
  
  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  Your compiler (g++ 4.8.5) may be ABI-incompatible with PyTorch!
  Please use a compiler that is ABI-compatible with GCC 5.0 and above.
  See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.
  
  See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6
  for instructions on how to install GCC 5 or higher.
  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  
                                !! WARNING !!
  
    warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler))
  building 'torch_lfilter_cpp' extension
  creating build/temp.linux-x86_64-cpython-38
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/TH -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/include/python3.8 -c torch_lfilter.cpp -o build/temp.linux-x86_64-cpython-38/torch_lfilter.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=torch_lfilter_cpp -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  gcc: error: unrecognized command line option ‘-std=c++14’
  error: command '/usr/bin/gcc' failed with exit code 1
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for torch-lfilter
Running setup.py clean for torch-lfilter
Failed to build torch-lfilter
Installing collected packages: torch-lfilter
Running setup.py install for torch-lfilter ... error
error: subprocess-exited-with-error

× Running setup.py install for torch-lfilter did not run successfully.
│ exit code: 1
╰─> [36 lines of output]
/tmp/pip-install-bdbj55hq/torch-lfilter_d2b3ef8627bb49dcaf58ecc60af4262b/torch_lfilter.py:33: UserWarning: no efficient C++ lfilter implementation for cpu-tensors found. falling back to a (much slower) pure python implementation.

  Maybe something went wrong during the compilation of torch_lfilter? Please check out the installation instructions at https://github.com/flaport/torch_lfilter.
    warnings.warn(
  running install
  /home/shrutibhilare/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
    warnings.warn(
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-cpython-38
  copying torch_lfilter.py -> build/lib.linux-x86_64-cpython-38
  running build_ext
  /home/shrutibhilare/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:352: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  /home/shrutibhilare/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:294: UserWarning:
  
                                 !! WARNING !!
  
  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  Your compiler (g++ 4.8.5) may be ABI-incompatible with PyTorch!
  Please use a compiler that is ABI-compatible with GCC 5.0 and above.
  See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.
  
  See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6
  for instructions on how to install GCC 5 or higher.
  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  
                                !! WARNING !!
  
    warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler))
  building 'torch_lfilter_cpp' extension
  creating build/temp.linux-x86_64-cpython-38
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/TH -I/home/shrutibhilare/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/include/python3.8 -c torch_lfilter.cpp -o build/temp.linux-x86_64-cpython-38/torch_lfilter.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=torch_lfilter_cpp -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  gcc: error: unrecognized command line option ‘-std=c++14’
  error: command '/usr/bin/gcc' failed with exit code 1
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> torch-lfilter

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Image processing

Hi, thank you for your beautiful coding , do you have any plan to extend this work to 2D-filter?

Doublt about zi

I noticed that the original scipy implementation has the zi parameter, but in your implementation there is no zi.Does this affect the generated results?

Support for filter initial conditions

I suppose there is no support for setting the filter initial conditions, as in scipy's lfilter using the zi argument, right? Or constructing them using something similar to lfilteric?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.