trinkle23897 / fast-poisson-image-editing Goto Github PK

I have tested with different backends provided by the repo. Most results are fine but only the cuda backend can not work as intended. When I run the cli command, the cuda backend printed the info of the gpus, then after a long-time computation, it output a wrong blending result (abs error [0, 0, 0]).
Do you have any clue on how to solve this problem? Thank you!

The latest version of CMAKE as of this writing is 3.24

Is it a typo or am I missing somethig?

I had some problems with installing the fpie package. I already updated the wheel and installed the cmake package manually.

• OS: Ubuntu debian 18.04.6 LTS
• CUDA 11.4
• Nvidia RTX A5000 and Nvidia GeForce RTX 3090
• Browser firefox

Hi @Trinkle23897. Awesome work. I came across the following environment issue.
On an older platform the installation proceeds to install gcc backend

$ fpie --check-backend
[Taichi] version 1.6.0, llvm 15.0.4, commit f1c6fbbd, linux, python 3.9.7
['numpy', 'numba', 'taichi-cpu', 'taichi-gpu', 'gcc', 'openmp']
$ gcc --version
gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-18)

However, on a newer platform the gcc backend does not install.

$ fpie --check-backend
[Taichi] version 1.6.0, llvm 15.0.4, commit f1c6fbbd, linux, python 3.10.12
['numpy', 'numba', 'taichi-cpu', 'taichi-gpu']
$ gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

Do you have any ideas what the issue could be?

I want to use it within my python project ? I don't know how to do.

Thanks !!!

Hello,
Is there any way to process images in batch? So let's say I have three folders, in which dst/src/masks are already rescaled and aligned and I just want to take triplets with the same name, blend them and output it in a separate folder.
I also wonder whether solver reinitialization produces significant overhead if you do it one by one.

Hello

Here are my logs from anaconda virutal env terminal :

pip install vie

Here is the log

(sd-inf) C:\Users\B\Desktop\stablediffusion\stablediffusion-infinity>pip install fpie
Collecting fpie
Using cached fpie-0.2.4.tar.gz (29 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: cmake>=3.4 in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from fpie) (3.24.1.1)
Requirement already satisfied: opencv-python-headless>=4.2 in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from fpie) (4.6.0.66)
Requirement already satisfied: numpy>=1.18 in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from fpie) (1.23.3)
Requirement already satisfied: taichi>=1.0 in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from fpie) (1.1.3)
Requirement already satisfied: numba>=0.51 in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from fpie) (0.56.3)
Requirement already satisfied: llvmlite<0.40,>=0.39.0dev0 in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from numba>=0.51->fpie) (0.39.1)
Requirement already satisfied: setuptools in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from numba>=0.51->fpie) (63.4.1)
Requirement already satisfied: sourceinspect>=0.0.4 in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from taichi>=1.0->fpie) (0.0.4)
Requirement already satisfied: colorama in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from taichi>=1.0->fpie) (0.4.5)
Requirement already satisfied: rich in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from taichi>=1.0->fpie) (12.6.0)
Requirement already satisfied: dill in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from sourceinspect>=0.0.4->taichi>=1.0->fpie) (0.3.5.1)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from rich->taichi>=1.0->fpie) (0.9.1)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in c:\users\b\anaconda3\envs\sd-inf\lib\site-packages (from rich->taichi>=1.0->fpie) (2.13.0)
Building wheels for collected packages: fpie
Building wheel for fpie (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [62 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-310
creating build\lib.win-amd64-cpython-310\fpie
copying fpie\args.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\cli.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\gui.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\io.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\np_solver.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\numba_solver.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\process.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\taichi_solver.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie_init_.py -> build\lib.win-amd64-cpython-310\fpie
running build_ext
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19044.
-- The C compiler identification is MSVC 19.29.30146.0
-- The CXX compiler identification is MSVC 19.29.30146.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.36.0.windows.1")
-- downloading/updating pybind11
-- pybind11 directory not found, cloning...
Cloning into 'C:/Users/B/AppData/Local/Temp/pip-install-5bykcwlg/fpie_1af82193c262414a803966f640d8358b/pybind11'...
--
Switched to a new branch 'stable'
-- branch 'stable' set up to track 'origin/stable'.

  -- pybind11 v2.10.0
  -- Found PythonInterp: C:/Users/B/anaconda3/envs/sd-inf/python.exe (found suitable version "3.10.6", minimum required is "3.6")
  -- Found PythonLibs: C:/Users/B/anaconda3/envs/sd-inf/libs/python310.lib
  -- Performing Test HAS_MSVC_GL_LTCG
  -- Performing Test HAS_MSVC_GL_LTCG - Success
  -- Found OpenMP_C: -openmp (found version "2.0")
  -- Found OpenMP_CXX: -openmp (found version "2.0")
  -- Found OpenMP: TRUE (found version "2.0")
  -- Could NOT find MPI_C (missing: MPI_C_LIB_NAMES MPI_C_HEADER_DIR MPI_C_WORKS)
  -- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS)
  -- Could NOT find MPI (missing: MPI_C_FOUND MPI_CXX_FOUND)
  -- Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7 (found version "11.7")
  -- The CUDA compiler identification is NVIDIA 11.7.99
  -- Detecting CUDA compiler ABI info
  -- Detecting CUDA compiler ABI info - done
  -- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/bin/nvcc.exe - skipped
  -- Detecting CUDA compile features
  -- Detecting CUDA compile features - done
  -- Found PythonInterp: C:/Users/B/anaconda3/envs/sd-inf/python.exe (found suitable version "3.10.6", minimum required is "3")
  -- Found PythonLibs: C:/Users/B/anaconda3/envs/sd-inf/libs/python310.lib (found suitable version "3.10.6", minimum required is "3.10")
  -- Configuring done
  -- Generating done
  -- Build files have been written to: C:/Users/B/AppData/Local/Temp/pip-install-5bykcwlg/fpie_1af82193c262414a803966f640d8358b/build/temp.win-amd64-cpython-310/Release
  error: [WinError 2] Le fichier spécifié est introuvable
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for fpie
Running setup.py clean for fpie
Failed to build fpie
Installing collected packages: fpie
Running setup.py install for fpie ... error
error: subprocess-exited-with-error

× Running setup.py install for fpie did not run successfully.
│ exit code: 1
╰─> [56 lines of output]
running install
C:\Users\B\anaconda3\envs\sd-inf\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-310
creating build\lib.win-amd64-cpython-310\fpie
copying fpie\args.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\cli.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\gui.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\io.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\np_solver.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\numba_solver.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\process.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie\taichi_solver.py -> build\lib.win-amd64-cpython-310\fpie
copying fpie_init_.py -> build\lib.win-amd64-cpython-310\fpie
running build_ext
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19044.
-- The C compiler identification is MSVC 19.29.30146.0
-- The CXX compiler identification is MSVC 19.29.30146.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- pybind11 v2.10.0
-- Found PythonInterp: C:/Users/B/anaconda3/envs/sd-inf/python.exe (found suitable version "3.10.6", minimum required is "3.6")
-- Found PythonLibs: C:/Users/B/anaconda3/envs/sd-inf/libs/python310.lib
-- Performing Test HAS_MSVC_GL_LTCG
-- Performing Test HAS_MSVC_GL_LTCG - Success
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Could NOT find MPI_C (missing: MPI_C_LIB_NAMES MPI_C_HEADER_DIR MPI_C_WORKS)
-- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS)
-- Could NOT find MPI (missing: MPI_C_FOUND MPI_CXX_FOUND)
-- Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7 (found version "11.7")
-- The CUDA compiler identification is NVIDIA 11.7.99
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/bin/nvcc.exe - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found PythonInterp: C:/Users/B/anaconda3/envs/sd-inf/python.exe (found suitable version "3.10.6", minimum required is "3")
-- Found PythonLibs: C:/Users/B/anaconda3/envs/sd-inf/libs/python310.lib (found suitable version "3.10.6", minimum required is "3.10")
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/B/AppData/Local/Temp/pip-install-5bykcwlg/fpie_1af82193c262414a803966f640d8358b/build/temp.win-amd64-cpython-310/Release
error: [WinError 2] Le fichier spécifié est introuvable
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> fpie

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

First of all, I highly appreciate this great work, which implements lots of methods to perform poisson image editing with different backends, where I really value the promising implementations of CUDA and Taichi which enable me to use poisson image editing during my deep network training although it isn't differentiate. Although it seems perfect right now, I may have a small issue about the special treatment of image boundary, because it's likely that the blending result is less satisfactory if the mask touches the image border.

Since my mask will touch the image border, so the codes above will forcely set the mask value to 0 (instead of keeping the original 255). I think this is because the current implementation don't give a special treatment to this case, and therefore if the target image have wrong values inside the masked region (e.g. all zeros inside the mask) and we use gradient="src", the filled values will definitely affect the blending results because the filled values are also involved in the boundary constraints (some mask values are set to zero, see above).

I wonder to know, if I want to address this, what should I do and modify the codes so as to keep the changes minimum?

Thank you very much!