Hi

THANK YOU for your work and it just saved my life.

I am wondering how to turn on the gpu support when I transform ERP to perspective image

Pylance identifies 'equi2pers' as a module. It is then used later in the code as a function. This is the cause for the error.

I have very limited knowledge of Python and coding in general. I think 'equi2pers' is identified as a module instead of a function is due to the source code is not interpreted as expected by Python 3.12 which I am running.

BACKGROUND
I attempted to run your example code you provided at: https://github.com/haruishi43/equilib

I first ran the example using ‘Equi2Pers’ and Pylance could not find ‘Equi2Pers’. I then attempted the second example using ‘equi2pers’. Pylance identified it as a module.

Here is what I was running which is identical to the example but for 'my_img.jpg' and the last to lines of code to print 'perspective_image.jpg':

#This is the example code for 'equilib'
import numpy as np
from PIL import Image
from equilib import equi2pers

Input equirectangular image

equi_img = Image.open("./my_image.jpg")
equi_img = np.asarray(equi_img)
equi_img = np.transpose(equi_img, (2, 0, 1))

rotations

rots = {
'roll': 0.,
'pitch': np.pi/4, # rotate vertical
'yaw': np.pi/4, # rotate horizontal
}

Run equi2pers

pers_img = equi2pers(
equi=equi_img,
rots=rots,
height=480,
width=640,
fov_x=90.0,
mode="bilinear",
)

pers_img = Image.fromarray(pers_img.transpose((1, 2, 0)))
pers_img.save("./perspective_image.jpg")

Here is the the message at the terminal:

File "c:\Users\xxxx\Desktop\STUFF\Test Area 19 - Y P R python\E2Pr.py", line 19, in
pers_img = equi2pers(
^^^^^^^^^^
TypeError: 'module' object is not callable

I noticed a lot of typing Hints in the repo using : dict or : list, they really should use the typing module type hints for these functions calls, like : Dict[str, Union[torch.Tensor,np.ndarray]] etc...

Have you considered Numba JIT support to the Numpy methods in this library? With numba in particular, you could enable Parallelization and maybe even fastmath if the floating point errors are reasonable.

Also, several of the functions in the NUMBA tend give higher performance than the default Numpy ones. Main issue is that not all constructs/calls are supported so you sometimes have to reimplement it or change it to equivalent function signatures.

Also for the Torch examples, Torch.JIT support might give a slight performance boost, especially as the JIT graph optimizer improves. It also would allow it to be useful for lots of deployment.

You could write in a way so that jitting the code is not necessary and it would therefore not be an additional hard dependency.

We were hoping to use this library for making differentiable perspective cutouts from panoramas.

We noticed two issues getting in the way:

1. Some of the function signatures have Python primitive type hints, such as:
   

   equilib/equilib/equi2pers/torch.py

   Line 24 in 3cd9be0

   fov_x: float,

2. Some of the torch_util functions are using np functions, such as:

   equilib/equilib/torch_utils/rotation.py

   Line 110 in 3cd9be0

   np.cos(yaw) * np.cos(pitch),

We are trying to decide if we should make the necessary changes. I wonder if this is something that's already in the works or if you would be interested in a pull request.

Hi! I'm trying to run the code from the example you have, but when I get to the function call to transform an equirectangular image into a perspective, the process hangs up and is killed.

Currently using numpy 1.24.2, tried both with torch 1.13.1 and also torch 2.0.1, but with either I get the same problem. Running python 3.10.12.

Any ideas?

Hi,

First of all, thank you so much for creating this tool. It's beneficial for me as a computer vision researcher. I have a question about passing a list of images, to get a panoramic image. What is the specific order of the cubemap images in the list before using the function cube2qui?

Thank you so much; keep up the amazing work!

According to py360convert , it will be deprecated when a new library is published.
Is "Equilib" this replacement? Or just 2 projects working on same issues?
@haruishi43 @sunset1995

Environment

python: '3.9.12'
torch: '2.0.0+cu118'
equilib: '0.5.4'

HI

I am seeing something unexpected using equi2pers on an image generated using cube2equi.

I generate an equirectangular image as follow:

import equilib 
import torch 

width = 64 
height = 64 
e_width = 240
e_height = 136
cubemap = {
        'F': torch.ones(3, height, width),
        'L': torch.zeros(3, height, width),
        'B': torch.zeros(3, height, width),
        'R': torch.zeros(3, height, width),
        'U': torch.zeros(3, height, width),
        'D': torch.zeros(3, height, width),
        }
equi = equilib.cube2equi(cubemap, width=e_width, height=e_height, cube_format='dict', mode='nearest')

And it looks like this.

Then I project this equirectangular image back to cubemap dictionary

cube = equilib.equi2cube(equi, w_face=64, rots={'roll':0, 'pitch':0, 'yaw': 0}, mode='nearest', cube_format='dict')

Then I see some black pixels in front face, and some white pixels occur in left and up faces.

front face

left face

up face

So, it looks like the entire equirectangular image is "shifted" to left upper direction for a bit. I have tried different image resolutions, but they all have the same problem. Using mode="bilinear" does not help neither.

I am not sure if this is related to the FIXME I found

Thanks for great repo.

As the link overview and the question from stackoverflow
I have same case, with my image , your module support to make 90-roll(horizontal)

Here is my image:

Here is my expected:

Thank you

Hi, I'm using Equi2Cube to convert equirectangular images into cubemap but it seems to return it in greyscale with repeated images. Could I ask why that might be and what I can do to solve this? Thank you

I want to rotate a equirectangular image and then turn it back to the original angle, so I use the equi2equi function twice, but the result is both the first choice. I also try to save the first result, reread the image and select again, but the result is the same。What should I do？

I was trying to use this for depth prediction, but I found it problematic that values will be forced into (0,1).

E.g.

There's many of them (we can search the keyword of clip) and currently there's no way to disable this clipping except changing the source code directly.

I find there are shifts if we run cube2equi and equi2cube. How can we avoid or decrease such shifts

cube_xt = rearrange(xt, "(b n) c h w -> b c h (n w)", n=6)

for _ in range(10):
    rec_equi_xt = cube2equi(
        cubemap = cube_xt,
        cube_format="horizon",
        height = height,
        width = width,
        # mode="bilinear",
        mode="bicubic",
        # mode="nearest",
    )

    cube_xt = equi2cube(
        equi=rec_equi_xt[None],
        rots=[{"roll": 0, "pitch": 0, "yaw": 0}],
        w_face=64,
        cube_format="horizon",
        # mode="bilinear",
        mode="bicubic",
        # mode="nearest",
    )

cube_xt = rearrange(cube_xt, "b c h (n w) -> (b n) c h w", n=6)
# show cube and xt