raboof / dualfisheye2equirectangular Goto Github PK

View Code? Open in Web Editor NEW

119.0 6.0 35.0 14.13 MB

Convert 'dual-fisheye' 360 image material to equirectangular mapping

License: MIT License

C 99.40% Makefile 0.60%

dualfisheye2equirectangular's Introduction

Dual Fisheye to Equirectangular Projection Mapping

Many '360°' camera's, such as the dokicam, consist of 2 fish-eye camera's.

Why DIY?

Those camera's typically come with desktop software or apps to manipulate the images and for example share to facebook.

It's fun to explore doing this without relying on the official software.

Storage

The dokicam stores its photos and videos on its memory card in JPG and MP4 format, easily accessible via USB storage without even removing the card.

Projection conversion

Those images and video's show the 'double fish-eye' nature of the device. Services like Facebook, however require 360° imagery to be mapped using the Equirectangular Projection. This can be achieved with ffmpeg using 2 'mapping files' for your image type.

Mapping generation

I did not find a suitable mapping for my camera online. However I did find projection.c by Floris Sluiter which could generate such mapping files for single-fisheye sources, and modified it to support double-fisheye.

Compile the generator code:

gcc -o projection projection.c -lm

Create mapping files for video and photo's:

./projection -x xmap_dokicam_video.pgm -y ymap_dokicam_video.pgm -h 1440 -w 2880 -r 1440 -c 2880 -b 35 -m samsung_gear_360
./projection -x xmap_dokicam.pgm -y ymap_dokicam.pgm -h 2048 -w 4096 -r 2048 -c 4096 -b 75 -m samsung_gear_360

Usage

Once you have created (or downloaded) the mapping files, use them with ffmpeg:

ffmpeg -i photo.jpg -i xmap_dokicam.pgm -i ymap_dokicam.pgm -filter_complex remap out.jpg
ffmpeg -i movie.mp4 -i xmap_dokicam_video.pgm -i ymap_dokicam_video.pgm -filter_complex remap out.mp4

For images, add exif metadata to help e.g. Facebook understand this is 360:

exiftool -ProjectionType="equirectangular" out.jpg

For videos, use Google's Spatial Metadata Injector with the following options:

Quality

The method used for mapping is a rather crude pixel-by-pixel conversion. You can clearly see the 'stitch' where the two images are joined together. You can probably achieve much better results with software that actually 'blends' together the images, like hugin, but that's also a bit more complicated ;).

dualfisheye2equirectangular's People

Contributors

Stargazers

Watchers

dualfisheye2equirectangular's Issues

Segmentation fault

I'm trying to generate remapping for the Theta S with 1280x720 resolution images from dual fish eye cameras. I'm getting a segmentation fault when I run:

./projection -x xmap_thetaS.pgm -y ymap_thetaS.pgm -h 720 -w 1280 -r 640 -c 1280 -b 75 -m thetas

which is confusing. I thought projection was supposed to generate the .pgm files, not use pre-existing ones. What am I missing here?

Rotation and One X

I am trying to find the perfect combination of args for the Insta360 OneX. Not only is this not to obvious, but the rotation of the images are off. The dual fisheye are head to head in the middle of the frame. each one needs to be rotated 90 degrees away from the center line. Do you have any example args for this camera?

Two remap filters for the same stream.

Hi there.
I have a half-baked idea for getting the blending of the two cameras.
I am going to try and modify your code to do this, but to be honest this might take a long time, and I just wanted to put this idea to you as well, just in case you are interested in trying for yourself.

In order to get blending, we need overlap of the images. Now correct me if I am wrong, but I don't think the remap filter supports the blending of two pixels.

So we will have to split the input video into two streams so that we can map a little more of the lens on the output canvas to create the overlap. To re-combine the two streams one stream can be merged with a specially crafted alpha map, and overlaid on the other stream.

This takes care of one seam between the two lenses, but not the other.
There are two seams between the cameras, because each lens is "joined" on both sides by the other lens. The way to correct for this would be to move the "front" camera to the middle of the output canvas, and split the back camera across what is left of the canvas, left and right. Then the video would look more like what ActionDirector or the Gear360 Manager outputs.

Kind regards,
Evert

Example for Kodak PIXPRO sp360

The code references kodak cameras so I hope you know a command that can create a good map for the sp360 (not the 4k version)

I've tried this

./project -x x.pgm -y y.pgm -h 1440 -w 1440 -r 1440 -c 1440 -m front235

then used google's spatial-media python script to add metadata, then played in vlc.

The video is 360, full 360, not 235 degrees.

What is the relationship between x,y and r,c? What is the crop argument?