aetilius / phash Goto Github PK

pHash - the open source perceptual hash library

License: GNU General Public License v3.0

CMake 1.99% Makefile 3.46% C# 0.42% Java 0.46% C++ 28.44% C 23.98% M4 6.79% Awk 0.23% Shell 27.87% JavaScript 0.02% PHP 6.23% sed 0.10%

hash perceptual perceptual-hashing similarity similarity-detection

phash's People

Contributors

Stargazers

Watchers

phash's Issues

Using pHash data from an older version of pHash w/the current version

Apologies, this isn't really a bug, it's more of a question. But it might be useful for someone else down the road...

I was thrilled to discover this project, as we've been using a very old version of pHash (https://github.com/hszcg/pHash-0.9.6, it has some patches applied to it for things I'm assuming you've since addressed here). We have over 2 million existing pHash values for photos in our database calculated by the older version.

We're also using a Ruby wrapper for the pHash lib to compare photos: https://github.com/toy/pHash

It seems that, for a given photo, the pHash values from our old version differ from the value that the latest version produces. It's different enough that we're not able to detect the images as being similar... Is this expected?

And more importantly, is it is possible that this could happen when the pHash library is updated in the future? That a pHash calculated from today's code could be different from one calculated in some future version of the lib?

pHash does not work with MKV and WebM files

GetNumberVideoFrames() only checks for per-stream frame counts or durations, which MKV and WebM containers do not include in their headers. This causes the hashing of these files to fail almost before it even begins.

Solution: adding another fallback resolves that situation and in my (limited) tests allows pHash to work with MKV and WebM containers as well:

if (nb_frames <= 0) {
    nb_frames = static_cast<float>(pFormatCtx->duration) *
        str->avg_frame_rate.num /
        str->avg_frame_rate.den /
        AV_TIME_BASE;
}

Rounding logic looks strange

There's a common rounding pattern throughout phash.cpp like (int)std::floor(d + ROUNDING_FACTOR(d)) where ROUNDING_FACTOR is #define ROUNDING_FACTOR(x) (((x) >= 0) ? 0.5 : -0.5). This produces some unusual results:

-1.0  → -2
-0.75 → -2
-0.5  → -1
-0.25 → -1
 0.0  →  0
 0.25 →  0
 0.5  →  1
 0.75 →  1
 1.0  →  1

The weirdest one there is -0.75 rounds to -2, which seems wrong. I'd be happy to fix this, but I can't tell what the desired behavior is.

Add support for HEIF/HEIC images via libheif

The HEIF/HEIC format just keeps growing in use, so needs to be supported.

cmake does not check for libtiff

leading to compile error:

[ 10%] Building CXX object CMakeFiles/pHash.dir/src/pHash.cpp.o
In file included from /home/vm/Documents/pHash/src/pHash.h:59,
                 from /home/vm/Documents/pHash/src/pHash.cpp:25:
/home/vm/Documents/pHash/third-party/CImg/CImg.h:476:10: fatal error: tiffio.h: No such file or directory
  476 | #include "tiffio.h"
      |          ^~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/pHash.dir/build.make:63: CMakeFiles/pHash.dir/src/pHash.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:110: CMakeFiles/pHash.dir/all] Error 2
make: *** [Makefile:152: all] Error 2

Build fails due to missing libraries.

Hi, I am trying to do a build and getting below error. It could not find jpeglib.h. Please see snapshot.

No ConvNet (pHashML) Documentation

Hi,
There does not appear to be any documentation on how pHashML works?

It only appears at:
http://phash.org/demo/
and there is nothing at:
http://www.phash.org/docs/design.html

I also could not find any references to it in the repository?
Is there any way I can access either the model itself, or the documentation/methodology for it?

Thanks!

Build instructions need updating

The build instructions in INSTALL still reference automake rather than CMake

build issue

I am trying to build with cmake from revised branch

cmake . succeeds but I get following error when I run make

➜ pHash git:(revised-v1.0) ✗ make Scanning dependencies of target pHash [ 3%] Building CXX object CMakeFiles/pHash.dir/src/ph_fft.cpp.o /Users/user/workspace/pHash/src/ph_fft.cpp:29:25: error: constexpr function never produces a constant expression [-Winvalid-constexpr] static constexpr double pi(){ ^ /Users/user/workspace/pHash/src/ph_fft.cpp:30:9: note: non-constexpr function 'atan<int>' cannot be used in a constant expression return std::atan(1)*4; ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:806:1: note: declared here atan(_A1 __lcpp_x) _NOEXCEPT {return ::atan((double)__lcpp_x);} ^ 1 error generated. make[2]: *** [CMakeFiles/pHash.dir/src/ph_fft.cpp.o] Error 1 make[1]: *** [CMakeFiles/pHash.dir/all] Error 2 make: *** [all] Error 2

What is the difference between pHash and pHash-1(other repo)?

You have one more repo called pHash and pHash-1? Which one am I supposed to use?

Moreover, there is one more repo which looks very similar to above two repos. https://github.com/clearscene/pHash

pHash.org Version Outdated

0.9.6 is the latest tar.gz released on pHash.org and is quite old

Doc for ph_hamming_distance is wrong

It currently reads

/* ! /brief dct video robust hash
 *   Compute video hash based on the dct of normalized video 32x32x64 cube
 *   /param file name of file
 *   /param hash ulong64 value for hash value
 *   /return int value - less than 0 for error
 */
int ph_hamming_distance(const ulong64 hash1,const ulong64 hash2);

Unable to create .pylib file with Mac Sonoma.

I want to compile the library into .dylib file in Mac.
However, cmake --build . fails with 'png.h' file not found error.

git clone https://github.com/aetilius/pHash.git
brew install libpng jpeg ffmpeg
mkdir build    
cd build
cmake ../pHash    
cmake --build .

[ 50%] Building CXX object CMakeFiles/pHash.dir/src/pHash.cpp.o
In file included from /Users/tenna/pHash/src/pHash.cpp:25:
In file included from /Users/tenna/pHash/src/pHash.h:60:
/Users/tenna/pHash/third-party/CImg/CImg.h:470:10: fatal error: 'png.h' file not found
#include "png.h"
         ^~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/pHash.dir/src/pHash.cpp.o] Error 1
make[1]: *** [CMakeFiles/pHash.dir/all] Error 2
make: *** [all] Error 2

libpng is already installed.

% brew install libpng            
Warning: libpng 1.6.43 is already installed and up-to-date.

but the png.h location was not in the default /usr/local/include, so I added this to CMakeLists.txt

include_directories("/Library/Frameworks/Mono.framework/Versions/6.12.0/include")

Then, I tried
% cmake ../pHash
% cmake --build .
which produced this error:


[ 50%] Building CXX object CMakeFiles/pHash.dir/src/pHash.cpp.o
In file included from /Users/tenna/pHash/src/pHash.cpp:25:
In file included from /Users/tenna/pHash/src/pHash.h:60:
/Users/tenna/pHash/third-party/CImg/CImg.h:2894:29: warning: 'vsprintf' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use vsnprintf(3) instead. [-Wdeprecated-declarations]
    const int result = std::vsprintf(s, format, args);
                            ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk/usr/include/stdio.h:199:1: note: 'vsprintf' has been explicitly marked deprecated here
__deprecated_msg("This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use vsnprintf(3) instead.")
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk/usr/include/sys/cdefs.h:218:48: note: expanded from macro '__deprecated_msg'
        #define __deprecated_msg(_msg) __attribute__((__deprecated__(_msg)))
                                                      ^
1 warning generated.
make[2]: *** No rule to make target `/opt/homebrew/lib/libmpg123.dylib', needed by `Release/libpHash.1.0.0.dylib'.  Stop.
make[1]: *** [CMakeFiles/pHash.dir/all] Error 2
make: *** [all] Error 2

then I tried to create phash.dylib from pHash.cpp like this.
% g++ -dynamiclib -o phash.dylib pHash.cpp
It fails with this error:

In file included from pHash.cpp:25:
./pHash.h:60:10: fatal error: 'CImg.h' file not found
#include "CImg.h"
         ^~~~~~~~
1 error generated.

./configure fails due to ffmpeg version

Output on Ubuntu 18.04:

*** Configuring image hash ***

checking CImg.h usability... no
checking CImg.h presence... no
checking for CImg.h... no
checking whether CImg.h is in the current or src directory.... no

*** Configuring video Hash ***

checking whether FFmpeg is present... checking for avcodec_alloc_frame in -lavcodec... no
configure: error:

*** libavcodec not found.
You need FFmpeg. Get it at <http://ffmpeg.org/>

Seems like its not compatible to ffmpeg version:
[(https://stackoverflow.com/questions/30107520/error-installing-phash-on-ubuntu)]

FFmpeg is definitely installed under /usr/local/bin. Do you know how to fix this?

Invalid argument(s): Failed to load dynamic library 'phash.so': dlopen failed: cannot locate symbol "TIFFSetWarningHandler" referenced by "/data/app/~~dsq_8wJXYPiEvZhp0h9C8w==/com.example.tcg-n4L1yUnzT6xjvaZKt_iDZA==/lib/arm64/phash.so"...

I built phase.so as below:

export NDK_PATH=/Users/tenna/Library/Android/sdk/ndk/26.2.11394342
export ARCH=arm64-v8a
export TOOLCHAIN=$NDK_PATH/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android34-clang++

$TOOLCHAIN -shared -fPIC -o /Users/tenna/desktop/appname/android/app/src/main/jniLibs/$ARCH/phash.so \
    -I/Library/Frameworks/Mono.framework/Versions/6.12.0/include \
    -I/Users/tenna/pHash/CImg-3.3.5_pre03042417 \
    -L/Users/tenna/anaconda3/pkgs/jpeg-9e-h80987f9_1/lib \
    -L/Users/tenna/anaconda3/pkgs/libtiff-4.5.0-h313beb8_2/lib \
    -L/Users/tenna/anaconda3/pkgs/libpng-1.6.39-h80987f9_0/lib \
    /Users/tenna/pHash/src/pHash.cpp

export NDK_PATH=/Users/tenna/Library/Android/sdk/ndk/26.2.11394342
export ARCH=arm64-v8a
export TOOLCHAIN=$NDK_PATH/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android34-clang++
$TOOLCHAIN -shared -fPIC -o /Users/tenna/desktop/app/android/app/src/main/jniLibs/$ARCH/phash.so \
    -I/Library/Frameworks/Mono.framework/Versions/6.12.0/include \
    -I/Users/tenna/pHash/third-party/CImg \
    -L/Library/Frameworks/Mono.framework/Versions/6.12.0/lib \
    /Users/tenna/pHash/src/pHash.cpp

When I try to open the phase.so file in flutter, it throws errors:

I/flutter ( 7373): getImagePhash.e Invalid argument(s): Failed to load dynamic library 'phash.so': dlopen failed: cannot locate symbol "TIFFSetWarningHandler" referenced by "/data/app/~~dsq_8wJXYPiEvZhp0h9C8w==/com.example.tcg-n4L1yUnzT6xjvaZKt_iDZA==/lib/arm64/phash.so"...
E/flutter ( 7373): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Invalid argument(s): Failed to load dynamic library 'phash.so': dlopen failed: cannot locate symbol "TIFFSetWarningHandler" referenced by "/data/app/~~dsq_8wJXYPiEvZhp0h9C8w==/com.example.tcg-n4L1yUnzT6xjvaZKt_iDZA==/lib/arm64/phash.so"...
E/flutter ( 7373): #0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:11:43)
E/flutter ( 7373): #1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:22:12)
E/flutter ( 7373): #2      lib (package:tcg/templates/hash2.dart:17:22)
E/flutter ( 7373): #3      lib (package:tcg/templates/hash2.dart)
E/flutter ( 7373): #4      calculateImageHash (package:tcg/templates/hash2.dart:23:39)
E/flutter ( 7373): #5      HashHelper2._computeImagePHash (package:tcg/templates/hash2.dart:90:16)
E/flutter ( 7373): #6      HashHelper2.getImagePhash (package:tcg/templates/hash2.dart:49:24)
E/flutter ( 7373): <asynchronous suspension>
E/flutter ( 7373): #7      _MyPageTabState.getdata (package:tcg/pages/my_page.dart:32:18)
E/flutter ( 7373): <asynchronous suspension>
E/flutter ( 7373):

TIFFSetWarningHandler is supposed be imported from libtiff in CImg.h.

I tried reinstalling libtiff.dylib through homebrew which did not solve the problem.
This made me suspect that the version of libtiff installed may differ from the version required for the library.
But I could not find any specification either.

example for video test_dctvideohash.cpp not compiling

I tried to compile the example file for video hashing. At first the compiler couldn't find phash.h and CImg.h. So i moved everything in the src folder. I figured that I have to activate a switch in pHash.h:

#define HAVE_VDEO_HASH
/* #undef HAVE_AUDIO_HASH */
/* #undef HAVE_VIDEO_HASH */
/* #undef HAVE_LIBMPG123 */

but then i get the following error messages when calling gcc test_dctvideohash.cpp:

In file included from test_dctvideohash.cpp:29:0:
cimgffmpeg.h:30:0: warning: "cimg_display" redefined
 #define cimg_display 0

In file included from test_dctvideohash.cpp:28:0:
CImg.h:375:0: note: this is the location of the previous definition
 #define cimg_display 1

In file included from test_dctvideohash.cpp:27:0:
pHash.h:97:5: error: 'CImg' does not name a type
     CImg<uint8_t> *R;           //contains projections of image of angled lines through center
     ^~~~
pHash.h:148:32: error: 'CImg' does not name a type
 int ph_radon_projections(const CImg<uint8_t> &img,int N,Projections &projs);
                                ^~~~
pHash.h:148:36: error: expected ',' or '...' before '<' token
 int ph_radon_projections(const CImg<uint8_t> &img,int N,Projections &projs);
                                    ^
pHash.h:187:28: error: 'CImg' does not name a type
 int _ph_image_digest(const CImg<uint8_t> &img,double sigma, double gamma,Digest &digest,int N=180);
                            ^~~~
pHash.h:187:32: error: expected ',' or '...' before '<' token
 int _ph_image_digest(const CImg<uint8_t> &img,double sigma, double gamma,Digest &digest,int N=180);
                                ^
pHash.h:210:30: error: 'CImg' does not name a type
 int _ph_compare_images(const CImg<uint8_t> &imA,const CImg<uint8_t> &imB,double &pcc, double sigma = 3.5, double gamma = 1.0,int N=180,double threshold=0.90);
                              ^~~~
pHash.h:210:34: error: expected ',' or '...' before '<' token
 int _ph_compare_images(const CImg<uint8_t> &imA,const CImg<uint8_t> &imB,double &pcc, double sigma = 3.5, double gamma = 1.0,int N=180,double threshold=0.90);
                                  ^
test_dctvideohash.cpp: In function 'int main(int, char**)':
test_dctvideohash.cpp:51:13: error: 'ph_dct_videohash' was not declared in this scope
     hash1 = ph_dct_videohash(file1, L1);
             ^~~~~~~~~~~~~~~~
test_dctvideohash.cpp:51:13: note: suggested alternative: 'ph_dct_imagehash'
     hash1 = ph_dct_videohash(file1, L1);
             ^~~~~~~~~~~~~~~~
             ph_dct_imagehash
test_dctvideohash.cpp:57:46: warning: format '%llx' expects argument of type 'long long unsigned int', but argument 3 has type 'ulong64 {aka long unsigned int}' [-Wformat=]
        printf("hash1[%d]=%llx\n", i, hash1[i]);
                                      ~~~~~~~~^
test_dctvideohash.cpp:71:8: error: 'ph_dct_videohash_dist' was not declared in this scope
  sim = ph_dct_videohash_dist(hash1, L1, hash2, L2, 21);
        ^~~~~~~~~~~~~~~~~~~~~
test_dctvideohash.cpp:71:8: note: suggested alternative: 'ph_dct_imagehash'
  sim = ph_dct_videohash_dist(hash1, L1, hash2, L2, 21);
        ^~~~~~~~~~~~~~~~~~~~~
        ph_dct_imagehash

Do you have any idea how to relove this ?

Radial hash ignores gamma parameter?

Looking at ph_image_digest function, I've discovered the following line:

(graysc / graysc.max()).pow(gamma);

It seems for me, that this line doesn't do anything, because graysc.max() returns scalar value, and operator/(const T value) returns new image instance. pow method is called on that instance, but it doesn't make any difference, because the result is not assigned anywhere.
Am I missing something there?

'FORCE' settings in CMakeLists.txt?

What is the reason for the FORCE keyword for these settings? This is overriding -D or set() commands. Isn't the point of these to toggle what isn't needed?

set(USE_IMAGE_HASH ON CACHE BOOLEAN "compile image hashes" FORCE)
set(USE_AUDIO_HASH ON CACHE BOOLEAN "compile audio hashes" FORCE)
set(USE_VIDEO_HASH ON CACHE BOOLEAN "compile video hash" FORCE)
set(USE_TEXT_HASH ON CACHE BOOLEAN "compile text hash" FORCE)

(Also, BOOLEAN needs to be STRING)

Running make fails when building examples/test_dctvideohash.cpp

I used ccmake to configure the Makefiles, setting WITH_VIDEO_HASH to ON. Subsequently building with make, the process throws an error when trying to build examples/test_dctvideohash.cpp.

Error details

/path/to/phash/examples/test_dctvideohash.cpp:44:25: error: use of undeclared identifier
      'ph_dct_videohash'; did you mean 'ph_dct_imagehash'?
        hashes[i - 1] = ph_dct_videohash(argv[i], lengths[i - 1]);
                        ^~~~~~~~~~~~~~~~
                        ph_dct_imagehash
/usr/local/include/pHash.h:229:5: note: 'ph_dct_imagehash' declared here
int ph_dct_imagehash(const char* file,ulong64 &hash);
    ^

Workaround

Looking in pHash.h, I can see #ifdef conditionally wrapping the signature for ph_dct_videohash. I used make -n to find the c++ command that was being used to compile the file that was throwing an error. I added -D HAVE_VIDEO_HASH=1 to that command, and ran it, which successfully compiled. I then continued to run make and make install which completed successfully.

I'm not sure if the problem here is my configuration, my environment (Mac OS 10.14), or something else.

_ph_image_digest function cimg exception when alpha channel present

Function throws the following exception with images that contain an alpha channel as it is not being treated as an RBG.

[CImg] *** CImgInstanceException *** [instance(316,324,1,4,0x7fbd3d307010,non-shared)] CImg<unsigned char>::RGBtoYCbCr(): Instance is not a RGB image. terminate called after throwing an instance of 'cimg_library::CImgInstanceException' what(): [instance(316,324,1,4,0x7fbd3d307010,non-shared)] CImg<unsigned char>::RGBtoYCbCr(): Instance is not a RGB image.

Create a image with a resolution of 195x64 pixels
Run TestRadish with this image
The phash is all 0's

Problem:

in ph_radon_projections, projs.nb_pix_perline at index 134 is not changed, i.e. this value is 0

in ph_feature_vector, there is a division with the 0 value from projs.nb_pix_perline[134] (nb_pixels):

feat_v[k] = (line_sum_sqd / nb_pixels) -
            (line_sum * line_sum) / (nb_pixels * nb_pixels);

aetilius / phash Goto Github PK

phash's People

Contributors

Stargazers

Watchers

Forkers

phash's Issues

Error details

Workaround

Recommend Projects

Recommend Topics

Recommend Org