zweigraf / face-landmarking-ios Goto Github PK

Hi, Luis Reisewitz.
First of all thank you for the excellent work, your app works very well.
My issue is that When i tried to reproduce your project I had slow fps on my iPhone (when I compiled Your project it worked with good fps)

Below what I did:

1. Created new project
2. Copied all source files (both DlibWrapper, ViewController, SessionHandler, Bridging-Header, Storyboard file)
3. Created lib directory and copied from your directory dlib source code, libdlib.a, shape_predictor_68_face_landmarks
4. In Build Settings I tuned next items:

• Header search path and Library search path similar with your settings
• Preprocessor macros were set as well.

5. In Build Phases I added Accelerate.framework and libdlib.a to Link Binary..
6. To Copy bundle resources was added shape_predictor_68_face_landmark

What's I forget?
P.S. here's my rap, just in case: github.com/WhiteRaby/Dlib

If I wanted to print the 68 points of dlib what would be the syntax? I'd like to print in the dlibwrapper.mm file

I can't draw correct circle at Portrait mode.
I find shape, but not circle, just line. like this.

How can I draw correct circle for face landmark at Portrait mode?

I only fixed two things.

1. Check Portrait at Device Orientation. (and select target iOS 13.2. but this setting is maybe unrelated this problem)

2. Add "AVCaptureVideoOrientation.portrait"

I checked #5 but I can't understand this problem.

My Environment

MacOS Catalina 10.15.2
Xcode 11.3.1
iPhone7 (iOS 13.3.1
commit version 748f2ea

Hi, wondering if you can speak or provide guidance to this specific link time issue I'm having

_USER_ERROR__missing_dlib_all_source_cpp_file__OR__inconsistent_use_of_DEBUG_or_ENABLE_ASSERTS_preprocessor_directives

This happened when I tried to port the libdlib.a and wrapper classes to my project- I have attempted to change all build settings in my project to look like the example project. Any ideas?

Open on iphone7, landmark undetectable？

Hi Zweigraf, great job with the detector, it's a great example to learn how to use dlib in iOS!

I noticed I don't see any compiler or linker flags... is this intended? How do you use lapack, blas, png support etc. without any flags? Also, won't there be some flags to speed up the program by deactivating some debug assert?

when I use dlib in my projects, Xcode says that 'X11/Xlib.h' not found. I have used the same instructions to build it: http://stackoverflow.com/questions/34591254/how-to-build-dlib-for-ios/35058969#35058969.
It also appears when I use your lib folder and set it in Header Search Paths. This is my first project where I have to change the build settings for a 3rd party lib

Lexical or Preprocessor Issue
-'X11/Xib.h' file not found nativefont.h /Users/jan.driesen/Projects/dlib-iphone-compile/dlib_app/dlib_app/dlib_app/library/dlib/all/source.cpp:64:10: In file included from /Users/jan.driesen/Projects/dlib-iphone-compile/dlib_app/dlib_app/lib/dlib/all/source.cpp:64:

Hi ,
Thank your for a amazing project , i need to get Coordinates for specific face parts like eye and mouth to crop it later
thanks on advance

Hey, nice project <3 I was wondering can you get landmarks without setting video settings to BGRA and using the default 420f? or Somehow convert the CMSampleBuffer in the captureoutput from BGRA to 420f?

HI, great job, very helpfull project. I'm working on face detection for now, but I need detect face landmarks from an UIImage. Could you give me a hand how to switch the detection from video output? I don't have matedata and faceObject rects as you in AVCaptureVideoDataOutputSampleBufferDelegate, sou I use custom CIDetectorFaceType. But rects from my detector are very diferent from yours, so detected points arent correct. Do you have any hint what to do? Thanks

Currently, landmarks seems noisy and jump a lot (clear environment with sufficient light ), this is however a lot more stable when compiling dlib for OSX, is there something that we can do to improve the noise level on iOS?

Many thanks for your awesome work!

Hi, first of all, thanks a lot for the library! It's great!!
I'm having this issue everytime, and I don't know how to fix it:

ios/lib/dlib/image_processing/scan_image.h:6:10: 'vector' file not found

This happens when I include this line in the .m file:

#include <dlib/image_processing.h>

Anyone knows why this is happening?

Thank you very much!!

Agustin

Is there any workaround to enhance this method?:

dlib::deserialize(modelFileNameCString) >> sp;

because when i debug, this method takes about 5s to finish

2017-08-04 18:20:20.906752+0700 xxapp[3460:819204] start deserialize: 1501845620.906730
2017-08-04 18:20:26.145505+0700 xxapp[3460:819204] finish deserialize: 1501845626.144248

I had tried to set the camera orientation to Landscape or Portrait but the below code (in DlibWrapper.mm) still return width = 640 and height = 480 (with preset is AVCaptureSessionPreset640x480).
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
Then I couldn't do the landmark detection in Portrait view. Could you fix it?

Hi, I was trying to make my own app using some dlib, content but I have this error.

ld: warning: ignoring file /Users/sebastiancorrea/Documents/UPV/Master_IARFID/TFM/Bloque2/mobile-emotion-classifier/mobile/IOS/FaceVision/FaceVision/lib/libdlib.a, file was built for archive which is not the architecture being linked (arm64): /Users/sebastiancorrea/Documents/UPV/Master_IARFID/TFM/Bloque2/mobile-emotion-classifier/mobile/IOS/FaceVision/FaceVision/lib/libdlib.a
Undefined symbols for architecture arm64:
"dlib::entropy_decoder_kernel_2::set_stream(std::__1::basic_istream<char, std::__1::char_traits >&)", referenced from:
dlib::compress_stream_kernel_1<dlib::entropy_encoder_model_kernel_5<257ul, dlib::entropy_encoder_kernel_2, 200000ul, 4ul>, dlib::entropy_decoder_model_kernel_5<257ul, dlib::entropy_decoder_kernel_2, 200000ul, 4ul>, dlib::crc32>::decompress(std::__1::basic_istream<char, std::__1::char_traits >&, std::__1::basic_ostream<char, std::__1::char_traits >&) const in OpenCVWrapper.o
"dlib::entropy_decoder_kernel_2::entropy_decoder_kernel_2()", referenced from:
dlib::compress_stream_kernel_1<dlib::entropy_encoder_model_kernel_5<257ul, dlib::entropy_encoder_kernel_2, 200000ul, 4ul>, dlib::entropy_decoder_model_kernel_5<257ul, dlib::entropy_decoder_kernel_2, 200000ul, 4ul>, dlib::crc32>::decompress(std::__1::basic_istream<char, std::__1::char_traits >&, std::__1::basic_ostream<char, std::__1::char_traits >&) const in OpenCVWrapper.o
"dlib::entropy_decoder_kernel_2::get_target(unsigned int)", referenced from:
dlib::compress_stream_kernel_1<dlib::entropy_encoder_model_kernel_5<257ul, dlib::entropy_encoder_kernel_2, 200000ul, 4ul>, dlib::entropy_decoder_model_kernel_5<257ul, dlib::entropy_decoder_kernel_2, 200000ul, 4ul>, dlib::crc32>::decompress(std::__1::basic_istream<char, std::__1::char_traits >&, std::__1::basic_ostream<char, std::__1::char_traits >&) const in OpenCVWrapper.o
dlib::entropy_decoder_model_kernel_5<257ul, dlib::entropy_decoder_kernel_2, 200000ul, 4ul>::decode(unsigned long&) in OpenCVWrapper.o
"dlib::entropy_decoder_kernel_2::decode(unsigned int, unsigned int)", referenced from:
dlib::compress_stream_kernel_1<dlib::entropy_encoder_model_kernel_5<257ul, dlib::entropy_encoder_kernel_2, 200000ul, 4ul>, dlib::entropy_decoder_model_kernel_5<257ul, dlib::entropy_decoder_kernel_2, 200000ul, 4ul>, dlib::crc32>::decompress(std::__1::basic_istream<char, std::__1::char_traits >&, std::__1::basic_ostream<char, std::__1::char_traits >&) const in OpenCVWrapper.o
dlib::entropy_decoder_model_kernel_5<257ul, dlib::entropy_decoder_kernel_2, 200000ul, 4ul>::decode(unsigned long&) in OpenCVWrapper.o
"dlib::entropy_decoder_kernel_2::~entropy_decoder_kernel_2()", referenced from:
dlib::compress_stream_kernel_1<dlib::entropy_encoder_model_kernel_5<257ul, dlib::entropy_encoder_kernel_2, 200000ul, 4ul>, dlib::entropy_decoder_model_kernel_5<257ul, dlib::entropy_decoder_kernel_2, 200000ul, 4ul>, dlib::crc32>::decompress(std::__1::basic_istream<char, std::__1::char_traits >&, std::__1::basic_ostream<char, std::__1::char_traits >&) const in OpenCVWrapper.o
"USER_ERROR__missing_dlib_all_source_cpp_file__OR__inconsistent_use_of_DEBUG_or_ENABLE_ASSERTS_preprocessor_directives", referenced from:
_dlib_check_consistent_assert_usage in OpenCVWrapper.o
"dlib::base64::decode(std::__1::basic_istream<char, std::__1::char_traits >&, std::__1::basic_ostream<char, std::__1::char_traits >&) const", referenced from:
dlib::get_serialized_frontal_faces() in OpenCVWrapper.o
"dlib::base64::base64()", referenced from:
dlib::get_serialized_frontal_faces() in OpenCVWrapper.o
"dlib::base64::~base64()", referenced from:
dlib::get_serialized_frontal_faces() in OpenCVWrapper.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Can some one help me, Thank you.

When i drag the libdlib.a in my Xcode,the error "Source file is not valid UTF-8" appeared,and could you tell me how to solve？

As the landmarks file is very big (around 90mb) its causes the IOS app to be much larger in size.
Do you think there is a way reduce the size of the file (maybe zip it in some way)?

How can I fill the color ? I try to use with uibezierpath and Point position are not same. I cloudn't find a way to draw polygon with dlib.

Is it possible to draw line and fill like uibezierpath in dlib or I need to recalculate the pixel ?

I think the high resolution causes this issue. on iphone 7, portrait camera does not recognize faces. If anyone solved this please help me too.

is Dlib library free and OpenSource?

First of all, congratulations for your good job!
My question is: Is there any way to grab the coordinates (x, y) of the points detected by the algorithm, inside the "SessionHandler.swift" file? (more specifically inside the "captureOutput" function). I'm trying to figure out how to do this for a few days, but I couldn't get any further, because I'm having difficuties to understand Objective C and adapt the code. Thank you for sharing!

Hi! During the restart of the application, the picture from the camera does not appear and writes a message:
Ignoring enqueueSampleBuffer: because status is "failed"
Are there any solutions?

Sorry guys. I have the same question. I develop my own app with face landmark detection and use this project as an example. I installed dlib with cocoa pod, also tried library provided in this repository and every time I include <dlib/image_processing.h> I get
.../Pods/Headers/Public/dlib/dlib/image_processing/scan_image.h:6:10: 'vector' file not found.
If I include <dlib/image_processing/frontal_face_detector.h> I get
.../Pods/dlib/build/install/include/dlib/algs.h:17:10: "Dlib requires C++11 support. Give your compiler the -std=c++11 option to enable it." .../Pods/dlib/build/install/include/dlib/algs.h:93:10: 'string' file not found
All code is implemented in .mm file without .h. All pre-compiler and complier flags correspond to @zweigraf project. Please help. Thank you

Hey, is there any way to slow process time? I want to get positions twice a second?

People, I'm getting crazy with something. Im trying to send a UIImage to the function, but it is very slow (more than 10 seconds)... Can this be related to the size of the image or it quality?

Show you part of my code:

First of all I created this function in the DLIB Wrapper:

`- (void)processImage:(UIImage*) uiImage {

CGFloat width = uiImage.size.width, height = uiImage.size.height;
CGContextRef context;
size_t pixelBits = CGImageGetBitsPerPixel(uiImage.CGImage);
size_t pixelBytes = pixelBits/8;
size_t dataSize = pixelBytes * ((size_t) width*height);
uchar* imageData = (uchar*) malloc(dataSize);
memset(imageData, 0, dataSize);
CGColorSpaceRef colorSpace = CGImageGetColorSpace(uiImage.CGImage);

CGBitmapInfo bitmapInfo = kCGImageAlphaPremultipliedLast;
bool isGray = false;
if (CGColorSpaceGetModel(colorSpace) == kCGColorSpaceModelMonochrome) {
    // gray image
    bitmapInfo = kCGImageAlphaNone;
    isGray = true;
}
else
{
    
}

  context = CGBitmapContextCreate(imageData, (size_t) width, (size_t) height,
                                  8, pixelBytes*((size_t)width), colorSpace,
                                 bitmapInfo);
 CGContextDrawImage(context, CGRectMake(0, 0, width, height), uiImage.CGImage);
  CGContextRelease(context);

  dlib::array2d<dlib::bgr_pixel> dlibImage;

  dlibImage.clear();
  dlibImage.set_size((long)height, (long)width);
  dlibImage.reset();
  long position = 0;
  while (dlibImage.move_next()){
      dlib::bgr_pixel& pixel = dlibImage.element();

     long offset = position*((long) pixelBytes);
      uchar b, g, r;
      if (isGray) {
          b = imageData[offset];
          g = imageData[offset];
          r = imageData[offset];
      } else {
          b = imageData[offset];
          g = imageData[offset+1];
          r = imageData[offset+2];
      }
      pixel = dlib::bgr_pixel(b, g, r);
      position++;
  }

  std::vector<dlib::rectangle> dets = detector(dlibImage);

 for (unsigned long j = 0; j < dets.size(); ++j)
  {
      dlib::full_object_detection shape = sp(dlibImage, dets[j]);

      for (unsigned long k = 0; k < shape.num_parts(); k++) {
          dlib::point p = shape.part(k);
          draw_solid_circle(dlibImage, p, 3, dlib::rgb_pixel(0, 255, 255));
      }
  }

  free(imageData);

}
`

By the other hand, this is the function i have defined in the Header:

- (void)processImage:(UIImage*)uiImage;

I'm calling this function in the next way:

`

                                                                  [self.libreria processImage:image];

`

Of course, thanks a lot for your help!!! Probably I'm doing something wrong!

Best regards

There is a strange problem with this lib when working on images with resolution 1080x1920 (WIDTHxHEIGHT). It produces artefacts like this:

Not only drawing is corrupted but also points positions.

To reproduce this bug You only need to add this:

if let connection = output.connectionWithMediaType(AVMediaTypeVideo) {
    if connection.supportsVideoOrientation {
        connection.videoOrientation = .Portrait
    }
}
session.sessionPreset = AVCaptureSessionPreset1920x1080

Just before session.commitConfiguration() in func openSession(). Not every device will be able work in this resolution in front camera, so changing to back camera might help.

Smaller images with resolution 720x1280 and 1280x720 works fine (I did not check smaller images than 720x1280). Even 2160x3840 and 3840x2160 works fine, so there is something wrong only with resolution 1080x1920. On other hand resolution 1920x1080 works correctly but it is for landscape and I need portrait orientation. So for now I'm using:
session.sessionPreset = AVCaptureSessionPreset1280x720

Do You know what could be wrong?

I download the code, works fine in xcode, how do I import the library to another xcode project? I tried adding it and it says it cant find the file, and I updated framework search paths, is there a compiler setting I need to use or something? Thanks for your time.....

__GLOBAL__sub_I_DlibWrapper.mm in DlibWrapper.o
ld: symbol(s) not found for architecture i386
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I downloaded the github files as is, but I am getting the above error. Are there any other steps I need to take to get dlib to compile correctly?

Hi this is great job !!! it's possible to detect face in video in view ? not live cam video something like this https://www.assetstore.unity3d.com/en/#!/content/66602

I want to use still image from my phone gallery for landmarking.How can I do that?

How can i put mask on face?
Anyone have idea about it.
Please share with me.

How can I convert SessionHandler.swift file to objective-c

Is it possible to get head rotation angle with only dlib? In every example ( like this ) people use OpenCV. This is a good solution, but increases app size too much (dlib + model + opencv = 250 mb).