Git Product home page Git Product logo

Comments (8)

uricamic avatar uricamic commented on July 21, 2024

Hi,

I think the problem is that the learned model available on the webpage is still quite weak. It was learned on around 3,500 examples only for the whole multi-view scenario (i.e. 5 yaw ranges).
I will upload some new models on the project webpage soon. It should be a bit better, however, we will soon learn a new models with much more examples.

It could be also interesting to visualize the face box as well. I believe some precision gain might be obtained by stabilizing the noisy face detection (e.g. by a Kalman filter).

from clandmark.

uricamic avatar uricamic commented on July 21, 2024

Hi @RafaRuiz,

I almost forgot to ask, do you use CFeaturePool * in your code? If not then the problem is that the model which you are using is supposing that you use it. So the resulting precision would be totally unpredictable.

from clandmark.

RafaRuiz avatar RafaRuiz commented on July 21, 2024

Hello @uricamic, This is the new code. Basically I added the example of the CFeaturePool* from static_input.cpp to the code after declaring the Flandmark flandmark:

main function:

int main( int argc, const char** argv )
{

    //CvCapture* capture = 0x0;
    //CvVideoWriter* writer = 0x0;
    VideoCapture capture;
    VideoWriter writer;

    Mat frame;

    double tic;
    bool saveoutput = false;
    string out_fname;

    Flandmark *flandmark = Flandmark::getInstanceOf(landmarks_path.c_str());
    CFeaturePool *featurePool = new CFeaturePool(flandmark->getBaseWindowSize()[0], flandmark->getBaseWindowSize()[1]);
    featurePool->addFeaturesToPool(
                                   new CSparseLBPFeatures(
                                                          featurePool->getWidth(),
                                                          featurePool->getHeight(),
                                                          featurePool->getPyramidLevels(),
                                                          featurePool->getCumulativeWidths()
                                                          )
                                   );

    flandmark->setNFfeaturesPool(featurePool);

    //-- 1. Load the cascades
    if( !face_cascade.load( cascade_path ) )
    {
        cerr << "Couldn't load the haar cascade. Exiting..." << endl;
        return -1;
    };


    capture.open(0);
    // window
    namedWindow(window_name, CV_WINDOW_KEEPRATIO);
    //    namedWindow(window_name, WINDOW_KEEPRATIO);

    //-- 2. Read the video stream
    if( capture.isOpened() )
    {
        while( true )
        {
            tic = (double)getTickCount();

            capture >> frame;

            resize(frame, frame, Size(frame.size().width/3, frame.size().height/3));

            //-- 3. Apply the classifier to the frame
            if( !frame.empty() )
            {
                detectAndDisplay( frame, flandmark );

                tic = ((double)getTickCount() - tic)/getTickFrequency() * 1000;

                stringstream fps;
                fps << "fps: " << setprecision(4) << setw(4) << 1000.0 / tic << " ";
                putText(frame, fps.str(), Point(10, 25), CV_FONT_HERSHEY_COMPLEX, 1, Scalar(255, 255, 0));
                //              putText(frame, fps.str(), Point(10, 25), FONT_HERSHEY_COMPLEX, 1, Scalar(255, 255, 0));

                imshow( window_name, frame );

            } else {
                cerr << "No frame --- break." << endl;
                break;
            }

            if (saveoutput)
            {
                writer << frame;
            }

            int c = waitKey(10);
            if( (char)c == 'c' )
            {
                break;
            }
        }
    }

    delete flandmark;

    return 0;
}

Detect and Display function

void detectAndDisplay( Mat &frame, Flandmark *flandmark)
{
    std::vector<Rect> faces;
    Mat frame_gray;
    //  int bbox[4];
    int bbox[8];
    fl_double_t *landmarks;

    cvtColor( frame, frame_gray, CV_BGR2GRAY );
    //    cvtColor( frame, frame_gray, COLOR_BGR2GRAY );
    equalizeHist( frame_gray, frame_gray );

    //-- Detect faces
    face_cascade.detectMultiScale( frame_gray, faces, 1.1, 2, 0|CV_HAAR_SCALE_IMAGE, Size(30, 30) );
    //  face_cascade.detectMultiScale( frame_gray, faces, 1.1, 2, 0|CASCADE_SCALE_IMAGE, Size(30, 30) );

    //    CImage *frm_gray = new CImage();
    cimg_library::CImg<unsigned char>* frm_gray = 0x0; //= new cimg_library::CImg<unsigned char>();

    for( uint32_t i = 0; i < faces.size(); i++ )
    {
        // Get detected face bounding box
        bbox[0] = faces[i].x;
        bbox[1] = faces[i].y;
        bbox[2] = faces[i].x + faces[i].width;
        bbox[3] = faces[i].y;
        bbox[4] = faces[i].x + faces[i].width;
        bbox[5] = faces[i].y + faces[i].height;
        bbox[6] = faces[i].x;
        bbox[7] = faces[i].y + faces[i].height;

        // Detect facial landmarks
        frm_gray = cvImgToCImg(frame_gray);
        flandmark->detect(frm_gray, bbox);

        delete frm_gray;

        // Get detected landmarks
        landmarks = flandmark->getLandmarks();

        // Draw bounding box and detected landmarks
        //      rectangle(frame, Point(bbox[0], bbox[1]), Point(bbox[2], bbox[3]), Scalar(255, 0, 0));
        circle(frame, Point(int(landmarks[0]), int(landmarks[1])), 2, Scalar(255, 0, 0), -1);
        rectangle(frame, faces.at(i), Scalar(0,255,0), 3);
        for (int i=2; i < 2*flandmark->getLandmarksCount(); i+=2)
        {
            circle(frame, Point(int(landmarks[i]), int(landmarks[i+1])), 2, Scalar(0, 0, 255), -1);
        }

        // Textual output
        printTimingStats(flandmark->timings);
        printLandmarks(landmarks, flandmark->getLandmarksCount());
        printLandmarks(flandmark->getLandmarksNF(), flandmark->getLandmarksCount());
    }

    //    delete frm_gray;
}

and this is the result:

su39b

from clandmark.

RafaRuiz avatar RafaRuiz commented on July 21, 2024

Also tried with:

string landmarks_path = "/Users/rafaelruizmunoz/Downloads/independent_mv_models/INDIVIDUAL_FRONTAL_AFLW_SPLIT_1.xml";

working in the same way

from clandmark.

uricamic avatar uricamic commented on July 21, 2024

Hi @RafaRuiz,

call flandmark->detect_optimized(frm_gray, bbox); instead of flandmark->detect(frm_gray, bbox);
That should do the job.

I am sorry for this inconvenience, it is not described anywhere yet, but there is an old version of computing features for compatibility reasons (our previous version of the library called flandmark).

The difference in INDIVIDUAL_FRONTAL_AFLW_SPLIT_1.xml and JOINT_MV_AFLW_SPLIT_1_frontal is that the first one was learned as an independent single-view detector, while the second is learned jointly with all 5 yaw ranges.

from clandmark.

RafaRuiz avatar RafaRuiz commented on July 21, 2024

now it seems to be more stable!
no sorry to be about :)

Thank you very much for your answers, I can't see the moment to start playing with this ;)

from clandmark.

uricamic avatar uricamic commented on July 21, 2024

Glad to hear that! :)

I will upload the learning scripts for MATLAB soon and Python interface will follow up shortly.

from clandmark.

karlhugle avatar karlhugle commented on July 21, 2024

@RafaRuiz Hi, can you write any instructions about setting up it using Xcode, tried but failed....

from clandmark.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.