Verifying the release caffemodel in the result of Megaface about sphereface HOT 10 OPEN

I get 73.6888%. This number is still too high. According to the paper, a 64-layer ResNet could get 72.729%. A 20-layer network should not do better than a 64-layer network.

I guess this is because of the alignment. Megaface is a strange dataset that if you do worse on alignment, you can get better result. I have tried my best to align the dataset, but I can't get accurate keypoints on lots of images. I directly crop a region as the aligned face on each of these images. This may cause the distractors too weak to compete with the probes, which would make the performance better......

from sphereface.

Hi,
Well, to be fair and the issue of practical, I only used the provided (groundtruth) landmark provided in json files to normalise the faces and I think others will do the similar procedure. The reason is that there is impossible to check 1M faces results using the other facial landmark algorithm as some images have more than one face and it is also very difficult to manual annotate the landmark when the algorithm fails.
Another issue I point out is the result of age-invariant, my result is significantly worse than the paper they released. I think the architecture of 20 layers they released may not the one they use for the megaface test as one of the authors in this forum has mentioned that.

from sphereface.

@happynear
hi, do you mean that you got 73.6888% by using the released 20 layer model? Could you please show the exact preprocessing code about MTCNN and alignment used for MegaFace and FaceScrub dataset? I can't get comparable results in MegaFace with yours'. Maybe there is something wrong in my preprocessing code.
BTW, what do you do if the face can't be detected using the algorithm?
Thanks in advance!

from sphereface.

@kalyo-zjl ,
I have uploaded the codes to my repository https://github.com/happynear/FaceVerification/tree/master/dataset/Megaface . The detection and alignment logic is also described in the ReadMe file.
You may not use Matlab and my implementation of MTCNN, but you may refer to my procedure of detection and alignment.

from sphereface.

@happynear ,
Thank you, I will check it. Yes, I will refer to your procedure of detection and alignment, and use the python version of MTCNN and alignment, in which the result should be similar.

from sphereface.

@happynear
Hi, I have a question. How to evaluate sphereface on Megaface ?
The sphereface uses cosine similarity to measure the distance.

from sphereface.

You can normalize the embedding feature first. when X, Y are normalized. d(X,Y) = 2 - 2 cos(X,Y). They just have the same effect

from sphereface.

@kalyo-zjl em...... you mean that once I save the normalized embeddings to '.bin' file, I can directly
run the code "run_experiment.py" to get correct results?

from sphereface.

@chichan01 @happynear @kalyo-zjl Hi all, I know d(X,Y) = 2 - 2 cos(X,Y). But how to define a new score model when I evaluate my model on Megaface?
run_experiment.py -s 1000000 -p ../templatelists/facescrub_features_list.json -m ???
Thanks!

from sphereface.

Hi,
Do we need to normalize the input images of Facescrub and Megaface before feeding them to the model?

from sphereface.