Openset f-measure calculation problem about openlongtailrecognition-oltr HOT 3 CLOSED

Hello @Lilyo , thank you very much for asking. Yes, I agree this metric is a little bit confusing. The reason why the authors treat images from the open set that are incorrectly classified as known samples as false negative is because this whole setting is fundamentally an open set detection problem. We need to identify open set samples at the same time be correct for the closed set classification. When considering both aspects, the authors designed this specific metric. This definitely is not a perfect metric, and we think it is still an open problem that needs improvement from all different directions. Does that make sense?

from openlongtailrecognition-oltr.

hi, @zhmiao , I have checked the paper you mention,

Original content:
"For example, when testing the system with images from validation set, fooling set and open set (see Fig. 3), true positives are defined as the correct classifications on the validation set, false positives are incorrect classifications on the validation set and false negatives are images from the fooling set and open set categories that the system incorrectly classified as known examples."

right now I known you follow the same setting from above paper!
BUT, the original meaning of "false negatives" is mean data from validation set, and incorrectly classified to fooling set and open set categories. Why they define as "images from the fooling set and open set categories that the system incorrectly classified as known examples"?

from openlongtailrecognition-oltr.

Yes, @zhmiao , this makes more sense now!

from openlongtailrecognition-oltr.