Question regarding THR about torchcp HOT 3 CLOSED

Dear Thomas，

Thank you for the interest in this toobox. We list our replies for each question below.

• The weakness of THR. Yes, THR can achieve excellent performance in the average size. However, it generally perform bad in conditional coverage, as shown in Table 11 of RAPS [1]. In addition, the introduction of conformal prediction (CP) [2] gives a detailed discussion of THR.
• "Naive" method. We cannot find the "naive" word in the paper [3]. As presented in README.md, we provide the list of implementations and their papers in the table. The method of paper [3] is implemented as "classification.predictors.cluster". The "classification.scores.margin" correspondings to the paper [4].

Please let us know if you have any more questions.

Best regards,
authors

[1] Uncertainty Sets for Image Classifiers using Conformal Prediction
[2] A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
[3] Class-Conditional Conformal Prediction with Many Classes
[4] Bias reduction through conditional conformal prediction

from torchcp.

Thanks for the detailed answers and providing helpful resources.
I will include "SSCV" then in my experiments. As a side note, it would be really cool if the metrics contained references as well.

Also, I unfortunately messed up the references. "naive" was in the RAPS paper aswell.
I however don't think it is Margin.

Best regards,
Thomas

from torchcp.

Dear Thomas，

Sorry for the late response. "naive" presented in Raps is not included in TorchCP.

Additionally, we have updated the references for the metrics. We hope this assists you in your research.

Please let us know if you have any more questions.

Best regards,
authors

from torchcp.