fine-tuning Few-Shot Relation Classification about ask2transformers HOT 2 CLOSED

Hi @hanjiale ,

The hyperparameters you have used looks fine to me. Maybe I can suggest you to try 1e-5 as learning rate and increase the epochs to 5. You can also use the load_best_model_at_end with accuracy (or defining the entailment f1_score) as metric_for_best_model to improve the checkpoint selection. But in any case, those hyperparameters should be fine. I am concerned about the optimal_threshold value... Did you use the development set, at least on a small portion of it, to optimize the threshold? Maybe that explains the huge gap between the precision and recall.

One of the weakness points of the approach is the need tof tunning the threshold, at least on a very small portion of data. We are working on it and have very promising results, but the improvements are still work in progress.

Let me know,
Oscar

from ask2transformers.

I see. The problem is that I remove the "dev_path" in the tacred.relation.config.json. Thank you for your reply and looking forward to your new work!

from ask2transformers.