Nice work!
I have some problems of the metrics used in the table1, especially about the task of text2text(paragraph-actions).
You have mentioned that “For the forward prediction task the metric is accuracy; for the retrosynthesis task the metric is roundtrip accuracy (Schwaller et al., 2020); for all the other tasks the BLEU score.”
So what metric do you use in the task of text2text(paragraph-actions)? BLEU-2 or BLEU-4?
Hi, thank you for sharing this interesting work.
I'm wondering if the method still requires a fine-tuning process? As I understand it, a unified model is obtained by mixing 4 different tasks in the pre-training stage, and then inference can be performed on the test set directly without fine-tuning. However, the Code Availability section in the paper mentions that "model is available for inference, training and finetuning via the GT4SD library". So I am a little confused about the process of training, fine-tuning and inference on the test set.
Traceback (most recent call last):
File "xxx/anaconda3/envs/gt4sd/bin/gt4sd-trainer", line 8, in
sys.exit(main())
File "xxx/anaconda3/envs/gt4sd/lib/python3.8/site-packages/gt4sd/cli/trainer.py", line 185, in main
pipeline().train(**config)
TypeError: train() missing 3 required positional arguments: 'pl_trainer_args', 'model_args', and 'dataset_args'
I cannot reproduce the results of the paper through the weights on the huggingface repo:GT4SD/multitask-text-and-chemistry-t5-base-augm ,whose accuracy is supposed to 0.322,but I test it only to be 0.196.
Same question, I cannot reproduce the paragraph2action results of the paper through the weights on the huggingface repo: https://huggingface.co/GT4SD/multitask-text-and-chemistry-t5-small-standard, whose BLEU score is supposed to 0.929, but I test it only to be 0.659. Does the model need to be further finetuned?