Comments (4)
In addition, I noticed that the F1@10 > F1 @5, while in your paper, F1@10 < F1@5
from opennmt-kpg-release.
Thank you for your questions. I cannot give any feedback without knowing your running arguments and dataset.
from opennmt-kpg-release.
Thanks for your reply!
I just run this command which is provided in README
python kp_gen_eval.py -tasks pred eval report -config config/test/config-test-keyphrase-one2seq.yml -data_dir data/keyphrase/meng17/ -ckpt_dir models/keyphrase/meng17-one2seq-kp20k-topmodels/ -output_dir output/meng17-one2seq-topbeam-selfterminating/meng17-one2many-beam10-maxlen40/ -testsets nus -gpu -1 --verbose --beam_size 10 --batch_size 32 --max_length 40 --onepass --beam_terminate topbeam
and got the results in the picture.
The F1@10 is 0.3772 while in the paper "Does order matters?" the result is 0.295.
I used your provided pre-trained models.
Thanks very much!
from opennmt-kpg-release.
The default search strategy is full
as set in opts.py L670. This means it during beam search every search sequence will not be terminated until it reaches the max_len (say 50). It can boosts scores a lot when beam_size is small at the cost of slow searching speed. topbeam
has merits of fast inference as well as comparable performance when beam_size is large enough. The F1 0.295 is based on the topbeam
decoding. We will provide more comparisons about these two settings in the paper a little later.
from opennmt-kpg-release.
Related Issues (20)
- Scoring question HOT 7
- Problem with generating kephrases (inference.ipynb file)
- Real train & eval dataset statistics HOT 3
- Missing model files HOT 3
- Is this project still working? HOT 3
- Error in inference.ipynb, when loading pre-trained model HOT 9
- Question about exact-matching between predicted and groundtruth keyphrases HOT 1
- Train configuration file does not exitst HOT 2
- Fine tune on custom dataset HOT 3
- How to use Bart model in OPenNMT HOT 3
- AttributeError: module 'torchtext.data' has no attribute 'Iterator' HOT 4
- Pump torchtext from 0.8 to 0.12 HOT 2
- I can not find json_process.ipynb in notebook HOT 1
- config/transfer_kp/train/transformer-presabs-kp20k.yml HOT 2
- MAGKP dataset HOT 4
- tokenizing of letters, on prediction HOT 1
- about keyphrase representation in General-to-Specific Transfer Labeling for Domain Adaptable Keyphrase Generation HOT 1
- can you release the total code of catseqd? HOT 1
- Vocab? HOT 1
- catSeq/catSeqD predictions
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opennmt-kpg-release.