Comments (4)
No, that's correct. The reason is because the reference edits in Codalab were annotated by humans, while the hypothesis edits were extracted automatically with ERRANT. Consequently, even though the corrected sentences are the same, there is a mismatch between the spans of the gold and automatic edits.
You can read a bit more about this in Section 4.1 of the ERRANT paper and also Appendix A.7 in the BEA-19 shared task paper.
from errant.
Thank you for your detailed explanation.
from errant.
There is a script on the BEA 2019 shared task website to apply the edits to get the corrected text: link
As for the original text, I just run:
grep ^S m2_file | cut -c 3- > output.orig
from errant.
There is a script on the BEA 2019 shared task website to apply the edits to get the corrected text: link
As for the original text, I just run:
grep ^S m2_file | cut -c 3- > output.orig
I run the code to get correct sentences from BEA2019 dev m2. But the F0.5 score is only 86.45 when I upload the correct sentences to Codalab of BEA2019. Did I do something wrong?
The command that I run:
python office_correct.py ~/Corpus/BEA2019/wi+locness/m2/ABCN.dev.gold.bea19.m2 -out office.gold
from errant.
Related Issues (20)
- Errant incompatible with spacy 3 HOT 9
- Expose errant_compare functionality via the API HOT 3
- Merge Casing Issue HOT 2
- Handling Missing Annotations on certain sentence HOT 5
- Edits missed for a substitute -> Delete -> Substitute sequence. HOT 3
- OSError: [E053] Could not read meta.json from en\meta.json HOT 3
- Implementation issue HOT 6
- UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2490: ordinal not in range(128) HOT 2
- Parallel_to_m2 is not working HOT 1
- Licensing concerns HOT 6
- Errant parse method not working HOT 5
- Wrong format for incorr_sentences.txt HOT 4
- ‘’AttributeError: 'English' object has no attribute 'tagger'” when running the "Quick Start" code in API given in README.md HOT 4
- Ignore temporary files generated by installation HOT 1
- cancelling
- Edit indices HOT 3
- Simulate Errors HOT 1
- API Quickstart script not working - Please update with fix provided HOT 2
- Is there any way to further improve the method of summarizing error types? HOT 1
- Questions about evaluating duplicate corrections HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from errant.