Git Product home page Git Product logo

ffci's Introduction

FFCI: A framework for interpretable automatic evaluation of summarization

About

FFCI is a framework for fine-grained summarization evaluation that comprises four elements: Faithfulness (degree of factual consistency with the source), Focus (precision of summary content relative to the reference), Coverage (recall of summary content relative to the reference), and Inter-sentential coherence (document fluency between adjacent sentences).

Paper

Fajri Koto, Timothy Baldwin, and Jey Han Lau. FFCI: A Framework for Interpretable Automatic Evaluation of Summarization. Under review for the Journal of Artificial Intelligence Research (JAIR 2021).

Data

You can find the data used in our experiments in folder Data.

ffci's People

Contributors

fajri91 avatar

Stargazers

avatar avatar

Watchers

avatar avatar avatar

ffci's Issues

Cnndm data ids

Hi,
First, I want to thank you for publishing your data!
How can I get the original data of cnndm ( your ids doesn't correspond to those of the original cnndm dataset)
Thanks!

Code and models

Hi,

the paper states "Data and code used in this paper can be accessed at https://github.com/fajri91/ffci" but as far as I see only the data is available atm. Are you planning to release the code including the trained metric (including trained model)? I would be interested in using your metric.

kind regards.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.