The rouge's discuss from kenlimmj

Constructor Uint8Array requires 'new'

I tried running the following:

var rouge = require('rouge');
const ref = 'police killed the gunman';
const cand = 'police kill the gunman';
console.log(rouge.l(cand, ref, { beta: 0.5 }));

I get the following error:

node_modules/rouge/src/rouge.js:48
                cTable.push(Uint8Array(trimmedCandidate.length + 1));
                            ^

TypeError: Constructor Uint8Array requires 'new'

I installed through npm install rouge.

Can't find V2 in NPM

https://www.npmjs.com/package/rouge?activeTab=versions

Calling with same input

Call with equal candidate and reference will cause a deadlock.

var rouge = require('rouge');
input = "the same input sentence"
console.log(rouge.l(input, input, { beta: 0.5 }));

I've been working with ROUGE for a while and I'm still not sure how to implement ROUGE-L correctly.
Both your implementation and the one I'm using (in Python) implements the summary level ROUGE-LCS score as described in the paper. The thing is, the score isn't close to the official scores (i.e. using the perl script).

Example

Ref:

brendan @entity8 is under pressure following @entity11 semi-final defeat . but the @entity10 boss says he will bounce back despite the criticism . @entity10 owners @entity9 maintain @entity8 wo n't be sacked . @entity13 hopes @entity18 commits his future to the @entity23 .

Summary:

brendan @entity8 insists he is the man to guide @entity10 to success . brendan @entity8 has not been rattled by the intensity of the criticism . @entity10 manager is under pressure following the semi-final defeat by @entity12 last sunday .

Experiment

Official scores: In fact, I'm using python wrappers (files2rouge, that uses pyrouge). I tested those wrappers by scoring some prediction / reference pairs and finding the exact same numbers.

---------------------------------------------
1 ROUGE-1 Average_R: 0.43902 (95%-conf.int. 0.43902 - 0.43902)
1 ROUGE-1 Average_P: 0.47368 (95%-conf.int. 0.47368 - 0.47368)
1 ROUGE-1 Average_F: 0.45569 (95%-conf.int. 0.45569 - 0.45569)
---------------------------------------------
1 ROUGE-2 Average_R: 0.20000 (95%-conf.int. 0.20000 - 0.20000)
1 ROUGE-2 Average_P: 0.21622 (95%-conf.int. 0.21622 - 0.21622)
1 ROUGE-2 Average_F: 0.20779 (95%-conf.int. 0.20779 - 0.20779)
---------------------------------------------
1 ROUGE-L Average_R: 0.41463 (95%-conf.int. 0.41463 - 0.41463)
1 ROUGE-L Average_P: 0.44737 (95%-conf.int. 0.44737 - 0.44737)
1 ROUGE-L Average_F: 0.43038 (95%-conf.int. 0.43038 - 0.43038)

Other implementations:
- rouge (python)

{
  "rouge-1": {
    "f": 0.43076922582721894,
    "p": 0.4827586206896552,
    "r": 0.3888888888888889
  },
  "rouge-2": {
    "f": 0.19999999501250013,
    "p": 0.21052631578947367,
    "r": 0.19047619047619047
  },
  "rouge-l": {
    "f": 0.048830315339539125,
    "p": 0.0507936507936508,
    "r": 0.047249907715024
  }
}

rouge (this repo) (JS)

> rougeL(hyp, ref);
0.07972913936216687

The difference between R1, R2 scores does not really bother me. But it seems like we're not using the right LCS.

kenlimmj / rouge Goto Github PK

rouge's Issues

Constructor Uint8Array requires 'new'

Can't find V2 in NPM

Calling with same input

Multi-sentence ROUGE-L scores

Example

Experiment

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent