yinan-zhao / vizwiz-caption Goto Github PK

Hi,

I cloned the repo and ran the notebook without modifying the code.
I got inconsistent results from the original notebook. Could you please confirm that the current results in the notebook are reproducible?

Below is my log from vizwizEval.evaluate()

tokenization...
setting up scorers...
computing Bleu score...
{'testlen': 86410, 'reflen': 83011, 'guess': [86410, 78868, 71335, 63802], 'correct': [31457, 12531, 8925, 7741]}
ratio: 1.0409463806001489
Bleu_1: 0.364
Bleu_2: 0.241
Bleu_3: 0.193
Bleu_4: 0.172
computing Rouge score...
ROUGE_L: 0.289
computing CIDEr score...
CIDEr: 0.318
computing SPICE score...
SPICE: 0.060
computing METEOR score...
METEOR: 0.120

My result screenshot:

Screenshot of current notebook file in this github:

My environment: Ubuntu 20.04 / python 3.8 / java 1.8.0

What does the is_precanned tag mean for a caption?

Hi,

When I was submitting on EvalAI, it gave me the following error:

And I think it is coming from the following code.

Seems it is directing comparing a JSON object with a list type, I think it should be

assert type(anns['results']) == list, 'results is not an array of objects'

As in the instruction, it expects the following format:

May you confirm? Thanks.

Hi,

Just want to let you know VizWiz_train_00022628.jpg seems to be corrupted, it's only 10x13px.