During evaluation metric_fn is called epoch_size/batch_size times with the same data about seq2seq HOT 2 CLOSED

This is actually intended and I don't think there is an easy way around it. The metric function is called with different data at each time step, but it accumulates all strings so far into a variable (see TextMetricSpec class), and then calls the actual metric calculation with "all-strings-so-far", so that it can run scripts like multi-bleu.perl on it. The reported metric is always the last on, which is the one that includes all the data and should be correct.

The data accumulation is needed because scripts like multi-bleu.perl need the whole data and can't be be used for batch-by-batch calculation.

I also can't just call the metric function once, because that's not how Tensorflow metrics works. They are supposed to be calculated in a "streaming" fashion and calculate results batch by batch.

Does that make sense?

from seq2seq.

Closing for now.

from seq2seq.