Questions about the results about demfi HOT 7 OPEN

@VongolaWu
Thank you for your interest.

1. 'prime' means the performances for the architecture of DeMFI-Net_bs(first stage)'s outputs, w/o 'prime' means the performances for those of DeMFI-Net_rb(second stage)'s.

2. 'Time' represents an inference time (runtime) for one image.

3. Performance results are shown based on the total average for each time index (1~7: intermediate time index for joint deblurring and interpolation [MFI x8], 8: time index for only deblurring).

from demfi.

I still feel a little confused. Could you further explain it:

The part below "------x8 MFI results----" should be the final result, right? Using the above image as an example, I have already split it as 5 parts: "Test after Epoch", "x8 MFI results[PSNR Stage 1]", "x8 MFI results[SSIM Stage 1]", "x8 MFI results[PSNR Stage 2]", "x8 MFI results[SSIM Stage 2]"

1. What is the relationship between "Test after Epoch" and the other 4 parts? Is the "Test after Epoch" whole report for all the test images? But I'm still confused about the (avg:) after each item. For example, "intp_testPSNR: 2.6699e+01 (avg:2.3329e+01)", what does the avg mean?
2. Also I still feel confused about the (avg) in "x8 MFI results[PSNR Stage 1]", "x8 MFI results[SSIM Stage 1]", "x8 MFI results[PSNR Stage 2]", "x8 MFI results[SSIM Stage 2]".
3. Does "Time" mean that generating 8 sharp images for 1 blur image needs 1.136 seconds?

from demfi.

@VongolaWu

1. Yes, first block is average of all results for all time indices. "avg:~" means that it shows literally average on that time index (in your example, "avg:2.3329e+01" is a final performance for intp_testPSNR).
2. The final DeMFI-Net_{rb} (Stage II) performance is based on "avg" values of "x8 MFI results[PSNR Stage 2]" and "x8 MFI results[SSIM Stage 2]" for each time index (as in Fig. 1). Therefore, the "Average" performance in Table is calculated by averaging all "avg" values for all time indices of "x8 MFI results[PSNR Stage 2]", "x8 MFI results[SSIM Stage 2]" for each metric.
3. No, "Time" means an average runtime for running DeMFI-Net once, i.e. synthesizing each pair of three frames (two deblurred frame - S0, S1 and one interpolated image - St)

from demfi.

Thank you for your explanation.
Let me ask a further question:

I make a table for this result.

Therefore, for the first table, the intp_testPSNR(avg) should be the final results for 7 interpolated frames? and deblur_testPSNR(avg) should the final results for the 1 deblurred frames?
But they are not consistent with the second table.
So I would like to ask if I would like to represent the average PSNR for the whole test dataset containing both interpolated frames and deblurred frames, which number should I use?

from demfi.

@VongolaWu
First table is for an average of "whole" samples, regardless for each scene's numbers of frames.
Second table shows an average of "average for each scene", as done in BIN paper's evaluation. It considers that the numbers of frames are different for each scene. Therefore, it is recommended to use the second table format to be robust to imbalance of numbers of frames per each scene. (in your case, average 30.8135 dB to evaluate joint performance)

from demfi.

@JihyongOh
Thank you for your reply. One more thing:
For PSNR_1 to PSNR_7, are they the generated frame at timestamp t*1/8 (eg. PSNR_1 is at time 1/8)?
And the PSNR_blur is the sharp frame of B0?

from demfi.

@VongolaWu
Yes, your understanding for PSNR_1 (1/8) to PSNR_7 (7/8) is right (multiple of 1/8), and PSNR_blur is measured for the sharp frame of S1 and S1 without overlapped evaluation by considering a temporal window. Please refer these two lines; line1 and line2.

from demfi.