I think you should add argument is_training for resnet model. tf.layers.batch_normalization use training=False by default. It means that the resnet model created by your code is for Evaluation, not for Training.

• Code Ref:

• Suggestion:

return tf.layers.batch_normalization(data, momentum=bn_momentum, training=is_training, name=('%s__bn' % name))

I think u should do UpSampling2D of the lf_conv and then add the hf_conv and upsampled data to get the final output. See figure 2(a) of the paper when alpha_out = 0 only two paths are left in which one is Upsample2D.