res2net / res2net-poolnet Goto Github PK

I have skimmed through the papers however didn't find the detailed explanation on accumulate gradients. Please help me understand. Generally simplified flow is like

predicted_output = model(input)
loss = loss_function(predicted_output, ground_truth)
optimizer.zero_grad()
loss.backward()
optimizer.step()

However in code, gradients are accumulated for 10 iterations and then reset. I am wondering what +ve or -ve impacts it will have if I

1: reset on each iteration means along the lines of above general algorithm flow
2: increase/decrease the self.iter_size
3: add support for multi-batching and multi-gpu

Many thanks.

Hi

Thank you for your work. I came across this and training. Initial values of the training are as

The number of parameters: 70452145
epoch: [ 0/48], iter: [ 0/65231] || Sal : 7288.5986
Learning rate: 5e-05
epoch: [ 0/48], iter: [ 50/65231] || Sal : 669608.8125
Learning rate: 5e-05

How do we know that model has learned well? does Sal value have to be near zero or when flatten on any number?

Hi,

Could you please share your results of Res2Net-PoolNet using joint training with edge information on DUT-TR? Many thanks in advance!

The inference speed of "res2net_poolnet_final.pth" is a little bit slow.