compute fisher matrix after train? about ewc HOT 2 OPEN

Yes, I am just computing the Fisher information for the trained neural network using the previous task's data only.

Unfortunately, TensorFlow does not expose the unaggregated gradients, which are required to compute the Fisher information. The workaround I chose was to hardcode the unaggregated gradients directly into the computation graph, using mini batches of fixed size 100 to allow parallelization (computing full-batch directly runs out of memory). It is then a simple matter to obtain the full-batch Fisher information by accumulating 550 mini-batches (see update_fisher_full_batch method). The simpler solution (which prevents parallelization) is to loop over the training data (55000 examples) using minibatches of size 1.

from ewc.

what is the difference with calculating a mean of fisher information on aggregated gradients?
The code is summing up and divide by the number of inputs, which seems the value of mean.

unaggregating the gradients make code more complicated, isn't it? why don't you use just mean of aggregated gradients?

from ewc.