Should the global model replace the client model? about fedprox HOT 2 CLOSED

Q1. w^t is the global model we would like to learn (and output). In theory, for fedprox, each device doesn't necessarily need to start solving h_k from w^t. All we need is w_k^{t+1} is a \gamma_k^t-inexact minimizer of h_k. FedProx allows to use any local algorithm to solve h_k. In practice (the code), we let each device start from the current global model w^t and run multiple local iterations of mini-batch SGD (this is one way [and a natural way] of obtaining an inexact solution), which will recover the algorithm of FedAvg when \mu=0, and will recover distributed SGD when the number of local iterations is 1.

Q2. I am not sure what catastrophic forgetting means in federated settings. Usually, people consider it in continual learning where they learn a sequence of tasks. While in federated learning, we simultaneously learn from multiple devices/tasks to produce a single global model or personalized models. But I think (variants) the proximal term could be used in other problems as well.

from fedprox.

Thank you for your detailed reply, I have gained a lot from it. I look forward to what you do next.

from fedprox.