xtra-computing / niid-bench Goto Github PK

因为Fednova比Fedavg低了四个千，fedavg比fedprox低了四个千，SCAFFOLD远高于其他方法，但是图上，fednova是比较低的，而另外三种方法的线很接近，似乎和数据不符

Hello,

There is a missing comma in line

Hi! I've read your paper 'Federated Learning on Non-IID Data Silos: An Experimental Study' and 'Model-Contrastive Federated Learning'. I think both works are very interesting!

However, I notice that the performance of Scaffold is unstable and even extremely poor (e.g. nan) in some cases of your implementation. I've also found that Scaffold is reliably stable and outperforms many baselines across different settings of data heterogeneity and datasets in other works like FedDyn (ICLR 2021), FedDC (CVPR2022). Therefore, I carefully read the source code of yours and found the controlling variable is not initialized as described in 'SCAFFOLD: Stochastic Controlled Averaging for Federated Learning' in Sec.4. Since a random initialization of controlling variable is meaningless and can be misleading when locally training the model (especially at the early stage of training), I wonder whether this is the reason accounting for the unstablility and poor performance of Scaffold.

你好，非常感谢你提供的代码。我有个问题想请问一下，使用noniid-#label2划分数据时，每次程序运行得到的结果都不一样，无法重复上一次的实验数据划分。希望能解答一下，万分感谢

Hello， thank you for your nice code.
Maybe here shoulbe be "default='mlp'" instead of "default='MLP'"

Thanks for your contribution, it is useful to me
When I change the mlp model to a bigger one , I find that scaffold's loss is 'nan' after several rounds, do you have any idea for this issue?

When setting partition='homo', for the same label, different clients have different numbers, even tens of differences .
The picture below is the heat map for the mnist dataset divided evenly among 10 clients using partition method 'homo'.

The test_ds_local is loaed by test_ds = dl_obj(datadir+'./val/', transform=transform_test).
Therefore, test_ds is the same for all clients.

femnist datatset seems to have only 10 classes?

Thanks for your work. I think there is probably a small mistake in line 673 of experiments.py. According to the pseudocode of SCAFFOLD, the denominator here should be the number of all clients instead of the number of selected clients per round. Please correct me if I 'm wrong. Thank you!

Hi, when I run the following command
python experiments.py --model=simple-cnn --dataset=cifar10 --alg=all_in --lr=0.01 --batch-size=64 --epochs=10 --n_parties=10 --rho=0.9 --comm_round=50 --partition=noniid-labeldir --beta=0.5 --device=cuda:0 --datadir=./data/ --logdir=./logs/ --noise=0 --init_seed=0
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

Maybe this line of code should be preceded by "nets[0].to(device)"

According to the readme file, we notice that there are different types of noise, and we can select 'noise_type' as level so that gaussian noise will be added to all the pixels of images. However, when we read the class AddGaussianNoise, we found that the list named 'filt' will select part of the pixels to add gaussian noise, which seems odd to us. Thus, we are here to ask the question of whether we should add a conditional judgment before implementing the 'filt' according to the 'noise_type'. (e.g. when noise_type == 'level', we do not implement the 'filt').

HI，I want to know how i find "-load_path" ?

Thanks

This parameter is not involved in the pseudocode.
When t = 0, global model $\omega^0$ is sent to all the selected clients.

Hi, when I tried to run experiments.py in default setting, there is a problem showed up

is there any way to fix it?

Hi, your implementation is good. The FedAvg algorithm in the original paper https://arxiv.org/pdf/2007.07481.pdf includes the learning rate $\eta$ both in computing normalized gradients and global updates, but this term doesn't seem to appear in your code. I‘m wondering if there are things I missed or the learning rate should be added here.

Hi, i want to know lodfir is what?

Number of local samples $|D^i|$ =len(train_dl_local.dateset) instead of len(train_dl_local).
Although the batch size is the same for all dataloader，there will be cases where the number of data cannot be divided by the batch size.
I think this will be a little different from the pseudo-code