lx10077 / fedavgpy Goto Github PK
View Code? Open in Web Editor NEWOn the Convergence of FedAvg on Non-IID Data
License: MIT License
On the Convergence of FedAvg on Non-IID Data
License: MIT License
I think there is a missing indentation in line 173 and 220.
Hello!
I think it maybe a little problem around understanding overall execution process.
I follow instructions to prepare and run data on README file.
I am using the parameters:
export DATASET=mnist
export SEED=10
export K=5
export B=4
export NET=2nn
export T=5
export E=5
and receiving this result:
python3 main.py --dataset $DATASET --clients_per_round $K --num_round $T --num_epoch $E --batch_size $B --seed $SEED --model $NET --noaverage --noprint
>>> Arguments:
algo : fedavg4
batch_size : 4
clients_per_round : 5
dataset : mnist
device : 0
dis :
eval_every : 5
gpu : False
input_shape : 784
lr : 0.1
model : 2nn
noaverage : True
noprint : True
num_class : 10
num_epoch : 5
num_round : 5
seed : 10
wd : 0.001
>>> Read data from:
./data/mnist/data/train/all_data_0_equal_niid.pkl
./data/mnist/data/train/all_data_1_random_niid.pkl
./data/mnist/data/test/all_data_0_equal_niid.pkl
./data/mnist/data/test/all_data_1_random_niid.pkl
>>> Don not use gpu
>>> Model statistic per layer
TwoHiddenLayerFc(
198.8 KMac, 100.000% MACs,
(fc1): Linear(156.8 KMac, 78.873% MACs, in_features=784, out_features=200, bias=True)
(fc2): Linear(40.0 KMac, 20.121% MACs, in_features=200, out_features=200, bias=True)
(fc3): Linear(2.0 KMac, 1.006% MACs, in_features=200, out_features=10, bias=True)
)
>>> Activate a worker for training
>>> Initialize 100 clients in total
>>> Weigh updates by sample numbers
>>> Select 5 clients per round
Traceback (most recent call last):
File "/home/user/fedavgpy/main.py", line 131, in <module>
main()
File "/home/user/fedavgpy/main.py", line 127, in main
trainer.train()
File "/home/user/fedavgpy/src/trainers/fedavg4.py", line 35, in train
self.test_latest_model_on_traindata(round_i)
File "/home/user/fedavgpy/src/trainers/base.py", line 155, in test_latest_model_on_traindata
stats_from_train_data = self.local_test(use_eval_data=False)
File "/home/user/fedavgpy/src/trainers/base.py", line 211, in local_test
tot_correct, num_sample, loss = c.local_test(use_eval_data=use_eval_data)
File "/home/user/fedavgpy/src/models/client.py", line 89, in local_test
tot_correct, loss = self.worker.local_test(dataloader)
File "/home/user/fedavgpy/src/models/worker.py", line 131, in local_test
pred = self.model(x)
File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/user/fedavgpy/src/models/model.py", line 25, in forward
out = F.relu(self.fc1(x))
File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (112x28 and 784x200)
In paper "On the convergence of FedAvg on non-iid data", the learning rate eta = eta0/(t+1), where t means the current number of iterations (including local iterations and communication rounds).
Additionally, in your code in src/models/worker.py, I understand that for epoch in range(self.num_epoch):
where num_epoch refers to E in the paper.
However, in your introdunction in github,
python main.py --gpu --dataset $DATASET --clients_per_round $K --num_round $T --num_epoch $E --batch_size $B --lr $LR --device $device --seed $SEED --model $NET --algo $ALGO --noaverage --noprint
where num_round refers to T and it is the thing that I cannot understand.
We could see the function local_train
from your code in src/trainers/fedavg.py,
I think num_round in your code is not T. Instead, it refers to T/E. Therefore, I think there are some revisions to make.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.