Git Product home page Git Product logo

fedala's Introduction

Biography

Only do what interests you in your life. –––Jianqing Zhang

I am Jianqing (Tsing) Zhang, a first-year PhD student in Computer Science at Shanghai Jiao Tong University under the supervision of Jian Cao. I completed my Master's degree at the same institution and was fortunate to work with Ruhui Ma and Tao Song in 2023. I earned my Bachelor's degree at Hangzhou Dianzi University and was fortunate to work with Dongjin Yu and Dongjing Wang in 2020. My research interests include Federated Learning, Transfer Learning, Edge AI, and Recommender Systems. Both my PhD advisor and I believe that collaboration, which is also one of the core ideas of Federated Learning, is essential for scientific research. So far, I have worked with Yang Liu from Tsinghua University, Yang Hua from Queen's University Belfast, and Hao Wang from Louisiana State University. I am sincerely grateful for their invaluable knowledge and support. Additionally, I am also a photographer with a passion for capturing the beauty of the world.

Internships

  • ByteDance | Machine Learning Platform - Security and Trust | Contribute to the open-source project fedlearner

Featured Projects and Publications (refer to the slides for a detailed introduction)

Stage Ⅱ: heterogeneous federated learning

  • 🎉[HtFLlib] Data-Free Federated Learning Algorithm Library in Data and Model Heterogeneous Scenarios. [code]
  • 🎉[CVPR'24] An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learning. >>>Jianqing Zhang, Yang Liu, Yang Hua, Jian Cao<<< [paper] [code]
  • 🎉[AAAI'24] FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning. >>>Jianqing Zhang, Yang Liu, Yang Hua, Jian Cao<<< [paper] [code]

Stage Ⅰ: personalized federated learning

  • 🎉[PFLlib] Personalized Federated Learning Algorithm Library. [paper] [code] >>>Jianqing Zhang, Yang Liu, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, Jian Cao<<<
  • 🎉[NeurIPS'23] Eliminating Domain Bias for Federated Learning in Representation Space. >>>Jianqing Zhang, Yang Hua, Jian Cao, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, Haibing Guan<<< [paper] [code]
  • 🎉[ICCV'23] GPFL: Simultaneously Learning Generic and Personalized Feature Information for Personalized Federated Learning. >>>Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, Jian Cao, Haibing Guan<<< [paper] [code]
  • 🎉[KDD'23] FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy. >>>Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, Haibing Guan<<< [paper] [code]
  • 🎉[AAAI'23] FedALA: Adaptive Local Aggregation for Personalized Federated Learning. >>>Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, Haibing Guan<<< [paper] [code]

Other work

  • 🎉[IEEE Transactions on Cognitive and Developmental Systems] pFedEff: An Efficient and Personalized Federated Cognitive Learning Framework in Multi-agent Systems. >>>Hongjian Shi, Jianqing Zhang, Shuming Fan, Ruhui Ma, Haibing Guan<<< [paper]
  • 🎉[Neurocomputing] TLSAN: Time-aware long-and short-term attention network for next-item recommendation. >>>Jianqing Zhang, Dongjing Wang, Dongjin Yu<<< [paper] [code]

Photography

My photography gallery can be found at: https://tsing.tuchong.com/work/

  • 🎉[2022] International Photography Award (IPA) official selection
  • 🎉[2020] Top 30 in the Metro Yunchuang Photo Contest

fedala's People

Contributors

tsingz0 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

fedala's Issues

Reproducing Table 2.

Hello,

First of all thank you for providing this work.

I am looking to replicate Table 2 from the publication. However, for many of these methods, there are many hyper-parameters to tune and it is difficult to tell if the drop in performance is due to the randomness of the dataset partition or due to poor hyperparameter selection.

Would it be possible for you to share either:

  1. The full set of hyperparameters used to train models used in table 2
  2. Trained model checkpoints, with the corresponding dataset splits (e.g. just as you have done here for MNIST, but also for CIFAR10/CIFAR100)

Thank you for your time.

公式推导问题

您好,拜读了您的论文,十分仰慕您做的工作,我想请问一下您在,论文3.3部分的关于Analysis of ALA的内容,是如何实现公式推导的?
也就是对于由公式(4)(5)得到(6)的过程,希望请您赐教。

关于论文和代码不符问题

image
这里这样取模型参数不是会把bias也可能取到吗?通常最后一层的最后一个都是bias,那只学习bias而不是w的话也不符合论文的**吧?望解答,谢谢

数据路径的相关问题

您好,我阅读你的代码然后经过单步调试对一个地方感到疑惑。就是在data_utils中的read_data函数中的一 行:train_data_dir = os.path.join('../dataset', dataset, 'train/')
调试过程中会报出../dataset/mnist……路径下的文件不存在,于是我尝试把“../”给删除就可以成功调试下去。我也觉得存在../的路径是合理的,但为什么会出现这种情况。

请教关于Cifar100数据集,ρ=0.5的对比实验问题

您好,我注意到在论文的extended version:https://arxiv.org/pdf/2212.01197v3.pdf 中有关于Cifar100数据集在节点参与率ρ=0.5下的对比实验(Table 7,结果如下图):
1694594085795
看起来它应该是Table 4的对比实验,除了ρ=0.5与ρ=1的设定不同外其他条件应该是一致的,但Table 4的结果如下图:
1694594067387
我发现在这两个对比实验中,在ρ=0.5的条件下FedAvg的准确度反而比ρ=1的要高出不少(39.51比31.95),这似乎不太符合直觉(节点参与率越高,模型准确度应该更高才对)。我想请问一下可能导致这个现象的原因是什么?是因为计算准确度的方式选择的是训练过程中的最优准确度吗?希望能得到您的解答,谢谢!

关于抽取本地数据训练权重的问题

您好,多次拜读了您FedALA的论文,请教您一个问题,在您的论文中,您在公式(5)中从客户端本地的数据训练自适应权重Wpi,并且在下面的内容中提到为了减少计算,可以随机抽取80%的本地数据训练这个权重。

但是在后面,看到您在每一个客户端上评价训练的本地个性化模型,将75%的数据作为训练集,25%的数据作为测试集。
所以请问,在抽取80%的本地数据训练权重Wpi的时候,应该是只抽取的本地的训练集的80%的数据吧?

超参数解释问题

论文中的p和s是什么超参数呀?我只在代码中看到 cid: int,
loss: nn.Module,
train_data: List[Tuple],
batch_size: int,
rand_percent: int,
layer_idx: int = 0,
eta: float = 1.0,
device: str = 'cpu',
threshold: float = 0.01,
num_pre_loss: int = 10) -> None: 这些参数

Hyperparameter grid search

I see the hyperparameters are available in the paper for the default practical setting but is the code for the grid search available if we want to apply hyperparameter tuning to other methods or different data distributions?

关于测试准确率结果的问题

您好!
很荣幸能读到您的论文,有一个问题想请教您,请问您的实验结果是取最后一轮的测试准确率还是运行期间最高的一轮测试准确率呢?

关于AG_News数据的生成问题

您好,在您的PFL-Non-IID的项目代码中,运行python generate_agnews.py noniid - dir生成数据,出现以下报错,希望您指点一下问题:

Traceback (most recent call last):
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\urllib3\connection.py", line 174, in _new_conn
conn = connection.create_connection(
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\urllib3\util\connection.py", line 95, in create_connection
raise err
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\urllib3\util\connection.py", line 85, in create_connection
sock.connect(sa)
TimeoutError: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应,连接尝试失败。

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\urllib3\connectionpool.py", line 714, in urlopen
httplib_response = self._make_request(
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\urllib3\connectionpool.py", line 403, in _make_request
self._validate_conn(conn)
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\urllib3\connectionpool.py", line 1053, in _validate_conn
conn.connect()
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\urllib3\connection.py", line 363, in connect
self.sock = conn = self._new_conn()
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\urllib3\connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x0000028978125D60>: Failed to establish a new connection: [WinError 10060] 由于连接方在一段时间
后没有正确答复或连接的主机没有反应,连接尝试失败。

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\requests\adapters.py", line 487, in send
resp = conn.urlopen(
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\urllib3\connectionpool.py", line 798, in urlopen
retries = retries.increment(
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\urllib3\util\retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='drive.google.com', port=443): Max retries exceeded with url: /uc?export=download&id=0Bz8a_Dbh9QhbUDNpeUdjb0wxRms (Caused b
y NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000028978125D60>: Failed to establish a new connection: [WinError 10060] 由于连接方在一段时间后没有正确答复或连
接的主机没有反应,连接尝试失败。'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "L:\Model\PFL-Non-IID-master -test\dataset\generate_agnews.py", line 92, in
generate_agnews(dir_path, num_clients, num_classes, niid, balance, partition)
File "L:\Model\PFL-Non-IID-master -test\dataset\generate_agnews.py", line 33, in generate_agnews
trainset, testset = torchtext.datasets.AG_NEWS(root=dir_path+"rawdata")
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\torchtext\datasets\text_classification.py", line 170, in AG_NEWS
return _setup_datasets(*(("AG_NEWS",) + args), **kwargs)
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\torchtext\datasets\text_classification.py", line 117, in _setup_datasets
dataset_tar = download_from_url(URLS[dataset_name], root=root)
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\torchtext\utils.py", line 96, in download_from_url
response = session.get(url, stream=True)
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\requests\sessions.py", line 600, in get
return self.request("GET", url, **kwargs)
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\requests\sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\requests\sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "C:\Users\like.conda\envs\pytorch\lib\site-packages\requests\adapters.py", line 520, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='drive.google.com', port=443): Max retries exceeded with url: /uc?export=download&id=0Bz8a_Dbh9QhbUDNpeUdjb0wxRms (Cause
d by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000028978125D60>: Failed to establish a new connection: [WinError 10060] 由于连接方在一段时间后没有正确答复
或连接的主机没有反应,连接尝试失败。'))

公式6的推导

你好,看了您的论文以及附录之后,还是不太懂6是如何推导的,附录的7可以看懂。请问论文中公式6前的一句话we can view updating Wi as updating ˆΘt i如何理解?公式6的梯度的系数为什么是两个(Θt-1 - Θt-1i)?

关于Applicability of ALA,ALA与Ditto结合的问题

您好,请教一些论文中关于4.4 Applicability of ALA部分的问题:

In Table 4, the accuracy improvement for FedAvg and FedProx is apparent, and the improvement for Per-FedAvg, Ditto, and FedPHP is also remarkable. This indicates the applicability of ALA to the traditional FL and pFL methods.

请问您是如何将ALA方法和Ditto联合使用的呢?有些不太理解,期待您的回答!

我想问一下,关于W的更新方式

你好,我想问一下,我理解的是:老哥你在论文中通过求Loss对tp_model的梯度,接着再计算W的梯度,之后采用梯度下降法更新W。
能不能直接在优化器的参数中只加入W,然后反向传播直接求Loss对W的梯度呢
谢谢你,祝你学业顺利,生活愉快!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.