pguijas / p2pfl Goto Github PK

P2P Federated Learning (p2pfl) is a decentralized federated learning library, it allows creating basic federated learning systems on p2p networks using gossip protocols.

Home Page: https://p2pfl.com

License: GNU General Public License v3.0

Python 99.74% Dockerfile 0.26%

decentralized deep-learning federated-learning machine-learning decentralized-federated-learning gossip-protocol open-source p2p-network python pytorch

p2pfl's Introduction

P2PFL - Federated Learning over P2P networks

P2PFL is a general-purpose open-source library for the execution (simulated and in real environments) of Decentralized Federated Learning systems, specifically making use of P2P networks and the Gossip protocol.

✨ Key Features

P2PFL offers a range of features designed to make decentralized federated learning accessible and efficient. For detailed information, please refer to our documentation.

Feature	Description
🚀 Easy to Use	Get started quickly with our intuitive API.
🛡️ Reliable	Built for fault tolerance and resilience.
🌐 Scalable	Leverages the power of peer-to-peer networks.
🧪 Versatile	Experiment in simulated or real-world environments.
🔒 Private	Prioritizes data privacy with decentralized architecture.
🧩 Flexible	Integrate with PyTorch and TensorFlow (coming soon!).
📈 Real-time Monitoring	Manage and track experiment through P2PFL Web Services.
🧠 Model Agnostic	Use any machine learning model you prefer (e.g., PyTorch models).
📡 Communication Protocol Agnostic	Choose the communication protocol that best suits your needs (e.g., gRPC).

📥 Installation

Note: We recommend using Python 3.9 or lower. We have found some compatibility issues with Python 3.10 and PyTorch.

👨🏼‍💻 For Users

pip install "p2pfl[torch]"

👨🏼‍🔧 For Developers

🐍 Python (using Poetry)

git clone https://github.com/pguijas/p2pfl.git
cd p2pfl
poetry install -E torch

Note: Use the extras (-E) flag to install specific dependencies (e.g., -E torch). Use --no-dev to exclude development dependencies.

🐳 Docker

docker build -t p2pfl .
docker run -it --rm p2pfl bash

🎬 Quickstart

To start using P2PFL, follow our quickstart guide in the documentation.

📚 Documentation & Resources

Documentation: https://pguijas.github.io/p2pfl/
End-of-Degree Project Report: other/memoria.pdf
Open Source Project Award Report: other/memoria-open-source.pdf

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines. Please adhere to the project's code of conduct in CODE_OF_CONDUCT.md.

💬 Community

Connect with us and stay updated:

GitHub Discussions: - For general discussions, questions, and ideas.
GitHub Issues: - For reporting bugs and requesting features.
Google Group: - For discussions and announcements.
Slack: - For real-time conversations and support.

⭐ Star History

A big thank you to the community for your interest in P2PFL! We appreciate your support and contributions.

📜 License

GNU General Public License, Version 3.0

p2pfl's People

Contributors

Stargazers

Watchers

Forkers

masterthesis-e22 joonghocho rmauri25 lifelikemaik saadiabadi ayush0016 dhanya8689 enriquetomasmb lushoupeng angel3245 chichaman33 kirle mmroshani

p2pfl's Issues

[FUTURE] New Federated Learning Aggregation Algorithms

💡 The feature, motivation and pitch

It would great to have a repertoire of aggregation algorithms

Alternatives

No response

Additional context

No response

[BUG] Votes not associated with rounds

Is there an existing issue for this?

I have searched the existing issues

Current Behavior

Votes not associated with rounds

Expected Behavior

Associate votes to rounds, so that if delays occur and votes arrive when rounds have already been executed, discrepancies do not occur.

Steps To Reproduce

No response

Anything else?

No response

In which system are you seeing the problem on?

No response

[FUTURE] Proposal to introduce a developing branch for ongoing contributions

Hi! I was looking to the project and I see that there's no development branch, which could help maintain stability as changes and features are added. Would you consider creating a 'develop' branch for ongoing development?

Additional context

I'm taking https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow as reference.

something wrong with Trainer log

hi, I met some problems in logger after modifying based on your example, specifically using pytorch lightning Trainer

| 127.0.0.1:64607 | ERROR ]: Evaluation error. Something went wrong with pytorch lightning. You are trying to self.log() but it is not managed by the Trainer control 
| ERROR ]: Error: You are trying to self.log() but it is not managed by the Trainer control flow
| INFO ]: Stopping node...
| INFO ]: Stopping gossiper...
| INFO ]: Cannot send message beat to 127.0.0.1:64609. Error: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.DEADLINE_EXCEEDED
	details = "Deadline Exceeded"
	debug_error_string = "UNKNOWN:Deadline Exceeded"

How can I fix it？

[FUTURE] Iterative terminal for node management

💡 The feature, motivation and pitch

It would be great to manage a node through an iterative terminal.

Alternatives

No response

Additional context

No response

[FUTURE] Node connection during learning.

💡 The feature, motivation and pitch

Node connection during learning.

Alternatives

No response

Additional context

No response

[BUG] Tolerance to incomplete message reception

Is there an existing issue for this?

I have searched the existing issues

Current Behavior

No response

Expected Behavior

Although highly improbable, it could happen that fragments of (incomplete) messages are read and produce errors.

Steps To Reproduce

No response

Anything else?

No response

In which system are you seeing the problem on?

No response

[FUTURE] GRCP

💡 The feature, motivation and pitch

communication protocol (pure TCP) to GRCP (https://grpc.io/) migration

consider whether it would improve robustness, performance and clarity in code

Alternatives

No response

Additional context

No response

[FUTURE] Optional Dependencies

💡 The feature, motivation and pitch

https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies

Alternatives

No response

Additional context

No response

[FUTURE] Node authentication

💡 The feature, motivation and pitch

Node authentication to control node access to the network.

Alternatives

No response

Additional context

No response

Import error while running examples

Hello, when I run the test "minst.py" , I get this error："Model Reception in a late round .". But I didn't find what went wrong in the code, do you have a good solution for this?

[FUTURE] TensorFlow support

💡 The feature, motivation and pitch

TensorFlow support

Alternatives

No response

Additional context

No response

[FUTURE] Secure Agregation

💡 The feature, motivation and pitch

Include Secure Agregation for an additional privacy layer

Alternatives

No response

Additional context

No response

[FUTURE] Simulation Optimization

💡 The feature, motivation and pitch

Create a new type of connections that avoids the use of TCP for local simulations.

The single instance pattern should be used and declare this operation as a new connection type.

Alternatives

No response

Additional context

No response

Be careful with the port number

At this moment, in p2pfl/base_node.py and p2pfl/node.py, they have as default port 0. Instead, you should put something not reserve by the system and which is not on the well-known ports.

For example you can iniciate it to None and made this check before open any one.

def __init__(self, host="127.0.0.1", port=None, simulation=True):
     port = port if port is not None else choose_random_port()

Import error while running examples

I have not found a way to run the examples inside the examples/ folder, as the package is installed in the p2pfl folder inside the root directory, and therefore when trying to run, for example, mnist.py, you get an import error ModuleNotFoundError: No module named 'test.utils.

Perhaps it would be good to add to the documentation more clearly how to run the examples. The easiest way I have found without changing the repository organization is to use python -m examples.mnist.py

[FUTURE] Asyncio to improve performance in GRPC

💡 The feature, motivation and pitch

This issue proposes integrating Asyncio into gRPC Python to enhance performance and scalability. Leveraging Asyncio's non-blocking I/O can significantly improve responsiveness and resource utilization, especially under high concurrency.

This would enable gRPC Python to handle more requests with lower latency, making it a better fit for demanding applications.

Alternatives

No response

Additional context

No response

pguijas / p2pfl Goto Github PK

p2pfl's Introduction

P2PFL - Federated Learning over P2P networks

✨ Key Features

📥 Installation

👨🏼‍💻 For Users

👨🏼‍🔧 For Developers

🐍 Python (using Poetry)

🐳 Docker

🎬 Quickstart

📚 Documentation & Resources

🤝 Contributing

💬 Community

⭐ Star History

📜 License

p2pfl's People

Contributors

Stargazers

Watchers

Forkers

p2pfl's Issues

💡 The feature, motivation and pitch

Alternatives

Additional context

Is there an existing issue for this?

Current Behavior

Expected Behavior

Steps To Reproduce

Anything else?

In which system are you seeing the problem on?

Additional context

💡 The feature, motivation and pitch

Alternatives

Additional context

💡 The feature, motivation and pitch

Alternatives

Additional context

Is there an existing issue for this?

Current Behavior

Expected Behavior

Steps To Reproduce

Anything else?

In which system are you seeing the problem on?

💡 The feature, motivation and pitch

Alternatives

Additional context

💡 The feature, motivation and pitch

Alternatives

Additional context

💡 The feature, motivation and pitch

Alternatives

Additional context

💡 The feature, motivation and pitch

Alternatives

Additional context

💡 The feature, motivation and pitch

Alternatives

Additional context

💡 The feature, motivation and pitch

Alternatives

Additional context

💡 The feature, motivation and pitch

Alternatives

Additional context

Recommend Projects

Recommend Topics

Recommend Org