Is your feature request related to a problem? Please describe. In

This can be viewed as a pre-requisite to issue <a class="issue-link js-issue-link" dat

Add general data transfer protocol between client and server [FR] about plato HOT 2 CLOSED

tl-system commented on July 26, 2024

Add general data transfer protocol between client and server [FR]

from plato.

Comments (2)

hcngac commented on July 26, 2024

This can be viewed as a pre-requisite to issue #119, for providing a unified data exchange format, such that different data processors can be plugged into the data-exchange, not only for reformatting, but also for other features such as local differential privacy and model/feature compression.

from plato.

baochunli commented on July 26, 2024

This can be quite easily achieved by converting framework-specific weights to numpy arrays in Algorithm.extract_weights() and Algorithm.load_weights():

plato/plato/algorithms/mindspore/fedavg.py

Lines 13 to 25 in c238f2d

 """MindSpore-based federated averaging algorithm, used by both the client and the server.""" 

 def extract_weights(self): 

 """Extract weights from the model.""" 

 # In MindSpore releases later than 1.1.3, including 1.5.0, do not support `pickle.load()` 

 # on `Tensor` objects (https://gitee.com/mindspore/mindspore/issues/I43RPP?from=project-issue). 

 # Therefore, Tensor objects must be converted to numpy arrays first 

 numpy_weights = OrderedDict() 

 for name, weight in self.model.parameters_dict().items(): 

 numpy_weights[name] = weight.asnumpy() 

 return numpy_weights

plato/plato/algorithms/mindspore/fedavg.py

Lines 32 to 41 in c238f2d

 def load_weights(self, weights): 

 """Load the model weights passed in as a parameter.""" 

 for name, weight in weights.items(): 

 weight_tensor = mindspore.Tensor(weight.astype(np.float32)) 

 weights[name] = mindspore.Parameter(weight_tensor, name=name) 

 # One can also use `self.model.load_parameter_slice(weights)', which 

 # seems to be equivalent to mindspore.load_param_into_net() in its effects 

 mindspore.load_param_into_net(self.model, weights, strict_load=True)

The only other location that requires a little bit of work to make sure fedavg works correctly is to implement the trainer.zeros() function to initialize a numpy-based zeros array. An example is shown in trainers/mindspore/basic.py:

plato/plato/trainers/mindspore/basic.py

Lines 60 to 64 in c238f2d

 def zeros(self, shape): 

 """Returns a zero numpy array with the given shape.""" 

 # This should only be called from a server 

 assert self.client_id == 0 

 return np.zeros(shape)

from plato.

Recommend Projects

Add general data transfer protocol between client and server [FR] about plato HOT 2 CLOSED

Comments (2)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	"""MindSpore-based federated averaging algorithm, used by both the client and the server."""
	def extract_weights(self):
	"""Extract weights from the model."""

	# In MindSpore releases later than 1.1.3, including 1.5.0, do not support `pickle.load()`
	# on `Tensor` objects (https://gitee.com/mindspore/mindspore/issues/I43RPP?from=project-issue).
	# Therefore, Tensor objects must be converted to numpy arrays first
	numpy_weights = OrderedDict()

	for name, weight in self.model.parameters_dict().items():
	numpy_weights[name] = weight.asnumpy()

	return numpy_weights

	def load_weights(self, weights):
	"""Load the model weights passed in as a parameter."""
	for name, weight in weights.items():
	weight_tensor = mindspore.Tensor(weight.astype(np.float32))
	weights[name] = mindspore.Parameter(weight_tensor, name=name)

	# One can also use `self.model.load_parameter_slice(weights)', which
	# seems to be equivalent to mindspore.load_param_into_net() in its effects

	mindspore.load_param_into_net(self.model, weights, strict_load=True)

	def zeros(self, shape):
	"""Returns a zero numpy array with the given shape."""
	# This should only be called from a server
	assert self.client_id == 0
	return np.zeros(shape)