Bug in Local platform:
The participant list in the aggregator is duplicated since it is updated everytime we call the receive and it contains a Notification.participant_joined but also when calling the method get_participants. I have created a task with 2 users (Marcos and Jaime) and this is the print from the ML library after calling get_participants:
Participants: ['Marcos', 'Jaime', 'Marcos', 'Jaime']

This version of the communications library is missing the option of sending a message to a specific user from the aggregator. We need this in order to implement algorithms under the POM3.

Summary

Make sure you know which tasks the user is participating in and which tasks he is creator of.

Justification

So that you can filter and know which tasks a user is involved in.

Example

We've identified two solutions:

1. Make sure that within the task model you know which tasks the user is participating in and the ones he is creator of: by name/id or boolean (the model could also be a DTO)

2. Have a get_tasks_by_user method to return a lists of tasks for which the user is a participant or aggregator

E.g. modify the receive methods for aggregator/participants to return a 204 code in case the corresponding queues were empty. With this control can be given back to the aggregator/participant part of the algorithm and avoid blocking the execution.

Summary

It would be helpful to have an intermediate status between CREATED and STARTED.

Justification

When a new task is added to the available ones it gets the CREATED status. As soon as its owner decides
to start the aggregation on the background the master waits for the workers number to reach the task quorum
in order to start the federated learning. Nevertheless, during this time the task status remains the same,
changing to STARTED only when the actual task execution starts. In this way, there is no way for other users
to understand whether that task owner has started the aggregation or not, getting an error if they choose
to join a task which has just been created and not yet aggregated.

"Must have" functionality

The additional task status (something like "PENDING") would have multiple benefits:

• Aggregator receives a real feedback after the aggregation by checking the task status.
• All the "participant" calls on tasks which are not waiting for workers are avoided.
• Improved browsing on the tasks list.

Summary

Define error classes.

Justification

In order to capture and handle in a custom way the error that occur.

Example

fflapi.py:

class ServerError(ValueError):
pass

class BadRequestError(ValueError):
pass

…
if 'error' in result:
raise ServerError(result['error'])

if 'calls' not in result:
raise BadRequestError("Malformed object: " + str(result))

controller.py:

from pycloudmessenger.ffl.fflapi import ServerError
from pycloudmessenger.ffl.fflapi import BadRequestError

@app.errorhandler(ServerError)
def server_error(error):
return jsonify({'message': str(error)}), 500, {'ContentType': 'application/json'}

@app.errorhandler(BadRequestError)
def server_error(error):
return jsonify({'message': str(error)}), 400, {'ContentType': 'application/json'}

Bug in Local platform:
The receive method for both the aggregator and the participant still get blocked and never returns a timeout exception when nothing is received within the fixed time.

Summary

Add this information at task level:
- task description
- the name of the task owner
- the POM
- algorithm name
- the quorum
- a “rewards” field to define how the task is rewarded
- a “datasets” field to define the topology and dataset description to be used
- the number or list of participants who have joined the task

Justification

Add useful information for the user who wants to join a task. All of this information should be set by the task creator.

"Must have" functionality

All of these information are stored during the task creation and retrievable by any user (ffl.get_tasks() method); some information may only be retrievable by the task creator.

Example

A JSON model example (we can further discuss about some of this information):

{
"task_name": "Test",
"status": "CREATED",
"description": "Task test",
"added": "2020-01-20T12:00:00",
"topology": "STAR",
"participants": 4,
"owner": "owner_name",
"model_type": "Kmeans",
"POM": 1,
"quorum": 5,
"reward": "collaborative"
"datasets": [
{
"type": "image",
"description": "High-res images depicting people",
"features": {
"extension": "png",
"maxHeight": 16000000,
"maxWidth": 16000000
}
},
{
"type": "tabular",
"description": "Data about people",
"sample": [
"Lucas",
35
],
"features": {
"extension": "csv",
"size": 223000000,
"columns": [
{
"name": "Name",
"type": "string",
"description": "Person first name",
"isLabel": false
},
{
"name": "Age",
"type": "number",
"description": "Person age",
"isLabel": true
}
]
}
}
],
"definition": {
"NC": 2,
"maxIterations": 100
}
}

Summary

Add a "delete_task" method inside fflapi.py

Justification

A user can remove its own tasks if he wants.

"Must have" functionality

Only the user who created his own task can remove it.

Summary

Trying to send certain objects results in an error Object of type ... is not JSON serializable.
Could those be serialized using an encoding like

base64.b64encode(pickle.dumps(x)).decode('utf-8')

Currently, when an numpy array is part of a sent message, the receiver will receive it as list. Could such implicit transformation be avoided?

Bug in Local platform:
Currently, a participant with the same name can join a task multiple times, this is not allowable since it can break the logic of the platform, but it is currently assumed to let users to control this issue. However, this issue should be resoloved by the platform itself.