PoC on new communication architecture based on gRPC about fedbiomed HOT 1 CLOSED

Long-lived polling from node to researcher:

There are several approaches for long-lived polling as using unary RPC and stream RPC. During the long-lived polling nodes asks for tasks from researcher such as train, search and etc.

Unary - Unary RPC

Creates a new request in a while loop. Waits until timeout or receives a task from researcher. Researcher answers as unary RPC as well.

• Creates new connection to request new task.
• Sending new task request requires a certain amount of time since it creates new connection.
• Requires to increase max message length (default 4MB)
• The time consumed for receiving task is not stable (test: between 2 seconds and 5 seconds for 30MB of data) it may be due to sending large dataset
• Does not require asynchronous programming.
• Risk of memory due to receiving large message in single request.

Unary - Streaming RPC

Creates a new request in a while loop.Waits until timeout or receives a task from researcher. Researcher answers with streaming to send large messages as 4MB chunks.

• Creates new connection to request new task - one the node side.
• Sending new task request requires time to create new connection.
• Does not require to increase max message length, large messages will be send as small message chunks
• The time consumed for receiving a large task message is stable (test: around 2.9 seconds for 30MB of data - with 4 other nodes requesting at the same time)
• Does not require asynchronous programming on the node side.
• No Risk of memory due to receiving large message in single request.
• Performs slower comapre to "Stream-Stream RPC" approach while the message size is smaller. It is due to time crating a new connection to request new task from researcher. (test: 60ms slower)

Streaming-Streaming RPC

Node creates a streaming request and receives task in a streaming. Connection is created once.

• Does not create new connection to request new task. Sends task request within the open stream.
• Sending new task request is faster since it does not create a new connection.
• Does not require to increase max message length, large messages will be send as small message chunks
• The time consumed for receiving task is stable (test: avg 3.2 seconds for 30MB data - with 4 other nodes requesting at the same time )
• Does require asynchronous programming where one co-routine listens for tasks, and another one asks for new tasks.
• No risk of memory due to receiving large message in single request.
• Performs better for small size messages compare to "Unary-Streaming RPC approach" (test: 60ms faster)

from fedbiomed.