This project is read-only.
1

Closed

Process-to-process direct communication

description

In the current design the inter-process communication is performed via sets. Although this approach is reliable, simple and requires minimum additional setup, in a large scale it may become an issue. All the IPC load will fall onto the directory and it will become a bottleneck. The direct process-to-process communication approach addresses this issue.

The key property of a process in Flower is that any process is run by a single processor which is a single machine. The processor exposes a service endpoint by which it receives notifications from the directory. These endpoints, potentially, can be used by the processors to communicate to each other without the directory.

The processors need to have a transactional storage (local in most of the cases, but not necessary) to save the incoming messages persistently (inbox). The storage has to support distributed transactions to be able to synchronize its state with the directory if required (see further).

Overall operation

To send a message, the sender process obtains the endpoint proxy of the processor running the recipient process. The recipient is identified by pid. The processor path can be inferred from the path of the process which can be obtained from the directory by pid. The service endpoint is called in a transaction context.

The recipient processor puts the message into the inbox and, if the recipient process is already waiting for the message, the recipient processor switches it to the pending state (here the synchronization of the inbox with the directory is required); otherwise, the recipient processor does nothing.

To receive a message, the recipient process checks the inbox. If there are no messages for it, the process goes to the waiting state; otherwise, it removes the message from the inbox, saves the process state and goes futher.

Inbox synchronisation

The put and get operations of the inbox for the same pid must be synchronized. This will prevent the cases when a process waits infinitely for a message that arrived right at the moment when it checked the inbox. If the sender and receiver transactions run simultaneously, it is posible that the recipient will see no message and the sender will see no waiter at the same time.

Processor proxy life-cycle

A processor must cache the other's processors proxies hence it must somehow know when a processor configuration changes. There are two ways to achieve that:
  • Directory notifications. The directory can track the changes of the processors and tell other processors to update their proxies. This may be done via almost the same mechanism as the Pending folder changes notifications.
  • Pollling. It doesn't scale well, but it should backup the notifications mechanism.
Also, if a send call fails, the processor must try to update the proxy and retry.
Closed Nov 3, 2013 at 11:28 AM by dbratus

comments

dbratus wrote Oct 28, 2013 at 7:40 PM

Since introduction of application callbacks and queue services in 0.4 the issue is resolved.