I’m currently creating a clustered application in WildFly which communicates through a custom JGROUPS channel. The applications sends an RPC call to the coordinator which checks if the desired changes are possible.
The coordinator again sends an RPC call to all cluster members to sync its state (tells them, that the changes will be made). The coordinator can’t process the response of the second call, because the first call is still being processed (‘max one thread per sender’ logic) and therefor the whole thing fails with a timeout. Similar to a thread deadlock.
How to overcome this limitation? In my case there are only two nested calls, the solution should also work for more nested calls (I can’t ensure, that more nested calls may happen in the future)
—
I’m using RPC calls because they are synced an so I can ensure, that all nodes always have the same state.
- Node A sends ‘change request’ to coordinator
- Coordiator send ‘change done’ to all nodes
- Node A will send ‘something requiring the previous change’ to all nodes
- Without sync #3 may be received by node B before #2, correct? As far as I know only the messages per sender are ordered but there is not a total order across all senders besides using the SEQUENCE protocoll which will always forward the mesages to the coordinator which sends the message for the real sender.
Advertisement
Answer
A possible solution is to implement an own RpcDispatcher based on asyncDispatching=true and it’s request-handler method handle(Message req, Response response). The own implementation simply adds the respone object to the args of the method invokation.
This way you do not block the JGROUPS thread of the sender.