JGroup is the corner stone for clustering. It is a reliable multicast system. As we know, using IP multicasting we can set a group of computers to use one multicast IP address, and all the messages sent to this multicast IP address will be received by all the computers in the group.
IP multicasting is usually used together with UDP protocol. Because UDP protocol is stateless, if one computer send an UDP datagram to one multicast IP address, this UDP datagram will be sent to all the members in the group. Meanwhile, the sender of this UDP datagram don't care whether destination computers received this datagram or not. Here is the diagram that shows a IP multicasting group that contains three members:
Here is the use case for above scenario:
UDP is suitable for this scenario. Because if we use TCP, then the sender must establish connections with all the members in a multicasting group, and check the messages are received successfully by all the members.
However, UDP is not a reliable protocol, the datagram maybe lost during transmission and it won't be checked:
Besides, UDP datagrams may also arrived in random sequence and not sorted by timeline: If the sender sends packet 1 firstly, and then packet 2, the receiver may receive packet 2 firstly and then 1:
So if we want to build a reliable cluster system based on IP multicast + UDP, we need to ensure the data stability by ourselves.
One example is like numbering all the datagrams we've sent in application layer. And the receiver will need to check the data is not lost and then sort the packets into correct order. For example:
- If sender sent packet 1, 2 to multicast group, and in this group there are two receivers called A and B. ReceiverA received packet 1, 2 successfully, but receiver B only get packet 1. Then receiverB will ask sender to retransmit packet2. Please note this time the sender don't have to send packet 1 to multicast address, because other receivers in the group didn't ask for it because they have gotten the packet 2 successfully. Sender just need to send packet 1 to receiverB again with an unicast to reduce bandwidth cost:
- If sender sent packet 1, packet 2, and receiverA received packet 2, packet 1, it will reorder the packet to packet 1, packet 2 by itself.
In above is some simple design concepts on how to build a reliable transmission system on top of IP multicasting + UDP. JGroup is certainly much more complex than above example but the design concept is basically the same.
In JGroup, the minimal group unit is called 'Channel'. Members in a channel can talk with each other. The data send/receive in a channel is called 'Message'. Here is the use case:
It's very similar to IP multicasting + UDP:
Generally speaking, JGroup is designed as a 'layered protocol stack'. In the bottom of the stack it's the 'UDP' protocol. Other protocols are built on top of it to meet different requirements of a cluster. For example, some protocol ensures the packet reliability (like the example shown above: all the packets should be sent to the members in cluster with 0% data loss); some protocol will check the liveness of the members in a cluster (for example, ping the members periodically); and some protocol will cut the big datagram into smaller one for UDP to be able to transmit them (as we know, the UDP packet is sensitive to the size of datagram).
Here is the default setup of JGroup protocol stack:
You can even guess some protocol's purpose by its name. The complete description of each protocol could be found here:
I wish you have caught the general design concepts of JGroup with above description. Now let's get into the code:
In JGroup a cluster is called a channel. The members in the channel can send messages to or receive messages from this channel. We will develop an example that creates a channel and have two members in it: A sender that sends messages to the channel and a receiver that receive messages from the channel.
Please note a member in a channel can be both a sender a receiver. We just simplify the scenario in the example for the code to be better explained. Here is the use case:
From the above diagram we can see the first step to setup a cluster is to ask members to join to a same channel, then they can communicate with other members in this group. In below are the real codes:
The first step is to create a JChannel instance:
If no setting provided, JChannel will provide a default protocol stack:
The default stack is like this:
Each protocol is focusing on its own task, for a complete definition of each stack, please check JGroup Protocol Link
After the loading process, the xml file is loaded into ProtocolStackConfigurator:
After loading the configure, JChannel will go on its initialization process:
A ProtocolStack instance is created with the current Channel config, and two xml files are loaded into ClassConfigurator:
Setup protocol stack (creates protocol, calls init() on them)
In the situation of default configuration:
- top_prot == STATE_TRANSFER
- bottom_prot == UDP
- initProtocolStack will setup each protocol in the stack: