JBoss.org Community Documentation

19.3.2. TCP configuration

Alternatively, a JGroups-based cluster can also work over TCP connections. Compared with UDP, TCP generates more network traffic when the cluster size increases. TCP is fundamentally a unicast protocol. To send multicast messages, JGroups uses multiple TCP unicasts. To use TCP as a transport protocol, you should define a TCP element in the JGroups Config element. Here is an example of the TCP element.

<TCP start_port="7800"
    bind_addr="192.168.5.1"
    loopback="true"
    down_thread="false" up_thread="false"/>
                

Below are the attributes available in the TCP element.

  • bind_addr specifies the binding address. It can also be set with the -Djgroups.bind_address command line option at server startup.

  • start_port, end_port define the range of TCP ports the server should bind to. The server socket is bound to the first available port from start_port. If no available port is found (e.g., because of a firewall) before the end_port, the server throws an exception. If no end_port is provided or end_port < start_port then there is no upper limit on the port range. If start_port == end_port, then we force JGroups to use the given port (start fails if port is not available). The default is 7800. If set to 0, then the operating system will pick a port. Please, bear in mind that setting it to 0 will work only if we use MPING or TCPGOSSIP as discovery protocol because TCCPING requires listing the nodes and their corresponding ports.

  • loopback specifies whether to loop outgoing message back up the stack. In unicast mode, the messages are sent to self. In mcast mode, a copy of the mcast message is sent. The default is false.

  • recv_buf_size, send_buf_size define receive and send buffer sizes. It is good to have a large receiver buffer size, so packets are less likely to get dropped due to buffer overflow.

  • conn_expire_time specifies the time (in milliseconds) after which a connection can be closed by the reaper if no traffic has been received.

  • reaper_interval specifies interval (in milliseconds) to run the reaper. If both values are 0, no reaping will be done. If either value is > 0, reaping will be enabled. By default, reaper_interval is 0, which means no reaper.

  • sock_conn_timeout specifies max time in millis for a socket creation. When doing the initial discovery, and a peer hangs, don't wait forever but go on after the timeout to ping other members. Reduces chances of *not* finding any members at all. The default is 2000.

  • use_send_queues specifies whether to use separate send queues for each connection. This prevents blocking on write if the peer hangs. The default is true.

  • external_addr specifies external IP address to broadcast to other group members (if different to local address). This is useful when you have use (Network Address Translation) NAT, e.g. a node on a private network, behind a firewall, but you can only route to it via an externally visible address, which is different from the local address it is bound to. Therefore, the node can be configured to broadcast its external address, while still able to bind to the local one. This avoids having to use the TUNNEL protocol, (and hence a requirement for a central gossip router) because nodes outside the firewall can still route to the node inside the firewall, but only on its external address. Without setting the external_addr, the node behind the firewall will broadcast its private address to the other nodes which will not be able to route to it.

  • skip_suspected_members specifies whether unicast messages should not be sent to suspected members. The default is true.

  • tcp_nodelay specifies TCP_NODELAY. TCP by default nagles messages, that is, conceptually, smaller messages are bundled into larger ones. If we want to invoke synchronous cluster method calls, then we need to disable nagling in addition to disabling message bundling (by setting enable_bundling to false). Nagling is disabled by setting tcp_nodelay to true. The default is false.