Clustering in AWS

Out of the box the Connect2id server is configured for standard UDP/IP multicast based clustering. This is the most efficient transport method due to its minimal networking overhead.

AWS and most other cloud providers however do not support multicast because servers are typically not provisioned on the same network switch and that would be difficult to implement anyway.

The Connect2id server employs an Infinispan datagrid from JBoss / RedHat to cluster its nodes. Infinispan in turn uses JGroups to handle the actual messaging down the network stack.

In EC2 the multicast node auto-discovery will have to be replaced with another method, e.g. S3_PING (that seems to be the recommended method). Then, the nodes will have to use direct TCP links instead of UDP multicast to replicate the data among themselves.

JGroups provides two modules for implementing the initial node discovery on AWS:

1. S3_PING

This module is included in JGroups and uses a shared S3 bucket to let the nodes discover one another and exchange their IP addresses.


This module is provided by a third-party developer, and uses an AWS API for discovery.

Configuring the Connect2id server for S3_PING

The location of the JGroups settings file is given in the Infinispan configuration file:


Replace the original multicast configuration with the following XML (also included in the infinispan JAR):

<stack-file name="jgroups-config" path="default-configs/default-jgroups-ec2.xml"/>

Set the following system properties at JVM startup, or alternatively, use the above file as a template and plug in your own custom values. In both cases the result should be the same.

System property Description Default Required?
jgroups.tcp.address IP address to use for the TCP transport. No
jgroups.tcp.port Port to use for TCP socket 7800 No
jgroups.s3.access_key The Amazon S3 access key used to access an S3 bucket Yes
jgroups.s3.secret_access_key The Amazon S3 secret key used to access an S3 bucket Yes
jgroups.s3.bucket Name of the Amazon S3 bucket to use. Must be unique and must already exist. Yes
jgroups.s3.pre_signed_delete_url The pre-signed URL to be used for the DELETE operation. Yes
jgroups.s3.pre_signed_put_url The pre-signed URL to be used for the PUT operation. Yes
jgroups.s3.prefix If set, S3_PING searches for a bucket with a name that starts with the prefix value. No

Configuring the Connect2id server for AWS_PING


comments powered by Disqus