Clustering in AWS

Out of the box the Connect2id server is configured for simple UDP/IP multicast based clustering. This is the most efficient transport method due to its minimal networking overhead.

AWS and most other cloud providers however do not support multicast because servers are typically not provisioned on the same network switch and that would be difficult to implement anyway.

The Connect2id server employs an Infinispan datagrid from JBoss / RedHat to cluster its nodes. Infinispan in turn uses JGroups to handle the actual messaging down the network stack.

In EC2 the multicast node auto-discovery will have to be replaced with another method, e.g. S3_PING (that seems to be the recommended method). Then, the nodes will have to use direct TCP links instead of UDP multicast to replicate the data among themselves.

JGroups provides two modules for implementing the initial node discovery on AWS:

1. S3_PING

This module is included in JGroups and uses a shared S3 bucket to let the nodes discover one another and exchange their IP addresses.

http://www.jgroups.org/manual/index.html#_s3_ping

2. AWS_PING

This module is provided by a third-party developer, and uses an AWS API for discovery.

http://www.jgroups.org/manual/index.html#_aws_ping

https://github.com/meltmedia/jgroups-aws

Configuring the Connect2id server for S3_PING

The location of the JGroups settings file is given in the Infinispan configuration file:

WEB-INF/infinispan.xml

Replace the original multicast configuration with the following XML (also included in the infinispan JAR):

<stack-file name="jgroups-config" path="default-configs/jgroups-ec2.xml"/>

Set the following system properties at JVM startup, or alternatively, use the above file as a template and plug in your own custom values. In both cases the result should be the same.

System property	Description	Default	Required?
jgroups.tcp.address	IP address to use for the TCP transport.	127.0.0.1	No
jgroups.tcp.port	Port to use for TCP socket	7800	No
jgroups.s3.access_key	The Amazon S3 access key used to access an S3 bucket		Yes
jgroups.s3.secret_access_key	The Amazon S3 secret key used to access an S3 bucket		Yes
jgroups.s3.bucket	Name of the Amazon S3 bucket to use. Must be unique and must already exist.		Yes
jgroups.s3.pre_signed_delete_url	The pre-signed URL to be used for the DELETE operation.		Yes
jgroups.s3.pre_signed_put_url	The pre-signed URL to be used for the PUT operation.		Yes
jgroups.s3.prefix	If set, S3_PING searches for a bucket with a name that starts with the prefix value.		No

Configuring the Connect2id server for AWS_PING

TBD