Connect2id server 6.15 introduces DynamoDB support, simplifies clustering

15 November 2017

DynamoDB persistence and simplified cluster deployment – these are the two highlights of release 6.15 of the OpenID Connect / OAuth 2.0 server.

DynamoDB

We love DynamoDB for two reasons. Reason one: It’s the most completely managed DBaaS in the AWS cloud, even ahead of Amazon’s fancy Aurora. Reason two: The data model of the Connect2id server is almost a precise fit to the database.

Deploying a Connect2id server with DynamoDB is designed to be a breeze. You only need to set the AWS region and the access credentials. At startup the server will automatically create all necessary tables for its operation. Once that’s done, specify read and write throughputs to match your apps’ requirements, or easier still, turn auto-scaling on.

If you’re running the Connect2id server cluster in “stateless” mode (see below), you even have a way to fire up cross-region replication, using special streams between the DynamoDB tables.

Simplified clustering

The Infinispan and JGroups configurations were reworked. You can now use identical protocol settings regardless of where you choose to deploy a Connect2id server cluster – on premise, or in the cloud.

TCP became the default transport for messaging. UDP/IP multicast was phased out, as it’s practically never available in a cloud environment.
JDBC_PING becomes the default membership discovery protocol for all Connect2id server deployments that have a relational database (MySQL, PosgreSQL) to persist data. Discovery will happen via a jgroupsping table where joining nodes can peek into to locate the coordinator. The PING protocol which relies on multicast will no longer be used, and in case of AWS deployments, there is no longer a need to fiddle with S3_PING.
Similarly, deployments with DynamoDB will use a native DYNAMODB_PING which was developed at Connect2id and was contributed to the JGroups project where it will soon appear in project’s GitHub repo.

Cluster deployments with an LDAP backend remain unchanged, i.e. multicast remains the default method for discovery and transport. Our customer records show that LDAP based deployments are mostly hosted on premise, in private data centres.

New cluster metrics

Two new gauges were added to the metrics endpoint, to report the current cluster size and whether the queried server is the coordinator:

infinispan.numClusterMembers – The number of Connect2id cluster members, as seen from the queried member. If the cluster has degraded into “split brain” state, for example due to networking issues, the reported number will not be consistent across the entire cluster. In that case check the server logs for more information. Zero (0) if clustering is disabled or “stateless”.
infinispan.isCoordinator – Indicates if the queried cluster member is the current coordinator (true or false). false if clustering is disabled or “stateless”.

Updated health-check

The health-check endpoint was also updated. Previously it only had checks that ping the underlying databases. Now it also includes a check for the “split brain” cluster condition, when communication between nodes has broken down, and there is risk of in-memory data becoming inconsistent.

The monitoring endpoint can now accept the access token passed as a query parameter. With that the health-check can be easily polled from a load balancer or reverse proxy using a simple URL like http://c2id.com//monitor/v1/healthcheck?access_token=ier2Aecoh....

New “stateless” cluster mode

Perhaps you’ve noticed that for the past year or so the Connect2id server supported an invalidation cluster mode using Infinispan together with Redis (the latter as primary in-memory store and cache). This mode was intended for large IdPs who occasionally need to deal with spikes in traffic. With invalidation objects are not replicated between the cluster nodes, which eases the process (and overhead) of scaling up and down to match demand. Requested objects are always loaded from the underlying stores - Redis for transient objects and the database, e.g. MySQL, for persisted ones. Only simple “delete key” commands get passed between the nodes, to cause objects for update to be invalidated (hence the mode name).

Unfortunately, the invalidation mode in Infinispan turned out to have some serious and rather hard to diagnose issues. For that reason we decided to discontinue it, until the problems get fixed.

We devised an alternative “stateless” cluster mode. In this mode no state at all is kept in the Connect2id server nodes (the web service layer). Instead, all objects, including sessions and cached information, are kept underlying data store layer – Redis + database, or DynamoDB (with optional DAX). Nodes can now be added and removed with zero extra overhead.

Customers who have a cluster in invalidation mode can perform a rolling upgrade to the new stateless mode. If you have questions about that, please contact Connect2id support.

Improved support for advanced claims sources

The claims sourcing SPI was updated to support reuse of the UserInfo access token with upstream distributed and aggregated claims providers.

For the complete list of changes and resolved issues check out the 6.15 change log.

Download

To download a ZIP package of Connect2id server 6.15:

https://c2id-downloads.s3.eu-central-1.amazonaws.com/server/6.15/Connect2id-server.zip

(SHA-256: 0077bf130ea1a2e140484b848b1858ff3a1425dd2e74c574083176e2330add90)

As WAR package only:

https://c2id-downloads.s3.eu-central-1.amazonaws.com/server/6.15/c2id.war

(SHA-256: c4f6a913c9b6227426155e46fc07f5691793b5f9c4be0c890ab9ab442cbd50e3)

Questions?

Get in touch with Connect2id support.

Release notes

6.15 (2017-11-09)

Summary

Adds support for persisting Connect2id server data to a AWS DynamoDB database.
Introduces a new “stateless” clustering mode. Caching and in-memory storage (of sessions, etc) is disabled in each Connect2id server node; all transient and persistent data is instead stored in the underlying database layer (Redis with MySQL, PostgreSQL or LDAP, or DynamoDB with optional DAX). The stateless clustering mode offers simplified scaling - Connect2id server nodes can be added or removed without the typical overhead (node discovery, state transfer and re-balancing) of Infinispan clustering in replication, distribution or invalidation mode.
Discontinues the Infinispan invalidation clustering mode configurations due to reliability issues with the invalidation mode in Infinispan 8.2. Users with Connect2id server clusters in invalidation mode are advised to switch to a matching stateless configuration.
Simplifies deployment of Connect2id server clusters with a MySQL or PostgreSQL backend database by modifying the JGroups configuration to for JDBC_PING for the initial membership discovery (instead of S3_PING) and TCP for the transport (instead of UDP / multicast).
Updates the monitoring API with new “infinispan.numClusterMembers” and “infinispan.isCoordinator” metrics for the cluster size and topology as well as new “[cache/map-name].availability” health checks for detecting degraded cluster conditions (“split brain”).
Updates the implementation of “Mutual TLS Profile for OAuth 2.0” to draft-ietf-oauth-mtls-04.

Configuration

/WEB-INF/oidcProvider.properties
- op.token.authMethods – renames the “pub_key_tls_client_auth” client authentication method to “self_signed_tls_client_auth” to reflect changes in draft-ietf-oauth-mtls-04. Existing registrations of clients using self-signed TLS / X.509 certificate authentication must be renewed or updated.
/WEB-INF/infinispan-*.xml
- infinispan-replication-dynamodb.xml – new Infinispan configuration for a cluster in replication mode, persisting data to a AWS DynamoDB database and using DYNAMODB_PING (a shared jgroups_ping DynamoDB table) for initial cluster membership discovery.
- infinispan-replication-mysql.xml – renames from infinispan-mysql.xml, switches the JGroups configuration to JDBC_PING initial membership discovery and TCP transport.
- infinispan-replication-postgres95.xml – renames from infinispan-postgres95.xml, switches the JGroups configuration to JDBC_PING initial membership discovery and TCP transport.
- infinispan-replication-ldap.xml – renames from infinispan-ldap.xml.
- infinispan-stateless-dynamodb.xml – new Infinispan configuration for a cluster in “stateless” mode where all transient in persistent data is stored in AWS DynamoDB. Caching may be transparently enabled via Amazon DynamoDB Accelerator (DAX).
- infinispan-stateless-redis-mysql.xml – new Infinispan configuration for a cluster in “stateless” mode where transient and cached data is stored in Redis and persisted data in a MySQL database.
- infinispan-stateless-redis-postgres95.xml – new Infinispan configuration for a cluster in “stateless” mode where transient and cached data is stored in Redis and persisted data in a PostgreSQL database.
- infinispan-stateless-redis-ldap.xml – new Infinispan configuration for a cluster in “stateless” mode where transient and cached data is stored in Redis and persisted data in a LDAP directory.
- infinispan-mysql-redis.xml – discontinued due to reliability issues with invalidation in Infinispan 8.2. Switch to the new stateless configuration infinispan-stateless-redis-mysql.xml.
- infinispan-postgres-redis.xml – discontinued due to reliability issues with invalidation in Infinispan 8.2. Switch to the new stateless configuration infinispan-stateless-redis-postgres.xml.
- infinispan-redis-ldap.xml – discontinued due to reliability issues with invalidation in Infinispan 8.2. Switch to the new stateless configuration infinispan-stateless-redis-ldap.xml.
- infinispan-local-h2.xml – renames from infinispan-h2.xml.
Other:
- The location of the preferred Infinispan configuration file in the web application (WAR) can also be set by a Java system property named “infinispan.configurationFile”. If that system property is not set, the configuration file will be looked up in the traditional “infinispan.configurationFile” servlet context parameter.

Web API

/token
- Returns detailed “error_description” when processing malformed OAuth 2.0 grants, such as grants with missing parameters.
/monitor/v1/
- Updates the monitoring API to also accept access tokens passed via an access_token query parameter. Previously the access token could only be passed via the Authorization header. With that the /monitor/v1/healthcheck endpoint can be polled from a load balancer as a simple URL where the access token is included as a query parameter. For security reasons, wherever possible, use the Authorization header for passing the access token, to prevent the token from appearing in web server logs. See The OAuth 2.0 Authorization Framework: Bearer Token Usage (RFC 6750), section 2.3 for more information about passing the access token as a query parameter.
/monitor/v1/metrics
- Adds new “infinispan.isCoordinator” gauge of type boolean to show of the Connect2id server node acts as a coordinator in the Infinispan cluster. If clustering is disabled the gauge outputs false.
- Adds new “infinispan.numClusterMembers” gauge of type integer to show the number of Infinispan cluster members as seen from the queried Connect2id server node. If clustering is disabled the gauge outputs zero.
/monitor/v1/healthcheck
- Adds new “[infinispan-cache/map-name].availability” health checks. The health check returns "healthy":true if the cache/map has started up and is available. The health check returns an error message if the cache/map is not available yet, or has entered a degraded state due to a “spit brain” condition (in a replicated or distributed cluster).

SPI

com.nimbusds.openid.connect.provider.spi.claims.AdvancedClaimsSource
- Adds the UserInfo access token and the client IP address to the ClaimsRequestContext. The claims source may use the UserInfo access token to retrieve aggregated and distributed claims, where the same token is recognised by the upstream claims providers. See OpenID Connect Core 1.0, section 5.6.

Resolved Issues

Fixes a bug which caused sourcing and advertising of OpenID claims from disabled claims sources (issue server/317).
Logs configured SQL data source class name and URL at Connect2id server startup (issue sql-store/11).
Updates the S3_PING JGroups discovery to cause the coordinator to remove stale ping entries from old coordinators after a view change (with remove_old_coords_on_view_change=“true”), switches to asynchronous discovery using separate threads (with async_discovery=“true” and async_discovery_use_separate_thread_per_request=“true”) (issue server/325).
Fixes the eviction strategy and size of op.consentSessionMap to ensure the consent session number is bounded in cases when the authorisation session proceeds directly to consent (issue server/330).

Dependency Changes

Adds dependency to com.nimbusds:infinispan-cachestore-dynamodb:1.5.2
Adds dependency to com.nimbusds:jgroups-dynamodb-ping:1.2.1
Upgrades to com.nimbusds:c2id-server-sdk:3.12
Upgrades to com.nimbusds:oauth2-oidc-sdk:5.40
Upgrades to com.nimbusds:oauth2-authz-store:5.20
Upgrades to com.nimbusds:oidc-session-store:5.2.13
Upgrades to com.nimbusds:common:2.14
Upgrades to com.nimbusds:infinispan-cachestore-sql:2.6.3
Upgrades to org.jooq:jooq:3.9.5
Upgrades to com.zaxxer:HikariCP:2.7.1
Upgrades to org.mariadb.jdbc:mariadb-java-client:2.1.1
Upgrades to org.postgresql:postgresql:42.1.4
Upgrades to com.h2database:h2:1.4.196