How to create a global DynamoDB table in AWS

2018-09-05

In November 2017 AWS announced a DynamoDB feature enabling transparent asynchronous replication of table data between any number of regions. This can be useful in a distributed application and for failover between AWS regions. Data can be written and read in each of the regions with replicas of the table.

This is the missing guide for creating global tables with the AWS Java SDK. You'll need a recent version of the SDK as older releases lack the classes for dealing with global DynamoDB tables.

The steps for creating a global DynamoDB table:

  1. Create a DynamoDB client for each of the AWS regions where you need to have replicas.

  2. In each region create the desired DynamoDB table, with the usual parameters, but also enable the table for streaming and specify what is to be streamed - both the new and old images of the item (StreamViewType.NEW_AND_OLD_IMAGES).

  3. Finally, run a request to turn all those created tables into a single global one, by putting them into a replication group. This request can run in any AWS region.

Here is a Java example:

import java.util.Collection;
import java.util.LinkedList;

import com.amazonaws.auth.DefaultAWSCredentialsProviderChain;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
import com.amazonaws.services.dynamodbv2.model.*;
import org.junit.Test;

public class AWSGlobalTableTest {

    private static final String EU_CENTRAL_1 = "eu-central-1";

    private static final String US_EAST_1 = "us-east-1";

    private static final String TABLE_NAME = "my_global_table";

    @Test
    public void createGlobalTable() {

        // Create a client for each of the regions with table replicas
        AmazonDynamoDB client1 = AmazonDynamoDBClientBuilder
            .standard()
            .withCredentials(DefaultAWSCredentialsProviderChain.getInstance())
            .withRegion(EU_CENTRAL_1)
            .build();

        AmazonDynamoDB client2 = AmazonDynamoDBClientBuilder
            .standard()
            .withCredentials(DefaultAWSCredentialsProviderChain.getInstance())
            .withRegion(US_EAST_1)
            .build();

        // The table spec, must be enabled for streaming
        Collection<KeySchemaElement> keyAttrs = new LinkedList<>();
        keyAttrs.add(new KeySchemaElement("id", KeyType.HASH));

        Collection<AttributeDefinition> attrs = new LinkedList<>();
        attrs.add(new AttributeDefinition("id", ScalarAttributeType.S));

        CreateTableRequest createTableRequest = new CreateTableRequest()
            .withTableName(TABLE_NAME)
            .withKeySchema(keyAttrs)
            .withAttributeDefinitions(attrs)
            .withStreamSpecification(
                new StreamSpecification()
                    .withStreamEnabled(true)
                    .withStreamViewType(StreamViewType.NEW_AND_OLD_IMAGES))
            .withProvisionedThroughput(new ProvisionedThroughput(1L, 1L));

        // Create the table in each region
        client1.createTable(createTableRequest);
        client2.createTable(createTableRequest);

        // The global table request can be executed in any of the regions
        CreateGlobalTableResult createGlobalTableResult = client1.createGlobalTable(
            new CreateGlobalTableRequest()
                .withGlobalTableName(TABLE_NAME)
                .withReplicationGroup(
                    new Replica().withRegionName(EU_CENTRAL_1),
                    new Replica().withRegionName(US_EAST_1)));
    }
}

Global tables will be automatically provisioned with three extra attributes - the timestamp of the item update, in which region that happened and a flag whether the item is pending deletion.

The Connect2id server for IdP, SSO and API security is going to support global DynamoDB tables in the next 7.6 release. This will make it possible to run a global IdP operation, with the ability to service clients and applications close to their deployment location for minimal latency.

Incremental authorisation, resource indicators and EdDSA in release 6.0 the OAuth 2.0 / OpenID Connect SDK

2018-09-03

The slow summer period is now over and it's now time to announce a new major release of our popular open source OAuth 2.0 / OpenID Connect SDK, on which a number of application frameworks as well as client and server implementations rely.

What's in the new 6.0 release?

  • Experimental support for incremental authorisation, which allows OAuth 2.0 clients to ask for more consent (access) while preserving any previously granted scopes. The intent of this draft to improve the user experience around consent, so that users are asked for things only as needed and when needed (and also make the process easier for application developers).

  • Also experimental support for the resource parameter draft, which lets clients indicate the URL of the resource server in authorisation and token requests. We are a bit sceptical of the actual utility of this proposed spec, save for servers that require it for some legacy reason. If you need to issue tokens for multiple resource servers (web APIs), we suggest you encode the resource server identity (or URL) in the scope value instead, e.g. by prefixing the resource URL to the scope, which should be easier to understand and handle, while also effectively preventing clashes of scope values between resource servers.

  • The Nimbus JOSE+JWT dependency was bumped to 6.0 to enable issue and handling of ID tokens secured with high-performance Ed25519 digital signatures. According to our benchmarks signing of tokens with Ed25519 is 63x cheaper compared to RSA.

  • The optional OpenSAML dependency was also bumped, to the latest stable 3.2, for those applications that need to exchange SAML 2.0 assertions for OAuth 2.0 access tokens.

  • The authorisation and token requests can now support multi-valued parameters. This is a breaking change which opens up the SDK to more potential use cases, such as handling multiple resource parameters.

  • A bunch of classes and methods marked as deprecated in 5.x were removed.

Check the release notes below for a full list of all changes.


version 6.0 (2018-09-02)

  • Adds experimental support for Resource Indicators for OAuth 2.0 (draft-ietf-oauth-resource-indicators-00).
  • Adds experimental support for OAuth 2.0 Incremental Authorization (draft-ietf-oauth-incremental-authz-00).
  • Updates AuthorizationRequest and AuthenticationRequest to support multi-valued query parameters (breaking change).
  • Updates the AuthorizationResponse classes to support multi-valued parameters (breaking change).
  • Updates TokenRequest to support multi-valued parameters in the request body (breaking change).
  • Updates the AuthorizationGrant classes to support multi-valued parameters in the request body (breaking change).
  • Updates HTTPRequest to support multi-valued parameters in the query and body (breaking change).
  • Switches com.nimbusds.openid.connect.sdk.op.AuthenticationRequestResolver to com.nimbusds.jose.util.ResourceRetriever (breaking change).
  • Updates HTTPRequest and HTTPResponse to support multi-valued HTTP headers (breaking change) (iss #252).
  • Removes deprecated com.nimbusds.oauth2.sdk.http.Resource class.
  • Removes deprecated com.nimbusds.oauth2.sdk.http.ResourceRetriever interface.
  • Removes deprecated com.nimbusds.oauth2.sdk.http.RestrictedResourceRetriever interface.
  • Removes deprecated com.nimbusds.oauth2.sdk.http.AbstractRestrictedResourceRetriever class.
  • Removes deprecated com.nimbusds.oauth2.sdk.http.DefaultResourceRetriever class.
  • Removes deprecated com.nimbusds.oauth2.sdk.jose.jwk.JWKSource interface.
  • Removes deprecated com.nimbusds.oauth2.sdk.jose.jwk.AbstractJWKSource class.
  • Removes deprecated com.nimbusds.oauth2.sdk.jose.jwk.ImmutableJWKSet class.
  • Removes deprecated com.nimbusds.oauth2.sdk.jose.jwk.ImmutableClientSecret class.
  • Removes deprecated com.nimbusds.oauth2.sdk.jose.jwk.RemoteJWKSet class.
  • Removes deprecated com.nimbusds.oauth2.sdk.jose.jwk.AbstractJWKSelector class.
  • Removes deprecated com.nimbusds.oauth2.sdk.jose.jwk.JWSVerificationKeySelector class.
  • Removes deprecated com.nimbusds.oauth2.sdk.jose.jwk.JWEDecryptionKeySelector class.
  • Removes deprecated com.nimbusds.oauth2.sdk.jose.jwk.KeyConverter class.
  • Removes deprecated com.nimbusds.openid.connect.sdk.id.AESBasedPairwiseSubjectCodec class.
  • Upgrades optional OpenSAML dependency to 3.0+ (breaking change for all OpenSAML APIs).
  • Updates Nimbus JOSE+JWT to 6.0.1 with Ed25519 and X25519 support.

High-performance Ed25519 and X25519 cryptography comes to Nimbus JOSE+JWT

2018-08-22

Asymmetric RSA and EC cryptography comes at a cost. Its overhead can be a limiting factor in how quickly an OpenID Connect / OAuth 2.0 server can mint tokens, and client applications and protected resources can validate them. This new 6.0 release of the Nimbus JOSE+JWT library introduces support for high-performance public key cryptography based on the Ed25519 and X25519 algorithms. The benchmarks showed signatures with Ed25519 receiving a 62x boost over 2048-bit RSA, with verification remaining roughly on par.

Operation Ed25519 RSA 2048 ECDSA P-256
Sign 14635 ops/s 236 ops/s 1083 ops/s
Verify 5065 ops/s 8598 ops/s 687 ops/s

Edwards-curve cryptography was defined for JOSE in RFC 8037, the last specification to come out of the working group. The standard Java Cryptography Architecture (JCA) still lacks Edwards-curve cryptography support, which meant we had to find a suitable external Ed25519 / X25519 implementation and integrate it directly, without the help of JCA abstractions. For that we chose Google's Tink, which is written entirely in Java and is already mature enough to be used in production.

To use Ed25515 and X25519 for JOSE you need to include the Tink dependency, which is marked as optional, in your Maven project. For Nimbus JOSE+JWT 6.0 that is

<dependency>
    <groupId>com.google.crypto.tink</groupId>
    <artifactId>tink</artifactId>
    <version>1.2.0-rc2</version>
</dependency>

Edwards-curve cryptography in JOSE requires a new type of key called octet key pair (OKP). To generate such a JWK of type OKP just specify the curve name and any key metadata required by your application:

import java.util.*;

import com.nimbusds.jose.jwk.*;
import com.nimbusds.jose.jwk.gen.*;

// Generate Ed25519 Octet key pair (OKP) in JWK format, attach some metadata
OctetKeyPair jwk = new OctetKeyPairGenerator(Curve.Ed25519)
    .keyUse(KeyUse.SIGNATURE) // indicate the intended use of the key
    .keyID(UUID.randomUUID().toString()) // give the key a unique ID
    .generate();

// Output OKP
System.out.println(jwk);

// Output public key only
System.out.println(jwk.toPublicJWK());

Example Ed25519 key in JWK format with public x parameter, private d parameter and metadata:

{
  "kty" : "OKP",
  "crv" : "Ed25519",
  "x"   : "11qYAYKxCrfVS_7TyWQHOg7hcvPapiMlrwIaaPcHURo",
  "d"   : "nWGxne_9WmC6hEr0kuwsxERJxWl7MmkZcDusAxyuf2A"
  "use" : "sig",
  "kid" : "FdFYFzERwC2uCBB46pZQi4GG85LujR8obt-KWRBICVQ"
}

Here is a code snippet showing signing and validation of a JSON Web Token (JWT) secured with Ed25519:

import java.util.Date;

import com.nimbusds.jose.*;
import com.nimbusds.jose.crypto.*;
import com.nimbusds.jose.jwk.*;
import com.nimbusds.jose.jwk.gen.*;
import com.nimbusds.jwt.*;


// Create the EdDSA signer with the generated OKP
JWSSigner signer = new Ed25519Signer(jwk);

// Prepare JWT with claims set
JWTClaimsSet claimsSet = new JWTClaimsSet.Builder()
    .subject("alice")
    .issuer("https://c2id.com")
    .expirationTime(new Date(new Date().getTime() + 60 * 1000))
    .build();

SignedJWT signedJWT = new SignedJWT(
    new JWSHeader.Builder(JWSAlgorithm.Ed25519).keyID(jwk.getKeyID()).build(),
    claimsSet);

// Compute the EC signature
signedJWT.sign(signer);

// Serialize the JWS to compact form
String s = signedJWT.serialize();

// On the consumer side, parse the JWS and verify its EdDSA signature with the
// public OKP
signedJWT = SignedJWT.parse(s);

JWSVerifier verifier = new Ed25519Verifier(publicJWK);
assertTrue(signedJWT.verify(verifier));

// Retrieve / verify the JWT claims according to the application requirements
assertEquals("alice", signedJWT.getJWTClaimsSet().getSubject());
assertEquals("https://c2id.com", signedJWT.getJWTClaimsSet().getIssuer());
assertTrue(new Date().before(signedJWT.getJWTClaimsSet().getExpirationTime()));

Release notes

version 6.0 (2018-08-07)

  • Adds an Ed25519Signer and Ed25519Verifier for EdDSA with an Ed25519 curve (RFC 8037, section 3.1). Requires the optional com.google.crypto.tink:tink:1.2.0-rc2 dependency.
  • Adds an X25519Encrypter and X25519Decrypter for ECDH-ES key agreement with an X25519 curve (RFC 8037, section 3.2). Requires the optional com.google.crypto.tink:tink:1.2.0-rc2 dependency.
  • Adds an OctetKeyPairGenerator for generating JSON Web Keys (JWT) of type "OKP" with Ed25519 and X25519 curves.
  • Breaking change: The ephemeral public key (epk) in JWEHeader is now represented by the more general JWK class instead of ECKey which only allowed EC JWKs. The change was necessary to implement ECDH-ES key agreement with an Octet Key Pair (OKP) JWK.
  • Refactors the Base64 and Base64URL codec utilities to prevent potential cache timing attacks due to use of table lookups. The Base64URL utility is used to decode the encrypted key portion in a JWE object (iss #270).
  • Fixes Maven build so that the output JDK 1.7 JAR has 1.7 classes and not 1.6 (iss #271).