Data migration

This guide outlines a safe strategy for migrating Connect2id server data between two server instances using the available server web APIs. Such a migration can be useful when switching to a different server backend database, for example from an LDAP to an SQL backend database.

Conventions in this guide:

Source server – The Connect2id server from which the data is migrated.
Target server – The Connect2id server to which the data is migrated.

Server versions:

The target server version should be equal to the source server or newer. Migrating data to an older Connect2id server version isn’t recommended and isn’t generally supported.
The target Connect2id server version must be 6.6 (2017-03-20) or newer. Earlier Connect2id server versions have API support for importing data that is limited to client registrations only.

1. Preparation

1.1 Server JWK set

Make sure the target Connect2id server has a JSON Web Key set identical to the source server before the data migration is performed.

This will guarantee that all tokens and identifiers that are secured with a cryptographic key will continue to work when traffic is switched to the target Connect2id server. After that, when the new server is up and running, the keys that permit rotation (the asymmetric signing and encryption keys) can be safely rotated.

2.2 Server configuration

There are no other particular requirements about the source and the target Connect2id server configurations apart from the mentioned to have identical JWK sets.

If the aim is to have a target Connect2id server with the same OpenID provider / OAuth 2.0 authorisation server properties as the source server make sure all applicable configurations and plugins (SPI implementations) are replicated.

2. Actual migration

To perform the actual data migration both source and target Connect2id server must be up and running.

A migration will encompass between one and three types of data:

Client registrations – required
Authorisation records – optional
End-user (subject) sessions – optional

A minimal strategy is to migrate the client registrations only, as explained in the next step. A more complete strategy is to include authorisation records (persisted consent). Finally, end-user sessions may also be migrated.

Step 1. Migrate client registrations

The registrations of the OAuth 2.0 clients / OpenID relying parties is the most important piece of data. Without that clients won’t be able to use the Connect2id server at all, and must be registered again.

Export all existing client registrations from the source Connect2id server with an HTTP request like this:

GET /clients HTTP/1.1
Host: source.c2id.com
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

The Connect2id server will return a JSON array witl all registered clients:

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: no-store
Pragma: no-cache

[
  { "client_id" : "s6BhdRkqt3", ... },
  { "client_id" : "joaix93Kf4", ... },
  ...
]

Post the JSON object for each individual client registration to the target Connect2id server. Use the preferred_client_id and preferred_client_secret parameters to force the Connect2id server to not provision new client credentials (the default behaviour) but use the specified values instead. To do that simply replace all "client_id" JSON member keys with "preferred_client_id" and all "client_secret" keys with "preferred_client_secret" in the JSON.

If the Connect2id server has a custom client secret codec it must allow an export of the secret value, in a way that can be understood when imported with preferred_client_secret.

Example POST request:

POST /c2id/clients HTTP/1.1
Host: target.c2id.com
Content-Type: application/json
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

{
  "preferred_client_id"     : "s6BhdRkqt3",
  "preferred_client_secret" : "ahL7AhthchiNg6beAo5HeijeThae3deiChab7ajiVuip2eodesoBie0ufohtiiK4",
  ...
}

Note, the following client metadata parameters will be ignored when imported with a POST request. For them the Connect2id server will automatically generate new values:

client_id_issued_at – The timestamp of the client_id creation. It will be reset to the current timestamp.
registration_access_token – The special access token intended to let a client registered via open registration to self-manage (update, read or delete) its registration. Contact support if you need to import such clients.

Step 2. Migrate authorisation records

The next step is to migrate the existing long-lived (persisted) authorisations. Without that all refresh tokens issued to clients will become invalid and users who have their consent remembered (persisted) will need to authorise their client applications again.

Export all long-lived authorisations from the source Connect2id server:

GET /authz-store/rest/v3/authorizations HTTP/1.1
Host: source.c2id.com
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

If the source Connect2id server version is older that v10.0 use this call instead, from v2 of the authorisation store API.

The Connect2id server will return a JSON array of the long-lived authorisations. Note that with a large user base this can be a potentially time-consuming operation.

HTTP/1.1 200 OK
Content-Type: application/json

[
  { ... },
  { ... },
  ...
]

Recreate each individual authorisation on the target Connect2id server.

Example import for a individual record:

POST /authz-store/rest/v3/authorizations?import=true HTTP/1.1
Host: target.c2id.com
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6
Content-Type: application/json

{
  "sub": "alice",
  "cid": "123",
  ...
}

Note, when the Connect2id server is configured to rotate the issued refreshed tokens (designating them as one-time use only) any refresh tokens issued by the source Connect2id server in the time window between the records export and the switch of client traffic to the target server will be deemed invalid when the client tries to make use of them at target server. The clients will then have to ask for re-authorisation at the Connect2id server for the affected users.

Step 3. Migrate end-user sessions

The final optional step is the migration of the subject (end-user) sessions with the Connect2id server. If that step is skipped all session cookies will become invalid and all users will be forced to authenticate on their next visit to the login page.

Export all sessions from the source Connect2id server:

GET /session-store/rest/v2/sessions HTTP/1.1
Host: source.c2id.com
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

The Connect2id server will return a JSON object of all sessions keyed by their session identifier (SID). Note that this can be a potentially time-consuming operation and for very large user bases the migration may not be feasible.

HTTP/1.1 200 OK
Content-Type: application/json

{
  "N4ICV2CQmTu6KLbrJKQSXw.KDtRkxEw76uDsobEL5zz1Q" : { ... },
  "a-wwPf7tckz2wbzpIEVtug.egmsAdB_sg3Au1hoD6InnA" : { ... },
  ...
}

Recreate each session on the target Connect2id server.

Use the SID-Key header parameter to specify the identifier of the posted session. The SID-Key value is obtained by stripping the appended HMAC from the exported SID (delimited with a dot . character). The target Connect2id server will recompute the exact same SID HMAC.

Example exported SID: N4ICV2CQmTu6KLbrJKQSXw.KDtRkxEw76uDsobEL5zz1Q
The SID key after stripping the HMAC portion: N4ICV2CQmTu6KLbrJKQSXw

Example import for a session:

POST /session-store/rest/v2/sessions HTTP/1.1
Host: target.c2id.com
SID-Key: N4ICV2CQmTu6KLbrJKQSXw
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6
Content-Type: application/json

{
  "sub"  : "alice",
  ...
}