Skip to content
Connect2id

Monitoring

1. Introduction

The Connect2id server provides over 100 useful metrics and backend health checks. These are implemented with the Dropwizard Metrics library.

Access to the monitoring endpoint is protected by means of a long-lived token. The token must be passed with each HTTP request in the Authorization header:

Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

The token can be alternatively passed as an access_token query parameter, for example, to enable URL polling of the healthcheck endpoint from a load balancer / reverse proxy:

https://demo.c2id.com/monitor/v1/healthcheck?access_token=ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

Security note: Avoid passing the token as a query parameter to prevent the credential from appearing in web server logs.

Note that the available metrics can also be reported to a remote data collection / dashboard server via JMX or the Graphite protocol.

More information can be found in the monitoring configuration reference.

2. Web API overview

Resources
Representations Errors

3. Resources

3.1 /monitor/v1/metrics

Connect2id server metrics.

3.1.1 GET

Retrieves a set of metrics collected by the Connect2id server.

Header parameters:

  • Authorization Must specify the configured bearer access token for this web API. The token can be alternatively passed as a query parameter (not recommended), see below.

Query parameters:

  • [ pretty = false ] {true|false} Optional query parameter, causes pretty JSON formatting to be applied to the response entity body. Defaults to false.

  • [ access_token ] {string} Alternative method to pass the configured bearer access token for this web API. Since v6.15.

Success:

Errors:

Example request, with pretty JSON printing:

GET /monitor/v1/metrics?pretty=true HTTP/1.1
Host: c2id.com
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

Example response:

HTTP/1.1 200 Success
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: application/json

{
  "version" : "3.0.0",
  "gauges"  : {
    "authzSessionStore.numSessions"                    : { "value" : 0 },
    "authzStore.ldapConnector.maxAvailableConnections" : { "value" : 5 },
    "authzStore.ldapConnector.numAvailableConnections" : { "value" : 1 },
  ...
  }
  ...
}

3.2 /monitor/v1/metrics/prometheus

Connect2id server metrics, in Prometheus format. This resource is available since v6.4.

3.2.1 GET

Retrieves a set of metrics collected by the Connect2id server, converted to Prometheus format.

Header parameters:

  • Authorization Must specify the configured bearer access token for this web API. The token can be alternatively passed as a query parameter (not recommended), see below.

Query parameters:

  • [ access_token ] {string} Alternative method to pass the configured bearer access token for this web API. Since v6.15.

Success:

Errors:

Example request:

GET /monitor/v1/metrics/prometheus HTTP/1.1
Host: c2id.com
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

Example response:

HTTP/1.1 200 Success
Content-Type: text/plain; version=0.0.4;charset=utf-8

# HELP authzSessionStore_numSessions Generated from Dropwizard metric import (metric=authzSessionStore.numSessions, type=com.nimbusds.openid.connect.provider.authz.session.InfinispanAuthorizationSessionStore$$Lambda$295/197373476)
# TYPE authzSessionStore_numSessions gauge
authzSessionStore_numSessions 0.0
# HELP authzStore_accessTokenMap_sqlStore_pool_ActiveConnections Generated from Dropwizard metric import (metric=authzStore.accessTokenMap.sqlStore.pool.ActiveConnections, type=com.zaxxer.hikari.metrics.dropwizard.CodaHaleMetricsTracker$3)
# TYPE authzStore_accessTokenMap_sqlStore_pool_ActiveConnections gauge
authzStore_accessTokenMap_sqlStore_pool_ActiveConnections 0.0
# HELP authzStore_accessTokenMap_sqlStore_pool_IdleConnections Generated from Dropwizard metric import (metric=authzStore.accessTokenMap.sqlStore.pool.IdleConnections, type=com.zaxxer.hikari.metrics.dropwizard.CodaHaleMetricsTracker$2)
# TYPE authzStore_accessTokenMap_sqlStore_pool_IdleConnections gauge
authzStore_accessTokenMap_sqlStore_pool_IdleConnections 5.0
...

3.2 /monitor/v1/ping

Endpoint for HTTP ping requests.

3.2.1 GET

Performs an HTTP ping request to the Connect2id server.

Header parameters:

  • Authorization Must specify the configured bearer access token for this web API. The token can be alternatively passed as a query parameter (not recommended), see below.

Query parameters:

  • [ access_token ] {string} Alternative method to pass the configured bearer access token for this web API. Since v6.15.

Success:

  • Code: 200

  • Content-Type: text/plain

  • Body: {string} The string “pong”.

Errors:

Example ping request:

GET /monitor/v1/ping HTTP/1.1
Host: c2id.com
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

Example response:

HTTP/1.1 200 Success
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/plain

pong

3.3 /monitor/v1/threads

Current thread dump of the Connect2id server.

3.3.1 GET

Gets a thread dump of the Connect2id server.

Header parameters:

  • Authorization Must specify the configured bearer access token for this web API. The token can be alternatively passed as a query parameter (not recommended), see below.

Query parameters:

  • [ pretty = false ] {true|false} Optional query parameter, causes pretty JSON formatting to be applied to the response entity body. Defaults to false.

  • [ access_token ] {string} Alternative method to pass the configured bearer access token for this web API. Since v6.15.

Success:

  • Code: 200

  • Content-Type: text/plain

  • Body: {text} The obtained Java thread dump.

Errors:

Example request:

GET /monitor/v1/threads HTTP/1.1
Host: c2id.com
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

Example response:

HTTP/1.1 200 Success
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/plain

main id=1 state=RUNNABLE (running in native)
    at java.net.PlainSocketImpl.socketAccept(Native Method)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
    at java.net.ServerSocket.implAccept(ServerSocket.java:530)
    at java.net.ServerSocket.accept(ServerSocket.java:498)
    at org.apache.catalina.core.StandardServer.await(StandardServer.java:446)
    at org.apache.catalina.startup.Catalina.await(Catalina.java:713)
    at org.apache.catalina.startup.Catalina.start(Catalina.java:659)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:351)
    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:485)

Reference Handler id=2 state=WAITING
    - waiting on <0x41febc07> (a java.lang.ref.Reference$Lock)
    - locked <0x41febc07> (a java.lang.ref.Reference$Lock)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:503)
    at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)

...

3.4 /monitor/v1/healthcheck

Health checks built into the Connect2id server.

3.4.1 GET

Runs built-in health checks and returns the result.

  • Authorization Must specify the configured bearer access token for this web API. The token can be alternatively passed as a query parameter (not recommended), see below.

Query parameters:

  • [ pretty = false ] {true|false} Optional query parameter, causes pretty JSON formatting to be applied to the response entity body. Defaults to false.

  • [ access_token ] {string} Alternative method to pass the configured bearer access token for this web API. Since v6.15.

Success:

  • Code: 200

  • Content-Type: application/json

  • Body: {object} A JSON object representing a successful health check result.

Errors:

Example request, with pretty JSON printing:

GET /monitor/v1/healthcheck?pretty=true HTTP/1.1
Host: c2id.com
Authorization: Bearer ztucZS1ZyFKgh0tUEruUtiSTXhnexmd6

Example response indicating good health:

HTTP/1.1 200 Success
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: application/json

{
  "clientStore.ldapConnector" : { "healthy" : true },
  "authzStore.ldapConnector"  : { "healthy" : true }
}

Example response indicating the backend LDAP server is down or unavailable:

{
  "authzStore.ldapConnector"  : { "healthy" : false,
                                  "message" : "An error occurred while attempting to connect to server 127.0.0.1:1389:  java.io.IOException: An error occurred while attempting to establish a connection to server /127.0.0.1:1389:  java.net.ConnectException: Connection refused" },
  "clientStore.ldapConnector" : { "healthy" : false,
                                  "message" : "An error occurred while attempting to connect to server 127.0.0.1:1389:  java.io.IOException: An error occurred while attempting to establish a connection to server /127.0.0.1:1389:  java.net.ConnectException: Connection refused" }
}

4. Representations

4.1 Core metrics set

Core Connect2id server metrics consisting of gauges, meters and timers:

  • version {string} Identifies the Dropwizard Metrics API version, set to “3.0.0”.

  • gauges {object} Gauges for tracking the count of current sessions and other objects:

    • authzSessionStore.numSessions The number of authorisation sessions. Indicates how many end-users are currently in the process of logging in.

    • sessionStore.numSessions The number of subject (end-user) sessions. Note that a user may have more than one active session.

    • sessionStore.queuedNotifications The number of queued notifications (session start, session end) to subscribers. This feature is not used by the Connect2id server at present.

    • clientStore.numRegistrations The number of OpenID Connect / OAuth 2.0 client registrations.

    • clientStore.numCachedRegistrations Deprecated in v6.1 The number of cached OpenID Connect / OAuth 2.0 client registrations. Use clientStore.numRegistrations instead.

    • clientStore.numCachedRemoteJWKSets The number of cached remote (URL referenced) client JWK sets.

    • numCachedRemoteRequestObjects The number of cached remote (URL referenced) OpenID request objects.

    • clientStore.numCachedExpendedTokens The number of cached expended initial access tokens for OpenID Connect / OAuth 2.0 client registration.

    • authzStore.numAuthzCodes The number of OAuth 2.0 authorisation codes waiting to be exchanged for an access token.

    • authzStore.numIdAccessTokens The number of currently active identifier- based access tokens.

    • authzStore.numLongLivedAuthorizations The number of (cached) long-lived authorisations.

    • authzStore.numRevocationJournalEntries The number of (cached) revocation journal entries.

    • infinispan.numClusterMembers The number of Connect2id cluster members, as seen from the queried member. If the cluster has degraded into “split brain” state, for example due to networking issues, the reported number will not be consistent across the entire cluster. Check the server logs for more information. Zero (0) if clustering is disabled or “stateless”. Since v6.15.

    • infinispan.isCoordinator Indicates if the queried cluster member is the current coordinator (true or false). false if clustering is disabled or “stateless”. Since v6.15.

  • meters {object} Event meters:

    • op.idTokenIssues Meters ID token issues.

    • authzEndpoint.successfulRequests Meters successful requests at the OAuth 2.0 authorisation endpoint.

    • authzEndpoint.failedSubjectAuthentications Meters failed subject (end-user) authentications at the OAuth 2.0 authorisation endpoint.

    • authzEndpoint.consentDenials Meters consent denials at the OAuth 2.0 authorisation endpoint.

    • directAuthzEndpoint.successfulRequests Meters successful direct authorisation requests.

    • clientStore.newRegistrations Meters new client registrations.

    • clientStore.reads Meters client registration reads.

    • clientStore.updates Meters client registration updates.

    • clientStore.deletions Meters client registration deletions.

    • authzStore.codeIssues Meters authorisation code issues.

    • authzStore.codeExchanges Meters authorisation code exchanges for an access / refresh / ID token.

    • authzStore.accessTokenIssues Meters access token issues.

    • authzStore.refreshTokenIssues Meters refresh token issues.

    • authzStore.tokenRefreshes Meters token refreshes.

    • authzStore.longLivedAuthz.newAdditions Meters addition of new long-lived (persisted) authorisations.

    • authzStore.longLivedAuthz.updates Meters updates of long-lived (persisted) authorisations.

    • authzStore.longLivedAuthz.revocations Meters revocations of long-lived (persisted) authorisations.

    • authzStore.shortLivedAuthz.newAdditions Meters addition of new short-lived (transient) authorisations.

    • sessionStore.newSessions Meters creation of new subject (end-user) sessions.

    • sessionStore.sessionRetrievals Meters retrieval of subject (end-user) sessions.

    • sessionStore.sessionAuthUpdates Meters updates of the authentication details of subject (end-user) sessions.

    • sessionStore.sessionDataUpdates Meters updates of the data attribute of subject (end-user) sessions.

    • sessionStore.sessionClosures Meters closures (or log-outs) of subject (end-user) sessions.

    • sessionStore.sessionExpirations Meters expirations of subject (end-user) sessions.

    • tokenEndpoint.code.successfulRequests Meters successful token requests with an OAuth 2.0 authorisation code grant.

    • tokenEndpoint.code.invalidClientErrors Meters failed token requests with an OAuth 2.0 authorisation code grant that produced an invalid_client error.

    • tokenEndpoint.code.unauthorizedClientErrors Meters failed token requests with an OAuth 2.0 authorisation code grant that produced an unauthorized_client error.

    • tokenEndpoint.code.invalidGrantErrors Meters failed token requests with an OAuth 2.0 authorisation code grant that produced an invalid_grant error.

    • tokenEndpoint.code.invalidScopeErrors Meters failed token requests with an OAuth 2.0 authorisation code grant that produced an invalid_scope error.

    • tokenEndpoint.code.serverErrors Meters failed token requests with an OAuth 2.0 authorisation code grant that produced an HTTP 500 error.

    • tokenEndpoint.refreshToken.successfulRequests Meters successful token requests with an OAuth 2.0 refresh token grant.

    • tokenEndpoint.refreshToken.invalidClientErrors Meters failed token requests with an OAuth 2.0 refresh token grant that produced an invalid_client error.

    • tokenEndpoint.refreshToken.unauthorizedClientErrors Meters failed token requests with an OAuth 2.0 refresh token grant that produced an unauthorized_client error.

    • tokenEndpoint.refreshToken.invalidGrantErrors Meters failed token requests with an OAuth 2.0 refresh token grant that produced an invalid_grant error.

    • tokenEndpoint.refreshToken.invalidScopeErrors Meters failed token requests with an OAuth 2.0 refresh token grant that produced an invalid_scope error.

    • tokenEndpoint.refreshToken.serverErrors Meters failed token requests with an OAuth 2.0 refresh token grant that produced an HTTP 500 error.

    • tokenEndpoint.password.successfulRequests Meters successful token requests with an OAuth 2.0 password grant.

    • tokenEndpoint.password.invalidClientErrors Meters failed token requests with an OAuth 2.0 password grant that produced an invalid_client error.

    • tokenEndpoint.password.unauthorizedClientErrors Meters failed token requests with an OAuth 2.0 password grant that produced an unauthorized_client error.

    • tokenEndpoint.password.invalidGrantErrors Meters failed token requests with an OAuth 2.0 password grant that produced an invalid_grant error.

    • tokenEndpoint.password.invalidScopeErrors Meters failed token requests with an OAuth 2.0 password grant that produced an invalid_scope error.

    • tokenEndpoint.password.serverErrors Meters failed token requests with an OAuth 2.0 password grant that produced an HTTP 500 error.

    • tokenEndpoint.clientCredentials.successfulRequests Meters successful token requests with an OAuth 2.0 client credentials grant.

    • tokenEndpoint.clientCredentials.invalidClientErrors Meters failed token requests with an OAuth 2.0 client credentials grant that produced an invalid_client error.

    • tokenEndpoint.clientCredentials.unauthorizedClientErrors Meters failed token requests with an OAuth 2.0 client credentials grant that produced an unauthorized_client error.

    • tokenEndpoint.clientCredentials.invalidGrantErrors Meters failed token requests with an OAuth 2.0 client credentials grant that produced an invalid_grant error.

    • tokenEndpoint.clientCredentials.invalidScopeErrors Meters failed token requests with an OAuth 2.0 client credentials grant that produced an invalid_scope error.

    • tokenEndpoint.clientCredentials.serverErrors Meters failed token requests with an OAuth 2.0 client credentials grant that produced an HTTP 500 error.

    • tokenEndpoint.jwtBearer.successfulRequests Meters successful token requests with an OAuth 2.0 JWT bearer assertion grant.

    • tokenEndpoint.jwtBearer.invalidClientErrors Meters failed token requests with an OAuth 2.0 JWT bearer assertion grant that produced an invalid_client error.

    • tokenEndpoint.jwtBearer.unauthorizedClientErrors Meters failed token requests with an OAuth 2.0 JWT bearer assertion grant that produced an unauthorized_client error.

    • tokenEndpoint.jwtBearer.invalidGrantErrors Meters failed token requests with an OAuth 2.0 JWT bearer assertion that produced an invalid_grant error.

    • tokenEndpoint.jwtBearer.invalidScopeErrors Meters failed token requests with an OAuth 2.0 JWT bearer assertion that produced an invalid_scope error.

    • tokenEndpoint.jwtBearer.serverErrors Meters failed token requests with an OAuth 2.0 JWT bearer assertion that produced an HTTP 500 error.

    • tokenEndpoint.saml2Bearer.successfulRequests Meters successful token requests with a SAML 2.0 bearer assertion grant.

    • tokenEndpoint.saml2Bearer.invalidClientErrors Meters failed token requests with a 2.0 SAML 2.0 bearer assertion grant that produced an invalid_client error.

    • tokenEndpoint.saml2Bearer.unauthorizedClientErrors Meters failed token requests with a SAML 2.0 bearer assertion grant that produced an unauthorized_client error.

    • tokenEndpoint.saml2Bearer.invalidGrantErrors Meters failed token requests with a SAML 2.0 bearer assertion grant that produced an invalid_grant error.

    • tokenEndpoint.saml2Bearer.invalidScopeErrors Meters failed token requests with a SAML 2.0 bearer assertion grant that produced an invalid_scope error.

    • tokenEndpoint.saml2Bearer.serverErrors Meters failed token requests with a SAML 2.0 bearer assertion grant that produced an HTTP 500 error.

    • tokenEndpoint.invalidRequests Meters failed token requests that produced an invalid_request error.

    • tokenEndpoint.unsupportedGrantRequests Meters failed token requests that produced an unsupportedGrant error.

    • tokenIntrospectionEndpoint.activeTokens Meters successful token introspection requests that returned a response for an active token.

    • tokenIntrospectionEndpoint.inactiveTokens Meters successful token introspection requests that returned a response for an inactive token.

    • tokenIntrospectionEndpoint.invalidRequests Meters failed token introspection requests that produced an HTTP 400 error.

    • tokenIntrospectionEndpoint.invalidClientErrors Meters failed token introspection requests that produced an HTTP 401 error.

    • tokenIntrospectionEndpoint.insufficientScopeErrors Meters failed token introspection requests that produced an HTTP 403 error.

    • tokenIntrospectionEndpoint.serverErrors Meters failed token introspection requests that produced an HTTP 500 error.

    • tokenRevocationEndpoint.successfulRequests Meters successful token revocation requests.

    • tokenRevocationEndpoint.invalidRequests Meters failed token revocation requests that produced an HTTP 400 error.

    • tokenRevocationEndpoint.invalidClientErrors Meters failed token revocation requests that produced an HTTP 401 error.

    • tokenRevocationEndpoint.invalidTokens Meters silently failed token revocation requests due to an invalid or expired token.

    • tokenRevocationEndpoint.serverErrors Meters failed token revocation requests that produced an HTTP 500 error.

    • userInfoEndpoint.successfulRequests Meters successful UserInfo requests.

    • userInfoEndpoint.invalidRequestErrors Meters failed UserInfo requests that produced an HTTP 400 error.

    • userInfoEndpoint.invalidTokenErrors Meters failed UserInfo requests that produced an HTTP 401 error.

    • userInfoEndpoint.insufficientScopeErrors Meters failed UserInfo requests that produced an HTTP 403 error.

    • userInfoEndpoint.serverErrors Meters failed UserInfo requests that produced an HTTP 500 error.

  • timers {object} Timers:

    • tokenEndpoint.code.handlerTimer Times processing of OAuth 2.0 authorisation code grants at the token endpoint.
    • tokenEndpoint.refreshToken.handlerTimer Times processing of OAuth 2.0 refresh token grants at the token endpoint.
    • tokenEndpoint.password.handlerTimer Times processing of OAuth 2.0 password grants at the token endpoint.
    • tokenEndpoint.clientCredentials.handlerTimer Times processing of OAuth 2.0 client credentials grants at the token endpoint.
    • tokenEndpoint.jwtBearer.handlerTimer Times processing of OAuth 2.0 JWT bearer assertion grants at the token endpoint.
    • sessionStore.expirationTimer Times purging of expired subject (end-user) sessions.
    • claimsSource.retrievalTimer Times retrieval of UserInfo and other claims from the claims source backend.

Example metrics set:

{
  "version" : "3.0.0",
  "gauges"  : { "authzStore.numAuthzCodes"               : { "value": 0 },
                "authzStore.numIdAccessTokens"           : { "value": 0 },
                "authzStore.numLongLivedAuthorizations"  : { "value": 1 },
                "authzStore.numRevocationJournalEntries" : { "value": 0 },
  ...
  }
  ...
}

4.2 LDAP store metrics

Additional metrics for each Infinispan map / cache configured with a persisting LDAP store:

  • gauges {object} Gauges:

    • [infinispan-cache-name].ldapStore.numAvailableConnections The number of connections currently available for use in the LDAP connection pool, if that information is available (else -1).
    • [infinispan-cache-name].ldapStore.maxAvailableConnections The maximum number of connections that may be available in the LDAP connection pool at any time, if that information is available (else -1).
    • [infinispan-cache-name].ldapStore.numSuccessfulConnectionAttempts The number of connections that have been successfully created for use in conjunction with the LDAP connection pool.
    • [infinispan-cache-name].ldapStore.numFailedConnectionAttempts The number of failed attempts to create a connection for use in the LDAP connection pool.
    • [infinispan-cache-name].ldapStore.numConnectionsClosedDefunct The number of connections that have been closed as defunct (i.e., they are no longer believed to be valid).
    • [infinispan-cache-name].ldapStore.numConnectionsClosedExpired The number of connections that have been closed as expired (i.e., they have been established for longer than the maximum connection age for the pool).
    • [infinispan-cache-name].ldapStore.numConnectionsClosedUnneeded The number of connections that have been closed as unneeded (i.e., they were created in response to heavy load but are no longer needed to meet the current load, or they were closed when the pool was closed).
    • [infinispan-cache-name].ldapStore.numSuccessfulCheckouts The number of successful attempts to check out a connection from the LDAP connection pool (including connections checked out for internal use by operations processed as part of the pool).
    • [infinispan-cache-name].ldapStore.numSuccessfulCheckoutsWithoutWaiting The number of successful attempts to check out a connection from the LDAP connection pool that were able to obtain an existing connection without waiting.
    • [infinispan-cache-name].ldapStore.numSuccessfulCheckoutsAfterWaiting The number of successful attempts to check out a connection from the LDAP connection pool that had to wait for a connection to become available.
    • [infinispan-cache-name].ldapStore.numSuccessfulCheckoutsNewConnection The number of successful attempts to check out a connection from the LDAP connection pool that had to create a new connection because no existing connections were available.
    • [infinispan-cache-name].ldapStore.numFailedCheckouts The number of failed attempts to check out a connection from the LDAP connection pool (including connections checked out for internal use by operations processed as part of the pool).
    • [infinispan-cache-name].ldapStore.numReleasedValid The number of times a valid, usable connection has been released back to the LDAP connection pool after being checked out (including connections checked out for internal use by operations processed within the pool).

  • timers {object} Timers:

    • [infinispan-cache-name].ldapStore.getTimer Times retrieval of entries from the LDAP directory (LDAP search operation with scope=base).
    • [infinispan-cache-name].ldapStore.searchTimer Times bulk retrieval of entries from the LDAP directory (LDAP search operations with scope=one).
    • [infinispan-cache-name].ldapStore.addTimer Times addition of entries to the LDAP directory (LDAP add operations).
    • [infinispan-cache-name].ldapStore.modifyTimer Times modification of entries in the LDAP directory (LDAP modify operations).
    • [infinispan-cache-name].ldapStore.deleteTimer Times deletion of entries from the LDAP directory (LDAP delete operations).

4.3 SQL store metrics

Additional metrics for each Infinispan map / cache configured with a persisting SQL store. The metrics are sourced from the underlying Hikari SQL connection pool:

  • gauges {object} Gauges:

    • [infinispan-cache-name].sqlStore.pool.TotalConnections A cached gauge, refreshed on demand at 1 second resolution, indicating the total number of SQL connections in the pool.

    • [infinispan-cache-name].sqlStore.pool.IdleConnections A cached gauge, refreshed on demand at 1 second resolution, indicating the number of idle SQL connections in the pool.

    • [infinispan-cache-name].sqlStore.pool.ActiveConnections A cached gauge, refreshed on demand at 1 second resolution, indicating the number of active (in-use) SQL connections in the pool.

    • [infinispan-cache-name].sqlStore.pool.PendingConnections A cached gauge, refreshed on demand at 1 second resolution, indicating the number of threads awating an SQL connection from the pool.

  • timers {object} Timers:

    • [infinispan-cache-name].sqlStore.pool.Wait Times the wait time of obtaining an SQL connection from the pool.

  • histograms {object} Histograms:

    • [infinispan-cache-name].sqlStore.pool.Usage Histogram of SQL connection usage time.

    • [infinispan-cache-name].sqlStore.pool.ConnectionCreation Histogram of SQL connection acquisition time.

4.4 Redis store metrics

Additional metrics for each Infinispan map / cache configured with a Redis store. The metrics are sourced from the underlying Jedis connection pool:

  • gauges {object} Gauges:

    • [infinispan-cache-name].redisStore.numActiveConnections The number of active Redis client connections in the pool.

    • [infinispan-cache-name].redisStore.numIdleConnections The number of idle Redis client connections in the pool.

    • [infinispan-cache-name].redisStore.numWaitingForConnection The number of threads waiting for a Redis client connection.

    • [infinispan-cache-name].redisStore.meanWaitingTimeForConnectionMs The mean time waiting to borrow a Redis client connection from the pool, in milliseconds.

    • [infinispan-cache-name].redisStore.maxWaitingTimeForConnectionMs The maximum time waiting to borrow a Redis client connection from the pool, in milliseconds.

4.5 Gauge

A gauge from a metrics set.

  • value {integer} The current gauge value.

Example gauge:

{
  "value" : 950140456
}

4.6 Meter

A meter from a metrics set.

  • count {integer} The number of metered events.
  • m1_rate {float} The one-minute exponentially-weighted moving average rate at which events have occurred since start up. This rate has the same exponential decay factor as the one-minute load average in the top Unix command.
  • m5_rate {float} The five-minute exponentially-weighted moving average rate at which events have occurred since start up. This rate has the same exponential decay factor as the five-minute load average in the top Unix command.
  • m15_rate {float} The fifteen-minute exponentially-weighted moving average rate at which events have occurred since start up. This rate has the same exponential decay factor as the fifteen-minute load average in the top Unix command.
  • mean_rate {float} The mean rate at which events have occurred since start up.
  • units = “events/second” The unit measure.

Example meter:

{
  "count"     : 2,
  "m15_rate"  : 0.00220381749348163,
  "m1_rate"   : 0.029527305437977176,
  "m5_rate"   : 0.006503044431934881,
  "mean_rate" : 4.2031348626575445E-4,
  "units"     : "events/second"
}

4.7 Timer

A timer from a metrics set.

  • count {integer} The number of timed events.
  • min {float} The lowest duration value.
  • mean {float} The arithmetic mean duration value.
  • max {float} The highest duration values.
  • p50 {float} The value at the 50th percentile in the distribution.
  • p75 {float} The value at the 75th percentile in the distribution.
  • p95 {float} The value at the 95th percentile in the distribution.
  • p98 {float} The value at the 98th percentile in the distribution.
  • p99 {float} The value at the 99th percentile in the distribution.
  • p999 {float} The value at the 99.9th percentile in the distribution.
  • stddev {float} The standard deviation of the sampled duration values.
  • m1_rate {float} The one-minute exponentially-weighted moving average rate at which events have occurred since start up. This rate has the same exponential decay factor as the one-minute load average in the top Unix command.
  • m5_rate {float} The five-minute exponentially-weighted moving average rate at which events have occurred since start up. This rate has the same exponential decay factor as the five-minute load average in the top Unix command.
  • m15_rate {float} The fifteen-minute exponentially-weighted moving average rate at which events have occurred since start up. This rate has the same exponential decay factor as the fifteen-minute load average in the top Unix command.
  • mean_rate {float} The mean rate at which events have occurred since start up.
  • duration_units = “seconds” The duration unit measure.
  • rate_units = “calls/second” The rate unit measure.

Example timer:

{
  "count"          : 6,
  "max"            : 0.01028953,
  "mean"           : 6.647419832286938E-4,
  "min"            : 6.56503E-4,
  "p50"            : 6.647430000000001E-4,
  "p75"            : 6.647430000000001E-4,
  "p95"            : 6.647430000000001E-4,
  "p98"            : 6.647430000000001E-4,
  "p99"            : 6.647430000000001E-4,
  "p999"           : 6.647430000000001E-4,
  "stddev"         : 9.152683616619665E-8,
  "m15_rate"       : 0.0016652947966130432,
  "m1_rate"        : 1.9309212342695557E-4,
  "m5_rate"        : 0.0015804743529140488,
  "mean_rate"      : 0.0015484305097827385,
  "duration_units" : "seconds",
  "rate_units"     : "calls/second"
}

4.8 Histograms

A histogram from a metrics set.

A timer from a metrics set.

  • count {integer} The number of timed events.
  • min {float} The lowest duration value.
  • mean {float} The arithmetic mean duration value.
  • max {float} The highest duration values.
  • p50 {float} The value at the 50th percentile in the distribution.
  • p75 {float} The value at the 75th percentile in the distribution.
  • p95 {float} The value at the 95th percentile in the distribution.
  • p98 {float} The value at the 98th percentile in the distribution.
  • p99 {float} The value at the 99th percentile in the distribution.
  • p999 {float} The value at the 99.9th percentile in the distribution.
  • stddev {float} The standard deviation of the sampled duration values.

Example histogram:

{
  "count": 4,
  "max": 4,
  "mean": 1.4949055507513855,
  "min": 0,
  "p50": 1,
  "p75": 4,
  "p95": 4,
  "p98": 4,
  "p99": 4,
  "p999": 4,
  "stddev": 1.5881400253003708
}

4.9 Health check results

Connect2id server health check results.

  • [infinispan-cache-name].availability {object} A JSON object representing the result of checking the availability of the given Infinispan map / cache. Reports degraded map / cache states due to a “spit brain” condition (in a replicated or distributed cluster). Since v6.15.

    • healthy {true|false} Indicates the health.

    • [ message ] {string} Diagnostic message if healthy is false.

  • [infinispan-cache-name].ldapStore {object} A JSON object representing the result of checking the health of the persisting LDAP store for the given Infinispan map / cache.

    • healthy {true|false} Indicates the health.

    • [ message ] {string} Diagnostic message if healthy is false.

  • [infinispan-cache-name].sqlStore {object} A JSON object representing the result of checking the health of the persisting SQL store for the given Infinispan map / cache.

    • healthy {true|false} Indicates the health.

    • [ message ] {string} Diagnostic message if healthy is false.

Example results indicating good health:

{
	"authzStore.accessTokenMap.availability": {
		"healthy": true
	},
	"authzStore.accessTokenMap.sqlStore.pool.ConnectivityCheck": {
		"healthy": true
	},
	"authzStore.codeMap.availability": {
		"healthy": true
	},
	"authzStore.longLivedAuthzMap.availability": {
		"healthy": true
	},
	"authzStore.longLivedAuthzMap.sqlStore.pool.ConnectivityCheck": {
		"healthy": true
	},
	"authzStore.revocationJournalMap.availability": {
		"healthy": true
	},
	"authzStore.revocationJournalMap.sqlStore.pool.ConnectivityCheck": {
		"healthy": true
	},
	"clients.registrationsMap.availability": {
		"healthy": true
	},
	"clients.registrationsMap.sqlStore.pool.ConnectivityCheck": {
		"healthy": true
	},
	"clients.remoteJWKSetCache.availability": {
		"healthy": true
	},
	"clients.remoteRequestJWTClaimsCache.availability": {
		"healthy": true
	},
	"op.authSessionMap.availability": {
		"healthy": true
	},
	"op.clientRegTokenMap.availability": {
		"healthy": true
	},
	"op.consentSessionMap.availability": {
		"healthy": true
	},
	"sessionStore.sessionMap.availability": {
		"healthy": true
	},
	"sessionStore.sessionMap.sqlStore.pool.ConnectivityCheck": {
		"healthy": true
	},
	"sessionStore.subjectMap.availability": {
		"healthy": true
	},
	"sessionStore.subjectMap.sqlStore.pool.ConnectivityCheck": {
		"healthy": true
	}
}

Example result indicating the backend LDAP server is down or unavailable:

{
  "clients.registrationsMap.ldapStore.pool.ConnectivityCheck"  :
        {
          "healthy" : false,
          "message" : "An error occurred while attempting to connect to server 127.0.0.1:1389:  java.io.IOException: An error occurred while attempting to establish a connection to server /127.0.0.1:1389:  java.net.ConnectException: Connection refused" }
        },
  ...
}

5. Errors

401 Unauthorized

The request was denied due to an invalid or missing bearer access token.

Example:

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Bearer
Content-Type: application/json

{
  "error"             : "missing_token",
  "error_description" : "Unauthorized: Missing Bearer access token"
}

500 Internal Server Error

An internal server error has occurred. Check the Connect2id server logs for details.

Example:

HTTP/1.1 500 Internal Server Error
Content-Type: application/json

{
  "error"             : "server_error",
  "error_description" : "Internal server error: Check the logs for details"
}