mirror of https://github.com/OpenIdentityPlatform/OpenDJ.git

Maxim Thomas
20 hours ago 3b20faae34218cdbb72008a3695f748b34cdff4a
refs
author Maxim Thomas <maxim.thomas@gmail.com>
Tuesday, October 20, 2026 08:45 +0200
committer GitHub <noreply@github.com>
Tuesday, October 20, 2026 08:45 +0200
commit3b20faae34218cdbb72008a3695f748b34cdff4a
tree 215a481530bf05294681115a70a5f74b99d5b307 tree | zip | gz
parent d954c4c53f3d626f2a60c86eed653569dfd907fe view | diff
[#1056] Add the RSASVE/KDF-SHA256/AESKWP key wrapping transformation for runtimes without RSA-OAEP (#1194)

Closes #1056.

## Summary

A Java runtime whose cryptography comes from a `SunPKCS11` provider
alone, such as `SunPKCS11-NSS-FIPS` on a Linux system in FIPS mode,
offers no RSA-OAEP. Its only RSA padding for key transport is PKCS#1
v1.5, which NIST SP 800-131A Rev. 2 disallows. On such a runtime the
crypto manager has no acceptable value for
`key-wrapping-transformation`.

This PR adds one: `RSASVE/KDF-SHA256/AESKWP`. It is a key wrapping
scheme of OpenDJ's own, built on the hybrid key transport of NIST SP
800-56B Rev. 2, section 9.3. It needs only `RSA/ECB/NoPadding`, SHA-256
and AES-KWP from the runtime. The scheme is opt-in only: the default
stays `RSA/ECB/OAEPWITHSHA-1ANDMGF1PADDING`, and neither `setup` nor the
server selects the new value by itself.

## The scheme

A new package-private class,
`org.opends.server.crypto.RsaSveKeyTransport`, wraps a secret key in
three steps:

| Step | Standard | What it does |
| --- | --- | --- |
| RSASVE.GENERATE | SP 800-56B Rev. 2, 7.2.1 | Draws z uniformly in [2,
n-2] by rejection sampling over nLen random bytes. Raw RSA of I2OS(z,
nLen) gives C, normalized to exactly nLen bytes. |
| One-step KDF | SP 800-56C Rev. 2, 4.1 | KWK = SHA-256(00000001 ‖ Z ‖
FixedInfo), used as an AES-256 key. |
| AES-KWP | SP 800-38F | W = KWP(KWK, secret key). Asks for `AESWrapPad`
first, the only name Bouncy Castle FIPS knows, so that Bouncy Castle
FIPS does the key wrap when it is installed first; then
`AES/KWP/NoPadding`. SunJCE and SunPKCS11 register `AESWrapPad` as an
alias of their `AES/KWP/NoPadding`. |

Before the wrap, the certificate of the receiving server is checked the
way `Cipher.init(WRAP_MODE, certificate)` checks it for RSA-OAEP: a
critical key usage without keyEncipherment is refused.

The wrapped key is `len16(C) ‖ C ‖ W`. It goes, hex-encoded, into the
fourth field of the existing `ds-cfg-symmetric-key` value
`wrappingKeyID:transformation:algorithm:hexWrappedKey`. The value format
does not change.

FixedInfo is length-prefixed (a 4-byte big-endian length before each
variable field) and binds, in order:
- the canonical scheme name as AlgorithmID;
- an empty PartyUInfo;
- the wrapping key ID (field 1 of the stored value) as PartyVInfo;
- the wrapped key's algorithm (field 3);
- L = 256.

So none of the fields stored next to the wrapped key can be changed
without the unwrap failing.

**No nonce.** Keys are wrapped for a server from its published
certificate alone, without that server taking part, so no nonce of the
receiving server enters the derivation. The scheme is therefore "RSASVE
+ one-step KDF + AES-KWP", built on section 9.3 but not the KAS1-basic
scheme to the letter. The documentation says so, and leaves to each
deployment's FIPS 140-3 assessment whether it counts as an approved key
transport method there.

**Unwrap.** It splits `len16(C) ‖ C ‖ W`. Before the private key
decrypts anything, it refuses an instance key pair without its private
key or without an RSA public key, and a C which is not on exactly nLen
bytes (RSASVE.RECOVER) or is not an integer with 1 < c < n − 1 (RSADP).
The modulus comes from the instance key certificate, because a private
key held by a PKCS#11 token does not always implement `RSAKey`. It then
raw-RSA-decrypts C and left-pads the result to nLen bytes, because
Bouncy Castle FIPS drops leading zero bytes. Last, it derives the KWK
and KWP-unwraps W. Both ciphers are asked for, and the raw RSA cipher is
initialized with the private key, before the decryption, since none of
this depends on the value: a runtime which lacks a cipher is reported
naming it, and a provider which cannot use the private key with its own
reason, not as a value which cannot be unwrapped. Every failure from the
decryption on is reported as one `InvalidKeyException`,
`RSASVE/KDF-SHA256/AESKWP cannot unwrap the key`, with the original
exception as its cause. So how a value is refused does not depend on
what the private key recovered from it. Bad keys and malformed values
are reported only as `GeneralSecurityException` subtypes, the only type
`CryptoManagerImpl` catches around the wrap and the unwrap.

## Crypto manager

`CryptoManagerImpl` routes to the new class by the transformation name,
matched case-insensitively like a JCE transformation, in three places:
- `isKeyWrappingTransformationSupported` checks that the runtime offers
the three algorithms. A missing one is refused through the existing
`ERR_CRYPTOMGR_KEY_WRAPPING_TRANSFORMATION_UNSUPPORTED`, which names the
property. The existing trial wrap that follows it then runs the whole
RSASVE wrap at configuration change and start time.
- `encodeSymmetricKeyAttribute` wraps with the scheme, passing the
certificate so that its key usage is checked.
- `decodeSymmetricKeyAttribute` unwraps by the name each stored value
records. Existing RSA-OAEP values keep decoding as before. It reads the
instance key certificate once: the instance key ID and, on the scheme's
branch, the public key both come from it.

No message and no message ordinal is added. `ConfigureDS`, the installer
and the setup warning are unchanged.

## Topology

A server without this change that meets a value wrapped with the scheme
fails in `Cipher.getInstance`. So does a server with this change on a
runtime without one of the three algorithms, such as Java 11 without
Bouncy Castle FIPS, which has no AES-KWP: since it does not use the
scheme itself, it refuses nothing. Both report
`ERR_CRYPTOMGR_DECODE_SYMMETRIC_KEY_ATTRIBUTE_DECIPHER`, which
`CryptoManagerSync` only logs; the second names the missing algorithm.
The property description and the install guide therefore say that every
server of the replication topology must run a version supporting the
scheme, on a runtime which offers the three algorithms, before any
server is switched to it.

## Documentation

- The description of `key-wrapping-transformation` in
`CryptoManagerConfiguration.xml`, shown by `dsconfig` help and the
generated configuration reference.
- The FIPS install procedure, `[#install-fips]` in `chap-install.adoc`.
It corrects what a SunPKCS11 runtime offers (`RSA/ECB/PKCS1Padding` and
`RSA/ECB/NoPadding`), describes the scheme, and uses it in the
`changes.ldif` example.

Both say that the `SunPKCS11` provider of OpenJDK 17 offers no AES-KWP,
so that the server refuses the value on an OpenJDK 17 runtime with that
provider alone, and that the provider offers it from OpenJDK 19 on
(`P11KeyWrapCipher` is absent from Temurin 17.0.7 and 17.0.11, present
in 19.0.1, 21.0.2, 24.0.2, 25.0.2 and Zulu 26).

## Tests

- `RsaSveKeyTransportTestCase` (new, 45 cases):
- round trips of five keys from 8 to 32 bytes (DES, AES-128, HmacSHA1,
DESede, AES-256), including the 20-byte HmacSHA1 key that needs KWP's
padding;
- a known-answer test pinning the stored layout, the KDF and the KWP
output, whose vector was computed independently of the implementation,
for the key wrap of both the JDK and Bouncy Castle FIPS;
- rejection of secret values outside [2, n-2];
- a secret value with a leading zero byte under Bouncy Castle FIPS;
- a 1024-bit modulus;
- another private key and changed stored fields failing the unwrap;
- malformed values refused with `InvalidKeyException`;
- encapsulated values of another length than the modulus, or outside 1 <
c < n − 1, refused before the decryption under both the JDK providers
and Bouncy Castle FIPS;
- one message for every unwrap failure after the decryption;
- a provider which drops the leading zero bytes of the encapsulated
value: the value is still stored on the modulus length;
- the support check and the unwrap naming the missing key wrap;
- the unwrap reporting the RSA cipher's own refusal of the private key,
not "cannot unwrap the key";
- an instance key pair without its private key or with a non-RSA public
key refused before any cipher;
- the key wrap served by Bouncy Castle FIPS when it is installed first,
even where the JDK offers `AES/KWP/NoPadding`.

The round trips of the five keys run under both the JDK providers and
Bouncy Castle FIPS (`AESWrapPad` only, and a raw RSA that drops leading
zeros).
- `CryptoManagerTestCase` (+8 cases):
- the scheme is accepted under both spellings;
- the scheme is refused, naming `key-wrapping-transformation` and
`AES/KWP/NoPadding`, on a runtime without AES-KWP;
- a key generated while it is configured is published under its name,
unwraps, and decrypts the data it encrypted;
- each value is unwrapped by the transformation it records: an RSA-OAEP
value while the scheme is configured, and the scheme's value, also with
its name in lower case, once RSA-OAEP is configured again;
- a secret key is re-wrapped for another instance only where its
certificate's key usage allows it: a critical key usage without
keyEncipherment is refused under both RSA-OAEP and the scheme, while one
with keyEncipherment, or one that is not critical, is accepted.

The cases which need AES-KWP run with Bouncy Castle FIPS first, so that
JDK 11 has it.
- `TestCaseUtils` gets two fixtures: `withBcFipsFirst` installs Bouncy
Castle FIPS first and puts the providers back afterwards (the existing
approved-only fixture of `CryptoManagerTestCase` now uses it), and
`withoutKeyWrap` runs an action on a runtime without AES-KWP, as Java 11
without Bouncy Castle FIPS is.
- The `CryptoManagerTestCase` case which generates a key wrapped with
the scheme deletes it afterwards, so that no later restart of the test
instance meets a value a Java 11 runtime cannot unwrap.

Local runs:

| Class | JDK 26.0.1 | JDK 11.0.25 |
| --- | --- | --- |
| `RsaSveKeyTransportTestCase` | 45/0/0/0 | 45/0/0/10 (the JDK-provider
rows and the 1024-bit case need `AES/KWP/NoPadding`, which JDK 11 lacks)
|
| `CryptoManagerTestCase` | 58/0/0/0 (50 before) | 58/0/0/0 |
| `GetSymmetricKeyExtendedOperationTestCase` | 2/0/0/0 | 2/0/0/0 |

Results are tests/failures/errors/skipped.

## Outstanding: one manual run on a SunPKCS11-only runtime

No CI cell runs a runtime whose only provider is SunPKCS11. One manual
run on a RHEL FIPS-mode OpenJDK 17+ with `SunPKCS11-NSS-FIPS` only is
still outstanding:

The runtime's `SunPKCS11` has to offer `AES/KWP/NoPadding`. In the
upstream OpenJDK sources, 17.0.7 and 17.0.11 register no AES-KWP in
`SunPKCS11`, while 19.0.1 and later do; a RHEL build may differ. On a
runtime without it, the server refuses the value at step 1 and names the
property.

1. Set `key-wrapping-transformation` to `RSASVE/KDF-SHA256/AESKWP` and
start the server. This also exercises the wrap side through the trial
wrap.
2. Make the server generate a secret key: an encrypted backup, or a
backend with confidentiality enabled.
3. Restart. Check that no `ERR_TRUSTSTORESYNC_EXCEPTION` appears and
that the data still decrypts. This is the only check of the unwrap side,
which no configuration-time check covers: a raw RSA decrypt with the
token-held private key, the KWP unwrap, and the import of the unwrapped
key.
4. Add a second server, to exercise the re-wrap for another instance.

If step 3 fails, data under that key cannot be read until the server
runs on a runtime that can unwrap it. The wrapped value stays in
`cn=secret keys`, so this is recoverable.

Also unverified without a token: wrapping a sensitive PKCS#11 secret
key, whose `getEncoded()` returns null, relies on the provider's
`C_WrapKey`.
5 files modified
2 files added
1497 ■■■■■ changed files
opendj-doc-generated-ref/src/main/asciidoc/install-guide/chap-install.adoc 6 ●●●●● diff | view | raw | blame | history
opendj-maven-plugin/src/main/resources/config/xml/org/forgerock/opendj/server/config/CryptoManagerConfiguration.xml 18 ●●●●● diff | view | raw | blame | history
opendj-server-legacy/src/main/java/org/opends/server/crypto/CryptoManagerImpl.java 69 ●●●●● diff | view | raw | blame | history
opendj-server-legacy/src/main/java/org/opends/server/crypto/RsaSveKeyTransport.java 402 ●●●●● diff | view | raw | blame | history
opendj-server-legacy/src/test/java/org/opends/server/TestCaseUtils.java 58 ●●●●● diff | view | raw | blame | history
opendj-server-legacy/src/test/java/org/opends/server/crypto/CryptoManagerTestCase.java 256 ●●●●● diff | view | raw | blame | history
opendj-server-legacy/src/test/java/org/opends/server/crypto/RsaSveKeyTransportTestCase.java 688 ●●●●● diff | view | raw | blame | history