mirror of https://github.com/OpenIdentityPlatform/OpenDJ.git
| refs | |
|---|---|
| author | Valery Kharseko <vharseko@3a-systems.ru> |
| Thursday, October 29, 2026 10:20 +0100 | |
| committer | GitHub <noreply@github.com> |
| Thursday, October 29, 2026 10:20 +0100 | |
| commit | 5055b6312ff61ef861be42bfb2827aa561bc1593 |
| tree | c5379dbe6c8dc61fc2a34982f628e97d945630bd tree | zip | gz |
| parent |
c4d91fab9e005179803a3ab978e8f2896a5a5b16
view | diff
|
[#1235] [#1236] [#1240] Replication: no mismatch warning for a peer without a generation ID, and tell the peers the one taken from a TopologyMsg or an update (#1237)
Fixes #1235
Fixes #1236
Fixes #1240
## Problem
**#1235.** A replication server resets the generation ID of a domain
that had no change to -1 once it sees no directory server left in the
topology (`ReplicationServerDomain.resetGenerationIdIfPossible()`), and
tells its peers. A peer that still holds the generation ID - its own
directory server not gone yet, or its view of the others not updated yet
- took -1 in the TopologyMsg for a different generation ID in
`receiveTopoInfoFromRS`. It logged msgID 77 at WARNING and sent an
ErrorMsg back, which the first one logged as msgID 79 at ERROR, both
asking for a re-initialization. A stop of the whole combined topology in
run
[39909337627](https://github.com/OpenIdentityPlatform/OpenDJ/actions/runs/39909337627)
logged it for `cn=schema`: two warnings on `cv-c1` and an error on each
of `cv-c0` and `cv-c3`. The handshake already leaves a peer at -1 alone
(`ReplicationServerHandler.checkGenerationId()`); only the TopologyMsg
path did not.
**#1236.** The replication server that reset takes a generation ID again
from the next TopologyMsg of a peer, through `setGenerationIdIfUnset`,
and told nobody. Its peers kept it at -1 and forwarded it no update
(`isDifferentGenerationId(rsHandler, updateMsg)`), until it sent a
TopologyMsg for some other reason. The handshake and
`changeGenerationId()` tell the peers; this path did not. Before #1235,
the warning 77 was the only trace of this in the logs.
**#1240.** The same silent take on the update path. A directory server
that connects with changes already made to a replication server without
a generation ID does not give it its generation ID at the handshake
(msgID 146). `put()` takes it from the first update of that directory
server, and told nobody either.
## Fix
In `ReplicationServerDomain`:
- `receiveTopoInfoFromRS` warns about a mismatch and sends the ErrorMsg
only for a peer which presents a generation ID (`> 0`), as
`checkGenerationId()` does. The generation ID of the peer is still
recorded, and updates are still not forwarded to a peer at -1: -1 is
also what `dsreplication pre-external-initialization` sets, so that the
replication servers take no change for the domain until it is
initialized again.
- `receiveTopoInfoFromRS` takes the generation ID only from a peer which
has directory servers, only while the replication server has none of its
own, and tells its peers when it takes it (`sendTopoInfoToAll()`).
- `put()` tells the peers when it takes the generation ID of an update.
Taking the generation ID only from a peer with directory servers is what
makes telling the peers safe. While those directory servers stay
connected, `resetGenerationIdIfPossible()` resets that generation ID
neither on the peer nor on the replication server that took it. A peer
without directory servers can reset the generation ID it presented as
soon as it receives a TopologyMsg. Two such replication servers whose
TopologyMsgs cross, one with a generation ID it has not saved and the
other at -1, would otherwise exchange it with no end: each one resets it
on the -1 of the other and takes it again from the next TopologyMsg of
the other, and each reset clears the changelog of the domain. A peer
without directory servers has no update of a directory server to forward
anyway, and the handshake still takes its generation ID.
A replication server with directory servers of its own takes no
generation ID from a TopologyMsg. With them connected,
`resetGenerationIdIfPossible()` never resets its generation ID, so it is
-1 only because a `ResetGenerationIdMsg` asked for it, such as the one
of `dsreplication pre-external-initialization`, or because they
connected with changes and gave none at the handshake (msgID 146), and
`put()` takes the one of their first update. A TopologyMsg that a peer
sent before it got the reset still presents the generation ID the reset
cleared, and it can be handled after the reset. build-maven
(ubuntu-latest, 21) of run
[40650839785](https://github.com/OpenIdentityPlatform/OpenDJ/actions/runs/40650839785)
did that: the reader of the peer had read the TopologyMsg while the
reset held the lock of the domain. The replication server took the old
generation ID from it and told the peer, which took it in turn, and the
reset was undone on both for as long as their directory servers stayed
connected. Before this PR, that take was silent, and only the
replication server that handled the TopologyMsg went back to the old
generation ID. A replication server without directory servers which
takes such a generation ID resets it again on the TopologyMsg that the
peer sends once it is reset.
The take in `put()` cannot start such an exchange. It comes from a
directory server connected to this replication server, or from a peer
that forwards an update of its own directory server, which that peer
stored first. `resetGenerationIdIfPossible()` does not reset a
generation ID while a directory server is connected, nor once the
changelog has taken an update for it.
## Tests
In `ReplicationDomainTest`, with two replication servers in the JVM:
- `peerWithoutGenerationIdIsNotWarnedAbout` (#1235): the second
replication server sends the first one, which has a replica, a
TopologyMsg with -1 and then one with another generation ID. Neither
replication server logs the warning about -1, and the warning about the
other one still comes.
- `peerWhichTakesTheGenerationIdAgainIsForwardedUpdates` (#1236): the
second replication server resets its generation ID, then takes it again
from the TopologyMsg the first one sends when a second replica connects
to it. The first one records the generation ID again, and an update of
its replica reaches the second one.
- `peerWithoutReplicaGivesNoGenerationId` (#1236): a replication server
at -1 handles a TopologyMsg of a peer without a replica, with a
generation ID, and stays at -1. The TopologyMsg is handled in the test
thread, so that what the replication server tells the peer cannot come
back before the assertion.
- `peerWithReplicaGivesItsGenerationIdRightAfterAReset` (#1236): the
second replication server holds a generation ID it has not saved. One
TopologyMsg of the first replication server, which has a replica and
another generation ID, makes it reset its own and take that one in the
same call, and the first one records it.
- `peersWithoutReplicaEndWithoutGenerationId` (#1236): two replication
servers without a replica send each other the same unsaved generation
ID. Both end at -1 and stay there for two seconds.
- `topologyMsgSentBeforeAResetDoesNotUndoIt` (#1236): two replication
servers with a replica each are reset to -1 by a
ResetGenerationIdMsg(-1), as `dsreplication pre-external-initialization`
does. The first one then handles, in the test thread, the TopologyMsg
the second one built before the reset, with its replica and the old
generation ID, and stays at -1.
- `peerWithReplicaAndWithoutGenerationIdIsNotAnswered` (#1236): after
the same reset, a replication server without a replica handles, in the
test thread, a TopologyMsg of its peer with a replica and -1. In the
next two seconds, the replica of the peer receives no TopologyMsg, and
it is still connected: a closed session receives none.
- `generationIdTakenFromAnUpdateIsToldToThePeers` (#1240): a replica
with changes connects to the second replication server, which is at -1.
Its first update makes the second replication server take its generation
ID, the first one records it, and an update of the replica of the first
one reaches the second one.
The cases that change the generation ID of a replication server next to
a peer with a replica first wait until it knows that replica, so that no
TopologyMsg of the peer is left to cross the change.
## Verification
- **Red, run directly with TestNG on the cases above, with the class
under test replaced:**
- at b41b948 (the first commit of this PR):
`peerWithoutReplicaGivesNoGenerationId` (RS(61) at 6666 instead of -1,
three runs out of three) and
`generationIdTakenFromAnUpdateIsToldToThePeers` (RS(91) still at -1 for
RS(92)).
- taking the generation ID of a peer without a replica too:
`peerWithoutReplicaGivesNoGenerationId`, three runs out of three.
- taking nothing in the call that resets:
`peerWithReplicaGivesItsGenerationIdRightAfterAReset`, RS(72) at -1.
- `setGenerationIdIfUnset` taking -1 from a peer at -1:
`peerWithReplicaAndWithoutGenerationIdIsNotAnswered`, the replica
receives 1 TopologyMsg instead of none, two runs out of two.
- taking the generation ID while the replication server has directory
servers of its own, as the PR did up to 7f3af5eb70:
`topologyMsgSentBeforeAResetDoesNotUndoIt`, RS(81) at 5555 instead of
-1, two runs out of two.
- `put()` taking the generation ID silently:
`generationIdTakenFromAnUpdateIsToldToThePeers`.
- Each mutant fails only its own case.
- **Without any fix:** `peerWithoutGenerationIdIsNotWarnedAbout` fails
on `msgID=77 ... RS(22) ... presented generation ID -1 ... RS(21) is
3333`, and `peerWhichTakesTheGenerationIdAgainIsForwardedUpdates` fails
with RS(41) still holding -1 for RS(42).
- **Green:** reactor `-Pprecommit verify -pl opendj-server-legacy -am
-Dit.test='org/opends/server/replication/**,org/opends/server/backends/ChangelogBackendTestCase'`
at this head, JDK 27: 4028 tests in 94 classes, no failure, none skipped
(1 h 11 min).
- **One-off probe, not kept as a case:** after the same reset, a
replication server without a replica handles a stale TopologyMsg of its
peer with a replica. It takes the old generation ID, the peer does not
take it back from it, and the next TopologyMsg of the peer, built after
the reset, makes it reset to -1 again.
## Not covered
- In a topology that mixes this version with an older one, a replication
server of this version still logs msgID 79 at ERROR when an older peer
warns it about its own -1. This lasts until every replication server
runs this version: the receiver cannot tell that warning apart from
others, because the generation IDs are only in the localized text.
- Updates published between the moment a replication server takes the
generation ID again and the moment its peers process its TopologyMsg are
still not forwarded to it. Closing this would mean reading the changelog
again for a peer replication server, which this PR does not do.
- A replication server at -1 next to a peer that has no directory server
but a saved generation ID no longer takes that generation ID from the
TopologyMsg of the peer, which says nothing of whether it is saved. It
takes it once a directory server connects to it or to that peer. A
directory server with changes that connects to it first gets msgID 146,
and the generation ID is taken from its first update (#1240). The same
holds for a replication server that took that generation ID at the
handshake and has not stored an update of that peer yet: the next
TopologyMsg of the peer resets it. If the directory server that connects
to it next has another generation ID, the replication server takes that
one from its first update, and it and the peer warn each other about the
mismatch (msgIDs 77 and 79) and exchange no update, where before this
change the directory server got BAD_GEN_ID. The mismatch is real either
way, since the peer holds changes of its own generation ID; only where
it is reported changes.
- A replication server whose directory servers connected with changes
while it was at -1 (msgID 146) no longer takes the generation ID of a
peer from its TopologyMsg. It takes the one of the first update of those
directory servers instead (#1240), which is the generation ID of their
data. Until that update, its peers forward it no update, as before this
PR, where it took the generation ID of the peer silently and the peers
still held -1 for it.
- Between `dsreplication pre-external-initialization` and
`post-external-initialization`, a replication server that takes the
generation ID of an update now tells its peers, so updates made in that
window are forwarded between replication servers that took the same
generation ID. This includes an update of a peer that was in flight when
the reset arrived: `put()` reads the generation ID of its source without
the lock of the domain, so it can take the generation ID from before the
reset, and a peer without directory servers of its own takes it in turn
from the TopologyMsg when this replication server has directory servers.
A directory server with an empty server state that connects to that peer
in this window then gets BAD_GEN_ID, where before this change the peer
stayed at -1 and took the generation ID of that directory server at the
handshake; the replication server that took the old one kept it to
itself. `post-external-initialization` sets the generation ID on every
replication server again.