* Every other tree of a backend is named from {@link EntryContainer#getTreePrefix()} and so * carries its base DN, but these two belong to the backend rather than to any one of its base * DNs and were named from a literal. That is harmless where a storage holds a single backend - * JE and PDB give each its own directory, and the Cassandra backend names its table after the * backend id - but the JDBC backend derives its table name from the tree name alone, so two * JDBC backends addressing one database met in this pair of tables. Each allocated tokens from * the size of its own in-memory map under its own lock, so both handed out the same token for * different attribute descriptions and overwrote each other's definitions; after a restart the * entries of the losing backend decoded as the wrong attributes, silently (issue #873). *
* Kept only to be read: {@link #load} migrates what it finds here into the backend's own trees * and never writes to it again. */ private static final String LEGACY_TREE_PREFIX = "compressed_schema"; private static final TreeName LEGACY_AD_TREE_NAME = new TreeName(LEGACY_TREE_PREFIX, DB_NAME_AD); private static final TreeName LEGACY_OC_TREE_NAME = new TreeName(LEGACY_TREE_PREFIX, DB_NAME_OC); /** The compressed attribute description schema tree of this backend. */ private final TreeName adTreeName; /** The compressed object class set schema tree of this backend. */ private final TreeName ocTreeName; /** The id of the backend these definitions belong to. */ private final String backendId; /** The storage in which the trees are held. */ private final Storage storage; /** How many definitions the migration copied, reported by {@link #reportMigration()} once committed. */ private long migratedDefinitions; private final ByteStringBuilder storeAttributeWriterBuffer = new ByteStringBuilder(); private final ASN1Writer storeAttributeWriter = ASN1.getWriter(storeAttributeWriterBuffer); private final ByteStringBuilder storeObjectClassesWriterBuffer = new ByteStringBuilder(); private final ASN1Writer storeObjectClassesWriter = ASN1.getWriter(storeObjectClassesWriterBuffer); /** * Creates a new instance of this compressed schema manager. * * @param serverContext * The server context. * @param backendId * The id of the backend whose definitions are held, which qualifies the trees so that * two backends sharing one database do not share one token space. * @param storage * A reference to the storage in which the trees will be held. * @param txn a non null transaction * @param accessMode specifies how the storage has been opened (read only or read/write) * * @throws StorageRuntimeException * If a problem occurs while loading the compressed schema * definitions from the tree. * @throws InitializationException * If an error occurs while loading and processing the compressed * schema definitions. */ PersistentCompressedSchema(ServerContext serverContext, String backendId, final Storage storage, WriteableTransaction txn, AccessMode accessMode) throws StorageRuntimeException, InitializationException { super(serverContext); this.storage = storage; this.backendId = backendId; final String treePrefix = treePrefix(backendId); this.adTreeName = new TreeName(treePrefix, DB_NAME_AD); this.ocTreeName = new TreeName(treePrefix, DB_NAME_OC); load(txn, accessMode.isWriteable()); } /** * Qualifies the legacy prefix with the backend id. {@link TreeName} documents that it assumes no * name component contains a '/', and {@code TreeName.valueOf} splits the string form at the last * one, so a prefix carrying a '/' would come back as a different name than it went in as. * Nothing rules a '/' out of a backend id - ds-cfg-backend-id is a plain string - so it is * escaped rather than passed through. The escape is reversible, so two distinct backend ids can * never produce one prefix. *
* The qualifier is the backend id rather than a base DN because this object is built once per
* {@link RootContainer}, before any entry container exists, and a backend holding two base DNs
* has no single prefix to borrow. The price is that these two trees, alone among the trees of a
* backend, do not follow the entries when a JDBC backend is deleted and re-created under another
* id over the same database: every other tree is named from its base DN and is found again,
* while these are not, leaving the token allocation to restart at zero over entries encoded with
* the definitions of the old id. Changing a backend id over populated storage needs an export
* and a re-import - as it always has on Cassandra, where the one table of a backend is named
* after the id and the entries do not survive the rename either.
*/
private static String treePrefix(String backendId)
{
return LEGACY_TREE_PREFIX + "_" + backendId.replace("%", "%25").replace("/", "%2F");
}
@Override
protected void storeAttribute(final byte[] encodedAttribute,
final String attributeName, final Iterable
* That invariant holds in one direction only. A version from before the separation resumes
* writing to the legacy pair, so after a downgrade and a second upgrade the counts can agree
* while the definitions behind them have diverged, and nothing is migrated. Downgrading across
* the separation is not a supported path - the legacy trees are left in place so that a backend
* still running an earlier version can go on reading them, not so that this one can go back.
*
* The question is settled again on every open, nothing recording that it has been answered
* before: two existence probes, and the record counts only where the legacy trees are still
* there. That is what not writing a marker of this backend's own into a database it may be
* sharing with another one costs.
*
* A backend created after the upgrade on a database that already holds legacy definitions copies
* them although it has no entries of its own. Deliberate: what it inherits is a consistent
* token-to-definition mapping and costs one copy, whereas asking instead whether its own trees
* are absent would answer "nothing to migrate" for the half-copied trees an interrupted
* migration leaves behind - the one case this method exists to catch.
*/
private boolean needsLegacyDefinitions(ReadableTransaction txn)
{
final long legacyAdCount = recordCount(txn, LEGACY_AD_TREE_NAME);
final long legacyOcCount = recordCount(txn, LEGACY_OC_TREE_NAME);
return (legacyAdCount > 0 || legacyOcCount > 0)
&& (recordCount(txn, adTreeName) < legacyAdCount || recordCount(txn, ocTreeName) < legacyOcCount);
}
/** The number of records of a tree, without asking a storage to materialize one that is absent. */
private long recordCount(ReadableTransaction txn, TreeName treeName)
{
return txn.treeExists(treeName) ? txn.getRecordCount(treeName) : 0;
}
private void migrateLegacyDefinitions(WriteableTransaction txn) throws InitializationException
{
migratedDefinitions = migrateTree(txn, LEGACY_AD_TREE_NAME, adTreeName)
+ migrateTree(txn, LEGACY_OC_TREE_NAME, ocTreeName);
}
private long migrateTree(WriteableTransaction txn, TreeName from, TreeName to) throws InitializationException
{
try
{
return copyMissingRecords(txn, from, to);
}
catch (final StorageRuntimeException e)
{
// Left with the type the storage gave it. The migration runs inside the WriteOperation of
// RootContainer.open(), and PDBStorage.write() decides by type what to do with what leaves
// it: a transaction conflict reaches its retry as a RollbackException only, so wrapping it
// here would turn a conflict that used to be replayed into a permanent failure of the open.
// Nothing is lost - a storage failure that is not a conflict still fails the open, through
// ERR_OPEN_ENV_FAIL, and leaves the trees as they were, since the transaction rolls back.
throw e;
}
catch (final Exception e)
{
logger.traceException(e);
// Deliberately fatal to the open. Loading no definitions at all would restart token
// allocation from zero and decode every entry written so far as the wrong attributes,
// without reporting anything.
throw new InitializationException(
ERR_COMPSCHEMA_CANNOT_MIGRATE.get(backendId, from, to, stackTraceToSingleLineString(e)), e);
}
}
/**
* Reports a migration that has been committed, and reports nothing where none was needed.
*
* Called by {@link RootContainer#open} once the transaction the migration ran in has committed,
* and only then: a copy is undone by a rollback - a later failure of the same transaction, or a
* conflict PersistIt replays - and this line is the only evidence the migration path emits, so
* one standing for a copy that was rolled back, or repeated once per replay, would be worse than
* none at all.
*/
void reportMigration()
{
if (migratedDefinitions > 0)
{
logger.info(NOTE_COMPSCHEMA_MIGRATED, migratedDefinitions, backendId,
LEGACY_AD_TREE_NAME, LEGACY_OC_TREE_NAME, adTreeName, ocTreeName);
}
}
/**
* Copies the records of {@code from} that {@code to} does not already hold, and returns how many
* were copied. The legacy tree is left in place: on a shared database it may still be the only
* copy a backend that has not been upgraded yet has.
*
* A key already present in {@code to} is never overwritten, which is what makes this safe to
* re-run after an interrupted migration and safe against a legacy tree that a backend of an
* earlier version is still writing to.
*/
private long copyMissingRecords(WriteableTransaction txn, TreeName from, TreeName to)
{
if (!txn.treeExists(from))
{
return 0;
}
long copied = 0;
try (Cursor