mirror of https://github.com/OpenIdentityPlatform/OpenDJ.git

Jean-Noel Rouvignac
17.08.2015 e73561d3b0db47696c578736a50489a454ad6f9c
refs
author Jean-Noel Rouvignac <jean-noel.rouvignac@forgerock.com>
Tuesday, March 17, 2015 14:08 +0100
committer Jean-Noel Rouvignac <jean-noel.rouvignac@forgerock.com>
Tuesday, March 17, 2015 14:08 +0100
commite73561d3b0db47696c578736a50489a454ad6f9c
tree bd1f2f2f566794160d326f8d706f0f86778c0508 tree | zip | gz
parent 9ba707c46e70ec1ee32d2d89c1d951c1df27a117 view | diff
OPENDJ-1708 Persistit: no rebuild-index support

r11899 fixed issues with index buffers and scratch files encoding/decoding.
After that, rebuild-index was still not working.
It seems the reason was due to creating a single WriteableStorage (equivalent to DB transaction) that was then used to:
- clean the indexes to rebuild (rebuild index thread, t1)
- set index state (rebuild index thread, t2)
- rebuild indexes (executor thread, t3)
Apparently, despite all the indexes having been cleaned at time t1 (but not committed yet), executor thread at time t3 could still see the cleaned data.
Once rebuild had finished and commit occurred, the data stored during rebuild index had disappeared.

To solve this problem, I split the RebuildManager.rebuildIndexes() into several methods allowing to do commits after each step and expose data to all threads for the final on-disk merge.
Now only remains to make the process truly multi-threaded by removing each places the number of threads have been hard-coded to 1.



ReadableStorage.java:
Now implements Closeable.

Storage.java:
Added getWriteableStorage().

PersistItStorage.java:
Rename StorageImpl.release() to close().
Consequence of the changes to ReadableStorage and Storage.

TracedStorage.java:
Consequence of the changes to ReadableStorage and Storage.

Importer.java:
In rebuildIndexes(), separated the clear degraded state case from the rebuild indexes case.
In the new rebuildIndexes() method, commit the changes before pahe one and two.
In submitIndexDBWriteTasks(), each task receives its own WriteableStorage to avoid problems with transactions used over several separate threads.
In RebuildManager:
- split rebuildIndexes() into several other methods: one for clearDegradedState() and several other ones for rebuild indexes: preRebuildIndex(), throwIfCancelled(), postRebuildIndex().
- renamed phaseOne() and phaseTwo() to rebuildIndexesPhaseOne() and rebuildIndexesPhaseTwo()
- passed transaction objects as methods parameters to easily trace down their origin
In Importer, renamed phaseOne() and phaseTwo() to importPhaseOne() and importPhaseTwo().
5 files modified
300 ■■■■■ changed files
opendj-server-legacy/src/main/java/org/opends/server/backends/persistit/PersistItStorage.java 24 ●●●●● diff | view | raw | blame | history
opendj-server-legacy/src/main/java/org/opends/server/backends/pluggable/Importer.java 233 ●●●●● diff | view | raw | blame | history
opendj-server-legacy/src/main/java/org/opends/server/backends/pluggable/TracedStorage.java 23 ●●●●● diff | view | raw | blame | history
opendj-server-legacy/src/main/java/org/opends/server/backends/pluggable/spi/ReadableStorage.java 7 ●●●● diff | view | raw | blame | history
opendj-server-legacy/src/main/java/org/opends/server/backends/pluggable/spi/Storage.java 13 ●●●● diff | view | raw | blame | history