OPENDJ-1708 Persistit: no rebuild-index support
r11899 fixed issues with index buffers and scratch files encoding/decoding.
After that, rebuild-index was still not working.
It seems the reason was due to creating a single WriteableStorage (equivalent to DB transaction) that was then used to:
- clean the indexes to rebuild (rebuild index thread, t1)
- set index state (rebuild index thread, t2)
- rebuild indexes (executor thread, t3)
Apparently, despite all the indexes having been cleaned at time t1 (but not committed yet), executor thread at time t3 could still see the cleaned data.
Once rebuild had finished and commit occurred, the data stored during rebuild index had disappeared.
To solve this problem, I split the RebuildManager.rebuildIndexes() into several methods allowing to do commits after each step and expose data to all threads for the final on-disk merge.
Now only remains to make the process truly multi-threaded by removing each places the number of threads have been hard-coded to 1.
ReadableStorage.java:
Now implements Closeable.
Storage.java:
Added getWriteableStorage().
PersistItStorage.java:
Rename StorageImpl.release() to close().
Consequence of the changes to ReadableStorage and Storage.
TracedStorage.java:
Consequence of the changes to ReadableStorage and Storage.
Importer.java:
In rebuildIndexes(), separated the clear degraded state case from the rebuild indexes case.
In the new rebuildIndexes() method, commit the changes before pahe one and two.
In submitIndexDBWriteTasks(), each task receives its own WriteableStorage to avoid problems with transactions used over several separate threads.
In RebuildManager:
- split rebuildIndexes() into several other methods: one for clearDegradedState() and several other ones for rebuild indexes: preRebuildIndex(), throwIfCancelled(), postRebuildIndex().
- renamed phaseOne() and phaseTwo() to rebuildIndexesPhaseOne() and rebuildIndexesPhaseTwo()
- passed transaction objects as methods parameters to easily trace down their origin
In Importer, renamed phaseOne() and phaseTwo() to importPhaseOne() and importPhaseTwo().