mirror of https://github.com/OpenIdentityPlatform/OpenDJ.git

boli
27.06.2007 05875933ae6929bc8e53f366ce116f2fc431fd46
refs
author boli <boli@localhost>
Friday, July 27, 2007 23:06 +0200
committer boli <boli@localhost>
Friday, July 27, 2007 23:06 +0200
commit05875933ae6929bc8e53f366ce116f2fc431fd46
tree 8d01ae2e1b2dd9146948cb43019d5ca9b2c83576 tree | zip | gz
parent 5d69e54585c4e6900bc761db41b1bdd82a318107 view | diff
These set of changes implement VLV and filter capability to OpenDS:

- A VLV index is defined by a name, base DN, search filter, search scope, sort order. A search request must match these parameters exactly to
use the VLV index.
- A VLV index made up of the entry IDs matching the definition criteria (above) and the corresponding attribute values that are part of the sort
order in the sort order. This information is broken up into blocks of sorted sets. The block size can be configured through admin framework.
Default block size is 4000. In the database, the sorted set is stored with the following format:

4 byte set size | entry IDs of 8 bytes each ... | attribute values of 16 bytes each ...

- Each sorted set is keyed by the entry ID and attribute values of the largest entry in the sorted set. A special comparator (VLVKeyComparator)
is used to sort the keys in the database in the order of the specified sort order.
- When entries are added to the VLV index, its sort values are extracted and inserted into the sorted set whose key (also the largest entry in
the set) is the smallest key that represents a entry that is greater or equal to the entry being inserted. If the sorted set exceeds the block
size, it is divided in two and stored back into the database with the new key. In this implementation, a sorted set's key is never changed after
it is created.
- On importing from LDIF, each entry's sort values and ID is written out to a intermediate file in order. These files are later merged and
inserted into the database.
- Index rebuild and verify also works with VLV indexes. The verify job ensures that all the entries stored in the VLV index is in the correct
order.
- With this implementation, once a VLV index is created, it can not be changed without a rebuild. The server will NOT warn the user if
the index
has changed offline. Until a rebuild is done, it can return incorrect results. This should be fixed later.
- Performance wise, modify, add, and delete performance will be degraded if the entry matches the indexing criteria. Searches not using the VLV
index should not see any notable performance degradation. If the block size is set too big, there is a potential that a large number of updates
will result in some JE lock timeouts since the few sorted sets are hotly contested. However, if the block size is too small, searches using the
VLV control with offsets could be slow since there are more records to look through. This area need further investigation to determine the
optimal default value.

Fix for issue 38
8 files added
21 files modified
7017 ■■■■■ changed files
opends/resource/admin/abbreviations.xsl 2 ●●● diff | view | raw | blame | history
opends/resource/config/config.ldif 5 ●●●●● diff | view | raw | blame | history
opends/resource/schema/02-config.ldif 25 ●●●●● diff | view | raw | blame | history
opends/src/admin/defn/org/opends/server/admin/std/JEBackendConfiguration.xml 16 ●●●●● diff | view | raw | blame | history
opends/src/admin/defn/org/opends/server/admin/std/VLVJEIndexConfiguration.xml 219 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/AttributeIndexBuilder.java 386 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/BackendImpl.java 10 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/EntryContainer.java 536 ●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/ImportJob.java 32 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/ImportThread.java 52 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/IndexBuilder.java 353 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/IndexRebuildThread.java 193 ●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/RebuildJob.java 65 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/SortValues.java 38 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/SortValuesSet.java 647 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/State.java 64 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/VLVIndex.java 1451 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/VLVIndexBuilder.java 324 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/VLVIndexMergeThread.java 481 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/VLVKeyComparator.java 352 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/backends/jeb/VerifyJob.java 306 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/messages/JebMessages.java 31 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/tools/VerifyIndex.java 4 ●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/types/SortKey.java 89 ●●●●● diff | view | raw | blame | history
opends/src/server/org/opends/server/types/SortOrder.java 60 ●●●●● diff | view | raw | blame | history
opends/tests/unit-tests-testng/resource/config-changes.ldif 66 ●●●●● diff | view | raw | blame | history
opends/tests/unit-tests-testng/src/server/org/opends/server/backends/jeb/TestRebuildJob.java 13 ●●●● diff | view | raw | blame | history
opends/tests/unit-tests-testng/src/server/org/opends/server/backends/jeb/TestVLVIndex.java 1105 ●●●●● diff | view | raw | blame | history
opends/tests/unit-tests-testng/src/server/org/opends/server/backends/jeb/TestVerifyJob.java 92 ●●●●● diff | view | raw | blame | history