<?xml version="1.0" encoding="UTF-8"?>
|
<!--
|
! CCPL HEADER START
|
!
|
! This work is licensed under the Creative Commons
|
! Attribution-NonCommercial-NoDerivs 3.0 Unported License.
|
! To view a copy of this license, visit
|
! http://creativecommons.org/licenses/by-nc-nd/3.0/
|
! or send a letter to Creative Commons, 444 Castro Street,
|
! Suite 900, Mountain View, California, 94041, USA.
|
!
|
! You can also obtain a copy of the license at
|
! trunk/opendj3/legal-notices/CC-BY-NC-ND.txt.
|
! See the License for the specific language governing permissions
|
! and limitations under the License.
|
!
|
! If applicable, add the following below this CCPL HEADER, with the fields
|
! enclosed by brackets "[]" replaced with your own identifying information:
|
! Portions Copyright [yyyy] [name of copyright owner]
|
!
|
! CCPL HEADER END
|
!
|
! Copyright 2011-2014 ForgeRock AS
|
!
|
-->
|
<chapter xml:id='chap-indexing'
|
xmlns='http://docbook.org/ns/docbook' version='5.0' xml:lang='en'
|
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
|
xsi:schemaLocation='http://docbook.org/ns/docbook
|
http://docbook.org/xml/5.0/xsd/docbook.xsd'
|
xmlns:xlink='http://www.w3.org/1999/xlink'>
|
<title>Indexing Attribute Values</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
</indexterm>
|
|
<para>OpenDJ provides several indexing schemes to speed up searches.</para>
|
|
<para>When a client requests a directory search operation, the client sends
|
the server a filter expression such as
|
<literal>(&(uid=*jensen*)(l=Stavanger))</literal>. The server then uses
|
applicable indexes to find entries with attribute values likely to match
|
the search. If no indexes are applicable, then the server potentially has
|
to go through all entries to look for candidate matches.</para>
|
|
<para>Looking through all entries is resource-intensive for large directories.
|
For this reason, the <literal>unindexed-search</literal> privilege, allowing
|
users to request searches for which no applicable index exists, is reserved
|
for the directory root user by default.</para>
|
|
<para>Rather than granting the <literal>unindexed-search</literal> privilege
|
to more users and client applications, you configure indexes to correspond
|
to the searches that clients need to perform. See
|
<xref linkend="debug-search-indexes" /> for details.</para>
|
|
<para>This chapter first describes index types, and demonstrates how to
|
index attribute values. This chapter also lists the default indexing
|
configuration for OpenDJ directory server.</para>
|
|
<section xml:id="indexes-overview">
|
<title>Index Types & What Each Does</title>
|
|
<para>OpenDJ provides several different index types, each corresponding
|
to a different type of search.</para>
|
|
<section xml:id="indexes-approximate">
|
<title>Approximate Index</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Approximate</secondary>
|
</indexterm>
|
|
<para>An approximate index is used to match values that "sound like" those
|
provided in the filter. An approximate index on <literal>cn</literal>
|
allows clients to find people even when they misspell names as in the
|
following example.</para>
|
|
<screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com "(cn~=Babs Jansen)" cn
|
dn: uid=bjensen,ou=People,dc=example,dc=com
|
cn: Barbara Jensen
|
cn: Babs Jensen</screen>
|
</section>
|
|
<section xml:id="indexes-equality">
|
<title>Equality Index</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Equality</secondary>
|
</indexterm>
|
|
<para>An equality index is used to match values that correspond exactly
|
(though generally without case sensitivity) to the value provided in
|
the search filter. An equality index requires clients to match values
|
without wildcards or misspellings.</para>
|
|
<screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com "(uid=bjensen)" mail
|
dn: uid=bjensen,ou=People,dc=example,dc=com
|
mail: bjensen@example.com</screen>
|
</section>
|
|
<section xml:id="indexes-ordering">
|
<title>Ordering Index</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Ordering</secondary>
|
</indexterm>
|
|
<para>An ordering index is used to match values for a filter that
|
specifies a range. The <literal>ds-sync-hist</literal> has an ordering
|
index by default because searches on that attribute often seek entries
|
with changes more recent than the last time a search was performed.</para>
|
|
<para>The following example shows a search that specifies ranges.</para>
|
|
<screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com
|
"(&(uidNumber>=1120)(roomNumber>=4500))" uid
|
dn: uid=charvey,ou=People,dc=example,dc=com
|
uid: charvey
|
|
dn: uid=eward,ou=People,dc=example,dc=com
|
uid: eward
|
|
dn: uid=mvaughan,ou=People,dc=example,dc=com
|
uid: mvaughan
|
|
dn: uid=pchassin,ou=People,dc=example,dc=com
|
uid: pchassin</screen>
|
</section>
|
|
<section xml:id="indexes-presence">
|
<title>Presence Index</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Presence</secondary>
|
</indexterm>
|
|
<para>A presence index is used to match the fact that an attribute is
|
present on the entry, regardless of the value. The <literal>aci</literal>
|
attribute is indexed for presence by default to allow quick retrieval
|
of entries with ACIs.</para>
|
|
<screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com "(aci=*)" -
|
dn: dc=example,dc=com
|
|
dn: ou=People,dc=example,dc=com</screen>
|
</section>
|
|
<section xml:id="indexes-substring">
|
<title>Substring Index</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Substring</secondary>
|
</indexterm>
|
|
<para>A substring index is used to match values specified with wildcards
|
in the filter. Substring indexes can be expensive to maintain, especially
|
for large attribute values.</para>
|
|
<screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com "(cn=Barb*)" cn
|
dn: uid=bfrancis,ou=People,dc=example,dc=com
|
cn: Barbara Francis
|
|
dn: uid=bhal2,ou=People,dc=example,dc=com
|
cn: Barbara Hall
|
|
dn: uid=bjablons,ou=People,dc=example,dc=com
|
cn: Barbara Jablonski
|
|
dn: uid=bjensen,ou=People,dc=example,dc=com
|
cn: Barbara Jensen
|
cn: Babs Jensen
|
|
dn: uid=bmaddox,ou=People,dc=example,dc=com
|
cn: Barbara Maddox</screen>
|
</section>
|
|
<section xml:id="indexes-vlv">
|
<title>Virtual List View (Browsing) Index</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Virtual list view (browsing)</secondary>
|
</indexterm>
|
|
<para>A VLV or browsing index are designed to help the server respond to
|
client applications that need virtual list view results, for example to
|
browse through a long list in a GUI. They also help the server respond
|
to clients that request server-side sorting of the search results.</para>
|
|
<para>VLV indexes correspond to particular searches. Configure your
|
VLV indexes using the Control Panel, and copy the command-line
|
equivalent from the Details pane for the operation, if necessary.</para>
|
</section>
|
|
<section xml:id="indexes-extensible">
|
<title>Extensible Matching Rule Index</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Extensible matching rule</secondary>
|
</indexterm>
|
|
<para>In some cases you need an index for a matching rule other than those
|
described above. For example, OpenDJ supports generalized time based
|
matching so applications can search for all times later than, or earlier
|
than a specified time.</para>
|
</section>
|
</section>
|
|
<section xml:id="debug-search-indexes">
|
<title>Determining What Needs Indexing</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Debugging searches</secondary>
|
</indexterm>
|
|
<para>OpenDJ search performance depends on indexes. As mentioned above,
|
unindexed searches are so resource intensive that by default OpenDJ refuses
|
to perform unindexed searches. This is because, in order to find candidate
|
matches for an unindexed search, OpenDJ has to scan the entire directory
|
database. Most searches should therefore use indexes.</para>
|
|
<para>A simple way of checking the indexes that match a search is to request
|
the <literal>debugsearchindex</literal> attribute in your results.</para>
|
|
<screen>$ ldapsearch
|
--port 1389
|
--baseDN dc=example,dc=com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
"(uid=user.1000)"
|
debugsearchindex
|
dn: cn=debugsearch
|
debugsearchindex: filter=(uid=user.1000)[INDEX:uid.equality][COUNT:1] final=[COU
|
NT:1]</screen>
|
|
<para>When you request the <literal>debugsearchindex</literal> attribute,
|
instead of performing the search, OpenDJ returns debug information indicating
|
how it would process the search operation. In the example above you notice
|
OpenDJ hits the equality index for <literal>uid</literal> right away.</para>
|
|
<para>A less exact search requires more work from OpenDJ. In the following
|
example OpenDJ would have to return over 10,000 entries.</para>
|
|
<screen>$ ldapsearch
|
--port 1389
|
--baseDN dc=example,dc=com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
"(uid=*)"
|
debugsearchindex
|
dn: cn=debugsearch
|
debugsearchindex: filter=(uid=*)[NOT-INDEXED] scope=wholeSubtree[LIMIT-EXCEEDED:
|
10002] final=[NOT-INDEXED]</screen>
|
|
<para>By default OpenDJ rejects unindexed searches when the number of
|
candidate entries goes beyond the search or look-though limit.</para>
|
|
<screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com "(uid=*)"
|
SEARCH operation failed
|
Result Code: 50 (Insufficient Access Rights)
|
Additional Information: You do not have sufficient privileges to perform
|
an unindexed search</screen>
|
|
<para>When an unindexed search is performed, it shows up in the access
|
log with the <literal>unindexed</literal> label.</para>
|
|
<programlisting language="none"
|
>...SEARCH RES ... result=50 message="You do not have sufficient privileges
|
to perform an unindexed search" nentries=0 unindexed etime=1</programlisting>
|
|
<para>If directory users tell you their client applications are getting this
|
error, then you can work with them either to help them make their search
|
filter specific enough to use existing indexes, or to index attributes they
|
need indexed in order to perform their searches. For example, if a
|
directory client application is having trouble performing a search with
|
a filters such as <literal>(objectClass=person)</literal>, you can suggest
|
that they adjust the search to be more specific, such as
|
<literal>(&(mail=username@maildomain.net)(objectClass=person))</literal>,
|
so that the server can use an index, in this case equality for mail, to
|
limit the number of candidate entries to check for matches.</para>
|
|
<para>You can view and edit what is indexed through OpenDJ Control Panel,
|
Indexes > Manage Indexes. Alternatively you can manage indexes using the
|
command-line tools demonstrated in <xref linkend="configure-indexes" />.
|
If an index already exists, but you suspect it is not working properly, see
|
<xref linkend="verify-index" />, too.</para>
|
|
<para>If you do need to allow some applications to perform unindexed searches,
|
because they need to retrieve very large numbers of entries for example, then
|
you can assign them the <literal>unindexed-search</literal> privilege. See
|
<link xlink:show="new" xlink:href="admin-guide#configure-privileges"
|
xlink:role="http://docbook.org/xlink/role/olink"><citetitle>Configuring
|
Privileges</citetitle></link> for details. A successful unindexed search also
|
shows up in the access log with the label <literal>unindexed</literal>,
|
usually with a large etime as well.</para>
|
|
<programlisting language="none"
|
>...SEARCH RES conn=11 op=1 msgID=2 result=0 nentries=10000 unindexed etime=1129</programlisting>
|
|
<para>There is a trade off between the cost of maintaining an index and the
|
value the index has in speeding up searches. Although monitoring index use
|
is not something to leave active in production due to the additional cost and
|
memory needed to maintain the statistics, in a test environment you can
|
activate index analysis using the <command>dsconfig set-backend-prop</command>
|
command.</para>
|
|
<screen>$ dsconfig
|
set-backend-prop
|
--port 4444
|
--hostname opendj.example.com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
--backend-name userRoot
|
--set index-filter-analyzer-enabled:true
|
--no-prompt
|
--trustAll</screen>
|
|
<para>The command causes OpenDJ to analyze filters used and keep the results
|
in memory, so that you can read them through the <literal>cn=monitor</literal>
|
interface.</para>
|
|
<screen>$ ldapsearch
|
--port 1389
|
--baseDN "cn=userRoot Database Environment,cn=monitor"
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
"(objectclass=*)"
|
filter-use
|
dn: cn=userRoot Database Environment,cn=monitor
|
filter-use: (mail=aa*@maildomain.net) hits:1 maxmatches:0 message:
|
filter-use: (objectClass=*) hits:1 maxmatches:-1 message:presence index type is
|
disabled for the objectClass attribute
|
filter-use: (uid=user.1000) hits:2 maxmatches:1 message:
|
filter-use: (uid=user.1001) hits:1 maxmatches:1 message:
|
filter-use: (cn=aa*) hits:1 maxmatches:10 message:
|
filter-use: (cn=b*) hits:1 maxmatches:834 message:</screen>
|
|
<para>The <literal>filter-use</literal> values consist of the filter, followed
|
by <literal>hits</literal> being the number of times the filter was used,
|
followed by <literal>maxmatches</literal> being the number of matches found
|
for the filter, followed by a message.</para>
|
|
<para>You can turn off index analysis with the <command>dsconfig
|
set-backend-prop</command> command as well.</para>
|
|
<screen>$ dsconfig
|
set-backend-prop
|
--port 4444
|
--hostname opendj.example.com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
--backend-name userRoot
|
--set index-filter-analyzer-enabled:false
|
--no-prompt
|
--trustAll</screen>
|
</section>
|
|
<section xml:id="configure-indexes">
|
<title>Configuring & Rebuilding Indexes</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Configuring</secondary>
|
</indexterm>
|
|
<para>You modify index configurations using the <command>dsconfig</command>
|
command. The configuration changes then take effect after you rebuild the
|
index according to the new configuration, using the
|
<command>rebuild-index</command>. The <command>dsconfig
|
--help-database</command> command lists subcommands for creating, reading,
|
updating, and deleting index configuration.</para>
|
|
<tip>
|
<para>Indexes are per directory backend rather than per suffix. To maintain
|
separate indexes for different suffixes on the same directory server, put
|
the suffixes in different backends.</para>
|
</tip>
|
|
<section xml:id="configure-standard-index">
|
<title>Configuring a Standard Index</title>
|
|
<para>You can configure standard indexes from the Control Panel, and also
|
on the command line using the <command>dsconfig</command> command. After
|
you finish configuring the index, you must rebuild the index for the changes
|
to take effect.</para>
|
|
<example xml:id="create-index-example">
|
<title>Create a New Index</title>
|
|
<para>The following example creates a new substring index for
|
<literal>description</literal>.</para>
|
|
<screen>$ dsconfig
|
create-local-db-index
|
--port 4444
|
--hostname opendj.example.com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
--backend-name userRoot
|
--index-name description
|
--set index-type:substring
|
--trustAll
|
--no-prompt</screen>
|
</example>
|
|
<example xml:id="approx-index-example">
|
<title>Configure an Approximate Index</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Approximate</secondary>
|
</indexterm>
|
|
<para>The following example configures an approximate index for
|
<literal>cn</literal> (common name).</para>
|
|
<screen>$ dsconfig
|
set-local-db-index-prop
|
--port 4444
|
--hostname opendj.example.com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
--backend-name userRoot
|
--index-name cn
|
--set index-type:approximate
|
--trustAll
|
--no-prompt</screen>
|
</example>
|
|
<example xml:id="extensible-match-index-example">
|
<title>Configure an Extensible Match Index</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Extensible matching rule</secondary>
|
</indexterm>
|
|
<para>The OpenDJ Control Panel New Index window does not help you set up
|
extensible matching rule indexes. Use the <command>dsconfig</command>
|
command instead.</para>
|
|
<para>The following example configures an extensible matching rule
|
index for "later than" and "earlier than" generalized time matching on
|
a <literal>lastLoginTime</literal> attribute.</para>
|
|
<screen>$ dsconfig
|
create-local-db-index
|
--port 4444
|
--hostname opendj.example.com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
--backend-name userRoot
|
--set index-type:extensible
|
--set index-extensible-matching-rule:1.3.6.1.4.1.26027.1.4.5
|
--set index-extensible-matching-rule:1.3.6.1.4.1.26027.1.4.6
|
--index-name lastLoginTime
|
--trustAll
|
--no-prompt</screen>
|
</example>
|
</section>
|
|
<section xml:id="configure-vlv">
|
<title>Configuring a Virtual List View Index</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Virtual list view (browsing)</secondary>
|
</indexterm>
|
|
<para>In the OpenDJ Control Panel, select Manage Indexes >
|
New VLV Index..., and then set up your VLV index using the New VLV
|
Index window.</para>
|
|
<mediaobject xml:id="figure-create-vlv-index">
|
<imageobject>
|
<imagedata fileref="images/create-vlv-index.png" format="PNG" />
|
</imageobject>
|
</mediaobject>
|
|
<para>After you finish configuring your index and click OK, the Control
|
Panel prompts you to make the additional changes necessary to complete the
|
VLV index configuration, and then to build the index.</para>
|
|
<para>You can also create the equivalent index configuration using the
|
<command>dsconfig</command> command.</para>
|
|
<screen>$ dsconfig
|
create-local-db-vlv-index
|
--port 4444
|
--hostname opendj.example.com
|
--bindDn "cn=Directory Manager"
|
--bindPassword password
|
--backend-name userRoot
|
--index-name people-by-last-name
|
--set base-dn:ou=People,dc=example,dc=com
|
--set filter:"(|(givenName=*)(sn=*))"
|
--set scope:single-level
|
--set sort-order:"+sn +givenName"
|
--trustAll
|
--no-prompt</screen>
|
|
<note>
|
<para>When referring to a virtual list view (VLV) index after creation, you
|
must add <literal>vlv.</literal> as a prefix. In other words, if you named
|
the VLV index <literal>people-by-last-name</literal>, you refer to it as
|
<literal>vlv.people-by-last-name</literal> when rebuilding indexes,
|
changing index properties such as the index entry limit, or verifying
|
indexes.</para>
|
</note>
|
</section>
|
|
<section xml:id="rebuild-index">
|
<title>Rebuilding Indexes</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Rebuilding</secondary>
|
</indexterm>
|
|
<para>After you change an index configuration, or when you find that
|
an index is corrupt, you can rebuild the index. When you rebuild indexes,
|
you specify the base DN of the data to index, and either the list of indexes
|
to rebuild or <option>--rebuildAll</option>. You can rebuild indexes while
|
the server is offline, or while the server is online. If you rebuild the
|
index while the server is online, then you must schedule the rebuild process
|
as a task.</para>
|
|
<example xml:id="rebuild-index-example">
|
<title>Rebuild Index</title>
|
|
<para>The following example rebuilds the <literal>cn</literal> index
|
immediately with the server online.</para>
|
|
<screen>$ rebuild-index
|
--port 4444
|
--hostname opendj.example.com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
--baseDN dc=example,dc=com
|
--index cn
|
--start 0
|
Rebuild Index task 20110607171639867 scheduled to start Jun 7, 2011 5:16:39 PM</screen>
|
</example>
|
|
<example xml:id="rebuild-degraded-indexes-example">
|
<title>Rebuild Degraded Indexes</title>
|
|
<para>The following example rebuilds degraded indexes immediately with
|
the server online.</para>
|
|
<screen>$ rebuild-index
|
--port 4444
|
--hostname opendj.example.com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
--baseDN dc=example,dc=com
|
--rebuildDegraded
|
...
|
[31/Jan/2012:16:43:25 +0100] severity="NOTICE" msgCount=7 msgID=8847510
|
message="Due to changes in the configuration, index
|
dc_example_dc_com_description is currently operating in a degraded state
|
and must be rebuilt before it can be used"
|
[31/Jan/2012:16:43:25 +0100] severity="NOTICE" msgCount=8 msgID=8847591
|
message="Rebuild of all degraded indexes started with 160 total entries
|
to process"
|
...
|
[31/Jan/2012:16:43:25 +0100] severity="NOTICE" msgCount=10 msgID=8847493
|
message="Rebuild complete. Processed 160 entries in 0 seconds (average
|
rate 1860.5/sec)"
|
...
|
Rebuild Index task 20120131164324838 has been successfully completed</screen>
|
</example>
|
|
<example xml:id="clear-degraded-indexes-example">
|
<title>Clear New, Unused, "Degraded" Indexes</title>
|
|
<para>When you add a new attribute as described in <link
|
xlink:href="admin-guide#update-schema"
|
xlink:role="http://docbook.org/xlink/role/olink"><citetitle>Updating
|
Directory Schema</citetitle></link>, and then create indexes for the new
|
attribute, the new indexes appear as degraded, even though the attribute
|
has not yet been used, and so indexes are sure to be empty, rather than
|
degraded.</para>
|
|
<para>In this special case, you can safely use the
|
<command>rebuild-index</command> command
|
<option>--clearDegradedState</option> option to avoid having to scan
|
the entire directory backend to rebuild the new, unused index. This
|
is shown in the following example, where an index has just been created
|
for <literal>newUnusedAttribute</literal>.</para>
|
|
<screen>$ dbtest
|
list-index-status
|
--backendID userRoot
|
--baseDN dc=example,dc=com
|
| grep newUnusedAttribute
|
newUnusedAttribute.equality Index ...newUnusedAttribute.equality false...
|
newUnusedAttribute.presence Index ...newUnusedAttribute.presence false...
|
newUnusedAttribute.substring Index ...newUnusedAttribute.substring false...
|
$ rebuild-index
|
--port 4444
|
--hostname opendj.example.com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
--baseDN dc=example,dc=com
|
--clearDegradedState
|
--index newUnusedAttribute
|
--start 0
|
Rebuild Index task 20130211175925012 scheduled to start Feb 11, 2013 5:59:25
|
PM CET
|
$ dbtest
|
list-index-status
|
--backendID userRoot
|
--baseDN dc=example,dc=com
|
| grep newUnusedAttribute
|
newUnusedAttribute.equality Index ...newUnusedAttribute.equality true...
|
newUnusedAttribute.presence Index ...newUnusedAttribute.presence true...
|
newUnusedAttribute.substring Index ...newUnusedAttribute.substring true...</screen>
|
|
<para>If the newly indexed attribute has already been used, rebuild indexes
|
instead.</para>
|
</example>
|
</section>
|
|
<section xml:id="index-entry-limits">
|
<title>Changing Index Entry Limits</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Entry limits</secondary>
|
</indexterm>
|
|
<para>As the number of entries in your directory grows, it can make sense
|
not to maintain indexes for particular values. For example, every entry
|
in the directory has the value <literal>top</literal> for the
|
<literal>objectClass</literal> attribute, so maintaining a list of entries
|
that match the filter <literal>(objectClass=top)</literal> is not a
|
reasonable use of resources. In a very, very large directory, the same can
|
be true for <literal>(givenName=John)</literal> and
|
<literal>(sn=Smith)</literal>.</para>
|
|
<para>In an index, each index key points to a list of entries that
|
are candidates to match. For the <literal>objectClass</literal> index key
|
that corresponds to <literal>=top</literal>, the list of entries can
|
include every entry in the directory.</para>
|
|
<para>OpenDJ directory server therefore defines an index entry limit. When
|
the number of entries that an index key points to exceeds the index entry
|
limit, OpenDJ stops maintaining the list of entries for that index key.</para>
|
|
<para>The default index entry limit value is 4000. 4000 is intended to be
|
large enough for most index keys, though it prevents OpenDJ from maintaining
|
indexes at any cost. You can use the <command>dbtest</command> command to
|
evaluate how well attributes are indexed, and consider whether to change
|
the index entry limit. Non-zero values in the "Undefined" column indicate
|
the number of index keys that have reached the limit and are no longer
|
maintained. The "Undefined keys" are then listed below.</para>
|
|
<informalexample><?dbfo pgwide="1"?>
|
<screen>$ dbtest list-index-status --backendID userRoot --baseDN dc=example,dc=com
|
Index Name Index Type JE Database Name Index Valid Record Count Undefined 95% 90% 85%
|
---------------------------------------------------------------------------------------------------------------------------------------
|
id2children Index dc_example_dc_com_id2children true 2 1 0 0 0
|
id2subtree Index dc_example_dc_com_id2subtree true 2 2 0 0 0
|
uid.equality Index dc_example_dc_com_uid.equality true 10000 0 0 0 0
|
aci.presence Index dc_example_dc_com_aci.presence true 0 0 0 0 0
|
ds-sync-conflict.equality Index dc_example_dc_com_ds-sync-conflict.equality true 0 0 0 0 0
|
givenName.equality Index dc_example_dc_com_givenName.equality true 8605 0 0 0 0
|
givenName.substring Index dc_example_dc_com_givenName.substring true 19629 0 0 0 0
|
objectClass.equality Index dc_example_dc_com_objectClass.equality true 6 4 0 0 0
|
member.equality Index dc_example_dc_com_member.equality true 0 0 0 0 0
|
uniqueMember.equality Index dc_example_dc_com_uniqueMember.equality true 0 0 0 0 0
|
cn.equality Index dc_example_dc_com_cn.equality true 10000 0 0 0 0
|
cn.substring Index dc_example_dc_com_cn.substring true 86040 0 0 0 0
|
sn.equality Index dc_example_dc_com_sn.equality true 10000 0 0 0 0
|
sn.substring Index dc_example_dc_com_sn.substring true 32217 0 0 0 0
|
telephoneNumber.equality Index dc_example_dc_com_telephoneNumber.equality true 10000 0 0 0 0
|
telephoneNumber.substring Index dc_example_dc_com_telephoneNumber.substring true 73235 0 0 0 0
|
ds-sync-hist.ordering Index dc_example_dc_com_ds-sync-hist.ordering true 0 0 0 0 0
|
mail.equality Index dc_example_dc_com_mail.equality true 10000 0 0 0 0
|
mail.substring Index dc_example_dc_com_mail.substring true 31235 15 0 0 0
|
entryUUID.equality Index dc_example_dc_com_entryUUID.equality true 10002 0 0 0 0
|
|
Total: 20
|
|
Index: objectClass.equality
|
Undefined keys: [inetorgperson] [organizationalperson] [person] [top]
|
|
Index: id2children
|
Undefined keys: [2]
|
|
Index: mail.substring
|
Undefined keys: [.net] [@maild] [aildom] [ain.ne] [domain] [et] [ildoma] [in.net] [ldomai] [maildo] [main.n] [n.net] [net] [omain.] [t]
|
|
Index: id2subtree
|
Undefined keys: [1] [2]</screen>
|
</informalexample>
|
|
<para>In this case (for a directory with only about 10,000 entries) the
|
list of undefined keys is perfectly reasonable. Every user entry has the
|
object classes listed, and every user entry has a mail address ending in
|
<literal>@maildomain.net</literal>, so those values are not specific enough
|
to be used in search filters. The <literal>id2children</literal> and
|
<literal>id2subtree</literal> are for OpenDJ's internal use.</para>
|
|
<para>
|
For an explanation of the output
|
of the <command>dbtest list-index-status</command> command, see
|
<link xlink:show="new" xlink:href="admin-guide#dbtest-1"
|
xlink:role="http://docbook.org/xlink/role/olink">dbtest(1)</link>.
|
</para>
|
|
<para>If you do find the limit is too low for a certain key, you can change
|
the index entry limit on a per index basis.</para>
|
|
<example xml:id="change-index-entry-limit">
|
<title>Change Index Entry Limit</title>
|
|
<para>The following example changes the index entry limit for the
|
<literal>objectClass</literal> index, and then rebuilds the index for the
|
configuration change to take effect. The example is contrived, but the
|
steps are the same for any other index.</para>
|
|
<important>
|
<para>Changing the index entry limit significantly can result in
|
serious performance degradation. Be prepared to test performance
|
thoroughly before you roll out an index entry limit change in
|
production.</para>
|
</important>
|
|
<screen>$ dsconfig
|
set-local-db-index-prop
|
--port 4444
|
--hostname opendj.example.com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
--backend-name userRoot
|
--index-name objectClass
|
--set index-entry-limit:5000
|
--trustAll
|
--no-prompt
|
$ rebuild-index
|
--port 4444
|
--hostname opendj.example.com
|
--bindDN "cn=Directory Manager"
|
--bindPassword password
|
--baseDN dc=example,dc=com
|
--index objectclass
|
--start 0
|
Rebuild Index task 20110607160349596 scheduled to start Jun 7, 2011 4:03:49 PM</screen>
|
</example>
|
|
<para>Alternatively, you can configure the index entry limit for all
|
indexes stored in a backend by using the <command>dsconfig
|
set-backend-prop</command> command with the <option>--backend-name
|
<replaceable>backendName</replaceable> --set
|
index-entry-limit:<replaceable>limitValue</replaceable></option>
|
options.</para>
|
</section>
|
</section>
|
|
<section xml:id="verify-index">
|
<title>Verifying Indexes</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Verifying</secondary>
|
</indexterm>
|
|
<para>You can verify that indexes correspond to current directory data,
|
and that indexes do not contain errors using the
|
<command>verify-index</command> command.</para>
|
|
<example xml:id="verify-index-example">
|
<title>Verify Index</title>
|
|
<para>The following example verifies the <literal>cn</literal> (common
|
name) index for completeness and for errors.</para>
|
|
<screen>$ verify-index
|
--baseDN dc=example,dc=com
|
--index cn
|
--clean
|
--countErrors
|
[07/Jun/2011:16:06:50 +0200] category=BACKEND severity=INFORMATION
|
msgID=9437595 msg=Local DB backend userRoot does not specify the number of
|
lock tables: defaulting to 97
|
[07/Jun/2011:16:06:50 +0200] category=BACKEND severity=INFORMATION
|
msgID=9437594 msg=Local DB backend userRoot does not specify the number of
|
cleaner threads: defaulting to 24 threads
|
[07/Jun/2011:16:06:51 +0200] category=JEB severity=NOTICE msgID=8847461
|
msg=Checked 1316 records and found 0 error(s) in 0 seconds
|
(average rate 2506.7/sec)
|
[07/Jun/2011:16:06:51 +0200] category=JEB severity=INFORMATION
|
msgID=8388710 msg=Number of records referencing more than one entry: 315
|
[07/Jun/2011:16:06:51 +0200] category=JEB severity=INFORMATION
|
msgID=8388711 msg=Number of records that exceed the entry limit: 0
|
[07/Jun/2011:16:06:51 +0200] category=JEB severity=INFORMATION
|
msgID=8388712 msg=Average number of entries referenced is 1.58/record
|
[07/Jun/2011:16:06:51 +0200] category=JEB severity=INFORMATION
|
msgID=8388713 msg=Maximum number of entries referenced by any
|
record is 32</screen>
|
|
<para>Ignore the messages regarding lock tables and cleaner threads. The
|
important information is whether any errors are found in the indexes.</para>
|
</example>
|
</section>
|
|
<section xml:id="default-indexes">
|
<title>Default Indexes</title>
|
<indexterm>
|
<primary>Indexes</primary>
|
<secondary>Default settings</secondary>
|
</indexterm>
|
|
<para>When you first install OpenDJ directory server and import your
|
data from LDIF, the following indexes are configured.</para>
|
|
<table pgwide="1" rules="none">
|
<title>Default Indexes</title>
|
<tgroup cols="7">
|
<colspec colnum="2" colname="c2" />
|
<colspec colnum="7" colname="c7" />
|
<thead>
|
<row>
|
<entry>Index</entry>
|
<entry>Approx.</entry>
|
<entry>Equality</entry>
|
<entry>Ordering</entry>
|
<entry>Presence</entry>
|
<entry>Substring</entry>
|
<entry>Entry Limit</entry>
|
</row>
|
</thead>
|
<tbody>
|
<row>
|
<entry><literal>aci</literal></entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>cn</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>dn2id</literal></entry>
|
<entry namest="c2" nameend="c7" align="center">Non-configurable
|
internal index</entry>
|
</row>
|
<row>
|
<entry><literal>ds-sync-conflict</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>ds-sync-hist</literal></entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>entryUUID</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>givenName</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>id2children</literal></entry>
|
<entry namest="c2" nameend="c7" align="center">Non-configurable
|
internal index</entry>
|
</row>
|
<row>
|
<entry><literal>id2subtree</literal></entry>
|
<entry namest="c2" nameend="c7" align="center">Non-configurable
|
internal index</entry>
|
</row>
|
<row>
|
<entry><literal>mail</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>member</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>objectClass</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>sn</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>telephone­Number</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>uid</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>4000</entry>
|
</row>
|
<row>
|
<entry><literal>unique­Member</literal></entry>
|
<entry>-</entry>
|
<entry>Yes</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>-</entry>
|
<entry>4000</entry>
|
</row>
|
</tbody>
|
</tgroup>
|
</table>
|
|
<para>When you create a JE backend using the <command>dsconfig</command>
|
command, OpenDJ creates the <literal>aci</literal> presence,
|
<literal>ds-sync-conflict</literal> equality,
|
<literal>ds-sync-hist</literal> ordering,
|
<literal>entryUUID</literal> equality, and
|
<literal>objectClass</literal> equality indexes automatically.</para>
|
</section>
|
</chapter>
|
