This chapter describes the monitoring capabilities that OpenDJ implements, and shows how to configure them. Monitoring OpenDJ Control Panel provides basic monitoring capabilities under Monitoring > Connection Handler, Monitoring > Connection Handler, and Monitoring > Manage Tasks. This chapter covers the other options for monitoring OpenDJ.

OpenDJ exposes monitoring information over LDAP under the entry cn=monitor. Many different types of information are exposed. The following example shows monitoring information about the userRoot backend holding Example.com data. $ ldapsearch --port 1389 --baseDN cn=monitor "(cn=userRoot backend)" dn: cn=userRoot backend,cn=Disk Space Monitor,cn=monitor disk-state: normal objectClass: top objectClass: ds-monitor-entry objectClass: extensibleObject disk-dir: /path/to/OpenDJ/db/userRoot disk-free: 343039315968 cn: userRoot backend dn: cn=userRoot Backend,cn=monitor objectClass: top objectClass: ds-monitor-entry objectClass: ds-backend-monitor-entry ds-backend-is-private: FALSE ds-backend-writability-mode: enabled cn: userRoot Backend ds-backend-entry-count: 163 ds-backend-id: userRoot ds-base-dn-entry-count: 163 dc=example,dc=com ds-backend-base-dn: dc=example,dc=com You can set global ACIs on the Access Control Handler if you want to limit read access under cn=monitor.

SNMP OpenDJ lets you monitor the server over the Simple Network Management Protocol (SNMP), with support for the Management Information Base described in RFC 2605: Directory Server Monitoring MIB. OpenDJ SNMP-based monitoring depends on OpenDMK, which you must download separately. Install the Full Binary Bundle alongside OpenDJ. OpenDJ that you download from ForgeRock is built with OpenDMK, but OpenDMK is not part of OpenDJ, and SNMP is therefore not enabled by default. You can set up a connection handler for SNMP by enabling the connection handler, and pointing OpenDJ to your installation of the OpenDMK jdmkrt.jar library. $ dsconfig set-connection-handler-prop --port 4444 --hostname opendj.example.com --bindDN "cn=Directory Manager" --bindPassword password --handler-name "SNMP Connection Handler" --set enabled:true --set opendmk-jarfile:/path/to/OpenDMK-bin/lib/jdmkrt.jar --trustAll --no-prompt By default, the SNMP Connection Handler listens on port 161 and uses port 162 for traps. On UNIX and Linux systems, only root can normally open these ports. Therefore if you install as a normal user, you might want to change the listen and trap ports. $ dsconfig set-connection-handler-prop --port 4444 --hostname opendj.example.com --bindDN "cn=Directory Manager" --bindPassword password --handler-name "SNMP Connection Handler" --set listen-port:11161 --set trap-port:11162 --trustAll --no-prompt $ snmpwalk -v 2c -c OpenDJ@OpenDJ localhost:11161 mib-2.66.1.1.2.1 SNMPv2-SMI::mib-2.66.1.1.2.1 = STRING: "/path/to/OpenDJ"

JMX OpenDJ provides Java Management eXtensions (JMX) based monitoring. A number of tools support JMX, including jconsole and jvisualvm, which are bundled with the Sun/Oracle Java platform. JMX is not configured by default. Use the dsconfig command to configure the JMX connection handler. $ dsconfig set-connection-handler-prop --port 4444 --hostname opendj.example.com --bindDN "cn=Directory Manager" --bindPassword password --handler-name "JMX Connection Handler" --set enabled:true --trustAll --no-prompt By default, no users have privileges to access the JMX connection. The following command adds JMX privileges for Directory Manager. $ dsconfig set-root-dn-prop --port 4444 --hostname opendj.example.com --bindDN "cn=Directory Manager" --bindPassword password --add default-root-privilege-name:jmx-notify --add default-root-privilege-name:jmx-read --add default-root-privilege-name:jmx-write --trustAll --no-prompt You must also configure security to login remotely. See the section on Using SSL in Monitoring and Management Using JMX for hints. Alternatively, you can connect to a local server process by using the server process identifier. $ cat ../logs/server.pid 3363 $ jvisualvm --openpid 3363 &

OpenDJ comes with two commands for monitoring server processes and tasks. The status command displays basic information about the local server, similar to what is seen in the default window of the Control Panel. The manage-tasks command lets you manage tasks scheduled on a server, such as nightly backup. The status command takes administrative credentials to read the configuration, as does the Control Panel. $ status --bindDN "cn=Directory Manager" --bindPassword password --- Server Status --- Server Run Status: Started Open Connections: 1 --- Server Details --- Host Name: localhost Administrative Users: cn=Directory Manager Installation Path: /path/to/OpenDJ Version: OpenDJ Java Version: 1.6.0_24 Administration Connector: Port 4444 (LDAPS) --- Connection Handlers --- Address:Port : Protocol : State -------------:----------:--------- -- : LDIF : Disabled 0.0.0.0:636 : LDAPS : Disabled 0.0.0.0:1389 : LDAP : Enabled 0.0.0.0:1689 : JMX : Disabled --- Data Sources --- Base DN: dc=example,dc=com Backend ID: userRoot Entries: 163 Replication: Disabled The manage-tasks command connects over the administration port, and so can connect to both local and remote servers. $ manage-tasks --hostname opendj.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --trustAll --no-prompt ID Type Status -------------------------------------------------------- example Backup Recurring example-20110623030000000 Backup Waiting on start time

Logs Replication Log By default OpenDJ stores access and errors logs as well as a server process ID file under the logs/ directory. For the replication service, OpenDJ also keeps a replication log there. You can also configure a debug log. Furthermore, you can configure policies about how logs are rotated, and how they are retained. You configure logging using the dsconfig command. The access log traces the operations the server processes including timestamps, connection information, and information about the operation itself. The access log can therefore grow quickly, as each client request results in at least one new log message. The following access log excerpt shows a search operation from the local host, with the first three lines wrapped for readability. [21/Jun/2011:08:01:53 +0200] CONNECT conn=4 from=127.0.0.1:49708 to=127.0.0.1:1389 protocol=LDAP [21/Jun/2011:08:01:53 +0200] SEARCH REQ conn=4 op=0 msgID=1 base="dc=example,dc=com" scope=wholeSubtree filter="(uid=bjensen)" attrs="ALL" [21/Jun/2011:08:01:53 +0200] SEARCH RES conn=4 op=0 msgID=1 result=0 nentries=1 etime=3 [21/Jun/2011:08:01:53 +0200] UNBIND REQ conn=4 op=1 msgID=2 [21/Jun/2011:08:01:53 +0200] DISCONNECT conn=4 reason="Client Unbind" The errors log traces server events, error conditions, and warnings, categorized and identified by severity. The following errors log excerpt shows log entries about a backup task, with lines wrapped for readability. [22/Jun/2011:12:32:23 +0200] category=BACKEND severity=NOTICE msgID=9896349 msg=Backup task 20110622123224088 started execution [22/Jun/2011:12:32:23 +0200] category=TOOLS severity=NOTICE msgID=10944792 msg=Starting backup for backend userRoot [22/Jun/2011:12:32:24 +0200] category=JEB severity=NOTICE msgID=8847446 msg=Archived: 00000000.jdb [22/Jun/2011:12:32:24 +0200] category=TOOLS severity=NOTICE msgID=10944795 msg=The backup process completed successfully [22/Jun/2011:12:32:24 +0200] category=BACKEND severity=NOTICE msgID=9896350 msg=Backup task 20110622123224088 finished execution The replication log traces replication events, with entries similar to the errors log. The following excerpt has lines wrapped for readability. [22/Jun/2011:14:37:34 +0200] category=SYNC severity=NOTICE msgID=15139026 msg=Finished total update: exported domain "dc=example,dc=com" from this directory server DS(24065) to all remote directory servers. [22/Jun/2011:14:37:35 +0200] category=SYNC severity=MILD_WARNING msgID=14745663 msg=Replication server RS(23947) at opendj.example.com/10.10.0.168:8989 has closed the connection to this directory server DS(24065). This directory server will now try to connect to another replication server in order to receive changes for the domain "dc=example,dc=com" [22/Jun/2011:14:37:35 +0200] category=SYNC severity=NOTICE msgID=15138894 msg=The generation ID for domain "dc=example,dc=com" has been reset to 3679640 Notice that the replication log does not trace replication operations. Use the external change log instead to get notifications about changes to directory data over protocol. You can alternatively configure an audit log, which is a type of access log that dumps changes in LDIF. A debug log traces details needed to troubleshoot a problem in the server. Debug logs can grow large quickly, and therefore no debug logs are enabled by default. Each log depends on a log publisher, whose type corresponds to the type of log. OpenDJ uses file-based log publishers. The design allows for custom log publishers, however, which could publish the logs elsewhere besides a file. Each log can also be associated with a log rotation policy, and a log retention policy. The former can specify when, after how much time, or at what maximum size a log is rotated. The latter can specify a maximum number or size of logs to retain, or an amount of free disk space to maintain. The design allows for custom policies as well. For debug logging, you also set a debug target to control what gets logged. By default the file-based logs are subject to rotation and retention policies that you can list with dsconfig list-rotation-policies and dsconfig list-retention-policies.

Logs Filtering Each time a client application sends a request to OpenDJ, the server writes to its access log. As shown above, a simple search operation results in five messages written to the access log. This volume of logging gives you the information to analyze overall access patterns, or to audit access when you do not know in advance what you are looking for. Yet when you do know what you are looking for, log filtering lets you limit what the server logs, and focus on what you want to see. You define the filter criteria, and also set the filtering policy. You can filter both access and also audit logs. Log filtering lets you define rules based these criteria. Client IP address, bind DN, group membership Port number Protocol used (such as LDAP, LDAPS, JMX) Response times Result codes (only log error results, for example) Search response criteria (number of entries returned, whether the search was indexed) Target DN Type of operation (connect, bind, add, delete, modify, rename, search, etc.) The filtering policy in the log publisher configuration specifies whether to include or exclude log messages that match the criteria you define. OpenDJ does not filter logs until you update the log publisher configuration. A common development troubleshooting technique consists of sending client requests while tailing the access log: $ tail -f /path/to/OpenDJ/logs/access Trouble is, when OpenDJ Control Panel is running, or when you are also adapting your configuration using the dsconfig command, OpenDJ writes access log messages related to administration. These might prevent you from noticing the messages that interest you. This example demonstrates how to filter out access log messages due to administrative connections over LDAPS on ports 1636 and 4444. Create access log filtering criteria rules. $ dsconfig create-access-log-filtering-criteria --port 4444 --hostname opendj.example.com --bindDN "cn=Directory Manager" --bindPassword password --publisher-name "File-Based Access Logger" --criteria-name "Exclude LDAPS on 1636 and 4444" --add connection-port-equal-to:1636 --add connection-port-equal-to:4444 --set connection-protocol-equal-to:ldaps --trustAll --no-prompt Activate filtering to exclude messages from the default access log according to the criteria you specified. $ dsconfig set-log-publisher-prop --port 4444 --hostname opendj.example.com --bindDN "cn=Directory Manager" --bindPassword password --publisher-name "File-Based Access Logger" --set filtering-policy:exclusive --trustAll --no-prompt At this point, OpenDJ filters out connections over LDAPS to ports 1636 and 4444. While performing operations in OpenDJ Control Panel, if you perform a simple ldapsearch --port 1389 --baseDN dc=example,dc=com uid=bjensen cn, then all you see in the access log is the effect of the ldapsearch command. $ tail -f /path/to/OpenDJ/logs/access [19/Oct/2011:16:37:16 +0200] CONNECT conn=8 from=127.0.0.1:54165 to=127.0.0.1:1389 protocol=LDAP [19/Oct/2011:16:37:16 +0200] SEARCH REQ conn=8 op=0 msgID=1 base="dc=example,dc=com" scope=wholeSubtree filter="(uid=bjensen)" attrs="cn" [19/Oct/2011:16:37:16 +0200] SEARCH RES conn=8 op=0 msgID=1 result=0 nentries=1 etime=14 [19/Oct/2011:16:37:16 +0200] UNBIND REQ conn=8 op=1 msgID=2 [19/Oct/2011:16:37:16 +0200] DISCONNECT conn=8 reason="Client Unbind" In addition to the filtering policy, you can also adjust how OpenDJ writes log messages. By default, OpenDJ writes one log message for a request, and another for a response. You can set the log publisher property log-format to combined to have OpenDJ write a single message per operation. This can be helpful, for example, when evaluating response times. In addition, you can change the log message time stamps with log-record-time-format, and specify whether to log LDAP control OIDs for operations by setting log-control-oids to true.

Alerts OpenDJ can send alerts to provide notifications of significant server events. Yet alert notifications are not enabled by default. You can use the dsconfig command to enable alert notifications. $ dsconfig set-alert-handler-prop --port 4444 --hostname opendj.example.com --bindDN "cn=Directory Manager" --bindPassword password --handler-name "JMX Alert Handler" --set enabled:true --trustAll --no-prompt OpenDJ can also send mail over SMTP instead of JMX notifications. Before you set up the SMTP-based alert handler, you must identify an SMTP server to which OpenDJ sends messages. $ dsconfig set-global-configuration-prop --port 4444 --hostname opendj.example.com --bindDN "cn=Directory Manager" --bindPassword password --set smtp-server:smtp.example.com --trustAll --no-prompt $ dsconfig create-alert-handler --port 4444 --hostname opendj.example.com --bindDN "cn=Directory Manager" --bindPassword password --handler-name "SMTP Alert Handler" --type smtp --set enabled:true --set message-subject:"OpenDJ Alert, Type: %%alert-type%%, ID: %%alert-id%%" --set message-body:"%%alert-message%%" --set recipient-address:kvaughan@example.com --set sender-address:opendj@example.com --trustAll --no-prompt