CR-1725 Fix for OPENDJ-893: Update admin guide section on server logs to me...

Mark Craig

22.37.2013 5f855b7663fbe23cec3a21c91cf51eebde7dd743

CR-1725 Fix for OPENDJ-893: Update admin guide section on server logs to mention HTTP

 opendj-sdk/opendj3/src/main/docbkx/admin-guide/chap-monitoring.xml

@@ -24,11 +24,11 @@
  !    
-->
<chapter xml:id='chap-monitoring'
 xmlns='http://docbook.org/ns/docbook' version='5.0' xml:lang='en'
 xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
 xsi:schemaLocation='http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd'
 xmlns:xlink='http://www.w3.org/1999/xlink'
 xmlns:xinclude='http://www.w3.org/2001/XInclude'>
         xmlns='http://docbook.org/ns/docbook' version='5.0' xml:lang='en'
         xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
         xsi:schemaLocation='http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd'
         xmlns:xlink='http://www.w3.org/1999/xlink'
        >
 <title>Monitoring, Logging, &amp; Alerts</title>
 
 <para>This chapter describes the monitoring capabilities that OpenDJ
@@ -332,6 +332,152 @@
[22/Jun/2011:12:32:24 +0200] category=BACKEND severity=NOTICE msgID=9896350
 msg=Backup task 20110622123224088 finished execution</screen>
   </listitem>

   <listitem>
    <para>If you use the HTTP Connection Handler, OpenDJ maintains a separate
    access log in <filename>logs/http-access</filename>. This access log, by
    default configured as the File Based HTTP Access Log Publisher, uses
    a different format than the LDAP access log. This HTTP access log uses
    <link xlink:href="http://www.w3.org/TR/WD-logfile.html" xlink:show="new"
    >Extended Log File Format</link> with fields described in <link
    xlink:show="new"
    xlink:href="http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/676400bc-8969-4aa7-851a-9319490a9bbb.mspx?mfr=true"
    >Microsoft's implementation</link> as well. The following default
    fields are shown here in the order they occur in the log file.</para>

    <variablelist>
     <varlistentry>
      <term><literal>cs-host</literal></term>
      <listitem>
       <para>Client host name</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>c-ip</literal></term>
      <listitem>
       <para>Client IP address</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>cs-username</literal></term>
      <listitem>
       <para>Username used to authenticate</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>x-datetime</literal></term>
      <listitem>
       <para>Completion timestamp for the HTTP request, which you can configure
        using the <literal>log-record-time-format</literal> property</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>cs-method</literal></term>
      <listitem>
       <para>HTTP method requested by the client</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>cs-uri-query</literal></term>
      <listitem>
       <para>Path and query string requested by the client</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>cs-version</literal></term>
      <listitem>
       <para>HTTP version requested by the client</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>sc-status</literal></term>
      <listitem>
       <para>HTTP status code for the operation</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>cs(User-Agent)</literal></term>
      <listitem>
       <para>User-Agent identifier</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>x-connection-id</literal></term>
      <listitem>
       <para>Connection ID used for OpenDJ internal operations</para>
       <para>When using this field to match HTTP requests with internal
       operations in the LDAP access log, first set the access log advanced
       property, <literal>suppress-internal-operations</literal>, to
       <literal>false</literal>. By default, internal operations do not appear
       in the LDAP access log.</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>x-etime</literal></term>
      <listitem>
       <para>Execution time in milliseconds needed by OpenDJ to service the
       HTTP request</para>
      </listitem>
     </varlistentry>
    </variablelist>

    <para>Missing values are replaced with <literal>-</literal>. Tabs separate
    the fields, and if a field contains a tab character, then the field is
    surrounded with double quotes. OpenDJ then doubles double quotes in the
    field to escape them.</para>

    <para>The following example shows an excerpt of an HTTP access log with
    the default configuration. Lines are folded and space reformatted for the
    printed page.</para>

    <screen>-  192.168.0.15  bjensen   22/May/2013:10:06:18 +0200
  GET  /users/bjensen?_prettyPrint=true                      HTTP/1.1    200
  curl/7.21.4  3    40
-  192.168.0.15  bjensen   22/May/2013:10:06:52 +0200
  GET  /groups/Directory%20Administrators?_prettyPrint=true  HTTP/1.1    200
  curl/7.21.4  4    41
-  192.168.0.12  bjensen   22/May/2013:10:07:07 +0200
  GET  /users/missing?_prettyPrint=true                      HTTP/1.1    200
  curl/7.21.4  5     9
-  192.168.0.12  -         22/May/2013:10:07:46 +0200
  GET  /users/missing?_prettyPrint=true                      HTTP/1.1    401
  curl/7.21.4  6     0
-  192.168.0.15  kvaughan  22/May/2013:10:09:10 +0200
  POST /users?_action=create&amp;_prettyPrint=true               HTTP/1.1    200
  curl/7.21.4  7   120</screen>

    <para>You can configure the <literal>log-format</literal> for the access log
    using the <command>dsconfig</command> command. In addition to the default
    fields, the following standard fields are supported.</para>

    <variablelist>
     <varlistentry>
      <term><literal>c-port</literal></term>
      <listitem>
       <para>Client port number</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>s-computername</literal></term>
      <listitem>
       <para>Server name where the access log was written</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>s-ip</literal></term>
      <listitem>
       <para>Server IP address</para>
      </listitem>
     </varlistentry>
     <varlistentry>
      <term><literal>s-port</literal></term>
      <listitem>
       <para>Server port number</para>
      </listitem>
     </varlistentry>
    </variablelist>
   </listitem>

   <listitem>
    <para>The <firstterm>replication log</firstterm> traces replication
    events, with entries similar to the errors log. The following excerpt has

 opendj-sdk/opendj3/src/main/docbkx/admin-guide/chap-troubleshooting.xml

@@ -24,12 +24,12 @@
  !    
-->
<chapter xml:id='chap-troubleshooting'
 xmlns='http://docbook.org/ns/docbook'
 version='5.0' xml:lang='en'
 xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
 xsi:schemaLocation='http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd'
 xmlns:xlink='http://www.w3.org/1999/xlink'
 xmlns:xinclude='http://www.w3.org/2001/XInclude'>
         xmlns='http://docbook.org/ns/docbook'
         version='5.0' xml:lang='en'
         xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
         xsi:schemaLocation='http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd'
         xmlns:xlink='http://www.w3.org/1999/xlink'
        >
 <title>Troubleshooting Server Problems</title>
 <indexterm><primary>Troubleshooting</primary></indexterm>
 
@@ -667,10 +667,11 @@
 <section xml:id="troubleshoot-connections">
  <title>Troubleshooting Client Operations</title>
 
  <para>By default OpenDJ logs information about all client operations in
  <filename>logs/access</filename>. The following lines are wrapped for
  <para>By default OpenDJ logs information about all LDAP client operations in
  <filename>logs/access</filename>, and all HTTP client operations in
  <filename>logs/http-access</filename>. The following lines are wrapped for
  readability, showing a search for the entry with
  <literal>uid=bjensen</literal> as traced in the access log. In the access
  <literal>uid=bjensen</literal> as traced in the LDAP access log. In the access
  log itself, each line starts with a time stamp.</para>
  
  <screen>[27/Jun/2011:17:23:00 +0200] CONNECT conn=19 from=127.0.0.1:56641
@@ -687,8 +688,51 @@
  performed, then including information about the connection, the operation
  number for the sequence of operations performed by the client, a message
  identification number, and additional information about the operation.</para>
  
  <para>Do help diagnose errors due to access permissions, OpenDJ supports the

  <para>To match HTTP client operations with related internal server operations,
  first prevent OpenDJ from suppressing internal operations from the LDAP access
  log by using the <command>dsconfig</command> command to set the LDAP access
  log publisher <literal>suppress-internal-operations</literal> advanced
  property to <literal>false</literal>. Then match the values of the
  <literal>x-connection-id</literal> field in the HTTP access log with
  <literal>conn=<replaceable>id</replaceable></literal> values in the LDAP
  access log.</para>

  <para>For example, consider an HTTP GET request for the <literal>_id</literal>
  field of the user <literal>newuser</literal>, which is handled by connection 4
  as shown in <filename>logs/http-access</filename>.</para>

  <screen>-  192.168.0.12  bjensen  22/May/2013:16:27:52 +0200
  GET  /users/newuser?_fields=_id  HTTP/1.1  200
  curl/7.21.4  4  12</screen>

  <para>With internal operations logged in <filename>logs/access</filename>,
  log lines for the related operations have <literal>conn=4</literal>.</para>

  <screen>[22/May/2013:16:27:52 +0200] CONNECT conn=4
  from=192.168.0.12:63593 to=192.168.0.12:8080 protocol=HTTP/1.1
[22/May/2013:16:27:52 +0200] SEARCH REQ conn=4
  op=0 msgID=0 base="ou=people,dc=example,dc=com" scope=wholeSubtree
   filter="(&amp;(objectClass=inetOrgPerson)(uid=bjensen))" attrs="1.1"
[22/May/2013:16:27:52 +0200] SEARCH RES conn=4
  op=0 msgID=0 result=0 nentries=1 etime=5
[22/May/2013:16:27:52 +0200] BIND REQ conn=4
  op=1 msgID=1 version=3 type=SIMPLE
   dn="uid=bjensen,ou=People,dc=example,dc=com"
[22/May/2013:16:27:52 +0200] BIND RES conn=4
  op=1 msgID=1 result=0 authDN="uid=bjensen,ou=People,dc=example,dc=com"
   etime=3
[22/May/2013:16:27:52 +0200] SEARCH REQ conn=4
  op=2 msgID=2 base="uid=newuser,ou=people,dc=example,dc=com" scope=baseObject
   filter="(objectClass=*)" attrs="uid,etag"
[22/May/2013:16:27:52 +0200] SEARCH RES conn=4
   op=2 msgID=2 result=0 nentries=1 etime=4
[22/May/2013:16:27:52 +0200] UNBIND REQ conn=4
   op=3 msgID=3
[22/May/2013:16:27:52 +0200] DISCONNECT conn=4
   reason="Client Unbind"</screen>

  <para>To help diagnose errors due to access permissions, OpenDJ supports the
  get effective rights control. The control OID,
  <literal>1.3.6.1.4.1.42.2.27.9.5.2</literal>, is not allowed by the default
  global ACIs. You must therefore add access to use the get effective rights