From 71571f860fb61d1d8028076c78d28cbb41d59c69 Mon Sep 17 00:00:00 2001 From: Mark Craig <mark.craig@forgerock.com> Date: Thu, 13 Oct 2011 16:04:13 +0000 Subject: [PATCH] Thanks to Danny Turner for doc review comments --- opendj-sdk/opendj3/src/main/docbkx/admin-guide/chap-import-export.xml | 462 ++++++++++++++++++++++++++++++--------------------------- 1 files changed, 242 insertions(+), 220 deletions(-) diff --git a/opendj-sdk/opendj3/src/main/docbkx/admin-guide/chap-import-export.xml b/opendj-sdk/opendj3/src/main/docbkx/admin-guide/chap-import-export.xml index 6004322..a6f8319 100644 --- a/opendj-sdk/opendj3/src/main/docbkx/admin-guide/chap-import-export.xml +++ b/opendj-sdk/opendj3/src/main/docbkx/admin-guide/chap-import-export.xml @@ -38,15 +38,15 @@ </indexterm> <indexterm><primary>Exporting data</primary></indexterm> <indexterm><primary>Backup</primary></indexterm> - <indexterm> - <primary>LDIF</primary> - <secondary>Import</secondary> - </indexterm> - <indexterm> - <primary>LDIF</primary> - <secondary>Export</secondary> - </indexterm> - + <indexterm> + <primary>LDIF</primary> + <secondary>Import</secondary> + </indexterm> + <indexterm> + <primary>LDIF</primary> + <secondary>Export</secondary> + </indexterm> + <para>LDAP Data Interchange Format provides a mechanism for representing directory data in text format. LDIF data is typically used to initialize directory databases, but also may be used to move data between different @@ -57,79 +57,249 @@ This chapter also covers creating test data in LDIF format, and manipulating LDIF data with command-line tools.</para> - <procedure xml:id="generate-ldif"> - <title>To Generate Test LDIF Data</title> - <indexterm><primary>Importing data</primary><secondary>Test data</secondary></indexterm> + <section xml:id="generating-ldif"> + <title>Generating Test Data</title> <para>When you install OpenDJ, you have the option of importing sample - data generated at installation time. This procedure demonstrates how to - generate LDIF on the command line using the - <command>make-ldif</command>.</para> + data generated during the installation. This procedure demonstrates how to + generate LDIF using the <command>make-ldif</command> command.</para> - <para>The <command>make-ldif</command> command uses templates to provide - sample data. Default templates are located in the - <filename>OpenDJ/config/MakeLDIF/</filename> directory. The - <filename>example.template</filename> file can be used to create - a suffix with <literal>inetOrgPerson</literal> entries of the type you have - the option to generate at installation time.</para> - - <step> - <para>Write a file to act as the template for your generated LDIF.</para> - <para>The resulting test data template depends on what data you expect to - encounter in production. Base your work on your knowledge of the production - data, and on the sample template, - <filename>OpenDJ/config/MakeLDIF/example.template</filename>, and - associated data.</para> - </step> - <step> - <para>Create additional data files for the content in your template to be - selected randomly from a file, rather than generated by an expression.</para> - <para>Additional data files are located in the same directory as your - template file.</para> - </step> - <step> - <para>Decide whether you want to generate the same test data each time - you run the <command>make-ldif</command> command with your template.</para> - <para>If so, pick an integer same random seed to provide each time you run - the command.</para> - </step> - <step> - <para>Before generating a very large LDIF file, make sure you have enough - space on disk.</para> - </step> - <step> - <para>Run the <command>make-ldif</command> command to generate your - LDIF file.</para> - <screen>$ make-ldif --randomSeed 0 -t /path/to/my.template -o generated.ldif + <procedure xml:id="generate-ldif"> + <title>To Generate Test LDIF Data</title> + <indexterm> + <primary>Importing data</primary> + <secondary>Test data</secondary> + </indexterm> + + <para>The <command>make-ldif</command> command uses templates to provide + sample data. Default templates are located in the + <filename>OpenDJ/config/MakeLDIF/</filename> directory. The + <filename>example.template</filename> file can be used to create + a suffix with entries of the type <literal>inetOrgPerson</literal>. You can + do the equivalent in OpenDJ Control Panel (Directory Data > New Base + DN... > Import Automatically Generated Example Data).</para> + + <step> + <para>Write a file to act as the template for your generated LDIF.</para> + <para>The resulting test data template depends on what data you expect to + encounter in production. Base your work on your knowledge of the production + data, and on the sample template, + <filename>OpenDJ/config/MakeLDIF/example.template</filename>, and + associated data.</para> + </step> + <step> + <para>Create additional data files for the content in your template to be + selected randomly from a file, rather than generated by an expression.</para> + <para>Additional data files are located in the same directory as your + template file.</para> + </step> + <step> + <para>Decide whether you want to generate the same test data each time + you run the <command>make-ldif</command> command with your template.</para> + <para>If so, provide the same <literal>randomSeed</literal> integer each + time you run the command.</para> + </step> + <step> + <para>Before generating a very large LDIF file, make sure you have enough + space on disk.</para> + </step> + <step> + <para>Run the <command>make-ldif</command> command to generate your + LDIF file.</para> + <screen>$ make-ldif --randomSeed 0 -t /path/to/my.template -o generated.ldif Processed 1000 entries Processed 2000 entries ... Processed 10000 entries LDIF processing complete. 10003 entries written</screen> + </step> + </procedure> + </section> + + <section xml:id="importing-exporting-ldif"> + <title>Importing & Exporting Data</title> + + <para>You can use the OpenDJ Control Panel to import data (Directory + Data > Import LDIF...) and to export data (Directory Data > Export + LDIF...). The following procedures demonstrate how to use the + <command>import-ldif</command> and <command>export-ldif</command> + commands.</para> + + <procedure xml:id="import-ldif"> + <title>To Import LDIF Data</title> + + <para>The most efficient method of importing LDIF data is to take the + OpenDJ server offline. Alternatively, you can schedule a task to import + the data while the server is online.</para> + + <step performance="optional"> + <para>If you do not want to use the default <literal>userRoot</literal> + backend, create a new JE backend for your data.</para> + <para>See <xref linkend="create-database-backend" /> for details.</para> + </step> + <step> + <para>The following example imports <literal>dc=example,dc=org</literal> + data into the <literal>userRoot</literal> backend, overwriting existing + data.</para> + <stepalternatives> + <step> + <para>If you want to speed up the process—for example because you + have millions of directory entries to import—first shut down the + server, and then run the <command>import-ldif</command> command.</para> + <screen>$ stop-ds +$ import-ldif -b dc=example,dc=org -n userRoot -l /path/to/generated.ldif</screen> + </step> + <step> + <para>If not, schedule a task to import the data while online.</para> + <screen>$ import-ldif + -X + -p 4444 + -h `hostname` + -D "cn=Directory Manager" + -w password + -b dc=example,dc=org + -n userRoot + -l /path/to/generated.ldif</screen> + <para>Notice that the task is scheduled through communication over SSL on + the administration port, by default <literal>4444</literal>. You can + schedule the import task to start at a particular time using the + <option>--start</option> option.</para> + <para>The <option>-X</option> option trusts all SSL certificates, such + as a default self-signed certificate used for testing.</para> + </step> + </stepalternatives> </step> </procedure> - <procedure xml:id="import-ldif"> - <title>To Import LDIF Data</title> + <procedure xml:id="export-ldif"> + <title>To Export LDIF Data</title> - <para>You can also use the OpenDJ Control Panel to import data (Directory - Data > Import LDIF...). This procedure demonstrates how to import LDIF using - the command line tool, <command>import-ldif</command>.</para> + <step> + <para>The following example exports <literal>dc=example,dc=org</literal> + data from the <literal>userRoot</literal> backend.</para> + <stepalternatives> + <step> + <para>If you want to speed up export, first shut down the server, and then + export data using the <command>export-ldif</command> command.</para> + <screen>$ stop-ds +$ export-ldif -b dc=example,dc=org -n userRoot -l /path/to/backup.ldif</screen> + </step> + <step> + <para>If not, schedule a task to export the data while online.</para> + <screen>$ export-ldif + -X + -p 4444 + -h `hostname` + -D "cn=Directory Manager" + -w password + -b dc=example,dc=org + -n userRoot + -l /path/to/backup.ldif + --start 20111221230000</screen> + <para>The <option>--start 20111221230000</option> option tells OpenDJ to + start the export at 11 PM on December 21, 2012.</para> + <para>If OpenDJ is stopped at this time, then when you start OpenDJ again, + the server attempts to perform the task after starting up.</para> + </step> + </stepalternatives> + </step> + </procedure> + </section> + + <section xml:id="ldif-tools"> + <title>Other Tools For Working With LDIF Data</title> + <indexterm> + <primary>LDIF</primary> + <secondary>Tools</secondary> + </indexterm> - <para>You can import LDIF most quickly after shutting down OpenDJ directory - server. Alternatively, if you do not want to shut the server down, you can - schedule a task to import the data while the server is online.</para> + <para>This section demonstrates the <command>ldifsearch</command>, + <command>ldifmodify</command>, and <command>ldif-diff</command> tools.</para> - <step performance="optional"> - <para>To speed up the operation for large data sets such as importing - millions of entries, first shut down the server where you plan to import - data.</para> - <screen>$ stop-ds</screen> - </step> - <step performance="optional"> - <para>Create a new JE backend for your data if you do not want to use - an existing backend, such as <literal>userRoot</literal>, and set the - base DN corresponding to your data.</para> - <screen>$ dsconfig create-backend --backend-name testData --type local-db + <section xml:id="ldifsearch-example"> + <title>Searching in LDIF With <command>ldifsearch</command></title> + + <para>The <command>ldifsearch</command> command lets you search LDIF files + in a similar way to how you search LDAP directories with the + <command>ldapsearch</command> command.</para> + + <screen>$ ldifsearch -b dc=example,dc=org -l generated.ldif "(sn=Grenier)" mobile +dn: uid=user.4630,ou=People,dc=example,dc=org +mobile: +1 728 983 6669</screen> + + <para>The <option>-l <replaceable>ldif-file</replaceable></option> option + replaces the <option>--hostname</option> and <option>--port</option> options + used to connect to an LDAP directory. Otherwise the command syntax and LDIF + output is familiar to <command>ldapsearch</command> users.</para> + </section> + + <section xml:id="ldifmodify-example"> + <title>Updating LDIF With <command>ldifmodify</command></title> + + <para>The <command>ldifmodify</command> command lets you apply changes to + LDIF files, generating a new, changed version of the original file.</para> + + <screen>$ cat changes.ldif +dn: uid=user.0,ou=People,dc=example,dc=org +changetype: modify +replace: description +description: This is the new description for Aaccf Amar. +- +replace: initials +initials: AAA + +$ ldifmodify -s generated.ldif -m changes.ldif -t new.ldif</screen> + + <para>Notice that the resulting new LDIF file is likely to be about the + same size as the source LDIF file.</para> + </section> + + <section xml:id="ldif-diff-example"> + <title>Comparing LDIF With <command>ldif-diff</command></title> + + <para>The <command>ldif-diff</command> command reports differences between + two LDIF files in LDIF format.</para> + + <screen>$ ldif-diff -s old.ldif -t new.ldif +dn: uid=user.0,ou=People,dc=example,dc=org +changetype: modify +add: initials +initials: AAA +- +delete: initials +initials: ASA +- +add: description +description: This is the new description for Aaccf Amar. +- +delete: description +description: This is the description for Aaccf Amar. + +</screen> + + <para>The <option>-o <replaceable>ldif-file</replaceable></option> option + can be used to save the output to a file.</para> + + <para>As the <command>ldif-diff</command> command reads both files into + memory, constructing tree maps to perform the comparison, the command + is designed to work with small files and fragments. The command can quickly + run out of memory when calculating differences between large files.</para> + </section> + </section> + + <section xml:id="create-database-backend"> + <title>Creating a New Database Backend</title> + <indexterm> + <primary>Database backend</primary> + <secondary>Creating</secondary> + </indexterm> + + <para>OpenDJ stores your data in a <firstterm>backend</firstterm>, which + server to store directory data, and permit backup and restore operations. + By default, OpenDJ stores your data in a backend named + <literal>userRoot</literal>. You can create new backends using the + <command>dsconfig</command> command. The following example creates a + local backend named <literal>testData</literal>.</para> + <screen>$ dsconfig create-backend --backend-name testData --type local-db >>>> Configuring the "base-dn" property @@ -189,156 +359,8 @@ Enter choice [f]: The Local DB Backend was created successfully</screen> - </step> - <step> - <para>The following example imports <literal>dc=example,dc=org</literal> - data into the existing <literal>userRoot</literal> JE backend, replacing - any data already loaded.</para> - <stepalternatives> - <step> - <para>If you stopped the server, import the data directly.</para> - <screen>$ import-ldif -b dc=example,dc=org -n userRoot -l /path/to/generated.ldif</screen> - </step> - <step> - <para>If not, schedule a task to import the data while online.</para> - <screen>$ import-ldif - -X - -p 4444 - -h `hostname` - -D "cn=Directory Manager" - -w password - -b dc=example,dc=org - -n userRoot - -l /path/to/generated.ldif</screen> - <para>Notice that the task is scheduled through communication over SSL on - the administration port, by default <literal>4444</literal>. You can - schedule the import task to start at a particular time using the - <option>--start</option> option.</para> - <para>The <option>-X</option> option trusts all SSL certificates, such - as a default self-signed certificate used for testing.</para> - </step> - </stepalternatives> - </step> - </procedure> - - <procedure xml:id="export-ldif"> - <title>To Export LDIF Data</title> - - <para>You can also use the OpenDJ Control Panel to export data (Directory - Data > Export LDIF...). This procedure demonstrates how to export LDIF using - the command line tool, <command>export-ldif</command>.</para> - - <step performance="optional"> - <para>To speed up the operation for large data sets such as exporting - millions of entries, first shut down the server.</para> - <screen>$ stop-ds</screen> - </step> - <step> - <para>The following example exports <literal>dc=example,dc=org</literal> - data from the <literal>userRoot</literal> JE backend.</para> - <stepalternatives> - <step> - <para>If you stopped the server, export the data directly.</para> - <screen>$ export-ldif -b dc=example,dc=org -n userRoot -l /path/to/backup.ldif</screen> - </step> - <step> - <para>If not, schedule a task to export the data while online.</para> - <screen>$ export-ldif - -X - -p 4444 - -h `hostname` - -D "cn=Directory Manager" - -w password - -b dc=example,dc=org - -n userRoot - -l /path/to/backup.ldif - --start 20111221230000</screen> - <para>The <option>--start 20111221230000</option> option tells OpenDJ to - start the export at 11 PM on December 21, 2012, assuming the world has - not ended yet.</para> - </step> - </stepalternatives> - </step> - </procedure> - - <section xml:id="ldif-tools"> - <title>Other Tools For Working With LDIF Data</title> - <indexterm> - <primary>LDIF</primary> - <secondary>Tools</secondary> - </indexterm> - - <para>This section demonstrates the <command>ldifsearch</command>, - <command>ldifmodify</command>, and <command>ldif-diff</command> tools.</para> - - <section xml:id="ldifsearch-example"> - <title>Searching in LDIF With <command>ldifsearch</command></title> - - <para>The <command>ldifsearch</command> command lets you search LDIF files - in a similar way to how you search LDAP directories with the - <command>ldapsearch</command> command.</para> - - <screen>$ ldifsearch -b dc=example,dc=org -l generated.ldif "(sn=Grenier)" mobile -dn: uid=user.4630,ou=People,dc=example,dc=org -mobile: +1 728 983 6669</screen> - - <para>The <option>-l <replaceable>ldif-file</replaceable></option> option - compares to the hostname and port options used to connect to an LDAP - directory. Otherwise the command syntax and LDIF output is familiar to - <command>ldapsearch</command> users.</para> - </section> - - <section xml:id="ldifmodify-example"> - <title>Updating LDIF With <command>ldifmodify</command></title> - - <para>The <command>ldifmodify</command> command lets you apply changes to - LDIF files, generating a new, changed version of the original file.</para> - - <screen>$ cat changes.ldif -dn: uid=user.0,ou=People,dc=example,dc=org -changetype: modify -replace: description -description: This is the new description for Aaccf Amar. -- -replace: initials -initials: AAA - -$ ldifmodify -s generated.ldif -m changes.ldif -t new.ldif</screen> - - <para>Notice that the resulting new LDIF file is likely to be about the - same size as the source LDIF file.</para> - </section> - - <section xml:id="ldif-diff-example"> - <title>Comparing LDIF With <command>ldif-diff</command></title> - - <para>The <command>ldif-diff</command> command reports differences between - two LDIF files in LDIF format.</para> - - <screen>$ ldif-diff -s old.ldif -t new.ldif -dn: uid=user.0,ou=People,dc=example,dc=org -changetype: modify -add: initials -initials: AAA -- -delete: initials -initials: ASA -- -add: description -description: This is the new description for Aaccf Amar. -- -delete: description -description: This is the description for Aaccf Amar. - -</screen> - - <para>The <option>-o <replaceable>ldif-file</replaceable></option> option - can be used to save the output to a file.</para> - - <para>As the <command>ldif-diff</command> command reads both files into - memory, constructing tree maps to perform the comparison, the command - is designed to work with small files and fragments. The command can quickly - run out of memory when calculating differences between large files.</para> - </section> + <para>Alternatively, you can create a new backend in OpenDJ Control Panel + (Directory Data > New Base DN... > Backend > New Backend: + <replaceable>backend-name</replaceable>).</para> </section> </chapter> -- Gitblit v1.10.0