Annotation of win32/pcre/README, revision 1.2

1.1       misha       1: README file for PCRE (Perl-compatible regular expression library)
                      2: -----------------------------------------------------------------
                      3: 
1.2     ! misha       4: The latest release of PCRE is always available in three alternative formats
        !             5: from:
1.1       misha       6: 
                      7:   ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
1.2     ! misha       8:   ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.bz2
        !             9:   ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.zip
1.1       misha      10: 
                     11: There is a mailing list for discussion about the development of PCRE at
                     12: 
                     13:   pcre-dev@exim.org
                     14: 
                     15: Please read the NEWS file if you are upgrading from a previous release.
                     16: The contents of this README file are:
                     17: 
                     18:   The PCRE APIs
                     19:   Documentation for PCRE
                     20:   Contributions by users of PCRE
                     21:   Building PCRE on non-Unix systems
                     22:   Building PCRE on Unix-like systems
                     23:   Retrieving configuration information on Unix-like systems
                     24:   Shared libraries on Unix-like systems
                     25:   Cross-compiling on Unix-like systems
                     26:   Using HP's ANSI C++ compiler (aCC)
                     27:   Making new tarballs
                     28:   Testing PCRE
                     29:   Character tables
                     30:   File manifest
                     31: 
                     32: 
                     33: The PCRE APIs
                     34: -------------
                     35: 
                     36: PCRE is written in C, and it has its own API. The distribution also includes a
                     37: set of C++ wrapper functions (see the pcrecpp man page for details), courtesy
                     38: of Google Inc.
                     39: 
                     40: In addition, there is a set of C wrapper functions that are based on the POSIX
                     41: regular expression API (see the pcreposix man page). These end up in the
                     42: library called libpcreposix. Note that this just provides a POSIX calling
                     43: interface to PCRE; the regular expressions themselves still follow Perl syntax
                     44: and semantics. The POSIX API is restricted, and does not give full access to
                     45: all of PCRE's facilities.
                     46: 
                     47: The header file for the POSIX-style functions is called pcreposix.h. The
                     48: official POSIX name is regex.h, but I did not want to risk possible problems
                     49: with existing files of that name by distributing it that way. To use PCRE with
                     50: an existing program that uses the POSIX API, pcreposix.h will have to be
                     51: renamed or pointed at by a link.
                     52: 
                     53: If you are using the POSIX interface to PCRE and there is already a POSIX regex
                     54: library installed on your system, as well as worrying about the regex.h header
                     55: file (as mentioned above), you must also take care when linking programs to
                     56: ensure that they link with PCRE's libpcreposix library. Otherwise they may pick
                     57: up the POSIX functions of the same name from the other library.
                     58: 
                     59: One way of avoiding this confusion is to compile PCRE with the addition of
                     60: -Dregcomp=PCREregcomp (and similarly for the other POSIX functions) to the
                     61: compiler flags (CFLAGS if you are using "configure" -- see below). This has the
                     62: effect of renaming the functions so that the names no longer clash. Of course,
                     63: you have to do the same thing for your applications, or write them using the
                     64: new names.
                     65: 
                     66: 
                     67: Documentation for PCRE
                     68: ----------------------
                     69: 
                     70: If you install PCRE in the normal way on a Unix-like system, you will end up
                     71: with a set of man pages whose names all start with "pcre". The one that is just
                     72: called "pcre" lists all the others. In addition to these man pages, the PCRE
                     73: documentation is supplied in two other forms:
                     74: 
                     75:   1. There are files called doc/pcre.txt, doc/pcregrep.txt, and
                     76:      doc/pcretest.txt in the source distribution. The first of these is a
                     77:      concatenation of the text forms of all the section 3 man pages except
                     78:      those that summarize individual functions. The other two are the text
                     79:      forms of the section 1 man pages for the pcregrep and pcretest commands.
                     80:      These text forms are provided for ease of scanning with text editors or
                     81:      similar tools. They are installed in <prefix>/share/doc/pcre, where
                     82:      <prefix> is the installation prefix (defaulting to /usr/local).
                     83: 
                     84:   2. A set of files containing all the documentation in HTML form, hyperlinked
                     85:      in various ways, and rooted in a file called index.html, is distributed in
                     86:      doc/html and installed in <prefix>/share/doc/pcre/html.
                     87: 
                     88: 
                     89: Contributions by users of PCRE
                     90: ------------------------------
                     91: 
                     92: You can find contributions from PCRE users in the directory
                     93: 
                     94:   ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
                     95: 
                     96: There is a README file giving brief descriptions of what they are. Some are
                     97: complete in themselves; others are pointers to URLs containing relevant files.
                     98: Some of this material is likely to be well out-of-date. Several of the earlier
                     99: contributions provided support for compiling PCRE on various flavours of
                    100: Windows (I myself do not use Windows). Nowadays there is more Windows support
                    101: in the standard distribution, so these contibutions have been archived.
                    102: 
                    103: 
                    104: Building PCRE on non-Unix systems
                    105: ---------------------------------
                    106: 
                    107: For a non-Unix system, please read the comments in the file NON-UNIX-USE,
                    108: though if your system supports the use of "configure" and "make" you may be
                    109: able to build PCRE in the same way as for Unix-like systems. PCRE can also be
                    110: configured in many platform environments using the GUI facility of CMake's
                    111: CMakeSetup. It creates Makefiles, solution files, etc.
                    112: 
                    113: PCRE has been compiled on many different operating systems. It should be
                    114: straightforward to build PCRE on any system that has a Standard C compiler and
                    115: library, because it uses only Standard C functions.
                    116: 
                    117: 
                    118: Building PCRE on Unix-like systems
                    119: ----------------------------------
                    120: 
                    121: If you are using HP's ANSI C++ compiler (aCC), please see the special note
                    122: in the section entitled "Using HP's ANSI C++ compiler (aCC)" below.
                    123: 
                    124: The following instructions assume the use of the widely used "configure, make,
                    125: make install" process. There is also support for CMake in the PCRE
                    126: distribution; there are some comments about using CMake in the NON-UNIX-USE
                    127: file, though it can also be used in Unix-like systems.
                    128: 
                    129: To build PCRE on a Unix-like system, first run the "configure" command from the
                    130: PCRE distribution directory, with your current directory set to the directory
                    131: where you want the files to be created. This command is a standard GNU
                    132: "autoconf" configuration script, for which generic instructions are supplied in
                    133: the file INSTALL.
                    134: 
                    135: Most commonly, people build PCRE within its own distribution directory, and in
                    136: this case, on many systems, just running "./configure" is sufficient. However,
                    137: the usual methods of changing standard defaults are available. For example:
                    138: 
                    139: CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
                    140: 
                    141: specifies that the C compiler should be run with the flags '-O2 -Wall' instead
                    142: of the default, and that "make install" should install PCRE under /opt/local
                    143: instead of the default /usr/local.
                    144: 
                    145: If you want to build in a different directory, just run "configure" with that
                    146: directory as current. For example, suppose you have unpacked the PCRE source
                    147: into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
                    148: 
                    149: cd /build/pcre/pcre-xxx
                    150: /source/pcre/pcre-xxx/configure
                    151: 
                    152: PCRE is written in C and is normally compiled as a C library. However, it is
                    153: possible to build it as a C++ library, though the provided building apparatus
                    154: does not have any features to support this.
                    155: 
                    156: There are some optional features that can be included or omitted from the PCRE
                    157: library. You can read more about them in the pcrebuild man page.
                    158: 
                    159: . If you want to suppress the building of the C++ wrapper library, you can add
                    160:   --disable-cpp to the "configure" command. Otherwise, when "configure" is run,
                    161:   it will try to find a C++ compiler and C++ header files, and if it succeeds,
                    162:   it will try to build the C++ wrapper.
                    163: 
                    164: . If you want to make use of the support for UTF-8 character strings in PCRE,
                    165:   you must add --enable-utf8 to the "configure" command. Without it, the code
                    166:   for handling UTF-8 is not included in the library. (Even when included, it
                    167:   still has to be enabled by an option at run time.)
                    168: 
                    169: . If, in addition to support for UTF-8 character strings, you want to include
                    170:   support for the \P, \p, and \X sequences that recognize Unicode character
                    171:   properties, you must add --enable-unicode-properties to the "configure"
                    172:   command. This adds about 30K to the size of the library (in the form of a
                    173:   property table); only the basic two-letter properties such as Lu are
                    174:   supported.
                    175: 
                    176: . You can build PCRE to recognize either CR or LF or the sequence CRLF or any
                    177:   of the preceding, or any of the Unicode newline sequences as indicating the
                    178:   end of a line. Whatever you specify at build time is the default; the caller
                    179:   of PCRE can change the selection at run time. The default newline indicator
                    180:   is a single LF character (the Unix standard). You can specify the default
                    181:   newline indicator by adding --enable-newline-is-cr or --enable-newline-is-lf
                    182:   or --enable-newline-is-crlf or --enable-newline-is-anycrlf or
                    183:   --enable-newline-is-any to the "configure" command, respectively.
                    184: 
                    185:   If you specify --enable-newline-is-cr or --enable-newline-is-crlf, some of
                    186:   the standard tests will fail, because the lines in the test files end with
                    187:   LF. Even if the files are edited to change the line endings, there are likely
                    188:   to be some failures. With --enable-newline-is-anycrlf or
                    189:   --enable-newline-is-any, many tests should succeed, but there may be some
                    190:   failures.
                    191: 
                    192: . By default, the sequence \R in a pattern matches any Unicode line ending
                    193:   sequence. This is independent of the option specifying what PCRE considers to
                    194:   be the end of a line (see above). However, the caller of PCRE can restrict \R
                    195:   to match only CR, LF, or CRLF. You can make this the default by adding
                    196:   --enable-bsr-anycrlf to the "configure" command (bsr = "backslash R").
                    197: 
                    198: . When called via the POSIX interface, PCRE uses malloc() to get additional
                    199:   storage for processing capturing parentheses if there are more than 10 of
                    200:   them in a pattern. You can increase this threshold by setting, for example,
                    201: 
                    202:   --with-posix-malloc-threshold=20
                    203: 
                    204:   on the "configure" command.
                    205: 
                    206: . PCRE has a counter that can be set to limit the amount of resources it uses.
                    207:   If the limit is exceeded during a match, the match fails. The default is ten
                    208:   million. You can change the default by setting, for example,
                    209: 
                    210:   --with-match-limit=500000
                    211: 
                    212:   on the "configure" command. This is just the default; individual calls to
                    213:   pcre_exec() can supply their own value. There is more discussion on the
                    214:   pcreapi man page.
                    215: 
                    216: . There is a separate counter that limits the depth of recursive function calls
                    217:   during a matching process. This also has a default of ten million, which is
                    218:   essentially "unlimited". You can change the default by setting, for example,
                    219: 
                    220:   --with-match-limit-recursion=500000
                    221: 
                    222:   Recursive function calls use up the runtime stack; running out of stack can
                    223:   cause programs to crash in strange ways. There is a discussion about stack
                    224:   sizes in the pcrestack man page.
                    225: 
                    226: . The default maximum compiled pattern size is around 64K. You can increase
                    227:   this by adding --with-link-size=3 to the "configure" command. You can
                    228:   increase it even more by setting --with-link-size=4, but this is unlikely
                    229:   ever to be necessary. Increasing the internal link size will reduce
                    230:   performance.
                    231: 
                    232: . You can build PCRE so that its internal match() function that is called from
                    233:   pcre_exec() does not call itself recursively. Instead, it uses memory blocks
                    234:   obtained from the heap via the special functions pcre_stack_malloc() and
                    235:   pcre_stack_free() to save data that would otherwise be saved on the stack. To
                    236:   build PCRE like this, use
                    237: 
                    238:   --disable-stack-for-recursion
                    239: 
                    240:   on the "configure" command. PCRE runs more slowly in this mode, but it may be
                    241:   necessary in environments with limited stack sizes. This applies only to the
                    242:   pcre_exec() function; it does not apply to pcre_dfa_exec(), which does not
                    243:   use deeply nested recursion. There is a discussion about stack sizes in the
                    244:   pcrestack man page.
                    245: 
                    246: . For speed, PCRE uses four tables for manipulating and identifying characters
                    247:   whose code point values are less than 256. By default, it uses a set of
                    248:   tables for ASCII encoding that is part of the distribution. If you specify
                    249: 
                    250:   --enable-rebuild-chartables
                    251: 
                    252:   a program called dftables is compiled and run in the default C locale when
                    253:   you obey "make". It builds a source file called pcre_chartables.c. If you do
                    254:   not specify this option, pcre_chartables.c is created as a copy of
                    255:   pcre_chartables.c.dist. See "Character tables" below for further information.
                    256: 
                    257: . It is possible to compile PCRE for use on systems that use EBCDIC as their
                    258:   default character code (as opposed to ASCII) by specifying
                    259: 
                    260:   --enable-ebcdic
                    261: 
                    262:   This automatically implies --enable-rebuild-chartables (see above).
                    263: 
                    264: . It is possible to compile pcregrep to use libz and/or libbz2, in order to
                    265:   read .gz and .bz2 files (respectively), by specifying one or both of
                    266: 
                    267:   --enable-pcregrep-libz
                    268:   --enable-pcregrep-libbz2
                    269: 
                    270:   Of course, the relevant libraries must be installed on your system.
                    271: 
                    272: . It is possible to compile pcretest so that it links with the libreadline
                    273:   library, by specifying
                    274: 
                    275:   --enable-pcretest-libreadline
                    276: 
                    277:   If this is done, when pcretest's input is from a terminal, it reads it using
                    278:   the readline() function. This provides line-editing and history facilities.
                    279:   Note that libreadline is GPL-licenced, so if you distribute a binary of
                    280:   pcretest linked in this way, there may be licensing issues.
                    281: 
                    282:   Setting this option causes the -lreadline option to be added to the pcretest
                    283:   build. In many operating environments with a sytem-installed readline
                    284:   library this is sufficient. However, in some environments (e.g. if an
                    285:   unmodified distribution version of readline is in use), it may be necessary
                    286:   to specify something like LIBS="-lncurses" as well. This is because, to quote
                    287:   the readline INSTALL, "Readline uses the termcap functions, but does not link
                    288:   with the termcap or curses library itself, allowing applications which link
                    289:   with readline the to choose an appropriate library."
                    290: 
                    291: The "configure" script builds the following files for the basic C library:
                    292: 
                    293: . Makefile is the makefile that builds the library
                    294: . config.h contains build-time configuration options for the library
                    295: . pcre.h is the public PCRE header file
                    296: . pcre-config is a script that shows the settings of "configure" options
                    297: . libpcre.pc is data for the pkg-config command
                    298: . libtool is a script that builds shared and/or static libraries
                    299: . RunTest is a script for running tests on the basic C library
                    300: . RunGrepTest is a script for running tests on the pcregrep command
                    301: 
                    302: Versions of config.h and pcre.h are distributed in the PCRE tarballs under
                    303: the names config.h.generic and pcre.h.generic. These are provided for the
                    304: benefit of those who have to built PCRE without the benefit of "configure". If
                    305: you use "configure", the .generic versions are not used.
                    306: 
                    307: If a C++ compiler is found, the following files are also built:
                    308: 
                    309: . libpcrecpp.pc is data for the pkg-config command
                    310: . pcrecpparg.h is a header file for programs that call PCRE via the C++ wrapper
                    311: . pcre_stringpiece.h is the header for the C++ "stringpiece" functions
                    312: 
                    313: The "configure" script also creates config.status, which is an executable
                    314: script that can be run to recreate the configuration, and config.log, which
                    315: contains compiler output from tests that "configure" runs.
                    316: 
                    317: Once "configure" has run, you can run "make". It builds two libraries, called
                    318: libpcre and libpcreposix, a test program called pcretest, and the pcregrep
                    319: command. If a C++ compiler was found on your system, "make" also builds the C++
                    320: wrapper library, which is called libpcrecpp, and some test programs called
                    321: pcrecpp_unittest, pcre_scanner_unittest, and pcre_stringpiece_unittest.
                    322: Building the C++ wrapper can be disabled by adding --disable-cpp to the
                    323: "configure" command.
                    324: 
                    325: The command "make check" runs all the appropriate tests. Details of the PCRE
                    326: tests are given below in a separate section of this document.
                    327: 
                    328: You can use "make install" to install PCRE into live directories on your
                    329: system. The following are installed (file names are all relative to the
                    330: <prefix> that is set when "configure" is run):
                    331: 
                    332:   Commands (bin):
                    333:     pcretest
                    334:     pcregrep
                    335:     pcre-config
                    336: 
                    337:   Libraries (lib):
                    338:     libpcre
                    339:     libpcreposix
                    340:     libpcrecpp (if C++ support is enabled)
                    341: 
                    342:   Configuration information (lib/pkgconfig):
                    343:     libpcre.pc
                    344:     libpcrecpp.pc (if C++ support is enabled)
                    345: 
                    346:   Header files (include):
                    347:     pcre.h
                    348:     pcreposix.h
                    349:     pcre_scanner.h      )
                    350:     pcre_stringpiece.h  ) if C++ support is enabled
                    351:     pcrecpp.h           )
                    352:     pcrecpparg.h        )
                    353: 
                    354:   Man pages (share/man/man{1,3}):
                    355:     pcregrep.1
                    356:     pcretest.1
                    357:     pcre.3
                    358:     pcre*.3 (lots more pages, all starting "pcre")
                    359: 
                    360:   HTML documentation (share/doc/pcre/html):
                    361:     index.html
                    362:     *.html (lots more pages, hyperlinked from index.html)
                    363: 
                    364:   Text file documentation (share/doc/pcre):
                    365:     AUTHORS
                    366:     COPYING
                    367:     ChangeLog
                    368:     LICENCE
                    369:     NEWS
                    370:     README
                    371:     pcre.txt       (a concatenation of the man(3) pages)
                    372:     pcretest.txt   the pcretest man page
                    373:     pcregrep.txt   the pcregrep man page
                    374: 
                    375: If you want to remove PCRE from your system, you can run "make uninstall".
                    376: This removes all the files that "make install" installed. However, it does not
                    377: remove any directories, because these are often shared with other programs.
                    378: 
                    379: 
                    380: Retrieving configuration information on Unix-like systems
                    381: ---------------------------------------------------------
                    382: 
                    383: Running "make install" installs the command pcre-config, which can be used to
                    384: recall information about the PCRE configuration and installation. For example:
                    385: 
                    386:   pcre-config --version
                    387: 
                    388: prints the version number, and
                    389: 
                    390:   pcre-config --libs
                    391: 
                    392: outputs information about where the library is installed. This command can be
                    393: included in makefiles for programs that use PCRE, saving the programmer from
                    394: having to remember too many details.
                    395: 
                    396: The pkg-config command is another system for saving and retrieving information
                    397: about installed libraries. Instead of separate commands for each library, a
                    398: single command is used. For example:
                    399: 
                    400:   pkg-config --cflags pcre
                    401: 
                    402: The data is held in *.pc files that are installed in a directory called
                    403: <prefix>/lib/pkgconfig.
                    404: 
                    405: 
                    406: Shared libraries on Unix-like systems
                    407: -------------------------------------
                    408: 
                    409: The default distribution builds PCRE as shared libraries and static libraries,
                    410: as long as the operating system supports shared libraries. Shared library
                    411: support relies on the "libtool" script which is built as part of the
                    412: "configure" process.
                    413: 
                    414: The libtool script is used to compile and link both shared and static
                    415: libraries. They are placed in a subdirectory called .libs when they are newly
                    416: built. The programs pcretest and pcregrep are built to use these uninstalled
                    417: libraries (by means of wrapper scripts in the case of shared libraries). When
                    418: you use "make install" to install shared libraries, pcregrep and pcretest are
                    419: automatically re-built to use the newly installed shared libraries before being
                    420: installed themselves. However, the versions left in the build directory still
                    421: use the uninstalled libraries.
                    422: 
                    423: To build PCRE using static libraries only you must use --disable-shared when
                    424: configuring it. For example:
                    425: 
                    426: ./configure --prefix=/usr/gnu --disable-shared
                    427: 
                    428: Then run "make" in the usual way. Similarly, you can use --disable-static to
                    429: build only shared libraries.
                    430: 
                    431: 
                    432: Cross-compiling on Unix-like systems
                    433: ------------------------------------
                    434: 
                    435: You can specify CC and CFLAGS in the normal way to the "configure" command, in
                    436: order to cross-compile PCRE for some other host. However, you should NOT
                    437: specify --enable-rebuild-chartables, because if you do, the dftables.c source
                    438: file is compiled and run on the local host, in order to generate the inbuilt
                    439: character tables (the pcre_chartables.c file). This will probably not work,
                    440: because dftables.c needs to be compiled with the local compiler, not the cross
                    441: compiler.
                    442: 
                    443: When --enable-rebuild-chartables is not specified, pcre_chartables.c is created
                    444: by making a copy of pcre_chartables.c.dist, which is a default set of tables
                    445: that assumes ASCII code. Cross-compiling with the default tables should not be
                    446: a problem.
                    447: 
                    448: If you need to modify the character tables when cross-compiling, you should
                    449: move pcre_chartables.c.dist out of the way, then compile dftables.c by hand and
                    450: run it on the local host to make a new version of pcre_chartables.c.dist.
                    451: Then when you cross-compile PCRE this new version of the tables will be used.
                    452: 
                    453: 
                    454: Using HP's ANSI C++ compiler (aCC)
                    455: ----------------------------------
                    456: 
                    457: Unless C++ support is disabled by specifying the "--disable-cpp" option of the
                    458: "configure" script, you must include the "-AA" option in the CXXFLAGS
                    459: environment variable in order for the C++ components to compile correctly.
                    460: 
                    461: Also, note that the aCC compiler on PA-RISC platforms may have a defect whereby
                    462: needed libraries fail to get included when specifying the "-AA" compiler
                    463: option. If you experience unresolved symbols when linking the C++ programs,
                    464: use the workaround of specifying the following environment variable prior to
                    465: running the "configure" script:
                    466: 
                    467:   CXXLDFLAGS="-lstd_v2 -lCsup_v2"
                    468: 
                    469: 
                    470: Making new tarballs
                    471: -------------------
                    472: 
                    473: The command "make dist" creates three PCRE tarballs, in tar.gz, tar.bz2, and
                    474: zip formats. The command "make distcheck" does the same, but then does a trial
                    475: build of the new distribution to ensure that it works.
                    476: 
                    477: If you have modified any of the man page sources in the doc directory, you
                    478: should first run the PrepareRelease script before making a distribution. This
                    479: script creates the .txt and HTML forms of the documentation from the man pages.
                    480: 
                    481: 
                    482: Testing PCRE
                    483: ------------
                    484: 
                    485: To test the basic PCRE library on a Unix system, run the RunTest script that is
                    486: created by the configuring process. There is also a script called RunGrepTest
                    487: that tests the options of the pcregrep command. If the C++ wrapper library is
                    488: built, three test programs called pcrecpp_unittest, pcre_scanner_unittest, and
                    489: pcre_stringpiece_unittest are also built.
                    490: 
                    491: Both the scripts and all the program tests are run if you obey "make check" or
                    492: "make test". For other systems, see the instructions in NON-UNIX-USE.
                    493: 
                    494: The RunTest script runs the pcretest test program (which is documented in its
                    495: own man page) on each of the testinput files in the testdata directory in
                    496: turn, and compares the output with the contents of the corresponding testoutput
                    497: files. A file called testtry is used to hold the main output from pcretest
                    498: (testsavedregex is also used as a working file). To run pcretest on just one of
                    499: the test files, give its number as an argument to RunTest, for example:
                    500: 
                    501:   RunTest 2
                    502: 
                    503: The first test file can also be fed directly into the perltest.pl script to
                    504: check that Perl gives the same results. The only difference you should see is
                    505: in the first few lines, where the Perl version is given instead of the PCRE
                    506: version.
                    507: 
                    508: The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(),
                    509: pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
                    510: detection, and run-time flags that are specific to PCRE, as well as the POSIX
                    511: wrapper API. It also uses the debugging flags to check some of the internals of
                    512: pcre_compile().
                    513: 
                    514: If you build PCRE with a locale setting that is not the standard C locale, the
                    515: character tables may be different (see next paragraph). In some cases, this may
                    516: cause failures in the second set of tests. For example, in a locale where the
                    517: isprint() function yields TRUE for characters in the range 128-255, the use of
                    518: [:isascii:] inside a character class defines a different set of characters, and
                    519: this shows up in this test as a difference in the compiled code, which is being
                    520: listed for checking. Where the comparison test output contains [\x00-\x7f] the
                    521: test will contain [\x00-\xff], and similarly in some other cases. This is not a
                    522: bug in PCRE.
                    523: 
                    524: The third set of tests checks pcre_maketables(), the facility for building a
                    525: set of character tables for a specific locale and using them instead of the
                    526: default tables. The tests make use of the "fr_FR" (French) locale. Before
                    527: running the test, the script checks for the presence of this locale by running
                    528: the "locale" command. If that command fails, or if it doesn't include "fr_FR"
                    529: in the list of available locales, the third test cannot be run, and a comment
                    530: is output to say why. If running this test produces instances of the error
                    531: 
                    532:   ** Failed to set locale "fr_FR"
                    533: 
                    534: in the comparison output, it means that locale is not available on your system,
                    535: despite being listed by "locale". This does not mean that PCRE is broken.
                    536: 
                    537: [If you are trying to run this test on Windows, you may be able to get it to
                    538: work by changing "fr_FR" to "french" everywhere it occurs. Alternatively, use
                    539: RunTest.bat. The version of RunTest.bat included with PCRE 7.4 and above uses
                    540: Windows versions of test 2. More info on using RunTest.bat is included in the
                    541: document entitled NON-UNIX-USE.]
                    542: 
                    543: The fourth test checks the UTF-8 support. It is not run automatically unless
                    544: PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when
                    545: running "configure". This file can be also fed directly to the perltest script,
                    546: provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch,
                    547: commented in the script, can be be used.)
                    548: 
                    549: The fifth test checks error handling with UTF-8 encoding, and internal UTF-8
                    550: features of PCRE that are not relevant to Perl.
                    551: 
                    552: The sixth test checks the support for Unicode character properties. It it not
                    553: run automatically unless PCRE is built with Unicode property support. To to
                    554: this you must set --enable-unicode-properties when running "configure".
                    555: 
                    556: The seventh, eighth, and ninth tests check the pcre_dfa_exec() alternative
                    557: matching function, in non-UTF-8 mode, UTF-8 mode, and UTF-8 mode with Unicode
                    558: property support, respectively. The eighth and ninth tests are not run
                    559: automatically unless PCRE is build with the relevant support.
                    560: 
                    561: 
                    562: Character tables
                    563: ----------------
                    564: 
                    565: For speed, PCRE uses four tables for manipulating and identifying characters
                    566: whose code point values are less than 256. The final argument of the
                    567: pcre_compile() function is a pointer to a block of memory containing the
                    568: concatenated tables. A call to pcre_maketables() can be used to generate a set
                    569: of tables in the current locale. If the final argument for pcre_compile() is
                    570: passed as NULL, a set of default tables that is built into the binary is used.
                    571: 
                    572: The source file called pcre_chartables.c contains the default set of tables. By
                    573: default, this is created as a copy of pcre_chartables.c.dist, which contains
                    574: tables for ASCII coding. However, if --enable-rebuild-chartables is specified
                    575: for ./configure, a different version of pcre_chartables.c is built by the
                    576: program dftables (compiled from dftables.c), which uses the ANSI C character
                    577: handling functions such as isalnum(), isalpha(), isupper(), islower(), etc. to
                    578: build the table sources. This means that the default C locale which is set for
                    579: your system will control the contents of these default tables. You can change
                    580: the default tables by editing pcre_chartables.c and then re-building PCRE. If
                    581: you do this, you should take care to ensure that the file does not get
                    582: automatically re-generated. The best way to do this is to move
                    583: pcre_chartables.c.dist out of the way and replace it with your customized
                    584: tables.
                    585: 
                    586: When the dftables program is run as a result of --enable-rebuild-chartables,
                    587: it uses the default C locale that is set on your system. It does not pay
                    588: attention to the LC_xxx environment variables. In other words, it uses the
                    589: system's default locale rather than whatever the compiling user happens to have
                    590: set. If you really do want to build a source set of character tables in a
                    591: locale that is specified by the LC_xxx variables, you can run the dftables
                    592: program by hand with the -L option. For example:
                    593: 
                    594:   ./dftables -L pcre_chartables.c.special
                    595: 
                    596: The first two 256-byte tables provide lower casing and case flipping functions,
                    597: respectively. The next table consists of three 32-byte bit maps which identify
                    598: digits, "word" characters, and white space, respectively. These are used when
                    599: building 32-byte bit maps that represent character classes for code points less
                    600: than 256.
                    601: 
                    602: The final 256-byte table has bits indicating various character types, as
                    603: follows:
                    604: 
                    605:     1   white space character
                    606:     2   letter
                    607:     4   decimal digit
                    608:     8   hexadecimal digit
                    609:    16   alphanumeric or '_'
                    610:   128   regular expression metacharacter or binary zero
                    611: 
                    612: You should not alter the set of characters that contain the 128 bit, as that
                    613: will cause PCRE to malfunction.
                    614: 
                    615: 
                    616: File manifest
                    617: -------------
                    618: 
                    619: The distribution should contain the following files:
                    620: 
                    621: (A) Source files of the PCRE library functions and their headers:
                    622: 
                    623:   dftables.c              auxiliary program for building pcre_chartables.c
                    624:                             when --enable-rebuild-chartables is specified
                    625: 
                    626:   pcre_chartables.c.dist  a default set of character tables that assume ASCII
                    627:                             coding; used, unless --enable-rebuild-chartables is
                    628:                             specified, by copying to pcre_chartables.c
                    629: 
                    630:   pcreposix.c             )
                    631:   pcre_compile.c          )
                    632:   pcre_config.c           )
                    633:   pcre_dfa_exec.c         )
                    634:   pcre_exec.c             )
                    635:   pcre_fullinfo.c         )
                    636:   pcre_get.c              ) sources for the functions in the library,
                    637:   pcre_globals.c          )   and some internal functions that they use
                    638:   pcre_info.c             )
                    639:   pcre_maketables.c       )
                    640:   pcre_newline.c          )
                    641:   pcre_ord2utf8.c         )
                    642:   pcre_refcount.c         )
                    643:   pcre_study.c            )
                    644:   pcre_tables.c           )
                    645:   pcre_try_flipped.c      )
1.2     ! misha     646:   pcre_ucd.c              )
1.1       misha     647:   pcre_valid_utf8.c       )
                    648:   pcre_version.c          )
                    649:   pcre_xclass.c           )
                    650:   pcre_printint.src       ) debugging function that is #included in pcretest,
                    651:                           )   and can also be #included in pcre_compile()
                    652:   pcre.h.in               template for pcre.h when built by "configure"
                    653:   pcreposix.h             header for the external POSIX wrapper API
                    654:   pcre_internal.h         header for internal use
1.2     ! misha     655:   ucp.h                   header for Unicode property handling
1.1       misha     656: 
                    657:   config.h.in             template for config.h, which is built by "configure"
                    658: 
                    659:   pcrecpp.h               public header file for the C++ wrapper
                    660:   pcrecpparg.h.in         template for another C++ header file
                    661:   pcre_scanner.h          public header file for C++ scanner functions
                    662:   pcrecpp.cc              )
                    663:   pcre_scanner.cc         ) source for the C++ wrapper library
                    664: 
                    665:   pcre_stringpiece.h.in   template for pcre_stringpiece.h, the header for the
                    666:                             C++ stringpiece functions
                    667:   pcre_stringpiece.cc     source for the C++ stringpiece functions
                    668: 
                    669: (B) Source files for programs that use PCRE:
                    670: 
                    671:   pcredemo.c              simple demonstration of coding calls to PCRE
                    672:   pcregrep.c              source of a grep utility that uses PCRE
                    673:   pcretest.c              comprehensive test program
                    674: 
                    675: (C) Auxiliary files:
                    676: 
                    677:   132html                 script to turn "man" pages into HTML
                    678:   AUTHORS                 information about the author of PCRE
                    679:   ChangeLog               log of changes to the code
                    680:   CleanTxt                script to clean nroff output for txt man pages
                    681:   Detrail                 script to remove trailing spaces
                    682:   HACKING                 some notes about the internals of PCRE
                    683:   INSTALL                 generic installation instructions
                    684:   LICENCE                 conditions for the use of PCRE
                    685:   COPYING                 the same, using GNU's standard name
                    686:   Makefile.in             ) template for Unix Makefile, which is built by
                    687:                           )   "configure"
                    688:   Makefile.am             ) the automake input that was used to create
                    689:                           )   Makefile.in
                    690:   NEWS                    important changes in this release
                    691:   NON-UNIX-USE            notes on building PCRE on non-Unix systems
                    692:   PrepareRelease          script to make preparations for "make dist"
                    693:   README                  this file
                    694:   RunTest                 a Unix shell script for running tests
                    695:   RunGrepTest             a Unix shell script for pcregrep tests
                    696:   aclocal.m4              m4 macros (generated by "aclocal")
                    697:   config.guess            ) files used by libtool,
                    698:   config.sub              )   used only when building a shared library
                    699:   configure               a configuring shell script (built by autoconf)
                    700:   configure.ac            ) the autoconf input that was used to build
                    701:                           )   "configure" and config.h
                    702:   depcomp                 ) script to find program dependencies, generated by
                    703:                           )   automake
                    704:   doc/*.3                 man page sources for the PCRE functions
                    705:   doc/*.1                 man page sources for pcregrep and pcretest
                    706:   doc/index.html.src      the base HTML page
                    707:   doc/html/*              HTML documentation
                    708:   doc/pcre.txt            plain text version of the man pages
                    709:   doc/pcretest.txt        plain text documentation of test program
                    710:   doc/perltest.txt        plain text documentation of Perl test program
                    711:   install-sh              a shell script for installing files
                    712:   libpcre.pc.in           template for libpcre.pc for pkg-config
                    713:   libpcrecpp.pc.in        template for libpcrecpp.pc for pkg-config
                    714:   ltmain.sh               file used to build a libtool script
                    715:   missing                 ) common stub for a few missing GNU programs while
                    716:                           )   installing, generated by automake
                    717:   mkinstalldirs           script for making install directories
                    718:   perltest.pl             Perl test program
                    719:   pcre-config.in          source of script which retains PCRE information
                    720:   pcrecpp_unittest.cc          )
                    721:   pcre_scanner_unittest.cc     ) test programs for the C++ wrapper
                    722:   pcre_stringpiece_unittest.cc )
                    723:   testdata/testinput*     test data for main library tests
                    724:   testdata/testoutput*    expected test results
                    725:   testdata/grep*          input and output for pcregrep tests
                    726: 
                    727: (D) Auxiliary files for cmake support
                    728: 
1.2     ! misha     729:   cmake/COPYING-CMAKE-SCRIPTS
        !           730:   cmake/FindPackageHandleStandardArgs.cmake
        !           731:   cmake/FindReadline.cmake
1.1       misha     732:   CMakeLists.txt
                    733:   config-cmake.h.in
                    734: 
                    735: (E) Auxiliary files for VPASCAL
                    736: 
                    737:   makevp.bat
                    738:   makevp_c.txt
                    739:   makevp_l.txt
                    740:   pcregexp.pas
                    741: 
                    742: (F) Auxiliary files for building PCRE "by hand"
                    743: 
                    744:   pcre.h.generic          ) a version of the public PCRE header file
                    745:                           )   for use in non-"configure" environments
                    746:   config.h.generic        ) a version of config.h for use in non-"configure"
                    747:                           )   environments
                    748: 
                    749: (F) Miscellaneous
                    750: 
                    751:   RunTest.bat            a script for running tests under Windows
                    752: 
                    753: Philip Hazel
                    754: Email local part: ph10
                    755: Email domain: cam.ac.uk
1.2     ! misha     756: Last updated: 05 September 2008

E-mail: