Annotation of win32/pcre/README, revision 1.7
1.1 misha 1: README file for PCRE (Perl-compatible regular expression library)
2: -----------------------------------------------------------------
3:
1.7 ! moko 4: NOTE: This set of files relates to PCRE releases that use the original API,
! 5: with library names libpcre, libpcre16, and libpcre32. January 2015 saw the
! 6: first release of a new API, known as PCRE2, with release numbers starting at
! 7: 10.00 and library names libpcre2-8, libpcre2-16, and libpcre2-32. The old
! 8: libraries (now called PCRE1) are still being maintained for bug fixes, but
! 9: there will be no new development. New projects are advised to use the new PCRE2
! 10: libraries.
! 11:
! 12:
! 13: The latest release of PCRE1 is always available in three alternative formats
1.2 misha 14: from:
1.1 misha 15:
16: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
1.2 misha 17: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.bz2
18: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.zip
1.1 misha 19:
20: There is a mailing list for discussion about the development of PCRE at
1.7 ! moko 21: pcre-dev@exim.org. You can access the archives and subscribe or manage your
! 22: subscription here:
1.1 misha 23:
1.7 ! moko 24: https://lists.exim.org/mailman/listinfo/pcre-dev
1.1 misha 25:
26: Please read the NEWS file if you are upgrading from a previous release.
27: The contents of this README file are:
28:
29: The PCRE APIs
30: Documentation for PCRE
31: Contributions by users of PCRE
1.6 misha 32: Building PCRE on non-Unix-like systems
33: Building PCRE without using autotools
34: Building PCRE using autotools
35: Retrieving configuration information
36: Shared libraries
37: Cross-compiling using autotools
1.1 misha 38: Using HP's ANSI C++ compiler (aCC)
1.6 misha 39: Compiling in Tru64 using native compilers
40: Using Sun's compilers for Solaris
1.4 misha 41: Using PCRE from MySQL
1.1 misha 42: Making new tarballs
43: Testing PCRE
44: Character tables
45: File manifest
46:
47:
48: The PCRE APIs
49: -------------
50:
1.6 misha 51: PCRE is written in C, and it has its own API. There are three sets of
52: functions, one for the 8-bit library, which processes strings of bytes, one for
53: the 16-bit library, which processes strings of 16-bit values, and one for the
54: 32-bit library, which processes strings of 32-bit values. The distribution also
1.5 misha 55: includes a set of C++ wrapper functions (see the pcrecpp man page for details),
56: courtesy of Google Inc., which can be used to call the 8-bit PCRE library from
1.7 ! moko 57: C++. Other C++ wrappers have been created from time to time. See, for example:
! 58: https://github.com/YasserAsmi/regexp, which aims to be simple and similar in
! 59: style to the C API.
! 60:
! 61: The distribution also contains a set of C wrapper functions (again, just for
! 62: the 8-bit library) that are based on the POSIX regular expression API (see the
! 63: pcreposix man page). These end up in the library called libpcreposix. Note that
! 64: this just provides a POSIX calling interface to PCRE; the regular expressions
! 65: themselves still follow Perl syntax and semantics. The POSIX API is restricted,
! 66: and does not give full access to all of PCRE's facilities.
1.1 misha 67:
68: The header file for the POSIX-style functions is called pcreposix.h. The
69: official POSIX name is regex.h, but I did not want to risk possible problems
70: with existing files of that name by distributing it that way. To use PCRE with
71: an existing program that uses the POSIX API, pcreposix.h will have to be
72: renamed or pointed at by a link.
73:
74: If you are using the POSIX interface to PCRE and there is already a POSIX regex
75: library installed on your system, as well as worrying about the regex.h header
76: file (as mentioned above), you must also take care when linking programs to
77: ensure that they link with PCRE's libpcreposix library. Otherwise they may pick
78: up the POSIX functions of the same name from the other library.
79:
80: One way of avoiding this confusion is to compile PCRE with the addition of
81: -Dregcomp=PCREregcomp (and similarly for the other POSIX functions) to the
82: compiler flags (CFLAGS if you are using "configure" -- see below). This has the
83: effect of renaming the functions so that the names no longer clash. Of course,
84: you have to do the same thing for your applications, or write them using the
85: new names.
86:
87:
88: Documentation for PCRE
89: ----------------------
90:
91: If you install PCRE in the normal way on a Unix-like system, you will end up
92: with a set of man pages whose names all start with "pcre". The one that is just
93: called "pcre" lists all the others. In addition to these man pages, the PCRE
94: documentation is supplied in two other forms:
95:
96: 1. There are files called doc/pcre.txt, doc/pcregrep.txt, and
97: doc/pcretest.txt in the source distribution. The first of these is a
98: concatenation of the text forms of all the section 3 man pages except
1.7 ! moko 99: the listing of pcredemo.c and those that summarize individual functions.
! 100: The other two are the text forms of the section 1 man pages for the
! 101: pcregrep and pcretest commands. These text forms are provided for ease of
! 102: scanning with text editors or similar tools. They are installed in
! 103: <prefix>/share/doc/pcre, where <prefix> is the installation prefix
! 104: (defaulting to /usr/local).
1.1 misha 105:
106: 2. A set of files containing all the documentation in HTML form, hyperlinked
107: in various ways, and rooted in a file called index.html, is distributed in
108: doc/html and installed in <prefix>/share/doc/pcre/html.
109:
1.3 misha 110: Users of PCRE have contributed files containing the documentation for various
111: releases in CHM format. These can be found in the Contrib directory of the FTP
112: site (see next section).
113:
1.1 misha 114:
115: Contributions by users of PCRE
116: ------------------------------
117:
118: You can find contributions from PCRE users in the directory
119:
120: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
121:
122: There is a README file giving brief descriptions of what they are. Some are
123: complete in themselves; others are pointers to URLs containing relevant files.
124: Some of this material is likely to be well out-of-date. Several of the earlier
125: contributions provided support for compiling PCRE on various flavours of
126: Windows (I myself do not use Windows). Nowadays there is more Windows support
127: in the standard distribution, so these contibutions have been archived.
128:
1.7 ! moko 129: A PCRE user maintains downloadable Windows binaries of the pcregrep and
! 130: pcretest programs here:
! 131:
! 132: http://www.rexegg.com/pcregrep-pcretest.html
! 133:
1.1 misha 134:
1.6 misha 135: Building PCRE on non-Unix-like systems
136: --------------------------------------
1.1 misha 137:
1.6 misha 138: For a non-Unix-like system, please read the comments in the file
139: NON-AUTOTOOLS-BUILD, though if your system supports the use of "configure" and
140: "make" you may be able to build PCRE using autotools in the same way as for
141: many Unix-like systems.
142:
143: PCRE can also be configured using the GUI facility provided by CMake's
144: cmake-gui command. This creates Makefiles, solution files, etc. The file
145: NON-AUTOTOOLS-BUILD has information about CMake.
1.1 misha 146:
147: PCRE has been compiled on many different operating systems. It should be
148: straightforward to build PCRE on any system that has a Standard C compiler and
149: library, because it uses only Standard C functions.
150:
151:
1.6 misha 152: Building PCRE without using autotools
153: -------------------------------------
154:
155: The use of autotools (in particular, libtool) is problematic in some
156: environments, even some that are Unix or Unix-like. See the NON-AUTOTOOLS-BUILD
157: file for ways of building PCRE without using autotools.
158:
159:
160: Building PCRE using autotools
161: -----------------------------
1.1 misha 162:
163: If you are using HP's ANSI C++ compiler (aCC), please see the special note
164: in the section entitled "Using HP's ANSI C++ compiler (aCC)" below.
165:
1.6 misha 166: The following instructions assume the use of the widely used "configure; make;
167: make install" (autotools) process.
168:
169: To build PCRE on system that supports autotools, first run the "configure"
170: command from the PCRE distribution directory, with your current directory set
171: to the directory where you want the files to be created. This command is a
172: standard GNU "autoconf" configuration script, for which generic instructions
173: are supplied in the file INSTALL.
1.1 misha 174:
175: Most commonly, people build PCRE within its own distribution directory, and in
176: this case, on many systems, just running "./configure" is sufficient. However,
177: the usual methods of changing standard defaults are available. For example:
178:
179: CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
180:
1.5 misha 181: This command specifies that the C compiler should be run with the flags '-O2
182: -Wall' instead of the default, and that "make install" should install PCRE
183: under /opt/local instead of the default /usr/local.
1.1 misha 184:
185: If you want to build in a different directory, just run "configure" with that
186: directory as current. For example, suppose you have unpacked the PCRE source
187: into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
188:
189: cd /build/pcre/pcre-xxx
190: /source/pcre/pcre-xxx/configure
191:
192: PCRE is written in C and is normally compiled as a C library. However, it is
193: possible to build it as a C++ library, though the provided building apparatus
194: does not have any features to support this.
195:
196: There are some optional features that can be included or omitted from the PCRE
1.5 misha 197: library. They are also documented in the pcrebuild man page.
1.1 misha 198:
1.5 misha 199: . By default, both shared and static libraries are built. You can change this
200: by adding one of these options to the "configure" command:
201:
202: --disable-shared
203: --disable-static
204:
205: (See also "Shared libraries on Unix-like systems" below.)
206:
207: . By default, only the 8-bit library is built. If you add --enable-pcre16 to
1.6 misha 208: the "configure" command, the 16-bit library is also built. If you add
209: --enable-pcre32 to the "configure" command, the 32-bit library is also built.
210: If you want only the 16-bit or 32-bit library, use --disable-pcre8 to disable
211: building the 8-bit library.
1.5 misha 212:
213: . If you are building the 8-bit library and want to suppress the building of
214: the C++ wrapper library, you can add --disable-cpp to the "configure"
215: command. Otherwise, when "configure" is run without --disable-pcre8, it will
216: try to find a C++ compiler and C++ header files, and if it succeeds, it will
217: try to build the C++ wrapper.
218:
219: . If you want to include support for just-in-time compiling, which can give
220: large performance improvements on certain platforms, add --enable-jit to the
221: "configure" command. This support is available only for certain hardware
222: architectures. If you try to enable it on an unsupported architecture, there
223: will be a compile time error.
224:
225: . When JIT support is enabled, pcregrep automatically makes use of it, unless
226: you add --disable-pcregrep-jit to the "configure" command.
1.1 misha 227:
1.3 misha 228: . If you want to make use of the support for UTF-8 Unicode character strings in
1.5 misha 229: the 8-bit library, or UTF-16 Unicode character strings in the 16-bit library,
1.6 misha 230: or UTF-32 Unicode character strings in the 32-bit library, you must add
231: --enable-utf to the "configure" command. Without it, the code for handling
232: UTF-8, UTF-16 and UTF-8 is not included in the relevant library. Even
1.5 misha 233: when --enable-utf is included, the use of a UTF encoding still has to be
234: enabled by an option at run time. When PCRE is compiled with this option, its
1.6 misha 235: input can only either be ASCII or UTF-8/16/32, even when running on EBCDIC
1.5 misha 236: platforms. It is not possible to use both --enable-utf and --enable-ebcdic at
237: the same time.
238:
1.6 misha 239: . There are no separate options for enabling UTF-8, UTF-16 and UTF-32
240: independently because that would allow ridiculous settings such as requesting
241: UTF-16 support while building only the 8-bit library. However, the option
1.5 misha 242: --enable-utf8 is retained for backwards compatibility with earlier releases
1.6 misha 243: that did not support 16-bit or 32-bit character strings. It is synonymous with
1.5 misha 244: --enable-utf. It is not possible to configure one library with UTF support
245: and the other without in the same configuration.
246:
1.6 misha 247: . If, in addition to support for UTF-8/16/32 character strings, you want to
1.5 misha 248: include support for the \P, \p, and \X sequences that recognize Unicode
249: character properties, you must add --enable-unicode-properties to the
250: "configure" command. This adds about 30K to the size of the library (in the
251: form of a property table); only the basic two-letter properties such as Lu
252: are supported.
1.1 misha 253:
254: . You can build PCRE to recognize either CR or LF or the sequence CRLF or any
255: of the preceding, or any of the Unicode newline sequences as indicating the
256: end of a line. Whatever you specify at build time is the default; the caller
257: of PCRE can change the selection at run time. The default newline indicator
258: is a single LF character (the Unix standard). You can specify the default
259: newline indicator by adding --enable-newline-is-cr or --enable-newline-is-lf
260: or --enable-newline-is-crlf or --enable-newline-is-anycrlf or
261: --enable-newline-is-any to the "configure" command, respectively.
262:
263: If you specify --enable-newline-is-cr or --enable-newline-is-crlf, some of
264: the standard tests will fail, because the lines in the test files end with
265: LF. Even if the files are edited to change the line endings, there are likely
266: to be some failures. With --enable-newline-is-anycrlf or
267: --enable-newline-is-any, many tests should succeed, but there may be some
268: failures.
269:
270: . By default, the sequence \R in a pattern matches any Unicode line ending
271: sequence. This is independent of the option specifying what PCRE considers to
272: be the end of a line (see above). However, the caller of PCRE can restrict \R
273: to match only CR, LF, or CRLF. You can make this the default by adding
274: --enable-bsr-anycrlf to the "configure" command (bsr = "backslash R").
275:
276: . When called via the POSIX interface, PCRE uses malloc() to get additional
277: storage for processing capturing parentheses if there are more than 10 of
278: them in a pattern. You can increase this threshold by setting, for example,
279:
280: --with-posix-malloc-threshold=20
281:
282: on the "configure" command.
283:
1.7 ! moko 284: . PCRE has a counter that limits the depth of nesting of parentheses in a
! 285: pattern. This limits the amount of system stack that a pattern uses when it
! 286: is compiled. The default is 250, but you can change it by setting, for
! 287: example,
! 288:
! 289: --with-parens-nest-limit=500
! 290:
! 291: . PCRE has a counter that can be set to limit the amount of resources it uses
! 292: when matching a pattern. If the limit is exceeded during a match, the match
! 293: fails. The default is ten million. You can change the default by setting, for
! 294: example,
1.1 misha 295:
296: --with-match-limit=500000
297:
298: on the "configure" command. This is just the default; individual calls to
299: pcre_exec() can supply their own value. There is more discussion on the
300: pcreapi man page.
301:
302: . There is a separate counter that limits the depth of recursive function calls
303: during a matching process. This also has a default of ten million, which is
304: essentially "unlimited". You can change the default by setting, for example,
305:
306: --with-match-limit-recursion=500000
307:
308: Recursive function calls use up the runtime stack; running out of stack can
309: cause programs to crash in strange ways. There is a discussion about stack
310: sizes in the pcrestack man page.
311:
312: . The default maximum compiled pattern size is around 64K. You can increase
1.5 misha 313: this by adding --with-link-size=3 to the "configure" command. In the 8-bit
314: library, PCRE then uses three bytes instead of two for offsets to different
315: parts of the compiled pattern. In the 16-bit library, --with-link-size=3 is
316: the same as --with-link-size=4, which (in both libraries) uses four-byte
1.6 misha 317: offsets. Increasing the internal link size reduces performance. In the 32-bit
318: library, the only supported link size is 4.
1.1 misha 319:
320: . You can build PCRE so that its internal match() function that is called from
321: pcre_exec() does not call itself recursively. Instead, it uses memory blocks
322: obtained from the heap via the special functions pcre_stack_malloc() and
323: pcre_stack_free() to save data that would otherwise be saved on the stack. To
324: build PCRE like this, use
325:
326: --disable-stack-for-recursion
327:
328: on the "configure" command. PCRE runs more slowly in this mode, but it may be
329: necessary in environments with limited stack sizes. This applies only to the
1.5 misha 330: normal execution of the pcre_exec() function; if JIT support is being
331: successfully used, it is not relevant. Equally, it does not apply to
332: pcre_dfa_exec(), which does not use deeply nested recursion. There is a
333: discussion about stack sizes in the pcrestack man page.
1.1 misha 334:
335: . For speed, PCRE uses four tables for manipulating and identifying characters
336: whose code point values are less than 256. By default, it uses a set of
337: tables for ASCII encoding that is part of the distribution. If you specify
338:
339: --enable-rebuild-chartables
340:
341: a program called dftables is compiled and run in the default C locale when
342: you obey "make". It builds a source file called pcre_chartables.c. If you do
343: not specify this option, pcre_chartables.c is created as a copy of
344: pcre_chartables.c.dist. See "Character tables" below for further information.
345:
346: . It is possible to compile PCRE for use on systems that use EBCDIC as their
1.6 misha 347: character code (as opposed to ASCII/Unicode) by specifying
1.1 misha 348:
349: --enable-ebcdic
350:
1.3 misha 351: This automatically implies --enable-rebuild-chartables (see above). However,
352: when PCRE is built this way, it always operates in EBCDIC. It cannot support
1.6 misha 353: both EBCDIC and UTF-8/16/32. There is a second option, --enable-ebcdic-nl25,
354: which specifies that the code value for the EBCDIC NL character is 0x25
355: instead of the default 0x15.
356:
357: . In environments where valgrind is installed, if you specify
358:
359: --enable-valgrind
360:
361: PCRE will use valgrind annotations to mark certain memory regions as
362: unaddressable. This allows it to detect invalid memory accesses, and is
363: mostly useful for debugging PCRE itself.
364:
365: . In environments where the gcc compiler is used and lcov version 1.6 or above
366: is installed, if you specify
367:
368: --enable-coverage
369:
370: the build process implements a code coverage report for the test suite. The
371: report is generated by running "make coverage". If ccache is installed on
372: your system, it must be disabled when building PCRE for coverage reporting.
373: You can do this by setting the environment variable CCACHE_DISABLE=1 before
1.7 ! moko 374: running "make" to build PCRE. There is more information about coverage
! 375: reporting in the "pcrebuild" documentation.
1.1 misha 376:
1.5 misha 377: . The pcregrep program currently supports only 8-bit data files, and so
378: requires the 8-bit PCRE library. It is possible to compile pcregrep to use
379: libz and/or libbz2, in order to read .gz and .bz2 files (respectively), by
380: specifying one or both of
1.1 misha 381:
382: --enable-pcregrep-libz
383: --enable-pcregrep-libbz2
384:
385: Of course, the relevant libraries must be installed on your system.
386:
1.7 ! moko 387: . The default size (in bytes) of the internal buffer used by pcregrep can be
! 388: set by, for example:
1.5 misha 389:
1.7 ! moko 390: --with-pcregrep-bufsize=51200
1.5 misha 391:
1.7 ! moko 392: The value must be a plain integer. The default is 20480.
1.5 misha 393:
1.1 misha 394: . It is possible to compile pcretest so that it links with the libreadline
1.6 misha 395: or libedit libraries, by specifying, respectively,
1.1 misha 396:
1.6 misha 397: --enable-pcretest-libreadline or --enable-pcretest-libedit
1.1 misha 398:
399: If this is done, when pcretest's input is from a terminal, it reads it using
400: the readline() function. This provides line-editing and history facilities.
401: Note that libreadline is GPL-licenced, so if you distribute a binary of
1.6 misha 402: pcretest linked in this way, there may be licensing issues. These can be
403: avoided by linking with libedit (which has a BSD licence) instead.
1.1 misha 404:
1.6 misha 405: Enabling libreadline causes the -lreadline option to be added to the pcretest
1.1 misha 406: build. In many operating environments with a sytem-installed readline
407: library this is sufficient. However, in some environments (e.g. if an
408: unmodified distribution version of readline is in use), it may be necessary
409: to specify something like LIBS="-lncurses" as well. This is because, to quote
410: the readline INSTALL, "Readline uses the termcap functions, but does not link
411: with the termcap or curses library itself, allowing applications which link
1.3 misha 412: with readline the to choose an appropriate library." If you get error
413: messages about missing functions tgetstr, tgetent, tputs, tgetflag, or tgoto,
414: this is the problem, and linking with the ncurses library should fix it.
1.1 misha 415:
416: The "configure" script builds the following files for the basic C library:
417:
1.5 misha 418: . Makefile the makefile that builds the library
419: . config.h build-time configuration options for the library
420: . pcre.h the public PCRE header file
421: . pcre-config script that shows the building settings such as CFLAGS
422: that were set for "configure"
423: . libpcre.pc ) data for the pkg-config command
424: . libpcre16.pc )
1.6 misha 425: . libpcre32.pc )
1.5 misha 426: . libpcreposix.pc )
427: . libtool script that builds shared and/or static libraries
1.1 misha 428:
1.4 misha 429: Versions of config.h and pcre.h are distributed in the PCRE tarballs under the
430: names config.h.generic and pcre.h.generic. These are provided for those who
431: have to built PCRE without using "configure" or CMake. If you use "configure"
432: or CMake, the .generic versions are not used.
1.1 misha 433:
1.5 misha 434: When building the 8-bit library, if a C++ compiler is found, the following
435: files are also built:
1.1 misha 436:
1.5 misha 437: . libpcrecpp.pc data for the pkg-config command
438: . pcrecpparg.h header file for calling PCRE via the C++ wrapper
439: . pcre_stringpiece.h header for the C++ "stringpiece" functions
1.1 misha 440:
441: The "configure" script also creates config.status, which is an executable
442: script that can be run to recreate the configuration, and config.log, which
443: contains compiler output from tests that "configure" runs.
444:
1.6 misha 445: Once "configure" has run, you can run "make". This builds the the libraries
446: libpcre, libpcre16 and/or libpcre32, and a test program called pcretest. If you
1.5 misha 447: enabled JIT support with --enable-jit, a test program called pcre_jit_test is
448: built as well.
449:
450: If the 8-bit library is built, libpcreposix and the pcregrep command are also
451: built, and if a C++ compiler was found on your system, and you did not disable
452: it with --disable-cpp, "make" builds the C++ wrapper library, which is called
453: libpcrecpp, as well as some test programs called pcrecpp_unittest,
454: pcre_scanner_unittest, and pcre_stringpiece_unittest.
1.1 misha 455:
456: The command "make check" runs all the appropriate tests. Details of the PCRE
457: tests are given below in a separate section of this document.
458:
459: You can use "make install" to install PCRE into live directories on your
460: system. The following are installed (file names are all relative to the
461: <prefix> that is set when "configure" is run):
462:
463: Commands (bin):
464: pcretest
1.5 misha 465: pcregrep (if 8-bit support is enabled)
1.1 misha 466: pcre-config
467:
468: Libraries (lib):
1.5 misha 469: libpcre16 (if 16-bit support is enabled)
1.6 misha 470: libpcre32 (if 32-bit support is enabled)
1.5 misha 471: libpcre (if 8-bit support is enabled)
472: libpcreposix (if 8-bit support is enabled)
473: libpcrecpp (if 8-bit and C++ support is enabled)
1.1 misha 474:
475: Configuration information (lib/pkgconfig):
1.5 misha 476: libpcre16.pc
1.6 misha 477: libpcre32.pc
1.1 misha 478: libpcre.pc
1.5 misha 479: libpcreposix.pc
1.1 misha 480: libpcrecpp.pc (if C++ support is enabled)
481:
482: Header files (include):
483: pcre.h
484: pcreposix.h
485: pcre_scanner.h )
486: pcre_stringpiece.h ) if C++ support is enabled
487: pcrecpp.h )
488: pcrecpparg.h )
489:
490: Man pages (share/man/man{1,3}):
491: pcregrep.1
492: pcretest.1
1.5 misha 493: pcre-config.1
1.1 misha 494: pcre.3
495: pcre*.3 (lots more pages, all starting "pcre")
496:
497: HTML documentation (share/doc/pcre/html):
498: index.html
499: *.html (lots more pages, hyperlinked from index.html)
500:
501: Text file documentation (share/doc/pcre):
502: AUTHORS
503: COPYING
504: ChangeLog
505: LICENCE
506: NEWS
507: README
1.5 misha 508: pcre.txt (a concatenation of the man(3) pages)
509: pcretest.txt the pcretest man page
510: pcregrep.txt the pcregrep man page
511: pcre-config.txt the pcre-config man page
1.1 misha 512:
513: If you want to remove PCRE from your system, you can run "make uninstall".
514: This removes all the files that "make install" installed. However, it does not
515: remove any directories, because these are often shared with other programs.
516:
517:
1.6 misha 518: Retrieving configuration information
519: ------------------------------------
1.1 misha 520:
521: Running "make install" installs the command pcre-config, which can be used to
522: recall information about the PCRE configuration and installation. For example:
523:
524: pcre-config --version
525:
526: prints the version number, and
527:
528: pcre-config --libs
529:
530: outputs information about where the library is installed. This command can be
531: included in makefiles for programs that use PCRE, saving the programmer from
532: having to remember too many details.
533:
534: The pkg-config command is another system for saving and retrieving information
535: about installed libraries. Instead of separate commands for each library, a
536: single command is used. For example:
537:
538: pkg-config --cflags pcre
539:
540: The data is held in *.pc files that are installed in a directory called
541: <prefix>/lib/pkgconfig.
542:
543:
1.6 misha 544: Shared libraries
545: ----------------
1.1 misha 546:
547: The default distribution builds PCRE as shared libraries and static libraries,
548: as long as the operating system supports shared libraries. Shared library
549: support relies on the "libtool" script which is built as part of the
550: "configure" process.
551:
552: The libtool script is used to compile and link both shared and static
553: libraries. They are placed in a subdirectory called .libs when they are newly
554: built. The programs pcretest and pcregrep are built to use these uninstalled
555: libraries (by means of wrapper scripts in the case of shared libraries). When
556: you use "make install" to install shared libraries, pcregrep and pcretest are
557: automatically re-built to use the newly installed shared libraries before being
558: installed themselves. However, the versions left in the build directory still
559: use the uninstalled libraries.
560:
561: To build PCRE using static libraries only you must use --disable-shared when
562: configuring it. For example:
563:
564: ./configure --prefix=/usr/gnu --disable-shared
565:
566: Then run "make" in the usual way. Similarly, you can use --disable-static to
567: build only shared libraries.
568:
569:
1.6 misha 570: Cross-compiling using autotools
571: -------------------------------
1.1 misha 572:
573: You can specify CC and CFLAGS in the normal way to the "configure" command, in
574: order to cross-compile PCRE for some other host. However, you should NOT
575: specify --enable-rebuild-chartables, because if you do, the dftables.c source
576: file is compiled and run on the local host, in order to generate the inbuilt
577: character tables (the pcre_chartables.c file). This will probably not work,
578: because dftables.c needs to be compiled with the local compiler, not the cross
579: compiler.
580:
581: When --enable-rebuild-chartables is not specified, pcre_chartables.c is created
582: by making a copy of pcre_chartables.c.dist, which is a default set of tables
583: that assumes ASCII code. Cross-compiling with the default tables should not be
584: a problem.
585:
586: If you need to modify the character tables when cross-compiling, you should
587: move pcre_chartables.c.dist out of the way, then compile dftables.c by hand and
588: run it on the local host to make a new version of pcre_chartables.c.dist.
589: Then when you cross-compile PCRE this new version of the tables will be used.
590:
591:
592: Using HP's ANSI C++ compiler (aCC)
593: ----------------------------------
594:
595: Unless C++ support is disabled by specifying the "--disable-cpp" option of the
596: "configure" script, you must include the "-AA" option in the CXXFLAGS
597: environment variable in order for the C++ components to compile correctly.
598:
599: Also, note that the aCC compiler on PA-RISC platforms may have a defect whereby
600: needed libraries fail to get included when specifying the "-AA" compiler
601: option. If you experience unresolved symbols when linking the C++ programs,
602: use the workaround of specifying the following environment variable prior to
603: running the "configure" script:
604:
605: CXXLDFLAGS="-lstd_v2 -lCsup_v2"
606:
607:
1.6 misha 608: Compiling in Tru64 using native compilers
609: -----------------------------------------
610:
611: The following error may occur when compiling with native compilers in the Tru64
612: operating system:
613:
614: CXX libpcrecpp_la-pcrecpp.lo
615: cxx: Error: /usr/lib/cmplrs/cxx/V7.1-006/include/cxx/iosfwd, line 58: #error
616: directive: "cannot include iosfwd -- define __USE_STD_IOSTREAM to
617: override default - see section 7.1.2 of the C++ Using Guide"
618: #error "cannot include iosfwd -- define __USE_STD_IOSTREAM to override default
619: - see section 7.1.2 of the C++ Using Guide"
620:
621: This may be followed by other errors, complaining that 'namespace "std" has no
622: member'. The solution to this is to add the line
623:
624: #define __USE_STD_IOSTREAM 1
625:
626: to the config.h file.
627:
628:
1.4 misha 629: Using Sun's compilers for Solaris
630: ---------------------------------
631:
632: A user reports that the following configurations work on Solaris 9 sparcv9 and
633: Solaris 9 x86 (32-bit):
634:
635: Solaris 9 sparcv9: ./configure --disable-cpp CC=/bin/cc CFLAGS="-m64 -g"
636: Solaris 9 x86: ./configure --disable-cpp CC=/bin/cc CFLAGS="-g"
637:
638:
639: Using PCRE from MySQL
640: ---------------------
641:
642: On systems where both PCRE and MySQL are installed, it is possible to make use
643: of PCRE from within MySQL, as an alternative to the built-in pattern matching.
644: There is a web page that tells you how to do this:
645:
646: http://www.mysqludf.org/lib_mysqludf_preg/index.php
647:
648:
1.1 misha 649: Making new tarballs
650: -------------------
651:
652: The command "make dist" creates three PCRE tarballs, in tar.gz, tar.bz2, and
653: zip formats. The command "make distcheck" does the same, but then does a trial
654: build of the new distribution to ensure that it works.
655:
656: If you have modified any of the man page sources in the doc directory, you
657: should first run the PrepareRelease script before making a distribution. This
658: script creates the .txt and HTML forms of the documentation from the man pages.
659:
660:
661: Testing PCRE
662: ------------
663:
1.6 misha 664: To test the basic PCRE library on a Unix-like system, run the RunTest script.
665: There is another script called RunGrepTest that tests the options of the
666: pcregrep command. If the C++ wrapper library is built, three test programs
667: called pcrecpp_unittest, pcre_scanner_unittest, and pcre_stringpiece_unittest
668: are also built. When JIT support is enabled, another test program called
669: pcre_jit_test is built.
1.1 misha 670:
671: Both the scripts and all the program tests are run if you obey "make check" or
1.6 misha 672: "make test". For other environments, see the instructions in
673: NON-AUTOTOOLS-BUILD.
1.1 misha 674:
675: The RunTest script runs the pcretest test program (which is documented in its
1.5 misha 676: own man page) on each of the relevant testinput files in the testdata
677: directory, and compares the output with the contents of the corresponding
1.6 misha 678: testoutput files. RunTest uses a file called testtry to hold the main output
679: from pcretest. Other files whose names begin with "test" are used as working
680: files in some tests.
681:
682: Some tests are relevant only when certain build-time options were selected. For
683: example, the tests for UTF-8/16/32 support are run only if --enable-utf was
684: used. RunTest outputs a comment when it skips a test.
1.5 misha 685:
686: Many of the tests that are not skipped are run up to three times. The second
687: run forces pcre_study() to be called for all patterns except for a few in some
688: tests that are marked "never study" (see the pcretest program for how this is
689: done). If JIT support is available, the non-DFA tests are run a third time,
690: this time with a forced pcre_study() with the PCRE_STUDY_JIT_COMPILE option.
1.6 misha 691: This testing can be suppressed by putting "nojit" on the RunTest command line.
1.5 misha 692:
1.6 misha 693: The entire set of tests is run once for each of the 8-bit, 16-bit and 32-bit
694: libraries that are enabled. If you want to run just one set of tests, call
695: RunTest with either the -8, -16 or -32 option.
696:
697: If valgrind is installed, you can run the tests under it by putting "valgrind"
698: on the RunTest command line. To run pcretest on just one or more specific test
699: files, give their numbers as arguments to RunTest, for example:
1.5 misha 700:
701: RunTest 2 7 11
702:
1.6 misha 703: You can also specify ranges of tests such as 3-6 or 3- (meaning 3 to the
704: end), or a number preceded by ~ to exclude a test. For example:
705:
706: Runtest 3-15 ~10
707:
708: This runs tests 3 to 15, excluding test 10, and just ~13 runs all the tests
709: except test 13. Whatever order the arguments are in, the tests are always run
710: in numerical order.
711:
712: You can also call RunTest with the single argument "list" to cause it to output
713: a list of tests.
714:
1.5 misha 715: The first test file can be fed directly into the perltest.pl script to check
716: that Perl gives the same results. The only difference you should see is in the
717: first few lines, where the Perl version is given instead of the PCRE version.
1.1 misha 718:
1.5 misha 719: The second set of tests check pcre_fullinfo(), pcre_study(),
1.1 misha 720: pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
721: detection, and run-time flags that are specific to PCRE, as well as the POSIX
722: wrapper API. It also uses the debugging flags to check some of the internals of
723: pcre_compile().
724:
725: If you build PCRE with a locale setting that is not the standard C locale, the
726: character tables may be different (see next paragraph). In some cases, this may
727: cause failures in the second set of tests. For example, in a locale where the
728: isprint() function yields TRUE for characters in the range 128-255, the use of
729: [:isascii:] inside a character class defines a different set of characters, and
730: this shows up in this test as a difference in the compiled code, which is being
731: listed for checking. Where the comparison test output contains [\x00-\x7f] the
732: test will contain [\x00-\xff], and similarly in some other cases. This is not a
733: bug in PCRE.
734:
735: The third set of tests checks pcre_maketables(), the facility for building a
736: set of character tables for a specific locale and using them instead of the
737: default tables. The tests make use of the "fr_FR" (French) locale. Before
738: running the test, the script checks for the presence of this locale by running
739: the "locale" command. If that command fails, or if it doesn't include "fr_FR"
740: in the list of available locales, the third test cannot be run, and a comment
741: is output to say why. If running this test produces instances of the error
742:
743: ** Failed to set locale "fr_FR"
744:
745: in the comparison output, it means that locale is not available on your system,
746: despite being listed by "locale". This does not mean that PCRE is broken.
747:
748: [If you are trying to run this test on Windows, you may be able to get it to
749: work by changing "fr_FR" to "french" everywhere it occurs. Alternatively, use
750: RunTest.bat. The version of RunTest.bat included with PCRE 7.4 and above uses
751: Windows versions of test 2. More info on using RunTest.bat is included in the
752: document entitled NON-UNIX-USE.]
753:
1.6 misha 754: The fourth and fifth tests check the UTF-8/16/32 support and error handling and
1.5 misha 755: internal UTF features of PCRE that are not relevant to Perl, respectively. The
756: sixth and seventh tests do the same for Unicode character properties support.
757:
758: The eighth, ninth, and tenth tests check the pcre_dfa_exec() alternative
1.6 misha 759: matching function, in non-UTF-8/16/32 mode, UTF-8/16/32 mode, and UTF-8/16/32
760: mode with Unicode property support, respectively.
1.5 misha 761:
762: The eleventh test checks some internal offsets and code size features; it is
763: run only when the default "link size" of 2 is set (in other cases the sizes
764: change) and when Unicode property support is enabled.
765:
766: The twelfth test is run only when JIT support is available, and the thirteenth
767: test is run only when JIT support is not available. They test some JIT-specific
768: features such as information output from pcretest about JIT compilation.
769:
770: The fourteenth, fifteenth, and sixteenth tests are run only in 8-bit mode, and
1.6 misha 771: the seventeenth, eighteenth, and nineteenth tests are run only in 16/32-bit
772: mode. These are tests that generate different output in the two modes. They are
773: for general cases, UTF-8/16/32 support, and Unicode property support,
774: respectively.
775:
776: The twentieth test is run only in 16/32-bit mode. It tests some specific
777: 16/32-bit features of the DFA matching engine.
778:
779: The twenty-first and twenty-second tests are run only in 16/32-bit mode, when
780: the link size is set to 2 for the 16-bit library. They test reloading
781: pre-compiled patterns.
1.5 misha 782:
1.6 misha 783: The twenty-third and twenty-fourth tests are run only in 16-bit mode. They are
784: for general cases, and UTF-16 support, respectively.
1.5 misha 785:
1.6 misha 786: The twenty-fifth and twenty-sixth tests are run only in 32-bit mode. They are
787: for general cases, and UTF-32 support, respectively.
1.4 misha 788:
1.1 misha 789:
790: Character tables
791: ----------------
792:
793: For speed, PCRE uses four tables for manipulating and identifying characters
794: whose code point values are less than 256. The final argument of the
795: pcre_compile() function is a pointer to a block of memory containing the
796: concatenated tables. A call to pcre_maketables() can be used to generate a set
797: of tables in the current locale. If the final argument for pcre_compile() is
798: passed as NULL, a set of default tables that is built into the binary is used.
799:
800: The source file called pcre_chartables.c contains the default set of tables. By
801: default, this is created as a copy of pcre_chartables.c.dist, which contains
802: tables for ASCII coding. However, if --enable-rebuild-chartables is specified
803: for ./configure, a different version of pcre_chartables.c is built by the
804: program dftables (compiled from dftables.c), which uses the ANSI C character
805: handling functions such as isalnum(), isalpha(), isupper(), islower(), etc. to
806: build the table sources. This means that the default C locale which is set for
807: your system will control the contents of these default tables. You can change
808: the default tables by editing pcre_chartables.c and then re-building PCRE. If
809: you do this, you should take care to ensure that the file does not get
810: automatically re-generated. The best way to do this is to move
811: pcre_chartables.c.dist out of the way and replace it with your customized
812: tables.
813:
814: When the dftables program is run as a result of --enable-rebuild-chartables,
815: it uses the default C locale that is set on your system. It does not pay
816: attention to the LC_xxx environment variables. In other words, it uses the
817: system's default locale rather than whatever the compiling user happens to have
818: set. If you really do want to build a source set of character tables in a
819: locale that is specified by the LC_xxx variables, you can run the dftables
820: program by hand with the -L option. For example:
821:
822: ./dftables -L pcre_chartables.c.special
823:
824: The first two 256-byte tables provide lower casing and case flipping functions,
825: respectively. The next table consists of three 32-byte bit maps which identify
826: digits, "word" characters, and white space, respectively. These are used when
827: building 32-byte bit maps that represent character classes for code points less
828: than 256.
829:
830: The final 256-byte table has bits indicating various character types, as
831: follows:
832:
833: 1 white space character
834: 2 letter
835: 4 decimal digit
836: 8 hexadecimal digit
837: 16 alphanumeric or '_'
838: 128 regular expression metacharacter or binary zero
839:
840: You should not alter the set of characters that contain the 128 bit, as that
841: will cause PCRE to malfunction.
842:
843:
844: File manifest
845: -------------
846:
1.5 misha 847: The distribution should contain the files listed below. Where a file name is
1.6 misha 848: given as pcre[16|32]_xxx it means that there are three files, one with the name
849: pcre_xxx, one with the name pcre16_xx, and a third with the name pcre32_xxx.
1.1 misha 850:
851: (A) Source files of the PCRE library functions and their headers:
852:
853: dftables.c auxiliary program for building pcre_chartables.c
1.6 misha 854: when --enable-rebuild-chartables is specified
1.1 misha 855:
856: pcre_chartables.c.dist a default set of character tables that assume ASCII
1.6 misha 857: coding; used, unless --enable-rebuild-chartables is
858: specified, by copying to pcre[16]_chartables.c
1.1 misha 859:
1.6 misha 860: pcreposix.c )
861: pcre[16|32]_byte_order.c )
862: pcre[16|32]_compile.c )
863: pcre[16|32]_config.c )
864: pcre[16|32]_dfa_exec.c )
865: pcre[16|32]_exec.c )
866: pcre[16|32]_fullinfo.c )
867: pcre[16|32]_get.c ) sources for the functions in the library,
868: pcre[16|32]_globals.c ) and some internal functions that they use
869: pcre[16|32]_jit_compile.c )
870: pcre[16|32]_maketables.c )
871: pcre[16|32]_newline.c )
872: pcre[16|32]_refcount.c )
873: pcre[16|32]_string_utils.c )
874: pcre[16|32]_study.c )
875: pcre[16|32]_tables.c )
876: pcre[16|32]_ucd.c )
877: pcre[16|32]_version.c )
878: pcre[16|32]_xclass.c )
879: pcre_ord2utf8.c )
880: pcre_valid_utf8.c )
881: pcre16_ord2utf16.c )
882: pcre16_utf16_utils.c )
883: pcre16_valid_utf16.c )
884: pcre32_utf32_utils.c )
885: pcre32_valid_utf32.c )
1.5 misha 886:
1.6 misha 887: pcre[16|32]_printint.c ) debugging function that is used by pcretest,
888: ) and can also be #included in pcre_compile()
1.5 misha 889:
1.1 misha 890: pcre.h.in template for pcre.h when built by "configure"
891: pcreposix.h header for the external POSIX wrapper API
892: pcre_internal.h header for internal use
1.5 misha 893: sljit/* 16 files that make up the JIT compiler
1.2 misha 894: ucp.h header for Unicode property handling
1.1 misha 895:
896: config.h.in template for config.h, which is built by "configure"
897:
898: pcrecpp.h public header file for the C++ wrapper
899: pcrecpparg.h.in template for another C++ header file
900: pcre_scanner.h public header file for C++ scanner functions
901: pcrecpp.cc )
902: pcre_scanner.cc ) source for the C++ wrapper library
903:
904: pcre_stringpiece.h.in template for pcre_stringpiece.h, the header for the
905: C++ stringpiece functions
906: pcre_stringpiece.cc source for the C++ stringpiece functions
907:
908: (B) Source files for programs that use PCRE:
909:
910: pcredemo.c simple demonstration of coding calls to PCRE
911: pcregrep.c source of a grep utility that uses PCRE
912: pcretest.c comprehensive test program
913:
914: (C) Auxiliary files:
915:
916: 132html script to turn "man" pages into HTML
917: AUTHORS information about the author of PCRE
918: ChangeLog log of changes to the code
919: CleanTxt script to clean nroff output for txt man pages
920: Detrail script to remove trailing spaces
921: HACKING some notes about the internals of PCRE
922: INSTALL generic installation instructions
923: LICENCE conditions for the use of PCRE
924: COPYING the same, using GNU's standard name
925: Makefile.in ) template for Unix Makefile, which is built by
926: ) "configure"
927: Makefile.am ) the automake input that was used to create
928: ) Makefile.in
929: NEWS important changes in this release
1.6 misha 930: NON-UNIX-USE the previous name for NON-AUTOTOOLS-BUILD
931: NON-AUTOTOOLS-BUILD notes on building PCRE without using autotools
1.1 misha 932: PrepareRelease script to make preparations for "make dist"
933: README this file
934: RunTest a Unix shell script for running tests
935: RunGrepTest a Unix shell script for pcregrep tests
936: aclocal.m4 m4 macros (generated by "aclocal")
937: config.guess ) files used by libtool,
938: config.sub ) used only when building a shared library
939: configure a configuring shell script (built by autoconf)
940: configure.ac ) the autoconf input that was used to build
941: ) "configure" and config.h
942: depcomp ) script to find program dependencies, generated by
943: ) automake
1.4 misha 944: doc/*.3 man page sources for PCRE
1.1 misha 945: doc/*.1 man page sources for pcregrep and pcretest
946: doc/index.html.src the base HTML page
947: doc/html/* HTML documentation
948: doc/pcre.txt plain text version of the man pages
949: doc/pcretest.txt plain text documentation of test program
950: doc/perltest.txt plain text documentation of Perl test program
951: install-sh a shell script for installing files
1.5 misha 952: libpcre16.pc.in template for libpcre16.pc for pkg-config
1.6 misha 953: libpcre32.pc.in template for libpcre32.pc for pkg-config
1.1 misha 954: libpcre.pc.in template for libpcre.pc for pkg-config
1.4 misha 955: libpcreposix.pc.in template for libpcreposix.pc for pkg-config
1.1 misha 956: libpcrecpp.pc.in template for libpcrecpp.pc for pkg-config
957: ltmain.sh file used to build a libtool script
958: missing ) common stub for a few missing GNU programs while
959: ) installing, generated by automake
960: mkinstalldirs script for making install directories
961: perltest.pl Perl test program
962: pcre-config.in source of script which retains PCRE information
1.5 misha 963: pcre_jit_test.c test program for the JIT compiler
1.1 misha 964: pcrecpp_unittest.cc )
965: pcre_scanner_unittest.cc ) test programs for the C++ wrapper
966: pcre_stringpiece_unittest.cc )
967: testdata/testinput* test data for main library tests
968: testdata/testoutput* expected test results
969: testdata/grep* input and output for pcregrep tests
1.5 misha 970: testdata/* other supporting test files
1.1 misha 971:
972: (D) Auxiliary files for cmake support
973:
1.2 misha 974: cmake/COPYING-CMAKE-SCRIPTS
975: cmake/FindPackageHandleStandardArgs.cmake
1.6 misha 976: cmake/FindEditline.cmake
1.2 misha 977: cmake/FindReadline.cmake
1.1 misha 978: CMakeLists.txt
979: config-cmake.h.in
980:
981: (E) Auxiliary files for VPASCAL
982:
983: makevp.bat
984: makevp_c.txt
985: makevp_l.txt
986: pcregexp.pas
987:
988: (F) Auxiliary files for building PCRE "by hand"
989:
990: pcre.h.generic ) a version of the public PCRE header file
991: ) for use in non-"configure" environments
992: config.h.generic ) a version of config.h for use in non-"configure"
993: ) environments
994:
995: (F) Miscellaneous
996:
997: RunTest.bat a script for running tests under Windows
998:
999: Philip Hazel
1000: Email local part: ph10
1001: Email domain: cam.ac.uk
1.7 ! moko 1002: Last updated: 10 February 2015
E-mail:
![[BACK]](/cvsweb/icons/back.gif){kind=icon}
Return to README CVS log ![[TXT]](/cvsweb/icons/text.gif){kind=icon}
![[DIR]](/cvsweb/icons/dir.gif){kind=icon}
Up to [parser3project] / win32 / pcre