-----------------------------------------------------------------------------

This file contains a concatenation of the PCRE man pages, converted to plain

text format for ease of searching with a text editor, or for use on systems

that do not have a man page processor. The small individual files that give

synopses of each function in the library have not been included. There are

separate text files for the pcregrep and pcretest commands.

-----------------------------------------------------------------------------

PCRE(3)                                                                PCRE(3)

NAME

       PCRE - Perl-compatible regular expressions

INTRODUCTION

       The  PCRE  library is a set of functions that implement regular expres-

       sion pattern matching using the same syntax and semantics as Perl, with

       just  a  few  differences. Certain features that appeared in Python and

       PCRE before they appeared in Perl are also available using  the  Python

       syntax.  There is also some support for certain .NET and Oniguruma syn-

       tax items, and there is an option for  requesting  some  minor  changes

       that give better JavaScript compatibility.

       The  current  implementation of PCRE (release 7.x) corresponds approxi-

       mately with Perl 5.10, including support for UTF-8 encoded strings  and

       Unicode general category properties. However, UTF-8 and Unicode support

       has to be explicitly enabled; it is not the default. The Unicode tables

       correspond to Unicode release 5.0.0.

       In  addition to the Perl-compatible matching function, PCRE contains an

       alternative matching function that matches the same  compiled  patterns

       in  a different way. In certain circumstances, the alternative function

       has some advantages. For a discussion of the two  matching  algorithms,

       see the pcrematching page.

       PCRE  is  written  in C and released as a C library. A number of people

       have written wrappers and interfaces of various kinds.  In  particular,

       Google  Inc.   have  provided  a comprehensive C++ wrapper. This is now

       included as part of the PCRE distribution. The pcrecpp page has details

       of  this  interface.  Other  people's contributions can be found in the

       Contrib directory at the primary FTP site, which is:

       ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre

       Details of exactly which Perl regular expression features are  and  are

       not supported by PCRE are given in separate documents. See the pcrepat-

       tern and pcrecompat pages. There is a syntax summary in the  pcresyntax

       page.

       Some  features  of  PCRE can be included, excluded, or changed when the

       library is built. The pcre_config() function makes it  possible  for  a

       client  to  discover  which  features are available. The features them-

       selves are described in the pcrebuild page. Documentation about  build-

       ing  PCRE for various operating systems can be found in the README file

       in the source distribution.

       The library contains a number of undocumented  internal  functions  and

       data  tables  that  are  used by more than one of the exported external

       functions, but which are not intended  for  use  by  external  callers.

       Their  names  all begin with "_pcre_", which hopefully will not provoke

       any name clashes. In some environments, it is possible to control which

       external  symbols  are  exported when a shared library is built, and in

       these cases the undocumented symbols are not exported.

USER DOCUMENTATION

       The user documentation for PCRE comprises a number  of  different  sec-

       tions.  In the "man" format, each of these is a separate "man page". In

       the HTML format, each is a separate page, linked from the  index  page.

       In  the  plain text format, all the sections are concatenated, for ease

       of searching. The sections are as follows:

         pcre              this document

         pcre-config       show PCRE installation configuration information

         pcreapi           details of PCRE's native C API

         pcrebuild         options for building PCRE

         pcrecallout       details of the callout feature

         pcrecompat        discussion of Perl compatibility

         pcrecpp           details of the C++ wrapper

         pcregrep          description of the pcregrep command

         pcrematching      discussion of the two matching algorithms

         pcrepartial       details of the partial matching facility

         pcrepattern       syntax and semantics of supported

                             regular expressions

         pcresyntax        quick syntax reference

         pcreperform       discussion of performance issues

         pcreposix         the POSIX-compatible C API

         pcreprecompile    details of saving and re-using precompiled patterns

         pcresample        discussion of the sample program

         pcrestack         discussion of stack usage

         pcretest          description of the pcretest testing command

       In  addition,  in the "man" and HTML formats, there is a short page for

       each C library function, listing its arguments and results.

LIMITATIONS

       There are some size limitations in PCRE but it is hoped that they  will

       never in practice be relevant.

       The  maximum  length of a compiled pattern is 65539 (sic) bytes if PCRE

       is compiled with the default internal linkage size of 2. If you want to

       process  regular  expressions  that are truly enormous, you can compile

       PCRE with an internal linkage size of 3 or 4 (see the  README  file  in

       the  source  distribution and the pcrebuild documentation for details).

       In these cases the limit is substantially larger.  However,  the  speed

       of execution is slower.

       All values in repeating quantifiers must be less than 65536.

       There is no limit to the number of parenthesized subpatterns, but there

       can be no more than 65535 capturing subpatterns.

       The maximum length of name for a named subpattern is 32 characters, and

       the maximum number of named subpatterns is 10000.

       The  maximum  length of a subject string is the largest positive number

       that an integer variable can hold. However, when using the  traditional

       matching function, PCRE uses recursion to handle subpatterns and indef-

       inite repetition.  This means that the available stack space may  limit

       the size of a subject string that can be processed by certain patterns.

       For a discussion of stack issues, see the pcrestack documentation.

UTF-8 AND UNICODE PROPERTY SUPPORT

       From release 3.3, PCRE has  had  some  support  for  character  strings

       encoded  in the UTF-8 format. For release 4.0 this was greatly extended

       to cover most common requirements, and in release 5.0  additional  sup-

       port for Unicode general category properties was added.

       In  order  process  UTF-8 strings, you must build PCRE to include UTF-8

       support in the code, and, in addition,  you  must  call  pcre_compile()

       with  the PCRE_UTF8 option flag. When you do this, both the pattern and

       any subject strings that are matched against it are  treated  as  UTF-8

       strings instead of just strings of bytes.

       If  you compile PCRE with UTF-8 support, but do not use it at run time,

       the library will be a bit bigger, but the additional run time  overhead

       is limited to testing the PCRE_UTF8 flag occasionally, so should not be

       very big.

       If PCRE is built with Unicode character property support (which implies

       UTF-8  support),  the  escape sequences \p{..}, \P{..}, and \X are sup-

       ported.  The available properties that can be tested are limited to the

       general  category  properties such as Lu for an upper case letter or Nd

       for a decimal number, the Unicode script names such as Arabic  or  Han,

       and  the  derived  properties  Any  and L&. A full list is given in the

       pcrepattern documentation. Only the short names for properties are sup-

       ported.  For example, \p{L} matches a letter. Its Perl synonym, \p{Let-

       ter}, is not supported.  Furthermore,  in  Perl,  many  properties  may

       optionally  be  prefixed by "Is", for compatibility with Perl 5.6. PCRE

       does not support this.

   Validity of UTF-8 strings

       When you set the PCRE_UTF8 flag, the strings  passed  as  patterns  and

       subjects are (by default) checked for validity on entry to the relevant

       functions. From release 7.3 of PCRE, the check is according  the  rules

       of  RFC  3629, which are themselves derived from the Unicode specifica-

       tion. Earlier releases of PCRE followed the rules of  RFC  2279,  which

       allows  the  full range of 31-bit values (0 to 0x7FFFFFFF). The current

       check allows only values in the range U+0 to U+10FFFF, excluding U+D800

       to U+DFFF.

       The  excluded  code  points are the "Low Surrogate Area" of Unicode, of

       which the Unicode Standard says this: "The Low Surrogate Area does  not

       contain  any  character  assignments,  consequently  no  character code

       charts or namelists are provided for this area. Surrogates are reserved

       for  use  with  UTF-16 and then must be used in pairs." The code points

       that are encoded by UTF-16 pairs  are  available  as  independent  code

       points  in  the  UTF-8  encoding.  (In other words, the whole surrogate

       thing is a fudge for UTF-16 which unfortunately messes up UTF-8.)

       If an  invalid  UTF-8  string  is  passed  to  PCRE,  an  error  return

       (PCRE_ERROR_BADUTF8) is given. In some situations, you may already know

       that your strings are valid, and therefore want to skip these checks in

       order to improve performance. If you set the PCRE_NO_UTF8_CHECK flag at

       compile time or at run time, PCRE assumes that the pattern  or  subject

       it  is  given  (respectively)  contains only valid UTF-8 codes. In this

       case, it does not diagnose an invalid UTF-8 string.

       If you pass an invalid UTF-8 string  when  PCRE_NO_UTF8_CHECK  is  set,

       what  happens  depends on why the string is invalid. If the string con-

       forms to the "old" definition of UTF-8 (RFC 2279), it is processed as a

       string  of  characters  in  the  range 0 to 0x7FFFFFFF. In other words,

       apart from the initial validity test, PCRE (when in UTF-8 mode) handles

       strings  according  to  the more liberal rules of RFC 2279. However, if

       the string does not even conform to RFC 2279, the result is  undefined.

       Your program may crash.

       If  you  want  to  process  strings  of  values  in the full range 0 to

       0x7FFFFFFF, encoded in a UTF-8-like manner as per the old RFC, you  can

       set PCRE_NO_UTF8_CHECK to bypass the more restrictive test. However, in

       this situation, you will have to apply your own validity check.

   General comments about UTF-8 mode

       1. An unbraced hexadecimal escape sequence (such  as  \xb3)  matches  a

       two-byte UTF-8 character if the value is greater than 127.

       2.  Octal  numbers  up to \777 are recognized, and match two-byte UTF-8

       characters for values greater than \177.

       3. Repeat quantifiers apply to complete UTF-8 characters, not to  indi-

       vidual bytes, for example: \x{100}{3}.

       4.  The dot metacharacter matches one UTF-8 character instead of a sin-

       gle byte.

       5. The escape sequence \C can be used to match a single byte  in  UTF-8

       mode,  but  its  use can lead to some strange effects. This facility is

       not available in the alternative matching function, pcre_dfa_exec().

       6. The character escapes \b, \B, \d, \D, \s, \S, \w, and  \W  correctly

       test  characters of any code value, but the characters that PCRE recog-

       nizes as digits, spaces, or word characters  remain  the  same  set  as

       before, all with values less than 256. This remains true even when PCRE

       includes Unicode property support, because to do otherwise  would  slow

       down  PCRE in many common cases. If you really want to test for a wider

       sense of, say, "digit", you must use Unicode  property  tests  such  as

       \p{Nd}.

       7.  Similarly,  characters that match the POSIX named character classes

       are all low-valued characters.

       8. However, the Perl 5.10 horizontal and vertical  whitespace  matching

       escapes (\h, \H, \v, and \V) do match all the appropriate Unicode char-

       acters.

       9. Case-insensitive matching applies only to  characters  whose  values

       are  less than 128, unless PCRE is built with Unicode property support.

       Even when Unicode property support is available, PCRE  still  uses  its

       own  character  tables when checking the case of low-valued characters,

       so as not to degrade performance.  The Unicode property information  is

       used only for characters with higher values. Even when Unicode property

       support is available, PCRE supports case-insensitive matching only when

       there  is  a  one-to-one  mapping between a letter's cases. There are a

       small number of many-to-one mappings in Unicode;  these  are  not  sup-

       ported by PCRE.

AUTHOR

       Philip Hazel

       University Computing Service

       Cambridge CB2 3QH, England.

       Putting  an actual email address here seems to have been a spam magnet,

       so I've taken it away. If you want to email me, use  my  two  initials,

       followed by the two digits 10, at the domain cam.ac.uk.

REVISION

       Last updated: 12 April 2008

       Copyright (c) 1997-2008 University of Cambridge.

------------------------------------------------------------------------------

PCREBUILD(3)                                                      PCREBUILD(3)

NAME

       PCRE - Perl-compatible regular expressions

PCRE BUILD-TIME OPTIONS

       This  document  describes  the  optional  features  of PCRE that can be

       selected when the library is compiled. It assumes use of the  configure

       script,  where the optional features are selected or deselected by pro-

       viding options to configure before running the make  command.  However,

       the  same  options  can be selected in both Unix-like and non-Unix-like

       environments using the GUI facility of  CMakeSetup  if  you  are  using

       CMake instead of configure to build PCRE.

       The complete list of options for configure (which includes the standard

       ones such as the  selection  of  the  installation  directory)  can  be

       obtained by running

         ./configure --help

       The  following  sections  include  descriptions  of options whose names

       begin with --enable or --disable. These settings specify changes to the

       defaults  for  the configure command. Because of the way that configure

       works, --enable and --disable always come in pairs, so  the  complemen-

       tary  option always exists as well, but as it specifies the default, it

       is not described.

C++ SUPPORT

       By default, the configure script will search for a C++ compiler and C++

       header files. If it finds them, it automatically builds the C++ wrapper

       library for PCRE. You can disable this by adding

         --disable-cpp

       to the configure command.

UTF-8 SUPPORT

       To build PCRE with support for UTF-8 character strings, add

         --enable-utf8

       to the configure command. Of itself, this  does  not  make  PCRE  treat

       strings  as UTF-8. As well as compiling PCRE with this option, you also

       have have to set the PCRE_UTF8 option when you call the  pcre_compile()

       function.

UNICODE CHARACTER PROPERTY SUPPORT

       UTF-8  support allows PCRE to process character values greater than 255

       in the strings that it handles. On its own, however, it does  not  pro-

       vide any facilities for accessing the properties of such characters. If

       you want to be able to use the pattern escapes \P, \p,  and  \X,  which

       refer to Unicode character properties, you must add

         --enable-unicode-properties

       to  the configure command. This implies UTF-8 support, even if you have

       not explicitly requested it.

       Including Unicode property support adds around 30K  of  tables  to  the

       PCRE  library.  Only  the general category properties such as Lu and Nd

       are supported. Details are given in the pcrepattern documentation.

CODE VALUE OF NEWLINE

       By default, PCRE interprets character 10 (linefeed, LF)  as  indicating

       the  end  of  a line. This is the normal newline character on Unix-like

       systems. You can compile PCRE to use character 13 (carriage return, CR)

       instead, by adding

         --enable-newline-is-cr

       to  the  configure  command.  There  is  also  a --enable-newline-is-lf

       option, which explicitly specifies linefeed as the newline character.

       Alternatively, you can specify that line endings are to be indicated by

       the two character sequence CRLF. If you want this, add

         --enable-newline-is-crlf

       to the configure command. There is a fourth option, specified by

         --enable-newline-is-anycrlf

       which  causes  PCRE  to recognize any of the three sequences CR, LF, or

       CRLF as indicating a line ending. Finally, a fifth option, specified by

         --enable-newline-is-any

       causes PCRE to recognize any Unicode newline sequence.

       Whatever  line  ending convention is selected when PCRE is built can be

       overridden when the library functions are called. At build time  it  is

       conventional to use the standard for your operating system.

WHAT \R MATCHES

       By  default,  the  sequence \R in a pattern matches any Unicode newline

       sequence, whatever has been selected as the line  ending  sequence.  If

       you specify

         --enable-bsr-anycrlf

       the  default  is changed so that \R matches only CR, LF, or CRLF. What-

       ever is selected when PCRE is built can be overridden when the  library

       functions are called.

BUILDING SHARED AND STATIC LIBRARIES

       The  PCRE building process uses libtool to build both shared and static

       Unix libraries by default. You can suppress one of these by adding  one

       of

         --disable-shared

         --disable-static

       to the configure command, as required.

POSIX MALLOC USAGE

       When PCRE is called through the POSIX interface (see the pcreposix doc-

       umentation), additional working storage is  required  for  holding  the

       pointers  to capturing substrings, because PCRE requires three integers

       per substring, whereas the POSIX interface provides only  two.  If  the

       number of expected substrings is small, the wrapper function uses space

       on the stack, because this is faster than using malloc() for each call.

       The default threshold above which the stack is no longer used is 10; it

       can be changed by adding a setting such as

         --with-posix-malloc-threshold=20

       to the configure command.

HANDLING VERY LARGE PATTERNS

       Within a compiled pattern, offset values are used  to  point  from  one

       part  to another (for example, from an opening parenthesis to an alter-

       nation metacharacter). By default, two-byte values are used  for  these

       offsets,  leading  to  a  maximum size for a compiled pattern of around

       64K. This is sufficient to handle all but the most  gigantic  patterns.

       Nevertheless,  some  people do want to process enormous patterns, so it

       is possible to compile PCRE to use three-byte or four-byte  offsets  by

       adding a setting such as

         --with-link-size=3

       to  the  configure  command.  The value given must be 2, 3, or 4. Using

       longer offsets slows down the operation of PCRE because it has to  load

       additional bytes when handling them.

AVOIDING EXCESSIVE STACK USAGE

       When matching with the pcre_exec() function, PCRE implements backtrack-

       ing by making recursive calls to an internal function  called  match().

       In  environments  where  the size of the stack is limited, this can se-

       verely limit PCRE's operation. (The Unix environment does  not  usually

       suffer from this problem, but it may sometimes be necessary to increase

       the maximum stack size.  There is a discussion in the  pcrestack  docu-

       mentation.)  An alternative approach to recursion that uses memory from

       the heap to remember data, instead of using recursive  function  calls,

       has  been  implemented to work round the problem of limited stack size.

       If you want to build a version of PCRE that works this way, add

         --disable-stack-for-recursion

       to the configure command. With this configuration, PCRE  will  use  the

       pcre_stack_malloc  and pcre_stack_free variables to call memory manage-

       ment functions. By default these point to malloc() and free(), but  you

       can replace the pointers so that your own functions are used.

       Separate  functions  are  provided  rather  than  using pcre_malloc and

       pcre_free because the  usage  is  very  predictable:  the  block  sizes

       requested  are  always  the  same,  and  the blocks are always freed in

       reverse order. A calling program might be able to  implement  optimized

       functions  that  perform  better  than  malloc()  and free(). PCRE runs

       noticeably more slowly when built in this way. This option affects only

       the   pcre_exec()   function;   it   is   not   relevant  for  the  the

       pcre_dfa_exec() function.

LIMITING PCRE RESOURCE USAGE

       Internally, PCRE has a function called match(), which it calls  repeat-

       edly   (sometimes   recursively)  when  matching  a  pattern  with  the

       pcre_exec() function. By controlling the maximum number of  times  this

       function  may be called during a single matching operation, a limit can

       be placed on the resources used by a single call  to  pcre_exec().  The

       limit  can be changed at run time, as described in the pcreapi documen-

       tation. The default is 10 million, but this can be changed by adding  a

       setting such as

         --with-match-limit=500000

       to   the   configure  command.  This  setting  has  no  effect  on  the

       pcre_dfa_exec() matching function.

       In some environments it is desirable to limit the  depth  of  recursive

       calls of match() more strictly than the total number of calls, in order

       to restrict the maximum amount of stack (or heap,  if  --disable-stack-

       for-recursion is specified) that is used. A second limit controls this;

       it defaults to the value that  is  set  for  --with-match-limit,  which

       imposes  no  additional constraints. However, you can set a lower limit

       by adding, for example,

         --with-match-limit-recursion=10000

       to the configure command. This value can  also  be  overridden  at  run

       time.

CREATING CHARACTER TABLES AT BUILD TIME

       PCRE  uses fixed tables for processing characters whose code values are

       less than 256. By default, PCRE is built with a set of tables that  are

       distributed  in  the  file pcre_chartables.c.dist. These tables are for

       ASCII codes only. If you add

         --enable-rebuild-chartables

       to the configure command, the distributed tables are  no  longer  used.

       Instead,  a  program  called dftables is compiled and run. This outputs

       the source for new set of tables, created in the default locale of your

       C runtime system. (This method of replacing the tables does not work if

       you are cross compiling, because dftables is run on the local host.  If

       you  need  to  create alternative tables when cross compiling, you will

       have to do so "by hand".)

USING EBCDIC CODE

       PCRE assumes by default that it will run in an  environment  where  the

       character  code  is  ASCII  (or Unicode, which is a superset of ASCII).

       This is the case for most computer operating systems.  PCRE  can,  how-

       ever, be compiled to run in an EBCDIC environment by adding

         --enable-ebcdic

       to the configure command. This setting implies --enable-rebuild-charta-

       bles. You should only use it if you know that  you  are  in  an  EBCDIC

       environment (for example, an IBM mainframe operating system).

PCREGREP OPTIONS FOR COMPRESSED FILE SUPPORT

       By default, pcregrep reads all files as plain text. You can build it so

       that it recognizes files whose names end in .gz or .bz2, and reads them

       with libz or libbz2, respectively, by adding one or both of

         --enable-pcregrep-libz

         --enable-pcregrep-libbz2