|
1 <html> |
|
2 <head> |
|
3 <title>pcrepattern specification</title> |
|
4 </head> |
|
5 <body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> |
|
6 <h1>pcrepattern man page</h1> |
|
7 <p> |
|
8 Return to the <a href="index.html">PCRE index page</a>. |
|
9 </p> |
|
10 <p> |
|
11 This page is part of the PCRE HTML documentation. It was generated automatically |
|
12 from the original man page. If there is any nonsense in it, please consult the |
|
13 man page, in case the conversion went wrong. |
|
14 <br> |
|
15 <ul> |
|
16 <li><a name="TOC1" href="#SEC1">PCRE REGULAR EXPRESSION DETAILS</a> |
|
17 <li><a name="TOC2" href="#SEC2">NEWLINE CONVENTIONS</a> |
|
18 <li><a name="TOC3" href="#SEC3">CHARACTERS AND METACHARACTERS</a> |
|
19 <li><a name="TOC4" href="#SEC4">BACKSLASH</a> |
|
20 <li><a name="TOC5" href="#SEC5">CIRCUMFLEX AND DOLLAR</a> |
|
21 <li><a name="TOC6" href="#SEC6">FULL STOP (PERIOD, DOT)</a> |
|
22 <li><a name="TOC7" href="#SEC7">MATCHING A SINGLE BYTE</a> |
|
23 <li><a name="TOC8" href="#SEC8">SQUARE BRACKETS AND CHARACTER CLASSES</a> |
|
24 <li><a name="TOC9" href="#SEC9">POSIX CHARACTER CLASSES</a> |
|
25 <li><a name="TOC10" href="#SEC10">VERTICAL BAR</a> |
|
26 <li><a name="TOC11" href="#SEC11">INTERNAL OPTION SETTING</a> |
|
27 <li><a name="TOC12" href="#SEC12">SUBPATTERNS</a> |
|
28 <li><a name="TOC13" href="#SEC13">DUPLICATE SUBPATTERN NUMBERS</a> |
|
29 <li><a name="TOC14" href="#SEC14">NAMED SUBPATTERNS</a> |
|
30 <li><a name="TOC15" href="#SEC15">REPETITION</a> |
|
31 <li><a name="TOC16" href="#SEC16">ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS</a> |
|
32 <li><a name="TOC17" href="#SEC17">BACK REFERENCES</a> |
|
33 <li><a name="TOC18" href="#SEC18">ASSERTIONS</a> |
|
34 <li><a name="TOC19" href="#SEC19">CONDITIONAL SUBPATTERNS</a> |
|
35 <li><a name="TOC20" href="#SEC20">COMMENTS</a> |
|
36 <li><a name="TOC21" href="#SEC21">RECURSIVE PATTERNS</a> |
|
37 <li><a name="TOC22" href="#SEC22">SUBPATTERNS AS SUBROUTINES</a> |
|
38 <li><a name="TOC23" href="#SEC23">ONIGURUMA SUBROUTINE SYNTAX</a> |
|
39 <li><a name="TOC24" href="#SEC24">CALLOUTS</a> |
|
40 <li><a name="TOC25" href="#SEC25">BACKTRACKING CONTROL</a> |
|
41 <li><a name="TOC26" href="#SEC26">SEE ALSO</a> |
|
42 <li><a name="TOC27" href="#SEC27">AUTHOR</a> |
|
43 <li><a name="TOC28" href="#SEC28">REVISION</a> |
|
44 </ul> |
|
45 <br><a name="SEC1" href="#TOC1">PCRE REGULAR EXPRESSION DETAILS</a><br> |
|
46 <P> |
|
47 The syntax and semantics of the regular expressions that are supported by PCRE |
|
48 are described in detail below. There is a quick-reference syntax summary in the |
|
49 <a href="pcresyntax.html"><b>pcresyntax</b></a> |
|
50 page. PCRE tries to match Perl syntax and semantics as closely as it can. PCRE |
|
51 also supports some alternative regular expression syntax (which does not |
|
52 conflict with the Perl syntax) in order to provide some compatibility with |
|
53 regular expressions in Python, .NET, and Oniguruma. |
|
54 </P> |
|
55 <P> |
|
56 Perl's regular expressions are described in its own documentation, and |
|
57 regular expressions in general are covered in a number of books, some of which |
|
58 have copious examples. Jeffrey Friedl's "Mastering Regular Expressions", |
|
59 published by O'Reilly, covers regular expressions in great detail. This |
|
60 description of PCRE's regular expressions is intended as reference material. |
|
61 </P> |
|
62 <P> |
|
63 The original operation of PCRE was on strings of one-byte characters. However, |
|
64 there is now also support for UTF-8 character strings. To use this, you must |
|
65 build PCRE to include UTF-8 support, and then call <b>pcre_compile()</b> with |
|
66 the PCRE_UTF8 option. How this affects pattern matching is mentioned in several |
|
67 places below. There is also a summary of UTF-8 features in the |
|
68 <a href="pcre.html#utf8support">section on UTF-8 support</a> |
|
69 in the main |
|
70 <a href="pcre.html"><b>pcre</b></a> |
|
71 page. |
|
72 </P> |
|
73 <P> |
|
74 The remainder of this document discusses the patterns that are supported by |
|
75 PCRE when its main matching function, <b>pcre_exec()</b>, is used. |
|
76 From release 6.0, PCRE offers a second matching function, |
|
77 <b>pcre_dfa_exec()</b>, which matches using a different algorithm that is not |
|
78 Perl-compatible. Some of the features discussed below are not available when |
|
79 <b>pcre_dfa_exec()</b> is used. The advantages and disadvantages of the |
|
80 alternative function, and how it differs from the normal function, are |
|
81 discussed in the |
|
82 <a href="pcrematching.html"><b>pcrematching</b></a> |
|
83 page. |
|
84 </P> |
|
85 <br><a name="SEC2" href="#TOC1">NEWLINE CONVENTIONS</a><br> |
|
86 <P> |
|
87 PCRE supports five different conventions for indicating line breaks in |
|
88 strings: a single CR (carriage return) character, a single LF (linefeed) |
|
89 character, the two-character sequence CRLF, any of the three preceding, or any |
|
90 Unicode newline sequence. The |
|
91 <a href="pcreapi.html"><b>pcreapi</b></a> |
|
92 page has |
|
93 <a href="pcreapi.html#newlines">further discussion</a> |
|
94 about newlines, and shows how to set the newline convention in the |
|
95 <i>options</i> arguments for the compiling and matching functions. |
|
96 </P> |
|
97 <P> |
|
98 It is also possible to specify a newline convention by starting a pattern |
|
99 string with one of the following five sequences: |
|
100 <pre> |
|
101 (*CR) carriage return |
|
102 (*LF) linefeed |
|
103 (*CRLF) carriage return, followed by linefeed |
|
104 (*ANYCRLF) any of the three above |
|
105 (*ANY) all Unicode newline sequences |
|
106 </pre> |
|
107 These override the default and the options given to <b>pcre_compile()</b>. For |
|
108 example, on a Unix system where LF is the default newline sequence, the pattern |
|
109 <pre> |
|
110 (*CR)a.b |
|
111 </pre> |
|
112 changes the convention to CR. That pattern matches "a\nb" because LF is no |
|
113 longer a newline. Note that these special settings, which are not |
|
114 Perl-compatible, are recognized only at the very start of a pattern, and that |
|
115 they must be in upper case. If more than one of them is present, the last one |
|
116 is used. |
|
117 </P> |
|
118 <P> |
|
119 The newline convention does not affect what the \R escape sequence matches. By |
|
120 default, this is any Unicode newline sequence, for Perl compatibility. However, |
|
121 this can be changed; see the description of \R in the section entitled |
|
122 <a href="#newlineseq">"Newline sequences"</a> |
|
123 below. A change of \R setting can be combined with a change of newline |
|
124 convention. |
|
125 </P> |
|
126 <br><a name="SEC3" href="#TOC1">CHARACTERS AND METACHARACTERS</a><br> |
|
127 <P> |
|
128 A regular expression is a pattern that is matched against a subject string from |
|
129 left to right. Most characters stand for themselves in a pattern, and match the |
|
130 corresponding characters in the subject. As a trivial example, the pattern |
|
131 <pre> |
|
132 The quick brown fox |
|
133 </pre> |
|
134 matches a portion of a subject string that is identical to itself. When |
|
135 caseless matching is specified (the PCRE_CASELESS option), letters are matched |
|
136 independently of case. In UTF-8 mode, PCRE always understands the concept of |
|
137 case for characters whose values are less than 128, so caseless matching is |
|
138 always possible. For characters with higher values, the concept of case is |
|
139 supported if PCRE is compiled with Unicode property support, but not otherwise. |
|
140 If you want to use caseless matching for characters 128 and above, you must |
|
141 ensure that PCRE is compiled with Unicode property support as well as with |
|
142 UTF-8 support. |
|
143 </P> |
|
144 <P> |
|
145 The power of regular expressions comes from the ability to include alternatives |
|
146 and repetitions in the pattern. These are encoded in the pattern by the use of |
|
147 <i>metacharacters</i>, which do not stand for themselves but instead are |
|
148 interpreted in some special way. |
|
149 </P> |
|
150 <P> |
|
151 There are two different sets of metacharacters: those that are recognized |
|
152 anywhere in the pattern except within square brackets, and those that are |
|
153 recognized within square brackets. Outside square brackets, the metacharacters |
|
154 are as follows: |
|
155 <pre> |
|
156 \ general escape character with several uses |
|
157 ^ assert start of string (or line, in multiline mode) |
|
158 $ assert end of string (or line, in multiline mode) |
|
159 . match any character except newline (by default) |
|
160 [ start character class definition |
|
161 | start of alternative branch |
|
162 ( start subpattern |
|
163 ) end subpattern |
|
164 ? extends the meaning of ( |
|
165 also 0 or 1 quantifier |
|
166 also quantifier minimizer |
|
167 * 0 or more quantifier |
|
168 + 1 or more quantifier |
|
169 also "possessive quantifier" |
|
170 { start min/max quantifier |
|
171 </pre> |
|
172 Part of a pattern that is in square brackets is called a "character class". In |
|
173 a character class the only metacharacters are: |
|
174 <pre> |
|
175 \ general escape character |
|
176 ^ negate the class, but only if the first character |
|
177 - indicates character range |
|
178 [ POSIX character class (only if followed by POSIX syntax) |
|
179 ] terminates the character class |
|
180 </pre> |
|
181 The following sections describe the use of each of the metacharacters. |
|
182 </P> |
|
183 <br><a name="SEC4" href="#TOC1">BACKSLASH</a><br> |
|
184 <P> |
|
185 The backslash character has several uses. Firstly, if it is followed by a |
|
186 non-alphanumeric character, it takes away any special meaning that character |
|
187 may have. This use of backslash as an escape character applies both inside and |
|
188 outside character classes. |
|
189 </P> |
|
190 <P> |
|
191 For example, if you want to match a * character, you write \* in the pattern. |
|
192 This escaping action applies whether or not the following character would |
|
193 otherwise be interpreted as a metacharacter, so it is always safe to precede a |
|
194 non-alphanumeric with backslash to specify that it stands for itself. In |
|
195 particular, if you want to match a backslash, you write \\. |
|
196 </P> |
|
197 <P> |
|
198 If a pattern is compiled with the PCRE_EXTENDED option, whitespace in the |
|
199 pattern (other than in a character class) and characters between a # outside |
|
200 a character class and the next newline are ignored. An escaping backslash can |
|
201 be used to include a whitespace or # character as part of the pattern. |
|
202 </P> |
|
203 <P> |
|
204 If you want to remove the special meaning from a sequence of characters, you |
|
205 can do so by putting them between \Q and \E. This is different from Perl in |
|
206 that $ and @ are handled as literals in \Q...\E sequences in PCRE, whereas in |
|
207 Perl, $ and @ cause variable interpolation. Note the following examples: |
|
208 <pre> |
|
209 Pattern PCRE matches Perl matches |
|
210 |
|
211 \Qabc$xyz\E abc$xyz abc followed by the contents of $xyz |
|
212 \Qabc\$xyz\E abc\$xyz abc\$xyz |
|
213 \Qabc\E\$\Qxyz\E abc$xyz abc$xyz |
|
214 </pre> |
|
215 The \Q...\E sequence is recognized both inside and outside character classes. |
|
216 <a name="digitsafterbackslash"></a></P> |
|
217 <br><b> |
|
218 Non-printing characters |
|
219 </b><br> |
|
220 <P> |
|
221 A second use of backslash provides a way of encoding non-printing characters |
|
222 in patterns in a visible manner. There is no restriction on the appearance of |
|
223 non-printing characters, apart from the binary zero that terminates a pattern, |
|
224 but when a pattern is being prepared by text editing, it is usually easier to |
|
225 use one of the following escape sequences than the binary character it |
|
226 represents: |
|
227 <pre> |
|
228 \a alarm, that is, the BEL character (hex 07) |
|
229 \cx "control-x", where x is any character |
|
230 \e escape (hex 1B) |
|
231 \f formfeed (hex 0C) |
|
232 \n linefeed (hex 0A) |
|
233 \r carriage return (hex 0D) |
|
234 \t tab (hex 09) |
|
235 \ddd character with octal code ddd, or backreference |
|
236 \xhh character with hex code hh |
|
237 \x{hhh..} character with hex code hhh.. |
|
238 </pre> |
|
239 The precise effect of \cx is as follows: if x is a lower case letter, it |
|
240 is converted to upper case. Then bit 6 of the character (hex 40) is inverted. |
|
241 Thus \cz becomes hex 1A, but \c{ becomes hex 3B, while \c; becomes hex |
|
242 7B. |
|
243 </P> |
|
244 <P> |
|
245 After \x, from zero to two hexadecimal digits are read (letters can be in |
|
246 upper or lower case). Any number of hexadecimal digits may appear between \x{ |
|
247 and }, but the value of the character code must be less than 256 in non-UTF-8 |
|
248 mode, and less than 2**31 in UTF-8 mode. That is, the maximum value in |
|
249 hexadecimal is 7FFFFFFF. Note that this is bigger than the largest Unicode code |
|
250 point, which is 10FFFF. |
|
251 </P> |
|
252 <P> |
|
253 If characters other than hexadecimal digits appear between \x{ and }, or if |
|
254 there is no terminating }, this form of escape is not recognized. Instead, the |
|
255 initial \x will be interpreted as a basic hexadecimal escape, with no |
|
256 following digits, giving a character whose value is zero. |
|
257 </P> |
|
258 <P> |
|
259 Characters whose value is less than 256 can be defined by either of the two |
|
260 syntaxes for \x. There is no difference in the way they are handled. For |
|
261 example, \xdc is exactly the same as \x{dc}. |
|
262 </P> |
|
263 <P> |
|
264 After \0 up to two further octal digits are read. If there are fewer than two |
|
265 digits, just those that are present are used. Thus the sequence \0\x\07 |
|
266 specifies two binary zeros followed by a BEL character (code value 7). Make |
|
267 sure you supply two digits after the initial zero if the pattern character that |
|
268 follows is itself an octal digit. |
|
269 </P> |
|
270 <P> |
|
271 The handling of a backslash followed by a digit other than 0 is complicated. |
|
272 Outside a character class, PCRE reads it and any following digits as a decimal |
|
273 number. If the number is less than 10, or if there have been at least that many |
|
274 previous capturing left parentheses in the expression, the entire sequence is |
|
275 taken as a <i>back reference</i>. A description of how this works is given |
|
276 <a href="#backreferences">later,</a> |
|
277 following the discussion of |
|
278 <a href="#subpattern">parenthesized subpatterns.</a> |
|
279 </P> |
|
280 <P> |
|
281 Inside a character class, or if the decimal number is greater than 9 and there |
|
282 have not been that many capturing subpatterns, PCRE re-reads up to three octal |
|
283 digits following the backslash, and uses them to generate a data character. Any |
|
284 subsequent digits stand for themselves. In non-UTF-8 mode, the value of a |
|
285 character specified in octal must be less than \400. In UTF-8 mode, values up |
|
286 to \777 are permitted. For example: |
|
287 <pre> |
|
288 \040 is another way of writing a space |
|
289 \40 is the same, provided there are fewer than 40 previous capturing subpatterns |
|
290 \7 is always a back reference |
|
291 \11 might be a back reference, or another way of writing a tab |
|
292 \011 is always a tab |
|
293 \0113 is a tab followed by the character "3" |
|
294 \113 might be a back reference, otherwise the character with octal code 113 |
|
295 \377 might be a back reference, otherwise the byte consisting entirely of 1 bits |
|
296 \81 is either a back reference, or a binary zero followed by the two characters "8" and "1" |
|
297 </pre> |
|
298 Note that octal values of 100 or greater must not be introduced by a leading |
|
299 zero, because no more than three octal digits are ever read. |
|
300 </P> |
|
301 <P> |
|
302 All the sequences that define a single character value can be used both inside |
|
303 and outside character classes. In addition, inside a character class, the |
|
304 sequence \b is interpreted as the backspace character (hex 08), and the |
|
305 sequences \R and \X are interpreted as the characters "R" and "X", |
|
306 respectively. Outside a character class, these sequences have different |
|
307 meanings |
|
308 <a href="#uniextseq">(see below).</a> |
|
309 </P> |
|
310 <br><b> |
|
311 Absolute and relative back references |
|
312 </b><br> |
|
313 <P> |
|
314 The sequence \g followed by an unsigned or a negative number, optionally |
|
315 enclosed in braces, is an absolute or relative back reference. A named back |
|
316 reference can be coded as \g{name}. Back references are discussed |
|
317 <a href="#backreferences">later,</a> |
|
318 following the discussion of |
|
319 <a href="#subpattern">parenthesized subpatterns.</a> |
|
320 </P> |
|
321 <br><b> |
|
322 Absolute and relative subroutine calls |
|
323 </b><br> |
|
324 <P> |
|
325 For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or |
|
326 a number enclosed either in angle brackets or single quotes, is an alternative |
|
327 syntax for referencing a subpattern as a "subroutine". Details are discussed |
|
328 <a href="#onigurumasubroutines">later.</a> |
|
329 Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are <i>not</i> |
|
330 synonymous. The former is a back reference; the latter is a subroutine call. |
|
331 </P> |
|
332 <br><b> |
|
333 Generic character types |
|
334 </b><br> |
|
335 <P> |
|
336 Another use of backslash is for specifying generic character types. The |
|
337 following are always recognized: |
|
338 <pre> |
|
339 \d any decimal digit |
|
340 \D any character that is not a decimal digit |
|
341 \h any horizontal whitespace character |
|
342 \H any character that is not a horizontal whitespace character |
|
343 \s any whitespace character |
|
344 \S any character that is not a whitespace character |
|
345 \v any vertical whitespace character |
|
346 \V any character that is not a vertical whitespace character |
|
347 \w any "word" character |
|
348 \W any "non-word" character |
|
349 </pre> |
|
350 Each pair of escape sequences partitions the complete set of characters into |
|
351 two disjoint sets. Any given character matches one, and only one, of each pair. |
|
352 </P> |
|
353 <P> |
|
354 These character type sequences can appear both inside and outside character |
|
355 classes. They each match one character of the appropriate type. If the current |
|
356 matching point is at the end of the subject string, all of them fail, since |
|
357 there is no character to match. |
|
358 </P> |
|
359 <P> |
|
360 For compatibility with Perl, \s does not match the VT character (code 11). |
|
361 This makes it different from the the POSIX "space" class. The \s characters |
|
362 are HT (9), LF (10), FF (12), CR (13), and space (32). If "use locale;" is |
|
363 included in a Perl script, \s may match the VT character. In PCRE, it never |
|
364 does. |
|
365 </P> |
|
366 <P> |
|
367 In UTF-8 mode, characters with values greater than 128 never match \d, \s, or |
|
368 \w, and always match \D, \S, and \W. This is true even when Unicode |
|
369 character property support is available. These sequences retain their original |
|
370 meanings from before UTF-8 support was available, mainly for efficiency |
|
371 reasons. |
|
372 </P> |
|
373 <P> |
|
374 The sequences \h, \H, \v, and \V are Perl 5.10 features. In contrast to the |
|
375 other sequences, these do match certain high-valued codepoints in UTF-8 mode. |
|
376 The horizontal space characters are: |
|
377 <pre> |
|
378 U+0009 Horizontal tab |
|
379 U+0020 Space |
|
380 U+00A0 Non-break space |
|
381 U+1680 Ogham space mark |
|
382 U+180E Mongolian vowel separator |
|
383 U+2000 En quad |
|
384 U+2001 Em quad |
|
385 U+2002 En space |
|
386 U+2003 Em space |
|
387 U+2004 Three-per-em space |
|
388 U+2005 Four-per-em space |
|
389 U+2006 Six-per-em space |
|
390 U+2007 Figure space |
|
391 U+2008 Punctuation space |
|
392 U+2009 Thin space |
|
393 U+200A Hair space |
|
394 U+202F Narrow no-break space |
|
395 U+205F Medium mathematical space |
|
396 U+3000 Ideographic space |
|
397 </pre> |
|
398 The vertical space characters are: |
|
399 <pre> |
|
400 U+000A Linefeed |
|
401 U+000B Vertical tab |
|
402 U+000C Formfeed |
|
403 U+000D Carriage return |
|
404 U+0085 Next line |
|
405 U+2028 Line separator |
|
406 U+2029 Paragraph separator |
|
407 </PRE> |
|
408 </P> |
|
409 <P> |
|
410 A "word" character is an underscore or any character less than 256 that is a |
|
411 letter or digit. The definition of letters and digits is controlled by PCRE's |
|
412 low-valued character tables, and may vary if locale-specific matching is taking |
|
413 place (see |
|
414 <a href="pcreapi.html#localesupport">"Locale support"</a> |
|
415 in the |
|
416 <a href="pcreapi.html"><b>pcreapi</b></a> |
|
417 page). For example, in a French locale such as "fr_FR" in Unix-like systems, |
|
418 or "french" in Windows, some character codes greater than 128 are used for |
|
419 accented letters, and these are matched by \w. The use of locales with Unicode |
|
420 is discouraged. |
|
421 <a name="newlineseq"></a></P> |
|
422 <br><b> |
|
423 Newline sequences |
|
424 </b><br> |
|
425 <P> |
|
426 Outside a character class, by default, the escape sequence \R matches any |
|
427 Unicode newline sequence. This is a Perl 5.10 feature. In non-UTF-8 mode \R is |
|
428 equivalent to the following: |
|
429 <pre> |
|
430 (?>\r\n|\n|\x0b|\f|\r|\x85) |
|
431 </pre> |
|
432 This is an example of an "atomic group", details of which are given |
|
433 <a href="#atomicgroup">below.</a> |
|
434 This particular group matches either the two-character sequence CR followed by |
|
435 LF, or one of the single characters LF (linefeed, U+000A), VT (vertical tab, |
|
436 U+000B), FF (formfeed, U+000C), CR (carriage return, U+000D), or NEL (next |
|
437 line, U+0085). The two-character sequence is treated as a single unit that |
|
438 cannot be split. |
|
439 </P> |
|
440 <P> |
|
441 In UTF-8 mode, two additional characters whose codepoints are greater than 255 |
|
442 are added: LS (line separator, U+2028) and PS (paragraph separator, U+2029). |
|
443 Unicode character property support is not needed for these characters to be |
|
444 recognized. |
|
445 </P> |
|
446 <P> |
|
447 It is possible to restrict \R to match only CR, LF, or CRLF (instead of the |
|
448 complete set of Unicode line endings) by setting the option PCRE_BSR_ANYCRLF |
|
449 either at compile time or when the pattern is matched. (BSR is an abbrevation |
|
450 for "backslash R".) This can be made the default when PCRE is built; if this is |
|
451 the case, the other behaviour can be requested via the PCRE_BSR_UNICODE option. |
|
452 It is also possible to specify these settings by starting a pattern string with |
|
453 one of the following sequences: |
|
454 <pre> |
|
455 (*BSR_ANYCRLF) CR, LF, or CRLF only |
|
456 (*BSR_UNICODE) any Unicode newline sequence |
|
457 </pre> |
|
458 These override the default and the options given to <b>pcre_compile()</b>, but |
|
459 they can be overridden by options given to <b>pcre_exec()</b>. Note that these |
|
460 special settings, which are not Perl-compatible, are recognized only at the |
|
461 very start of a pattern, and that they must be in upper case. If more than one |
|
462 of them is present, the last one is used. They can be combined with a change of |
|
463 newline convention, for example, a pattern can start with: |
|
464 <pre> |
|
465 (*ANY)(*BSR_ANYCRLF) |
|
466 </pre> |
|
467 Inside a character class, \R matches the letter "R". |
|
468 <a name="uniextseq"></a></P> |
|
469 <br><b> |
|
470 Unicode character properties |
|
471 </b><br> |
|
472 <P> |
|
473 When PCRE is built with Unicode character property support, three additional |
|
474 escape sequences that match characters with specific properties are available. |
|
475 When not in UTF-8 mode, these sequences are of course limited to testing |
|
476 characters whose codepoints are less than 256, but they do work in this mode. |
|
477 The extra escape sequences are: |
|
478 <pre> |
|
479 \p{<i>xx</i>} a character with the <i>xx</i> property |
|
480 \P{<i>xx</i>} a character without the <i>xx</i> property |
|
481 \X an extended Unicode sequence |
|
482 </pre> |
|
483 The property names represented by <i>xx</i> above are limited to the Unicode |
|
484 script names, the general category properties, and "Any", which matches any |
|
485 character (including newline). Other properties such as "InMusicalSymbols" are |
|
486 not currently supported by PCRE. Note that \P{Any} does not match any |
|
487 characters, so always causes a match failure. |
|
488 </P> |
|
489 <P> |
|
490 Sets of Unicode characters are defined as belonging to certain scripts. A |
|
491 character from one of these sets can be matched using a script name. For |
|
492 example: |
|
493 <pre> |
|
494 \p{Greek} |
|
495 \P{Han} |
|
496 </pre> |
|
497 Those that are not part of an identified script are lumped together as |
|
498 "Common". The current list of scripts is: |
|
499 </P> |
|
500 <P> |
|
501 Arabic, |
|
502 Armenian, |
|
503 Balinese, |
|
504 Bengali, |
|
505 Bopomofo, |
|
506 Braille, |
|
507 Buginese, |
|
508 Buhid, |
|
509 Canadian_Aboriginal, |
|
510 Cherokee, |
|
511 Common, |
|
512 Coptic, |
|
513 Cuneiform, |
|
514 Cypriot, |
|
515 Cyrillic, |
|
516 Deseret, |
|
517 Devanagari, |
|
518 Ethiopic, |
|
519 Georgian, |
|
520 Glagolitic, |
|
521 Gothic, |
|
522 Greek, |
|
523 Gujarati, |
|
524 Gurmukhi, |
|
525 Han, |
|
526 Hangul, |
|
527 Hanunoo, |
|
528 Hebrew, |
|
529 Hiragana, |
|
530 Inherited, |
|
531 Kannada, |
|
532 Katakana, |
|
533 Kharoshthi, |
|
534 Khmer, |
|
535 Lao, |
|
536 Latin, |
|
537 Limbu, |
|
538 Linear_B, |
|
539 Malayalam, |
|
540 Mongolian, |
|
541 Myanmar, |
|
542 New_Tai_Lue, |
|
543 Nko, |
|
544 Ogham, |
|
545 Old_Italic, |
|
546 Old_Persian, |
|
547 Oriya, |
|
548 Osmanya, |
|
549 Phags_Pa, |
|
550 Phoenician, |
|
551 Runic, |
|
552 Shavian, |
|
553 Sinhala, |
|
554 Syloti_Nagri, |
|
555 Syriac, |
|
556 Tagalog, |
|
557 Tagbanwa, |
|
558 Tai_Le, |
|
559 Tamil, |
|
560 Telugu, |
|
561 Thaana, |
|
562 Thai, |
|
563 Tibetan, |
|
564 Tifinagh, |
|
565 Ugaritic, |
|
566 Yi. |
|
567 </P> |
|
568 <P> |
|
569 Each character has exactly one general category property, specified by a |
|
570 two-letter abbreviation. For compatibility with Perl, negation can be specified |
|
571 by including a circumflex between the opening brace and the property name. For |
|
572 example, \p{^Lu} is the same as \P{Lu}. |
|
573 </P> |
|
574 <P> |
|
575 If only one letter is specified with \p or \P, it includes all the general |
|
576 category properties that start with that letter. In this case, in the absence |
|
577 of negation, the curly brackets in the escape sequence are optional; these two |
|
578 examples have the same effect: |
|
579 <pre> |
|
580 \p{L} |
|
581 \pL |
|
582 </pre> |
|
583 The following general category property codes are supported: |
|
584 <pre> |
|
585 C Other |
|
586 Cc Control |
|
587 Cf Format |
|
588 Cn Unassigned |
|
589 Co Private use |
|
590 Cs Surrogate |
|
591 |
|
592 L Letter |
|
593 Ll Lower case letter |
|
594 Lm Modifier letter |
|
595 Lo Other letter |
|
596 Lt Title case letter |
|
597 Lu Upper case letter |
|
598 |
|
599 M Mark |
|
600 Mc Spacing mark |
|
601 Me Enclosing mark |
|
602 Mn Non-spacing mark |
|
603 |
|
604 N Number |
|
605 Nd Decimal number |
|
606 Nl Letter number |
|
607 No Other number |
|
608 |
|
609 P Punctuation |
|
610 Pc Connector punctuation |
|
611 Pd Dash punctuation |
|
612 Pe Close punctuation |
|
613 Pf Final punctuation |
|
614 Pi Initial punctuation |
|
615 Po Other punctuation |
|
616 Ps Open punctuation |
|
617 |
|
618 S Symbol |
|
619 Sc Currency symbol |
|
620 Sk Modifier symbol |
|
621 Sm Mathematical symbol |
|
622 So Other symbol |
|
623 |
|
624 Z Separator |
|
625 Zl Line separator |
|
626 Zp Paragraph separator |
|
627 Zs Space separator |
|
628 </pre> |
|
629 The special property L& is also supported: it matches a character that has |
|
630 the Lu, Ll, or Lt property, in other words, a letter that is not classified as |
|
631 a modifier or "other". |
|
632 </P> |
|
633 <P> |
|
634 The Cs (Surrogate) property applies only to characters in the range U+D800 to |
|
635 U+DFFF. Such characters are not valid in UTF-8 strings (see RFC 3629) and so |
|
636 cannot be tested by PCRE, unless UTF-8 validity checking has been turned off |
|
637 (see the discussion of PCRE_NO_UTF8_CHECK in the |
|
638 <a href="pcreapi.html"><b>pcreapi</b></a> |
|
639 page). |
|
640 </P> |
|
641 <P> |
|
642 The long synonyms for these properties that Perl supports (such as \p{Letter}) |
|
643 are not supported by PCRE, nor is it permitted to prefix any of these |
|
644 properties with "Is". |
|
645 </P> |
|
646 <P> |
|
647 No character that is in the Unicode table has the Cn (unassigned) property. |
|
648 Instead, this property is assumed for any code point that is not in the |
|
649 Unicode table. |
|
650 </P> |
|
651 <P> |
|
652 Specifying caseless matching does not affect these escape sequences. For |
|
653 example, \p{Lu} always matches only upper case letters. |
|
654 </P> |
|
655 <P> |
|
656 The \X escape matches any number of Unicode characters that form an extended |
|
657 Unicode sequence. \X is equivalent to |
|
658 <pre> |
|
659 (?>\PM\pM*) |
|
660 </pre> |
|
661 That is, it matches a character without the "mark" property, followed by zero |
|
662 or more characters with the "mark" property, and treats the sequence as an |
|
663 atomic group |
|
664 <a href="#atomicgroup">(see below).</a> |
|
665 Characters with the "mark" property are typically accents that affect the |
|
666 preceding character. None of them have codepoints less than 256, so in |
|
667 non-UTF-8 mode \X matches any one character. |
|
668 </P> |
|
669 <P> |
|
670 Matching characters by Unicode property is not fast, because PCRE has to search |
|
671 a structure that contains data for over fifteen thousand characters. That is |
|
672 why the traditional escape sequences such as \d and \w do not use Unicode |
|
673 properties in PCRE. |
|
674 <a name="resetmatchstart"></a></P> |
|
675 <br><b> |
|
676 Resetting the match start |
|
677 </b><br> |
|
678 <P> |
|
679 The escape sequence \K, which is a Perl 5.10 feature, causes any previously |
|
680 matched characters not to be included in the final matched sequence. For |
|
681 example, the pattern: |
|
682 <pre> |
|
683 foo\Kbar |
|
684 </pre> |
|
685 matches "foobar", but reports that it has matched "bar". This feature is |
|
686 similar to a lookbehind assertion |
|
687 <a href="#lookbehind">(described below).</a> |
|
688 However, in this case, the part of the subject before the real match does not |
|
689 have to be of fixed length, as lookbehind assertions do. The use of \K does |
|
690 not interfere with the setting of |
|
691 <a href="#subpattern">captured substrings.</a> |
|
692 For example, when the pattern |
|
693 <pre> |
|
694 (foo)\Kbar |
|
695 </pre> |
|
696 matches "foobar", the first substring is still set to "foo". |
|
697 <a name="smallassertions"></a></P> |
|
698 <br><b> |
|
699 Simple assertions |
|
700 </b><br> |
|
701 <P> |
|
702 The final use of backslash is for certain simple assertions. An assertion |
|
703 specifies a condition that has to be met at a particular point in a match, |
|
704 without consuming any characters from the subject string. The use of |
|
705 subpatterns for more complicated assertions is described |
|
706 <a href="#bigassertions">below.</a> |
|
707 The backslashed assertions are: |
|
708 <pre> |
|
709 \b matches at a word boundary |
|
710 \B matches when not at a word boundary |
|
711 \A matches at the start of the subject |
|
712 \Z matches at the end of the subject |
|
713 also matches before a newline at the end of the subject |
|
714 \z matches only at the end of the subject |
|
715 \G matches at the first matching position in the subject |
|
716 </pre> |
|
717 These assertions may not appear in character classes (but note that \b has a |
|
718 different meaning, namely the backspace character, inside a character class). |
|
719 </P> |
|
720 <P> |
|
721 A word boundary is a position in the subject string where the current character |
|
722 and the previous character do not both match \w or \W (i.e. one matches |
|
723 \w and the other matches \W), or the start or end of the string if the |
|
724 first or last character matches \w, respectively. |
|
725 </P> |
|
726 <P> |
|
727 The \A, \Z, and \z assertions differ from the traditional circumflex and |
|
728 dollar (described in the next section) in that they only ever match at the very |
|
729 start and end of the subject string, whatever options are set. Thus, they are |
|
730 independent of multiline mode. These three assertions are not affected by the |
|
731 PCRE_NOTBOL or PCRE_NOTEOL options, which affect only the behaviour of the |
|
732 circumflex and dollar metacharacters. However, if the <i>startoffset</i> |
|
733 argument of <b>pcre_exec()</b> is non-zero, indicating that matching is to start |
|
734 at a point other than the beginning of the subject, \A can never match. The |
|
735 difference between \Z and \z is that \Z matches before a newline at the end |
|
736 of the string as well as at the very end, whereas \z matches only at the end. |
|
737 </P> |
|
738 <P> |
|
739 The \G assertion is true only when the current matching position is at the |
|
740 start point of the match, as specified by the <i>startoffset</i> argument of |
|
741 <b>pcre_exec()</b>. It differs from \A when the value of <i>startoffset</i> is |
|
742 non-zero. By calling <b>pcre_exec()</b> multiple times with appropriate |
|
743 arguments, you can mimic Perl's /g option, and it is in this kind of |
|
744 implementation where \G can be useful. |
|
745 </P> |
|
746 <P> |
|
747 Note, however, that PCRE's interpretation of \G, as the start of the current |
|
748 match, is subtly different from Perl's, which defines it as the end of the |
|
749 previous match. In Perl, these can be different when the previously matched |
|
750 string was empty. Because PCRE does just one match at a time, it cannot |
|
751 reproduce this behaviour. |
|
752 </P> |
|
753 <P> |
|
754 If all the alternatives of a pattern begin with \G, the expression is anchored |
|
755 to the starting match position, and the "anchored" flag is set in the compiled |
|
756 regular expression. |
|
757 </P> |
|
758 <br><a name="SEC5" href="#TOC1">CIRCUMFLEX AND DOLLAR</a><br> |
|
759 <P> |
|
760 Outside a character class, in the default matching mode, the circumflex |
|
761 character is an assertion that is true only if the current matching point is |
|
762 at the start of the subject string. If the <i>startoffset</i> argument of |
|
763 <b>pcre_exec()</b> is non-zero, circumflex can never match if the PCRE_MULTILINE |
|
764 option is unset. Inside a character class, circumflex has an entirely different |
|
765 meaning |
|
766 <a href="#characterclass">(see below).</a> |
|
767 </P> |
|
768 <P> |
|
769 Circumflex need not be the first character of the pattern if a number of |
|
770 alternatives are involved, but it should be the first thing in each alternative |
|
771 in which it appears if the pattern is ever to match that branch. If all |
|
772 possible alternatives start with a circumflex, that is, if the pattern is |
|
773 constrained to match only at the start of the subject, it is said to be an |
|
774 "anchored" pattern. (There are also other constructs that can cause a pattern |
|
775 to be anchored.) |
|
776 </P> |
|
777 <P> |
|
778 A dollar character is an assertion that is true only if the current matching |
|
779 point is at the end of the subject string, or immediately before a newline |
|
780 at the end of the string (by default). Dollar need not be the last character of |
|
781 the pattern if a number of alternatives are involved, but it should be the last |
|
782 item in any branch in which it appears. Dollar has no special meaning in a |
|
783 character class. |
|
784 </P> |
|
785 <P> |
|
786 The meaning of dollar can be changed so that it matches only at the very end of |
|
787 the string, by setting the PCRE_DOLLAR_ENDONLY option at compile time. This |
|
788 does not affect the \Z assertion. |
|
789 </P> |
|
790 <P> |
|
791 The meanings of the circumflex and dollar characters are changed if the |
|
792 PCRE_MULTILINE option is set. When this is the case, a circumflex matches |
|
793 immediately after internal newlines as well as at the start of the subject |
|
794 string. It does not match after a newline that ends the string. A dollar |
|
795 matches before any newlines in the string, as well as at the very end, when |
|
796 PCRE_MULTILINE is set. When newline is specified as the two-character |
|
797 sequence CRLF, isolated CR and LF characters do not indicate newlines. |
|
798 </P> |
|
799 <P> |
|
800 For example, the pattern /^abc$/ matches the subject string "def\nabc" (where |
|
801 \n represents a newline) in multiline mode, but not otherwise. Consequently, |
|
802 patterns that are anchored in single line mode because all branches start with |
|
803 ^ are not anchored in multiline mode, and a match for circumflex is possible |
|
804 when the <i>startoffset</i> argument of <b>pcre_exec()</b> is non-zero. The |
|
805 PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. |
|
806 </P> |
|
807 <P> |
|
808 Note that the sequences \A, \Z, and \z can be used to match the start and |
|
809 end of the subject in both modes, and if all branches of a pattern start with |
|
810 \A it is always anchored, whether or not PCRE_MULTILINE is set. |
|
811 </P> |
|
812 <br><a name="SEC6" href="#TOC1">FULL STOP (PERIOD, DOT)</a><br> |
|
813 <P> |
|
814 Outside a character class, a dot in the pattern matches any one character in |
|
815 the subject string except (by default) a character that signifies the end of a |
|
816 line. In UTF-8 mode, the matched character may be more than one byte long. |
|
817 </P> |
|
818 <P> |
|
819 When a line ending is defined as a single character, dot never matches that |
|
820 character; when the two-character sequence CRLF is used, dot does not match CR |
|
821 if it is immediately followed by LF, but otherwise it matches all characters |
|
822 (including isolated CRs and LFs). When any Unicode line endings are being |
|
823 recognized, dot does not match CR or LF or any of the other line ending |
|
824 characters. |
|
825 </P> |
|
826 <P> |
|
827 The behaviour of dot with regard to newlines can be changed. If the PCRE_DOTALL |
|
828 option is set, a dot matches any one character, without exception. If the |
|
829 two-character sequence CRLF is present in the subject string, it takes two dots |
|
830 to match it. |
|
831 </P> |
|
832 <P> |
|
833 The handling of dot is entirely independent of the handling of circumflex and |
|
834 dollar, the only relationship being that they both involve newlines. Dot has no |
|
835 special meaning in a character class. |
|
836 </P> |
|
837 <br><a name="SEC7" href="#TOC1">MATCHING A SINGLE BYTE</a><br> |
|
838 <P> |
|
839 Outside a character class, the escape sequence \C matches any one byte, both |
|
840 in and out of UTF-8 mode. Unlike a dot, it always matches any line-ending |
|
841 characters. The feature is provided in Perl in order to match individual bytes |
|
842 in UTF-8 mode. Because it breaks up UTF-8 characters into individual bytes, |
|
843 what remains in the string may be a malformed UTF-8 string. For this reason, |
|
844 the \C escape sequence is best avoided. |
|
845 </P> |
|
846 <P> |
|
847 PCRE does not allow \C to appear in lookbehind assertions |
|
848 <a href="#lookbehind">(described below),</a> |
|
849 because in UTF-8 mode this would make it impossible to calculate the length of |
|
850 the lookbehind. |
|
851 <a name="characterclass"></a></P> |
|
852 <br><a name="SEC8" href="#TOC1">SQUARE BRACKETS AND CHARACTER CLASSES</a><br> |
|
853 <P> |
|
854 An opening square bracket introduces a character class, terminated by a closing |
|
855 square bracket. A closing square bracket on its own is not special. If a |
|
856 closing square bracket is required as a member of the class, it should be the |
|
857 first data character in the class (after an initial circumflex, if present) or |
|
858 escaped with a backslash. |
|
859 </P> |
|
860 <P> |
|
861 A character class matches a single character in the subject. In UTF-8 mode, the |
|
862 character may occupy more than one byte. A matched character must be in the set |
|
863 of characters defined by the class, unless the first character in the class |
|
864 definition is a circumflex, in which case the subject character must not be in |
|
865 the set defined by the class. If a circumflex is actually required as a member |
|
866 of the class, ensure it is not the first character, or escape it with a |
|
867 backslash. |
|
868 </P> |
|
869 <P> |
|
870 For example, the character class [aeiou] matches any lower case vowel, while |
|
871 [^aeiou] matches any character that is not a lower case vowel. Note that a |
|
872 circumflex is just a convenient notation for specifying the characters that |
|
873 are in the class by enumerating those that are not. A class that starts with a |
|
874 circumflex is not an assertion: it still consumes a character from the subject |
|
875 string, and therefore it fails if the current pointer is at the end of the |
|
876 string. |
|
877 </P> |
|
878 <P> |
|
879 In UTF-8 mode, characters with values greater than 255 can be included in a |
|
880 class as a literal string of bytes, or by using the \x{ escaping mechanism. |
|
881 </P> |
|
882 <P> |
|
883 When caseless matching is set, any letters in a class represent both their |
|
884 upper case and lower case versions, so for example, a caseless [aeiou] matches |
|
885 "A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a |
|
886 caseful version would. In UTF-8 mode, PCRE always understands the concept of |
|
887 case for characters whose values are less than 128, so caseless matching is |
|
888 always possible. For characters with higher values, the concept of case is |
|
889 supported if PCRE is compiled with Unicode property support, but not otherwise. |
|
890 If you want to use caseless matching for characters 128 and above, you must |
|
891 ensure that PCRE is compiled with Unicode property support as well as with |
|
892 UTF-8 support. |
|
893 </P> |
|
894 <P> |
|
895 Characters that might indicate line breaks are never treated in any special way |
|
896 when matching character classes, whatever line-ending sequence is in use, and |
|
897 whatever setting of the PCRE_DOTALL and PCRE_MULTILINE options is used. A class |
|
898 such as [^a] always matches one of these characters. |
|
899 </P> |
|
900 <P> |
|
901 The minus (hyphen) character can be used to specify a range of characters in a |
|
902 character class. For example, [d-m] matches any letter between d and m, |
|
903 inclusive. If a minus character is required in a class, it must be escaped with |
|
904 a backslash or appear in a position where it cannot be interpreted as |
|
905 indicating a range, typically as the first or last character in the class. |
|
906 </P> |
|
907 <P> |
|
908 It is not possible to have the literal character "]" as the end character of a |
|
909 range. A pattern such as [W-]46] is interpreted as a class of two characters |
|
910 ("W" and "-") followed by a literal string "46]", so it would match "W46]" or |
|
911 "-46]". However, if the "]" is escaped with a backslash it is interpreted as |
|
912 the end of range, so [W-\]46] is interpreted as a class containing a range |
|
913 followed by two other characters. The octal or hexadecimal representation of |
|
914 "]" can also be used to end a range. |
|
915 </P> |
|
916 <P> |
|
917 Ranges operate in the collating sequence of character values. They can also be |
|
918 used for characters specified numerically, for example [\000-\037]. In UTF-8 |
|
919 mode, ranges can include characters whose values are greater than 255, for |
|
920 example [\x{100}-\x{2ff}]. |
|
921 </P> |
|
922 <P> |
|
923 If a range that includes letters is used when caseless matching is set, it |
|
924 matches the letters in either case. For example, [W-c] is equivalent to |
|
925 [][\\^_`wxyzabc], matched caselessly, and in non-UTF-8 mode, if character |
|
926 tables for a French locale are in use, [\xc8-\xcb] matches accented E |
|
927 characters in both cases. In UTF-8 mode, PCRE supports the concept of case for |
|
928 characters with values greater than 128 only when it is compiled with Unicode |
|
929 property support. |
|
930 </P> |
|
931 <P> |
|
932 The character types \d, \D, \p, \P, \s, \S, \w, and \W may also appear |
|
933 in a character class, and add the characters that they match to the class. For |
|
934 example, [\dABCDEF] matches any hexadecimal digit. A circumflex can |
|
935 conveniently be used with the upper case character types to specify a more |
|
936 restricted set of characters than the matching lower case type. For example, |
|
937 the class [^\W_] matches any letter or digit, but not underscore. |
|
938 </P> |
|
939 <P> |
|
940 The only metacharacters that are recognized in character classes are backslash, |
|
941 hyphen (only where it can be interpreted as specifying a range), circumflex |
|
942 (only at the start), opening square bracket (only when it can be interpreted as |
|
943 introducing a POSIX class name - see the next section), and the terminating |
|
944 closing square bracket. However, escaping other non-alphanumeric characters |
|
945 does no harm. |
|
946 </P> |
|
947 <br><a name="SEC9" href="#TOC1">POSIX CHARACTER CLASSES</a><br> |
|
948 <P> |
|
949 Perl supports the POSIX notation for character classes. This uses names |
|
950 enclosed by [: and :] within the enclosing square brackets. PCRE also supports |
|
951 this notation. For example, |
|
952 <pre> |
|
953 [01[:alpha:]%] |
|
954 </pre> |
|
955 matches "0", "1", any alphabetic character, or "%". The supported class names |
|
956 are |
|
957 <pre> |
|
958 alnum letters and digits |
|
959 alpha letters |
|
960 ascii character codes 0 - 127 |
|
961 blank space or tab only |
|
962 cntrl control characters |
|
963 digit decimal digits (same as \d) |
|
964 graph printing characters, excluding space |
|
965 lower lower case letters |
|
966 print printing characters, including space |
|
967 punct printing characters, excluding letters and digits |
|
968 space white space (not quite the same as \s) |
|
969 upper upper case letters |
|
970 word "word" characters (same as \w) |
|
971 xdigit hexadecimal digits |
|
972 </pre> |
|
973 The "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), and |
|
974 space (32). Notice that this list includes the VT character (code 11). This |
|
975 makes "space" different to \s, which does not include VT (for Perl |
|
976 compatibility). |
|
977 </P> |
|
978 <P> |
|
979 The name "word" is a Perl extension, and "blank" is a GNU extension from Perl |
|
980 5.8. Another Perl extension is negation, which is indicated by a ^ character |
|
981 after the colon. For example, |
|
982 <pre> |
|
983 [12[:^digit:]] |
|
984 </pre> |
|
985 matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the POSIX |
|
986 syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not |
|
987 supported, and an error is given if they are encountered. |
|
988 </P> |
|
989 <P> |
|
990 In UTF-8 mode, characters with values greater than 128 do not match any of |
|
991 the POSIX character classes. |
|
992 </P> |
|
993 <br><a name="SEC10" href="#TOC1">VERTICAL BAR</a><br> |
|
994 <P> |
|
995 Vertical bar characters are used to separate alternative patterns. For example, |
|
996 the pattern |
|
997 <pre> |
|
998 gilbert|sullivan |
|
999 </pre> |
|
1000 matches either "gilbert" or "sullivan". Any number of alternatives may appear, |
|
1001 and an empty alternative is permitted (matching the empty string). The matching |
|
1002 process tries each alternative in turn, from left to right, and the first one |
|
1003 that succeeds is used. If the alternatives are within a subpattern |
|
1004 <a href="#subpattern">(defined below),</a> |
|
1005 "succeeds" means matching the rest of the main pattern as well as the |
|
1006 alternative in the subpattern. |
|
1007 </P> |
|
1008 <br><a name="SEC11" href="#TOC1">INTERNAL OPTION SETTING</a><br> |
|
1009 <P> |
|
1010 The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and |
|
1011 PCRE_EXTENDED options (which are Perl-compatible) can be changed from within |
|
1012 the pattern by a sequence of Perl option letters enclosed between "(?" and ")". |
|
1013 The option letters are |
|
1014 <pre> |
|
1015 i for PCRE_CASELESS |
|
1016 m for PCRE_MULTILINE |
|
1017 s for PCRE_DOTALL |
|
1018 x for PCRE_EXTENDED |
|
1019 </pre> |
|
1020 For example, (?im) sets caseless, multiline matching. It is also possible to |
|
1021 unset these options by preceding the letter with a hyphen, and a combined |
|
1022 setting and unsetting such as (?im-sx), which sets PCRE_CASELESS and |
|
1023 PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, is also |
|
1024 permitted. If a letter appears both before and after the hyphen, the option is |
|
1025 unset. |
|
1026 </P> |
|
1027 <P> |
|
1028 The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA can be |
|
1029 changed in the same way as the Perl-compatible options by using the characters |
|
1030 J, U and X respectively. |
|
1031 </P> |
|
1032 <P> |
|
1033 When an option change occurs at top level (that is, not inside subpattern |
|
1034 parentheses), the change applies to the remainder of the pattern that follows. |
|
1035 If the change is placed right at the start of a pattern, PCRE extracts it into |
|
1036 the global options (and it will therefore show up in data extracted by the |
|
1037 <b>pcre_fullinfo()</b> function). |
|
1038 </P> |
|
1039 <P> |
|
1040 An option change within a subpattern (see below for a description of |
|
1041 subpatterns) affects only that part of the current pattern that follows it, so |
|
1042 <pre> |
|
1043 (a(?i)b)c |
|
1044 </pre> |
|
1045 matches abc and aBc and no other strings (assuming PCRE_CASELESS is not used). |
|
1046 By this means, options can be made to have different settings in different |
|
1047 parts of the pattern. Any changes made in one alternative do carry on |
|
1048 into subsequent branches within the same subpattern. For example, |
|
1049 <pre> |
|
1050 (a(?i)b|c) |
|
1051 </pre> |
|
1052 matches "ab", "aB", "c", and "C", even though when matching "C" the first |
|
1053 branch is abandoned before the option setting. This is because the effects of |
|
1054 option settings happen at compile time. There would be some very weird |
|
1055 behaviour otherwise. |
|
1056 </P> |
|
1057 <P> |
|
1058 <b>Note:</b> There are other PCRE-specific options that can be set by the |
|
1059 application when the compile or match functions are called. In some cases the |
|
1060 pattern can contain special leading sequences to override what the application |
|
1061 has set or what has been defaulted. Details are given in the section entitled |
|
1062 <a href="#newlineseq">"Newline sequences"</a> |
|
1063 above. |
|
1064 <a name="subpattern"></a></P> |
|
1065 <br><a name="SEC12" href="#TOC1">SUBPATTERNS</a><br> |
|
1066 <P> |
|
1067 Subpatterns are delimited by parentheses (round brackets), which can be nested. |
|
1068 Turning part of a pattern into a subpattern does two things: |
|
1069 <br> |
|
1070 <br> |
|
1071 1. It localizes a set of alternatives. For example, the pattern |
|
1072 <pre> |
|
1073 cat(aract|erpillar|) |
|
1074 </pre> |
|
1075 matches one of the words "cat", "cataract", or "caterpillar". Without the |
|
1076 parentheses, it would match "cataract", "erpillar" or an empty string. |
|
1077 <br> |
|
1078 <br> |
|
1079 2. It sets up the subpattern as a capturing subpattern. This means that, when |
|
1080 the whole pattern matches, that portion of the subject string that matched the |
|
1081 subpattern is passed back to the caller via the <i>ovector</i> argument of |
|
1082 <b>pcre_exec()</b>. Opening parentheses are counted from left to right (starting |
|
1083 from 1) to obtain numbers for the capturing subpatterns. |
|
1084 </P> |
|
1085 <P> |
|
1086 For example, if the string "the red king" is matched against the pattern |
|
1087 <pre> |
|
1088 the ((red|white) (king|queen)) |
|
1089 </pre> |
|
1090 the captured substrings are "red king", "red", and "king", and are numbered 1, |
|
1091 2, and 3, respectively. |
|
1092 </P> |
|
1093 <P> |
|
1094 The fact that plain parentheses fulfil two functions is not always helpful. |
|
1095 There are often times when a grouping subpattern is required without a |
|
1096 capturing requirement. If an opening parenthesis is followed by a question mark |
|
1097 and a colon, the subpattern does not do any capturing, and is not counted when |
|
1098 computing the number of any subsequent capturing subpatterns. For example, if |
|
1099 the string "the white queen" is matched against the pattern |
|
1100 <pre> |
|
1101 the ((?:red|white) (king|queen)) |
|
1102 </pre> |
|
1103 the captured substrings are "white queen" and "queen", and are numbered 1 and |
|
1104 2. The maximum number of capturing subpatterns is 65535. |
|
1105 </P> |
|
1106 <P> |
|
1107 As a convenient shorthand, if any option settings are required at the start of |
|
1108 a non-capturing subpattern, the option letters may appear between the "?" and |
|
1109 the ":". Thus the two patterns |
|
1110 <pre> |
|
1111 (?i:saturday|sunday) |
|
1112 (?:(?i)saturday|sunday) |
|
1113 </pre> |
|
1114 match exactly the same set of strings. Because alternative branches are tried |
|
1115 from left to right, and options are not reset until the end of the subpattern |
|
1116 is reached, an option setting in one branch does affect subsequent branches, so |
|
1117 the above patterns match "SUNDAY" as well as "Saturday". |
|
1118 </P> |
|
1119 <br><a name="SEC13" href="#TOC1">DUPLICATE SUBPATTERN NUMBERS</a><br> |
|
1120 <P> |
|
1121 Perl 5.10 introduced a feature whereby each alternative in a subpattern uses |
|
1122 the same numbers for its capturing parentheses. Such a subpattern starts with |
|
1123 (?| and is itself a non-capturing subpattern. For example, consider this |
|
1124 pattern: |
|
1125 <pre> |
|
1126 (?|(Sat)ur|(Sun))day |
|
1127 </pre> |
|
1128 Because the two alternatives are inside a (?| group, both sets of capturing |
|
1129 parentheses are numbered one. Thus, when the pattern matches, you can look |
|
1130 at captured substring number one, whichever alternative matched. This construct |
|
1131 is useful when you want to capture part, but not all, of one of a number of |
|
1132 alternatives. Inside a (?| group, parentheses are numbered as usual, but the |
|
1133 number is reset at the start of each branch. The numbers of any capturing |
|
1134 buffers that follow the subpattern start after the highest number used in any |
|
1135 branch. The following example is taken from the Perl documentation. |
|
1136 The numbers underneath show in which buffer the captured content will be |
|
1137 stored. |
|
1138 <pre> |
|
1139 # before ---------------branch-reset----------- after |
|
1140 / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x |
|
1141 # 1 2 2 3 2 3 4 |
|
1142 </pre> |
|
1143 A backreference or a recursive call to a numbered subpattern always refers to |
|
1144 the first one in the pattern with the given number. |
|
1145 </P> |
|
1146 <P> |
|
1147 An alternative approach to using this "branch reset" feature is to use |
|
1148 duplicate named subpatterns, as described in the next section. |
|
1149 </P> |
|
1150 <br><a name="SEC14" href="#TOC1">NAMED SUBPATTERNS</a><br> |
|
1151 <P> |
|
1152 Identifying capturing parentheses by number is simple, but it can be very hard |
|
1153 to keep track of the numbers in complicated regular expressions. Furthermore, |
|
1154 if an expression is modified, the numbers may change. To help with this |
|
1155 difficulty, PCRE supports the naming of subpatterns. This feature was not |
|
1156 added to Perl until release 5.10. Python had the feature earlier, and PCRE |
|
1157 introduced it at release 4.0, using the Python syntax. PCRE now supports both |
|
1158 the Perl and the Python syntax. |
|
1159 </P> |
|
1160 <P> |
|
1161 In PCRE, a subpattern can be named in one of three ways: (?<name>...) or |
|
1162 (?'name'...) as in Perl, or (?P<name>...) as in Python. References to capturing |
|
1163 parentheses from other parts of the pattern, such as |
|
1164 <a href="#backreferences">backreferences,</a> |
|
1165 <a href="#recursion">recursion,</a> |
|
1166 and |
|
1167 <a href="#conditions">conditions,</a> |
|
1168 can be made by name as well as by number. |
|
1169 </P> |
|
1170 <P> |
|
1171 Names consist of up to 32 alphanumeric characters and underscores. Named |
|
1172 capturing parentheses are still allocated numbers as well as names, exactly as |
|
1173 if the names were not present. The PCRE API provides function calls for |
|
1174 extracting the name-to-number translation table from a compiled pattern. There |
|
1175 is also a convenience function for extracting a captured substring by name. |
|
1176 </P> |
|
1177 <P> |
|
1178 By default, a name must be unique within a pattern, but it is possible to relax |
|
1179 this constraint by setting the PCRE_DUPNAMES option at compile time. This can |
|
1180 be useful for patterns where only one instance of the named parentheses can |
|
1181 match. Suppose you want to match the name of a weekday, either as a 3-letter |
|
1182 abbreviation or as the full name, and in both cases you want to extract the |
|
1183 abbreviation. This pattern (ignoring the line breaks) does the job: |
|
1184 <pre> |
|
1185 (?<DN>Mon|Fri|Sun)(?:day)?| |
|
1186 (?<DN>Tue)(?:sday)?| |
|
1187 (?<DN>Wed)(?:nesday)?| |
|
1188 (?<DN>Thu)(?:rsday)?| |
|
1189 (?<DN>Sat)(?:urday)? |
|
1190 </pre> |
|
1191 There are five capturing substrings, but only one is ever set after a match. |
|
1192 (An alternative way of solving this problem is to use a "branch reset" |
|
1193 subpattern, as described in the previous section.) |
|
1194 </P> |
|
1195 <P> |
|
1196 The convenience function for extracting the data by name returns the substring |
|
1197 for the first (and in this example, the only) subpattern of that name that |
|
1198 matched. This saves searching to find which numbered subpattern it was. If you |
|
1199 make a reference to a non-unique named subpattern from elsewhere in the |
|
1200 pattern, the one that corresponds to the lowest number is used. For further |
|
1201 details of the interfaces for handling named subpatterns, see the |
|
1202 <a href="pcreapi.html"><b>pcreapi</b></a> |
|
1203 documentation. |
|
1204 </P> |
|
1205 <br><a name="SEC15" href="#TOC1">REPETITION</a><br> |
|
1206 <P> |
|
1207 Repetition is specified by quantifiers, which can follow any of the following |
|
1208 items: |
|
1209 <pre> |
|
1210 a literal data character |
|
1211 the dot metacharacter |
|
1212 the \C escape sequence |
|
1213 the \X escape sequence (in UTF-8 mode with Unicode properties) |
|
1214 the \R escape sequence |
|
1215 an escape such as \d that matches a single character |
|
1216 a character class |
|
1217 a back reference (see next section) |
|
1218 a parenthesized subpattern (unless it is an assertion) |
|
1219 </pre> |
|
1220 The general repetition quantifier specifies a minimum and maximum number of |
|
1221 permitted matches, by giving the two numbers in curly brackets (braces), |
|
1222 separated by a comma. The numbers must be less than 65536, and the first must |
|
1223 be less than or equal to the second. For example: |
|
1224 <pre> |
|
1225 z{2,4} |
|
1226 </pre> |
|
1227 matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special |
|
1228 character. If the second number is omitted, but the comma is present, there is |
|
1229 no upper limit; if the second number and the comma are both omitted, the |
|
1230 quantifier specifies an exact number of required matches. Thus |
|
1231 <pre> |
|
1232 [aeiou]{3,} |
|
1233 </pre> |
|
1234 matches at least 3 successive vowels, but may match many more, while |
|
1235 <pre> |
|
1236 \d{8} |
|
1237 </pre> |
|
1238 matches exactly 8 digits. An opening curly bracket that appears in a position |
|
1239 where a quantifier is not allowed, or one that does not match the syntax of a |
|
1240 quantifier, is taken as a literal character. For example, {,6} is not a |
|
1241 quantifier, but a literal string of four characters. |
|
1242 </P> |
|
1243 <P> |
|
1244 In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to individual |
|
1245 bytes. Thus, for example, \x{100}{2} matches two UTF-8 characters, each of |
|
1246 which is represented by a two-byte sequence. Similarly, when Unicode property |
|
1247 support is available, \X{3} matches three Unicode extended sequences, each of |
|
1248 which may be several bytes long (and they may be of different lengths). |
|
1249 </P> |
|
1250 <P> |
|
1251 The quantifier {0} is permitted, causing the expression to behave as if the |
|
1252 previous item and the quantifier were not present. This may be useful for |
|
1253 subpatterns that are referenced as |
|
1254 <a href="#subpatternsassubroutines">subroutines</a> |
|
1255 from elsewhere in the pattern. Items other than subpatterns that have a {0} |
|
1256 quantifier are omitted from the compiled pattern. |
|
1257 </P> |
|
1258 <P> |
|
1259 For convenience, the three most common quantifiers have single-character |
|
1260 abbreviations: |
|
1261 <pre> |
|
1262 * is equivalent to {0,} |
|
1263 + is equivalent to {1,} |
|
1264 ? is equivalent to {0,1} |
|
1265 </pre> |
|
1266 It is possible to construct infinite loops by following a subpattern that can |
|
1267 match no characters with a quantifier that has no upper limit, for example: |
|
1268 <pre> |
|
1269 (a?)* |
|
1270 </pre> |
|
1271 Earlier versions of Perl and PCRE used to give an error at compile time for |
|
1272 such patterns. However, because there are cases where this can be useful, such |
|
1273 patterns are now accepted, but if any repetition of the subpattern does in fact |
|
1274 match no characters, the loop is forcibly broken. |
|
1275 </P> |
|
1276 <P> |
|
1277 By default, the quantifiers are "greedy", that is, they match as much as |
|
1278 possible (up to the maximum number of permitted times), without causing the |
|
1279 rest of the pattern to fail. The classic example of where this gives problems |
|
1280 is in trying to match comments in C programs. These appear between /* and */ |
|
1281 and within the comment, individual * and / characters may appear. An attempt to |
|
1282 match C comments by applying the pattern |
|
1283 <pre> |
|
1284 /\*.*\*/ |
|
1285 </pre> |
|
1286 to the string |
|
1287 <pre> |
|
1288 /* first comment */ not comment /* second comment */ |
|
1289 </pre> |
|
1290 fails, because it matches the entire string owing to the greediness of the .* |
|
1291 item. |
|
1292 </P> |
|
1293 <P> |
|
1294 However, if a quantifier is followed by a question mark, it ceases to be |
|
1295 greedy, and instead matches the minimum number of times possible, so the |
|
1296 pattern |
|
1297 <pre> |
|
1298 /\*.*?\*/ |
|
1299 </pre> |
|
1300 does the right thing with the C comments. The meaning of the various |
|
1301 quantifiers is not otherwise changed, just the preferred number of matches. |
|
1302 Do not confuse this use of question mark with its use as a quantifier in its |
|
1303 own right. Because it has two uses, it can sometimes appear doubled, as in |
|
1304 <pre> |
|
1305 \d??\d |
|
1306 </pre> |
|
1307 which matches one digit by preference, but can match two if that is the only |
|
1308 way the rest of the pattern matches. |
|
1309 </P> |
|
1310 <P> |
|
1311 If the PCRE_UNGREEDY option is set (an option that is not available in Perl), |
|
1312 the quantifiers are not greedy by default, but individual ones can be made |
|
1313 greedy by following them with a question mark. In other words, it inverts the |
|
1314 default behaviour. |
|
1315 </P> |
|
1316 <P> |
|
1317 When a parenthesized subpattern is quantified with a minimum repeat count that |
|
1318 is greater than 1 or with a limited maximum, more memory is required for the |
|
1319 compiled pattern, in proportion to the size of the minimum or maximum. |
|
1320 </P> |
|
1321 <P> |
|
1322 If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equivalent |
|
1323 to Perl's /s) is set, thus allowing the dot to match newlines, the pattern is |
|
1324 implicitly anchored, because whatever follows will be tried against every |
|
1325 character position in the subject string, so there is no point in retrying the |
|
1326 overall match at any position after the first. PCRE normally treats such a |
|
1327 pattern as though it were preceded by \A. |
|
1328 </P> |
|
1329 <P> |
|
1330 In cases where it is known that the subject string contains no newlines, it is |
|
1331 worth setting PCRE_DOTALL in order to obtain this optimization, or |
|
1332 alternatively using ^ to indicate anchoring explicitly. |
|
1333 </P> |
|
1334 <P> |
|
1335 However, there is one situation where the optimization cannot be used. When .* |
|
1336 is inside capturing parentheses that are the subject of a backreference |
|
1337 elsewhere in the pattern, a match at the start may fail where a later one |
|
1338 succeeds. Consider, for example: |
|
1339 <pre> |
|
1340 (.*)abc\1 |
|
1341 </pre> |
|
1342 If the subject is "xyz123abc123" the match point is the fourth character. For |
|
1343 this reason, such a pattern is not implicitly anchored. |
|
1344 </P> |
|
1345 <P> |
|
1346 When a capturing subpattern is repeated, the value captured is the substring |
|
1347 that matched the final iteration. For example, after |
|
1348 <pre> |
|
1349 (tweedle[dume]{3}\s*)+ |
|
1350 </pre> |
|
1351 has matched "tweedledum tweedledee" the value of the captured substring is |
|
1352 "tweedledee". However, if there are nested capturing subpatterns, the |
|
1353 corresponding captured values may have been set in previous iterations. For |
|
1354 example, after |
|
1355 <pre> |
|
1356 /(a|(b))+/ |
|
1357 </pre> |
|
1358 matches "aba" the value of the second captured substring is "b". |
|
1359 <a name="atomicgroup"></a></P> |
|
1360 <br><a name="SEC16" href="#TOC1">ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS</a><br> |
|
1361 <P> |
|
1362 With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") |
|
1363 repetition, failure of what follows normally causes the repeated item to be |
|
1364 re-evaluated to see if a different number of repeats allows the rest of the |
|
1365 pattern to match. Sometimes it is useful to prevent this, either to change the |
|
1366 nature of the match, or to cause it fail earlier than it otherwise might, when |
|
1367 the author of the pattern knows there is no point in carrying on. |
|
1368 </P> |
|
1369 <P> |
|
1370 Consider, for example, the pattern \d+foo when applied to the subject line |
|
1371 <pre> |
|
1372 123456bar |
|
1373 </pre> |
|
1374 After matching all 6 digits and then failing to match "foo", the normal |
|
1375 action of the matcher is to try again with only 5 digits matching the \d+ |
|
1376 item, and then with 4, and so on, before ultimately failing. "Atomic grouping" |
|
1377 (a term taken from Jeffrey Friedl's book) provides the means for specifying |
|
1378 that once a subpattern has matched, it is not to be re-evaluated in this way. |
|
1379 </P> |
|
1380 <P> |
|
1381 If we use atomic grouping for the previous example, the matcher gives up |
|
1382 immediately on failing to match "foo" the first time. The notation is a kind of |
|
1383 special parenthesis, starting with (?> as in this example: |
|
1384 <pre> |
|
1385 (?>\d+)foo |
|
1386 </pre> |
|
1387 This kind of parenthesis "locks up" the part of the pattern it contains once |
|
1388 it has matched, and a failure further into the pattern is prevented from |
|
1389 backtracking into it. Backtracking past it to previous items, however, works as |
|
1390 normal. |
|
1391 </P> |
|
1392 <P> |
|
1393 An alternative description is that a subpattern of this type matches the string |
|
1394 of characters that an identical standalone pattern would match, if anchored at |
|
1395 the current point in the subject string. |
|
1396 </P> |
|
1397 <P> |
|
1398 Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as |
|
1399 the above example can be thought of as a maximizing repeat that must swallow |
|
1400 everything it can. So, while both \d+ and \d+? are prepared to adjust the |
|
1401 number of digits they match in order to make the rest of the pattern match, |
|
1402 (?>\d+) can only match an entire sequence of digits. |
|
1403 </P> |
|
1404 <P> |
|
1405 Atomic groups in general can of course contain arbitrarily complicated |
|
1406 subpatterns, and can be nested. However, when the subpattern for an atomic |
|
1407 group is just a single repeated item, as in the example above, a simpler |
|
1408 notation, called a "possessive quantifier" can be used. This consists of an |
|
1409 additional + character following a quantifier. Using this notation, the |
|
1410 previous example can be rewritten as |
|
1411 <pre> |
|
1412 \d++foo |
|
1413 </pre> |
|
1414 Note that a possessive quantifier can be used with an entire group, for |
|
1415 example: |
|
1416 <pre> |
|
1417 (abc|xyz){2,3}+ |
|
1418 </pre> |
|
1419 Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY |
|
1420 option is ignored. They are a convenient notation for the simpler forms of |
|
1421 atomic group. However, there is no difference in the meaning of a possessive |
|
1422 quantifier and the equivalent atomic group, though there may be a performance |
|
1423 difference; possessive quantifiers should be slightly faster. |
|
1424 </P> |
|
1425 <P> |
|
1426 The possessive quantifier syntax is an extension to the Perl 5.8 syntax. |
|
1427 Jeffrey Friedl originated the idea (and the name) in the first edition of his |
|
1428 book. Mike McCloskey liked it, so implemented it when he built Sun's Java |
|
1429 package, and PCRE copied it from there. It ultimately found its way into Perl |
|
1430 at release 5.10. |
|
1431 </P> |
|
1432 <P> |
|
1433 PCRE has an optimization that automatically "possessifies" certain simple |
|
1434 pattern constructs. For example, the sequence A+B is treated as A++B because |
|
1435 there is no point in backtracking into a sequence of A's when B must follow. |
|
1436 </P> |
|
1437 <P> |
|
1438 When a pattern contains an unlimited repeat inside a subpattern that can itself |
|
1439 be repeated an unlimited number of times, the use of an atomic group is the |
|
1440 only way to avoid some failing matches taking a very long time indeed. The |
|
1441 pattern |
|
1442 <pre> |
|
1443 (\D+|<\d+>)*[!?] |
|
1444 </pre> |
|
1445 matches an unlimited number of substrings that either consist of non-digits, or |
|
1446 digits enclosed in <>, followed by either ! or ?. When it matches, it runs |
|
1447 quickly. However, if it is applied to |
|
1448 <pre> |
|
1449 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa |
|
1450 </pre> |
|
1451 it takes a long time before reporting failure. This is because the string can |
|
1452 be divided between the internal \D+ repeat and the external * repeat in a |
|
1453 large number of ways, and all have to be tried. (The example uses [!?] rather |
|
1454 than a single character at the end, because both PCRE and Perl have an |
|
1455 optimization that allows for fast failure when a single character is used. They |
|
1456 remember the last single character that is required for a match, and fail early |
|
1457 if it is not present in the string.) If the pattern is changed so that it uses |
|
1458 an atomic group, like this: |
|
1459 <pre> |
|
1460 ((?>\D+)|<\d+>)*[!?] |
|
1461 </pre> |
|
1462 sequences of non-digits cannot be broken, and failure happens quickly. |
|
1463 <a name="backreferences"></a></P> |
|
1464 <br><a name="SEC17" href="#TOC1">BACK REFERENCES</a><br> |
|
1465 <P> |
|
1466 Outside a character class, a backslash followed by a digit greater than 0 (and |
|
1467 possibly further digits) is a back reference to a capturing subpattern earlier |
|
1468 (that is, to its left) in the pattern, provided there have been that many |
|
1469 previous capturing left parentheses. |
|
1470 </P> |
|
1471 <P> |
|
1472 However, if the decimal number following the backslash is less than 10, it is |
|
1473 always taken as a back reference, and causes an error only if there are not |
|
1474 that many capturing left parentheses in the entire pattern. In other words, the |
|
1475 parentheses that are referenced need not be to the left of the reference for |
|
1476 numbers less than 10. A "forward back reference" of this type can make sense |
|
1477 when a repetition is involved and the subpattern to the right has participated |
|
1478 in an earlier iteration. |
|
1479 </P> |
|
1480 <P> |
|
1481 It is not possible to have a numerical "forward back reference" to a subpattern |
|
1482 whose number is 10 or more using this syntax because a sequence such as \50 is |
|
1483 interpreted as a character defined in octal. See the subsection entitled |
|
1484 "Non-printing characters" |
|
1485 <a href="#digitsafterbackslash">above</a> |
|
1486 for further details of the handling of digits following a backslash. There is |
|
1487 no such problem when named parentheses are used. A back reference to any |
|
1488 subpattern is possible using named parentheses (see below). |
|
1489 </P> |
|
1490 <P> |
|
1491 Another way of avoiding the ambiguity inherent in the use of digits following a |
|
1492 backslash is to use the \g escape sequence, which is a feature introduced in |
|
1493 Perl 5.10. This escape must be followed by an unsigned number or a negative |
|
1494 number, optionally enclosed in braces. These examples are all identical: |
|
1495 <pre> |
|
1496 (ring), \1 |
|
1497 (ring), \g1 |
|
1498 (ring), \g{1} |
|
1499 </pre> |
|
1500 An unsigned number specifies an absolute reference without the ambiguity that |
|
1501 is present in the older syntax. It is also useful when literal digits follow |
|
1502 the reference. A negative number is a relative reference. Consider this |
|
1503 example: |
|
1504 <pre> |
|
1505 (abc(def)ghi)\g{-1} |
|
1506 </pre> |
|
1507 The sequence \g{-1} is a reference to the most recently started capturing |
|
1508 subpattern before \g, that is, is it equivalent to \2. Similarly, \g{-2} |
|
1509 would be equivalent to \1. The use of relative references can be helpful in |
|
1510 long patterns, and also in patterns that are created by joining together |
|
1511 fragments that contain references within themselves. |
|
1512 </P> |
|
1513 <P> |
|
1514 A back reference matches whatever actually matched the capturing subpattern in |
|
1515 the current subject string, rather than anything matching the subpattern |
|
1516 itself (see |
|
1517 <a href="#subpatternsassubroutines">"Subpatterns as subroutines"</a> |
|
1518 below for a way of doing that). So the pattern |
|
1519 <pre> |
|
1520 (sens|respons)e and \1ibility |
|
1521 </pre> |
|
1522 matches "sense and sensibility" and "response and responsibility", but not |
|
1523 "sense and responsibility". If caseful matching is in force at the time of the |
|
1524 back reference, the case of letters is relevant. For example, |
|
1525 <pre> |
|
1526 ((?i)rah)\s+\1 |
|
1527 </pre> |
|
1528 matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original |
|
1529 capturing subpattern is matched caselessly. |
|
1530 </P> |
|
1531 <P> |
|
1532 There are several different ways of writing back references to named |
|
1533 subpatterns. The .NET syntax \k{name} and the Perl syntax \k<name> or |
|
1534 \k'name' are supported, as is the Python syntax (?P=name). Perl 5.10's unified |
|
1535 back reference syntax, in which \g can be used for both numeric and named |
|
1536 references, is also supported. We could rewrite the above example in any of |
|
1537 the following ways: |
|
1538 <pre> |
|
1539 (?<p1>(?i)rah)\s+\k<p1> |
|
1540 (?'p1'(?i)rah)\s+\k{p1} |
|
1541 (?P<p1>(?i)rah)\s+(?P=p1) |
|
1542 (?<p1>(?i)rah)\s+\g{p1} |
|
1543 </pre> |
|
1544 A subpattern that is referenced by name may appear in the pattern before or |
|
1545 after the reference. |
|
1546 </P> |
|
1547 <P> |
|
1548 There may be more than one back reference to the same subpattern. If a |
|
1549 subpattern has not actually been used in a particular match, any back |
|
1550 references to it always fail. For example, the pattern |
|
1551 <pre> |
|
1552 (a|(bc))\2 |
|
1553 </pre> |
|
1554 always fails if it starts to match "a" rather than "bc". Because there may be |
|
1555 many capturing parentheses in a pattern, all digits following the backslash are |
|
1556 taken as part of a potential back reference number. If the pattern continues |
|
1557 with a digit character, some delimiter must be used to terminate the back |
|
1558 reference. If the PCRE_EXTENDED option is set, this can be whitespace. |
|
1559 Otherwise an empty comment (see |
|
1560 <a href="#comments">"Comments"</a> |
|
1561 below) can be used. |
|
1562 </P> |
|
1563 <P> |
|
1564 A back reference that occurs inside the parentheses to which it refers fails |
|
1565 when the subpattern is first used, so, for example, (a\1) never matches. |
|
1566 However, such references can be useful inside repeated subpatterns. For |
|
1567 example, the pattern |
|
1568 <pre> |
|
1569 (a|b\1)+ |
|
1570 </pre> |
|
1571 matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of |
|
1572 the subpattern, the back reference matches the character string corresponding |
|
1573 to the previous iteration. In order for this to work, the pattern must be such |
|
1574 that the first iteration does not need to match the back reference. This can be |
|
1575 done using alternation, as in the example above, or by a quantifier with a |
|
1576 minimum of zero. |
|
1577 <a name="bigassertions"></a></P> |
|
1578 <br><a name="SEC18" href="#TOC1">ASSERTIONS</a><br> |
|
1579 <P> |
|
1580 An assertion is a test on the characters following or preceding the current |
|
1581 matching point that does not actually consume any characters. The simple |
|
1582 assertions coded as \b, \B, \A, \G, \Z, \z, ^ and $ are described |
|
1583 <a href="#smallassertions">above.</a> |
|
1584 </P> |
|
1585 <P> |
|
1586 More complicated assertions are coded as subpatterns. There are two kinds: |
|
1587 those that look ahead of the current position in the subject string, and those |
|
1588 that look behind it. An assertion subpattern is matched in the normal way, |
|
1589 except that it does not cause the current matching position to be changed. |
|
1590 </P> |
|
1591 <P> |
|
1592 Assertion subpatterns are not capturing subpatterns, and may not be repeated, |
|
1593 because it makes no sense to assert the same thing several times. If any kind |
|
1594 of assertion contains capturing subpatterns within it, these are counted for |
|
1595 the purposes of numbering the capturing subpatterns in the whole pattern. |
|
1596 However, substring capturing is carried out only for positive assertions, |
|
1597 because it does not make sense for negative assertions. |
|
1598 </P> |
|
1599 <br><b> |
|
1600 Lookahead assertions |
|
1601 </b><br> |
|
1602 <P> |
|
1603 Lookahead assertions start with (?= for positive assertions and (?! for |
|
1604 negative assertions. For example, |
|
1605 <pre> |
|
1606 \w+(?=;) |
|
1607 </pre> |
|
1608 matches a word followed by a semicolon, but does not include the semicolon in |
|
1609 the match, and |
|
1610 <pre> |
|
1611 foo(?!bar) |
|
1612 </pre> |
|
1613 matches any occurrence of "foo" that is not followed by "bar". Note that the |
|
1614 apparently similar pattern |
|
1615 <pre> |
|
1616 (?!foo)bar |
|
1617 </pre> |
|
1618 does not find an occurrence of "bar" that is preceded by something other than |
|
1619 "foo"; it finds any occurrence of "bar" whatsoever, because the assertion |
|
1620 (?!foo) is always true when the next three characters are "bar". A |
|
1621 lookbehind assertion is needed to achieve the other effect. |
|
1622 </P> |
|
1623 <P> |
|
1624 If you want to force a matching failure at some point in a pattern, the most |
|
1625 convenient way to do it is with (?!) because an empty string always matches, so |
|
1626 an assertion that requires there not to be an empty string must always fail. |
|
1627 <a name="lookbehind"></a></P> |
|
1628 <br><b> |
|
1629 Lookbehind assertions |
|
1630 </b><br> |
|
1631 <P> |
|
1632 Lookbehind assertions start with (?<= for positive assertions and (?<! for |
|
1633 negative assertions. For example, |
|
1634 <pre> |
|
1635 (?<!foo)bar |
|
1636 </pre> |
|
1637 does find an occurrence of "bar" that is not preceded by "foo". The contents of |
|
1638 a lookbehind assertion are restricted such that all the strings it matches must |
|
1639 have a fixed length. However, if there are several top-level alternatives, they |
|
1640 do not all have to have the same fixed length. Thus |
|
1641 <pre> |
|
1642 (?<=bullock|donkey) |
|
1643 </pre> |
|
1644 is permitted, but |
|
1645 <pre> |
|
1646 (?<!dogs?|cats?) |
|
1647 </pre> |
|
1648 causes an error at compile time. Branches that match different length strings |
|
1649 are permitted only at the top level of a lookbehind assertion. This is an |
|
1650 extension compared with Perl (at least for 5.8), which requires all branches to |
|
1651 match the same length of string. An assertion such as |
|
1652 <pre> |
|
1653 (?<=ab(c|de)) |
|
1654 </pre> |
|
1655 is not permitted, because its single top-level branch can match two different |
|
1656 lengths, but it is acceptable if rewritten to use two top-level branches: |
|
1657 <pre> |
|
1658 (?<=abc|abde) |
|
1659 </pre> |
|
1660 In some cases, the Perl 5.10 escape sequence \K |
|
1661 <a href="#resetmatchstart">(see above)</a> |
|
1662 can be used instead of a lookbehind assertion; this is not restricted to a |
|
1663 fixed-length. |
|
1664 </P> |
|
1665 <P> |
|
1666 The implementation of lookbehind assertions is, for each alternative, to |
|
1667 temporarily move the current position back by the fixed length and then try to |
|
1668 match. If there are insufficient characters before the current position, the |
|
1669 assertion fails. |
|
1670 </P> |
|
1671 <P> |
|
1672 PCRE does not allow the \C escape (which matches a single byte in UTF-8 mode) |
|
1673 to appear in lookbehind assertions, because it makes it impossible to calculate |
|
1674 the length of the lookbehind. The \X and \R escapes, which can match |
|
1675 different numbers of bytes, are also not permitted. |
|
1676 </P> |
|
1677 <P> |
|
1678 Possessive quantifiers can be used in conjunction with lookbehind assertions to |
|
1679 specify efficient matching at the end of the subject string. Consider a simple |
|
1680 pattern such as |
|
1681 <pre> |
|
1682 abcd$ |
|
1683 </pre> |
|
1684 when applied to a long string that does not match. Because matching proceeds |
|
1685 from left to right, PCRE will look for each "a" in the subject and then see if |
|
1686 what follows matches the rest of the pattern. If the pattern is specified as |
|
1687 <pre> |
|
1688 ^.*abcd$ |
|
1689 </pre> |
|
1690 the initial .* matches the entire string at first, but when this fails (because |
|
1691 there is no following "a"), it backtracks to match all but the last character, |
|
1692 then all but the last two characters, and so on. Once again the search for "a" |
|
1693 covers the entire string, from right to left, so we are no better off. However, |
|
1694 if the pattern is written as |
|
1695 <pre> |
|
1696 ^.*+(?<=abcd) |
|
1697 </pre> |
|
1698 there can be no backtracking for the .*+ item; it can match only the entire |
|
1699 string. The subsequent lookbehind assertion does a single test on the last four |
|
1700 characters. If it fails, the match fails immediately. For long strings, this |
|
1701 approach makes a significant difference to the processing time. |
|
1702 </P> |
|
1703 <br><b> |
|
1704 Using multiple assertions |
|
1705 </b><br> |
|
1706 <P> |
|
1707 Several assertions (of any sort) may occur in succession. For example, |
|
1708 <pre> |
|
1709 (?<=\d{3})(?<!999)foo |
|
1710 </pre> |
|
1711 matches "foo" preceded by three digits that are not "999". Notice that each of |
|
1712 the assertions is applied independently at the same point in the subject |
|
1713 string. First there is a check that the previous three characters are all |
|
1714 digits, and then there is a check that the same three characters are not "999". |
|
1715 This pattern does <i>not</i> match "foo" preceded by six characters, the first |
|
1716 of which are digits and the last three of which are not "999". For example, it |
|
1717 doesn't match "123abcfoo". A pattern to do that is |
|
1718 <pre> |
|
1719 (?<=\d{3}...)(?<!999)foo |
|
1720 </pre> |
|
1721 This time the first assertion looks at the preceding six characters, checking |
|
1722 that the first three are digits, and then the second assertion checks that the |
|
1723 preceding three characters are not "999". |
|
1724 </P> |
|
1725 <P> |
|
1726 Assertions can be nested in any combination. For example, |
|
1727 <pre> |
|
1728 (?<=(?<!foo)bar)baz |
|
1729 </pre> |
|
1730 matches an occurrence of "baz" that is preceded by "bar" which in turn is not |
|
1731 preceded by "foo", while |
|
1732 <pre> |
|
1733 (?<=\d{3}(?!999)...)foo |
|
1734 </pre> |
|
1735 is another pattern that matches "foo" preceded by three digits and any three |
|
1736 characters that are not "999". |
|
1737 <a name="conditions"></a></P> |
|
1738 <br><a name="SEC19" href="#TOC1">CONDITIONAL SUBPATTERNS</a><br> |
|
1739 <P> |
|
1740 It is possible to cause the matching process to obey a subpattern |
|
1741 conditionally or to choose between two alternative subpatterns, depending on |
|
1742 the result of an assertion, or whether a previous capturing subpattern matched |
|
1743 or not. The two possible forms of conditional subpattern are |
|
1744 <pre> |
|
1745 (?(condition)yes-pattern) |
|
1746 (?(condition)yes-pattern|no-pattern) |
|
1747 </pre> |
|
1748 If the condition is satisfied, the yes-pattern is used; otherwise the |
|
1749 no-pattern (if present) is used. If there are more than two alternatives in the |
|
1750 subpattern, a compile-time error occurs. |
|
1751 </P> |
|
1752 <P> |
|
1753 There are four kinds of condition: references to subpatterns, references to |
|
1754 recursion, a pseudo-condition called DEFINE, and assertions. |
|
1755 </P> |
|
1756 <br><b> |
|
1757 Checking for a used subpattern by number |
|
1758 </b><br> |
|
1759 <P> |
|
1760 If the text between the parentheses consists of a sequence of digits, the |
|
1761 condition is true if the capturing subpattern of that number has previously |
|
1762 matched. An alternative notation is to precede the digits with a plus or minus |
|
1763 sign. In this case, the subpattern number is relative rather than absolute. |
|
1764 The most recently opened parentheses can be referenced by (?(-1), the next most |
|
1765 recent by (?(-2), and so on. In looping constructs it can also make sense to |
|
1766 refer to subsequent groups with constructs such as (?(+2). |
|
1767 </P> |
|
1768 <P> |
|
1769 Consider the following pattern, which contains non-significant white space to |
|
1770 make it more readable (assume the PCRE_EXTENDED option) and to divide it into |
|
1771 three parts for ease of discussion: |
|
1772 <pre> |
|
1773 ( \( )? [^()]+ (?(1) \) ) |
|
1774 </pre> |
|
1775 The first part matches an optional opening parenthesis, and if that |
|
1776 character is present, sets it as the first captured substring. The second part |
|
1777 matches one or more characters that are not parentheses. The third part is a |
|
1778 conditional subpattern that tests whether the first set of parentheses matched |
|
1779 or not. If they did, that is, if subject started with an opening parenthesis, |
|
1780 the condition is true, and so the yes-pattern is executed and a closing |
|
1781 parenthesis is required. Otherwise, since no-pattern is not present, the |
|
1782 subpattern matches nothing. In other words, this pattern matches a sequence of |
|
1783 non-parentheses, optionally enclosed in parentheses. |
|
1784 </P> |
|
1785 <P> |
|
1786 If you were embedding this pattern in a larger one, you could use a relative |
|
1787 reference: |
|
1788 <pre> |
|
1789 ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... |
|
1790 </pre> |
|
1791 This makes the fragment independent of the parentheses in the larger pattern. |
|
1792 </P> |
|
1793 <br><b> |
|
1794 Checking for a used subpattern by name |
|
1795 </b><br> |
|
1796 <P> |
|
1797 Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a used |
|
1798 subpattern by name. For compatibility with earlier versions of PCRE, which had |
|
1799 this facility before Perl, the syntax (?(name)...) is also recognized. However, |
|
1800 there is a possible ambiguity with this syntax, because subpattern names may |
|
1801 consist entirely of digits. PCRE looks first for a named subpattern; if it |
|
1802 cannot find one and the name consists entirely of digits, PCRE looks for a |
|
1803 subpattern of that number, which must be greater than zero. Using subpattern |
|
1804 names that consist entirely of digits is not recommended. |
|
1805 </P> |
|
1806 <P> |
|
1807 Rewriting the above example to use a named subpattern gives this: |
|
1808 <pre> |
|
1809 (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) |
|
1810 |
|
1811 </PRE> |
|
1812 </P> |
|
1813 <br><b> |
|
1814 Checking for pattern recursion |
|
1815 </b><br> |
|
1816 <P> |
|
1817 If the condition is the string (R), and there is no subpattern with the name R, |
|
1818 the condition is true if a recursive call to the whole pattern or any |
|
1819 subpattern has been made. If digits or a name preceded by ampersand follow the |
|
1820 letter R, for example: |
|
1821 <pre> |
|
1822 (?(R3)...) or (?(R&name)...) |
|
1823 </pre> |
|
1824 the condition is true if the most recent recursion is into the subpattern whose |
|
1825 number or name is given. This condition does not check the entire recursion |
|
1826 stack. |
|
1827 </P> |
|
1828 <P> |
|
1829 At "top level", all these recursion test conditions are false. Recursive |
|
1830 patterns are described below. |
|
1831 </P> |
|
1832 <br><b> |
|
1833 Defining subpatterns for use by reference only |
|
1834 </b><br> |
|
1835 <P> |
|
1836 If the condition is the string (DEFINE), and there is no subpattern with the |
|
1837 name DEFINE, the condition is always false. In this case, there may be only one |
|
1838 alternative in the subpattern. It is always skipped if control reaches this |
|
1839 point in the pattern; the idea of DEFINE is that it can be used to define |
|
1840 "subroutines" that can be referenced from elsewhere. (The use of "subroutines" |
|
1841 is described below.) For example, a pattern to match an IPv4 address could be |
|
1842 written like this (ignore whitespace and line breaks): |
|
1843 <pre> |
|
1844 (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) |
|
1845 \b (?&byte) (\.(?&byte)){3} \b |
|
1846 </pre> |
|
1847 The first part of the pattern is a DEFINE group inside which a another group |
|
1848 named "byte" is defined. This matches an individual component of an IPv4 |
|
1849 address (a number less than 256). When matching takes place, this part of the |
|
1850 pattern is skipped because DEFINE acts like a false condition. |
|
1851 </P> |
|
1852 <P> |
|
1853 The rest of the pattern uses references to the named group to match the four |
|
1854 dot-separated components of an IPv4 address, insisting on a word boundary at |
|
1855 each end. |
|
1856 </P> |
|
1857 <br><b> |
|
1858 Assertion conditions |
|
1859 </b><br> |
|
1860 <P> |
|
1861 If the condition is not in any of the above formats, it must be an assertion. |
|
1862 This may be a positive or negative lookahead or lookbehind assertion. Consider |
|
1863 this pattern, again containing non-significant white space, and with the two |
|
1864 alternatives on the second line: |
|
1865 <pre> |
|
1866 (?(?=[^a-z]*[a-z]) |
|
1867 \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) |
|
1868 </pre> |
|
1869 The condition is a positive lookahead assertion that matches an optional |
|
1870 sequence of non-letters followed by a letter. In other words, it tests for the |
|
1871 presence of at least one letter in the subject. If a letter is found, the |
|
1872 subject is matched against the first alternative; otherwise it is matched |
|
1873 against the second. This pattern matches strings in one of the two forms |
|
1874 dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. |
|
1875 <a name="comments"></a></P> |
|
1876 <br><a name="SEC20" href="#TOC1">COMMENTS</a><br> |
|
1877 <P> |
|
1878 The sequence (?# marks the start of a comment that continues up to the next |
|
1879 closing parenthesis. Nested parentheses are not permitted. The characters |
|
1880 that make up a comment play no part in the pattern matching at all. |
|
1881 </P> |
|
1882 <P> |
|
1883 If the PCRE_EXTENDED option is set, an unescaped # character outside a |
|
1884 character class introduces a comment that continues to immediately after the |
|
1885 next newline in the pattern. |
|
1886 <a name="recursion"></a></P> |
|
1887 <br><a name="SEC21" href="#TOC1">RECURSIVE PATTERNS</a><br> |
|
1888 <P> |
|
1889 Consider the problem of matching a string in parentheses, allowing for |
|
1890 unlimited nested parentheses. Without the use of recursion, the best that can |
|
1891 be done is to use a pattern that matches up to some fixed depth of nesting. It |
|
1892 is not possible to handle an arbitrary nesting depth. |
|
1893 </P> |
|
1894 <P> |
|
1895 For some time, Perl has provided a facility that allows regular expressions to |
|
1896 recurse (amongst other things). It does this by interpolating Perl code in the |
|
1897 expression at run time, and the code can refer to the expression itself. A Perl |
|
1898 pattern using code interpolation to solve the parentheses problem can be |
|
1899 created like this: |
|
1900 <pre> |
|
1901 $re = qr{\( (?: (?>[^()]+) | (?p{$re}) )* \)}x; |
|
1902 </pre> |
|
1903 The (?p{...}) item interpolates Perl code at run time, and in this case refers |
|
1904 recursively to the pattern in which it appears. |
|
1905 </P> |
|
1906 <P> |
|
1907 Obviously, PCRE cannot support the interpolation of Perl code. Instead, it |
|
1908 supports special syntax for recursion of the entire pattern, and also for |
|
1909 individual subpattern recursion. After its introduction in PCRE and Python, |
|
1910 this kind of recursion was introduced into Perl at release 5.10. |
|
1911 </P> |
|
1912 <P> |
|
1913 A special item that consists of (? followed by a number greater than zero and a |
|
1914 closing parenthesis is a recursive call of the subpattern of the given number, |
|
1915 provided that it occurs inside that subpattern. (If not, it is a "subroutine" |
|
1916 call, which is described in the next section.) The special item (?R) or (?0) is |
|
1917 a recursive call of the entire regular expression. |
|
1918 </P> |
|
1919 <P> |
|
1920 In PCRE (like Python, but unlike Perl), a recursive subpattern call is always |
|
1921 treated as an atomic group. That is, once it has matched some of the subject |
|
1922 string, it is never re-entered, even if it contains untried alternatives and |
|
1923 there is a subsequent matching failure. |
|
1924 </P> |
|
1925 <P> |
|
1926 This PCRE pattern solves the nested parentheses problem (assume the |
|
1927 PCRE_EXTENDED option is set so that white space is ignored): |
|
1928 <pre> |
|
1929 \( ( (?>[^()]+) | (?R) )* \) |
|
1930 </pre> |
|
1931 First it matches an opening parenthesis. Then it matches any number of |
|
1932 substrings which can either be a sequence of non-parentheses, or a recursive |
|
1933 match of the pattern itself (that is, a correctly parenthesized substring). |
|
1934 Finally there is a closing parenthesis. |
|
1935 </P> |
|
1936 <P> |
|
1937 If this were part of a larger pattern, you would not want to recurse the entire |
|
1938 pattern, so instead you could use this: |
|
1939 <pre> |
|
1940 ( \( ( (?>[^()]+) | (?1) )* \) ) |
|
1941 </pre> |
|
1942 We have put the pattern into parentheses, and caused the recursion to refer to |
|
1943 them instead of the whole pattern. |
|
1944 </P> |
|
1945 <P> |
|
1946 In a larger pattern, keeping track of parenthesis numbers can be tricky. This |
|
1947 is made easier by the use of relative references. (A Perl 5.10 feature.) |
|
1948 Instead of (?1) in the pattern above you can write (?-2) to refer to the second |
|
1949 most recently opened parentheses preceding the recursion. In other words, a |
|
1950 negative number counts capturing parentheses leftwards from the point at which |
|
1951 it is encountered. |
|
1952 </P> |
|
1953 <P> |
|
1954 It is also possible to refer to subsequently opened parentheses, by writing |
|
1955 references such as (?+2). However, these cannot be recursive because the |
|
1956 reference is not inside the parentheses that are referenced. They are always |
|
1957 "subroutine" calls, as described in the next section. |
|
1958 </P> |
|
1959 <P> |
|
1960 An alternative approach is to use named parentheses instead. The Perl syntax |
|
1961 for this is (?&name); PCRE's earlier syntax (?P>name) is also supported. We |
|
1962 could rewrite the above example as follows: |
|
1963 <pre> |
|
1964 (?<pn> \( ( (?>[^()]+) | (?&pn) )* \) ) |
|
1965 </pre> |
|
1966 If there is more than one subpattern with the same name, the earliest one is |
|
1967 used. |
|
1968 </P> |
|
1969 <P> |
|
1970 This particular example pattern that we have been looking at contains nested |
|
1971 unlimited repeats, and so the use of atomic grouping for matching strings of |
|
1972 non-parentheses is important when applying the pattern to strings that do not |
|
1973 match. For example, when this pattern is applied to |
|
1974 <pre> |
|
1975 (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() |
|
1976 </pre> |
|
1977 it yields "no match" quickly. However, if atomic grouping is not used, |
|
1978 the match runs for a very long time indeed because there are so many different |
|
1979 ways the + and * repeats can carve up the subject, and all have to be tested |
|
1980 before failure can be reported. |
|
1981 </P> |
|
1982 <P> |
|
1983 At the end of a match, the values set for any capturing subpatterns are those |
|
1984 from the outermost level of the recursion at which the subpattern value is set. |
|
1985 If you want to obtain intermediate values, a callout function can be used (see |
|
1986 below and the |
|
1987 <a href="pcrecallout.html"><b>pcrecallout</b></a> |
|
1988 documentation). If the pattern above is matched against |
|
1989 <pre> |
|
1990 (ab(cd)ef) |
|
1991 </pre> |
|
1992 the value for the capturing parentheses is "ef", which is the last value taken |
|
1993 on at the top level. If additional parentheses are added, giving |
|
1994 <pre> |
|
1995 \( ( ( (?>[^()]+) | (?R) )* ) \) |
|
1996 ^ ^ |
|
1997 ^ ^ |
|
1998 </pre> |
|
1999 the string they capture is "ab(cd)ef", the contents of the top level |
|
2000 parentheses. If there are more than 15 capturing parentheses in a pattern, PCRE |
|
2001 has to obtain extra memory to store data during a recursion, which it does by |
|
2002 using <b>pcre_malloc</b>, freeing it via <b>pcre_free</b> afterwards. If no |
|
2003 memory can be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. |
|
2004 </P> |
|
2005 <P> |
|
2006 Do not confuse the (?R) item with the condition (R), which tests for recursion. |
|
2007 Consider this pattern, which matches text in angle brackets, allowing for |
|
2008 arbitrary nesting. Only digits are allowed in nested brackets (that is, when |
|
2009 recursing), whereas any characters are permitted at the outer level. |
|
2010 <pre> |
|
2011 < (?: (?(R) \d++ | [^<>]*+) | (?R)) * > |
|
2012 </pre> |
|
2013 In this pattern, (?(R) is the start of a conditional subpattern, with two |
|
2014 different alternatives for the recursive and non-recursive cases. The (?R) item |
|
2015 is the actual recursive call. |
|
2016 <a name="subpatternsassubroutines"></a></P> |
|
2017 <br><a name="SEC22" href="#TOC1">SUBPATTERNS AS SUBROUTINES</a><br> |
|
2018 <P> |
|
2019 If the syntax for a recursive subpattern reference (either by number or by |
|
2020 name) is used outside the parentheses to which it refers, it operates like a |
|
2021 subroutine in a programming language. The "called" subpattern may be defined |
|
2022 before or after the reference. A numbered reference can be absolute or |
|
2023 relative, as in these examples: |
|
2024 <pre> |
|
2025 (...(absolute)...)...(?2)... |
|
2026 (...(relative)...)...(?-1)... |
|
2027 (...(?+1)...(relative)... |
|
2028 </pre> |
|
2029 An earlier example pointed out that the pattern |
|
2030 <pre> |
|
2031 (sens|respons)e and \1ibility |
|
2032 </pre> |
|
2033 matches "sense and sensibility" and "response and responsibility", but not |
|
2034 "sense and responsibility". If instead the pattern |
|
2035 <pre> |
|
2036 (sens|respons)e and (?1)ibility |
|
2037 </pre> |
|
2038 is used, it does match "sense and responsibility" as well as the other two |
|
2039 strings. Another example is given in the discussion of DEFINE above. |
|
2040 </P> |
|
2041 <P> |
|
2042 Like recursive subpatterns, a "subroutine" call is always treated as an atomic |
|
2043 group. That is, once it has matched some of the subject string, it is never |
|
2044 re-entered, even if it contains untried alternatives and there is a subsequent |
|
2045 matching failure. |
|
2046 </P> |
|
2047 <P> |
|
2048 When a subpattern is used as a subroutine, processing options such as |
|
2049 case-independence are fixed when the subpattern is defined. They cannot be |
|
2050 changed for different calls. For example, consider this pattern: |
|
2051 <pre> |
|
2052 (abc)(?i:(?-1)) |
|
2053 </pre> |
|
2054 It matches "abcabc". It does not match "abcABC" because the change of |
|
2055 processing option does not affect the called subpattern. |
|
2056 <a name="onigurumasubroutines"></a></P> |
|
2057 <br><a name="SEC23" href="#TOC1">ONIGURUMA SUBROUTINE SYNTAX</a><br> |
|
2058 <P> |
|
2059 For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or |
|
2060 a number enclosed either in angle brackets or single quotes, is an alternative |
|
2061 syntax for referencing a subpattern as a subroutine, possibly recursively. Here |
|
2062 are two of the examples used above, rewritten using this syntax: |
|
2063 <pre> |
|
2064 (?<pn> \( ( (?>[^()]+) | \g<pn> )* \) ) |
|
2065 (sens|respons)e and \g'1'ibility |
|
2066 </pre> |
|
2067 PCRE supports an extension to Oniguruma: if a number is preceded by a |
|
2068 plus or a minus sign it is taken as a relative reference. For example: |
|
2069 <pre> |
|
2070 (abc)(?i:\g<-1>) |
|
2071 </pre> |
|
2072 Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are <i>not</i> |
|
2073 synonymous. The former is a back reference; the latter is a subroutine call. |
|
2074 </P> |
|
2075 <br><a name="SEC24" href="#TOC1">CALLOUTS</a><br> |
|
2076 <P> |
|
2077 Perl has a feature whereby using the sequence (?{...}) causes arbitrary Perl |
|
2078 code to be obeyed in the middle of matching a regular expression. This makes it |
|
2079 possible, amongst other things, to extract different substrings that match the |
|
2080 same pair of parentheses when there is a repetition. |
|
2081 </P> |
|
2082 <P> |
|
2083 PCRE provides a similar feature, but of course it cannot obey arbitrary Perl |
|
2084 code. The feature is called "callout". The caller of PCRE provides an external |
|
2085 function by putting its entry point in the global variable <i>pcre_callout</i>. |
|
2086 By default, this variable contains NULL, which disables all calling out. |
|
2087 </P> |
|
2088 <P> |
|
2089 Within a regular expression, (?C) indicates the points at which the external |
|
2090 function is to be called. If you want to identify different callout points, you |
|
2091 can put a number less than 256 after the letter C. The default value is zero. |
|
2092 For example, this pattern has two callout points: |
|
2093 <pre> |
|
2094 (?C1)abc(?C2)def |
|
2095 </pre> |
|
2096 If the PCRE_AUTO_CALLOUT flag is passed to <b>pcre_compile()</b>, callouts are |
|
2097 automatically installed before each item in the pattern. They are all numbered |
|
2098 255. |
|
2099 </P> |
|
2100 <P> |
|
2101 During matching, when PCRE reaches a callout point (and <i>pcre_callout</i> is |
|
2102 set), the external function is called. It is provided with the number of the |
|
2103 callout, the position in the pattern, and, optionally, one item of data |
|
2104 originally supplied by the caller of <b>pcre_exec()</b>. The callout function |
|
2105 may cause matching to proceed, to backtrack, or to fail altogether. A complete |
|
2106 description of the interface to the callout function is given in the |
|
2107 <a href="pcrecallout.html"><b>pcrecallout</b></a> |
|
2108 documentation. |
|
2109 </P> |
|
2110 <br><a name="SEC25" href="#TOC1">BACKTRACKING CONTROL</a><br> |
|
2111 <P> |
|
2112 Perl 5.10 introduced a number of "Special Backtracking Control Verbs", which |
|
2113 are described in the Perl documentation as "experimental and subject to change |
|
2114 or removal in a future version of Perl". It goes on to say: "Their usage in |
|
2115 production code should be noted to avoid problems during upgrades." The same |
|
2116 remarks apply to the PCRE features described in this section. |
|
2117 </P> |
|
2118 <P> |
|
2119 Since these verbs are specifically related to backtracking, most of them can be |
|
2120 used only when the pattern is to be matched using <b>pcre_exec()</b>, which uses |
|
2121 a backtracking algorithm. With the exception of (*FAIL), which behaves like a |
|
2122 failing negative assertion, they cause an error if encountered by |
|
2123 <b>pcre_dfa_exec()</b>. |
|
2124 </P> |
|
2125 <P> |
|
2126 The new verbs make use of what was previously invalid syntax: an opening |
|
2127 parenthesis followed by an asterisk. In Perl, they are generally of the form |
|
2128 (*VERB:ARG) but PCRE does not support the use of arguments, so its general |
|
2129 form is just (*VERB). Any number of these verbs may occur in a pattern. There |
|
2130 are two kinds: |
|
2131 </P> |
|
2132 <br><b> |
|
2133 Verbs that act immediately |
|
2134 </b><br> |
|
2135 <P> |
|
2136 The following verbs act as soon as they are encountered: |
|
2137 <pre> |
|
2138 (*ACCEPT) |
|
2139 </pre> |
|
2140 This verb causes the match to end successfully, skipping the remainder of the |
|
2141 pattern. When inside a recursion, only the innermost pattern is ended |
|
2142 immediately. PCRE differs from Perl in what happens if the (*ACCEPT) is inside |
|
2143 capturing parentheses. In Perl, the data so far is captured: in PCRE no data is |
|
2144 captured. For example: |
|
2145 <pre> |
|
2146 A(A|B(*ACCEPT)|C)D |
|
2147 </pre> |
|
2148 This matches "AB", "AAD", or "ACD", but when it matches "AB", no data is |
|
2149 captured. |
|
2150 <pre> |
|
2151 (*FAIL) or (*F) |
|
2152 </pre> |
|
2153 This verb causes the match to fail, forcing backtracking to occur. It is |
|
2154 equivalent to (?!) but easier to read. The Perl documentation notes that it is |
|
2155 probably useful only when combined with (?{}) or (??{}). Those are, of course, |
|
2156 Perl features that are not present in PCRE. The nearest equivalent is the |
|
2157 callout feature, as for example in this pattern: |
|
2158 <pre> |
|
2159 a+(?C)(*FAIL) |
|
2160 </pre> |
|
2161 A match with the string "aaaa" always fails, but the callout is taken before |
|
2162 each backtrack happens (in this example, 10 times). |
|
2163 </P> |
|
2164 <br><b> |
|
2165 Verbs that act after backtracking |
|
2166 </b><br> |
|
2167 <P> |
|
2168 The following verbs do nothing when they are encountered. Matching continues |
|
2169 with what follows, but if there is no subsequent match, a failure is forced. |
|
2170 The verbs differ in exactly what kind of failure occurs. |
|
2171 <pre> |
|
2172 (*COMMIT) |
|
2173 </pre> |
|
2174 This verb causes the whole match to fail outright if the rest of the pattern |
|
2175 does not match. Even if the pattern is unanchored, no further attempts to find |
|
2176 a match by advancing the start point take place. Once (*COMMIT) has been |
|
2177 passed, <b>pcre_exec()</b> is committed to finding a match at the current |
|
2178 starting point, or not at all. For example: |
|
2179 <pre> |
|
2180 a+(*COMMIT)b |
|
2181 </pre> |
|
2182 This matches "xxaab" but not "aacaab". It can be thought of as a kind of |
|
2183 dynamic anchor, or "I've started, so I must finish." |
|
2184 <pre> |
|
2185 (*PRUNE) |
|
2186 </pre> |
|
2187 This verb causes the match to fail at the current position if the rest of the |
|
2188 pattern does not match. If the pattern is unanchored, the normal "bumpalong" |
|
2189 advance to the next starting character then happens. Backtracking can occur as |
|
2190 usual to the left of (*PRUNE), or when matching to the right of (*PRUNE), but |
|
2191 if there is no match to the right, backtracking cannot cross (*PRUNE). |
|
2192 In simple cases, the use of (*PRUNE) is just an alternative to an atomic |
|
2193 group or possessive quantifier, but there are some uses of (*PRUNE) that cannot |
|
2194 be expressed in any other way. |
|
2195 <pre> |
|
2196 (*SKIP) |
|
2197 </pre> |
|
2198 This verb is like (*PRUNE), except that if the pattern is unanchored, the |
|
2199 "bumpalong" advance is not to the next character, but to the position in the |
|
2200 subject where (*SKIP) was encountered. (*SKIP) signifies that whatever text |
|
2201 was matched leading up to it cannot be part of a successful match. Consider: |
|
2202 <pre> |
|
2203 a+(*SKIP)b |
|
2204 </pre> |
|
2205 If the subject is "aaaac...", after the first match attempt fails (starting at |
|
2206 the first character in the string), the starting point skips on to start the |
|
2207 next attempt at "c". Note that a possessive quantifer does not have the same |
|
2208 effect in this example; although it would suppress backtracking during the |
|
2209 first match attempt, the second attempt would start at the second character |
|
2210 instead of skipping on to "c". |
|
2211 <pre> |
|
2212 (*THEN) |
|
2213 </pre> |
|
2214 This verb causes a skip to the next alternation if the rest of the pattern does |
|
2215 not match. That is, it cancels pending backtracking, but only within the |
|
2216 current alternation. Its name comes from the observation that it can be used |
|
2217 for a pattern-based if-then-else block: |
|
2218 <pre> |
|
2219 ( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ... |
|
2220 </pre> |
|
2221 If the COND1 pattern matches, FOO is tried (and possibly further items after |
|
2222 the end of the group if FOO succeeds); on failure the matcher skips to the |
|
2223 second alternative and tries COND2, without backtracking into COND1. If (*THEN) |
|
2224 is used outside of any alternation, it acts exactly like (*PRUNE). |
|
2225 </P> |
|
2226 <br><a name="SEC26" href="#TOC1">SEE ALSO</a><br> |
|
2227 <P> |
|
2228 <b>pcreapi</b>(3), <b>pcrecallout</b>(3), <b>pcrematching</b>(3), <b>pcre</b>(3). |
|
2229 </P> |
|
2230 <br><a name="SEC27" href="#TOC1">AUTHOR</a><br> |
|
2231 <P> |
|
2232 Philip Hazel |
|
2233 <br> |
|
2234 University Computing Service |
|
2235 <br> |
|
2236 Cambridge CB2 3QH, England. |
|
2237 <br> |
|
2238 </P> |
|
2239 <br><a name="SEC28" href="#TOC1">REVISION</a><br> |
|
2240 <P> |
|
2241 Last updated: 19 April 2008 |
|
2242 <br> |
|
2243 Copyright © 1997-2008 University of Cambridge. |
|
2244 <br> |
|
2245 <p> |
|
2246 Return to the <a href="index.html">PCRE index page</a>. |
|
2247 </p> |