[BACK]Return to ChangeLog CVS log [TXT] [DIR] Up to [parser3project] / win32 / pcre

Diff for /win32/pcre/ChangeLog between versions 1.7 and 1.8

version 1.7, 2015/10/13 00:25:39 version 1.8, 2019/12/03 16:37:21
Line 1 Line 1
 ChangeLog for PCRE  ChangeLog for PCRE
 ------------------  ------------------
   
   Note that the PCRE 8.xx series (PCRE1) is now in a bugfix-only state. All
   development is happening in the PCRE2 10.xx series.
   
   
   Version 8.43 23-February-2019
   -----------------------------
   
   1. Some time ago the config macro SUPPORT_UTF8 was changed to SUPPORT_UTF
   because it also applies to UTF-16 and UTF-32. However, this change was not made
   in the pcre2cpp files; consequently the C++ wrapper has from then been compiled
   with a bug in it, which would have been picked up by the unit test except that
   it also had its UTF8 code cut out. The bug was in a global replace when moving
   forward after matching an empty string.
   
   2. The C++ wrapper got broken a long time ago (version 7.3, August 2007) when
   (*CR) was invented (assuming it was the first such start-of-pattern option).
   The wrapper could never handle such patterns because it wraps patterns in
   (?:...)\z in order to support end anchoring. I have hacked in some code to fix
   this, that is, move the wrapping till after any existing start-of-pattern
   special settings.
   
   3. "pcre2grep" (sic) was accidentally mentioned in an error message (fix was
   ported from PCRE2).
   
   4. Typo LCC_ALL for LC_ALL fixed in pcregrep.
   
   5. In a pattern such as /[^\x{100}-\x{ffff}]*[\x80-\xff]/ which has a repeated
   negative class with no characters less than 0x100 followed by a positive class
   with only characters less than 0x100, the first class was incorrectly being
   auto-possessified, causing incorrect match failures.
   
   6. If the only branch in a conditional subpattern was anchored, the whole
   subpattern was treated as anchored, when it should not have been, since the
   assumed empty second branch cannot be anchored. Demonstrated by test patterns
   such as /(?(1)^())b/ or /(?(?=^))b/.
   
   7. Fix subject buffer overread in JIT when UTF is disabled and \X or \R has
   a greater than 1 fixed quantifier. This issue was found by Yunho Kim.
   
   8. If a pattern started with a subroutine call that had a quantifier with a
   minimum of zero, an incorrect "match must start with this character" could be
   recorded. Example: /(?&xxx)*ABC(?<xxx>XYZ)/ would (incorrectly) expect 'A' to
   be the first character of a match.
   
   9. Improve MAP_JIT flag usage on MacOS. Patch by Rich Siegel.
   
   
   Version 8.42 20-March-2018
   --------------------------
   
   1.  Fixed a MIPS issue in the JIT compiler reported by Joshua Kinard.
   
   2.  Fixed outdated real_pcre definitions in pcre.h.in (patch by Evgeny Kotkov).
   
   3.  pcregrep was truncating components of file names to 128 characters when
   processing files with the -r option, and also (some very odd code) truncating
   path names to 512 characters. There is now a check on the absolute length of
   full path file names, which may be up to 2047 characters long.
   
   4.  Using pcre_dfa_exec(), in UTF mode when UCP support was not defined, there
   was the possibility of a false positive match when caselessly matching a "not
   this character" item such as [^\x{1234}] (with a code point greater than 127)
   because the "other case" variable was not being initialized.
   
   5. Although pcre_jit_exec checks whether the pattern is compiled
   in a given mode, it was also expected that at least one mode is available.
   This is fixed and pcre_jit_exec returns with PCRE_ERROR_JIT_BADOPTION
   when the pattern is not optimized by JIT at all.
   
   6. The line number and related variables such as match counts in pcregrep
   were all int variables, causing overflow when files with more than 2147483647
   lines were processed (assuming 32-bit ints). They have all been changed to
   unsigned long ints.
   
   7. If a backreference with a minimum repeat count of zero was first in a
   pattern, apart from assertions, an incorrect first matching character could be
   recorded. For example, for the pattern /(?=(a))\1?b/, "b" was incorrectly set
   as the first character of a match.
   
   8. Fix out-of-bounds read for partial matching of /./ against an empty string
   when the newline type is CRLF.
   
   9. When matching using the the REG_STARTEND feature of the POSIX API with a
   non-zero starting offset, unset capturing groups with lower numbers than a
   group that did capture something were not being correctly returned as "unset"
   (that is, with offset values of -1).
   
   10. Matching the pattern /(*UTF)\C[^\v]+\x80/ against an 8-bit string
   containing multi-code-unit characters caused bad behaviour and possibly a
   crash. This issue was fixed for other kinds of repeat in release 8.37 by change
   38, but repeating character classes were overlooked.
   
   11. A small fix to pcregrep to avoid compiler warnings for -Wformat-overflow=2.
   
   12. Added --enable-jit=auto support to configure.ac.
   
   13. Fix misleading error message in configure.ac.
   
   
   Version 8.41 05-July-2017
   -------------------------
   
   1.  Fixed typo in CMakeLists.txt (wrong number of arguments for
   PCRE_STATIC_RUNTIME (affects MSVC only).
   
   2.  Issue 1 for 8.40 below was not correctly fixed. If pcregrep in multiline
   mode with --only-matching matched several lines, it restarted scanning at the
   next line instead of moving on to the end of the matched string, which can be
   several lines after the start.
   
   3.  Fix a missing else in the JIT compiler reported by 'idaifish'.
   
   4.  A (?# style comment is now ignored between a basic quantifier and a
   following '+' or '?' (example: /X+(?#comment)?Y/.
   
   5.  Avoid use of a potentially overflowing buffer in pcregrep (patch by Petr
   Pisar).
   
   6.  Fuzzers have reported issues in pcretest. These are NOT serious (it is,
   after all, just a test program). However, to stop the reports, some easy ones
   are fixed:
   
       (a) Check for values < 256 when calling isprint() in pcretest.
       (b) Give an error for too big a number after \O.
   
   7.  In the 32-bit library in non-UTF mode, an attempt to find a Unicode
   property for a character with a code point greater than 0x10ffff (the Unicode
   maximum) caused a crash.
   
   8. The alternative matching function, pcre_dfa_exec() misbehaved if it
   encountered a character class with a possessive repeat, for example [a-f]{3}+.
   
   9. When pcretest called pcre_copy_substring() in 32-bit mode, it set the buffer
   length incorrectly, which could result in buffer overflow.
   
   10. Remove redundant line of code (accidentally left in ages ago).
   
   11. Applied C++ patch from Irfan Adilovic to guard 'using std::' directives
   with namespace pcrecpp (Bugzilla #2084).
   
   12. Remove a duplication typo in pcre_tables.c.
   
   13. Fix returned offsets from regexec() when REG_STARTEND is used with a
   starting offset greater than zero.
   
   
   Version 8.40 11-January-2017
   ----------------------------
   
   1.  Using -o with -M in pcregrep could cause unnecessary repeated output when
       the match extended over a line boundary.
   
   2.  Applied Chris Wilson's second patch (Bugzilla #1681) to CMakeLists.txt for
       MSVC static compilation, putting the first patch under a new option.
   
   3.  Fix register overwite in JIT when SSE2 acceleration is enabled.
   
   4.  Ignore "show all captures" (/=) for DFA matching.
   
   5.  Fix JIT unaligned accesses on x86. Patch by Marc Mutz.
   
   6.  In any wide-character mode (8-bit UTF or any 16-bit or 32-bit mode),
       without PCRE_UCP set, a negative character type such as \D in a positive
       class should cause all characters greater than 255 to match, whatever else
       is in the class. There was a bug that caused this not to happen if a
       Unicode property item was added to such a class, for example [\D\P{Nd}] or
       [\W\pL].
   
   7.  When pcretest was outputing information from a callout, the caret indicator
       for the current position in the subject line was incorrect if it was after
       an escape sequence for a character whose code point was greater than
       \x{ff}.
   
   8.  A pattern such as (?<RA>abc)(?(R)xyz) was incorrectly compiled such that
       the conditional was interpreted as a reference to capturing group 1 instead
       of a test for recursion. Any group whose name began with R was
       misinterpreted in this way. (The reference interpretation should only
       happen if the group's name is precisely "R".)
   
   9.  A number of bugs have been mended relating to match start-up optimizations
       when the first thing in a pattern is a positive lookahead. These all
       applied only when PCRE_NO_START_OPTIMIZE was *not* set:
   
       (a) A pattern such as (?=.*X)X$ was incorrectly optimized as if it needed
           both an initial 'X' and a following 'X'.
       (b) Some patterns starting with an assertion that started with .* were
           incorrectly optimized as having to match at the start of the subject or
           after a newline. There are cases where this is not true, for example,
           (?=.*[A-Z])(?=.{8,16})(?!.*[\s]) matches after the start in lines that
           start with spaces. Starting .* in an assertion is no longer taken as an
           indication of matching at the start (or after a newline).
   
   
   Version 8.39 14-June-2016
   -------------------------
   
   1.  If PCRE_AUTO_CALLOUT was set on a pattern that had a (?# comment between
       an item and its qualifier (for example, A(?#comment)?B) pcre_compile()
       misbehaved. This bug was found by the LLVM fuzzer.
   
   2.  Similar to the above, if an isolated \E was present between an item and its
       qualifier when PCRE_AUTO_CALLOUT was set, pcre_compile() misbehaved. This
       bug was found by the LLVM fuzzer.
   
   3.  Further to 8.38/46, negated classes such as [^[:^ascii:]\d] were also not
       working correctly in UCP mode.
   
   4.  The POSIX wrapper function regexec() crashed if the option REG_STARTEND
       was set when the pmatch argument was NULL. It now returns REG_INVARG.
   
   5.  Allow for up to 32-bit numbers in the ordin() function in pcregrep.
   
   6.  An empty \Q\E sequence between an item and its qualifier caused
       pcre_compile() to misbehave when auto callouts were enabled. This bug was
       found by the LLVM fuzzer.
   
   7.  If a pattern that was compiled with PCRE_EXTENDED started with white
       space or a #-type comment that was followed by (?-x), which turns off
       PCRE_EXTENDED, and there was no subsequent (?x) to turn it on again,
       pcre_compile() assumed that (?-x) applied to the whole pattern and
       consequently mis-compiled it. This bug was found by the LLVM fuzzer.
   
   8.  A call of pcre_copy_named_substring() for a named substring whose number
       was greater than the space in the ovector could cause a crash.
   
   9.  Yet another buffer overflow bug involved duplicate named groups with a
       group that reset capture numbers (compare 8.38/7 below). Once again, I have
       just allowed for more memory, even if not needed. (A proper fix is
       implemented in PCRE2, but it involves a lot of refactoring.)
   
   10. pcre_get_substring_list() crashed if the use of \K in a match caused the
       start of the match to be earlier than the end.
   
   11. Migrating appropriate PCRE2 JIT improvements to PCRE.
   
   12. A pattern such as /(?<=((?C)0))/, which has a callout inside a lookbehind
       assertion, caused pcretest to generate incorrect output, and also to read
       uninitialized memory (detected by ASAN or valgrind).
   
   13. A pattern that included (*ACCEPT) in the middle of a sufficiently deeply
       nested set of parentheses of sufficient size caused an overflow of the
       compiling workspace (which was diagnosed, but of course is not desirable).
   
   14. And yet another buffer overflow bug involving duplicate named groups, this
       time nested, with a nested back reference. Yet again, I have just allowed
       for more memory, because anything more needs all the refactoring that has
       been done for PCRE2. An example pattern that provoked this bug is:
       /((?J)(?'R'(?'R'(?'R'(?'R'(?'R'(?|(\k'R'))))))))/ and the bug was
       registered as CVE-2016-1283.
   
   15. pcretest went into a loop if global matching was requested with an ovector
       size less than 2. It now gives an error message. This bug was found by
       afl-fuzz.
   
   16. An invalid pattern fragment such as (?(?C)0 was not diagnosing an error
       ("assertion expected") when (?(?C) was not followed by an opening
       parenthesis.
   
   17. Fixed typo ("&&" for "&") in pcre_study(). Fortunately, this could not
       actually affect anything, by sheer luck.
   
   18. Applied Chris Wilson's patch (Bugzilla #1681) to CMakeLists.txt for MSVC
       static compilation.
   
   19. Modified the RunTest script to incorporate a valgrind suppressions file so
       that certain errors, provoked by the SSE2 instruction set when JIT is used,
       are ignored.
   
   20. A racing condition is fixed in JIT reported by Mozilla.
   
   21. Minor code refactor to avoid "array subscript is below array bounds"
       compiler warning.
   
   22. Minor code refactor to avoid "left shift of negative number" warning.
   
   23. Fix typo causing compile error when 16- or 32-bit JIT is compiled without
       UCP support.
   
   24. Refactor to avoid compiler warnings in pcrecpp.cc.
   
   25. Refactor to fix a typo in pcre_jit_test.c
   
   26. Patch to support compiling pcrecpp.cc with Intel compiler.
   
   
   Version 8.38 23-November-2015
   -----------------------------
   
   1.  If a group that contained a recursive back reference also contained a
       forward reference subroutine call followed by a non-forward-reference
       subroutine call, for example /.((?2)(?R)\1)()/, pcre_compile() failed to
       compile correct code, leading to undefined behaviour or an internally
       detected error. This bug was discovered by the LLVM fuzzer.
   
   2.  Quantification of certain items (e.g. atomic back references) could cause
       incorrect code to be compiled when recursive forward references were
       involved. For example, in this pattern: /(?1)()((((((\1++))\x85)+)|))/.
       This bug was discovered by the LLVM fuzzer.
   
   3.  A repeated conditional group whose condition was a reference by name caused
       a buffer overflow if there was more than one group with the given name.
       This bug was discovered by the LLVM fuzzer.
   
   4.  A recursive back reference by name within a group that had the same name as
       another group caused a buffer overflow. For example:
       /(?J)(?'d'(?'d'\g{d}))/. This bug was discovered by the LLVM fuzzer.
   
   5.  A forward reference by name to a group whose number is the same as the
       current group, for example in this pattern: /(?|(\k'Pm')|(?'Pm'))/, caused
       a buffer overflow at compile time. This bug was discovered by the LLVM
       fuzzer.
   
   6.  A lookbehind assertion within a set of mutually recursive subpatterns could
       provoke a buffer overflow. This bug was discovered by the LLVM fuzzer.
   
   7.  Another buffer overflow bug involved duplicate named groups with a
       reference between their definition, with a group that reset capture
       numbers, for example: /(?J:(?|(?'R')(\k'R')|((?'R'))))/. This has been
       fixed by always allowing for more memory, even if not needed. (A proper fix
       is implemented in PCRE2, but it involves more refactoring.)
   
   8.  There was no check for integer overflow in subroutine calls such as (?123).
   
   9.  The table entry for \l in EBCDIC environments was incorrect, leading to its
       being treated as a literal 'l' instead of causing an error.
   
   10. There was a buffer overflow if pcre_exec() was called with an ovector of
       size 1. This bug was found by american fuzzy lop.
   
   11. If a non-capturing group containing a conditional group that could match
       an empty string was repeated, it was not identified as matching an empty
       string itself. For example: /^(?:(?(1)x|)+)+$()/.
   
   12. In an EBCDIC environment, pcretest was mishandling the escape sequences
       \a and \e in test subject lines.
   
   13. In an EBCDIC environment, \a in a pattern was converted to the ASCII
       instead of the EBCDIC value.
   
   14. The handling of \c in an EBCDIC environment has been revised so that it is
       now compatible with the specification in Perl's perlebcdic page.
   
   15. The EBCDIC character 0x41 is a non-breaking space, equivalent to 0xa0 in
       ASCII/Unicode. This has now been added to the list of characters that are
       recognized as white space in EBCDIC.
   
   16. When PCRE was compiled without UCP support, the use of \p and \P gave an
       error (correctly) when used outside a class, but did not give an error
       within a class.
   
   17. \h within a class was incorrectly compiled in EBCDIC environments.
   
   18. A pattern with an unmatched closing parenthesis that contained a backward
       assertion which itself contained a forward reference caused buffer
       overflow. And example pattern is: /(?=di(?<=(?1))|(?=(.))))/.
   
   19. JIT should return with error when the compiled pattern requires more stack
       space than the maximum.
   
   20. A possessively repeated conditional group that could match an empty string,
       for example, /(?(R))*+/, was incorrectly compiled.
   
   21. Fix infinite recursion in the JIT compiler when certain patterns such as
       /(?:|a|){100}x/ are analysed.
   
   22. Some patterns with character classes involving [: and \\ were incorrectly
       compiled and could cause reading from uninitialized memory or an incorrect
       error diagnosis.
   
   23. Pathological patterns containing many nested occurrences of [: caused
       pcre_compile() to run for a very long time.
   
   24. A conditional group with only one branch has an implicit empty alternative
       branch and must therefore be treated as potentially matching an empty
       string.
   
   25. If (?R was followed by - or + incorrect behaviour happened instead of a
       diagnostic.
   
   26. Arrange to give up on finding the minimum matching length for overly
       complex patterns.
   
   27. Similar to (4) above: in a pattern with duplicated named groups and an
       occurrence of (?| it is possible for an apparently non-recursive back
       reference to become recursive if a later named group with the relevant
       number is encountered. This could lead to a buffer overflow. Wen Guanxing
       from Venustech ADLAB discovered this bug.
   
   28. If pcregrep was given the -q option with -c or -l, or when handling a
       binary file, it incorrectly wrote output to stdout.
   
   29. The JIT compiler did not restore the control verb head in case of *THEN
       control verbs. This issue was found by Karl Skomski with a custom LLVM
       fuzzer.
   
   30. Error messages for syntax errors following \g and \k were giving inaccurate
       offsets in the pattern.
   
   31. Added a check for integer overflow in conditions (?(<digits>) and
       (?(R<digits>). This omission was discovered by Karl Skomski with the LLVM
       fuzzer.
   
   32. Handling recursive references such as (?2) when the reference is to a group
       later in the pattern uses code that is very hacked about and error-prone.
       It has been re-written for PCRE2. Here in PCRE1, a check has been added to
       give an internal error if it is obvious that compiling has gone wrong.
   
   33. The JIT compiler should not check repeats after a {0,1} repeat byte code.
       This issue was found by Karl Skomski with a custom LLVM fuzzer.
   
   34. The JIT compiler should restore the control chain for empty possessive
       repeats. This issue was found by Karl Skomski with a custom LLVM fuzzer.
   
   35. Match limit check added to JIT recursion. This issue was found by Karl
       Skomski with a custom LLVM fuzzer.
   
   36. Yet another case similar to 27 above has been circumvented by an
       unconditional allocation of extra memory. This issue is fixed "properly" in
       PCRE2 by refactoring the way references are handled. Wen Guanxing
       from Venustech ADLAB discovered this bug.
   
   37. Fix two assertion fails in JIT. These issues were found by Karl Skomski
       with a custom LLVM fuzzer.
   
   38. Fixed a corner case of range optimization in JIT.
   
   39. An incorrect error "overran compiling workspace" was given if there were
       exactly enough group forward references such that the last one extended
       into the workspace safety margin. The next one would have expanded the
       workspace. The test for overflow was not including the safety margin.
   
   40. A match limit issue is fixed in JIT which was found by Karl Skomski
       with a custom LLVM fuzzer.
   
   41. Remove the use of /dev/null in testdata/testinput2, because it doesn't
       work under Windows. (Why has it taken so long for anyone to notice?)
   
   42. In a character class such as [\W\p{Any}] where both a negative-type escape
       ("not a word character") and a property escape were present, the property
       escape was being ignored.
   
   43. Fix crash caused by very long (*MARK) or (*THEN) names.
   
   44. A sequence such as [[:punct:]b] that is, a POSIX character class followed
       by a single ASCII character in a class item, was incorrectly compiled in
       UCP mode. The POSIX class got lost, but only if the single character
       followed it.
   
   45. [:punct:] in UCP mode was matching some characters in the range 128-255
       that should not have been matched.
   
   46. If [:^ascii:] or [:^xdigit:] or [:^cntrl:] are present in a non-negated
       class, all characters with code points greater than 255 are in the class.
       When a Unicode property was also in the class (if PCRE_UCP is set, escapes
       such as \w are turned into Unicode properties), wide characters were not
       correctly handled, and could fail to match.
   
   
 Version 8.37 28-April-2015  Version 8.37 28-April-2015
 --------------------------  --------------------------
   
Removed from v.1.7  
changed lines
  Added in v.1.8

E-mail: