CVS log for parser3/src/main/pa_common.C

Up to [parser3project] / parser3 / src / main

Request diff between arbitrary revisions - Display revisions graphically

Keyword substitution: kv
Default branch: MAIN

Copyright year updated, websites links changed to https://

inline-storage hash used to avoid allocations for MethodFrame args and locals (imlpements feature #1293)
_stack uses for_each to avoid Iterator for InlineHashString

feature #1273 core implementation: value::as_wint added, implemented for int/double/string, and used where applicable; vint now holds pa_wint type; format_double moved to pa_int

typo fixed (related to bug #1282)

minor optimization + create_dir_for_file checks for \ (related to bug #1282)

_wrename, _wrmdir и _wmkdir are now used under Windows (fixes bug #1282)

file:delete for UTF-8 files now works under Windows (fixes bug #1282)

fixes for --with-build-warnings

errno should not be used on locking errors (related to bug #1231)

Copyright year updated

format replaced with pa_uitoa where applicable, $.date[unix-timestamp] fixed for handling large dates

minor error messages spelling fixes

faster templated pa_itoa/pa_uitoa added to work with any integer type, String::Body::Format removed

Adopted patch from Yavor Doganov <yavor@gnu.org> to add pcre2 library support (implements feature #1236)

clip2int,clip2uint added to sync double->int overflow behavour under different architectures (fixes bug #1232)

avoid (uint)<negative double> conversion as on Apple M1 it's 0

copyright, authors actualized

MSVC SOCKET warnings war

pa_strncpy added ans used to avoid strncpy drawbacks

create_dir_for_file should not create dir for trailing / (fixes bug #1166)

it is more safe to have versions of pa_strdup with one and two arguments,
as helper_length=0 not always mean that strlen() should be called, but means zero.

Copyright year updated

slow file_read_binary no longer used, send_range added

unified uint64_t lseek implemented to support >4Gb files under x86, including Windows

minor bugfix: $.limit() should be checked with max_file_size

support for processing files sized >4Gb on 32 bit planforms

pa_filename added to avoid useless rsplit

httpd config_handler added, stdout flush added.

*** empty log message ***

pa_strcat implemented and used

check_file_size filespec is now optional

entry_exists -> entry_ifdir to fix Windows compilation issue (related to bug #1201)

entry_readable -> entry_exists, undocumented access(fname, R_OK) call removed (fixes bug #1201)
-f and -d now works under Windows in UTF-8 with Russian filenames.

optimization: offset and limit options parsing moved inside file_load.

optimization: file_read replaced with simplified file_read_binary

whitespace

never used params removed

pa_base64.h and pa_base64.C added, base64 implementation moved from pa_common.h and pa_common.C

cleanup: sql_ variables declaration moved to pa_common.h

fixed spelling typos from lintian reported by Sergey B Kirpichev (fixes bug #1156)

Copyright year: 2015->2017

minor fixes, whitespace

wchar filenames API under Windows now used (implements feature #1081)

check_file_size added and used in curl / file load from disk and http, http file load rewritten (related to bug #1014)

warning war

stat -> pa_stat (related to feature #1014)

-wAll warning war

file_stat now return 64-bit size (implements #1014 under 32-bit unix versions)

charsets -> pa_charsets, vs2015 warning war

vs2015 compilation fixes

vs2015 compilation fixes

vs2015 compilation fixes

charsets.get now does str_upper inside (related to feature #1061)

^curl:load[] now support multiple headers, ^file::load supports headers transcoding (implements feature #1042)

pa_common.h: File_read_action: fname, as_text arguments removed;
image.C - absolute path shown in exception (implements feature #1071)

Copyright year updated

checkBOM detects charset if it's not enforced, skips BOM signature if it complies charset (fixes bug #1037)

reduce js flags checks count (related to feature #120)

new double date implementation with internal support for extended 0..9999 year range, default timezone support (related to feature #36)

snprintf comments updated

^string:idna[xn--...], ^string.idna[] - idna conversion support implemented (related to feature #849)

removed most of the clang compiler warnings (closes issue #1018)

$request:headers, $request:method added (implements feature #1007)

- supports body with PUT method as well ( new feature: #1006 )

json chars 0x01-0x1F now are escaped as \u00XX, minor optimizations (closes issue #896)

minor rearrangements and cleanup, code moved from .h to .C, no changes in code

vs2003 warning war

no message

warning war

WIN32 -> _MSC_VER, minor cleanup

includes cleanup

warning war

read error now reported cottectly (fixes issue #933

no more double slashes in file path (fixes bug #872)

- option for keeping empty dirs is added

- search_stop method was moved from pa_vcookie.C to pa_common.C

ident now works under Linux + ident displays filenames (closes issue #818)
Copyright updated

- ^string:base64[encoded;$.strict(true)] now detects invalid base64 chars in the middle of encoded stricg ( new feature: #55 )

- $.strict(true|false) option is added to base64 decode methods ( new feature: #55 )

- base64 decode memory usage was decreased ( new feature: #819 )

- transcode file from utf-8 to $request:charset during loading if the BOM code is detected ( new feature: #98 )

- exception comment strings "options must be hash", "options must be hash, not code" and "called with invalid option" were replaced by constatns
- exception comment string "invalid option passed" was replaced by "called with invalid option" (now in different methods are the same exception comment string)

- junction points should not be deleted on win32 during dirs cleanup ( bugfix: #83 )
- optimisation in dirs cleanup

- bugfix #74 -- memmove should be used instead of memcopy for overlaped regions

- wanring about unused vars was removed

new feature: #71 ^string:js-unescape and cookie parser decode \uXXXX as well as %uXXXX

fixes #64 Now file.missing exception generates in unsafe mode to.

- bugfix: there are no excaption.type if trying to open file by path /existing-file/non-exixting-file

- don't allocate memory during capitalization if input string is already capitalized

- in safe mode clear executable bits during file writing

- use pa_malloc_atomic instead of new

- some magic numbers were removed

- pa_base64_encode uses less memory now

- file_write accepts Request_charsets and asked charset and cound transcode content before writing

- back to storing response http headers in lowercase
- capitalize them during output

- not needed parameter was removed

- cstr(params) and cstrm(params) were deleted. taint_cstr[m] and untaint_cstr[m] should be used

- optimizations and simplification of string.cstr(...)

- one method detect_charset was deleted
- bug was fixed in this method (error while detecting charset in content_type withot last ';')

- result optimisation
- helper_length parameter removed from string constructors

- method print_pcre_exec_error_text was moved to class vregex

- describe one more UTF-8 related error during PCRE compile/execute

- ^file::load[...;http://...;] now can post files (new option $.encode[multipart-form/data] should be specified)
- $.method[] option for file::load now is not case-sensitive

- changes in formating

- some formating changes

- some exceptions while operations with files not typeless anymore

- method for print pcre_exec text error was added

- skip BOM code before transcode
- escape method has new option for skip converting '+' to a space char

- size must be int but not size_t or we can't compare with 0 sprintf result

- option for disable transcoding while file_read and file_read_text added

- some polish

- more helpers methods moved here
- unescape_chars accepn one charset now

- new method for caseless search c-substring in c-string
- use isxdigit instead of is_hex_digit

- use store_Char instead of transcodeCharFromUTF8

- unescape_chars can decode %uXXXX if charset specified
- checks for format before print number (^d.format[>...<], ^eval($d)[>...<])
- formating with spaces (instead of tabs) removed

- comment added

- bugfix for february at leap year

- remove_crlf optimize whitespaces now

- cosmetic changes

- "parser.runtime" strings were removed

- $.charset option for ^file::load[text;/local.txt] and ^table::load[/table.txt] was added
- not needed transcodes were removed from ^file::load[...;http://...]

- definitions for ^file:base64[filespec]

- added file_block_read with read error detection
- file_block_read used instead of native read

- little optimization in getMonthDays
- small changes in crc32 calculation

- CRC32_MAX_BUFFER_SIZE was renamed to FILE_BUFFER_SIZE

- some functions for crc32 calculation added

merged from HEAD
! bugfix: decoding from base64

! bugfix: decoding from base64

. merged latest fixes from 3.2.0

. less warnings

merged from HEAD:
! changed file/dir_readable to simple file/dir_exist, this would help in situations "class not found because .p file has bad rights" << in that case error would be explicit "access denied to 'this' file"

! changed file/dir_readable to simple file/dir_exist, this would help in situations "class not found because .p file has bad rights" << in that case error would be explicit "access denied to 'this' file"

merged from HEAD:
+! ^file::exec/cgi[script;$.charset[changed] $.QUERY_STRING[^untaint[URI]{aaa=$form:text} << now %HH would be encoded in $.charset charset

. extracted http:// into separate file [preparation for ^file::cgi[script;$.form[$.field1[]

. less gcc warnings

. less gcc warnings

merged from HEAD:
+ removed "use either uri with ?params or $.form option" limitation

+ removed "use either uri with ?params or $.form option" limitation

merged from HEAD:
+ ^file.base64[] encode
+ ^file::base64[encoded] decode

+ ^file.base64[] encode
+ ^file::base64[encoded] decode

+ ^string:base64[in] encode
+ ^string.base64[] decode

! merged: file::load[binary;fileOfZeroSize] now loads VFile.ptr!=0, so such files can be saved now

! file::load[binary;fileOfZeroSize] now loads VFile.ptr!=0, so such files can be saved now

no message

no message

! merged from 3.1.5: fixed file load memory issue: now tries to guess content-length and allocates one big piece if possible [regretfully gc_realloc works as malloc+free, leaving lots of holes behind]

! fixed file load memory issue: now tries to guess content-length and allocates one big piece if possible [regretfully gc_realloc works as malloc+free, leaving lots of holes behind]

merged to HEAD:
! table::save with enclosers now doubles them: "->""
! table::save/load do not remove elements from options hash

! table::save with enclosers now doubles them: "->""
! table::save/load do not remove elements from options hash

2005

. 2005 ;)

! bugfix -- recv()==0 is not an error

! bugfix ^file::load[mode;name;$.offset $.limit] now work again [support was broken in 3.1.4]

beauty: hashfile::open now creates dir for new file

DONT_LINGER can cause subsequent failures though defined in .h

comment on volatile

http: unix: alarm function were not used since the beginning due to stupid error [setsigjmp were not checked in configure.in]

http: timeout setsockopt [if possible] on unix too

http: exception status of send/recv errors change to most probable http.timeout

win32: http connection $.timeout option now works [setsockopt on send/receive]

win32 beauty: socket errors properly decoded

bugfix: loads headers with both \r\n and \n separators
bugfix: loads header with endings \r\n\r\n and \n\n
(bloody yandex.server)

beauty: less warnings

beauty: do not bother with charset detection when ^file::load[binary

hack: for yandex.server http server

bugfix: isspace((unsigned char)c) everywhere. failed on russian letters

bugfix: isspace((unsigned char)c) everywhere. failed on russian letters

new: $response:body/download[
	$.file[name on disk]
	$.name[of file for user]
	$.mdate[date of last-modified. default from directory]
]

patched:  ^file:send by Victor Fedoseev <vvf_ru@mail.ru>
todo: turn it to $response:download[
$.filename[filename]
$.option[]
,,
]

beauty: gcc warning removed

bugfix: of 1.175 bugfix: http://i.p.a.ddress gethostbyaddr added [on some platforms gethostbyname failed with such 'domains']
now checked properly, were: needless reverse/forward dns lookup

bugfix: of 1.175 bugfix: http://i.p.a.ddress gethostbyaddr added [on some platforms gethostbyname failed with such 'domains']
now checked properly, were: needless reverse/forward dns lookup

merged with 3.1.3

removed warning

bugfix: ^image.polyline and http status line parsing
checked number of columns in table [user reported an assert]

new: ^file::load[...][options]
        $.form[
            !$.field1[string]
            !$.field2[^table::create{one_column_only^#0Avalue1^#0Avalue2}]
        ]
        !$.body[string]

GET ?here
notGET(POST, HEAD, others) in content-type: application/x-www-form-urlencoded

todo: upload files ability

new: file::load[;http:// GET/POST $.form[$.fields
started, not tested yet
todo:$.field[table] for multiple values

beauty: link to rtf added

new: http:// options $.user $.password basic authorization

beauty: removed needless vars

bugfix: INADDR_NONE not everywhere defined

bugfix: http://i.p.a.ddress gethostbyaddr added [on some platforms gethostbyname failed with such 'domains']

2004

typo: %l -> %u

merged: bugfix from 3.1.1 on memory overflow

bugfix: buffer overflow
* in libxml&xsl error-reporting code
* in parser-error reporting code [when no @unhandled_exception defined]

fixed in one place: my *snprintf override

bugfix: ^file::load[binary;http:// now not garbled  [pieces after zero bytes were removed]

bugfix: empty http response caused gpf

bugfix: ^file::load[binary;http://...]   now not transcodes response body

static const char * const IDENT

more warnings --

turned on warnings level4 on all projects (except libltdl=off & libsdbm=level3)
found several unitialized vars

cache: ^cache[] fallback
todo: kinda ^try:  ^cache[...]{body}{catch code with $exception.cache field}

^hashfile.delete[] removes files altogether [and dir, if could]

no message

locking move to pa_os [along with pa_sleep]
apr-impl locking done

^table::load[; options:
        !$.column-separator[^#09]
        !$.column-encloser["]

read errors would now be reported 'actually read -1 bytes'

bugfix: utf signature were not ignored due to typo error
http://i2/tasks/edit/?id=4573354650786434584

no message

String.for_each bug fixed [omited one-char cases]

string_fragments_to_cord merged to HEAD

v() functions of String::Body, String::Languages and String itself
StringBody->String::Body

merged
$f[^file::load[binary;http://...]]
$f.tables

printf is buffered, write(1 is not. can't flush stdout without referencing 'stdout' symbol. to hell with non-ansi libraries without 'stdout' symbol

got rid of 'stdout' symbol reference on unix (users reported problems with some old libc)

$date.TZ
^date.roll[TZ;new zone]

moved tempate_gc to HEAD

http:// CRLF now [merged from HEAD]

http:// CRLF now

introducing
$f[^file::load[binary;http://...]]
$f.tables

fix_line_breaks bug fix [terminating zero were not appended] which violated string invariant

date format now
Sun, 06 Nov 1994 08:49:37 GMT  ; RFC 822, updated by RFC 1123

^file::load[mode;name;
    $.offset
    $.limit

gcc didn't like (stat xxx,

gcc didn't like (stat xxx,

http://
  $.charset[] param
done

started http://
  $.charset[] param
and
  http response charset detection
just compiled.
todo:test

merged PA_USE_ALARM bugfix from HEAD

merged from head:safe mode error message now includes numbers

safe mode error message now includes numbers

uncommented http:// file loading

uncommented http:// file loading

all calls to String::String(str,helper_length) are checked.
found/fixed one place:
$request:body now zero-terminated,

no message

moved memory debugging to global level: to pa_gc_malloc

introducing append_help_length [radical improvement here]

pre-evaluated .count() in some places

a number of bugfixes [while testing first real site (aval)]

http://

introducing string[body]::pos(char)

size_t pos everywhere checks changed to check for eq STRING_NOT_FOUND

attempt to do gif encoding to CORD_ec [bad]
would rewrite as realloc now

pa_request.C compiled

execute.C compiled

introducing StringBody [C++ CORD wrapper]

} typo bug fix

*** empty log message ***

} typo bug fix

more tests OK

test compiled

untaint.C 99% [except mail&sql&optimize]

untaint.C 30%,
pa_common.C [done, without http for now]

started test

string reimplementation with cord+array<fragment>: 70%

started porting to gc:
PA_Object done
lots of replacements also

*** empty log message ***

*** empty log message ***

libgc attempt

test:
97 file::load http://  [bugs fixed]

typo fixed

stupid gcc [2.95.4] generated bad code
which failed to handle sigsetjmp+throw: crashed inside of pre-throw code.
rewritten simplier [though duplicating closesocket code]

stupid gcc [2.95.4] generated bad code
which failed to handle sigsetjmp+throw: crashed inside of pre-throw code.
rewritten simplier [though duplicating closesocket code]

fixed more leaks resulted from pooled::malloc calls

13 link errors

classes.lib - 0 error(s), 0 warning(s)
parser.C compiled

image compiled

file compiled

hash 50% compiled

removed stupid value_includes.h

libmain.a gcc
and
mail.lib msvc++
compiled

started gcc compiling
pa_common compiled

most pa_request compiled

grammar compiled

ConstStringPtr died, long live StringPtr

#include "pa_value_includes.h"
everywhere in main

common compiled

paused on sql_manager

common compiled

wcontext compiled

more patched

dictionary compiled

resurrected pool in new sense: now it's factory, producing&accounting memory chunks for read[autofree] buffers

connect_string allocated on heap[wes on stack] now. and exception can be reported OK now [can be reported outside of pro
c with that stack]

2002->2003

http://
introducing
	$.any-status(1)

http request now return status. and not fail on status!=200

changing SAFE_MODE politics

pool::copy on zero size|ptr fixed [were really called with zero size when .html?a=&b=]

merged small changes from branch 6 to HEAD

ftruncate having checked

mail header ',' allowed.
still bugs in smtp [only one receiptient works, cc, bcc ignored now, and MAILED ;)]

[strncpy killed, memnchr used]

http header lang of tainted parts becomes http-header

http:// param values now handled exactly like $response:header values.
e.g. can be hash [can have subattributes] & contain date values.

http:// param values lang forced URI

http:// tainted partes now %xx

no message

utf8 prefix ignored @read text

http:// response status!=200 made exception: http.status with source=bad status#

http://
and image const void related probs fixed

http fields now UPPERCASE

$file[^file::load[http://there]]
$file.SERVER

checked http options [invalid onces now reported]
made default
user-agent: paf

table/xdoc/file::load now understand http:// prefix and additional params, sample:
$rates[^xdoc::load[http://www.cbr.ru/scripts/XML_daily.asp?date_req=02/03/2002;
   $.USER-AGENT[parser3]
]]

no message

no message

image.measure internals rewritten. no there's reader.seek, and all's ready for EXIF extraction
[now we fail to measure files with EXIF info]

O_TRUNCATE killed, ftruncate  used instead:
O_TRUNC truncates even exclusevely write-locked file [thanks to Igor Milyakov <virtan@rotabanner.com> for discovering]

^file:exec/cgi [pa_exec] pipe read errors now checked

-d "DIR/" now true

auto.p exists but unreadable - now this - fatal error

when auto.p beside binary [cgi, isapi] not accessible [due to bad rights or whatever] it's error now

$response:field[date]
$response:field[$.xxx[date]]

write checked

write checked

write checked

write checked

$Date: now$

ident.C* removed

lock failures reported now

merged fixed -d (bug)

fixed -d (bug)

int/double get_string now not pool.malloc(MAX_NUMBER) but really neaded

no message

assigned exception types

--disable-foreign-group-files

truncation never occured when writing files on unix [since i've changed cache mech]
fix

strnchr: sanity check added

name spelling

name spelling

2002

compiled under cygwin
added yet another locking function: fcntl

rewritten ^cache to use non-blocking-exclusive-caches

text file read mode on win32 adjusted [speed impact. was off for debugging purpose, but forgotten afterwards]

^file:lock[filename]{code}

removed redundant pool param to some pa_common funcs

removed signed mismatch warnings

merged from r14_simple_cache

exclusive lock file write
shared lock file read

^cache operator

merged from _0014, 1

hashfile bugfix2 more wrong sizes.
^cache behaviour changed: if ^cache(33) would change

0660 bits in file_write & hashfile create

snprintf replacement was wrong

snprintf replacement was wrong

snprintf on buffer full returns
win32 -1
solaris >buffer_size

checked all that

snprintf on buffer full returns
win32 -1
solaris >buffer_size

checked all that

line endings fixing fixed last piece[was still cstr oriented]

line endings fixing got rid of cstr, now must work ok with post multipart

$status:db
fixed used not initialized in db_connection & db_table

started status parser class

completely empty file now considered "read", not ignored [not perfect trick used]

simplier common ifdefs

typing bug in common - bad ifdeff

^string.save[[append;]path]

form post data line endings not reformatted now.

debuggging memory.
string::as_int/double now usually uses stack, not heap [200K out of 17M :)]
VString(String&) not copies reference [29M -> 17M]

form fix_line_breaks

.am

^hashfile.hash[]

makes

switched to c++ exceptions 0

\r\n -> \n   DOS
\r -> \n Macintosh
on all systems

cookie urlencoded [netscape doc reread]

header untainting to UL_HTTP_HEADER, UL_MAIL_HEADER made nonforced [only tainted pieces]

remove_crlf @ SAPI::log

added ident.awk and main.dsp splitted to main+classes+types along to .am

merged3 with before_xml
[merge2 actually didn't happen - i've created tags on wrong branch. so this merge is in fact re-merge]

merged with before_xml

merged latest bugfixes from before_xml branch
^if(def $hash) now true only when ^hash:_count[]!=0
moved
.string .save .file from dom to dnode

dom.xslt  stylesheet compiled&cached

$MAIN:CLASS_PATH now
can be string now
more informative error messages

file:move autocreate/remove dest/src dir

removed debug info from pa_common

configure for solaris -lsocket

file:move

file_read close @ eof when imgsize bug fixed

$file created by file:state += .atime .mtime .ctime
+found&fixed bug with exceptions on get_junction-created objects [they were on wrong pool]

date roll
table calendar

*/
static const char *RCSId="$Id: double.C,v 1.31 2001/06/28 07:41:59 parser Exp $";

static char *RCSId="$Id$";

common:  actual filename '%s'

#include "pa_config_includes.h"
removed from most .C

code documentation ++

-d

common: file_write /// @test mkdirs  file_delete rmdirs

untaint - 	@test mail-header

changed urlencode here and in untaint.C to HTTP standard's " and \" mech

exec @jav

compile-1

fixed vstring:as_vfile  length

image:measure -90

image:measure -100 just compiled

sendmail unix added defaults

exec3. decided exec:cgi to move to file:cgi

exec class just compiled. moved post read to request core

mail:send -10 just compiled

string:match replace 2.  string.cstr(forced lang)

z

todo/tests

vfile in response:body

file:load 1. tainted

started file:load. wrong write. must be self.set

fixed r/w context of code-params2, fixed ntfs hardlink slow dir update

continue on sort

table:save decided to have as it were. stepped back.
grammar: added 'in' 'is' 'lt'&co follow-space check

moved ::save to pa_table.
disabled @auto invocation in ^process

z

file_read stringified

nt error log experiments failed - cgi under iis got no access to log. registereventsource simply failed with 5(illegal call) code, while worked OK as standalone executable.
isapi: used HSE_APPEND_LOG_PARAMETER feature

fixed default content-type allocation storage

cyclic uses ignored. defaulted default content-type

auto.p monkey every target

auto.p monkey

cgi: reinvent document_root and request_uri under iis. autoconf updated. introducing ap_config_includes

some comments

shifted apache branch on main trunc

starting mod_parser3

automake compiled and works. todo:make win32 ifdefs work and add ifdef unistd

z

removing locking

String::UL_

^table:save

file class just compiled

renamed Hash::Value to Val so to doxygen would finlly stop confusing those Value-s

z

auto @brief

cookie class

^taint 0

before error show with parser