CVS log for parser3/src/main/pa_charset.C

Up to [parser3project] / parser3 / src / main

Request diff between arbitrary revisions - Display revisions graphically

Keyword substitution: kv
Default branch: MAIN

Copyright year updated, websites links changed to https://

typo fixed (related to feature #1281)

libxml 2.14 compatibility fixed (implements feature #1281)

Adopted patch from Yavor Doganov <yavor@gnu.org> with extra libxml2 headers including for Debian package

Copyright year updated

minor error messages spelling fixes

copyright, authors actualized

bugfix: +static xml_encoding_handlers as xmlInitCharEncodingHandlers is called in libxml static initializers before gc_substitute_memory_management_functions in pa_globals and thus gc may not see reference to our xmlCharEncodingHandler and free it.

memcpy is more correct and efficient in this case then strncpy

Copyright year updated

regular new/delete no longer used in our code, stubs defend from accidental use (implements feature #1170)

fixed spelling typos from lintian reported by Sergey B Kirpichev (fixes bug #1156)

Copyright year: 2015->2017

cleanup

vs2015 warning war

vs2015 warning war

Copyright year updated

String::Body(String::C) used

exceptions unified

declarations unified

warning war (-pedantic flag)

new(UseGC) replaced with new, malloc/strdup/free replaced with pa_ version and checks for accidental use of non-pa_ version added (related to issue #1018)

*** empty log message ***

avoid compilation bug in Debian 6.0.8 x32, related to issue #896

fixUTF8 method added to replace invalid UTF-8 to '?', related to issue #938

json chars 0x01-0x1F now are escaped as \u00XX, minor optimizations (closes issue #896)

vs2003 warning war

pa_atoui added for out of range checks, zero division in vint/vdouble check added ( fixes #832 )

ident now works under Linux + ident displays filenames (closes issue #818)
Copyright updated

- use pa_isalpha and pa_isalnum instead of isalpha and isalnum
- bug with redundand quoting lowercased latin chars while building email body was fixed

- bugfix: in some cases the calculating string size for transcoding gave too small value (should ever look for availability char in dest charset inspite of the char size)

parser charset tables declare only white-space before 0x20, thus adding the missing chars

small optimization, just to decrease number of lines. :)

- taint[json] was added
- escaping was slightly modified

- method lengthUTF8Char was added

- little hacking: for dealing with transcoding+^taint[uri] transcode_and_untaint was replaced by untaint_and_transcode

no message

- use iterators in pa_charset.C itself

- class UTF8_string_iterator was added

- some code cleanup

- cstr(params) and cstrm(params) were deleted. taint_cstr[m] and untaint_cstr[m] should be used

- more safe creation on String:Body

- hash_code caching was implemented (seaches should be faster now)

- String constructors don't have that stupid true/false param which means 'tainted'. they accept lang instead.

no message

- result optimisation
- helper_length parameter removed from string constructors

- little optimisation: somewhere length() was replaced by is_empty()

- make it x64 friendly

- parser is x64 friendly now

- methods readChar and skipChar which is used for read utf8-strings were renamed
- is_escaped was renamed to isEscaped (to the same name convention)

- rollback changeing readChar to skipChar. these methods read different strings

- readChar => skipChar (in this place we need just skip char)

- type changed for making compiler happy

no message

- moved to new PCRE library

- more escape method-layers added (with different params)

no message

no message

- methods for working with pos/mid for strings in utf-8 were added

- added method escape for escaping cookies as %uXXXX
- before transcode calculate required space for dest string. it reduce mem usage for transcode

no message

- method store_Char added, transcodeCharFromUTF8 -- removed

- method for get char in requested charset from utf code was added

- "parser.runtime" strings were removed

+ ^break[] ^continue[], in ^for, ^while, ^menu, ^hash/hashfile.foreach

! switched to VS2005 (projects and object<info explicitly typed>.for_each(, info))
+ $response:field[] setting void means removing
+ grammar $name` name stops now, for mysql `$field`
+ started ^break[] ^continue[], in ^for. TODO: to other iterators (while, menu, foreach)
+ all log messages += [uri=, method=, cl=]

. started killing gdome

. merged latest fixes from 3.2.0

. less warnings

2005

. 2005 ;)

bugfix: UTF8->one-byte-per-char-encoding bugfix: when there is no char in charset one byte produces 6 (&#255; <<max). increased buffer size!

bugfix: two-bytes invalid chars in TranscodeFromUTF8 considered valid. only >2bytes-long now %HH encoded

convinience: transcodeFromUTF8 now never fails. in case on input appears nonutf,
those bytes will be printed in %HH form. that can be easily decoded/recovered.

this form is quite noticable, and hopefully would not go noticed

2004

bugfix: forgot to undo some

beauty: uncomment PA_PATCHED_LIBXML_BACKWARD to link with old patched libxml libraries

bugfix: memory for charsethandler structure for libxml now allocated properly

$status:sql.cache back operational

more warnings --

more warnings --

static const char * const IDENT

turned on warnings level4 on all projects (except libltdl=off & libsdbm=level3)
found several unitialized vars

hashfile: started again with sdbm from apache

bugfix: had broken binary search [copied thoughtlessly from xalan]

utf-8 upper/lower

string_fragments_to_cord merged to HEAD

v() functions of String::Body, String::Languages and String itself
StringBody->String::Body

merged 3.1.0 latest changes

bad #endif fix

moved tempate_gc to HEAD

merged from HEAD file::exec/cgi .charset

re-added libgdome patch. regretfully libgdome bug can not be worked around

couple more asserts

little transcodes speedup

couple assertions on fantastic situations added [may be those is the case now?]

bugfix on memory buffer overrun [but that's memory from gc_malloc, not from g_malloc :(, but would hope]

bugfix on memory buffer overrun [but that's memory from gc_malloc, not from g_malloc :(, but would hope]

checked custom malloc for returning 0

http://
  $.charset[] param
done

started http://
  $.charset[] param
and
  http response charset detection
just compiled.
todo:test

pa_vform violated String::invariant. started fixing [not compiled now]

*** empty log message ***

--enable-assertions autoconf-ed

worked around  xmlRegisterCharEncodingHandler limitation [currently imposed limit of 10 user-defined charsets]

pre-evaluated .count() in some places

untaint.C L_MAIL_HEADER

fixed dom language [values are now considered tainted.
it was a bug in parser. updated 107 test result

memory: new 'new' overloads for structure handling

xdoc.C compiled
todo: xnode.C / finalizers

started XML

2*2 worked :)

pa_charsets.C compiled

introducing StringBody [C++ CORD wrapper]

$file::exec/cgi[script;
   $.charset[this is script's charset]

command line, env values, input got transcoded before call
stdout, stderr got transcoded after call

untaint.C 30%,
pa_common.C [done, without http for now]

started test

started porting to gc:
PA_Object done
lots of replacements also

libgc attempt

charsets: fixed problems when transcode from charset A to A.

fixed: mem leak from copy/paste bug

xml: test: 107 bug fixes

xml: charset two mem alloc functions used for different cases [libxml, libxsl]

xml: exceptions

xml: charsets

charset_utf8 declaration moved to charsets.C

hash cloning fixed

fixed all cstr's that must use pool [as the one in ^process]

found&kill all remaning leaks resulted from pooled::malloc calls
[excluding in gd -- planning complete rewrite extremely ugly code]

another bug of unitialized simple-typed field fixed.
todo: find&kill all like that one

fixed more leaks resulted from pooled::malloc calls

removed stupid value_includes.h

grammar compiled

ConstStringPtr died, long live StringPtr

vrequest,vdouble,vint compiled

charset/s simplified [charset::pool_for_load introduced]

pa_charsets compiled

charset compiled

2002->2003

From: "Victor Fedoseev" <vvf_ru@mail.ru>
To: "Alexandr Petrosian (PAF)" <PAF@design.ru>
Sent: Monday, December 23, 2002 4:22 AM
Subject: bug â Charset::transcode_buf2xchar

xml->parser strings now have origin, which points to place where value left xml library and came to parser: place of dom field extraction/call

while fixing ( xmlCharEncodingInput/OutputFunc callbacks returned bad value )
forgot to check users of those funcs. not all were using that return value convention

xmlCharEncodingInput/OutputFunc callbacks returned bad value in case of unfinished in buffer processing, causing accidental transcode stop [in case that source enc != utf-8 & there is incomplete utf-8 sequence at the end of 16000block iside of libxml lib]
tfm readed & code updated

$Date: now$

ident.C* removed

comment

UTF-8 to 1byte charset convert, no char in table, &#decimal;

merged latest updates to head

^xdoc::create[[uri]]... base uri for document being created, imports and other relative file names would be relative to this uri
default uri=path_translated

assigned exception types

removed some tested @tests

size_t

name spelling

2002

typedef XMLCh... not stupid illegal define

checked empty transcoders in transcode_cstr|buf

autoconf gnome

started output options parsing

xslt transform0.
TODO:
  use output options
  handle errors

xpath selectNode 1

GdomeDOMString_auto_ptr  c++ wrapper [calls refcounter]

xdoc
^set
^string with glib works 0

started glib transcoders

gnome libs just compiled in.
no refcounting
no xpath
no xslt
  yet

going away from xalan&xerces, started the process.
globals [initialization
pool [charset update
charset [transcodings
vxnode vxdoc [DOM calls

charset key globalized [bug]

charset->charset transcoding via unicode intermediate

ifndef XML were bad type

empty bufs transcode [forgot check :(]

charset_connection&manager replaced by charset&charsets

name_cstr 0

introducing Charset