License:
BSD style: see license.txt
Version:
Initial release: December 2005
author:
Kris
- enum
Encoding
;
- see http://icu.sourceforge.net/docs/papers/forms_of_unicode/#t2
- class
UnicodeBom
(T): BomSniffer;
- Convert unicode content
Unicode is an encoding of textual material. The purpose of this module
is to interface external-encoding with a programmer-defined internal-
encoding. This internal encoding is declared via the template argument
T, whilst the external encoding is either specified or derived.
Three internal encodings are supported: char, wchar, and dchar. The
methods herein operate upon arrays of this type. That is, decode()
returns an array of the type, while encode() expect an array of said
type.
Supported external encodings are as follow:
Encoding.Unknown
Encoding.UTF_8
Encoding.UTF_8N
Encoding.UTF_16
Encoding.UTF_16BE
Encoding.UTF_16LE
Encoding.UTF_32
Encoding.UTF_32BE
Encoding.UTF_32LE
These can be divided into non-explicit and explicit encodings:
Encoding.Unknown
Encoding.UTF_8
Encoding.UTF_16
Encoding.UTF_32
Encoding.UTF_8N
Encoding.UTF_16BE
Encoding.UTF_16LE
Encoding.UTF_32BE
Encoding.UTF_32LE
The former group of non-explicit encodings may be used to 'discover'
an unknown encoding, by examining the first few bytes of the content
for a signature. This signature is optional, but is often written such
that the content is self-describing. When an encoding is unknown, using
one of the non-explicit encodings will cause the decode() method to look
for a signature and adjust itself accordingly. It is possible that a
ZWNBSP character might be confused with the signature; today's unicode
content is supposed to use the WORD-JOINER character instead.
The group of explicit encodings are for use when the content encoding
is known. These *must* be used when converting back to external encoding,
since written content must be in a known format. It should be noted that,
during a decode() operation, the existence of a signature is in conflict
with these explicit varieties.
See
http://www.utf-8.com/
http://www.hackcraft.net/xmlUnicode/
http://www.unicode.org/faq/utf_bom.html/
http://www.azillionmonkeys.com/qed/unicode.html/
http://icu.sourceforge.net/docs/papers/forms_of_unicode/
- this(Encoding encoding);
- Construct a instance using the given external encoding ~ one
of the Encoding.xx types
- T[]
decode
(void[] content, T[] dst = null, uint* ate = null);
- Convert the provided content. The content is inspected
for a BOM signature, which is stripped. An exception is
thrown if a signature is present when, according to the
encoding type, it should not be. Conversely, An exception
is thrown if there is no known signature where the current
encoding expects one to be present
- void[]
encode
(T[] content, void[] dst = null, uint* ate = null);
- Perform encoding of content. Note that the encoding must be
of the explicit variety by the time we get here
- void[]
swapBytes
(void[] content);
- Swap bytes around, as required by the encoding
- T[]
into
(void[] x, uint type, T[] dst = null, uint* ate = null);
- void[]
from
(T[] x, uint type, void[] dst = null, uint* ate = null);
- class
BomSniffer
;
- final Encoding
encoding
();
- Return the current
encoding
. This is either the originally
specified
encoding
, or a derived one obtained by inspecting
the content for a BOM. The latter is performed as part of
the decode() method
- final void[]
signature
();
- Return the
signature
(BOM) of the current encoding
- final void
setup
(Encoding encoding);
- Configure this instance with unicode converters
- static final Info *
test
(void[] content);
- Scan the BOM signatures looking for a match. We scan in
reverse order to get the longest match first
|
|