diff -r ffa851df0825 -r 2fb8b9db1c86 symbian-qemu-0.9.1-12/python-2.6.1/Doc/c-api/unicode.rst --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/symbian-qemu-0.9.1-12/python-2.6.1/Doc/c-api/unicode.rst Fri Jul 31 15:01:17 2009 +0100 @@ -0,0 +1,811 @@ +.. highlightlang:: c + +.. _unicodeobjects: + +Unicode Objects and Codecs +-------------------------- + +.. sectionauthor:: Marc-Andre Lemburg + +Unicode Objects +^^^^^^^^^^^^^^^ + + +These are the basic Unicode object types used for the Unicode implementation in +Python: + +.. % --- Unicode Type ------------------------------------------------------- + + +.. ctype:: Py_UNICODE + + This type represents the storage type which is used by Python internally as + basis for holding Unicode ordinals. Python's default builds use a 16-bit type + for :ctype:`Py_UNICODE` and store Unicode values internally as UCS2. It is also + possible to build a UCS4 version of Python (most recent Linux distributions come + with UCS4 builds of Python). These builds then use a 32-bit type for + :ctype:`Py_UNICODE` and store Unicode data internally as UCS4. On platforms + where :ctype:`wchar_t` is available and compatible with the chosen Python + Unicode build variant, :ctype:`Py_UNICODE` is a typedef alias for + :ctype:`wchar_t` to enhance native platform compatibility. On all other + platforms, :ctype:`Py_UNICODE` is a typedef alias for either :ctype:`unsigned + short` (UCS2) or :ctype:`unsigned long` (UCS4). + +Note that UCS2 and UCS4 Python builds are not binary compatible. Please keep +this in mind when writing extensions or interfaces. + + +.. ctype:: PyUnicodeObject + + This subtype of :ctype:`PyObject` represents a Python Unicode object. + + +.. cvar:: PyTypeObject PyUnicode_Type + + This instance of :ctype:`PyTypeObject` represents the Python Unicode type. It + is exposed to Python code as ``unicode`` and ``types.UnicodeType``. + +The following APIs are really C macros and can be used to do fast checks and to +access internal read-only data of Unicode objects: + + +.. cfunction:: int PyUnicode_Check(PyObject *o) + + Return true if the object *o* is a Unicode object or an instance of a Unicode + subtype. + + .. versionchanged:: 2.2 + Allowed subtypes to be accepted. + + +.. cfunction:: int PyUnicode_CheckExact(PyObject *o) + + Return true if the object *o* is a Unicode object, but not an instance of a + subtype. + + .. versionadded:: 2.2 + + +.. cfunction:: Py_ssize_t PyUnicode_GET_SIZE(PyObject *o) + + Return the size of the object. *o* has to be a :ctype:`PyUnicodeObject` (not + checked). + + +.. cfunction:: Py_ssize_t PyUnicode_GET_DATA_SIZE(PyObject *o) + + Return the size of the object's internal buffer in bytes. *o* has to be a + :ctype:`PyUnicodeObject` (not checked). + + +.. cfunction:: Py_UNICODE* PyUnicode_AS_UNICODE(PyObject *o) + + Return a pointer to the internal :ctype:`Py_UNICODE` buffer of the object. *o* + has to be a :ctype:`PyUnicodeObject` (not checked). + + +.. cfunction:: const char* PyUnicode_AS_DATA(PyObject *o) + + Return a pointer to the internal buffer of the object. *o* has to be a + :ctype:`PyUnicodeObject` (not checked). + + +.. cfunction:: int PyUnicode_ClearFreeList(void) + + Clear the free list. Return the total number of freed items. + + .. versionadded:: 2.6 + +Unicode provides many different character properties. The most often needed ones +are available through these macros which are mapped to C functions depending on +the Python configuration. + +.. % --- Unicode character properties --------------------------------------- + + +.. cfunction:: int Py_UNICODE_ISSPACE(Py_UNICODE ch) + + Return 1 or 0 depending on whether *ch* is a whitespace character. + + +.. cfunction:: int Py_UNICODE_ISLOWER(Py_UNICODE ch) + + Return 1 or 0 depending on whether *ch* is a lowercase character. + + +.. cfunction:: int Py_UNICODE_ISUPPER(Py_UNICODE ch) + + Return 1 or 0 depending on whether *ch* is an uppercase character. + + +.. cfunction:: int Py_UNICODE_ISTITLE(Py_UNICODE ch) + + Return 1 or 0 depending on whether *ch* is a titlecase character. + + +.. cfunction:: int Py_UNICODE_ISLINEBREAK(Py_UNICODE ch) + + Return 1 or 0 depending on whether *ch* is a linebreak character. + + +.. cfunction:: int Py_UNICODE_ISDECIMAL(Py_UNICODE ch) + + Return 1 or 0 depending on whether *ch* is a decimal character. + + +.. cfunction:: int Py_UNICODE_ISDIGIT(Py_UNICODE ch) + + Return 1 or 0 depending on whether *ch* is a digit character. + + +.. cfunction:: int Py_UNICODE_ISNUMERIC(Py_UNICODE ch) + + Return 1 or 0 depending on whether *ch* is a numeric character. + + +.. cfunction:: int Py_UNICODE_ISALPHA(Py_UNICODE ch) + + Return 1 or 0 depending on whether *ch* is an alphabetic character. + + +.. cfunction:: int Py_UNICODE_ISALNUM(Py_UNICODE ch) + + Return 1 or 0 depending on whether *ch* is an alphanumeric character. + +These APIs can be used for fast direct character conversions: + + +.. cfunction:: Py_UNICODE Py_UNICODE_TOLOWER(Py_UNICODE ch) + + Return the character *ch* converted to lower case. + + +.. cfunction:: Py_UNICODE Py_UNICODE_TOUPPER(Py_UNICODE ch) + + Return the character *ch* converted to upper case. + + +.. cfunction:: Py_UNICODE Py_UNICODE_TOTITLE(Py_UNICODE ch) + + Return the character *ch* converted to title case. + + +.. cfunction:: int Py_UNICODE_TODECIMAL(Py_UNICODE ch) + + Return the character *ch* converted to a decimal positive integer. Return + ``-1`` if this is not possible. This macro does not raise exceptions. + + +.. cfunction:: int Py_UNICODE_TODIGIT(Py_UNICODE ch) + + Return the character *ch* converted to a single digit integer. Return ``-1`` if + this is not possible. This macro does not raise exceptions. + + +.. cfunction:: double Py_UNICODE_TONUMERIC(Py_UNICODE ch) + + Return the character *ch* converted to a double. Return ``-1.0`` if this is not + possible. This macro does not raise exceptions. + +To create Unicode objects and access their basic sequence properties, use these +APIs: + +.. % --- Plain Py_UNICODE --------------------------------------------------- + + +.. cfunction:: PyObject* PyUnicode_FromUnicode(const Py_UNICODE *u, Py_ssize_t size) + + Create a Unicode Object from the Py_UNICODE buffer *u* of the given size. *u* + may be *NULL* which causes the contents to be undefined. It is the user's + responsibility to fill in the needed data. The buffer is copied into the new + object. If the buffer is not *NULL*, the return value might be a shared object. + Therefore, modification of the resulting Unicode object is only allowed when *u* + is *NULL*. + + +.. cfunction:: Py_UNICODE* PyUnicode_AsUnicode(PyObject *unicode) + + Return a read-only pointer to the Unicode object's internal :ctype:`Py_UNICODE` + buffer, *NULL* if *unicode* is not a Unicode object. + + +.. cfunction:: Py_ssize_t PyUnicode_GetSize(PyObject *unicode) + + Return the length of the Unicode object. + + +.. cfunction:: PyObject* PyUnicode_FromEncodedObject(PyObject *obj, const char *encoding, const char *errors) + + Coerce an encoded object *obj* to an Unicode object and return a reference with + incremented refcount. + + String and other char buffer compatible objects are decoded according to the + given encoding and using the error handling defined by errors. Both can be + *NULL* to have the interface use the default values (see the next section for + details). + + All other objects, including Unicode objects, cause a :exc:`TypeError` to be + set. + + The API returns *NULL* if there was an error. The caller is responsible for + decref'ing the returned objects. + + +.. cfunction:: PyObject* PyUnicode_FromObject(PyObject *obj) + + Shortcut for ``PyUnicode_FromEncodedObject(obj, NULL, "strict")`` which is used + throughout the interpreter whenever coercion to Unicode is needed. + +If the platform supports :ctype:`wchar_t` and provides a header file wchar.h, +Python can interface directly to this type using the following functions. +Support is optimized if Python's own :ctype:`Py_UNICODE` type is identical to +the system's :ctype:`wchar_t`. + +.. % --- wchar_t support for platforms which support it --------------------- + + +.. cfunction:: PyObject* PyUnicode_FromWideChar(const wchar_t *w, Py_ssize_t size) + + Create a Unicode object from the :ctype:`wchar_t` buffer *w* of the given size. + Return *NULL* on failure. + + +.. cfunction:: Py_ssize_t PyUnicode_AsWideChar(PyUnicodeObject *unicode, wchar_t *w, Py_ssize_t size) + + Copy the Unicode object contents into the :ctype:`wchar_t` buffer *w*. At most + *size* :ctype:`wchar_t` characters are copied (excluding a possibly trailing + 0-termination character). Return the number of :ctype:`wchar_t` characters + copied or -1 in case of an error. Note that the resulting :ctype:`wchar_t` + string may or may not be 0-terminated. It is the responsibility of the caller + to make sure that the :ctype:`wchar_t` string is 0-terminated in case this is + required by the application. + + +.. _builtincodecs: + +Built-in Codecs +^^^^^^^^^^^^^^^ + +Python provides a set of builtin codecs which are written in C for speed. All of +these codecs are directly usable via the following functions. + +Many of the following APIs take two arguments encoding and errors. These +parameters encoding and errors have the same semantics as the ones of the +builtin unicode() Unicode object constructor. + +Setting encoding to *NULL* causes the default encoding to be used which is +ASCII. The file system calls should use :cdata:`Py_FileSystemDefaultEncoding` +as the encoding for file names. This variable should be treated as read-only: On +some systems, it will be a pointer to a static string, on others, it will change +at run-time (such as when the application invokes setlocale). + +Error handling is set by errors which may also be set to *NULL* meaning to use +the default handling defined for the codec. Default error handling for all +builtin codecs is "strict" (:exc:`ValueError` is raised). + +The codecs all use a similar interface. Only deviation from the following +generic ones are documented for simplicity. + +These are the generic codec APIs: + +.. % --- Generic Codecs ----------------------------------------------------- + + +.. cfunction:: PyObject* PyUnicode_Decode(const char *s, Py_ssize_t size, const char *encoding, const char *errors) + + Create a Unicode object by decoding *size* bytes of the encoded string *s*. + *encoding* and *errors* have the same meaning as the parameters of the same name + in the :func:`unicode` builtin function. The codec to be used is looked up + using the Python codec registry. Return *NULL* if an exception was raised by + the codec. + + +.. cfunction:: PyObject* PyUnicode_Encode(const Py_UNICODE *s, Py_ssize_t size, const char *encoding, const char *errors) + + Encode the :ctype:`Py_UNICODE` buffer of the given size and return a Python + string object. *encoding* and *errors* have the same meaning as the parameters + of the same name in the Unicode :meth:`encode` method. The codec to be used is + looked up using the Python codec registry. Return *NULL* if an exception was + raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_AsEncodedString(PyObject *unicode, const char *encoding, const char *errors) + + Encode a Unicode object and return the result as Python string object. + *encoding* and *errors* have the same meaning as the parameters of the same name + in the Unicode :meth:`encode` method. The codec to be used is looked up using + the Python codec registry. Return *NULL* if an exception was raised by the + codec. + +These are the UTF-8 codec APIs: + +.. % --- UTF-8 Codecs ------------------------------------------------------- + + +.. cfunction:: PyObject* PyUnicode_DecodeUTF8(const char *s, Py_ssize_t size, const char *errors) + + Create a Unicode object by decoding *size* bytes of the UTF-8 encoded string + *s*. Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_DecodeUTF8Stateful(const char *s, Py_ssize_t size, const char *errors, Py_ssize_t *consumed) + + If *consumed* is *NULL*, behave like :cfunc:`PyUnicode_DecodeUTF8`. If + *consumed* is not *NULL*, trailing incomplete UTF-8 byte sequences will not be + treated as an error. Those bytes will not be decoded and the number of bytes + that have been decoded will be stored in *consumed*. + + .. versionadded:: 2.4 + + +.. cfunction:: PyObject* PyUnicode_EncodeUTF8(const Py_UNICODE *s, Py_ssize_t size, const char *errors) + + Encode the :ctype:`Py_UNICODE` buffer of the given size using UTF-8 and return a + Python string object. Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_AsUTF8String(PyObject *unicode) + + Encode a Unicode object using UTF-8 and return the result as Python string + object. Error handling is "strict". Return *NULL* if an exception was raised + by the codec. + +These are the UTF-32 codec APIs: + +.. % --- UTF-32 Codecs ------------------------------------------------------ */ + + +.. cfunction:: PyObject* PyUnicode_DecodeUTF32(const char *s, Py_ssize_t size, const char *errors, int *byteorder) + + Decode *length* bytes from a UTF-32 encoded buffer string and return the + corresponding Unicode object. *errors* (if non-*NULL*) defines the error + handling. It defaults to "strict". + + If *byteorder* is non-*NULL*, the decoder starts decoding using the given byte + order:: + + *byteorder == -1: little endian + *byteorder == 0: native order + *byteorder == 1: big endian + + and then switches if the first four bytes of the input data are a byte order mark + (BOM) and the specified byte order is native order. This BOM is not copied into + the resulting Unicode string. After completion, *\*byteorder* is set to the + current byte order at the end of input data. + + In a narrow build codepoints outside the BMP will be decoded as surrogate pairs. + + If *byteorder* is *NULL*, the codec starts in native order mode. + + Return *NULL* if an exception was raised by the codec. + + .. versionadded:: 2.6 + + +.. cfunction:: PyObject* PyUnicode_DecodeUTF32Stateful(const char *s, Py_ssize_t size, const char *errors, int *byteorder, Py_ssize_t *consumed) + + If *consumed* is *NULL*, behave like :cfunc:`PyUnicode_DecodeUTF32`. If + *consumed* is not *NULL*, :cfunc:`PyUnicode_DecodeUTF32Stateful` will not treat + trailing incomplete UTF-32 byte sequences (such as a number of bytes not divisible + by four) as an error. Those bytes will not be decoded and the number of bytes + that have been decoded will be stored in *consumed*. + + .. versionadded:: 2.6 + + +.. cfunction:: PyObject* PyUnicode_EncodeUTF32(const Py_UNICODE *s, Py_ssize_t size, const char *errors, int byteorder) + + Return a Python bytes object holding the UTF-32 encoded value of the Unicode + data in *s*. If *byteorder* is not ``0``, output is written according to the + following byte order:: + + byteorder == -1: little endian + byteorder == 0: native byte order (writes a BOM mark) + byteorder == 1: big endian + + If byteorder is ``0``, the output string will always start with the Unicode BOM + mark (U+FEFF). In the other two modes, no BOM mark is prepended. + + If *Py_UNICODE_WIDE* is not defined, surrogate pairs will be output + as a single codepoint. + + Return *NULL* if an exception was raised by the codec. + + .. versionadded:: 2.6 + + +.. cfunction:: PyObject* PyUnicode_AsUTF32String(PyObject *unicode) + + Return a Python string using the UTF-32 encoding in native byte order. The + string always starts with a BOM mark. Error handling is "strict". Return + *NULL* if an exception was raised by the codec. + + .. versionadded:: 2.6 + + +These are the UTF-16 codec APIs: + +.. % --- UTF-16 Codecs ------------------------------------------------------ */ + + +.. cfunction:: PyObject* PyUnicode_DecodeUTF16(const char *s, Py_ssize_t size, const char *errors, int *byteorder) + + Decode *length* bytes from a UTF-16 encoded buffer string and return the + corresponding Unicode object. *errors* (if non-*NULL*) defines the error + handling. It defaults to "strict". + + If *byteorder* is non-*NULL*, the decoder starts decoding using the given byte + order:: + + *byteorder == -1: little endian + *byteorder == 0: native order + *byteorder == 1: big endian + + and then switches if the first two bytes of the input data are a byte order mark + (BOM) and the specified byte order is native order. This BOM is not copied into + the resulting Unicode string. After completion, *\*byteorder* is set to the + current byte order at the. + + If *byteorder* is *NULL*, the codec starts in native order mode. + + Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_DecodeUTF16Stateful(const char *s, Py_ssize_t size, const char *errors, int *byteorder, Py_ssize_t *consumed) + + If *consumed* is *NULL*, behave like :cfunc:`PyUnicode_DecodeUTF16`. If + *consumed* is not *NULL*, :cfunc:`PyUnicode_DecodeUTF16Stateful` will not treat + trailing incomplete UTF-16 byte sequences (such as an odd number of bytes or a + split surrogate pair) as an error. Those bytes will not be decoded and the + number of bytes that have been decoded will be stored in *consumed*. + + .. versionadded:: 2.4 + + +.. cfunction:: PyObject* PyUnicode_EncodeUTF16(const Py_UNICODE *s, Py_ssize_t size, const char *errors, int byteorder) + + Return a Python string object holding the UTF-16 encoded value of the Unicode + data in *s*. If *byteorder* is not ``0``, output is written according to the + following byte order:: + + byteorder == -1: little endian + byteorder == 0: native byte order (writes a BOM mark) + byteorder == 1: big endian + + If byteorder is ``0``, the output string will always start with the Unicode BOM + mark (U+FEFF). In the other two modes, no BOM mark is prepended. + + If *Py_UNICODE_WIDE* is defined, a single :ctype:`Py_UNICODE` value may get + represented as a surrogate pair. If it is not defined, each :ctype:`Py_UNICODE` + values is interpreted as an UCS-2 character. + + Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_AsUTF16String(PyObject *unicode) + + Return a Python string using the UTF-16 encoding in native byte order. The + string always starts with a BOM mark. Error handling is "strict". Return + *NULL* if an exception was raised by the codec. + +These are the "Unicode Escape" codec APIs: + +.. % --- Unicode-Escape Codecs ---------------------------------------------- + + +.. cfunction:: PyObject* PyUnicode_DecodeUnicodeEscape(const char *s, Py_ssize_t size, const char *errors) + + Create a Unicode object by decoding *size* bytes of the Unicode-Escape encoded + string *s*. Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_EncodeUnicodeEscape(const Py_UNICODE *s, Py_ssize_t size) + + Encode the :ctype:`Py_UNICODE` buffer of the given size using Unicode-Escape and + return a Python string object. Return *NULL* if an exception was raised by the + codec. + + +.. cfunction:: PyObject* PyUnicode_AsUnicodeEscapeString(PyObject *unicode) + + Encode a Unicode object using Unicode-Escape and return the result as Python + string object. Error handling is "strict". Return *NULL* if an exception was + raised by the codec. + +These are the "Raw Unicode Escape" codec APIs: + +.. % --- Raw-Unicode-Escape Codecs ------------------------------------------ + + +.. cfunction:: PyObject* PyUnicode_DecodeRawUnicodeEscape(const char *s, Py_ssize_t size, const char *errors) + + Create a Unicode object by decoding *size* bytes of the Raw-Unicode-Escape + encoded string *s*. Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_EncodeRawUnicodeEscape(const Py_UNICODE *s, Py_ssize_t size, const char *errors) + + Encode the :ctype:`Py_UNICODE` buffer of the given size using Raw-Unicode-Escape + and return a Python string object. Return *NULL* if an exception was raised by + the codec. + + +.. cfunction:: PyObject* PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode) + + Encode a Unicode object using Raw-Unicode-Escape and return the result as + Python string object. Error handling is "strict". Return *NULL* if an exception + was raised by the codec. + +These are the Latin-1 codec APIs: Latin-1 corresponds to the first 256 Unicode +ordinals and only these are accepted by the codecs during encoding. + +.. % --- Latin-1 Codecs ----------------------------------------------------- + + +.. cfunction:: PyObject* PyUnicode_DecodeLatin1(const char *s, Py_ssize_t size, const char *errors) + + Create a Unicode object by decoding *size* bytes of the Latin-1 encoded string + *s*. Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_EncodeLatin1(const Py_UNICODE *s, Py_ssize_t size, const char *errors) + + Encode the :ctype:`Py_UNICODE` buffer of the given size using Latin-1 and return + a Python string object. Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_AsLatin1String(PyObject *unicode) + + Encode a Unicode object using Latin-1 and return the result as Python string + object. Error handling is "strict". Return *NULL* if an exception was raised + by the codec. + +These are the ASCII codec APIs. Only 7-bit ASCII data is accepted. All other +codes generate errors. + +.. % --- ASCII Codecs ------------------------------------------------------- + + +.. cfunction:: PyObject* PyUnicode_DecodeASCII(const char *s, Py_ssize_t size, const char *errors) + + Create a Unicode object by decoding *size* bytes of the ASCII encoded string + *s*. Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_EncodeASCII(const Py_UNICODE *s, Py_ssize_t size, const char *errors) + + Encode the :ctype:`Py_UNICODE` buffer of the given size using ASCII and return a + Python string object. Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_AsASCIIString(PyObject *unicode) + + Encode a Unicode object using ASCII and return the result as Python string + object. Error handling is "strict". Return *NULL* if an exception was raised + by the codec. + +These are the mapping codec APIs: + +.. % --- Character Map Codecs ----------------------------------------------- + +This codec is special in that it can be used to implement many different codecs +(and this is in fact what was done to obtain most of the standard codecs +included in the :mod:`encodings` package). The codec uses mapping to encode and +decode characters. + +Decoding mappings must map single string characters to single Unicode +characters, integers (which are then interpreted as Unicode ordinals) or None +(meaning "undefined mapping" and causing an error). + +Encoding mappings must map single Unicode characters to single string +characters, integers (which are then interpreted as Latin-1 ordinals) or None +(meaning "undefined mapping" and causing an error). + +The mapping objects provided must only support the __getitem__ mapping +interface. + +If a character lookup fails with a LookupError, the character is copied as-is +meaning that its ordinal value will be interpreted as Unicode or Latin-1 ordinal +resp. Because of this, mappings only need to contain those mappings which map +characters to different code points. + + +.. cfunction:: PyObject* PyUnicode_DecodeCharmap(const char *s, Py_ssize_t size, PyObject *mapping, const char *errors) + + Create a Unicode object by decoding *size* bytes of the encoded string *s* using + the given *mapping* object. Return *NULL* if an exception was raised by the + codec. If *mapping* is *NULL* latin-1 decoding will be done. Else it can be a + dictionary mapping byte or a unicode string, which is treated as a lookup table. + Byte values greater that the length of the string and U+FFFE "characters" are + treated as "undefined mapping". + + .. versionchanged:: 2.4 + Allowed unicode string as mapping argument. + + +.. cfunction:: PyObject* PyUnicode_EncodeCharmap(const Py_UNICODE *s, Py_ssize_t size, PyObject *mapping, const char *errors) + + Encode the :ctype:`Py_UNICODE` buffer of the given size using the given + *mapping* object and return a Python string object. Return *NULL* if an + exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_AsCharmapString(PyObject *unicode, PyObject *mapping) + + Encode a Unicode object using the given *mapping* object and return the result + as Python string object. Error handling is "strict". Return *NULL* if an + exception was raised by the codec. + +The following codec API is special in that maps Unicode to Unicode. + + +.. cfunction:: PyObject* PyUnicode_TranslateCharmap(const Py_UNICODE *s, Py_ssize_t size, PyObject *table, const char *errors) + + Translate a :ctype:`Py_UNICODE` buffer of the given length by applying a + character mapping *table* to it and return the resulting Unicode object. Return + *NULL* when an exception was raised by the codec. + + The *mapping* table must map Unicode ordinal integers to Unicode ordinal + integers or None (causing deletion of the character). + + Mapping tables need only provide the :meth:`__getitem__` interface; dictionaries + and sequences work well. Unmapped character ordinals (ones which cause a + :exc:`LookupError`) are left untouched and are copied as-is. + +These are the MBCS codec APIs. They are currently only available on Windows and +use the Win32 MBCS converters to implement the conversions. Note that MBCS (or +DBCS) is a class of encodings, not just one. The target encoding is defined by +the user settings on the machine running the codec. + +.. % --- MBCS codecs for Windows -------------------------------------------- + + +.. cfunction:: PyObject* PyUnicode_DecodeMBCS(const char *s, Py_ssize_t size, const char *errors) + + Create a Unicode object by decoding *size* bytes of the MBCS encoded string *s*. + Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_DecodeMBCSStateful(const char *s, int size, const char *errors, int *consumed) + + If *consumed* is *NULL*, behave like :cfunc:`PyUnicode_DecodeMBCS`. If + *consumed* is not *NULL*, :cfunc:`PyUnicode_DecodeMBCSStateful` will not decode + trailing lead byte and the number of bytes that have been decoded will be stored + in *consumed*. + + .. versionadded:: 2.5 + + +.. cfunction:: PyObject* PyUnicode_EncodeMBCS(const Py_UNICODE *s, Py_ssize_t size, const char *errors) + + Encode the :ctype:`Py_UNICODE` buffer of the given size using MBCS and return a + Python string object. Return *NULL* if an exception was raised by the codec. + + +.. cfunction:: PyObject* PyUnicode_AsMBCSString(PyObject *unicode) + + Encode a Unicode object using MBCS and return the result as Python string + object. Error handling is "strict". Return *NULL* if an exception was raised + by the codec. + +.. % --- Methods & Slots ---------------------------------------------------- + + +.. _unicodemethodsandslots: + +Methods and Slot Functions +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The following APIs are capable of handling Unicode objects and strings on input +(we refer to them as strings in the descriptions) and return Unicode objects or +integers as appropriate. + +They all return *NULL* or ``-1`` if an exception occurs. + + +.. cfunction:: PyObject* PyUnicode_Concat(PyObject *left, PyObject *right) + + Concat two strings giving a new Unicode string. + + +.. cfunction:: PyObject* PyUnicode_Split(PyObject *s, PyObject *sep, Py_ssize_t maxsplit) + + Split a string giving a list of Unicode strings. If sep is *NULL*, splitting + will be done at all whitespace substrings. Otherwise, splits occur at the given + separator. At most *maxsplit* splits will be done. If negative, no limit is + set. Separators are not included in the resulting list. + + +.. cfunction:: PyObject* PyUnicode_Splitlines(PyObject *s, int keepend) + + Split a Unicode string at line breaks, returning a list of Unicode strings. + CRLF is considered to be one line break. If *keepend* is 0, the Line break + characters are not included in the resulting strings. + + +.. cfunction:: PyObject* PyUnicode_Translate(PyObject *str, PyObject *table, const char *errors) + + Translate a string by applying a character mapping table to it and return the + resulting Unicode object. + + The mapping table must map Unicode ordinal integers to Unicode ordinal integers + or None (causing deletion of the character). + + Mapping tables need only provide the :meth:`__getitem__` interface; dictionaries + and sequences work well. Unmapped character ordinals (ones which cause a + :exc:`LookupError`) are left untouched and are copied as-is. + + *errors* has the usual meaning for codecs. It may be *NULL* which indicates to + use the default error handling. + + +.. cfunction:: PyObject* PyUnicode_Join(PyObject *separator, PyObject *seq) + + Join a sequence of strings using the given separator and return the resulting + Unicode string. + + +.. cfunction:: int PyUnicode_Tailmatch(PyObject *str, PyObject *substr, Py_ssize_t start, Py_ssize_t end, int direction) + + Return 1 if *substr* matches *str*[*start*:*end*] at the given tail end + (*direction* == -1 means to do a prefix match, *direction* == 1 a suffix match), + 0 otherwise. Return ``-1`` if an error occurred. + + +.. cfunction:: Py_ssize_t PyUnicode_Find(PyObject *str, PyObject *substr, Py_ssize_t start, Py_ssize_t end, int direction) + + Return the first position of *substr* in *str*[*start*:*end*] using the given + *direction* (*direction* == 1 means to do a forward search, *direction* == -1 a + backward search). The return value is the index of the first match; a value of + ``-1`` indicates that no match was found, and ``-2`` indicates that an error + occurred and an exception has been set. + + +.. cfunction:: Py_ssize_t PyUnicode_Count(PyObject *str, PyObject *substr, Py_ssize_t start, Py_ssize_t end) + + Return the number of non-overlapping occurrences of *substr* in + ``str[start:end]``. Return ``-1`` if an error occurred. + + +.. cfunction:: PyObject* PyUnicode_Replace(PyObject *str, PyObject *substr, PyObject *replstr, Py_ssize_t maxcount) + + Replace at most *maxcount* occurrences of *substr* in *str* with *replstr* and + return the resulting Unicode object. *maxcount* == -1 means replace all + occurrences. + + +.. cfunction:: int PyUnicode_Compare(PyObject *left, PyObject *right) + + Compare two strings and return -1, 0, 1 for less than, equal, and greater than, + respectively. + + +.. cfunction:: int PyUnicode_RichCompare(PyObject *left, PyObject *right, int op) + + Rich compare two unicode strings and return one of the following: + + * ``NULL`` in case an exception was raised + * :const:`Py_True` or :const:`Py_False` for successful comparisons + * :const:`Py_NotImplemented` in case the type combination is unknown + + Note that :const:`Py_EQ` and :const:`Py_NE` comparisons can cause a + :exc:`UnicodeWarning` in case the conversion of the arguments to Unicode fails + with a :exc:`UnicodeDecodeError`. + + Possible values for *op* are :const:`Py_GT`, :const:`Py_GE`, :const:`Py_EQ`, + :const:`Py_NE`, :const:`Py_LT`, and :const:`Py_LE`. + + +.. cfunction:: PyObject* PyUnicode_Format(PyObject *format, PyObject *args) + + Return a new string object from *format* and *args*; this is analogous to + ``format % args``. The *args* argument must be a tuple. + + +.. cfunction:: int PyUnicode_Contains(PyObject *container, PyObject *element) + + Check whether *element* is contained in *container* and return true or false + accordingly. + + *element* has to coerce to a one element Unicode string. ``-1`` is returned if + there was an error.