<?xml version="1.0" encoding="utf-8"?>

<!-- Copyright (c) 2007-2010 Nokia Corporation and/or its subsidiary(-ies) All rights reserved. -->

<!-- This component and the accompanying materials are made available under the terms of the License 

"Eclipse Public License v1.0" which accompanies this distribution, 

and is available at the URL "http://www.eclipse.org/legal/epl-v10.html". -->

<!-- Initial Contributors:

    Nokia Corporation - initial contribution.

Contributors: 

-->

<!DOCTYPE concept

  PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">

<concept id="GUID-FE94596E-B5BB-51FE-BE38-069840323915" xml:lang="en"><title>Encoding

Types</title><prolog><metadata><keywords/></metadata></prolog><conbody>

<p>This topic describes the types of SMS encoding. </p>

<section id="GUID-F7D1E6C8-9605-57FA-9788-AF7FC72BD94C"><title>7-bit GSM encoding</title> <p>7-bit

GSM encoding supports the GSM 7-bit default alphabet and GSM 7-bit default

alphabet extension table through an escape mechanism. </p> <p>Figure 1 </p> <fig id="GUID-CDEE59FC-F035-5B75-8838-96E94A6714E8">

<title>              Escape mechanism            </title>

<image href="GUID-08A6B93F-92CD-5182-B142-D353E78016F3_d0e406599_href.png" placement="inline"/>

</fig> <p>The GSM 7-bit default alphabet consists of 128 characters. Each

character is represented by 7 bits. 10 extra characters are defined in the

GSM 7-bit default extension table. These characters are represented by an

escape mechanism using the escape character (0x1B). For example, 0x1B65 maps

to the Euro sign € (U+20AC). If an escape character byte is followed by a

character that is not included in the 10 characters, the escape character

is just ignored. This means 0x1B41 maps to Latin capital letter A (U+0041). </p> <p>For

more information about the GSM 7-bit default table, extension table and escape

mechanism, see 3GPP TS 23.038 V8.1.0. </p> </section>

<section id="GUID-918FF2E3-B9F4-5C61-8DBA-F9143DB16460"><title>Lossy 7-bit

encoding</title> <p>Lossy 7-bit encoding enlarges the character set supported

by 7-bit GSM encoding. Some Unicode Characters do not exist in the target

7-bit set. These characters are converted to ones that do exist in the target

7-bit set and closely resemble the original, intended character. A lossy encoding

using a 7-bit encoding is more cost effective than a UCS-2 encoding. </p> <p> <b>Example

of 7-bit encoding</b>  </p> <p>Accented Latin characters are not supported

by 7-bit GSM encoding. Figure 2 describes how an accented Latin characters

Á, is sent by SMS. Á has a Unicode value of 0x00C1. When it is processed by

the Lossy converter the character is converted from the Unicode to 7-bit code

letter A. A has a 7-bit code of 0x41. The SMS receiver reads A instead of

Á. By substituting the character that is similar enough to the original, the

reader can understand the word. The process of converting Á to A is called

a lossy conversion. </p> <p> <b>Note</b>: The 7-bit code of A (0x41) can only

be decoded back to the same Unicode letter A instead of Á. </p> <p>Figure

2 </p> <fig id="GUID-ACFF9511-D5E0-5558-8008-4CD48EE0B7A1">

<title>              Lossy conversion            </title>

<image href="GUID-8862E271-ABA4-5A25-8990-C0B3931E370D_d0e406639_href.png" placement="inline"/>

</fig> </section>

<section id="GUID-D2F0E6BE-932E-545D-A0C8-39017E3D67B4"><title>16-bit Unicode

encoding</title> <p>Unicode is an international standard character set. It

includes the characters of every language. In Unicode, each character is usually

encoded in two 8-bit bytes, and takes up more space than 7-bit encoding. </p> </section>

<section id="GUID-93B3DDF2-8EB1-5853-9DFD-3ABF42ADCB40"><title>National language

encoding</title> <p>According to 3GPP TS 23.038 V8.1.0, National Language

Encoding supports additional characters for certain languages which cannot

be represented in the GSM default 7-bit alphabet. It defines two mechanisms

for doing this: </p> <ul>

<li id="GUID-9ECCA8BD-0BA0-5AE2-B2D6-4677D2CD1BD7"><p>Locking shift mechanism–the

default GSM table is replaced with a table containing the character set needed

for a language. The table is referred to as locking shift table. </p> </li>

<li id="GUID-3900D849-350A-5722-9759-D1D768FE6A84"><p>Single shift mechanism–the

GSM extension table is replaced with a table containing the character set

needed for a language. The table is referred to as single shift table. </p> </li>

</ul> <p>When the locking shift mechanism is used, the escape table can be

the existing GSM extension table or it can be the escape table used by the

single shift mechanism. This supports three possible mappings as shown in

Figure 3: </p> <ul>

<li id="GUID-34ECF450-6265-58E2-9CB6-00E0C5DDA6F8"><p>The GSM 7-bit default

escapes to language-specific escape table. It is referred to as GSM-single. </p> </li>

<li id="GUID-6E8A53BF-0572-5DE2-8D41-FB588B6FB812"><p>The Language-specific

basic table escapes to GSM 7-bit default extension table. It is referred to

as locking-GSM ext. </p> </li>

<li id="GUID-830569B1-8ACD-5924-AF7F-15705FEF76B0"><p>The Language-specific

basic table escapes to language-specific extension table. It is referred to

as locking-single. </p> </li>

</ul> <p>Figure 3 </p> <fig id="GUID-541CED9A-2450-5C9D-AADF-93EE59E4D77E">

<title>              National language encoding            </title>

<image href="GUID-44347376-702D-5648-8938-EB55AFA329EC_d0e406701_href.png" placement="inline"/>

</fig><p>The single shift mechanism is useful when a message contains only

a few characters outside the default GSM table. It is however inefficient

when a message contains many unsupported characters, because each escaped

character must occupy 2 bytes. GSM-single supports more characters than locking-GSM

ext, but these characters are in the single table, which takes 2 bytes. Locking-single

is used more for the decoding purpose in case the extra characters can come

from the locking or single table. </p><p>The locking or single table is not

a complete replacement. For example, the locking table for Turkish redefines

only 8-character codes from the default GSM table, as shown in table 1. The

escape table for Turkish adds 7 characters to the GSM extension, as shown

in table 2. </p><table id="GUID-4AE6F58D-A5DA-4AD9-B39E-A61AA378F3F6"><title>Table 1</title>

<tgroup cols="3"><colspec colname="col1"/><colspec colname="col2"/><colspec colname="col3"/>

<thead>

<row>

<entry><p>GSM 7-Bit Code</p></entry>

<entry><p>Turkish Locking Shift Table</p></entry>

<entry><p>GSM 7-Bit Default Table</p></entry>

</row>

</thead>

<tbody>

<row>

<entry><p><codeph>0x40</codeph></p></entry>

<entry><p>I LATIN CAPITAL LETTER I WITH DOT ABOVE</p></entry>

<entry><p>¡ INVERTED EXCLAMATION MARK </p></entry>

</row>

<row>

<entry><p><codeph>0x60</codeph></p></entry>

<entry><p>ç LATIN SMALL LETTER C WITH CEDILLA</p></entry>

<entry><p>¿ INVERTED QUESTION MARK</p></entry>

</row>

<row>

<entry><p><codeph>0x04</codeph></p></entry>

<entry><p>€ EURO SIGN</p></entry>

<entry><p>è LATIN SMALL LETTER E WITH GRAVE</p></entry>

</row>

<row>

<entry><p><codeph>0x07</codeph></p></entry>

<entry><p>i LATIN SMALL LETTER DOTLESS</p></entry>

<entry><p>ì LATIN SMALL LETTER I WITH GRAVE</p></entry>

</row>

<row>

<entry><p><codeph>0x0B</codeph></p></entry>

<entry><p>G LATIN CAPITAL LETTER G WITH BREVE</p></entry>

<entry><p>Ø LATIN CAPITAL LETTER O WITH STROKE</p></entry>

</row>

<row>

<entry><p><codeph>0x0C</codeph></p></entry>

<entry><p>g LATIN SMALL LETTER G WITH BREVE</p></entry>

<entry><p>ø LATIN SMALL LETTER O WITH STROKE</p></entry>

</row>

<row>

<entry><p><codeph>0x1C</codeph></p></entry>

<entry><p>S LATIN CAPITAL LETTER S WITH CEDILLA *</p></entry>

<entry><p>Æ LATIN CAPITAL LETTER AE</p></entry>

</row>

<row>

<entry><p><codeph>0x1D</codeph></p></entry>

<entry><p>s LATIN SMALL LETTER S WITH CEDILLA *</p></entry>

<entry><p>æ LATIN SMALL LETTER AE</p></entry>

</row>

</tbody>

</tgroup>

</table> <table id="GUID-EC345039-0CB5-4F51-8CFA-83286790AC75"><title>Table 2</title>

<tgroup cols="3"><colspec colname="col1"/><colspec colname="col2"/><colspec colname="col3"/>

<thead>

<row>

<entry><p>GSM 7-Bit Code</p></entry>

<entry><p>Turkish Single Shift Table</p></entry>

<entry><p>GSM 7-Bit Extension Table</p></entry>

</row>

</thead>

<tbody>

<row>

<entry><p><codeph>0x1B49</codeph></p></entry>

<entry><p>I LATIN CAPITAL LETTER I WITH DOT ABOVE</p></entry>

<entry><p/></entry>

</row>

<row>

<entry><p><codeph>0x1B63</codeph></p></entry>

<entry><p>ç LATIN SMALL LETTER C WITH CEDILLA</p></entry>

<entry><p/></entry>

</row>

<row>

<entry><p><codeph>0x1B69</codeph></p></entry>

<entry><p>i LATIN SMALL LETTER DOTLESS</p></entry>

<entry><p/></entry>

</row>

<row>

<entry><p><codeph>0x1B47</codeph></p></entry>

<entry><p>G LATIN CAPITAL LETTER G WITH BREVE</p></entry>

<entry><p/></entry>

</row>

<row>

<entry><p><codeph>0x1B67</codeph></p></entry>

<entry><p>g LATIN SMALL LETTER G WITH BREVE</p></entry>

<entry><p/></entry>

</row>

<row>

<entry><p><codeph>0x1B53</codeph></p></entry>

<entry><p>S LATIN CAPITAL LETTER S WITH CEDILLA *</p></entry>

<entry><p/></entry>

</row>

<row>

<entry><p><codeph>0x1B73</codeph></p></entry>

<entry><p>s LATIN SMALL LETTER S WITH CEDILLA *</p></entry>

<entry><p/></entry>

</row>

</tbody>

</tgroup>

</table><p>For more information about the National Language Identifier, Single

or Locking mechanism, see 3GPP TS 23.038 V8.1.0: National Language Identifier.</p></section>

<section><title>See also</title> <p> <xref href="GUID-0BC9A9A1-DB99-5095-8390-E1C1B04D0080.dita">SMS

Encodings and Converters Overview</xref> </p> </section>