1 <?xml version="1.0" encoding="utf-8"?>

     2 <!-- Copyright (c) 2007-2010 Nokia Corporation and/or its subsidiary(-ies) All rights reserved. -->

     3 <!-- This component and the accompanying materials are made available under the terms of the License 

     4 "Eclipse Public License v1.0" which accompanies this distribution, 

     5 and is available at the URL "http://www.eclipse.org/legal/epl-v10.html". -->

     6 <!-- Initial Contributors:

     7     Nokia Corporation - initial contribution.

     8 Contributors: 

     9 -->

    10 <!DOCTYPE concept

    11   PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">

    12 <concept id="GUID-FE888721-C479-5D8A-810F-1A79575A1D58" xml:lang="en"><title>Introduction

    13 to Lexical Analysis</title><shortdesc>A <codeph>TLex</codeph> object is used to perform lexical analysis. </shortdesc><prolog><metadata><keywords/></metadata></prolog><conbody>

    14 <p>The class is used to:</p>

    15 <ul>

    16 <li id="GUID-4D79DE44-4E35-56C3-B6ED-5A82CD015FE9"><p>store a string</p> </li>

    17 <li id="GUID-537012CB-CFCC-55C3-82DE-1D9B18539BEC"><p>maintain an extraction

    18 mark to indicate the current lexical element being analysed</p> </li>

    19 <li id="GUID-7B398A9C-42F1-5ABE-80ED-4237F72806BD"><p>maintain a pointer to

    20 indicate the next character to be examined.</p> </li>

    21 </ul>

    22 <p>Member functions provide facilities to allow analysis and extraction of

    23 lexical elements from a <codeph>TLex</codeph> object, and a variety of string-to-number

    24 conversions.</p>

    25 <p>The <codeph>TLex</codeph> class provides general-purpose lexical analysis

    26 functions, using the locale-dependent functions of <codeph>TChar</codeph> to

    27 identify whether characters are digits, letters etc. The data for these locale-dependent

    28 functions is held in kernel-side locale tables. Thus, <codeph>TLex</codeph> functions

    29 which use them, require an executive call. If an application requires very

    30 fast lexical analysis of a language which is not locale-specific, then there

    31 may be some advantage in designing and implementing a special-purpose lexical

    32 analyzer class.</p>

    33 </conbody></concept>