<?xml version="1.0" encoding="utf-8"?>

<!-- Copyright (c) 2007-2010 Nokia Corporation and/or its subsidiary(-ies) All rights reserved. -->

<!-- This component and the accompanying materials are made available under the terms of the License 

"Eclipse Public License v1.0" which accompanies this distribution, 

and is available at the URL "http://www.eclipse.org/legal/epl-v10.html". -->

<!-- Initial Contributors:

    Nokia Corporation - initial contribution.

Contributors: 

-->

<!DOCTYPE concept

  PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">

<concept id="GUID-2E22CB5B-E648-5760-AB74-8947C7EDE2CF" xml:lang="en"><title>Lexical

Analysis Overview</title><shortdesc>Provides string analysis and string-to-number conversions</shortdesc><prolog><metadata><keywords/></metadata></prolog><conbody>

<section id="GUID-A64D2A5B-2A8C-4FEF-9F4A-6AC4F997BB7F"><title>Description</title> <p>The API has two key concepts: lexical

analyser, and extraction mark. </p> </section>

<section id="GUID-12BA42C0-484F-46FF-A9EB-8F8968B2F272"><title>Lexical analyser</title><p>The lexical analyser provides facilities

to identify and extract characters and tokens (substrings delimited by white

space) from a descriptor or NULL-terminated string. It also provides string-to-number

conversions.</p> <p>The lexical analyser interface is provided by <xref href="GUID-76BFE569-34F7-3FEC-8F01-38AEC8C29D68.dita"><apiname>TLex16</apiname></xref> for

wide strings, and <xref href="GUID-5B7955BC-1375-35F1-953D-D7F5C3583756.dita"><apiname>TLex8</apiname></xref> for narrow strings. <xref href="GUID-C94AB4F0-AE42-3957-A037-77CB145DDBF8.dita"><apiname>TLex</apiname></xref> is

a typedef for <xref href="GUID-76BFE569-34F7-3FEC-8F01-38AEC8C29D68.dita"><apiname>TLex16</apiname></xref>.</p></section>

<section id="GUID-A1CF4E86-0B08-4D16-A352-2240983CD651"><title>Extraction mark</title><p>The extraction mark allows positions

in a string that is being analysed to be remembered. Later lexical analysis

operations can then operate on this position in the string.</p> <p>The extraction

mark interface is provided by <xref href="GUID-284AF88B-910B-3483-8797-E85ECB08BB2A.dita"><apiname>TLexMark16</apiname></xref> for wide strings,

and <xref href="GUID-11EFB436-59E4-3E21-8950-B7F3F16FDD92.dita"><apiname>TLexMark8</apiname></xref> for narrow strings. <xref href="GUID-CCFA0F79-F758-3728-AA4E-DDF59D3817C1.dita"><apiname>TLexMark</apiname></xref> is

a typedef for <xref href="GUID-284AF88B-910B-3483-8797-E85ECB08BB2A.dita"><apiname>TLexMark16</apiname></xref>.</p></section>

</conbody><related-links>

<link>

<desc><xref href="GUID-0817AD1D-58CF-5108-ACBF-26DFD4BA395E.dita">Descriptors Overview</xref></desc>

</link>

</related-links></concept>