<?xml version="1.0" encoding="utf-8"?>
<!-- Copyright (c) 2007-2010 Nokia Corporation and/or its subsidiary(-ies) All rights reserved. -->
<!-- This component and the accompanying materials are made available under the terms of the License 
"Eclipse Public License v1.0" which accompanies this distribution, 
and is available at the URL "http://www.eclipse.org/legal/epl-v10.html". -->
<!-- Initial Contributors:
    Nokia Corporation - initial contribution.
Contributors: 
-->
<!DOCTYPE concept
  PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">
<concept id="GUID-2E22CB5B-E648-5760-AB74-8947C7EDE2CF" xml:lang="en"><title>Lexical
Analysis Overview</title><shortdesc>Provides string analysis and string-to-number conversions</shortdesc><prolog><metadata><keywords/></metadata></prolog><conbody>
<section id="GUID-A64D2A5B-2A8C-4FEF-9F4A-6AC4F997BB7F"><title>Description</title> <p>The API has two key concepts: lexical
analyser, and extraction mark. </p> </section>
<section id="GUID-12BA42C0-484F-46FF-A9EB-8F8968B2F272"><title>Lexical analyser</title><p>The lexical analyser provides facilities
to identify and extract characters and tokens (substrings delimited by white
space) from a descriptor or NULL-terminated string. It also provides string-to-number
conversions.</p> <p>The lexical analyser interface is provided by <xref href="GUID-76BFE569-34F7-3FEC-8F01-38AEC8C29D68.dita"><apiname>TLex16</apiname></xref> for
wide strings, and <xref href="GUID-5B7955BC-1375-35F1-953D-D7F5C3583756.dita"><apiname>TLex8</apiname></xref> for narrow strings. <xref href="GUID-C94AB4F0-AE42-3957-A037-77CB145DDBF8.dita"><apiname>TLex</apiname></xref> is
a typedef for <xref href="GUID-76BFE569-34F7-3FEC-8F01-38AEC8C29D68.dita"><apiname>TLex16</apiname></xref>.</p></section>
<section id="GUID-A1CF4E86-0B08-4D16-A352-2240983CD651"><title>Extraction mark</title><p>The extraction mark allows positions
in a string that is being analysed to be remembered. Later lexical analysis
operations can then operate on this position in the string.</p> <p>The extraction
mark interface is provided by <xref href="GUID-284AF88B-910B-3483-8797-E85ECB08BB2A.dita"><apiname>TLexMark16</apiname></xref> for wide strings,
and <xref href="GUID-11EFB436-59E4-3E21-8950-B7F3F16FDD92.dita"><apiname>TLexMark8</apiname></xref> for narrow strings. <xref href="GUID-CCFA0F79-F758-3728-AA4E-DDF59D3817C1.dita"><apiname>TLexMark</apiname></xref> is
a typedef for <xref href="GUID-284AF88B-910B-3483-8797-E85ECB08BB2A.dita"><apiname>TLexMark16</apiname></xref>.</p></section>
</conbody><related-links>
<link>
<desc><xref href="GUID-0817AD1D-58CF-5108-ACBF-26DFD4BA395E.dita">Descriptors Overview</xref></desc>
</link>
</related-links></concept>