/****************************************************************************

**

** Copyright (C) 2009 Nokia Corporation and/or its subsidiary(-ies).

** All rights reserved.

** Contact: Nokia Corporation (qt-info@nokia.com)

**

** This file is part of the documentation of the Qt Toolkit.

**

** $QT_BEGIN_LICENSE:LGPL$

** No Commercial Usage

** This file contains pre-release code and may not be distributed.

** You may use this file in accordance with the terms and conditions

** contained in the Technology Preview License Agreement accompanying

** this package.

**

** GNU Lesser General Public License Usage

** Alternatively, this file may be used under the terms of the GNU Lesser

** General Public License version 2.1 as published by the Free Software

** Foundation and appearing in the file LICENSE.LGPL included in the

** packaging of this file.  Please review the following information to

** ensure the GNU Lesser General Public License version 2.1 requirements

** will be met: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html.

**

** In addition, as a special exception, Nokia gives you certain additional

** rights.  These rights are described in the Nokia Qt LGPL Exception

** version 1.1, included in the file LGPL_EXCEPTION.txt in this package.

**

** If you have questions regarding the use of this file, please contact

** Nokia at qt-info@nokia.com.

**

**

**

**

**

**

**

**

** $QT_END_LICENSE$

**

****************************************************************************/

/*!

    \page xmlprocessing.html

    \title Using XML Technologies

    \previouspage Working with the DOM Tree

    \contentspage XML Processing

    \keyword Patternist

    \brief An overview of Qt's support for using XML technologies in

    Qt programs.

    \tableofcontents

    \section1 Introduction

    XQuery is a language for traversing XML documents to select and

    aggregate items of interest and to transform them for output as

    XML or some other format. XPath is the \e{element selection} part

    of XQuery.

    The QtXmlPatterns module supports using

    \l{http://www.w3.org/TR/xquery} {XQuery 1.0} and

    \l{http://www.w3.org/TR/xpath20} {XPath 2.0} in Qt applications,

    for querying XML data \e{and} for querying

    \l{QAbstractXmlNodeModel} {non-XML data that can be modeled to

    look like XML}. The QtXmlPatterns module is included in the \l{Qt

    Full Framework Edition}, and the \l{Open Source Versions of Qt}.

    Readers who are not familiar with the XQuery/XPath language can read

    \l {A Short Path to XQuery} for a brief introduction.

    \section1 Advantages of using QtXmlPatterns and XQuery

    The XQuery/XPath language simplifies data searching and

    transformation tasks by eliminating the need for doing a lot of

    C++ or Java procedural programming for each new query task. Here

    is an XQuery that constructs a bibliography of the contents of a

    library:

    \target qtxmlpatterns_example_query

    \quotefile snippets/patternist/introductionExample.xq

    First, the query opens a \c{<bibliography>} element in the

    output. The

    \l{xquery-introduction.html#using-path-expressions-to-match-select-items}

    {embedded path expression} then loads the XML document describing

    the contents of the library (\c{library.xml}) and begins the

    search. For each \c{<book>} element it finds, where the publisher

    was Addison-Wesley and the publication year was after 1991, it

    creates a new \c{<book>} element in the output as a child of the

    open \c{<bibliography>} element. Each new \c{<book>} element gets

    the book's title as its contents and the book's publication year

    as an attribute. Finally, the \c{<bibliography>} element is

    closed.

    The advantages of using QtXmlPatterns and XQuery in your Qt

    programs are summarized as follows:

    \list

    \o \bold{Ease of development}: All the C++ programming required to

       perform data query tasks can be replaced by a simple XQuery

       like the example above.

    \o \bold{Comprehensive functionality}: The

       \l{http://www.w3.org/TR/xquery/#id-expressions} {expression

       syntax} and rich set of

       \l{http://www.w3.org/TR/xpath-functions} {functions and

       operators} provided by XQuery are sufficient for performing any

       data searching, selecting, and sorting tasks.

    \o \bold{Conformance to standards}: Conformance to all applicable

       XML and XQuery standards ensures that QtXmlPatterns can always

       process XML documents generated by other conformant

       applications, and that XML documents created with QtXmlPatterns

       can be processed by other conformant applications.

    \o \bold{Maximal flexibility} The QtXmlPatterns module can be used

       to query XML data \e{and} non-XML data that can be

       \l{QAbstractXmlNodeModel} {modeled to look like XML}.

    \endlist

    \section1 Using the QtXmlPatterns module

    There are two ways QtXmlPatterns can be used to evaluate queries.

    You can run the query engine in your Qt application using the

    QtXmlPatterns C++ API, or you can run the query engine from the

    command line using Qt's \c{xmlpatterns} command line utility.

    \section2 Running the query engine from your Qt application

    If we save the example XQuery shown above in a text file (e.g.

    \c{myquery.xq}), we can run it from a Qt application using a

    standard QtXmlPatterns code sequence:

    \snippet doc/src/snippets/code/src_xmlpatterns_api_qxmlquery.cpp 3

    First construct a QFile for the text file containing the XQuery

    (\c{myquery.xq}). Then create an instance of QXmlQuery and call

    its \l{QXmlQuery::}{setQuery()} function to load and parse the

    XQuery file. Then create an \l{QXmlSerializer} {XML serializer} to

    output the query's result set as unformatted XML. Finally, call

    the \l{QXmlQuery::}{evaluateTo()} function to evaluate the query

    and serialize the results as XML.

    \note If you compile Qt yourself, the QtXmlPatterns module will

    \e{not} be built if exceptions are disabled, or if you compile Qt

    with a compiler that doesn't support member templates, e.g., MSVC

    6.

    See the QXmlQuery documentation for more information about the

    QtXmlPatterns C++ API.

    \section2 Running the query engine from the command line utility

    \e xmlpatterns is a command line utility for running XQueries.  It

    expects the name of a file containing the XQuery text.

    \snippet doc/src/snippets/code/doc_src_qtxmlpatterns.qdoc 2

    The XQuery in \c{myQuery.xq} will be evaluated and its output

    written to \c stdout. Pass the \c -help switch to get the list of

    input flags and their meanings.

    xmlpatterns can be used in scripting. However, the descriptions

    and messages it outputs were not meant to be parsed and may be

    changed in future releases of Qt.

    \target QtXDM

    \section1 The XQuery Data Model

    XQuery represents data items as \e{atomic values} or \e{nodes}. An

    atomic value is a value in the domain of one of the

    \l{http://www.w3.org/TR/xmlschema-2/#built-in-datatypes} {built-in

    datatypes} defined in \l{http://www.w3.org/TR/xmlschema-2} {Part

    2} of the W3C XML Schema. A node is normally an XML element or

    attribute, but when non-XML data is \l{QAbstractXmlNodeModel}

    {modeled to look like XML}, a node can also represent a non-XML

    data items.

    When you run an XQuery using the C++ API in a Qt application, you

    will often want to bind program variables to $variables in the

    XQuery. After the query is evaluated, you will want to interpret

    the sequence of data items in the result set.

    \section2 Binding program variables to XQuery variables

    When you want to run a parameterized XQuery from your Qt

    application, you will need to \l{QXmlQuery::bindVariable()} {bind

    variables} in your program to $name variables in your XQuery.

    Suppose you want to parameterize the bibliography XQuery in the

    example above. You could define variables for the catalog that

    contains the library (\c{$file}), the publisher name

    (\c{$publisher}), and the year of publication (\c{$year}):

    \target qtxmlpatterns_example_query2

    \quotefile snippets/patternist/introExample2.xq

    Modify the QtXmlPatterns code to use one of the \l{QXmlQuery::}

    {bindVariable()} functions to bind a program variable to each

    XQuery $variable:

    \snippet doc/src/snippets/code/src_xmlpatterns_api_qxmlquery.cpp 4

    Each program variable is passed to QtXmlPatterns as a QVariant of

    the type of the C++ variable or constant from which it is

    constructed. Note that QtXmlPatterns assumes that the type of the

    QVariant in the bindVariable() call is the correct type, so the

    $variable it is bound to must be used in the XQuery accordingly.

    The following table shows how QVariant types are mapped to XQuery

    $variable types:

    \table

    \header

        \o QVariant type

        \o XQuery $variable type

    \row

        \o QVariant::LongLong

        \o \c xs:integer

    \row

        \o QVariant::Int

        \o \c xs:integer

    \row

        \o QVariant::UInt

        \o \c xs:nonNegativeInteger

    \row

        \o QVariant::ULongLong

        \o \c xs:unsignedLong

    \row

        \o QVariant::String

        \o \c xs:string

    \row

        \o QVariant::Double

        \o \c xs:double

    \row

        \o QVariant::Bool

        \o \c xs:boolean

    \row

        \o QVariant::Double

        \o \c xs:decimal

    \row

        \o QVariant::ByteArray

        \o \c xs:base64Binary

    \row

        \o QVariant::StringList

        \o \c xs:string*

    \row

        \o QVariant::Url

        \o \c xs:string

    \row

        \o QVariant::Date

        \o \c xs:date.

    \row

        \o QVariant::DateTime

        \o \c xs:dateTime

    \row

        \o QVariant::Time.

        \o \c xs:time. (see \l{Binding To Time}{Binding To

        QVariant::Time} below)

    \row

        \o QVariantList

        \o (see \l{Binding To QVariantList}{Binding To QVariantList}

           below)

    \endtable

    A type not shown in the table is not supported and will cause

    undefined XQuery behavior or a $variable binding error, depending

    on the context in the XQuery where the variable is used.

    \target Binding To Time

    \section3 Binding To QVariant::Time

    Because the instance of QTime used in QVariant::Time does not

    include a zone offset, an instance of QVariant::Time should not be

    bound to an XQuery variable of type \c xs:time, unless the QTime is

    UTC. When binding a non-UTC QTime to an XQuery variable, it should

    first be passed as a string, or converted to a QDateTime with an arbitrary

    date, and then bound to an XQuery variable of type \c xs:dateTime.

    \target Binding To QVariantList

    \section3 Binding To QVariantList

    A QVariantList can be bound to an XQuery $variable. All the

    \l{QVariant}s in the list must be of the same atomic type, and the

    $variable the variant list is bound to must be of that same atomic

    type. If the QVariants in the list are not all of the same atomic

    type, the XQuery behavior is undefined.

    \section2 Interpreting XQuery results

    When the results of an XQuery are returned in a sequence of \l

    {QXmlResultItems} {result items}, atomic values in the sequence

    are treated as instances of QVariant. Suppose that instead of

    serializing the results of the XQuery as XML, we process the

    results programatically. Modify the standard QtXmlPatterns code

    sequence to call the overload of QXmlQuery::evaluateTo() that

    populates a sequence of \l {QXmlResultItems} {result items} with

    the XQuery results:

    \snippet doc/src/snippets/code/src_xmlpatterns_api_qxmlquery.cpp 5

    Iterate through the \l {QXmlResultItems} {result items} and test

    each QXmlItem to see if it is an atomic value or a node. If it is

    an atomic value, convert it to a QVariant with \l {QXmlItem::}

    {toAtomicValue()} and switch on its \l {QVariant::type()} {variant

    type} to handle all the atomic values your XQuery might return.

    The following table shows the QVariant type to expect for each

    atomic value type (or QXmlName):

    \table

    \header

        \o XQuery result item type

        \o QVariant type returned

    \row

        \o \c xs:QName

        \o QXmlName (see \l{Handling QXmlNames}{Handling QXmlNames}

        below)

    \row

        \o \c xs:integer

        \o QVariant::LongLong

    \row

        \o \c xs:string

        \o QVariant::String

    \row

        \o \c xs:string*

        \o QVariant::StringList

    \row

        \o \c xs:double

        \o QVariant::Double

    \row

        \o \c xs:float

        \o QVariant::Double

    \row

        \o \c xs:boolean

        \o QVariant::Bool

    \row

        \o \c xs:decimal

        \o QVariant::Double

    \row

        \o \c xs:hexBinary

        \o QVariant::ByteArray

    \row

        \o \c xs:base64Binary

        \o QVariant::ByteArray

    \row

        \o \c xs:gYear

        \o QVariant::DateTime

    \row

        \o \c xs:gYearMonth

        \o QVariant::DateTime

    \row

        \o \c xs:gMonthDay

        \o QVariant::DateTime

    \row

        \o \c xs:gDay

        \o QVariant::DateTime

    \row

        \o \c xs:gMonth

        \o QVariant::DateTime

    \row

        \o \c xs:anyURI

        \o QVariant::Url

    \row

        \o \c xs:untypedAtomic

        \o QVariant::String

    \row

        \o \c xs:ENTITY

        \o QVariant::String

    \row

        \o \c xs:date

        \o QVariant::DateTime

    \row

        \o \c xs:dateTime

        \o QVariant::DateTime

    \row

        \o \c xs:time

        \o (see \l{xstime-not-mapped}{No mapping for xs:time} below)

    \endtable

    \target Handling QXmlNames

    \section3 Handling QXmlNames

    If your XQuery can return atomic value items of type \c{xs:QName},

    they will appear in your QXmlResultItems as instances of QXmlName.

    Since the QVariant class does not support the QXmlName class

    directly, extracting them from QXmlResultItems requires a bit of

    slight-of-hand using the \l{QMetaType} {Qt metatype system}. We

    must modify our example to use a couple of template functions, a

    friend of QMetaType (qMetaTypeId<T>()) and a friend of QVariant

    (qVariantValue<T>()):

    \snippet doc/src/snippets/code/src_xmlpatterns_api_qxmlquery.cpp 6

    To access the strings in a QXmlName returned by an

    \l{QXmlQuery::evaluateTo()} {XQuery evaluation}, the QXmlName must

    be accessed with the \l{QXmlNamePool} {name pool} from the

    instance of QXmlQuery that was used for the evaluation.

    \target xstime-not-mapped

    \section3 No mapping for xs:time

    An instance of \c xs:time can't be represented correctly as an

    instance of QVariant::Time, unless the \c xs:time is a UTC time.

    This is because xs:time has a zone offset (0 for UTC) in addition

    to the time value, which the QTime in QVariant::Time does not

    have. This means that if an XQuery tries to return an atomic value

    of type \c xs:time, an invalid QVariant will be returned. A query

    can return an atomic value of type xs:time by either converting it

    to an \c xs:dateTime with an arbitrary date, or to an \c xs:string.

    \section1 Using XQuery with Non-XML Data

    Although the XQuery language was designed for querying XML, with

    QtXmlPatterns one can use XQuery for querying any data that can

    be modeled to look like XML. Non-XML data is modeled to look like

    XML by loading it into a custom subclass of QAbstractXmlNodeModel,

    where it is then presented to the QtXmlPatterns XQuery engine via

    the same API the XQuery engine uses for querying XML.

    When QtXmlPatterns loads and queries XML files and produces XML

    output, it can always load the XML data into its default XML node

    model, where it can be traversed efficiently. The XQuery below

    traverses the product orders found in the XML file \e myOrders.xml

    to find all the skin care product orders and output them ordered

    by shipping date.

    \quotefile snippets/patternist/introAcneRemover.xq

    QtXmlPatterns can be used out of the box to perform this

    query, provided \e myOrders.xml actually contains well-formed XML. It

    can be loaded directly into the default XML node model and

    traversed. But suppose we want QtXmlPatterns to perform queries on

    the hierarchical structure of the local file system. The default

    XML node model in QtXmlPatterns is not suitable for navigating the

    file system, because there is no XML file to load that contains a

    description of it. Such an XML file, if it existed, might look

    something like this:

    \quotefile snippets/patternist/introFileHierarchy.xml

    The \l{File System Example}{File System Example} does exactly this.

    There is no such file to load into the default XML node model, but

    one can write a subclass of QAbstractXmlNodeModel to represent the

    file system. This custom XML node model, once populated with all

    the directory and file descriptors obtained directly from the

    system, presents the complete file system hierarchy to the query

    engine via the same API used by the default XML node model to

    present the contents of an XML file. In other words, once the

    custom XML node model is populated, it presents the file system to

    the query engine as if a description of it had been loaded into

    the default XML node model from an XML file like the one shown

    above.

    Now we can write an XQuery to find all the XML files and parse

    them to find the ones that don't contain well-formed XML.

    \quotefromfile snippets/patternist/introNavigateFS.xq

    \skipto <html>

    \printuntil

    Without QtXmlPatterns, there is no simple way to solve this kind

    of problem. You might do it by writing a C++ program to traverse

    the file system, sniff out all the XML files, and submit each one

    to an XML parser to test that it contains valid XML. The C++ code

    required to write that program will probably be more complex than

    the C++ code required to subclass QAbstractXmlNodeModel, but even

    if the two are comparable, your custom C++ program can be used

    only for that one task, while your custom XML node model can be

    used by any XQuery that must navigate the file system.

    The general approach to using XQuery to perform queries on non-XML

    data has been a three step process. In the first step, the data is

    loaded into a non-XML data model. In the second step, the non-XML

    data model is serialized as XML and output to XML (text) files. In

    the final step, an XML tool loads the XML files into a second, XML

    data model, where the XQueries can be performed.  The development

    cost of implementing this process is often high, and the three

    step system that results is inefficient because the two data

    models must be built and maintained separately.

    With QtXmlPatterns, subclassing QAbstractXmlNodeModel eliminates

    the transformation required to convert the non-XML data model to

    the XML data model, because there is only ever one data model

    required. The non-XML data model presents the non-XML data to the

    query engine via the XML data model API. Also, since the query