<?xml version="1.0" encoding="utf-8"?>

<!-- Copyright (c) 2007-2010 Nokia Corporation and/or its subsidiary(-ies) All rights reserved. -->

<!-- This component and the accompanying materials are made available under the terms of the License 

"Eclipse Public License v1.0" which accompanies this distribution, 

and is available at the URL "http://www.eclipse.org/legal/epl-v10.html". -->

<!-- Initial Contributors:

    Nokia Corporation - initial contribution.

Contributors: 

-->

<!DOCTYPE concept

  PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">

<concept id="GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3" xml:lang="en"><title>Migration

Tutorial: Demand Paging and Media Drivers</title><shortdesc>Describes how to change media drivers when demand paging is used.</shortdesc><prolog><metadata><keywords/></metadata></prolog><conbody>

<p>Demand paging is a change made from Symbian platform v9.3 to how the Kernel

uses RAM and storage media. This topic </p>

<section id="GUID-AD70C4EE-F7E8-473D-B917-819B9852CB31"><title>Introduction</title> <p>If the ROM has been built with paging

enabled, the image is divided into two sections: an unpaged section and a

paged section. In addition to this, code that is not part of the ROM can be

loaded on demand into RAM from other non-XIP partitions and/or media, for

example FAT or ROFS partitions. </p> <p>Two types of paging are currently

supported: </p> <ul>

<li id="GUID-E838F239-2F8A-5441-AE76-59382AA75FF6"><p>ROM paging - paging

from the paged section of the ROM image </p> </li>

<li id="GUID-28970AFC-33C6-56B7-BBD3-383220359A90"><p>code paging - paging

from non-removable media, for example, a FAT partition on an internal Multi

Media Card (MMC) drive or an internal NAND ROFS/FAT partition. </p> <p>See

also <xref href="GUID-90B5FDD9-7D59-5035-BF53-2B177655DCD6.dita">Paging from an

internal MMC</xref>. </p> </li>

</ul> <p> <note>: A difference between ROM paging and code paging is that

all ROM pages are contiguous, whereas code that is paged from other drives

may be split over several potentially non-contiguous clusters. This puts an

extra burden on the paging subsystem as it needs to identify the layout of

the DLL on the media before it is deemed pageable. This is achieved by using

the file servers blockmap API.</note> </p> <p>Media drivers are typically

PDDs with a filename of <filepath>med.pdd</filepath>. Normally they are declared

in the <filepath>rombuild.oby</filepath> file with the keyword <codeph>extension</codeph> or <codeph>device</codeph> and

are therefore flagged as unpaged. That is, once loaded their code and read-only

data sections are not paged out. </p> <p>A media driver that is capable of

servicing page requests from the paging subsystem must ensure that the thread

in which the media driver runs takes the page fault itself otherwise a deadlock

could occur. In theory, the only time this can happen is when a media driver

accepts a write request from a user side client that points to data in the

paged section of the ROM or to code that has been loaded into RAM from code

paging enabled media. To remedy this, the local media subsystem has been modified

to lock write requests to paging media drivers before they are dispatched

and to split large write requests into a series of smaller ones to avoid exhausting

available RAM. </p> <p>The two initial stages relevant to this discussion

are: </p> <ul>

<li id="GUID-02D2CBAA-8393-52C2-B940-31603B4AF9FA"><p>the kernel extension

entry point - identified by the <codeph>DECLARE_STANDARD_EXTENSION</codeph> macro </p> </li>

<li id="GUID-1EFE9978-780D-507C-B408-E0C3AE97A0A7"><p>the PDD entry point

- identified by the <codeph>DECLARE_EXTENSION_PDD</codeph> macro. </p> </li>

</ul> <p>To enable demand paging as soon as possible in the boot sequence

it is desirable to instantiate and install the PDD factory object earlier,

for example in the kernel extension entry point. </p> <p> <note> Some media

drivers have no kernel extension entry point defined, the MMC media driver

is an example. These drivers have a <xref href="GUID-73947402-2F32-35C7-94C4-22B4D63D5FB3.dita"><apiname>DECLARE_STANDARD_PDD</apiname></xref> macro

defined rather than <xref href="GUID-52853C0D-CA98-3B92-B7D4-FF1C1F06C1A6.dita"><apiname>DECLARE_EXTENSION_PDD</apiname></xref>. They require

modification to have a <xref href="GUID-52853C0D-CA98-3B92-B7D4-FF1C1F06C1A6.dita"><apiname>DECLARE_EXTENSION_PDD</apiname></xref> and a <xref href="GUID-8B6DF6D7-4995-3564-9303-272500D7E747.dita"><apiname>DECLARE_STANDARD_EXTENSION</apiname></xref>.</note> </p> <p>The

steps needed to support ROM and/or code paging are as follows: </p> <ol id="GUID-CADDDC61-A3A2-5AC3-AD69-739FD61FA9CC">

<li id="GUID-4D14D36E-5F3A-55E6-BAB8-A45243B34928"><p>determine whether code

paging is supported, and if so, identify the relevant local drive number or

numbers </p> </li>

<li id="GUID-F97B3965-020F-5C16-9D09-5861C0B6C695"><p> <xref href="GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3.dita#GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3/GUID-77FBA53A-CC78-5CD1-8FDF-F6A8001DE466">modify variantmediadef.h</xref>  </p> </li>

<li id="GUID-D608C61F-E297-5A0C-B0FD-4F0F98B36EEF"><p> <xref href="GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3.dita#GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3/GUID-686EFEF4-A514-51BC-8378-962A8899F270">modify the media drivers kernel extension entry point</xref> to register

the media driver with the paging subsystem and to instantiate and install

the <xref href="GUID-A5484A7F-94B9-34C7-9F88-82B1BF516930.dita"><apiname>DPhysicalDevice</apiname></xref> derived factory object </p> </li>

<li id="GUID-5DEA2654-8954-52EC-B498-066AE05AB209"><p> <xref href="GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3.dita#GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3/GUID-C32CF25E-553B-5786-88ED-4587AC0DF3BA">modify the DLocalDrive::Ecaps() handling</xref>  </p> </li>

<li id="GUID-CED0CA1D-44EF-51DC-BC56-60B88A9D0494"><p>modify the media drivers <xref href="GUID-A0D4EB25-0BA4-39EE-874B-465EB9628DCC.dita#GUID-A0D4EB25-0BA4-39EE-874B-465EB9628DCC/GUID-EB244422-0BEC-3D3F-BDDA-3F050A2697A7"><apiname>DMediaDriver::Request</apiname></xref> function

to accept the four new <xref href="GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3.dita#GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3/GUID-AE44672E-F60D-563C-BB6A-B70AF187C366">paging

request</xref> types </p> </li>

<li id="GUID-A7CFB8AA-4D95-5DD3-AF3F-0FAB0448F218"><p> <xref href="GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3.dita#GUID-EB2566BD-8F65-5A81-B215-E8B05CFE21C3/GUID-0EA673E6-FEE4-51F6-9C84-9411316D7357">Handling fragmented write requests</xref>. </p> </li>

</ol> </section>

<section id="GUID-77FBA53A-CC78-5CD1-8FDF-F6A8001DE466"><title>Changes to

variantmediadef.h</title> <p>The following should be defined using appropriate

names in the variant's <i>variantmediadef.h</i> file: </p> <ul>

<li id="GUID-F5EFB1C8-B7C4-52AD-88A2-8A7659527373"><p> <codeph>PAGING_TYPE</codeph> -

flags that indicate whether code paging and/or ROM paging is supported </p> </li>

<li id="GUID-2745304A-2FAF-5DCB-88D3-74BD56B475E0"><p> <codeph>NAND_PAGEDRIVELIST</codeph> -

a list of the paging drives </p> </li>

<li id="GUID-507FB105-E254-501E-B22A-33E93C3CABA5"><p> <codeph>NAND_PAGEDRIVECOUNT</codeph> -

the total number of paging drives </p> </li>

<li id="GUID-F3D6494E-A14D-5FB1-A3E7-3D75D3753E12"><p> <codeph>NUM_PAGES</codeph> -

if a write request points to data that resides in paged ROM, the request is

split up into separate fragments of the specified size. This value needs to

be chosen with care, as if it is too small writes can take too long to finish. </p> </li>

</ul><p> <note> Normal write requests which point to data that is not in paged

ROM are not fragmented. However, large requests are split up into smaller

requests by the file server, providing clients with a more responsive system.</note> </p> <p>The

macros defined in the file <i>variantmediadef.h</i> are passed to <xref href="GUID-9E60E8D9-619E-3A76-BAC8-93A60D62C7DF.dita#GUID-9E60E8D9-619E-3A76-BAC8-93A60D62C7DF/GUID-5FDD89C6-C34A-3A0D-A422-D148DDE23E42"><apiname>LocDrv::RegisterPagingDevice()</apiname></xref>.

This function is similar to <xref href="GUID-9E60E8D9-619E-3A76-BAC8-93A60D62C7DF.dita#GUID-9E60E8D9-619E-3A76-BAC8-93A60D62C7DF/GUID-647D0858-FE04-3A4F-99CE-81CD0B34CE7B"><apiname>LocDrv::RegisterMediaDevice()</apiname></xref> in

that it takes a drive list as a parameter, but it identifies the drive or

drives to be used for code paging. If code only ROM paging is needed, set

the drive count to zero. </p> <p>Changes made to support paging on NAND: </p> <codeblock id="GUID-1425F457-8F28-5CD7-B8A4-3FC3D2777D73" xml:space="preserve">// Variant parameters for NAND flash media driver (mednand.pdd)

#define NAND_DRIVECOUNT 8

#define NAND_DRIVELIST 2,3,5,6,7,9,10,11

#define NAND_NUMMEDIA 1

#define NAND_DRIVENAME "Nand"

#define PAGING_TYPE DPagingDevice::ERom | DPagingDevice::ECode

// code paging from writeable FAT, Composite FAT and first ROFS

#define NAND_PAGEDRIVELIST 2,5,6    

#define NAND_PAGEDRIVECOUNT 3

// defines the size of fragment

#define NUM_PAGES 8</codeblock> </section>

<section id="GUID-686EFEF4-A514-51BC-8378-962A8899F270"><title>Changes to

the media driver kernel extension point</title> <p>The kernel-extension entry

point must create a DFC queue to satisfy any page fault that occurs in the

drive thread. Failure to do so results in a kernel fault. The entry point

must then create a <xref href="GUID-3715787E-9ADD-39E7-B22A-62CBBD2CEF1B.dita"><apiname>DPrimaryMediaBase</apiname></xref> object and register

it with the local media subsystem. To support paging, the entry point needs

altering to register the paging device with the demand paging subsystem and

instantiate and install the factory object. </p> <codeblock id="GUID-885C8178-46FB-5F4B-9755-E048A48C42A7" xml:space="preserve">DECLARE_STANDARD_EXTENSION()

    {   

    TInt r=Kern::DfcQInit(&TestMediaDfcQ,KTestThreadPriority,&KTestMediaThreadName);

    if (r|=KErrNone)

        return r;

    DPrimaryMediaBase* pM=new DPrimaryMediaBase;

    if (!pM)

        return r;

    pM->iDfcQ=&TestMediaDfcQ;

    r=LocDrv::RegisterMediaDevice(

        MEDIA_DEVICE_NAND,

        NAND_DRIVECOUNT,

        NAND_DRIVELIST,

        pM,

        NAND_NUMMEDIA,

        NAND_DRIVENAME);

    if (r != KErrNone)

        return r;

    r = LocDrv::RegisterPagingDevice(

        pM,

        NAND_PAGEDRIVELIST,

        NAND_PAGEDRIVECOUNT,

        PAGING_TYPE,

        SECTOR_SHIFT,

        NUM_PAGES);

    if (r == KErrNone)

        {

        device = new DPhysicalDeviceMediaTest;

        if (device == NULL)

            return KErrNoMemory;

        r = Kern::InstallPhysicalDevice(device);

        }

    // Ignore error if demand paging not supported by kernel

    else if (r == KErrNotSupported)

        r = KErrNone;

    else

        return r;

    pM->iMsgQ.Receive();

    return KErrNone;

    }</codeblock> <p>The fifth parameter passed to the function <xref href="GUID-9E60E8D9-619E-3A76-BAC8-93A60D62C7DF.dita#GUID-9E60E8D9-619E-3A76-BAC8-93A60D62C7DF/GUID-5FDD89C6-C34A-3A0D-A422-D148DDE23E42"><apiname>LocDrv::RegisterPagingDevice()</apiname></xref> named <codeph>SECTOR_SHIFT</codeph> is the log2 of the sector size for the given media. For example, passing

a value of 9 corresponds to a sector size of 512 for most media. </p> <p> <note> The <xref href="GUID-52853C0D-CA98-3B92-B7D4-FF1C1F06C1A6.dita"><apiname>DECLARE_EXTENSION_PDD</apiname></xref> entry

point is called some time later when the file server tries to load all the

media drivers in the system. When this happens a second factory object is

created by the media driver, but this is deleted by the kernel when it discovers

that another factory object bearing the same name is already in its internal

list.</note> </p> </section>

<section id="GUID-C32CF25E-553B-5786-88ED-4587AC0DF3BA"><title>Changes to

DLocalDrive::ECaps handling</title> <p>The <xref href="GUID-C57F8D34-DAB5-388F-A99F-A952916B7EA6.dita"><apiname>TLocalDriveCaps</apiname></xref> structure

needs to be modified so that: </p> <ul>

<li id="GUID-FECDE295-A9D3-5373-BD01-F2A7EAB83897"><p>the <xref href="GUID-FF7F3425-9F7C-3146-BA0A-FB68D1A2381C.dita"><apiname>KMediaAttPageable</apiname></xref> flag

is set in <xref href="GUID-722A1F4D-3A76-3BE3-95AB-2AA7745930D0.dita"><apiname>iMediaAtt</apiname></xref>  </p> </li>

<li id="GUID-82EC6B7C-FF34-5E2C-BFF1-832C0769B517"><p>the <xref href="GUID-ED80FF32-9226-31E4-8717-D4A29948A75B.dita"><apiname>KDriveAttPageable</apiname></xref> flag

is set if a particular drive has been registered as code paging. This is determined

by testing <xref href="GUID-D84A9903-AE0F-3F54-8833-E8956A88E26C.dita#GUID-D84A9903-AE0F-3F54-8833-E8956A88E26C/GUID-F5F80522-E23D-3A3D-86B5-6DC6C9E5C3B1"><apiname>TLocDrvRequest::Drive()</apiname></xref> <codeph>-&gt;iPagingDrv</codeph>. </p> </li>

</ul> <p>Additionally, the <xref href="GUID-C57F8D34-DAB5-388F-A99F-A952916B7EA6.dita"><apiname>TLocalDriveCaps</apiname></xref> <codeph>::iDriveAtt</codeph> must

have the <xref href="GUID-89ABFCAC-65EF-3DF3-83B8-2B0A169CCAFA.dita"><apiname>KDriveAttLocal</apiname></xref> and <xref href="GUID-78343374-6F5D-365D-BF3E-9AD19C77FB80.dita"><apiname>KDriveAttInternal</apiname></xref> bits

set and the <xref href="GUID-88C9C181-3BC3-35FB-BF82-A81C40EEDEB8.dita"><apiname>KDriveAttRemovable</apiname></xref> bit cleared. </p> <codeblock id="GUID-04F15DFB-144E-5966-88B5-227B7FC97B55" xml:space="preserve">TInt DMediaDriverTest::Request(TLocDrvRequest&amp; aRequest)

    {

    TInt r=KErrNotSupported;

    TInt id=aRequest.Id();

    if (id == DLocalDrive::ECaps)

        {

        TLocDrv* drive = aRequest.Drive();

        TLocalDriveCapsV4& c = *(TLocalDriveCapsV4*)aRequest.RemoteDes();   

        r=Caps(*drive,c);

        }

    // etc…

    }

TInt DMediaDriverTest::Caps(TLocDrv& aDrive, TLocalDriveCapsV4& caps)

    {

    // fill in rest of caps structure as usual…

    if(aDrive.iPrimaryMedia->iPagingMedia)

        caps.iMediaAtt|=KMediaAttPageable;

    if(aDrive.iPagingDrv)

        caps.iDriveAtt|=KDriveAttPageable; 

    }</codeblock> </section>

<section id="GUID-AE44672E-F60D-563C-BB6A-B70AF187C366"><title>Handling page

requests</title> <p>Four new request types need to be handled to support paging: </p> <ul>

<li id="GUID-703713F1-9276-583A-9963-325C04F8D4B6"><p> <xref href="GUID-951FB996-24B3-3340-8386-24B1A895EA16.dita"><apiname>EWriteRequestFragment</apiname></xref> marks

the start and middle of a sequence of writes. </p> </li>

<li id="GUID-F40CCE14-17DF-52EB-937F-CEBC2B78CC94"><p>Each sequence is terminated

by a <xref href="GUID-8B4109E6-B9F8-3C18-9F9A-0AB20FDA5E86.dita"><apiname>EWriteRequestFragmentLast</apiname></xref> request as long as none

of the prior requests completed with an error. </p> </li>

<li id="GUID-47C8C482-77B4-5447-8497-E8185BF1350B"><p> <xref href="GUID-F20251F4-A72D-32E5-B2AF-87F71CD5CD87.dita"><apiname>ERomPageInRequest</apiname></xref> is

treated as a normal read except that: </p> <ol id="GUID-DEEF7484-32E1-5739-839D-D1257A51B83C">

<li id="GUID-A90D4A45-05E4-5F9C-B47E-26F67F5462EE"><p>the list of partitions

reported by <xref href="GUID-A0D4EB25-0BA4-39EE-874B-465EB9628DCC.dita#GUID-A0D4EB25-0BA4-39EE-874B-465EB9628DCC/GUID-B2F42829-028C-341A-B34D-A8243CA434EB"><apiname>DMediaDriver::PartitionInfo</apiname></xref> does not normally

include the partition containing the ROM image. Therefore, the local media

subsystem does not know the absolute position from the start of the media

of a particular ROM page. The position stored in <xref href="GUID-F20251F4-A72D-32E5-B2AF-87F71CD5CD87.dita"><apiname>ERomPageInRequest</apiname></xref> is

offset from the start of the ROM image, rather than the start of the media.

Therefore, the media driver must add the offset of the start of the ROM image

to the position stored in <xref href="GUID-F20251F4-A72D-32E5-B2AF-87F71CD5CD87.dita"><apiname>ERomPageInRequest</apiname></xref> to obtain the

absolute position before issuing a read request. </p> </li>

<li id="GUID-E020A5D1-82FA-5857-AB1D-42A59D337EFD"><p>when the read is complete

the media driver needs to call <xref href="GUID-D84A9903-AE0F-3F54-8833-E8956A88E26C.dita#GUID-D84A9903-AE0F-3F54-8833-E8956A88E26C/GUID-362AB6B2-9C3D-342F-BB4A-F252ECD7EF3A"><apiname>TLocDrvRequest::WriteToPageHandler</apiname></xref> to

write the data back to the client, rather than <xref href="GUID-D84A9903-AE0F-3F54-8833-E8956A88E26C.dita#GUID-D84A9903-AE0F-3F54-8833-E8956A88E26C/GUID-2AC4DBA4-5C87-39C7-B6CF-95ECB7286339"><apiname>TLocDrvRequest::WriteRemote</apiname></xref> as

for a normal read, </p> </li>

</ol> </li>

<li id="GUID-50298F39-D1A0-5731-8E50-D18514941D70"><p> <xref href="GUID-E42B407D-8E18-337B-9CBB-4DCDD0AABBE9.dita"><apiname>ECodePageInRequest</apiname></xref> is

treated as a normal read, except that the function <xref href="GUID-D84A9903-AE0F-3F54-8833-E8956A88E26C.dita#GUID-D84A9903-AE0F-3F54-8833-E8956A88E26C/GUID-362AB6B2-9C3D-342F-BB4A-F252ECD7EF3A"><apiname>TLocDrvRequest::WriteToPageHandler</apiname></xref> should

be used to write the data back to the demand paging subsystem. However, the

position in the request is offset from the start of the media as for a normal

read. </p> </li>

</ul> <p>These request types are enumerated in the <xref href="GUID-6C039CFA-BF2F-3272-80FE-71430C12F0F4.dita"><apiname>DMediaPagingDevice</apiname></xref> class: </p> <codeblock id="GUID-AD9B48CF-7757-5C8A-A2EF-B27F2DC9BDD8" xml:space="preserve">NONSHARABLE_CLASS(DMediaPagingDevice) : public DPagingDevice

    {

public:

    enum TPagingRequestId

        {

        /** 

        Identifies any middle fragment of a Write request on a partition of a media that supports paging.

        */

        EWriteRequestFragment = 

            DLocalDrive::EFirstReqNumberReservedForPaging,

        /** 

        Identifies the last fragment of a Write request on a partition of a media that supports paging.

        */

        EWriteRequestFragmentLast = 

            DLocalDrive::EFirstReqNumberReservedForPaging+1,

        /** 

        Request for paging in (read) data from the ROM store area.

        */

        ERomPageInRequest =

            DLocalDrive::EFirstReqNumberReservedForPaging+2,

        /** 

        Request for paging in (read) data from the code store area.

        */

        ECodePageInRequest =

            DLocalDrive::ELastReqNumberReservedForPaging

        };

        //etc…

    }</codeblock> </section>

<section id="GUID-0EA673E6-FEE4-51F6-9C84-9411316D7357"><title>Handling fragmented

write requests</title> <p>In many respects, <xref href="GUID-951FB996-24B3-3340-8386-24B1A895EA16.dita"><apiname>EWriteRequestFragment</apiname></xref> and <xref href="GUID-8B4109E6-B9F8-3C18-9F9A-0AB20FDA5E86.dita"><apiname>EWriteRequestFragmentLast</apiname></xref> can

be treated as normal write requests. It should be noted however, that these

write requests can be interleaved with requests from other file server threads

if the media supports more than one partition, the resulting operations may

be perceived as a functional break in behaviour. </p> <p>If it is important

to maintain backwards compatibility and to prevent write requests from being

interleaved, the media driver must keep track of the current write-request

chain and defer requests from other drive threads while a write-fragment chain

is in progress by: </p> <ul>

<li id="GUID-311A0DF2-601D-5623-A059-7E20FAFEEF1C"><p>ensuring the local media

subsystem LDD has been built with the <codeph>__ALLOW_CONCURRENT_FRAGMENTATION__</codeph> macro

undefined. This ensures that the local media subsystem never issues more than

one write fragment at a time </p> </li>

<li id="GUID-9504A0D1-2147-5968-AB8C-2B9D99A48C0F"><p>modifying the paging-media

driver so that it keeps track of write-request chains and defers any read

or format requests received after the first fragment and before the last in

a sequence. When undefined, the macro subsystem does not issue more than one

write-request chain at a time. </p> </li>

</ul><p><note> Write fragments should never be deferred, only read or format

requests may be deferred. </note></p>  <p>To achieve this the media driver

can maintain a bit mask, each bit of which represents a write in progress

flag for a particular drive: </p> <codeblock id="GUID-CB127C9B-51D5-5D1A-BAF6-4F8CEF308912" xml:space="preserve">iFragmenting|=(0x1&lt;&lt;iCurrentReq-&gt;Drive()-&gt;iDriveNumber);</codeblock> <p>If a read or format request is received while any of the bits in <codeph>iFragmenting</codeph> are

set, that request may be deferred. </p> </section>