Migration -Tutorial: Fair Scheduling and File Caching

Describes how to implement and configure the fair -scheduling and file caching features for file systems implementations.

Enabling fair -scheduling

The MExtendedFileInterface API enables -fair scheduling. The FAT, FAT32 and LFFS file systems support MExtendedFileInterface, -this API allows requests to be split into one or more segments with the aOffset parameter. -Support has been built in for large files by changing the file’s position -parameter (aPos) from a TInt to a TInt64.

To -enable fair scheduling, the file system mounted on a particular drive must -support CFileCB::MExtendedFileInterface. The object returned -by CFileSystem::NewFileL() must be derived from CFileCB and CFileCB::MExtendedFileInterface. -The file server determines whether this is the case by calling CFileCB::GetInterface.

The CFileCB::MExtendedFileInterface interface is as follows:

class MExtendedFileInterface -{ -public: - virtual void ReadL(TInt64 aPos, TInt& aLength, TDes8* aDes, const RMessagePtr2& aMessage, TInt aOffset) = 0; - virtual void WriteL(TInt64 aPos, TInt& aLength, const TDesC8* aDes, const RMessagePtr2& aMessage, TInt aOffset) = 0; - virtual void SetSizeL(TInt64 aSize) = 0; -};

The file system indicates its support for MExtendedFileInterface by -intercepting the CFileCB::EExtendedFileInterface enumeration -in the CFileCB::GetInterface() method as in the following -example:

TInt CFatFileCB::GetInterface(TInt aInterfaceId, TAny*& aInterface, TAny* aInput) - { - switch(aInterfaceId) - { - case EExtendedFileInterface: - ((CFileCB::MExtendedFileInterface*&) aInterface) = this; - return KErrNone; - - //etc. - - default: - return CFileCB::GetInterface(aInterfaceId, aInterface, aInput); - } - }

Enabling read -and write caching

To enable caching, the file system must support -reads and writes to local buffers, which are buffers created and owned by -the file server itself rather by a client of the file server.

The local -media subsystem already supports local messages, but the file system and any -file system extensions need to be examined carefully to ensure that they do -not call any RMessagePtr2 methods, otherwise a panic results.

The -file server calls CMountCB::LocalBufferSupport() to find -out whether the file system and all file system extensions mounted on a particular -drive fully support local buffers before it enables caching. The file system -handles the CMountCB::ELocalBufferSupport enumeration in -its CMountCB::GetInterface() method and passes the request -on to the first proxy drive, as in the following example:

TInt CFatMountCB::GetInterface(TInt aInterfaceId, TAny*& aInterface, TAny* /*aInput*/) - { - TInt r= KErrNone; - switch(aInterfaceId) - { - // file caching supported, so pass query on to first proxy drive - case CMountCB::ELocalBufferSupport: - r = LocalDrive()->LocalBufferSupport(); - break; - default: - r=KErrNotSupported; - } - return r; - }

If no file system extensions are loaded, then the query -is eventually handled by CLocalProxyDrive::GetInterface(), -which returns KErrNone to indicate that TBusLocalDrive and -the local media sub-system support local buffers.

If there are any -file system extensions, they must handle the CProxyDrive::ELocalBufferSupport enumeration, -as in the following example:

TInt CBitExtProxyDrive::GetInterface(TInt aInterfaceId, TAny*& aInterface, TAny* aInput) - { - switch(aInterfaceId) - { - // file caching supported, so pass query on to next proxy drive - case ELocalBufferSupport: - return CBaseExtProxyDrive::LocalBufferSupport(); - - default: - return CBaseExtProxyDrive::GetInterface(aInterfaceId, aInterface, aInput); - } - }

Configuring -caching

The global and drive-specific cache property defaults are -defined in f32\sfile\sf_file_cache_defs.h. They may be -overridden for a particular drive by appropriate entries in the Base Starter -component's ESTART.TXT configuration file. The format -of this file has been enhanced to support the .ini file style -syntax:

C: 0 ELOCAL FAT 0 FS_FORMAT_COLD,FS_SYNC_DRIVE # IRAM -D: 1 ELOCAL FAT 0 FS_SCANDRIVE # MMC (textshell) -I: 2 ELOCAL FAT 0 FS_FORMAT_CORRUPT # NAND - USER DATA -K: 8 ELFFS LFFS 0 FS_FORMAT_CORRUPT # LFFS - -[DriveD] -FileCacheSize 256 -FairSchedulingLen 128 -FileCacheRead ON -FileCacheReadAhead ON -FileCacheWrite ON -ClosedFileKeepAliveTime 3000 -DirtyDataFlushTime 3000 -FileCacheReadAsync ON - -[DriveI] -FileCacheWrite ENABLED - -[DriveK] -FileCacheWrite ENABLED - -[FileCache] -GlobalCacheEnabled ON -GlobalCacheSize 32768 -GlobalCacheMaxLockedSize 1024 -LowMemoryThreshold 10

In this example, write caching is enabled -on drives I and K and has been turned on by default on drive D.

Most -of the properties in the example are set to their default values: the definitions -are redundant but are shown for illustrative purposes.

For details, -see Base Starter Technology.

Global cache settings

The -following table holds the global caching properties:

- - - -

Property

Type

Comments

Default value

- - -

GlobalCacheEnabled

boolean

on or off

- - -

GlobalCacheSize

integer

Size of disconnected chunk shared by all files in kilobytes.

32768K (32MB)

- - -

GlobalCacheMaxLockedSize

integer

Maximum amount of locked RAM for all files in kilobytes.

1024K (1 MB)

- - -

LowMemoryThreshold

integer

Low memory threshold expressed as a percentage of total RAM. If -the amount of RAM drops below this value, attempts to allocate memory for -file caching fails.

10%

- - - -

File cache settings

The -properties FileCacheRead, FileCacheReadAhead and FileCacheWrite may -each be in one of 3 states:

OFF - -file caching is permanently off and cannot be turned on.
ENABLED - -file caching is off by default but may be turned on using an appropriate file -open mode, such as EFileReadBuffered
ON - -file caching is on by default but may be turned off using an appropriate file -open mode, such as EFileReadDirectIO.

See Run time -cache settings.

If FileCacheReadAsync is -set, media driver reads on this drive are asynchronous. This results in read -ahead requests being issued before completing a client read request. However, -if FileCacheReadAsync is OFF, then media driver reads on -this drive are synchronous. So, issuing a read ahead is likely to block any -client thread that is normally running at lower priority than the media driver's -thread. In this case read ahead requests only happen during periods of inactivity -to improve latency.

- - - -

Property

Type

Comments

Default value

- - -

FileCacheSize

integer

Maximum amount of locked or unlocked RAM per file.

128K

- - -

FairSchedulingLen

integer

Read & write requests greater than this length are split up -into 2 or more requests.

128K

- - -

FileCacheRead

string

OFF, ENABLED or ON.

ENABLED

- - -

FileCacheReadAhead

string

OFF, ENABLED or ON.

- - -

FileCacheWrite

string

OFF, ENABLED or ON.

ENABLED

- - -

FileCacheReadAsync

boolean

ON or OFF. If set, media driver reads on this drive are asynchronous.

OFF

- - -

ClosedFileKeepAliveTime

integer

Time after which a file is removed from the closed file queue, in -milliseconds.

3000ms (3 secs)

- - -

DirtyDataFlushTime

integer

Time after which a file containing dirty data is flushed, in milliseconds.

3000ms (3 secs)

- - - -

+ + + + + +Migration +Tutorial: Fair Scheduling and File Caching +

Describes how to implement and configure the fair +scheduling and file caching features for file systems implementations.

Enabling fair +scheduling

The MExtendedFileInterface API enables +fair scheduling. The FAT, FAT32 and LFFS file systems support MExtendedFileInterface, +this API allows requests to be split into one or more segments with the aOffset parameter. +Support has been built in for large files by changing the file’s position +parameter (aPos) from a TInt to a TInt64.

To +enable fair scheduling, the file system mounted on a particular drive must +support CFileCB::MExtendedFileInterface. The object returned +by CFileSystem::NewFileL() must be derived from CFileCB and CFileCB::MExtendedFileInterface. +The file server determines whether this is the case by calling CFileCB::GetInterface.

The CFileCB::MExtendedFileInterface interface is as follows:

class MExtendedFileInterface +{ +public: + virtual void ReadL(TInt64 aPos, TInt& aLength, TDes8* aDes, const RMessagePtr2& aMessage, TInt aOffset) = 0; + virtual void WriteL(TInt64 aPos, TInt& aLength, const TDesC8* aDes, const RMessagePtr2& aMessage, TInt aOffset) = 0; + virtual void SetSizeL(TInt64 aSize) = 0; +};

The file system indicates its support for MExtendedFileInterface by +intercepting the CFileCB::EExtendedFileInterface enumeration +in the CFileCB::GetInterface() method as in the following +example:

TInt CFatFileCB::GetInterface(TInt aInterfaceId, TAny*& aInterface, TAny* aInput) + { + switch(aInterfaceId) + { + case EExtendedFileInterface: + ((CFileCB::MExtendedFileInterface*&) aInterface) = this; + return KErrNone; + + //etc. + + default: + return CFileCB::GetInterface(aInterfaceId, aInterface, aInput); + } + }

Enabling read +and write caching

To enable caching, the file system must support +reads and writes to local buffers, which are buffers created and owned by +the file server itself rather by a client of the file server.

The local +media subsystem already supports local messages, but the file system and any +file system extensions need to be examined carefully to ensure that they do +not call any RMessagePtr2 methods, otherwise a panic results.

The +file server calls CMountCB::LocalBufferSupport() to find +out whether the file system and all file system extensions mounted on a particular +drive fully support local buffers before it enables caching. The file system +handles the CMountCB::ELocalBufferSupport enumeration in +its CMountCB::GetInterface() method and passes the request +on to the first proxy drive, as in the following example:

TInt CFatMountCB::GetInterface(TInt aInterfaceId, TAny*& aInterface, TAny* /*aInput*/) + { + TInt r= KErrNone; + switch(aInterfaceId) + { + // file caching supported, so pass query on to first proxy drive + case CMountCB::ELocalBufferSupport: + r = LocalDrive()->LocalBufferSupport(); + break; + default: + r=KErrNotSupported; + } + return r; + }

If no file system extensions are loaded, then the query +is eventually handled by CLocalProxyDrive::GetInterface(), +which returns KErrNone to indicate that TBusLocalDrive and +the local media sub-system support local buffers.

If there are any +file system extensions, they must handle the CProxyDrive::ELocalBufferSupport enumeration, +as in the following example:

TInt CBitExtProxyDrive::GetInterface(TInt aInterfaceId, TAny*& aInterface, TAny* aInput) + { + switch(aInterfaceId) + { + // file caching supported, so pass query on to next proxy drive + case ELocalBufferSupport: + return CBaseExtProxyDrive::LocalBufferSupport(); + + default: + return CBaseExtProxyDrive::GetInterface(aInterfaceId, aInterface, aInput); + } + }

Configuring +caching

The global and drive-specific cache property defaults are +defined in f32\sfile\sf_file_cache_defs.h. They may be +overridden for a particular drive by appropriate entries in the Base Starter +component's ESTART.TXT configuration file. The format +of this file has been enhanced to support the .ini file style +syntax:

C: 0 ELOCAL FAT 0 FS_FORMAT_COLD,FS_SYNC_DRIVE # IRAM +D: 1 ELOCAL FAT 0 FS_SCANDRIVE # MMC (textshell) +I: 2 ELOCAL FAT 0 FS_FORMAT_CORRUPT # NAND - USER DATA +K: 8 ELFFS LFFS 0 FS_FORMAT_CORRUPT # LFFS + +[DriveD] +FileCacheSize 256 +FairSchedulingLen 128 +FileCacheRead ON +FileCacheReadAhead ON +FileCacheWrite ON +ClosedFileKeepAliveTime 3000 +DirtyDataFlushTime 3000 +FileCacheReadAsync ON + +[DriveI] +FileCacheWrite ENABLED + +[DriveK] +FileCacheWrite ENABLED + +[FileCache] +GlobalCacheEnabled ON +GlobalCacheSize 32768 +GlobalCacheMaxLockedSize 1024 +LowMemoryThreshold 10

In this example, write caching is enabled +on drives I and K and has been turned on by default on drive D.

Most +of the properties in the example are set to their default values: the definitions +are redundant but are shown for illustrative purposes.

For details, +see Base Starter Technology.

Global cache settings

The +following table holds the global caching properties:

+ + + +

Property

Type

Comments

Default value

+ + +

GlobalCacheEnabled

boolean

on or off

+ + +

GlobalCacheSize

integer

Size of disconnected chunk shared by all files in kilobytes.

32768K (32MB)

+ + +

GlobalCacheMaxLockedSize

integer

Maximum amount of locked RAM for all files in kilobytes.

1024K (1 MB)

+ + +

LowMemoryThreshold

integer

Low memory threshold expressed as a percentage of total RAM. If +the amount of RAM drops below this value, attempts to allocate memory for +file caching fails.

10%

+ + + +

File cache settings

The +properties FileCacheRead, FileCacheReadAhead and FileCacheWrite may +each be in one of 3 states:

OFF - +file caching is permanently off and cannot be turned on.
ENABLED - +file caching is off by default but may be turned on using an appropriate file +open mode, such as EFileReadBuffered
ON - +file caching is on by default but may be turned off using an appropriate file +open mode, such as EFileReadDirectIO.

See Run time +cache settings.

If FileCacheReadAsync is +set, media driver reads on this drive are asynchronous. This results in read +ahead requests being issued before completing a client read request. However, +if FileCacheReadAsync is OFF, then media driver reads on +this drive are synchronous. So, issuing a read ahead is likely to block any +client thread that is normally running at lower priority than the media driver's +thread. In this case read ahead requests only happen during periods of inactivity +to improve latency.

+ + + +

Property

Type

Comments

Default value

+ + +

FileCacheSize

integer

Maximum amount of locked or unlocked RAM per file.

128K

+ + +

FairSchedulingLen

integer

Read & write requests greater than this length are split up +into 2 or more requests.

128K

+ + +

FileCacheRead

string

OFF, ENABLED or ON.

ENABLED

+ + +

FileCacheReadAhead

string

OFF, ENABLED or ON.

+ + +

FileCacheWrite

string

OFF, ENABLED or ON.

ENABLED

+ + +

FileCacheReadAsync

boolean

ON or OFF. If set, media driver reads on this drive are asynchronous.

OFF

+ + +

ClosedFileKeepAliveTime

integer

Time after which a file is removed from the closed file queue, in +milliseconds.

3000ms (3 secs)

+ + +

DirtyDataFlushTime

integer

Time after which a file containing dirty data is flushed, in milliseconds.

3000ms (3 secs)

+ + + +

\ No newline at end of file