<?xml version="1.0" encoding="utf-8"?>

<!-- Copyright (c) 2007-2010 Nokia Corporation and/or its subsidiary(-ies) All rights reserved. -->

<!-- This component and the accompanying materials are made available under the terms of the License 

"Eclipse Public License v1.0" which accompanies this distribution, 

and is available at the URL "http://www.eclipse.org/legal/epl-v10.html". -->

<!-- Initial Contributors:

    Nokia Corporation - initial contribution.

Contributors: 

-->

<!DOCTYPE concept

  PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">

<concept id="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01" xml:lang="en"><title>Reclaiming Free Space</title><prolog><metadata><keywords/></metadata></prolog><conbody>

<p>This document discusses the benefits to be had, such as performance

improvements, from performing a <b>compaction</b> operation. </p>

<section id="GUID-9719243A-8391-4B2B-9462-65BF4F2755E7"><title>Purpose</title> <p>Database compaction frees up diskspace

and improves overall performance when performing certain database

operations. </p> </section>

<section id="GUID-E7319024-5988-5398-8F77-FAD6F644D1DB"><title>Database

compaction</title> <p>Databases using database compaction will experience

a performance improvement for some operations, such as DELETE. There

will also be a better trade-off between performance and disk usage. </p> <p>Legacy databases will be automatically migrated to use the new

default free space reclaim policy when they are used with the 9.5

version of Symbian SQL. The migration takes place the first time the

legacy database is opened with the new version of Symbian SQL. The

database will be configured to use the default free space reclaim

policy from then on. <b>Note</b>: Such migration will not be possible

(or necessary) for databases that are on read-only media, but these

databases will still be readable by Symbian SQL. </p> <p>Databases

can be configured to use a free space reclaim policy of their choice.

There are three modes of database compaction: </p> <ul>

<li id="GUID-7394A159-1BFA-53FF-ABBB-50D1DC8453F0"><p> <xref href="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01.dita#GUID-A64625D4-F101-5DB7-896B-F91A21BABE01/GUID-4AF5A27E-9793-5931-B2D0-3673B556DB6D">background</xref>, </p> </li>

<li id="GUID-DCA4B080-B17B-5AFB-916A-9B84C1C3DC5E"><p> <xref href="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01.dita#GUID-A64625D4-F101-5DB7-896B-F91A21BABE01/GUID-CE6F567F-7300-574A-A180-E42786DC04DB">synchronous</xref>, </p> </li>

<li id="GUID-72035C8D-9B12-5EDB-BD3D-59762DFB46B9"><p> <xref href="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01.dita#GUID-A64625D4-F101-5DB7-896B-F91A21BABE01/GUID-F9D40D87-B383-5EF5-9959-2A344436477A">manual</xref>. </p> </li>

</ul> <p>The compaction mode is set when the database is created and

cannot be changed later. Only the database creator can specify the

compaction mode. This avoids the possibility of conflicting policy

change requests by clients of shared databases. When a database compaction

mode is not specified, background mode is the default mode. </p> <p id="GUID-4AF5A27E-9793-5931-B2D0-3673B556DB6D"><b>Background compaction</b> </p> <p>Compaction activities are deferred and scheduled by the

server allowing client requests that delete records to complete more

quickly. This mode offers clients no guarantees on when space will

be reclaimed. When under high load the server will defer reclamation

indefinitely. </p> <p>The server maintains an <xref href="GUID-4000D46C-29C0-3497-8E20-FFC908670EC8.dita"><apiname>EPriorityIdle</apiname></xref> active object for each open database that is set to background compaction

mode. </p> <p>The amount of reclaimable disk space is retrieved after

an SQL statement is executed on a database. If this value exceeds

the compaction threshold then the database is marked as <i>dirty</i> by setting its active object as ready to run: </p> <ul>

<li id="GUID-93948590-87D5-541A-99FE-F166656E85B1"><p> The compaction

threshold is a server-wide parameter that specifies the amount of

reclaimable disk space (in bytes) that must exist in a database before

the server compacts that database. A default value for this threshold

is supplied by Symbian but device creators can override this in the

server configuration file.  </p> </li>

<li id="GUID-C5E412DF-E015-592D-8BF9-16D2E82E5514"><p>Each active

object contains a <i>free pages</i> threshold that is derived from

the compaction threshold. The <i>free pages</i> threshold is the number

of pages that must be in the database for it to be marked as <i>dirty</i>. </p> </li>

<li id="GUID-2B3A1FEB-C351-5CA9-870D-F87051A36476"><p>Each time a <xref href="GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF.dita#GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF/GUID-6D2120DC-C193-3E2E-A802-B4CD180BDF11"><apiname>RSqlDatabase::Exec()</apiname></xref> or <xref href="GUID-0176BF07-DF94-3259-8F90-DE030E35CE9A.dita#GUID-0176BF07-DF94-3259-8F90-DE030E35CE9A/GUID-52E3CE72-D495-388F-8829-952E32F0F37D"><apiname>RSqlStatement::Exec()</apiname></xref> operation has been completed the number of free pages in the database

is checked against the free pages threshold. If the threshold is met

then the database is marked as <i>dirty</i>. </p> </li>

</ul> <p>Whenever the Symbian SQL server is idle (not processing a

client request) runnable idle active objects are processed that cause

free space to be reclaimed from the dirty databases: </p> <ul>

<li id="GUID-D82FD580-ECDA-5449-BC40-C016723D7510"><p> Each active

object reclaims no more pages from its database than the single reclaim

limit. This limit is a server-wide parameter specifying the maximum

number of free pages that the server will attempt to reclaim in one

step and should be small enough to ensure that the server can remain

responsive to client requests. An appropriate default value is supplied

by Symbian but device creators can override this in the server configuration

file.  </p> </li>

<li id="GUID-84319C82-076C-5EA8-AF5D-F6D907678254"><p>Following the

reclaim action an active object will again set itself as ready to

run if the number of free pages following the reclaim meets the free

pages threshold. </p> </li>

<li id="GUID-F10B583B-F098-542D-9D6F-0F8A63A08CD0"><p>The idle priority

active objects for background compaction continues to be processed

until either: </p> <ul>

<li id="GUID-21EE7484-817C-5EEF-80F1-0600440FD74A"><p>there is no

more free space left to reclaim, </p> </li>

<li id="GUID-AA4694AB-D403-5807-9CC9-3608F9E86246"><p>a higher priority

active object becomes ready to run. </p> </li>

</ul> </li>

</ul> <p><b>Scheduling of background compaction</b> </p> <p>The server’s active

scheduler maintains a priority queue for servicing the active objects.

The scheduling algorithm is such that following the successful execution

of any active object's <codeph>RunL()</codeph>, the scheduler will

reset to the beginning of the queue. This has two implications: </p> <ul>

<li id="GUID-CF778079-5E0F-5D74-9A8A-5DB69E4B5B00"><p>There is an

implicit priority scheme among the active objects for background compaction

– a database opened earlier will always be given priority over one

opened later due to its location in the queue. </p> </li>

<li id="GUID-9EECC098-FC0A-5DB7-8A88-FB59C826582E"><p>Compaction of

a later database in the queue cannot begin until an earlier database

has been fully compacted. This is because the active scheduler resets

to the beginning of the queue following every active object execution. </p> </li>

</ul> <p>With this lack of <i>fair scheduling</i> it is advised that

certain situations are avoided: </p> <ul>

<li id="GUID-EAE8BC01-15F8-5A8E-99A6-48F4AB73AD68"><p>Background compaction

is not intended to be an antidote for critical disk space rescue scenarios

– a fairer scheduling algorithm may help but the system would already

be in severe difficulties by this stage. </p> </li>

<li id="GUID-D17C590B-31E4-582E-A07B-AA411B123BA2"><p>The compaction

operation itself requires free disk space and thus would be inoperable

in critically low disk space situations. Thus a fairer scheduling

algorithm would not help here. </p> </li>

<li id="GUID-80E5F115-4602-5B56-AEAC-B4490992101F"><p>background compaction

is not designed to work efficiently when the server load is such that

idle periods do not occur in sufficient quantity. </p> </li>

</ul> <p id="GUID-CE6F567F-7300-574A-A180-E42786DC04DB"><b>Synchronous compaction</b> </p> <p>compaction occurs at the end of the client request. A request

is not completed until compaction has finished. This mode is equivalent

to the compaction profile of previous versions of Symbian SQL. </p> <p id="GUID-F9D40D87-B383-5EF5-9959-2A344436477A"><b>Manual compaction</b> </p> <p>No automatic compaction occurs. This mode is intended for

clients that wish to manage compaction directly. </p> <p>Clients wishing

to exercise direct control over database compaction may select manual

mode. The server performs no automated compaction in manual mode. </p> <p>Clients can request database compaction with <xref href="GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF.dita#GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF/GUID-6F94DDD2-2586-3DEC-B5E6-B74C3A7E6DB5"><apiname>RSqlDatabase::Compact()</apiname></xref>. The client can specify a limit to the number of bytes to reclaim.

Compaction can be performed as either a synchronous or an asynchronous

operation. <codeph>Compact()</codeph> can be called on either a “manual”

or a “background” database (it can also be called on a “synchronous”

database although this will have no effect). </p> <p>The database

file size and the amount of reclaimable free space in the database

can be retrieved with <xref href="GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF.dita#GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF/GUID-F45D9371-F212-3D62-877A-3C31C4F49CA0"><apiname>RSqlDatabase::Size()</apiname></xref>. This

API supports clients wishing to trigger compaction based on either

absolute free space or a proportion of free space in the database. </p> </section>

</conbody><related-links>

<link href="GUID-22844C28-AB5B-5A6F-8863-7269464684B4.dita"><linktext>SQL

Overview</linktext></link>

<link href="GUID-4EE4E79E-F6D5-5F14-BA8D-4DD10D229B74.dita">

<linktext>Handling Blobs</linktext></link>

</related-links></concept>