Reclaiming Free Space

This document discusses the benefits to be had, such as performance improvements, from performing a compaction operation.

Database compaction

Databases using database compaction will experience a performance improvement for some operations, such as DELETE. There will also be a better trade-off between performance and disk usage.

Legacy databases will be automatically migrated to use the new default free space reclaim policy when they are used with the 9.5 version of Symbian SQL. The migration takes place the first time the legacy database is opened with the new version of Symbian SQL. The database will be configured to use the default free space reclaim policy from then on. Note: Such migration will not be possible (or necessary) for databases that are on read-only media, but these databases will still be readable by Symbian SQL.

Databases can be configured to use a free space reclaim policy of their choice. There are three modes of database compaction:

background,
synchronous,
manual.

The compaction mode is set when the database is created and cannot be changed later. Only the database creator can specify the compaction mode. This avoids the possibility of conflicting policy change requests by clients of shared databases. When a database compaction mode is not specified, background mode is the default mode.

Background compaction

Compaction activities are deferred and scheduled by the server allowing client requests that delete records to complete more quickly. This mode offers clients no guarantees on when space will be reclaimed. When under high load the server will defer reclamation indefinitely.

The server maintains an EPriorityIdle active object for each open database that is set to background compaction mode.

The amount of reclaimable disk space is retrieved after an SQL statement is executed on a database. If this value exceeds the compaction threshold then the database is marked as dirty by setting its active object as ready to run:

The compaction threshold is a server-wide parameter that specifies the amount of reclaimable disk space (in bytes) that must exist in a database before the server compacts that database. A default value for this threshold is supplied by Symbian but device creators can override this in the server configuration file.
Each active object contains a free pages threshold that is derived from the compaction threshold. The free pages threshold is the number of pages that must be in the database for it to be marked as dirty.
Each time a RSqlDatabase::Exec() or RSqlStatement::Exec() operation has been completed the number of free pages in the database is checked against the free pages threshold. If the threshold is met then the database is marked as dirty.

Whenever the Symbian SQL server is idle (not processing a client request) runnable idle active objects are processed that cause free space to be reclaimed from the dirty databases:

Each active object reclaims no more pages from its database than the single reclaim limit. This limit is a server-wide parameter specifying the maximum number of free pages that the server will attempt to reclaim in one step and should be small enough to ensure that the server can remain responsive to client requests. An appropriate default value is supplied by Symbian but device creators can override this in the server configuration file.
Following the reclaim action an active object will again set itself as ready to run if the number of free pages following the reclaim meets the free pages threshold.
The idle priority active objects for background compaction continues to be processed until either:
- there is no more free space left to reclaim,
- a higher priority active object becomes ready to run.

Scheduling of background compaction

The server’s active scheduler maintains a priority queue for servicing the active objects. The scheduling algorithm is such that following the successful execution of any active object's RunL(), the scheduler will reset to the beginning of the queue. This has two implications:

There is an implicit priority scheme among the active objects for background compaction – a database opened earlier will always be given priority over one opened later due to its location in the queue.
Compaction of a later database in the queue cannot begin until an earlier database has been fully compacted. This is because the active scheduler resets to the beginning of the queue following every active object execution.

With this lack of fair scheduling it is advised that certain situations are avoided:

Background compaction is not intended to be an antidote for critical disk space rescue scenarios – a fairer scheduling algorithm may help but the system would already be in severe difficulties by this stage.
The compaction operation itself requires free disk space and thus would be inoperable in critically low disk space situations. Thus a fairer scheduling algorithm would not help here.
background compaction is not designed to work efficiently when the server load is such that idle periods do not occur in sufficient quantity.

Synchronous compaction

compaction occurs at the end of the client request. A request is not completed until compaction has finished. This mode is equivalent to the compaction profile of previous versions of Symbian SQL.

Manual compaction

No automatic compaction occurs. This mode is intended for clients that wish to manage compaction directly.

Clients wishing to exercise direct control over database compaction may select manual mode. The server performs no automated compaction in manual mode.

Clients can request database compaction with RSqlDatabase::Compact(). The client can specify a limit to the number of bytes to reclaim. Compaction can be performed as either a synchronous or an asynchronous operation. Compact() can be called on either a “manual” or a “background” database (it can also be called on a “synchronous” database although this will have no effect).

The database file size and the amount of reclaimable free space in the database can be retrieved with RSqlDatabase::Size(). This API supports clients wishing to trigger compaction based on either absolute free space or a proportion of free space in the database.