7 Nokia Corporation - initial contribution. |
7 Nokia Corporation - initial contribution. |
8 Contributors: |
8 Contributors: |
9 --> |
9 --> |
10 <!DOCTYPE concept |
10 <!DOCTYPE concept |
11 PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd"> |
11 PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd"> |
12 <concept xml:lang="en" id="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01"><title>Reclaiming Free Space</title><prolog><metadata><keywords/></metadata></prolog><conbody><p>This document discusses the benefits to be had, such as performance improvements, from performing a <b>compaction</b> operation. </p> <section><title>Purpose</title> <p>Database compaction frees up diskspace and improves overall performance when performing certain database operations. </p> </section> <section id="GUID-E7319024-5988-5398-8F77-FAD6F644D1DB"><title>Database compaction</title> <p>Databases using database compaction will experience a performance improvement for some operations, such as DELETE. There will also be a better trade-off between performance and disk usage. </p> <p>Legacy databases will be automatically migrated to use the new default free space reclaim policy when they are used with the 9.5 version of Symbian SQL. The migration takes place the first time the legacy database is opened with the new version of Symbian SQL. The database will be configured to use the default free space reclaim policy from then on. <b>Note</b>: Such migration will not be possible (or necessary) for databases that are on read-only media, but these databases will still be readable by Symbian SQL. </p> <p>Databases can be configured to use a free space reclaim policy of their choice. There are three modes of database compaction: </p> <ul><li id="GUID-7394A159-1BFA-53FF-ABBB-50D1DC8453F0"><p> <xref href="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01.dita#GUID-A64625D4-F101-5DB7-896B-F91A21BABE01/GUID-4AF5A27E-9793-5931-B2D0-3673B556DB6D">background</xref>, </p> </li> <li id="GUID-DCA4B080-B17B-5AFB-916A-9B84C1C3DC5E"><p> <xref href="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01.dita#GUID-A64625D4-F101-5DB7-896B-F91A21BABE01/GUID-CE6F567F-7300-574A-A180-E42786DC04DB">synchronous</xref>, </p> </li> <li id="GUID-72035C8D-9B12-5EDB-BD3D-59762DFB46B9"><p> <xref href="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01.dita#GUID-A64625D4-F101-5DB7-896B-F91A21BABE01/GUID-F9D40D87-B383-5EF5-9959-2A344436477A">manual</xref>. </p> </li> </ul> <p>The compaction mode is set when the database is created and cannot be changed later. Only the database creator can specify the compaction mode. This avoids the possibility of conflicting policy change requests by clients of shared databases. When a database compaction mode is not specified, background mode is the default mode. </p> <p id="GUID-4AF5A27E-9793-5931-B2D0-3673B556DB6D"><b>Background compaction</b> </p> <p>Compaction activities are deferred and scheduled by the server allowing client requests that delete records to complete more quickly. This mode offers clients no guarantees on when space will be reclaimed. When under high load the server will defer reclamation indefinitely. </p> <p>The server maintains an <xref href="GUID-4000D46C-29C0-3497-8E20-FFC908670EC8.dita"><apiname>EPriorityIdle</apiname></xref> active object for each open database that is set to background compaction mode. </p> <p>The amount of reclaimable disk space is retrieved after an SQL statement is executed on a database. If this value exceeds the compaction threshold then the database is marked as <i>dirty</i> by setting its active object as ready to run: </p> <ul><li id="GUID-93948590-87D5-541A-99FE-F166656E85B1"><p> The compaction threshold is a server-wide parameter that specifies the amount of reclaimable disk space (in bytes) that must exist in a database before the server compacts that database. A default value for this threshold is supplied by Symbian but device creators can override this in the server configuration file. </p> </li> <li id="GUID-C5E412DF-E015-592D-8BF9-16D2E82E5514"><p>Each active object contains a <i>free pages</i> threshold that is derived from the compaction threshold. The <i>free pages</i> threshold is the number of pages that must be in the database for it to be marked as <i>dirty</i>. </p> </li> <li id="GUID-2B3A1FEB-C351-5CA9-870D-F87051A36476"><p>Each time a <xref href="GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF.dita#GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF/GUID-6D2120DC-C193-3E2E-A802-B4CD180BDF11"><apiname>RSqlDatabase::Exec()</apiname></xref> or <xref href="GUID-0176BF07-DF94-3259-8F90-DE030E35CE9A.dita#GUID-0176BF07-DF94-3259-8F90-DE030E35CE9A/GUID-52E3CE72-D495-388F-8829-952E32F0F37D"><apiname>RSqlStatement::Exec()</apiname></xref> operation has been completed the number of free pages in the database is checked against the free pages threshold. If the threshold is met then the database is marked as <i>dirty</i>. </p> </li> </ul> <p>Whenever the Symbian SQL server is idle (not processing a client request) runnable idle active objects are processed that cause free space to be reclaimed from the dirty databases: </p> <ul><li id="GUID-D82FD580-ECDA-5449-BC40-C016723D7510"><p> Each active object reclaims no more pages from its database than the single reclaim limit. This limit is a server-wide parameter specifying the maximum number of free pages that the server will attempt to reclaim in one step and should be small enough to ensure that the server can remain responsive to client requests. An appropriate default value is supplied by Symbian but device creators can override this in the server configuration file. </p> </li> <li id="GUID-84319C82-076C-5EA8-AF5D-F6D907678254"><p>Following the reclaim action an active object will again set itself as ready to run if the number of free pages following the reclaim meets the free pages threshold. </p> </li> <li id="GUID-F10B583B-F098-542D-9D6F-0F8A63A08CD0"><p>The idle priority active objects for background compaction continues to be processed until either: </p> <ul><li id="GUID-21EE7484-817C-5EEF-80F1-0600440FD74A"><p>there is no more free space left to reclaim, </p> </li> <li id="GUID-AA4694AB-D403-5807-9CC9-3608F9E86246"><p>a higher priority active object becomes ready to run. </p> </li> </ul> </li> </ul> <p><b>Scheduling of background compaction</b> </p> <p>The server’s active scheduler maintains a priority queue for servicing the active objects. The scheduling algorithm is such that following the successful execution of any active object's <codeph>RunL()</codeph>, the scheduler will reset to the beginning of the queue. This has two implications: </p> <ul><li id="GUID-CF778079-5E0F-5D74-9A8A-5DB69E4B5B00"><p>There is an implicit priority scheme among the active objects for background compaction – a database opened earlier will always be given priority over one opened later due to its location in the queue. </p> </li> <li id="GUID-9EECC098-FC0A-5DB7-8A88-FB59C826582E"><p>Compaction of a later database in the queue cannot begin until an earlier database has been fully compacted. This is because the active scheduler resets to the beginning of the queue following every active object execution. </p> </li> </ul> <p>With this lack of <i>fair scheduling</i> it is advised that certain situations are avoided: </p> <ul><li id="GUID-EAE8BC01-15F8-5A8E-99A6-48F4AB73AD68"><p>Background compaction is not intended to be an antidote for critical disk space rescue scenarios – a fairer scheduling algorithm may help but the system would already be in severe difficulties by this stage. </p> </li> <li id="GUID-D17C590B-31E4-582E-A07B-AA411B123BA2"><p>The compaction operation itself requires free disk space and thus would be inoperable in critically low disk space situations. Thus a fairer scheduling algorithm would not help here. </p> </li> <li id="GUID-80E5F115-4602-5B56-AEAC-B4490992101F"><p>background compaction is not designed to work efficiently when the server load is such that idle periods do not occur in sufficient quantity. </p> </li> </ul> <p id="GUID-CE6F567F-7300-574A-A180-E42786DC04DB"><b>Synchronous compaction</b> </p> <p>compaction occurs at the end of the client request. A request is not completed until compaction has finished. This mode is equivalent to the compaction profile of previous versions of Symbian SQL. </p> <p id="GUID-F9D40D87-B383-5EF5-9959-2A344436477A"><b>Manual compaction</b> </p> <p>No automatic compaction occurs. This mode is intended for clients that wish to manage compaction directly. </p> <p>Clients wishing to exercise direct control over database compaction may select manual mode. The server performs no automated compaction in manual mode. </p> <p>Clients can request database compaction with <xref href="GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF.dita#GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF/GUID-6F94DDD2-2586-3DEC-B5E6-B74C3A7E6DB5"><apiname>RSqlDatabase::Compact()</apiname></xref>. The client can specify a limit to the number of bytes to reclaim. Compaction can be performed as either a synchronous or an asynchronous operation. <codeph>Compact()</codeph> can be called on either a “manual” or a “background” database (it can also be called on a “synchronous” database although this will have no effect). </p> <p>The database file size and the amount of reclaimable free space in the database can be retrieved with <xref href="GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF.dita#GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF/GUID-F45D9371-F212-3D62-877A-3C31C4F49CA0"><apiname>RSqlDatabase::Size()</apiname></xref>. This API supports clients wishing to trigger compaction based on either absolute free space or a proportion of free space in the database. </p> </section> </conbody><related-links><link href="GUID-22844C28-AB5B-5A6F-8863-7269464684B4.dita"><linktext>SQL Overview</linktext> </link> <link><linktext/></link><link href="GUID-4EE4E79E-F6D5-5F14-BA8D-4DD10D229B74.dita"><linktext>Handling Blobs</linktext> </link> </related-links></concept> |
12 <concept id="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01" xml:lang="en"><title>Reclaiming Free Space</title><prolog><metadata><keywords/></metadata></prolog><conbody> |
|
13 <p>This document discusses the benefits to be had, such as performance |
|
14 improvements, from performing a <b>compaction</b> operation. </p> |
|
15 <section id="GUID-9719243A-8391-4B2B-9462-65BF4F2755E7"><title>Purpose</title> <p>Database compaction frees up diskspace |
|
16 and improves overall performance when performing certain database |
|
17 operations. </p> </section> |
|
18 <section id="GUID-E7319024-5988-5398-8F77-FAD6F644D1DB"><title>Database |
|
19 compaction</title> <p>Databases using database compaction will experience |
|
20 a performance improvement for some operations, such as DELETE. There |
|
21 will also be a better trade-off between performance and disk usage. </p> <p>Legacy databases will be automatically migrated to use the new |
|
22 default free space reclaim policy when they are used with the 9.5 |
|
23 version of Symbian SQL. The migration takes place the first time the |
|
24 legacy database is opened with the new version of Symbian SQL. The |
|
25 database will be configured to use the default free space reclaim |
|
26 policy from then on. <b>Note</b>: Such migration will not be possible |
|
27 (or necessary) for databases that are on read-only media, but these |
|
28 databases will still be readable by Symbian SQL. </p> <p>Databases |
|
29 can be configured to use a free space reclaim policy of their choice. |
|
30 There are three modes of database compaction: </p> <ul> |
|
31 <li id="GUID-7394A159-1BFA-53FF-ABBB-50D1DC8453F0"><p> <xref href="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01.dita#GUID-A64625D4-F101-5DB7-896B-F91A21BABE01/GUID-4AF5A27E-9793-5931-B2D0-3673B556DB6D">background</xref>, </p> </li> |
|
32 <li id="GUID-DCA4B080-B17B-5AFB-916A-9B84C1C3DC5E"><p> <xref href="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01.dita#GUID-A64625D4-F101-5DB7-896B-F91A21BABE01/GUID-CE6F567F-7300-574A-A180-E42786DC04DB">synchronous</xref>, </p> </li> |
|
33 <li id="GUID-72035C8D-9B12-5EDB-BD3D-59762DFB46B9"><p> <xref href="GUID-A64625D4-F101-5DB7-896B-F91A21BABE01.dita#GUID-A64625D4-F101-5DB7-896B-F91A21BABE01/GUID-F9D40D87-B383-5EF5-9959-2A344436477A">manual</xref>. </p> </li> |
|
34 </ul> <p>The compaction mode is set when the database is created and |
|
35 cannot be changed later. Only the database creator can specify the |
|
36 compaction mode. This avoids the possibility of conflicting policy |
|
37 change requests by clients of shared databases. When a database compaction |
|
38 mode is not specified, background mode is the default mode. </p> <p id="GUID-4AF5A27E-9793-5931-B2D0-3673B556DB6D"><b>Background compaction</b> </p> <p>Compaction activities are deferred and scheduled by the |
|
39 server allowing client requests that delete records to complete more |
|
40 quickly. This mode offers clients no guarantees on when space will |
|
41 be reclaimed. When under high load the server will defer reclamation |
|
42 indefinitely. </p> <p>The server maintains an <xref href="GUID-4000D46C-29C0-3497-8E20-FFC908670EC8.dita"><apiname>EPriorityIdle</apiname></xref> active object for each open database that is set to background compaction |
|
43 mode. </p> <p>The amount of reclaimable disk space is retrieved after |
|
44 an SQL statement is executed on a database. If this value exceeds |
|
45 the compaction threshold then the database is marked as <i>dirty</i> by setting its active object as ready to run: </p> <ul> |
|
46 <li id="GUID-93948590-87D5-541A-99FE-F166656E85B1"><p> The compaction |
|
47 threshold is a server-wide parameter that specifies the amount of |
|
48 reclaimable disk space (in bytes) that must exist in a database before |
|
49 the server compacts that database. A default value for this threshold |
|
50 is supplied by Symbian but device creators can override this in the |
|
51 server configuration file. </p> </li> |
|
52 <li id="GUID-C5E412DF-E015-592D-8BF9-16D2E82E5514"><p>Each active |
|
53 object contains a <i>free pages</i> threshold that is derived from |
|
54 the compaction threshold. The <i>free pages</i> threshold is the number |
|
55 of pages that must be in the database for it to be marked as <i>dirty</i>. </p> </li> |
|
56 <li id="GUID-2B3A1FEB-C351-5CA9-870D-F87051A36476"><p>Each time a <xref href="GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF.dita#GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF/GUID-6D2120DC-C193-3E2E-A802-B4CD180BDF11"><apiname>RSqlDatabase::Exec()</apiname></xref> or <xref href="GUID-0176BF07-DF94-3259-8F90-DE030E35CE9A.dita#GUID-0176BF07-DF94-3259-8F90-DE030E35CE9A/GUID-52E3CE72-D495-388F-8829-952E32F0F37D"><apiname>RSqlStatement::Exec()</apiname></xref> operation has been completed the number of free pages in the database |
|
57 is checked against the free pages threshold. If the threshold is met |
|
58 then the database is marked as <i>dirty</i>. </p> </li> |
|
59 </ul> <p>Whenever the Symbian SQL server is idle (not processing a |
|
60 client request) runnable idle active objects are processed that cause |
|
61 free space to be reclaimed from the dirty databases: </p> <ul> |
|
62 <li id="GUID-D82FD580-ECDA-5449-BC40-C016723D7510"><p> Each active |
|
63 object reclaims no more pages from its database than the single reclaim |
|
64 limit. This limit is a server-wide parameter specifying the maximum |
|
65 number of free pages that the server will attempt to reclaim in one |
|
66 step and should be small enough to ensure that the server can remain |
|
67 responsive to client requests. An appropriate default value is supplied |
|
68 by Symbian but device creators can override this in the server configuration |
|
69 file. </p> </li> |
|
70 <li id="GUID-84319C82-076C-5EA8-AF5D-F6D907678254"><p>Following the |
|
71 reclaim action an active object will again set itself as ready to |
|
72 run if the number of free pages following the reclaim meets the free |
|
73 pages threshold. </p> </li> |
|
74 <li id="GUID-F10B583B-F098-542D-9D6F-0F8A63A08CD0"><p>The idle priority |
|
75 active objects for background compaction continues to be processed |
|
76 until either: </p> <ul> |
|
77 <li id="GUID-21EE7484-817C-5EEF-80F1-0600440FD74A"><p>there is no |
|
78 more free space left to reclaim, </p> </li> |
|
79 <li id="GUID-AA4694AB-D403-5807-9CC9-3608F9E86246"><p>a higher priority |
|
80 active object becomes ready to run. </p> </li> |
|
81 </ul> </li> |
|
82 </ul> <p><b>Scheduling of background compaction</b> </p> <p>The server’s active |
|
83 scheduler maintains a priority queue for servicing the active objects. |
|
84 The scheduling algorithm is such that following the successful execution |
|
85 of any active object's <codeph>RunL()</codeph>, the scheduler will |
|
86 reset to the beginning of the queue. This has two implications: </p> <ul> |
|
87 <li id="GUID-CF778079-5E0F-5D74-9A8A-5DB69E4B5B00"><p>There is an |
|
88 implicit priority scheme among the active objects for background compaction |
|
89 – a database opened earlier will always be given priority over one |
|
90 opened later due to its location in the queue. </p> </li> |
|
91 <li id="GUID-9EECC098-FC0A-5DB7-8A88-FB59C826582E"><p>Compaction of |
|
92 a later database in the queue cannot begin until an earlier database |
|
93 has been fully compacted. This is because the active scheduler resets |
|
94 to the beginning of the queue following every active object execution. </p> </li> |
|
95 </ul> <p>With this lack of <i>fair scheduling</i> it is advised that |
|
96 certain situations are avoided: </p> <ul> |
|
97 <li id="GUID-EAE8BC01-15F8-5A8E-99A6-48F4AB73AD68"><p>Background compaction |
|
98 is not intended to be an antidote for critical disk space rescue scenarios |
|
99 – a fairer scheduling algorithm may help but the system would already |
|
100 be in severe difficulties by this stage. </p> </li> |
|
101 <li id="GUID-D17C590B-31E4-582E-A07B-AA411B123BA2"><p>The compaction |
|
102 operation itself requires free disk space and thus would be inoperable |
|
103 in critically low disk space situations. Thus a fairer scheduling |
|
104 algorithm would not help here. </p> </li> |
|
105 <li id="GUID-80E5F115-4602-5B56-AEAC-B4490992101F"><p>background compaction |
|
106 is not designed to work efficiently when the server load is such that |
|
107 idle periods do not occur in sufficient quantity. </p> </li> |
|
108 </ul> <p id="GUID-CE6F567F-7300-574A-A180-E42786DC04DB"><b>Synchronous compaction</b> </p> <p>compaction occurs at the end of the client request. A request |
|
109 is not completed until compaction has finished. This mode is equivalent |
|
110 to the compaction profile of previous versions of Symbian SQL. </p> <p id="GUID-F9D40D87-B383-5EF5-9959-2A344436477A"><b>Manual compaction</b> </p> <p>No automatic compaction occurs. This mode is intended for |
|
111 clients that wish to manage compaction directly. </p> <p>Clients wishing |
|
112 to exercise direct control over database compaction may select manual |
|
113 mode. The server performs no automated compaction in manual mode. </p> <p>Clients can request database compaction with <xref href="GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF.dita#GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF/GUID-6F94DDD2-2586-3DEC-B5E6-B74C3A7E6DB5"><apiname>RSqlDatabase::Compact()</apiname></xref>. The client can specify a limit to the number of bytes to reclaim. |
|
114 Compaction can be performed as either a synchronous or an asynchronous |
|
115 operation. <codeph>Compact()</codeph> can be called on either a “manual” |
|
116 or a “background” database (it can also be called on a “synchronous” |
|
117 database although this will have no effect). </p> <p>The database |
|
118 file size and the amount of reclaimable free space in the database |
|
119 can be retrieved with <xref href="GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF.dita#GUID-4688F6B7-E1B0-37CF-BAA2-C6BD103D4FDF/GUID-F45D9371-F212-3D62-877A-3C31C4F49CA0"><apiname>RSqlDatabase::Size()</apiname></xref>. This |
|
120 API supports clients wishing to trigger compaction based on either |
|
121 absolute free space or a proportion of free space in the database. </p> </section> |
|
122 </conbody><related-links> |
|
123 <link href="GUID-22844C28-AB5B-5A6F-8863-7269464684B4.dita"><linktext>SQL |
|
124 Overview</linktext></link> |
|
125 <link href="GUID-4EE4E79E-F6D5-5F14-BA8D-4DD10D229B74.dita"> |
|
126 <linktext>Handling Blobs</linktext></link> |
|
127 </related-links></concept> |