902 return ext; |
902 return ext; |
903 } |
903 } |
904 return ::ExtentL(Host(),iMark,Coord().Base(),aStream); |
904 return ::ExtentL(Host(),iMark,Coord().Base(),aStream); |
905 } |
905 } |
906 |
906 |
907 void CPermanentStoreCollector::RelocateStreamL(const CPermanentStoreCollector::TEntry& aReloc, TInt aExtent) |
|
908 /* relocate a stream into [iFree, aExtent) |
907 /* relocate a stream into [iFree, aExtent) |
909 |
908 |
910 During compaction the sequence of operations is: |
909 During compaction, for each string which is to be moved from position A1 to B1, the sequence of operations is: |
911 |
910 |
912 1. Copy stream S1 content from position A1 to position B1 . The copy never overlaps so the old stream content is still good at this point. |
911 1. Copy stream S1 content from position A1 to position B1 . The copy never overlaps so the old stream content is still good at this point. |
913 2. Optionally rewrite the file header to state that stream S1 is being relocated to B1 (more about the ‘optional below’) |
912 2. Optionally rewrite the file header to state that stream S1 is being relocated to B1 (more about the ‘optional below’) |
914 3. Overwrite the TOC entry for S1 to state that the content is now at B1 |
913 3. Overwrite the TOC entry for S1 to state that the content is now at B1 |
915 4. Carry on with stream S2 |
914 |
916 5. … etc |
915 This function completes 3 steps above and will be called again and again for every string to be moved. |
917 |
916 |
918 |
917 In terms of data consistency, first consider the impact of a mid-write failure in any of these steps (when write caching is disabled): |
919 First consider the impact of a mid-write failure in any of these steps (when write caching is disabled): |
|
920 1. If step #1 only partially completes the file is good as the original content is intact and the new content was being written to otherwise free space |
918 1. If step #1 only partially completes the file is good as the original content is intact and the new content was being written to otherwise free space |
921 2. If step #2 only partially completes the header CRC fails and only the TOC reference is considered valid (so the corrupt stream relocation record is ignored). The TOC will be good because it is being overwritten with the same content. |
919 2. If step #2 only partially completes the header CRC fails and only the TOC reference is considered valid (so the corrupt stream relocation record is ignored). |
922 3. If step #3 only partially completes the entry for S1 in the TOC is corrupt, BUT the relocation record for S1 in the file header is good and will override the entry in the TOC. |
920 The TOC will be good because it is being overwritten with the same content. |
|
921 3. If step #3 only partially completes the entry for S1 in the TOC is corrupt, BUT the relocation record for S1 in the file header is good and will |
|
922 override the entry in the TOC. |
923 |
923 |
924 In all cases the file is never broken by a crash in mid-compaction. |
924 In all cases the file is never broken by a crash in mid-compaction. |
925 |
925 |
926 Step #2 is optional – there are many cases when step #3 cannot fail ‘halfway through’ because the underlying media makes atomic block/page based updates and the write does not cross any block boundaries. In STORE we assume that blocks cannot be smaller than 512 bytes and any flash based media provides the required behavior. Thus 99% of the step #2 writes are eliminated. |
926 Step #2 is optional – there are many cases when step #3 cannot fail ‘halfway through’ because the underlying media makes atomic block/page based |
927 |
927 updates and the write does not cross any block boundaries. In STORE we assume that blocks cannot be smaller than 512 bytes and any flash based |
928 Note that sequencing MATTERS even for just one stream. If the TOC update hits the disk before the content is moved, and then the device fails we will have a broken file: S1 points to B1 which contains garbage. Equally in the case where step #2 is required (i.e. when step #3 straddles a block boundary and could fail) step 2 has to go before the step 3. Otherwise write #3 could go to disk and fail part way through before write #2 and leave the TOC corrupt with no recovery in the file header. |
928 media provides the required behavior. Thus 99% of the step #2 writes are eliminated. |
929 |
929 |
930 Consider the case that step 2 was omitted, so the Store relies on step 3 being completed in order to know that S1 is in location B1; and that no flush is done after step 3. In step 4 the stream S2 is moved – at this point the old space for stream S1 at A1 is considered empty – and suppose it gets moved from A2 to B2 where B2 overlaps/overwrites A1. If the writes in step 3 and step 4 are re-ordered and the step 3 write does not happen – then the TOC will claim that S1 is still at A1 but this location in the file has been overwritten with data from S2. A corrupted file. |
930 Note that sequencing MATTERS even for just one stream. If the TOC update hits the disk before the content is moved, and then the device fails |
931 |
931 we will have a broken file: S1 points to B1 which contains garbage. Equally in the case where step #2 is required (i.e. when step #3 straddles |
932 Based on the knowledge above, it is strongly recommended to set EFileWriteDirectIO bit when opening the file so that the order is maintained when writing to the file. |
932 a block boundary and could fail) step 2 has to go before the step 3. Otherwise write #3 could go to disk and fail part way through before write #2 |
933 |
933 and leave the TOC corrupt with no recovery in the file header. |
|
934 |
|
935 Consider the case that step 2 was omitted, so the Store relies on step 3 being completed in order to know that S1 is in location B1; |
|
936 and that no flush is done after step 3. In step 4 the stream S2 is moved – at this point the old space for stream S1 at A1 is considered empty |
|
937 – and suppose it gets moved from A2 to B2 where B2 overlaps/overwrites A1. If the writes in step 3 and step 4 are re-ordered and the step 3 |
|
938 write does not happen – then the TOC will claim that S1 is still at A1 but this location in the file has been overwritten with data from S2. |
|
939 A corrupted file. |
|
940 |
|
941 Based on the knowledge above, it is strongly recommended to set EFileWriteDirectIO bit when opening the file so that the order is maintained |
|
942 when writing to the file. |
934 */ |
943 */ |
|
944 void CPermanentStoreCollector::RelocateStreamL(const CPermanentStoreCollector::TEntry& aReloc, TInt aExtent) |
935 { |
945 { |
936 if (Coord().Accessed()) // must have exclusive access to relocate the stream |
946 if (Coord().Accessed()) // must have exclusive access to relocate the stream |
937 __LEAVE(KErrInUse); |
947 __LEAVE(KErrInUse); |
938 // |
948 // |
939 TInt end=RelocateL(aReloc.entry.ref,aReloc.len,aReloc.entry.handle == KHandleTocBase ? EFrameDescriptive16 : EFrameData16, aExtent); |
949 TInt end=RelocateL(aReloc.entry.ref,aReloc.len,aReloc.entry.handle == KHandleTocBase ? EFrameDescriptive16 : EFrameData16, aExtent); |