diff -r 80ef3a206772 -r 48780e181b38 Symbian3/SDK/Source/GUID-CCE5DBCC-41D6-53D0-B929-ADB478B53F12.dita --- a/Symbian3/SDK/Source/GUID-CCE5DBCC-41D6-53D0-B929-ADB478B53F12.dita Fri Jul 16 17:23:46 2010 +0100 +++ b/Symbian3/SDK/Source/GUID-CCE5DBCC-41D6-53D0-B929-ADB478B53F12.dita Tue Jul 20 12:00:49 2010 +0100 @@ -30,7 +30,7 @@ MAKEDEF WARNING: 1 export(s) not yet Frozen in \src\example\MyDLLU.DEF: ..\..\..\EPOC32\BUILD\src\example\group\MyDLL\ARMV5\MyDll{000a0000}.def(7) : _ZThn12_N7Derived3fooEv @6
This shows a problem with a frozen DEF file: the export at ordinal 2 is missing, and a new unfrozen export has been added at ordinal 6. When comparing the two symbols, they look suspiciously similar to each other, and to a third symbol in the DEF file:
The exports beginning with _ZTh are compiler generated functions called thunks (see
These generated functions are a feature of the Itanium C++ ABI, on which the ABI for the ARM architecture builds upon. Hence the name "ARM ABI Thunk Offset Problem".
What causes this problem?
The problem is caused because the symbol name generated for the thunk contains an offset number. More details can be found in
Another condition to trigger the problem needs to hold as well: multiple inheritance with virtual functions in more than one of the base classes. If this condition does not hold, the compiler will not generate a thunk and thus there is no problem.
Note that this is always a Binary Compatibility break, which shows up as a change to symbols in DEF files.
How do I fix it?
There are three choices to fix it:
The first option is to refreeze the DEF file: this would be OK if you are not maintaining a frozen interface, and your customers will in any case need to rebuild because of the Binary Compatibility break. The easiest way to refreeze is to delete all of the exports from your existing DEF file, build again, and then use "abld freeze armv5 " to update the DEF file. After updating the DEF file, build again: this time it should build cleanly.
The second option is to use the attached script to fix the ABI Thunk Offsets. It expects to read a build log containing the MAKEDEF errors and warnings and will modify the DEF file to replace each missing export with the corresponding unfrozen export. Run the script with no arguments to get further details.
After fixing the DEF file, you will need to rebuild the DLL which uses the DEF file.
The last option is that you could change your mind about adding that extra member data. This will only be an option if it is your change which causes the problem: if your supplier has changed the size of a class that they own and caused this problem, then you are forced to change your DEF file.
If you own the class which has the extra member data, it is worth noting that this change is likely to affect your customers as well. They will have to rebuild because of the BC break. Adding to this they will also see the ABI Thunk Offset Problem if they derive from your class. This includes simple inheritance from a class which shows the problem, if it re-implements any of the virtual functions which require thunks.
When Symbian breaks compatibility in a way likely to cause this problem, the corresponding entry in the Compatibility Break spreadsheet will say "BC+ Break: Rebuild & Check/Fix Def-File EABI Thunk Offsets".
Tell me the full details: What is a thunk? What causes its offset to change?
In a C++ class hierarchy involving both virtual functions and multiple inheritance, objects can be accessed as though they were several different types. A typical Symbian platform example would be a CBase-derived class which also derives from an M-class, perhaps to provide an observer interface: for example CCoeControl, which derives from both CBase and MObjectProvider.
The virtual functions which can be called on an object depends on the type it currently appears to be.A CCoeControl object can be viewed as a CBase object, in which case it has one set of virtual functions, or as an MObjectProvider, in which case it has another. The compiler constructs separate virtual tables for each of the possible interfaces, and these tables contain information about how to convert back to the underlying CCoeControl object. When converting from a CCoeControl pointer to an MObjectProvider pointer, the compiler will adjust the value of the pointer, so that it points to the "MObjectProvider" part of the object, and not the full CCoeControl object.
The MObjectProvider class defines a pure virtual function MopSupplyObject, which is implemented in CCoeControl. Even when the object is presenting it's MObjectProvider interface, the vtable must use the correct implementation of MopSupplyObject, which expects to be used in the context of a CCoeControl. The solution used by the compiler is to create a virtual function override thunk function which makes any necessary adjustments between the calling context (a pointer to MObjectProvider) and the execution context (a pointer to CCoeControl).
This could have been implemented using the names of the two contexts, but instead the ABI uses the amount by which the this pointer needs to be adjusted to make the switch: this is the offset encoded in the name.
Here is a small example:
eabi_thunk_offset_problem.cpp
The exports beginning with _ZTh are compiler generated functions called thunks (see
These generated functions are a feature of the Itanium C++ ABI, on which the ABI for the ARM architecture builds upon. Hence the name "ARM ABI Thunk Offset Problem".
What causes this problem?
The problem is caused because the symbol name generated for the thunk contains an offset number. More details can be found in
Another condition to trigger the problem needs to hold as well: multiple inheritance with virtual functions in more than one of the base classes. If this condition does not hold, the compiler will not generate a thunk and thus there is no problem.
Note that this is always a Binary Compatibility break, which shows up as a change to symbols in DEF files.
How do I fix it?
There are three choices to fix it:
The first option is to refreeze the DEF file: this would be OK if you are not maintaining a frozen interface, and your customers will in any case need to rebuild because of the Binary Compatibility break. The easiest way to refreeze is to delete all of the exports from your existing DEF file, build again, and then use "abld freeze armv5 " to update the DEF file. After updating the DEF file, build again: this time it should build cleanly.
The second option is to use the attached script to fix the ABI Thunk Offsets. It expects to read a build log containing the MAKEDEF errors and warnings and will modify the DEF file to replace each missing export with the corresponding unfrozen export. Run the script with no arguments to get further details.
After fixing the DEF file, you will need to rebuild the DLL which uses the DEF file.
The last option is that you could change your mind about adding that extra member data. This will only be an option if it is your change which causes the problem: if your supplier has changed the size of a class that they own and caused this problem, then you are forced to change your DEF file.
If you own the class which has the extra member data, it is worth noting that this change is likely to affect your customers as well. They will have to rebuild because of the BC break. Adding to this they will also see the ABI Thunk Offset Problem if they derive from your class. This includes simple inheritance from a class which shows the problem, if it re-implements any of the virtual functions which require thunks.
When Symbian breaks compatibility in a way likely to cause this problem, the corresponding entry in the Compatibility Break spreadsheet will say "BC+ Break: Rebuild & Check/Fix Def-File EABI Thunk Offsets".
Tell me the full details: What is a thunk? What causes its offset to change?
In a C++ class hierarchy involving both virtual functions and multiple inheritance, objects can be accessed as though they were several different types. A typical Symbian platform example would be a CBase-derived class which also derives from an M-class, perhaps to provide an observer interface: for example CCoeControl, which derives from both CBase and MObjectProvider.
The virtual functions which can be called on an object depends on the type it currently appears to be.A CCoeControl object can be viewed as a CBase object, in which case it has one set of virtual functions, or as an MObjectProvider, in which case it has another. The compiler constructs separate virtual tables for each of the possible interfaces, and these tables contain information about how to convert back to the underlying CCoeControl object. When converting from a CCoeControl pointer to an MObjectProvider pointer, the compiler will adjust the value of the pointer, so that it points to the "MObjectProvider" part of the object, and not the full CCoeControl object.
The MObjectProvider class defines a pure virtual function MopSupplyObject, which is implemented in CCoeControl. Even when the object is presenting it's MObjectProvider interface, the vtable must use the correct implementation of MopSupplyObject, which expects to be used in the context of a CCoeControl. The solution used by the compiler is to create a virtual function override thunk function which makes any necessary adjustments between the calling context (a pointer to MObjectProvider) and the execution context (a pointer to CCoeControl).
This could have been implemented using the names of the two contexts, but instead the ABI uses the amount by which the this pointer needs to be adjusted to make the switch: this is the offset encoded in the name.
Here is a small example:
eabi_thunk_offset_problem.cpp
Compile this with armcc -S eabi_thunk_offset_problem.cpp to get an assembly listing. Compile it again with an extra argument "-DCOUNT=2" to change the size of the base class, and compare the two files: there will be various differences in the code, but also differences in the _ZTh symbols - including the differences used in the "typical symptom" above.
If you use virtual inheritance, then you may see another version of the problem. With virtual inheritance, there are two offsets involved and the thunk symbols will begin with _ZTv. The same symbol may appear in several thunks, each with different offsets.
What is the Problem with Shared DEF Files?
The class exporting rules (see
Use-cases for Fixing Shared DEF Files
Use-Case 1: Polymorphic “Plug-ins”
Several DLLs are built using the same DLL interface (DEF file). Typically the DEF file has very few entries (1 or 2) and is maintained manually. This means that new functions are added by editing the shared DEF file. Also typically no import libraries are needed as the knowledge about the DLL interface is hard-coded into the client code of the "plug-in". The plug-ins do not have to be loaded at run-time. Some are always built but not always included in the ROM.
The Fix:
If the shared DEF file is in \epoc32\include\def\EABI then locate the original DEF file by searching all BLD.INF files for the appropriate line in PRJ_EXPORTS
Remove all non-callable exports that have caused warnings or errors from the original DEF file.
Add NOEXPORTLIBRARY to all MMP files that share that component, ensuring that the build system does NOT try and re-freeze these automatically the next time you build. Otherwise the build system will re-introduce these non-callable exports.
Note:
If you want to use the re-freeze mechanism – say to add a new export, you have to temporarily remove NOEXPORTLIBRARY from the MMP file, then generate a new DEF file by re-building the component, re-freeze, possibly edit (to remove unwanted non-callable exports) and then insert the keyword NOEXPORTLIBRARY into the MMP file again.
Use-Case 2: Polymorphic “Plug-ins” on which Other Components Depend
This is very similar to use-case 1, except that some other component depends on one of the plug-ins. This means that an import library is required.
The Fix: The build structure must be such that
One MMP file generates the import library from the shared DEF file using the target type IMPLIB. It may be necessary to create a new MMP file which does this.
All the other MMP files use NOEXPORTLIBRARY as described in use-case 1
Note:
If you want to use the re-freeze mechanism – say to add a new export, you have to temporarily remove NOEXPORTLIBRARY from the MMP file, then generate a new DEF file by re-building the component, re-freeze, possibly edit (to remove unwanted non-callable exports) and then insert the keyword NOEXPORTLIBRARY into the MMP file again.
Use-Case 3: Annotate Classes as Non-sharable
Where a DEF file must be shared between components for whatever reason and none of the above use-cases can be applied, the build would fail for at least one component. An example of this may be a class MyPrivateClass that exists in the debug build (UDEV) of the OS, but not in the release build (UREL).
In such a case all classes that should not contribute to the DEF file, i.e. that are really private to the implementation of a component, must be annotated in the source as NONSHARABLE_CLASS(X) or NONSHARABLE_STRUCT(X). As a result no non-callable exports will be generated for such a class. Say for example, class MyPrivateClass is truly private to a component that must share a DEF file with another component. Then it should be declared:
Compile this with armcc -S eabi_thunk_offset_problem.cpp to get an assembly listing. Compile it again with an extra argument "-DCOUNT=2" to change the size of the base class, and compare the two files: there will be various differences in the code, but also differences in the _ZTh symbols - including the differences used in the "typical symptom" above.
If you use virtual inheritance, then you may see another version of the problem. With virtual inheritance, there are two offsets involved and the thunk symbols will begin with _ZTv. The same symbol may appear in several thunks, each with different offsets.
What is the Problem with Shared DEF Files?
The class exporting rules (see
Use-cases for Fixing Shared DEF Files
Use-Case 1: Polymorphic “Plug-ins”
Several DLLs are built using the same DLL interface (DEF file). Typically the DEF file has very few entries (1 or 2) and is maintained manually. This means that new functions are added by editing the shared DEF file. Also typically no import libraries are needed as the knowledge about the DLL interface is hard-coded into the client code of the "plug-in". The plug-ins do not have to be loaded at run-time. Some are always built but not always included in the ROM.
The Fix:
If the shared DEF file is in \epoc32\include\def\EABI then locate the original DEF file by searching all BLD.INF files for the appropriate line in PRJ_EXPORTS
Remove all non-callable exports that have caused warnings or errors from the original DEF file.
Add NOEXPORTLIBRARY to all MMP files that share that component, ensuring that the build system does NOT try and re-freeze these automatically the next time you build. Otherwise the build system will re-introduce these non-callable exports.
Note:
If you want to use the re-freeze mechanism – say to add a new export, you have to temporarily remove NOEXPORTLIBRARY from the MMP file, then generate a new DEF file by re-building the component, re-freeze, possibly edit (to remove unwanted non-callable exports) and then insert the keyword NOEXPORTLIBRARY into the MMP file again.
Use-Case 2: Polymorphic “Plug-ins” on which Other Components Depend
This is very similar to use-case 1, except that some other component depends on one of the plug-ins. This means that an import library is required.
The Fix: The build structure must be such that
One MMP file generates the import library from the shared DEF file using the target type IMPLIB. It may be necessary to create a new MMP file which does this.
All the other MMP files use NOEXPORTLIBRARY as described in use-case 1
Note:
If you want to use the re-freeze mechanism – say to add a new export, you have to temporarily remove NOEXPORTLIBRARY from the MMP file, then generate a new DEF file by re-building the component, re-freeze, possibly edit (to remove unwanted non-callable exports) and then insert the keyword NOEXPORTLIBRARY into the MMP file again.
Use-Case 3: Annotate Classes as Non-sharable
Where a DEF file must be shared between components for whatever reason and none of the above use-cases can be applied, the build would fail for at least one component. An example of this may be a class MyPrivateClass that exists in the debug build (UDEV) of the OS, but not in the release build (UREL).
In such a case all classes that should not contribute to the DEF file, i.e. that are really private to the implementation of a component, must be annotated in the source as NONSHARABLE_CLASS(X) or NONSHARABLE_STRUCT(X). As a result no non-callable exports will be generated for such a class. Say for example, class MyPrivateClass is truly private to a component that must share a DEF file with another component. Then it should be declared:
This will prevent the compiler from exporting non-callables for MyPrivateClass. However this means that it is not possible to DLL-derive (for the definition of DLL-derive see
Use-Case 4: Optimisation
A consequence of the Simple Rule (see
For code that is private to an implementation, i.e. where it is known that a class would never be used outside of that component, this footprint increase is unnecessary. In order to avoid the footprint increase mark all private classes (and classes derived from them) in the source as NONSHARABLE_CLASS(X) or NONSHARABLE_STRUCT(X) as described for use-case 3.
Use-Case 5: The Build Tools Automatically Ignore Non-callable Exports
For most components the build tools automatically ignore all non-callable exports. This is the case because the build tools know the situations when non-callable exports cannot be needed. Non-callable exports are only needed, if:
The target type is either DLL, EXEDLL or EXEXP and the MMP file has no NOEXPORTLIBRARY keyword
If the MMP file contains the DEFFILE keyword and the MMP file has no NOEXPORTLIBRARY keyword
The reason for this is that target types, such as APP, always map directly onto one of the above use-cases. For example the target type APP is an example of use-case 1, i.e. the APP is a DLL that always has the same DEF file. However no other DLL but the APP loader will ever link against this binary, unless its MMP file contains the DEFFILE keyword.
Use-Case 6: Best Practice
Note that it is good practice to avoid unnecessary footprint increases by marking private classes as non-sharable as outlined in use-case 4. Further note, that at some point in the future Symbian may add this to the Symbian Coding Standards or withdraw tools support for some of the cases described above.
Optimisation
This section discusses advantages of marking “private” classes as described in use-cases 4 and 6 in the previous section as non-sharable
Reasons for Optimisation:
Small footprint saving
DEF files are more “pretty”, i.e. they will have fewer entries for non-callable exports in it and may have fewer holes in them.
When changing code that is private to a module as described in use-case 4 and not marked non-sharable, DEF file changes are required when:
Renaming a private class
Removing a private class
Adding a private class
This makes it harder to maintain DEF files and will ultimately lead to less “pretty” DEF files when Binary Compatibility must be maintained.
More non-sharable classes mean that it is less likely to have problems with shared DEF files as outlined in use-cases 1 and 2 in the previous section.
When not to use Optimisation:
It is not advisable to use optimisation in the following circumstances, as the build tools suppress non-callable exports automatically in these cases:
The target type of the component containing private classes is neither DLL, EXEDLL nor EXEXP and no DEFFILE keyword is contained the components MMP file
The MMP file of the component contains the NOEXPORTLIBRARY keyword.
If the NOEXPORTLIBRARY keyword has been introduced to work around problems introduced by shared DEF files and the Simple Rule it may be better to remove the NOEXPORTLIBRARY keyword and mark private classes as non-sharable instead.