Surface Update Component Overview

Purpose

The Surface Update component provides a communication channel interface for use by components that render graphics content to composition surfaces. Examples of such components are EGL, the video renderer (DevVideo), the camera viewfinder (ECam) and render stages. This is illustrated in the following diagram.

The client renders the graphics content to a buffer within the surface and then sends a request for composition. The request includes the surface ID, the ID of the current buffer within the surface and optionally the following:

• The number of the screen on which the surface is to be displayed. Alternatively, the KAllScreens constant can be used to indicate that the request should be broadcast to all screens. This is called a global surface update and its use is encouraged. It makes the client code simpler because it does not need to be concerned with which screen a surface is currently visible on. Note that you must not mix global updates with updates to specific screens within a single session.

• The region of the surface that has changed. This is sometimes referred to as the dirty rectangle. If the client does not specify this, the whole surface is recomposed.

The client can request notification of the following composition-related events:

• The buffer has become available for rendering.

• The buffer has been displayed for the first time after the update was submitted. This notification is accompanied by timestamp information.

• The buffer has been displayed a certain number of times. This can be used to ensure that each frame gets displayed for at least a given period of time.

Architecture

The Surface Update component is an adaptation component, which means that it can be replaced by device creators to suit the exact hardware. The reference implementation consists of the Surface Update Server and its client-side API. The Surface Update Server runs within the Window Server process along with the composition engine. The Surface Update Server uses the standard Symbian client/server model for Inter Process Communication (IPC) to communicate with clients and Inter Thread Communication (using active objects and semaphores) to communicate with the composition engine.

The client sends requests for composition through the command channel and the composition engine sends notification messages through the notification channel.

The following diagram shows communication between the client, Surface Update server and composition engine. The composition engine is shown as a single component, although it can be implemented as multiple components. For example, the OpenWF-C implementation consists of the OpenWF-C Support component and the OpenWF-C Engine component. The Surface Update Server in fact communicates with the OpenWF-C Support component, which in turn communicates with the OpenWF-C Engine.

Some key points to note include:

• The composition engine is a device-specific adaptation that may delegate some of its functionality (such as composing hardware-rendered surfaces) to a GPU or display controller.

• There is a composition engine instance for each internal screen and external screen connection point on the device. Each composition engine must be registered with the Surface Update Server at startup.

• Each composition engine instance has a unique 32-bit priority number, which represents the relative priority of its associated screen. The higher the number, the higher the priority of the screen. The composition engine instance passes this to the Surface Update Server at registration. The Surface Update Server adds the priority value to its priority list. After it is set up, the priority list is fixed and does not change when a screen becomes unavailable—for example, because it is put on standby or an external screen is disconnected.

• Each composition engine instance incorporates its own thread within the Window Server process. However, the external interface is accessed directly from the thread that makes the call. This means that MCompositionSurfaceUpdate::ContentUpdated() is called from the Surface Update Server thread.

• The Window Server's main thread is a client of the Surface Update Server. Although it is running in the same process as the Surface Update Server, it uses RSurfaceUpdateSession in exactly the same way as a client in a separate process.

The following diagram provides a more detailed view of the architecture. Notice that there is a composition engine thread for each screen.

Do not confuse the Surface Update Server with the Surface Manager. The Surface Manager creates, deletes and manages access to the surface data, whereas the Surface Update Server is concerned with communicating the fact that surface data has changed.

The Surface Update client API

Here is a simplified RSurfaceUpdateSession declaration showing only the key functions.

class RSurfaceUpdateSession : public RSessionBase
    {
public:
    IMPORT_C RSurfaceUpdateSession();

    IMPORT_C TInt Connect(TInt aMessageSlots = KDefaultMessageSlot);

    IMPORT_C void NotifyWhenAvailable(TRequestStatus& aStatus); 
    
    IMPORT_C void NotifyWhenDisplayed(TRequestStatus& aStatus, TTimeStamp& aTimeStamp);
    
    IMPORT_C void NotifyWhenDisplayedXTimes(TInt aCount, TRequestStatus&  aStatus);
   
    IMPORT_C void CancelAllUpdateNotifications();
        
    IMPORT_C TInt SubmitUpdate(TInt aScreen, const TSurfaceId& aSurfaceId, TInt aBuffer, 
                        const TRegion* aDirtyRegion = NULL);
};

To use the API:

1. Connect to the Surface Update Server.

2. Set up one or more notification requests.

   This part of the process is optional. You need to do it only if the client wants to know when the composition engine has used the data on the surface. To set up a request, call one of the NotifyWhen...() functions.

3. Send notification that your surface data has been updated.

   Call the SubmitUpdate() function to send a message to the Surface Update Server that there is new data on the specified surface. The message contains the surface ID, the buffer number and the number of the screen on which the surface is displayed or a constant that indicates that the request should be broadcast to all screens (this is called a global surface update). The message can also specify the region of the surface that has changed. This allows the composition engine to ignore parts that have not changed. Before it sends the update, the function sends all of the notification requests that the client has set up.

4. Wait for notifications from the Surface Update Server.

   If the client requests notification(s), it must wait (User::WaitForRequest()) for the notification to arrive. However, clients normally use active objects and the Active Scheduler to handle the wait rather than calling User::WaitForRequest() directly.

5. Close the connection.

   If your client is still waiting for a notification you must call CancelAllUpdateNotifiations() before closing the connection, otherwise the client thread panics.

Example

Here is a snippet of test code which illustrates the process of setting up notification requests before submitting an update. This piece of code results in four messages being sent to the Surface Update Server in SubmitUpdate():

TRequestStatus statusAvailable = KRequestPending ;
TRequestStatus statusDisplayed = KRequestPending ;
TRequestStatus statusDisplayedXTimes = KRequestPending ;

TTimeStamp timeStamp ; 
    
session.NotifyWhenAvailable( statusAvailable ) ;    
session.NotifyWhenDisplayed( statusDisplayed, timeStamp ) ;    
session.NotifyWhenDisplayedXTimes( 10, statusDisplayedXTimes ) ;    

err = session.SubmitUpdate( screen, surface, buffer, &region ) ;
        
User::WaitForRequest( statusAvailable ) ;
User::WaitForRequest( statusDisplayed ) ;
User::WaitForRequest( statusDisplayedXTimes ) ;

Notes:

• Because this is test code and to avoid complexity, this example calls User::WaitForRequest() directly rather than using active objects.

• In some hardware configurations where composition and display are fast, the buffer available and first displayed notifications may occur very close together for single-buffered surfaces. You should then use only one of these notifications at a time.

Sequence diagrams

These sequence diagrams primarily illustrate the protocol linking the sending of the buffers and notification. For simplicity some detail is omitted. For example, the client typically renders to the buffer before sending it to the composition engine. This is not shown. Similarly the diagrams omit detail from the composition engine and some omit the Surface Update Server altogether. In addition, they assume that a specific screen number is specified. When global surface updates are in use, the sequence is more complex and is described in Global Surface Updates.

Surface buffer availability: single-buffered surface

When using a single-buffered surface, the client typically does the following:

1. Render the graphics content to the buffer.

2. Call RSurfaceUpdateSession::NotifyWhenAvailable().

3. Call RSurfaceUpdateSession::SubmitUpdate().

4. Wait for notification that the buffer is available before rendering further content to it and repeating the cycle for as long as necessary.

This is shown in the following diagram.

Notice that the notification signal is issued to the client immediately after the buffer has been consumed by the composition engine. When single buffers are used, tearing artefacts are always a risk. Therefore double-buffered surfaces are often used.

Surface buffer availability: double-buffered surface

When double-buffering is used, the client renders to one buffer (called A in this example) while the other buffer (B) is on the screen and vice versa. In order for this to work correctly and to be free of tearing artefacts, the client must use the following sequence:

1. Render the graphics content to buffer A, and call NotifyWhenAvailable() followed by SubmitUpdate() for buffer A.

2. Render the graphics content to buffer B, and call NotifyWhenAvailable() followed by SubmitUpdate() for buffer B.

3. Wait for notification that buffer A is available. When it becomes available, render further content to it and call NotifyWhenAvailable() followed by SubmitUpdate() for buffer A.

4. Wait for notification that buffer B is available. When it becomes available, render further content to it and call NotifyWhenAvailable() followed by SubmitUpdate() for buffer B.

5. Repeat steps 3 and 4 for as long as required.

This is shown in the next diagram. After sending the first two buffers to the composition engine, the client waits for notification before sending a further buffer. The composition engine always returns notification after receiving a new buffer even if an error condition is detected.

Notes:

• The buffer that is on the screen is called the front buffer and the one that is being rendered into is called the back buffer.

• Although double-buffering is shown, three or more buffers can be used.

Frame update

The following diagram shows the sequence when a client submits a request to be notified when the buffer has been displayed three times. However, the exact details depend on how the composition engine is implemented. If the composition engine knows the screen refresh rate, it can post the composed buffer to the Display Driver and wait for notification that the buffer is on the screen. It could then uses a timer to wait for three frames to be displayed, before it sends the notification.

If the client sends a new request for update while the previous one is still in progress, the previous request is cancelled with the KErrOverflow error code. This is illustrated in the next diagram.

Cancelling of all outstanding requests

If the client is waiting for a notification when you need to remove the active object that is handling the notification and close the thread, you must call CancelAllUpdateNotifiations() first—otherwise the client thread panics.

However, CancelAllUpdateNotifiations() only cancels the notifications and not the associated command request. The notifications complete immediate with the KErrCancel error code.