diff -r 43e37759235e -r 51a74ef9ed63 Symbian3/SDK/Source/GUID-859CAA08-59C9-5FD3-98DE-6BDD0D6ED50B.dita --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Symbian3/SDK/Source/GUID-859CAA08-59C9-5FD3-98DE-6BDD0D6ED50B.dita Wed Mar 31 11:11:55 2010 +0100 @@ -0,0 +1,150 @@ + + + + + +Graphics +Composition +

Composition is the process of putting together the output elements from +various different sources to create the screen display that the end user sees +on the device.

+

Variant: ScreenPlay.

+

In a multi-tasking device many of the activities taking place simultaneously +generate output for display on the screen. Their output can include words, +pictures, video, games and the screen furniture (scroll bars, buttons, icons, +borders, tabs, menus, title bars) familiar to every computer user.

+

Many of these output elements can appear at the same time, either next +to each other or overlapping each other. They can be opaque, such that they +obscure anything behind, or semi-transparent such that the elements underneath +are partially visible.

+

The diagram below illustrates how the display that the viewer sees (looking +down from the top) is a two-dimensional representation composed from a series +of layers.

+ + The display is an orthogonal view of a series of layers. + + + +

Composition requires:

+ +

For this reason the two are handled separately.

+

UI Composition

+

UI Composition is performed by the Window Server. Each application has +its own window group containing all of its child windows. The Window Server +keeps track of the windows' positions, sizes, visibilities, transparencies +and z-order and is able to establish which windows, and which bits of each +window, are visible. It ensures that each visible bit of window is kept up-to-date +by calling its application when necessary.

+

Image composition

+

Prior to the introduction of ScreenPlay the Window Server did rudimentary +composition in its main thread and rendered composited output to the screen +buffer using the GDI (Graphics Device Interface). To achieve high frame rates, +applications typically bypassed the Window Server using Direct Screen Access +(DSA) and animation DLLs.

+

In ScreenPlay sources +that generate complex graphical output render directly to graphics surfaces. +The Window Server delegates the composition of surfaces to a composition engine +that device creators can adapt to take advantage of graphics processing hardware.

+

Although the composition engine runs in the Window Server's process, it +is largely transparent to application developers and works quite differently. +Instead of using windows, it uses surfaces to store pixel data and elements +to manipulate size, position, z-order, visibility and transparency.

+

While an element is a simple lightweight object, and easy to manipulate, +a surface stores a large amount of data and its handling requires more consideration.

+

The diagram below is a simplistic representation of how applications create +output which is rendered, composited and displayed.

+ + Graphics Composition + + +

In a device with graphics acceleration hardware (a Graphics Processing +Unit or GPU) there might, in addition to the memory managed by the CPU, be +additional memory managed by the GPU. Image data may therefore be considered +to have been software rendered (onto a surface in CPU memory) or hardware +rendered (onto a surface in hardware accelerated GPU memory). The following +diagram shows applications and other graphical data sources rendering to surfaces +in software and hardware.

+ + Hardware rendered graphics - the wrong way! + + +

The diagram, however, represents a system with several problem areas that +would render it unsuitable for any practical implementation

+ +

In ScreenPlay, the UI is therefore composed and rendered onto a single +surface before being composited with any other surfaces. This surface is termed +the UI surface and is displayed on a layer placed in front of all of +the others. In fact, the UI surface is created for the Window Server during +system start up and is then passed to the composition components as a 'special +case' surface for composition.

+
Combining Window Server and Composition

The compositing +of surfaces according to their origin means that the physical composition +process behaves differently from the logical composition process that is based +on what the user and the UI are doing. Logically the windows in the UI and +memory rendered surfaces may be on layers that are interleaved yet the memory +rendered surfaces are physically composed behind.

ScreenPlay addresses +this issue by associating 'external' surfaces with windows in the Window Server +using the RWindowBase::SetBackgroundSurface() method. This +means that the Window Server is able to include them in its logical composition +and make provision for them during data composition. A window with its background +set to a surface in this way becomes transparent in the UI rendered UI surface.

In +most cases this is not apparent to the viewer. Surfaces that are physically +composed behind the UI appear in the correct position on the two-dimensional +display. The diagram below illustrates how Window Server-rendered UI content +and external surfaces are composited using the UI surface.

+ Hardware composition and the Flattened UI + +

Although this method of composition is flexible and powerful, it +does have some limitations, particularly with respect to semi-transparent +hardware-accelerated surfaces.

It is not possible, for example, to +include a semi-transparent surface in front of the UI surface. In such cases +the composition engine determines how the display is composed.

Here +is a second version of the diagram at the top of the page showing how the +same composition might be achieved in practice. The UI menus, windows and +dialogs are composited by the Window Server onto a single surface. The light +green layer displays a hardware rendered surface so it is actually behind +the layer on which it appears.

+ The UI surface + +
+
Composition +Examples

This illustrations below illustrate the use of hardware +accelerated surfaces and the UI surface

+ Video rendered to a hardware accelerated surface mapped +to a layer behind the UI surface + + + As above with a semi-transparent dialog on the UI surface + + +
+
+The ScreenPlay +Architecture +Hardware +Acceleration + +
\ No newline at end of file