The -ScreenPlay Graphics Architecture

The -ScreenPlay Graphics ArchitectureThis topic provides an introduction to ScreenPlay and its architecture. -ScreenPlay is a new graphics architecture, introduced in Symbian^3 (S^3). -ScreenPlay enables device creators to take advantage of improved software -performance, hardware acceleration and third party graphics engines. ScreenPlay -is sometimes known as the New Graphics Architecture (NGA). -

ScreenPlay is a response to new requirements and developments in device -hardware models. For example, ScreenPlay can support graphics accelerators -and graphics processing units (GPUs) and non-uniform memory models as well -as uniform memory models. A non-uniform memory model is an architecture in -which a GPU has a completely different processing area from the CPU such that -the GPU memory is not available to the CPU and vice versa. ScreenPlay can -handle non-uniform memory models without the need to copy buffers between -the different processing areas.

Key features

Asynchronous hardware-accelerated -rendering and composition on devices on which dedicated graphics acceleration -hardware is available. This is achieved through a Hardware Adaptation Layer -(HAL).
The ability to composit -a semi-transparent UI buffer over highly dynamic content, such as OpenGL ES -games, video and the camera viewfinder.
The separation of control -and data flow. This has advantages when running on non-uniform memory architectures -and means that video decoding, UI rendering, and so on can take place and -remain in the GPU memory domain.
A foundation for secure -screen content and Digital Rights Management (DRM). Applications no longer -have direct access to the screen. Read and write access to the screen is controlled -by the Window Server.
Direct Screen Access -(DSA) is supported in order to provide backwards compatibility. However, because -in ScreenPlay the screen is no longer controlled by the Screen Driver, the -DSA frame buffer is just another buffer that can be allocated dynamically -on demand. ScreenPlay provides alternatives to DSA.

Architecture

The -following diagram shows the key components in the Symbian Foundation Graphics -package and some closely related components in other packages.

- Symbian^3 component architecture - -

The key ScreenPlay components are introduced below under separate -subheadings.

Graphics Composition

The -composition engine composes content, possibly from several different sources, -before it is displayed on the screen. Composition involves the important concepts -of scene elements (or layers) and surfaces. Scene elements describe the geometric -position, size and orientation of items to be displayed on the screen; whereas -surfaces are pixel buffers for holding an image or part of a scene.

The -composition engine maintains the stack of scene elements and computes what -is visible. For example, it culls invisible areas and maintains a list of -dirty rectangles. It blends the pixels if necessary and can perform limited -transformations, such as scaling and rotation (in 90° increments). The composition -engine is an adaptation component, which means that device creators -can adapt or replace it to suit the exact hardware on the device. The composition -engine can utilize GPU hardware composition and LCD hardware rotation if they -are available.

The composition components are specific to ScreenPlay. -For more information, see Graphics -Composition.

Surface Manager -and Surface Update Server

The Surface Manager component creates -and manages graphics composition surfaces. The Surface Manager reference implementation -implements surfaces as shared chunks because they must be accessible by user-side -processes and the kernel and composition hardware. Surfaces can be multi-buffered -and are identified by a 128 bit identifier (called the surface ID). This gets -resolved to an actual memory address by calling the Surface Manager Map Surface -API. Surfaces can be used by other Symbian components such as the Multimedia -Framework (MMF) and ECam -viewfinder and by applications such as OpenGL ES games. The Surface -Manager is an adaptation component and so can be adapted or replaced to suit -the hardware.

The Surface Update component provides a communication -mechanism between the composition engine and clients. This is particularly -useful for clients (such as video) that produce fast updates and use multi-buffered -surfaces.

Window Server

The -Window Server has been extended with a render stage framework, which enables -the last stage of the Window Server rendering to be customizable through render -stage plug-ins. This process, known as "deferred rendering" is achieved by -intercepting the output of the Window Server and then deciding how that output -should be rendered. For example, the output can be hardware accelerated or -it can be sent to a third party graphics engine. The render stage framework -enables device creators to integrate different UIs and runtime environments -(such as Flash or Silverlight) and to achieve transition effects such as slide, -zoom and fade.

Symbian provides more than one render stage solution. -The following diagram provides a simplified representation of one possible -solution (called solution A in this topic). This solution is full featured. -The diagram focuses on the more relevant components and does not attempt to -show all components in the complete solution. This solution has a dependency -on the S60 middleware layer, in particular on the Hitchcock component (which -is in the UI Accelerator package).

- Render stage solution A - -

Another possible solution (called solution B) is based on -the DirectGDI and Graphics Resource components (which are described next), -both of which have interface and adaptation layers. This solution is not full -featured. Like the previous diagram, this diagram focuses on the more relevant -components and does not attempt to show everything.

-Render stage solution B - -

Both of these render stage solutions mean that existing Window Server -applications can take advantage of hardware acceleration if it is available -(and therefore run faster) without recompiling the code.

ScreenPlay -provides extensions to the Window Server client-side API, which enable mobile -devices to respond to events from a number of pointers, including their proximity -and pressure. This feature is known as advanced -pointers.

A new API, RWsSession::Finish(), -has been added to allow Window Server client applications to synchronize with -the completion of Window Server rendering. The existing API, RWsSession::Flush(), -is redefined to simply flush the client-side command buffer, whereas previously -it also provided a guarantee that Window Server had completed the command -buffer’s operations. This behavioral change allows legacy clients to benefit -from the asynchronous hardware rendering when supported by the render stage -plug-in(s) that are in use.

For more information, see Windowing -Collection.

DirectGDI

DirectGDI -provides a graphics context that can be hardware accelerated and allows an -asynchronous interface. DirectGDI has two parts: a generic layer, which provides -a client API and an adaptation layer. Device creators can replace the adaptation -layer with an implementation that takes advantage of graphics accelerated -hardware, if it is available, or a software implementation, if it is not available.

DirectGDI -was introduced as a prototype in the development of ScreenPlay. It is deprecated -in Symbian^3.

Graphics Resource

The -Graphics Resource component provides an abstraction layer for the memory management -of pixel and non-pixel data (such as OpenVG command lists). Like DirectGDI, -it has a generic part, which provides a client API and an adaptation part, -which device creators can adapt to take advantage of graphics hardware when -it is available.

The Graphics Resource component was introduced as -a prototype in the development of ScreenPlay. It is deprecated in Symbian^3 -and will be removed in Symbian^4. However, a new Graphics Resource Interface -component is planned for S^4. This new component will provide a similar but -reduced API that is optimized for sharing images across processes.

OpenVG, OpenGL -ES and EGL

Symbian provides support for OpenVG, OpenGL ES and EGL. -The main advantage of ScreenPlay with regard to EGL is that EGL can render -into composition surfaces. For application developers this offers the ability -to have semi-transparent GDI content on top of EGL content. The EGL client -can query whether these new features are supported on the particular device.

For -more information, see Khronos -API Support.

Screen Driver

In -ScreenPlay, the implementation of the Screen Driver has been changed so that -DSA content can be passed into the composition engine.

-Graphics -Composition -Graphics -Hardware Acceleration -Graphics -Concepts - - - - + + + + + +The +ScreenPlay Graphics ArchitectureThis topic provides an introduction to ScreenPlay and its architecture. +ScreenPlay is a new graphics architecture, introduced in Symbian^3 (S^3). +ScreenPlay enables device creators to take advantage of improved software +performance, hardware acceleration and third party graphics engines. ScreenPlay +is sometimes known as the New Graphics Architecture (NGA). +

ScreenPlay is a response to new requirements and developments in device +hardware models. For example, ScreenPlay can support graphics accelerators +and graphics processing units (GPUs) and non-uniform memory models as well +as uniform memory models. A non-uniform memory model is an architecture in +which a GPU has a completely different processing area from the CPU such that +the GPU memory is not available to the CPU and vice versa. ScreenPlay can +handle non-uniform memory models without the need to copy buffers between +the different processing areas.

Key features

Asynchronous hardware-accelerated +rendering and composition on devices on which dedicated graphics acceleration +hardware is available. This is achieved through a Hardware Adaptation Layer +(HAL).
The ability to composit +a semi-transparent UI buffer over highly dynamic content, such as OpenGL ES +games, video and the camera viewfinder.
The separation of control +and data flow. This has advantages when running on non-uniform memory architectures +and means that video decoding, UI rendering, and so on can take place and +remain in the GPU memory domain.
A foundation for secure +screen content and Digital Rights Management (DRM). Applications no longer +have direct access to the screen. Read and write access to the screen is controlled +by the Window Server.
Direct Screen Access +(DSA) is supported in order to provide backwards compatibility. However, because +in ScreenPlay the screen is no longer controlled by the Screen Driver, the +DSA frame buffer is just another buffer that can be allocated dynamically +on demand. ScreenPlay provides alternatives to DSA.

Architecture

The +following diagram shows the key components in the Symbian Foundation Graphics +package and some closely related components in other packages.

+ Symbian^3 component architecture + +

The key ScreenPlay components are introduced below under separate +subheadings.

Graphics Composition

The +composition engine composes content, possibly from several different sources, +before it is displayed on the screen. Composition involves the important concepts +of scene elements (or layers) and surfaces. Scene elements describe the geometric +position, size and orientation of items to be displayed on the screen; whereas +surfaces are pixel buffers for holding an image or part of a scene.

The +composition engine maintains the stack of scene elements and computes what +is visible. For example, it culls invisible areas and maintains a list of +dirty rectangles. It blends the pixels if necessary and can perform limited +transformations, such as scaling and rotation (in 90° increments). The composition +engine is an adaptation component, which means that device creators +can adapt or replace it to suit the exact hardware on the device. The composition +engine can utilize GPU hardware composition and LCD hardware rotation if they +are available.

The composition components are specific to ScreenPlay. +For more information, see Graphics +Composition.

Surface Manager +and Surface Update Server

The Surface Manager component creates +and manages graphics composition surfaces. The Surface Manager reference implementation +implements surfaces as shared chunks because they must be accessible by user-side +processes and the kernel and composition hardware. Surfaces can be multi-buffered +and are identified by a 128 bit identifier (called the surface ID). This gets +resolved to an actual memory address by calling the Surface Manager Map Surface +API. Surfaces can be used by other Symbian components such as the Multimedia +Framework (MMF) and ECam +viewfinder and by applications such as OpenGL ES games. The Surface +Manager is an adaptation component and so can be adapted or replaced to suit +the hardware.

The Surface Update component provides a communication +mechanism between the composition engine and clients. This is particularly +useful for clients (such as video) that produce fast updates and use multi-buffered +surfaces.

Window Server

The +Window Server has been extended with a render stage framework, which enables +the last stage of the Window Server rendering to be customizable through render +stage plug-ins. This process, known as "deferred rendering" is achieved by +intercepting the output of the Window Server and then deciding how that output +should be rendered. For example, the output can be hardware accelerated or +it can be sent to a third party graphics engine. The render stage framework +enables device creators to integrate different UIs and runtime environments +(such as Flash or Silverlight) and to achieve transition effects such as slide, +zoom and fade.

Symbian provides more than one render stage solution. +The following diagram provides a simplified representation of one possible +solution (called solution A in this topic). This solution is full featured. +The diagram focuses on the more relevant components and does not attempt to +show all components in the complete solution. This solution has a dependency +on the S60 middleware layer, in particular on the Hitchcock component (which +is in the UI Accelerator package).

+ Render stage solution A + +

Another possible solution (called solution B) is based on +the DirectGDI and Graphics Resource components (which are described next), +both of which have interface and adaptation layers. This solution is not full +featured. Like the previous diagram, this diagram focuses on the more relevant +components and does not attempt to show everything.

+Render stage solution B + +

Both of these render stage solutions mean that existing Window Server +applications can take advantage of hardware acceleration if it is available +(and therefore run faster) without recompiling the code.

ScreenPlay +provides extensions to the Window Server client-side API, which enable mobile +devices to respond to events from a number of pointers, including their proximity +and pressure. This feature is known as advanced +pointers.

A new API, RWsSession::Finish(), +has been added to allow Window Server client applications to synchronize with +the completion of Window Server rendering. The existing API, RWsSession::Flush(), +is redefined to simply flush the client-side command buffer, whereas previously +it also provided a guarantee that Window Server had completed the command +buffer’s operations. This behavioral change allows legacy clients to benefit +from the asynchronous hardware rendering when supported by the render stage +plug-in(s) that are in use.

For more information, see Windowing +Collection.

DirectGDI

DirectGDI +provides a graphics context that can be hardware accelerated and allows an +asynchronous interface. DirectGDI has two parts: a generic layer, which provides +a client API and an adaptation layer. Device creators can replace the adaptation +layer with an implementation that takes advantage of graphics accelerated +hardware, if it is available, or a software implementation, if it is not available.

DirectGDI +was introduced as a prototype in the development of ScreenPlay. It is deprecated +in Symbian^3.

Graphics Resource

The +Graphics Resource component provides an abstraction layer for the memory management +of pixel and non-pixel data (such as OpenVG command lists). Like DirectGDI, +it has a generic part, which provides a client API and an adaptation part, +which device creators can adapt to take advantage of graphics hardware when +it is available.

The Graphics Resource component was introduced as +a prototype in the development of ScreenPlay. It is deprecated in Symbian^3 +and will be removed in Symbian^4. However, a new Graphics Resource Interface +component is planned for S^4. This new component will provide a similar but +reduced API that is optimized for sharing images across processes.

OpenVG, OpenGL +ES and EGL

Symbian provides support for OpenVG, OpenGL ES and EGL. +The main advantage of ScreenPlay with regard to EGL is that EGL can render +into composition surfaces. For application developers this offers the ability +to have semi-transparent GDI content on top of EGL content. The EGL client +can query whether these new features are supported on the particular device.

For +more information, see Khronos +API Support.

Screen Driver

In +ScreenPlay, the implementation of the Screen Driver has been changed so that +DSA content can be passed into the composition engine.

+Graphics +Composition +Graphics +Hardware Acceleration +Graphics +Concepts + + + +

\ No newline at end of file