Hardware overlay

In computing, hardware overlay, a type of video overlay, provides a method of rendering an image to a display screen with a dedicated memory buffer inside computer video hardware. The technique aims to improve the display of a fast-moving video image — such as a computer game, a DVD, or the signal from a TV card. Most video cards manufactured since about 1998 and most media players support hardware overlay.

The overlay is a dedicated buffer into which one app can render (typically video), without incurring the significant performance cost of checking for clipping and overlapping rendering by other apps. The framebuffer has hardware support for importing and rendering the buffer contents without going through the GPU.

Overview

The use of a hardware overlay is important for several reasons:

1. In a graphical user interface (GUI) operating system such as Windows, one display-device can typically display multiple applications simultaneously.
2. Consider how a display works without a hardware overlay. When each application draws to the screen, the operating system's graphical subsystem must constantly check to ensure that the objects being drawn appear on the appropriate location on the screen, and that they don't collide with overlapping and neighboring windows. The graphical subsystem must clip objects while they are being drawn when a collision occurs. This constant checking and clipping ensures that different applications can cooperate with one another in sharing a display, but also consumes a significant proportion of computing power.

A computer draws on its display by writing a bitmapped representation of the graphics into a special portion of its memory known as video memory. Without any hardware overlays, only one chunk of video memory exists which all applications must share - and the location of a given application's video memory moves whenever the user changes the position of the application's window. With shared video memory, an application must constantly check that it is only writing to memory that belongs to that application.

When running a high-bandwidth video application such as a movie player or some games, the computing power and complexity needed to perform constant clipping and checking negatively impacts performance and compatibility. A hardware overlay escapes these limitations. In addition, the graphics processing unit (GPU) provides an efficient way to scale the video in size and often performs color-format conversions (such as MPEG-2's YCbCr into RGB).

An application using a hardware overlay gets a completely separate section of video memory that belongs only to that application. Because nothing else uses it, the program never needs to waste resources in checking whether a given piece of the memory belongs to it, nor does it need to monitor whether the user moves the window and changes the location of the video memory. To get the image from the separate video memory to display in tandem with the remaining shared elements on the display, the graphical subsystem associates a certain attribute (for example, a particular color) as a "mask" for that overlay, which the graphics card understands to mean that it is to draw from the separate overlay buffer onto the screen. (This technique has become known as "chroma key".)

As an example, assume that the graphical subsystem defines the color "purple" as the mask color that triggers the overlay to appear. An application that plays a DVD will draw a solid purple rectangle on the shared screen, and then "play" the DVD into the special region of memory dedicated to the overlay. The graphics card will render the DVD playback only inside the purple area. If another window moves over top of the purple area and obscures a part of it, then the graphics hardware performs the clipping by itself. In practice, the actual color "purple" is not used - rather, either a nondescript near-black color is used (as the system often has thousands of such shades to choose from), or the mask region is denoted using some similar masking method not involving colors.

Screen shots

As a consequence of hardware overlay use, a screenshot program (for example, the one automatically built into Windows that activates when a user presses the PrtSc key) often does not capture the content appearing in the hardware overlay window. Rather, a blank region containing only the special mask color is captured. This is because the screen capture routine doesn't consider the special video memory regions dedicated to overlays – it simply captures the shared main screen as rendered by the software's graphical subsystem.

Secondary displays

Many newer graphics cards can support more than one monitor and/or a TV screen as output devices. Typically one of these output devices has to be declared the "primary" device, and only the primary device can display hardware overlays. There are exceptions: Intel writes in the FAQ for their Embedded Graphics Drivers that the overlay can be attached to either one of the displays but not to both (note: the Intel 945, G33-Q965 chipsets now have dual hardware overlay, and are capable of good quality mpeg2 on secondary monitors when appropriate software is installed), and some newer Matrox graphics card support overlay on both displays (for example, the Parhelia Series).

Secondary displays require both hardware and driver support; some graphics cards may support overlay on the second display while their drivers may not yet support it (note: recent (July 2008) graphics chipset driver bugs can cause most video formats apart from mpeg2 to work on both monitors, and mpeg2 only on the primary with most players).

Some users note that DVD movies display properly on a laptop screen but don't display on a TV connected to the laptop; in these cases it may be possible to designate the TV as the primary display. Sometimes, the use of hardware overlays may have to be disabled in the media player. However, some graphics cards have the option to completely redirect hardware overlay to the TV screen. In this case, starting a DVD player on the main screen with overlay enabled would result in video being displayed on the attached TV screen.

Implementations in various operating systems

Windows Vista's enhanced graphics capabilities replace the basic concept of hardware overlays with full hardware compositing for every application window running on the system, not just movie players or games, through the Desktop Window Manager. Mac OS X has used hardware compositing since the introduction of Quartz Extreme into Mac OS X 10.2. To improve performance, each program draws to its own independent memory buffer instead of to a slow graphical subsystem. (In Windows Vista, each hardware overlay is more correctly known as a Direct3D surface). Then the system's GPU assembles each of the windows into a single display screen in real time. With modern GPUs capable of advanced 3D graphics (as a consequence of the video game industry), operating systems can apply computationally intensive motion, scaling, and lighting effects to normal 2D windows.

Due to a need to reduce power consumption, Windows 7 added back limited support for overlays, and Windows 8.1 added support for multi-plane overlays. Multi-plane overlays allow the Desktop Window Manager to automatically render portions of the desktop into overlays, saving power in more circumstances.

In the X Window System, the windowing system of most Unix operating systems, the XVideo extension can allow applications to employ hardware overlays. Compositing is also used, with compiz and Beryl compositing window managers being the most prominent examples since 2006. They are able to take advantage of OpenGL (through glx extension) for 3D and 2D overlay visual effects. Other implementations like Metacity and xfwm have been available since 2004.

Some dedicated hardware overlay devices use embedded Linux as an operating system, for example the video logger [1] by Racelogic uses a Texas Instruments DM355 micro-controller to blend a frame buffer containing graphics onto a live video feed, and then store the result in MPEG-4 format on a flash card.

Overlay support aka Picture In Picture (PIP) was introduced to AmigaOS with the introduction of the PC graphic cards, Picasso96 and CyberGraphX graphic driver systems aka ReTargetable Graphics, for Picasso IV and Voodoo3 cards, picasso’96 driver system became standard in AmigaOS 3.5. Moovid was one of the first video players that supported hardware overlay (PIP) on AmigaOS.