How Windows manages playing multimedia
Windows automatic behaviour
When a file is played on Windows, most player software simply lets Windows handle the playback (Media Player Classic Home Cinema is an exception). Windows automatically selects an appropriate splitter to “split” the container, and then selects the appropriate decoders to decode each of the streams. Each decoded stream is then sent to a final stage known as a renderer. The splitter, decoders and renderers are known as filters and are sometimes referred to as codecs. The term codec has several common uses, but is generally used to mean a program which can carry out either encoding or decoding. In this guide we will use the term codec to refer to complete software packages, which may contain multiple filters and support multiple stream types. Each filter is a self-contained unit, usually stored inside an .ax file in the system32 folder. The process of selecting the splitter, decoders and renderers is known as building a graph. There are utilities available which allow you to see what graphs Windows will build for a particular file, as well as building custom graphs to test playback
Renderers are the final stage of the graph; and are the interface between the decoded streams and the hardware that outputs the stream – for example the video card drivers, or the sound card drivers. They manage the way the output fits into the Windows system. So for example you may play back a video in a window (as opposed to full screen) and also have another program open. Windows needs to how big to display the video, and where on the screen. You may also move another program or window partially in front of the video and Windows needs to know to discard this part of the video. This is all managed by the renderers.
Historically all decoding was done by the CPU. The CPU is designed to be very flexible, which means it can undertake any task, but is not especially efficient at anything. The increase in HD video content which requires substantial processing to decode has led to a demand for more efficient video decoding hardware. All modern video cards now include some level of “hardware decoding” of the 3 BluRay encoding standards (mpeg2, h.264, VC1). The level of support varies but with each new generation of hardware more standards are supported. The latest generation of CPUs from Intel now also include dedicated decoding hardware which frees the main parts of the CPU for other tasks, and reduces power consumption. Dedicated GPU and CPU hardware decoding can only be utilised if the codec used for decoding supports hardware decoding. The level of support for this feature is variable, but it is growing, and several of the free codecs provide good support. The codecs Microsoft provides with Windows also offer some support. The system of hardware acceleration is commonly known as DXVA (DirectX Video Acceleration). Many recent dedicated hardware decoders also support various image enhancing features.
In many cases, hardware decoding is incompatible with the use of subtitles, however, DXVA with subtitles is possible via the free open-source FFDshow tryouts.
Raw stream output
In some cases no decoder is required. Some audio on BluRay disks is uncompressed and can be sent directly from the splitter to the renderer without need of a decoder. In addition to this, many home theatre amplifiers include digital connectors (SPDIF or HDMI) and have their own hardware decoders for the 2 leading systems: Dolby and DTS. If you have an amplifier which supports hardware decoding you may wish to leave the audio stream encoded and send the encoded stream to the amplifier for decoding. To do this the stream from the splitter must be sent to the appropriate hardware (either a sound card in the case of SPDIF, or the video card for HDMI). This stream must be sent in the correct way so the hardware knows to output it over the digital connection. To do this still requires a filter, but the filter will not decode the stream. The most common codec used for this purpose is called ac3filter.
Video is never normally raw and so always requires a decoder filter. It must also pass through the renderer. Subtitle information is also passed to the renderer for rendering on top of the video.
New codec system in Vista
The system Windows uses to manage all these filters is called DirectShow. The vast majority of filters are designed to conform to the DirectShow standards. However, with Windows Vista, Microsoft introduced a new system called Media Foundation. This new system runs in parallel with DirectShow, but takes precedence. Microsoft has include Media Foundation codecs in all Windows releases since Vista, and when a file is played back, Windows will try to use the Media Foundation codecs to playback the file. This has made it more difficult to use alternative codecs.
Think we've missed something? Let us know by commenting below. If you would like to subscribe please use the subscribe link on the menu at the top right. You can also share this with your friends by using the social links below. Cheers.