0The Ultimate Codec Guide

How Windows manages playing multimedia

Win­dows auto­mat­ic behaviour

When a file is played on Win­dows, most play­er soft­ware simply lets Win­dows handle the play­back (Media Play­er Clas­sic Home Cinema is an excep­tion).  Win­dows auto­mat­ic­ally selects an appro­pri­ate split­ter to “split” the con­tain­er, and then selects the appro­pri­ate decoders to decode each of the streams. Each decoded stream is then sent to a final stage known as a ren­der­er. The split­ter, decoders and ren­der­ers are known as fil­ters and are some­times referred to as codecs. The term codec has sev­er­al com­mon uses, but is gen­er­ally used to mean a pro­gram which can carry out either encod­ing or decod­ing. In this guide we will use the term codec to refer to com­plete soft­ware pack­ages, which may con­tain mul­tiple fil­ters and sup­port mul­tiple stream types. Each fil­ter is a self-con­tained unit, usu­ally stored inside an .ax file in the system32 folder. The pro­cess of select­ing the split­ter, decoders and ren­der­ers is known as build­ing a graph. There are util­it­ies avail­able which allow you to see what graphs Win­dows will build for a par­tic­u­lar file, as well as build­ing cus­tom graphs to test playback

Ren­der­ers

Ren­der­ers are the final stage of the graph; and are the inter­face between the decoded streams and the hard­ware that out­puts the stream – for example the video card drivers, or the sound card drivers. They man­age the way the out­put fits into the Win­dows sys­tem. So for example you may play back a video in a win­dow (as opposed to full screen) and also have anoth­er pro­gram open. Win­dows needs to how big to dis­play the video, and where on the screen. You may also move anoth­er pro­gram or win­dow par­tially in front of the video and Win­dows needs to know to dis­card this part of the video. This is all man­aged by the renderers.

[google_adsense]

Hard­ware decoding

His­tor­ic­ally all decod­ing was done by the CPU.  The CPU is designed to be very flex­ible, which means it can under­take any task, but is not espe­cially effi­cient at any­thing. The increase in HD video con­tent which requires sub­stan­tial pro­cessing to decode has led to a demand for more effi­cient video decod­ing hard­ware. All mod­ern video cards now include some level of “hard­ware decod­ing” of the 3 BluRay encod­ing stand­ards (mpeg2, h.264, VC1).  The level of sup­port var­ies but with each new gen­er­a­tion of hard­ware more stand­ards are sup­por­ted. The latest gen­er­a­tion of CPUs from Intel now also include ded­ic­ated decod­ing hard­ware which frees the main parts of the CPU for oth­er tasks, and reduces power con­sump­tion. Ded­ic­ated GPU and CPU hard­ware decod­ing can only be util­ised if the codec used for decod­ing sup­ports hard­ware decod­ing. The level of sup­port for this fea­ture is vari­able, but it is grow­ing, and sev­er­al of the free codecs provide good sup­port. The codecs Microsoft provides with Win­dows also offer some sup­port. The sys­tem of hard­ware accel­er­a­tion is com­monly known as DXVA (Dir­ectX Video Accel­er­a­tion).  Many recent ded­ic­ated hard­ware decoders also sup­port vari­ous image enhan­cing features.

In many cases, hard­ware decod­ing is incom­pat­ible with the use of sub­titles, how­ever, DXVA with sub­titles is pos­sible via the free open-source FFD­show tryouts.

Raw stream output

In some cases no decoder is required. Some audio on BluRay disks is uncom­pressed and can be sent dir­ectly from the split­ter to the ren­der­er without need of a decoder. In addi­tion to this, many home theatre amp­li­fi­ers include digit­al con­nect­ors (SPDIF or HDMI) and have their own hard­ware decoders for the 2 lead­ing sys­tems: Dolby and DTS. If you have an amp­li­fi­er which sup­ports hard­ware decod­ing you may wish to leave the audio stream encoded and send the encoded stream to the amp­li­fi­er for decod­ing. To do this the stream from the split­ter must be sent to the appro­pri­ate hard­ware (either a sound card in the case of SPDIF, or the video card for HDMI).  This stream must be sent in the cor­rect way so the hard­ware knows to out­put it over the digit­al con­nec­tion. To do this still requires a fil­ter, but the fil­ter will not decode the stream. The most com­mon codec used for this pur­pose is called ac3filter.

Video is nev­er nor­mally raw and so always requires a decoder fil­ter. It must also pass through the ren­der­er. Sub­title inform­a­tion is also passed to the ren­der­er for ren­der­ing on top of the video.

New codec sys­tem in Vista

The sys­tem Win­dows uses to man­age all these fil­ters is called Dir­ect­Show. The vast major­ity of fil­ters are designed to con­form to the Dir­ect­Show stand­ards. How­ever, with Win­dows Vista, Microsoft intro­duced a new sys­tem called Media Found­a­tion. This new sys­tem runs in par­al­lel with Dir­ect­Show, but takes pre­ced­ence. Microsoft has include Media Found­a­tion codecs in all Win­dows releases since Vista, and when a file is played back, Win­dows will try to use the Media Found­a­tion codecs to play­back the file. This has made it more dif­fi­cult to use altern­at­ive codecs.