0FFmpeg LogoThe Ultimate Codec Guide

Encoding

Why AV streams are encoded

Each pixel of an image has at least an 8bit value for each of the 3 primary col­ours.  So each pixel requires 3 bytes of inform­a­tion (8bits in a byte).  A full-HD video frame is 1920×1080 pixels.  So a stand­ard single frame of uncom­pressed HD video will require 6,220,800 bytes (5.93Mbytes) of data.  If a typ­ic­al movie is 2 hours long, and has 30 frames per second there will be a total of 216,000 frames in the movie.  The total movie size, if uncom­pressed, would be 1.2 Tera­bytes.  Mod­ern Double-layered BluRay disks can hold only 50 Giga­bytes so a movie would span 25 disks.

Both audio and video streams can be com­pressed and there are many dif­fer­ent sys­tems of com­pres­sion.  Both can be “losslessly” com­pressed – this means that no inform­a­tion is lost.  For example, if a 100×100 pixel image is pure red then instead of describ­ing every pixel in the image as red (which would take 100×100×3 = 30,000 bytes), the com­pres­sion sys­tem might allow us to just say “the square is 100 by 100 and every pixel is red”.  This sys­tem is clearly much more effi­cient, and we still get an identic­al red square.  For­tu­nately no movies involve a con­stant, single col­our image, and audio is rarely a mono­tone.  This means there are lim­its to how much we can reduce the size of an audio or video stream with lossless com­pres­sion.  Many lossless audio com­pres­sion sys­tems can achieve approx­im­ately  a halv­ing of the raw audio data which in some instances is suf­fi­cient.  A movie with 2 hours of audio for 6 chan­nels (com­monly called 5.1, mean­ing 2 front, 2 rear and 1 centre chan­nel plus 1 sub­woof­er) requires approx­im­ately 3GBytes of raw audio, or 1.5Gig of lossless audio.  For a 50Gig BluRay disk this rep­res­ents 3% of the total capa­city and lossless or even raw audio formats are com­monly used.  For older 9Gig DVDs this rep­res­en­ted nearly 20% of total capa­city which was too large and con­sequently the audio on DVDs is always com­pressed in a way that does lose some data.

The com­mon MP3 format is a good example of what can be achieved by com­pres­sion that does lose data (oth­er­wise known as lossy com­pres­sion).  A raw audio file can be reduced to around 15% of its ori­gin­al size using typ­ic­al MP3 com­pres­sion.  The details of this pro­cess are com­plex, but in essence lossy com­pres­sion sys­tems attempt to identi­fy data which the human senses will not notice is miss­ing, and dis­card it.

All stand­ard video com­pres­sion schemes are lossy – lossless com­pres­sion of video is simply not suf­fi­cient to make the data size man­age­able.  Video com­pres­sion schemes have a com­plic­ated 3 way trade-off.  In an ideal world the scheme will pro­duce a video file with the smal­lest data size for the low­est qual­ity loss.  How­ever, these types of files require a lot of cal­cu­la­tion to decode – to restore to an uncom­pressed format when they are out­put.  The most power­ful schemes require so much pro­cessing that they could not be played using the hard­ware avail­able at con­sumer prices, and so a third com­prom­ise is neces­sary.  As con­sumer hard­ware gets more power­ful, new com­pres­sion schemes are intro­duced that increase qual­ity or reduce data size at the cost of requir­ing great­er pro­cessing.  As great­er pro­cessing is now avail­able this is an appro­pri­ate evolution.

[google_adsense]

Com­mon video formats

The most com­mon video com­pres­sion sys­tem is MPEG2.  This sys­tem is the stand­ard used on all DVDs.  MPEG2 is also used on some BluRay disks.  Oth­er BluRay disks use a sys­tem known as AVC or H.264.  The 3rd and final third format approved for the BluRay stand­ard is known as VC1.  VC1 is a vari­ant of Microsoft’s WMV3.

Oth­er com­mon formats are DivX, XviD, Flash, VP6, MPEG1, WMV1 and WMV2

Com­mon audio formats

The most com­mon type of audio is in fact raw audio as this is used on Audio CDs.  This is stored as .wav files on PCs.  The MP3 format is anoth­er very widely used format.  The vast major­ity of movies use mul­tichan­nel sound, rather than ste­reo and have tra­di­tion­ally been encoded using one of the Dolby schemes (usu­ally called AC3) or one of the DTS schemes, the most com­mon of which are Dolby Digit­al and the ori­gin­al DTS.  There are sev­er­al new­er formats, some of which are quite com­mon on BluRay disk includ­ing Dolby Digit­al Plus (E‑AC3), Dolby True HD and DTS-HD Mas­ter Audio (DTS-MA).  Some BluRays also include raw mul­tichan­nel audio, com­monly called LPCM.

Oth­er com­mon formats include MP1, MP2, AAC, Ape, FLAC and Ogg.

File types (file extensions)

File exten­sions and video streams

File exten­sions are related to con­tain­ers, but imply noth­ing about the streams with­in the con­tain­er.  A com­mon mis­con­cep­tion is that the file exten­sion belongs to “the video” – it doesn’t!  For example, the file exten­sion .m2ts can con­tain video streams encoded with mpeg2, h.264 or VC1, three com­pletely dif­fer­ent encod­ing sys­tems.  Equally, a h.264 stream can be found with­in all of the most com­mon con­tain­er formats except VOB.  In some rare cir­cum­stances, some of the streams can be inferred from the con­tain­er (for example VOB files con­tain mpeg2 video streams) but the file­type doesn’t always identi­fy the con­tain­er.  File­types are inten­ded to identi­fy the con­tain­er, how­ever, whilst in some cases this works reli­ably, there are oth­ers where the cor­rel­a­tion is not per­fect.  For example, M2TS and TS are largely inter­change­able, although BluRay disks only use M2TS, and live streams are nor­mally stored as TS.

File icons

File icons tell you very little about a file.  Icons are matched to exten­sions by Win­dows but each file type doesn’t have its own par­tic­u­lar icon.  The icons are fre­quently changed when soft­ware is installed or the file is played back in a dif­fer­ent play­er.  It would be easy to make Win­dows show the same icon for every type of file on the sys­tem!  The icons nor­mally only tell you what Win­dows thinks the default play­er soft­ware for the file is.

You might also like...

Leave a Reply