0FFmpeg LogoThe Ultimate Codec Guide


Why AV streams are encoded

Each pixel of an image has at least an 8bit value for each of the 3 primary col­ours.  So each pixel requires 3 bytes of inform­a­tion (8bits in a byte).  A full-HD video frame is 1920×1080 pixels.  So a stand­ard single frame of uncom­pressed HD video will require 6,220,800 bytes (5.93Mbytes) of data.  If a typ­ic­al movie is 2 hours long, and has 30 frames per second there will be a total of 216,000 frames in the movie.  The total movie size, if uncom­pressed, would be 1.2 Tera­bytes.  Mod­ern Double-layered BluRay disks can hold only 50 Giga­bytes so a movie would span 25 disks.

Both audio and video streams can be com­pressed and there are many dif­fer­ent sys­tems of com­pres­sion.  Both can be “losslessly” com­pressed – this means that no inform­a­tion is lost.  For example, if a 100×100 pixel image is pure red then instead of describ­ing every pixel in the image as red (which would take 100×100×3 = 30,000 bytes), the com­pres­sion sys­tem might allow us to just say “the square is 100 by 100 and every pixel is red”.  This sys­tem is clearly much more effi­cient, and we still get an identic­al red square.  For­tu­nately no movies involve a con­stant, single col­our image, and audio is rarely a mono­tone.  This means there are lim­its to how much we can reduce the size of an audio or video stream with lossless com­pres­sion.  Many lossless audio com­pres­sion sys­tems can achieve approx­im­ately  a halv­ing of the raw audio data which in some instances is suf­fi­cient.  A movie with 2 hours of audio for 6 chan­nels (com­monly called 5.1, mean­ing 2 front, 2 rear and 1 centre chan­nel plus 1 sub­woof­er) requires approx­im­ately 3GBytes of raw audio, or 1.5Gig of lossless audio.  For a 50Gig BluRay disk this rep­res­ents 3% of the total capa­city and lossless or even raw audio formats are com­monly used.  For older 9Gig DVDs this rep­res­en­ted nearly 20% of total capa­city which was too large and con­sequently the audio on DVDs is always com­pressed in a way that does lose some data.

The com­mon MP3 format is a good example of what can be achieved by com­pres­sion that does lose data (oth­er­wise known as lossy com­pres­sion).  A raw audio file can be reduced to around 15% of its ori­gin­al size using typ­ic­al MP3 com­pres­sion.  The details of this pro­cess are com­plex, but in essence lossy com­pres­sion sys­tems attempt to identi­fy data which the human senses will not notice is miss­ing, and dis­card it.

All stand­ard video com­pres­sion schemes are lossy – lossless com­pres­sion of video is simply not suf­fi­cient to make the data size man­age­able.  Video com­pres­sion schemes have a com­plic­ated 3 way trade-off.  In an ideal world the scheme will pro­duce a video file with the smal­lest data size for the low­est qual­ity loss.  How­ever, these types of files require a lot of cal­cu­la­tion to decode – to restore to an uncom­pressed format when they are out­put.  The most power­ful schemes require so much pro­cessing that they could not be played using the hard­ware avail­able at con­sumer prices, and so a third com­prom­ise is neces­sary.  As con­sumer hard­ware gets more power­ful, new com­pres­sion schemes are intro­duced that increase qual­ity or reduce data size at the cost of requir­ing great­er pro­cessing.  As great­er pro­cessing is now avail­able this is an appro­pri­ate evol­u­tion.

Com­mon video formats

The most com­mon video com­pres­sion sys­tem is MPEG2.  This sys­tem is the stand­ard used on all DVDs.  MPEG2 is also used on some BluRay disks.  Oth­er BluRay disks use a sys­tem known as AVC or H.264.  The 3rd and final third format approved for the BluRay stand­ard is known as VC1.  VC1 is a vari­ant of Microsoft’s WMV3.

Oth­er com­mon formats are DivX, XviD, Flash, VP6, MPEG1, WMV1 and WMV2

Com­mon audio formats

The most com­mon type of audio is in fact raw audio as this is used on Audio CDs.  This is stored as .wav files on PCs.  The MP3 format is anoth­er very widely used format.  The vast major­ity of movies use mul­tichan­nel sound, rather than ste­reo and have tra­di­tion­ally been encoded using one of the Dolby schemes (usu­ally called AC3) or one of the DTS schemes, the most com­mon of which are Dolby Digit­al and the ori­gin­al DTS.  There are sev­er­al new­er formats, some of which are quite com­mon on BluRay disk includ­ing Dolby Digit­al Plus (E‑AC3), Dolby True HD and DTS-HD Mas­ter Audio (DTS-MA).  Some BluRays also include raw mul­tichan­nel audio, com­monly called LPCM.

Oth­er com­mon formats include MP1, MP2, AAC, Ape, FLAC and Ogg.

File types (file extensions)

File exten­sions and video streams

File exten­sions are related to con­tain­ers, but imply noth­ing about the streams with­in the con­tain­er.  A com­mon mis­con­cep­tion is that the file exten­sion belongs to “the video” – it doesn’t!  For example, the file exten­sion .m2ts can con­tain video streams encoded with mpeg2, h.264 or VC1, three com­pletely dif­fer­ent encod­ing sys­tems.  Equally, a h.264 stream can be found with­in all of the most com­mon con­tain­er formats except VOB.  In some rare cir­cum­stances, some of the streams can be inferred from the con­tain­er (for example VOB files con­tain mpeg2 video streams) but the file­type doesn’t always identi­fy the con­tain­er.  File­types are inten­ded to identi­fy the con­tain­er, how­ever, whilst in some cases this works reli­ably, there are oth­ers where the cor­rel­a­tion is not per­fect.  For example, M2TS and TS are largely inter­change­able, although BluRay disks only use M2TS, and live streams are nor­mally stored as TS.

File icons

File icons tell you very little about a file.  Icons are matched to exten­sions by Win­dows but each file type doesn’t have its own par­tic­u­lar icon.  The icons are fre­quently changed when soft­ware is installed or the file is played back in a dif­fer­ent play­er.  It would be easy to make Win­dows show the same icon for every type of file on the sys­tem!  The icons nor­mally only tell you what Win­dows thinks the default play­er soft­ware for the file is.

Got some thoughts of your own? Indulge yourself below by commenting! If you would like to subscribe please use the subscribe link on the menu at the top right. You can also share this with your friends by using the social links below. Cheers.

You might also like...

Leave a Reply