0The Ultimate Codec Guide

How data is stored in a mul­ti­me­dia file

Files on a com­puter (or on an optical disk like DVD or BluRay) need to con­tain more than 1 type of data.  A typ­ical movie will include at least 1 video “stream” and one audio “stream”.  Most movies include mul­tiple lan­guages and sub­titles, each of which requires an addi­tional stream.  Each of these streams is effect­ively a file in its own right, but they are all stored together within a “con­tainer” which also starts each of them at the right time (sub­titles don’t start imme­di­ately at the start of a movie for example) and keeps them in sync as well as stor­ing meta-data about each of them – i.e. what lan­guage they are, what fram­er­ate and res­ol­u­tion the video is, and what com­pres­sion stand­ards have been used.

Video streams

Video streams con­tain only the video part of the movie.  They are com­pressed using a par­tic­u­lar sys­tem and there should be meta-data about their res­ol­u­tion, fram­er­ate, if they are inter­laced or pro­gress­ive and details of the encod­ing sys­tem used.

Audio streams

Audio streams con­tain only the audio part of the movie.  There are usu­ally mul­tiple audio streams included with the movie and each has its own stream.  They can be com­pressed in a range of ways or in some cases they are uncom­pressed raw audio.  There should be meta-data about their bitrate, res­ol­u­tion, lan­guage, num­ber of chan­nels and details of the encod­ing sys­tem used.

Other streams (e.g. Closed Captions)

Many movies con­tain other streams, most com­monly closed cap­tions (also known as sub­titles).  These come in sev­eral formats, but are typ­ic­ally just text with time stamps.  They are so small rel­at­ively that no com­pres­sion is used.  There should be meta-data about their lan­guage.

All of the streams that com­prise the movie are bundled together and kept in sync by a con­tainer.  The con­tainer should con­tain (and provide) all the meta-data about each stream.  The con­tainer also enables the col­lec­tion of streams to be stored as a single file.  The most com­mon con­tainer formats are VOB (used by DVD) and M2TS (used by BluRay).  Other com­mon con­tainer formats on PC are TS, MPEG, AVI, MKV, WMV, WTV, DVR-MS and MP4.  The obsol­ete HD–DVD format used EVO containers.