This post is also available in: Russian

Year by year, we become more involved with media technology than ever before. The World Wide Web is also sensitive to this trend as shown by immense success and popularity of online video services, such as YouTube. However, you cannot run such systems without the technology for video content playback at the end user level.

A classical approach is to build a video player with Flash. Probably you can hardly find a more popular Web browser plugin than Flash Player today. However, a weakness of this approach is the necessity to install the Flash plugin, missing from many devices by default. For example, whatsoever popular iPhone and iPad are, they cannot run Flash applications from within the browser. The way out of this situation is to use HTML5.

This modern version of HTML standard has introduced many innovations. But for us, the key feature is to play back videos from a standard Web browser. To do this, HTML5 specification has been enhanced by the <video> tag. It offers everything you need for easy video file playback. All  you need is to use the tag and set the value of its attributes:

• Autoplay – if the ‘autoplay‘ attribute is present, the playback starts immediately after a sufficient amount of video has been downloaded.
• Controls – if the ‘controls‘ attribute is present, the playback controls are shown.
• Height – the player height in pixels.
• Loop – if the ‘loop’ attribute is present, the video playback is looped.
• Muted – if the ‘muted’ value is present, the video sound is turned off.
• Poster – this attribute accepts the URL of an image to be displayed prior to content playback.
• Preload – this attribute defines the browser strategy for video content download. It can take on several values:
  • ‘Auto‘ – video download starts at HTML page load.
  • ‘Metadata‘ – only the content metadata is downloaded at page load.
  • ‘None‘ – video download at page load is disabled.
• Src – this attribute contains the URL of the video file.
• Width – contains the height of the player.

First Difficulties and Pitfalls

However, all this is not as easy as it sounds. HTML5 is a young enough standard. So the first thing to be kept in mind is limited support of HTML5 by browsers. Now, you can use the <video> tag in the following browser versions:

• IE 9+
• Firefox 4.0+
• Chrome 6+
• Safari 5+
• Opera 10.6+

Also, please note that different browsers can play back various video formats, so it is necessary to prepare the same content in various formats for different browsers. The following table outlines formats support by the basic browsers:

Web browser	Formats
	Ogg Theora	H.264	VP8 (WebM)
Internet Explorer	Extension required	9.0	Extension required
Mozilla Firefox	3.0	None	4.0
Google Chrome	3.0	Yes	9.0
Safari	Extension required	3.1	Extension required
Opera	10.50	None	10.60

It is possible to enable cross-browser playback in HTML5. To do this, inside <video>, insert the <source> tag pointing to the identical content in different formats. Hence the browser will identify the first playable video file and load it into the player. Here is a sample code:

<video>
  <source src=”path_to_ogg_file” type=”video/ogg” />
  <source src=”path_to_mp4_file” type=”video/mp4” />
  <source src=”path_to_webm_file” type=”video/webm” />
</video>

Please note that the playback controls will look different in different browsers. Therefore, to unify the layout, you have to develop your own player interface. The idea is quite simple: the controls are DIVs positioned on top of the video container.

Please note that browsers may vary in their HTML5 video handling behavior. For instance, on iOS devices you cannot automatically start playback.

Also, you cannot manage the video buffer size. Moreover, it is also imposible to detect neither the size of the played back file nor the download rate.

To Begin With

In HTML5, media elements have a powerful API, allowing not only to develop a common player interface, but also to support additional functionality. As a tool to interact with HTML5, Media API supports JavaScript.

Say, we have the following video container on a page:

<video id=”player” src=”some_path_to_video_file” width=640 height=480>
</video>

To start accessing the player from the API, let’s define an object including a container with the media content. For this, let’s use JQuery selector:

var player = $ ('# player') [0];

Now we can control our player using JS! To start or pause playback, insert the following lines, respectively:

player.start();
player.pause();

HTML5 Media API provides information about the currently played video. For instance, you can use the ‘currentSrc’ field to manipulate the content:

var currentSource = player.currentSrc; // fetch URL of the played 
                                       // back content
player.currentSrc = ‘path_to_new_media_source’; // replace content 
                                                // in the player

Similarly, you can enable the seek feature. To implement this feature, you can use the ‘currentTime‘ field (it accepts a float value of the current playback time in seconds):

var currentTime = player.currentTime; // current playback
                                      // time
player.currentTime = 60 // rewind to 1 minute

You can get the file duration using the ‘duration‘ field. This, together with the current play back time, allows you to calculate the viewing proportion:

var duration = player.duration; // content duration in seconds
var proportion = currentTime / duration; // ratio of current 	
                                         //playback position
                                         // to video
                                         // duration.

Using the ‘volume’ field, you can adjust playback volume :

player.volume = 1; // turn on full volume (to mute,
                   // set the value to “0”)

However the capabilities of HTML5 Media API do not end here. You can implement various sophisticated event-based controls using the addEventListenter() function. For example, to subscribe to the playback end event, do the following:

var onEndFunc = function() { // function called at playback
                             // end
  // some actions
}

player.addEventListener(‘ended’, onEndFunc); // subscribe a function
                                             // to video end
player.removeEventListener(‘ended’, onEndFunc); // unsubscribe the functions
                                                // from video end

Below are all events that exist in HTML5 Media API.

• onabort – playback abort event
• oncanplay – a "can play" event triggered when a sufficient part of video has been loaded
• oncanplaythrough – a "can play" event triggered when the content has been fully buffered
• ondurationchange – content duration change event
• onemptied – an event raised when connection has been lost
• onended – playback ended event
• onerror – content file download error event
• onloadeddata – content download event
• onloadedmetadata – metadata load event.
• onloadstart – file download start event
• onpause – playback stop event
• onplay – playback start event
• onplaying – playback event (to be run until the video stops)
• onprogress – video download progress event (to be run until the entire content is downloaded)
• onratechange – playback rate change event
• onreadystatechange – video player ready state change event
• onseeked – video seek end event
• onseeking – an event raised at content seek
• onstalled – an event raised by browser failure to get the content
• onsuspend – an event raised on content download suspension
• ontimeupdate – an event raised at change of the current playback position
• onvolumechange – a volume change event
• onwaiting – playback suspended for data buffering event

Using events, you can implement such controls as playlists, subtitles, showing of a target frame at seek and many others.

Dynamic Bitrate

As it has been mentioned above, HTML5 lacks a tool to handle buffer size, but the Media API allows you to estimate duration of the buffered content part to show available video on the seek bar. For this purpose, run the following code:

var bufferedTime = player.buffered.end(0); // time limit 
                                           // of the buffered part

And now the fun part starts. Knowing the duration of buffered content, you can dynamically define the bitrate of your content. To do this, you need to implement a method that will evaluate the rate of data buffering and, depending on its, dynamically switch the video bitrate.

The idea behind the analysis is as follows: We estimate the buffer growth rate and, if it is less than the playback rate, then we estimate the time left until buffering is complete and compare it with the remaining playback time. If the buffering time at the estimated rate is higher than the remaining playback time, then the content is switched to a lower bitrate.

var  bufferAnalizer = function (playbackStartPoint, playbackEndPoint,
                       bufferStartPoint , bufferEndPoint, duration) {
  var oldQualityObj = this.qualityObj,
    playbackStart = 0, // new playback start point
    bufferStart = 0; // new buffer start point

    if (oldQualityObj) {
      playbackStart =  oldQualityObj.playbackEndPoint;
      bufferStart = oldQualityObj.bufferEndPoint;
    } else {
       playbackStart = playbackStartPoint;
       bufferStart = bufferStartPoint;
    }

    this.qualityObj = {
      'playbackEndPoint': playbackEndPoint,
      'bufferEndPoint': bufferEndPoint,
      'deltaBuffer': bufferEndPoint - bufferStart, // buffered video size
      'bufferSpeed':  (bufferEndPoint - bufferStart) /
                      (playbackEndPoint - playbackStart),
      'deltaPlayback': playbackEndPoint - playbackStart, // played back
                                                         //video size
      'availTime': bufferEndPoint - playbackEndPoint // difference between 
                                                     // the buffer and the 
                                                     // playback position
    }

    var restTime = duration - playbackEndPoint,
        bufferTime = (duration - bufferEndPoint) / this.qualityObj.bufferSpeed;

    if ((bufferTime > restTime) && ((this.qualityObj.availTime /
                                        this.qualityObj.deltaPlayback ) < 2)) {
       if (this.quality == 'normal') {
          this.quality = 'low';
        }
    }
}

Attach Subtitles

HTML5 has a special <track> tag to display subtitles, but no popular browser has supported it as of the date of publication. However, you can implement this functionality manually. Let the subtitles be stored in the ‘subs.srt’ file. To begin with, let’s attach the subtitle file and put the content into a specific object:

var toSeconds = function(time) {
  var seconds = 0.0;
  if (time) {
    var p = time.split(':');
    for (var i = 0; i < p.length; i++)
      seconds = seconds * 60 + parseFloat(p[i].replace(',', '.'))
    }

  return seconds;
}

$.get(subsSrc, function (data) {
  data = data.replace(/\r\n\r\n/g, "").split('');

  for (var item in data) {
    var subItem = data[item].split(/\r\n/g);

    if (subItem.length > 3) {
      for (var i = 3; i < subItem.length; i++) {
        subItem[2] += '<br />' + subItem[i];
       }
    }

    var time = subItem[1].split(' --> ');

    self.subs.push({
      id:subItem[0],
      sTime:self.toSeconds(time[0]),
      eTime:self.toSeconds(time[1]),
      text:subItem[2]
    });
  }
}, 'html');

Now, we have to create a timer that will, depending on the current time, display relevant subtitles in a DIV with the ‘.subs’ class ‘.

var self = this,
     lastSub = 'empty',
     currentSub;

this.subsTimer = setInterval(function () {
  for (var item in self.subs) {
    var currentTime = self.player.currentTime;

    if ((self.subs[item].eTime > currentTime) &&

       (self.subs[item].sTime <= currentTime)) {
      currentSub = self.subs[item].text;
    } else if ((self.subs[item].eTime < currentTime)) { // no subs now
      currentSub = ' ';
    }
 }

 if (currentSub && (currentSub != lastSub)) {
   self.container.find('.subs').html(currentSub);
    lastSub = currentSub;
 }
}, 500);

Preview Thumbnails on Seek

You might hardly find a person who would not be happy with the feature of thumbnails preview on seek. Implementing of this functionality is not a tough task. To enable preview, on top of the video container, create a new smaller container with the same video content, and position it precisely above the cursor:

self.container.find('.controls').append('<video class="thumb" src="' +
                                  self.player.currentSrc + '"></video>');

And then seek video within this container:

var self = this;

$('.seekbar).bind('mousemove', function (e) {
  self.cursorX = e.pageX;
  self.seek(self);
});

this.scale = this.container.find('.seekbar').width() / player.duration;

var seek = function (context) {
  $('.thumb')[0].currentTime = (self.cursorX -
           self.container.find('.seekbar').offset().left) / self.scale;
}

Transition to Full Screen

In such browsers as Firefox, Safari and Chrome, you have a special API for a full-screen mode. It allows you to expand a selected DIV to the full screen. The features enabling such actions in different browsers are named differently, so you have to implement all of them.

var element = document.getElementById(‘player_container’);

if (element.mozRequestFullScreen) {
  element.mozRequestFullScreen(); // Expand in Firefox
} else if (element.webkitRequestFullScreen) {
  element.width('100%');
  element.height('100%');
  element.webkitRequestFullScreen(); // Expand in Chrome and Safari
}

Note that, for Chrome and Safari, in addition to expanding a container to full screen you should set its height and width to 100%. This is because webkitRequestFullScreen() could only black out the screen and place it in the center of the target container. Scaling of the container is fully delegated to the developer.

Expanding a container to full screen can be identified by the status of such container fields as document.mozFullScreenElement for Firefox and document.webkitIsFullScreen for WebKit based browsers.

To revert to the normal container display, you can use the cancel functions.

// Let's now first detect the full screen status of a given container.
if (document.mozFullScreenElement || document.webkitIsFullScreen) {
  if (document.mozCancelFullScreen) {
     // Collapse full screen in Mozilla
     document.mozCancelFullScreen();  
  } else if (document.webkitCancelFullScreen) {
    // Collapse full screen in Chrome and Safari
    document.webkitCancelFullScreen();  
  }
}

FullScreen API also supports full screen status change events, such as ‘mozfullscreenchange’ and ‘webkitfullscreenchange’, respectively. As has been shown above, in WebKit, scaling of the container is developer’s responsibility, so to revert to normal display, you have to scale down the player container:

document.addEventListener('webkitfullscreenchange',
                   this.WebkitFullscreenEvent = function() {
  if (!document.webkitIsFullScreen) {
    self.container.width(width);
    self.container.height(height);
  }
});

Other Features

Determining Downloadable File Size.

As we have already mentioned, the developers can not determine the size of the video using HTML5. However, they might well fetch the size either from the backend or while making a playlist or at requesting file URL or by invoking an applicable method.

Switching of Audio Tracks

At the moment, switching of audio tracks has not been enabled yet. Of course, you may create multiple audio players per page, but this makes syncing between video and audio a highly non-trivial task.

Getting a Snapshot of the Current Video

If you need to take a snapshot of your video, use <canvas> as outlined here.

Effects:

A nice feature of HTML5 Video is that you can apply any CSS3 transforms and masks to your video. A sample for this feature is available here.

HTML5 vs Flash

There is a widespread belief that HTML5 Video is a Flash "killer" of sorts, as it offers a similar functionality. In fact, this is not particularly the case. HTML5 is not suitable for deploying of the premium video services, as it is agnostic of the following vital features:

• Video streaming. HTML5 can be used for playing back video files only.
• Digital Rights Management However, Flash provides support for DRM technologies.
• A common API implemented in all browsers.
• A standard video format. Different browsers = different video formats for HTML5.

However it is worth noting that the work to extend HTML5 Video to match Flash is underway. The initiative stems primarily from the Google search giant. For more, please read an article on Google’s keynote at Streaming Media.

A Sample Video Player Implementation

Open a video player page.

Conclusions

The emergence of HTML5 has empowered us with a new tool for delivering video content to users. However, as HTML5 is quite young, at this point there are certain cases where Flash is indispensable. The decision as to which mechanism to use totally depends on the goals and needs of the service.

Useful Links

• HTML5 Media API
• Developer Guide on Using HTML5 Audio and Video
• Video Effect Demonstration
• Taking A Video Screen Shot
• Report on the Future of HTML5

Post-Scriptum

Stay with us for an article on HTML5 Video playback analytics.