A major advantage of HTTP Live Streaming, and it is still unbeaten by any other standard, is that you can use any technology to compose multimedia content. You don’t need expensive media servers to handle media streams. You can easily delegate media composition to some simple PHP / Python / Ruby / Node.js modules. As a long-time Flash lover I was somewhat prejudiced against the Apple’s video delivery standard. But as a developer, every day I use it to solve tasks which have been almost unattainable with the previous stack (or required expensive software). Adobe HDS playlists have been really hard to deal with. Just think of binary data in f4m playlists. They require much more time to develop & debug the solution. MPEG DASH is also far from being intuitive.
In this post, we are going to discuss how to make an alternative audio track for your video. Although HTTP Live Streaming can streamline this task, yet there are some limitations, so you need to make certain hacks on the client side. In our Together project, we had to implement alternate sound tracks for user videos. Luckily, we use HTTP Live Streaming throughout the system.
HTTP Live Streaming provides support for “overriding” the main media presentation using EXT-X-MEDIA tag. So, in the source audio/video stream you can override the audio track or even the main video. This feature is supported since iOS 5. Here you can find a technical note how to alternate media in HTTP Live Streaming. So far it has been the best way to alternate media in a stream. But in case you have inserted ads into your stream by using EXT-X-DISCONTINUITY, you won’t be able to play alternative media at discontinuities smoothly.
Here you can find a demo of how we do it in Together’s video albums: we stitch video with HLS and override the audio track. Have a look at this sample with Amsterdam Westerkerk Video.
Another solution may consist in client-side playback of various streams, like playing an audio tag and rendering a video tag. The only problem is that in iOS Safari only one stream can exist at a time. Let’s check some workarounds applicable here. First of all, here are some helpful materials:
- Safari HTML5 Audio and Video Guide – Syncing Multiple Media Elements Together
- HTML5 Media Controllers
The fact is that iOS Safari fully supports Media Controllers, but simultaneous playback is prevented. I spent a whole day trying to find a hack to avoid this prevention. It seems that with timeouts, user interaction and media controllers, you can play media simultaneously.
[gist id=6397764]
Unfortunately, this hack fails on some Android phones. Also, you have to implement your own player controls to keep audio/video in sync. I would recommend to stick to the first recipe and substitute the audio/video track by using HLS.
Links:
[1] Dynamic Adaptive Streaming over HTTP.
[2] Dynamic Adaptive Streaming over HTTP: Part 1 – Media Presentation Description and Segment Formats.
[3] Alternate media in HTTP Live Streaming Playlists.
[4] Syncing Multiple Media Elements.
[5] MediaController object description.