I have been mucking around with the audio tag as part of my quest to understand where HTML5 is going. The <video> tag gets all the press but I think there are many more opportunities to use audio in web apps. HTML5 is closing the gap between plugin-based apps (Flash, Silverlight, Java, etc) and sound support is an important part of that goal.

(Those of you who don’t care how it works should go directly to the TV Themes demo puzzle. It works best in Firefox3.6 and the latest version of Safari, although most browsers should function to some degree.)

The audio tag is pretty flexible, able to handle both long form audio (songs and spoken passages – the theme medley on the demo page for example) and short snippets of background audio (alerts, and confirmations – the demo plays one of two short tones when you type an answer. Video game sound effects are another example.) Optionally, the audio tag can provide a user interface for starting and stopping the audio, useful for playing long streams of audio. Different browsers have different ideas about how this should look, but they all function much the same way.

In theory, the audio tag is as easy as embedding an image into HTML:

1
2
3
4

<audio controls>
	<source src="music.mp3">
	You can put HTML here that will be displayed if the browser does not understand the audio tag
</audio>

However, the devil is in the details. There are two problems with the audio tag that complicate matters. The first is that only the very latest browsers support the audio tag at all. This means that if you want to provide audio that everyone can use, you are going to have a fall-back method available. Before the audio tag, people used to use Flash for this purpose and it still works. A number of sites provide simple Flash-based audio players that you can embed – I ended up using the player provided by Google.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

<object codebase="http://fpdownload.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=7,0,0,0" height="27" width="400" align="middle" classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000">
	<param name="_cx" value="10583"><param name="_cy" value="714"><param name="FlashVars" value="">
	<param name="Movie" value="http://www.google.com/reader/ui/3247397568-audio-player.swf?audioUrl=http://full/path/to/music.mp3">
	<param name="Src" value="http://www.google.com/reader/ui/3247397568-audio-player.swf?audioUrl=http://full/path/to/music.mp3">
	<param name="WMode" value="Window"><param name="Play" value="0">
 
	<param name="Loop" value="-1">
	<param name="Quality" value="High">
	<param name="SAlign" value="LT">
	<param name="Menu" value="-1">
	<param name="Base" value="">
	<param name="AllowScriptAccess" value="never">
	<param name="Scale" value="NoScale">
	<param name="DeviceFont" value="0">
	<param name="EmbedMovie" value="0">
 
	<param name="BGColor" value="">
	<param name="SWRemote" value="">
	<param name="MovieData" value="">
	<param name="SeamlessTabbing" value="1">
	<param name="Profile" value="0">
	<param name="ProfileAddress" value="">
	<param name="ProfilePort" value="0">
	<param name="AllowNetworking" value="all">
	<param name="AllowFullScreen" value="false">
 
	<embed type="application/x-shockwave-flash" src="http://www.google.com/reader/ui/3247397568-audio-player.swf?audioUrl=http://full/path/to/music.mp3" allowscriptaccess="never" quality="best" bgcolor="#ffffff" wmode="window" flashvars="playerMode=embedded" pluginspage="http://www.macromedia.com/go/getflashplayer" height="27" width="400" />
</object>

Not exactly elegant, is it? Apart from being uuuuug-ly, the full URI of the sound file must be used (the audio tag can use relative paths). Also, the Flash players are not scriptable in the same way as inbuilt audio tag is, which can make doing tricky stuff like animating other content in response to the audio more difficult.

The second problem with the audio tag is the same codec problem I talked about in a previous rant (The HTML5 Video Tag’s Fatal Flaw) For legal reasons, different browsers play different formats of audio – most notably Firefox will not play mp3s while Safari will not play ogg. There is no single format that will play in all browsers except for uncompressed wavs, which are too fat to be useful except for very short snippets.

To get around this problem the audio tag allows multiple files to be specified. The first file that the browser thinks it can play will be used, but it does mean you have to encode and store multiple versions of each audio file.

1
2
3
4
5

<!-- Only one of these files will be downloaded -->
<audio controls>
	<source src="music.ogg" type="audio/ogg">
	<source src="music.mp3" type="audio/mpeg">
</audio>

The demo page also uses the audio tag to play sound effects in the background, using audio elements that do not have a user interface. For simplicity I used wav files (download from this awesome source of free effects.) Since they have no user interface, Javascript must be used to play them:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

<audio id="clicksound" preload="auto">
	<source src="click.wav" type="audio/wav">
</audio>
 
<script type="text/javascript">
function playSound( )
{
	var a = document.getElementById( "clicksound" );
	if ( !a ) return;
	if ( !a.play ) return; // will exit if the browser does not understand the audio tag
 
	a.play();
}
</script>

It is all pretty simple but as always there are problems. I did not find a good way of replicating this using Flash, so browsers that do not understand the audio tag do not play these background noises. Also, Google Chrome (which has otherwise excellent support) contains a weird bug that prevents it playing the first couple of seconds of an audio file, making it useless for short sounds. Apparently Firefox3.5 had the same problem, but it works perfectly in 3.6.

I created the demo to see if the audio tag could replicate the functionality of Flash-based applications for both long-form audio and background sound effects. It does seem to be possible provided you are targeting a modern browser and are prepared to work around certain annoyances. Hopefully the next few years will see an improvement in support for audio, I can see many uses for it especially if the iPad (which does not support Flash) takes off.