Tag Archives: safari

The HTML5 audio tag

I have been mucking around with the audio tag as part of my quest to understand where HTML5 is going. The <video> tag gets all the press but I think there are many more opportunities to use audio in web apps. HTML5 is closing the gap between plugin-based apps (Flash, Silverlight, Java, etc) and sound support is an important part of that goal.

(Those of you who don’t care how it works should go directly to the TV Themes demo puzzle. It works best in Firefox3.6 and the latest version of Safari, although most browsers should function to some degree.)

The audio tag is pretty flexible, able to handle both long form audio (songs and spoken passages – the theme medley on the demo page for example) and short snippets of background audio (alerts, and confirmations – the demo plays one of two short tones when you type an answer. Video game sound effects are another example.) Optionally, the audio tag can provide a user interface for starting and stopping the audio, useful for playing long streams of audio. Different browsers have different ideas about how this should look, but they all function much the same way.

In theory, the audio tag is as easy as embedding an image into HTML:

1
2
3
4
<audio controls>
	<source src="music.mp3">
	You can put HTML here that will be displayed if the browser does not understand the audio tag
</audio>

However, the devil is in the details. There are two problems with the audio tag that complicate matters. The first is that only the very latest browsers support the audio tag at all. This means that if you want to provide audio that everyone can use, you are going to have a fall-back method available. Before the audio tag, people used to use Flash for this purpose and it still works. A number of sites provide simple Flash-based audio players that you can embed – I ended up using the player provided by Google.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
<object codebase="http://fpdownload.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=7,0,0,0" height="27" width="400" align="middle" classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000">
	<param name="_cx" value="10583"><param name="_cy" value="714"><param name="FlashVars" value="">
	<param name="Movie" value="http://www.google.com/reader/ui/3247397568-audio-player.swf?audioUrl=http://full/path/to/music.mp3">
	<param name="Src" value="http://www.google.com/reader/ui/3247397568-audio-player.swf?audioUrl=http://full/path/to/music.mp3">
	<param name="WMode" value="Window"><param name="Play" value="0">
 
	<param name="Loop" value="-1">
	<param name="Quality" value="High">
	<param name="SAlign" value="LT">
	<param name="Menu" value="-1">
	<param name="Base" value="">
	<param name="AllowScriptAccess" value="never">
	<param name="Scale" value="NoScale">
	<param name="DeviceFont" value="0">
	<param name="EmbedMovie" value="0">
 
	<param name="BGColor" value="">
	<param name="SWRemote" value="">
	<param name="MovieData" value="">
	<param name="SeamlessTabbing" value="1">
	<param name="Profile" value="0">
	<param name="ProfileAddress" value="">
	<param name="ProfilePort" value="0">
	<param name="AllowNetworking" value="all">
	<param name="AllowFullScreen" value="false">
 
	<embed type="application/x-shockwave-flash" src="http://www.google.com/reader/ui/3247397568-audio-player.swf?audioUrl=http://full/path/to/music.mp3" allowscriptaccess="never" quality="best" bgcolor="#ffffff" wmode="window" flashvars="playerMode=embedded" pluginspage="http://www.macromedia.com/go/getflashplayer" height="27" width="400" />
</object>

Not exactly elegant, is it? Apart from being uuuuug-ly, the full URI of the sound file must be used (the audio tag can use relative paths). Also, the Flash players are not scriptable in the same way as inbuilt audio tag is, which can make doing tricky stuff like animating other content in response to the audio more difficult.

The second problem with the audio tag is the same codec problem I talked about in a previous rant (The HTML5 Video Tag’s Fatal Flaw) For legal reasons, different browsers play different formats of audio – most notably Firefox will not play mp3s while Safari will not play ogg. There is no single format that will play in all browsers except for uncompressed wavs, which are too fat to be useful except for very short snippets.

To get around this problem the audio tag allows multiple files to be specified. The first file that the browser thinks it can play will be used, but it does mean you have to encode and store multiple versions of each audio file.

1
2
3
4
5
<!-- Only one of these files will be downloaded -->
<audio controls>
	<source src="music.ogg" type="audio/ogg">
	<source src="music.mp3" type="audio/mpeg">
</audio>

The demo page also uses the audio tag to play sound effects in the background, using audio elements that do not have a user interface. For simplicity I used wav files (download from this awesome source of free effects.) Since they have no user interface, Javascript must be used to play them:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<audio id="clicksound" preload="auto">
	<source src="click.wav" type="audio/wav">
</audio>
 
<script type="text/javascript">
function playSound( )
{
	var a = document.getElementById( "clicksound" );
	if ( !a ) return;
	if ( !a.play ) return; // will exit if the browser does not understand the audio tag
 
	a.play();
}
</script>

It is all pretty simple but as always there are problems. I did not find a good way of replicating this using Flash, so browsers that do not understand the audio tag do not play these background noises. Also, Google Chrome (which has otherwise excellent support) contains a weird bug that prevents it playing the first couple of seconds of an audio file, making it useless for short sounds. Apparently Firefox3.5 had the same problem, but it works perfectly in 3.6.

I created the demo to see if the audio tag could replicate the functionality of Flash-based applications for both long-form audio and background sound effects. It does seem to be possible provided you are targeting a modern browser and are prepared to work around certain annoyances. Hopefully the next few years will see an improvement in support for audio, I can see many uses for it especially if the iPad (which does not support Flash) takes off.

Safari 4 is Pretty Good

Safari 4 has been out for a couple of days now, and I must say I am enjoying using it. On the Mac, Safari has always had a great overall browser experience but Firefox always managed to stay my weapon of choice for viewing the Internet. This may change, Safari 4 is a very nice piece of software.

Apple is clearly wanting to make Safari an integral part of the Mac experience – Safari is very well integrated into the Mac OS (you can use Spotlight to search your history, passwords are stored in the keychain, etc), and the UI has all the polish you would expect from Apple. I particularly like the graphical Top Sites view that Safari presents when you first start it.

One thing I have always liked about Safari is the way it renders text and graphics. It always seemed to be just that little bit more polished than other browsers – correctly anti-aliasing fonts and respecting the embedded colour profiles of images. In my opinion Safari is still the best looking browser.

Apple are making a big song-and-dance about how Safari 4 is much faster than other browsers. What they mean is that they have included a very good Javascript JIT compiler which speeds up script-heavy sites by a large margin. This is excellent news, but Safari is hardly alone in focusing on Javascript performance and very recent versions of other browsers have very similar performance.

On Windows, Safari is something of an oddity. It works just as well and still renders sites better than most other Windows browsers, but its awkward neither-Mac-fish-or-Windows-fowl UI doesn’t help. The Windows version also lacks the smooth GUI animation that makes the Mac version so pleasant to use. Apple are doing their best to push Safari onto iTunes users, but I can’t see it taking off except among web developers, Safari’s built in web development tools are very cool.

There are still a few problem areas. Safari doesn’t seem to enjoy displaying animated GIFs, which often stutter before they are fully loaded. It very occasionally beachballs for a second on some pages, not all the time but enough to be annoying.

Finally, another rant about the HTML video tag (since the last one got quite a bit of attention): Safari supports the video tag but farms out the video to Quicktime. I suppose this is better than nothing, but you can tell that it is not well integrated. Videos do not show up in the list of page assets and it is clear that Quicktime is downloading the video itself, bypassing the browser’s cache. On top of this, performance is quite poor when the Javascript controls are used. The video quality is top-notch, but the experience is disappointing.

It sounds like I am dumping on Safari, but really they are minor niggles in a sea of greatness. I still think Firefox has the edge (particular Firefox3.5, which is shaping up nicely) but you could do worse.

The HTML5 Video Tag’s Fatal Flaw

Back in the day there was no standard way to publish video on the web. You could put any kind of video file you wanted on the server, but there was no guarantee that your readers would have the correct plugin required to view it. Everyone had to have a bunch of plugins installed to have any hope of viewing the majority of video files.

Flash video solve this problem. Flash was installed on nearly every computer anyway, so once they added a video decoder it seemed obvious to provide video content in Flash, even if it was in many ways not as good as the older plugins. Flash video uses massive amounts of processor time and slows down everything else on your computer. On the other hand, websites like Flash because it is easy to skin the player to fit in with the look of the site, and it makes downloading the raw video file (slightly) more difficult.

The <video> tag is supposed to replace Flash by linking to video files in the same way that the <img> tag links to images. In practice this is more complicated than it sounds because videos typically require the ability to skip and rewind content. This means that the browser must be prepared to download different parts of the file and cache things carefully to maintain performance. But these problems have long been solved.

I have been waiting for the big sites to make announcements, and today seems to be <video> day all over the internet. Both YouTube and Dailymotion have demo pages up showing <video> content:

YouTube’s HTML5 Page is designed to show how the <video> tag can replicate the functionality of their famous Flash-based player exactly. Unless you looked at the source (or your OS’s process monitor) you would never know you were using a different player.

The Dailymotion HTML5 Demo is even more impressive, using the <video> tag in combination with fancy Javascript to post-process the video and extract frames.

All this is very cool, but the two demos reveal the video tag’s fatal flaw : codecs. When the video tag was proposed, all browsers were supposed to support an unencumbered decoder named Ogg Theora (no seriously, that’s its name). There were just three problems:

  • Ogg Theora’s quality was not as good as other codecs
  • Although Ogg Theora was designed to be free of patent issues, it was felt that it may be a bit of a lightning rod for litigation.
  • Certain companies may have a vested interest in seeing their own codecs used.

So the requirement to support Ogg Theora was dropped. This means that although all HTML5 browsers will support <video>, there is no guarantee that they will be able to play and particular file. Firefox (at least the 3.5 beta) plays Ogg Theora, but Safari plays H.264 (a superior but expensive to license codec) but not vice versa. For instance, one of the demos above plays in Firefox, the other plays in Safari. This puts us in the farcical situation of having no standard way to publish video, exactly where we started.

There is also the small point that the most widely used browser (IE) does not support the video tag, and probably won’t for years. I predict that Flash video will be around for a while yet, and I am not happy about it.

Chrome

So Google is going where angels fear to tread, and has released a browser of their own: Chrome. This is a very interesting move; I can think of a few reasons why Google might think a custom browser might be a good idea:

  • although everyone thinks of Google as a search engine they are really in the cryto-marketing field. Firefox plugins that remove ads must be a worrying development for them.
  • anything that helps people view more web pages is a win for Google, since more pages viewed equals more ad impressions.
  • having their own browser gets them a seat at the table when new web-standards are being created.
  • Google has a vested interest in promoting internet commerce, releasing a secure browser with safety features built in supports that goal.
  • most browsers have a search box that redirects to Google or another search engine. Chrome has a search/location bar that only redirects to Google – AdWords ahoy!
  • perhaps even sneakier, the search bar widget uses Google Suggest, so it is sending back information to Google as you type. I am sure Google can think of useful things to do with this information.
  • the search/location widget is a very interesting idea in its own right, effectively minimising the utility of domain names (which can be spoofed or camped.) Expect to see lawsuits fly when people work that their expensive .com domain name is not worth as much as they thought if this idea catches on.

I used Chrome for a couple of hours today at work (luckily I am in a line of work were this is not considered goofing off.) The interface is very clean and slick, even better than Safari. Having the location bar as part of the tab rather than above it make a lot of sense, and Chrome does it much better than IE, which I have always found visually confusing. I also like the way that Chrome uses the window title bar when full screen, giving you an extra few pixels of vertical height.

Chrome uses the infamous WebKit HTML layout engine, as seen in Safari and various Linux browsers. It is very fast at complex pages and supports all the fancy -webkit extensions to CSS. Sadly it does not include the excellent graphics renderer that comes with Safari, so fonts and images still look jaggy. Also, no support for @font-face – come on!

The network code seems pretty tight. Browsing through proxies worked well (a lot better than Safari) although it never seemed to do automatic NTLM authentication like IE.

There are a few omissions. Weirdly, Java applets do not seem to be supported. I am not sure if that is deliberate, or just something they haven’t gotten around to yet. No Mac version as yet, although it is apparently on the way.

Finally I must observe that commissioning a comic book for announce a product launch is one of the weirdly cool things I have ever seen.

A new font

To celebrate my blog reaching 100 published posts, I decided to buy it a new font. After researching font creation for a couple of days, I decided that it was all too complicated and sought professional help: Fontifier is a web service that takes the pain (and believe me it is a pain) out of making a font. For the princely sum of 9 US dollars and 10 minutes effort you too can have a reasonable looking typeface.

The font I got back after uploading my sample sheet was OK, but I wasn’t happy with some of the letterform and the kerning was not very good. Enter FontForge, an open-source font editor. Although a little buggy and a lot hideous, FontForge let me fix up the problems without too much trouble.

And now for a rant about embedded fonts: I see today that Microsoft is trying to resurrect their EOT font embedding technology. This was the next big thing 10 years ago – a way for the browsers to download custom fonts for display while supposedly protecting the font from being pirated. What it is in practice is a non-standard font format that is a pain in the neck for legitimate users and no hinderance to the fiendish font-pirates at all. Everybody saw this 10 years ago, which is why you have never seen a page with embedded EOT fonts even though the technology has been around for a decade – nobody can be bothered.

The upshot is that IE8 will not support standard OpenType fonts in stylesheets. This is terrible news – if you are viewing this page using Safari on the Mac then you are seeing text rendered in my handwriting. This is purely decorative (it is so, shut up!) but I can think of several more practical reasons why a page may want to embed a custom font – think mathematical equations or hieroglyphs. With two completely non-compatible font formats, few authors are going to make the effort, and everybody loses.