Introducing HTML5 features… now! (Part one)

We think the HTML5 specification, published and maintained by the WHATWG, and the Open Web initiative are beautiful entry points to jump start a new generation of Internet applications. There is one downside to these endeavors: we web developers need to wait for browser vendors to implement features like the <audio> tag, <video> tag, undo/ redo, local storage, cross-domain XHR, validation, etc. And even when these features are implemented inside the new browser versions, we still need to wait for them to hit mainstream and replace the old versions. But we don’t want to wait, we want them now!

Note: The screencast software I’m using seemed to flicker a bit. I’m a first time user, I’ll hope to get it fixed next time.

The last couple of months we at Javeline have been working hard to get the open source Javeline Platform (JPF) ready for this new generation. The first showcase I present to you here is an implementation of the HTML5 <video> and <audio> elements, with two key differences:

1. The ‘j’ - namespace, so they are called <j:audio> and <j:video>. This difference is not really shocking, I know, but quite essential, because it enables a lot of features that Javeline Platform can provide. You can see it in action in the screencast below.
2. Since we cannot wait for the right A/V codec for the browser to stand up, we have to support them all: Apple Quicktime for .mov files and podcasts for example, Silverlight and Windows Media Player for .wmv files and streaming WMP, Flash for .flv and .mp3 files and VLC for .ogg files. In fact, VLC is also the default player for Linux clients, because it features the only reasonably functioning browser-plugin on that platform.

So how does it work?

Well, the basic syntax for a standard <j:video> element looks like this:

<j:video 
  id  = "myVideo1" 
  src = "http://url_to_your_video.mov">  
    Video Codec not supported. 
</j:video>

And the code for a standard <j:audio> element looks like this:

<j:audio 
  id  = "myAudio1" 
  src = "http://url_to_your_audio.mp3">  
    Audio Codec not supported. 
</j:audio>

Basically what I’ve done is

• implemented the HTMLMediaElement interface
• gathered all the knowledge that is available about scripting the media player controls that can play media inside browsers and implemented those API’s as a layer on top of the HTMLMediaElement interface.

The result is a cross-browser and cross-platform solution for embedding rich media into your web application or static webpage - like on this blog - that can be controlled fully with Javascript! In other words: the <audio> and <video> tags are now a real thing, whereas you - webdeveloper - don’t need to wait any longer. We want HTML5 and we want it now!
At the bottom of this article you will find a full demonstration of the <j:video> tag. It shows a screencast with me explaining how to use the <j:video> and <j:audio> for your own web application(s).

Next, I will show you how to add player controls to both <j:video> and <j:audio> elements. Since they share the same ancestor - the HTMLMediaElement interface - I can show the player controls for the <j:video> example first and simply reuse it for the <j:audio> example.

Playback control.

The HTMLMediaElement specifies a couple of accessor methods that enable developers to control the playback of a media file, like ‘play()’, ‘pause()’ and ‘canPlayType()‘.
Example: say you want to give a visitor basic control over the video and/ or audio files you provide, but want them to have the same look and feel as your web application. From a usability and interface design perspective this is quite important. Since you can now control the <j:video> and <j:audio> elements with simple accessor methods you may use your own button widgets to add a Play and Pause button. The onclick event would in this case fire ‘myVideo1.play()’ and ‘myVideo1.pause()’ respectively. Really, it’s that simple:

<j:video id = "myVideo1" 
  src      = "http://url_to_your_video.mov" 
  autoplay = "true" 
  controls = "false" 
  volume   = "90">  
    Video Codec not supported. 
</j:video> 
 
<j:button 
  background = "{!myVideo1.paused?'pause.png':'play.png'}" 
  onclick    = "myVideo1.paused ? myVideo1.play() : myVideo1.pause()" /> 
 
<j:label value = "{myVideo1.getCounter(myVideo1.currentTime, '%M:%S')}" /> 
 
<j:slider 
  realtime = "false" 
  value    = "[myVideo1.position]" /> 
 
<j:label 
  value = "{myVideo1.getCounter(myVideo1.currentTime, '%M:%S', true)}" 
  class = "jpf_countdown" /> 
 
<j:label 
  value = "{Math.round(myVideo1.volume)}" />
 
<j:slider 
  width = "80" 
  value = "[myVideo1.volume]" 
  min   = "0" 
  max   = "100" />

You can use the ‘getCounter()’ function to transform the currentTime from milliseconds to a human readable format. The modifier ‘%M:%S’ displays a currentTime value of 360,000 as ‘06:00′. A third argument specified as ‘true’ (Boolean) will return a countdown.

Javeline Platform supports two-way property binding. In other words: your controls are not limited to listening to changing properties, but may also change it themselves. You can now grab the knob of the slider control and seek through the movie or sound! I’d say give it a whirl, it’s quite fun to do once you know it’s Javascript doing the dirty work…

// Curly brackets: one way property binding (read-only)
{Math.round(myVideo1.volume)} 
 
// Square brackets: two way property binding (read-write)
[myVideo1.position]

Let’s add some style

Styling in JPF is done with skins, which is something I would love to explain in a later article, but for now I’ll stick with the notion of their existence.
When I apply some basic styling rules, the code will look like this:

<j:bar resizable="true" width="482" height="353" minwidth="400" 
  minheight="250" xmlns:j="http://www.javeline.com/2005/jml">
    <j:video id = "myVideo1" 
      src      = "http://url_to_your_video.mov" 
      autoplay = "true" 
      controls = "false" 
      volume   = "90"
      left = "2" right = "4" top = "2" bottom = "30">  
        Video Codec not supported. 
    </j:video> 
 
    <j:button 
      background = "{!myVideo1.paused?'pause.png':'play.png'}" 
      onclick    = "myVideo1.paused ? myVideo1.play() : myVideo1.pause()" /> 
 
    <j:label 
      value = "{myVideo1.getCounter(myVideo1.currentTime, '%M:%S')}" 
      class = "jpf_counter" /> 
 
    <j:slider 
      realtime = "false" 
      skin     = "slider16" 
      value    = "[myVideo1.position]" 
      left     = "80" 
      right    = "182" /> 
 
    <j:label 
      value = "{myVideo1.getCounter(myVideo1.currentTime, '%M:%S', true)}" 
      class = "jpf_countdown" /> 
 
    <j:label 
      value="{Math.round(myVideo1.volume)}" />
 
    <j:slider 
      width = "80" 
      value = "[myVideo1.volume]" 
      min   = "0" 
      max   = "100" /> 
</j:bar>

As you might have noticed, I just added a ‘<j:bar>’ element, which basically serves as a container element for the video and player control elements. The advantages of using a <j:bar> here instead of a <div> are twofold: 1) I’m able to use a skin like with all the other elements and 2) I can now use the ‘resizable’ attribute, with which I can make the entire video player resizable with drag and drop by setting it to ‘true’. Read all about it - and more - on the next page…

ResizeMe!

Well, I’ll be short on this one: setting the ‘resizable’ attribute on any visual Javeline Markup Language (JML) element makes it resizable by dragging the borders and dropping them at their new position(s). With additional properties like ‘minwidth’, ‘minheight’, ‘maxwidth’ and ‘maxheight’ you can enforce the necessary constraints on the resizing behavior. Simply put: check out the video player in this article, grab one of its corners and drag it around!

DOM operations

Both <j:video> and <j:audio> elements can be modified and controlled in real time using standard DOM operations.
Example: say we have a list of videos that a visitor of your site can pick from. Since it is a web application, no page refresh should happen, not even when the user starts switching between videos. That’s where setAttribute comes in to play; when the user selects another video, a simple setAttribute(’src’, ‘new_video.mov’) will do the trick! You can see me showing a couple more DOM operations in the screencast below.

Cut and paste for your blog

You can copy and paste the code below and you’ll get the same result on your own pages instantly (it includes the elements in a way that works well with blog editors like wordpress and blogspot). What’s left for you to do is build beautiful interfaces and enhance those web application with stunning rich media!

<script src="http://cdn.ajax.org/platform/1.0rc1/jpf_av5.1.0rc1.js"><!--
    <j:bar resizable="true" width="482" height="353" minwidth="400" minheight="250" maxwidth="500" xmlns:j="http://www.javeline.com/2005/jml">
        <j:video id="player1" autoplay="true" volume="50" controls="false"
          left="2" right="4" top="2" bottom="30">
            <j:source src="http://cdn.mikedeboer.nl/jaudio_jvideo_1_1.flv"></j:source>
            <j:source src="http://cdn.mikedeboer.nl/jaudio_jvideo_1_1.mov"></j:source>
            <j:source src="http://cdn.mikedeboer.nl/jaudio_jvideo_1_1.wmv" type="video/silverlight"></j:source>
            <j:nomedia>Video Codec not supported.</j:nomedia>
        </j:video>
 
        <j:button background="{!player1.paused?'pause.png':'play.png'}"
            onclick="player1.paused ? player1.play() : player1.pause()">
        </j:button>
 
        <j:label value="{player1.getCounter(player1.currentTime, '%M:%S')}" class="jpf_counter" disabled="{!player1.READY}"></j:label>
        <j:slider skin="slider16" value="[player1.position]" enabled="{player1.READY}" left="85" right="182" realtime="false"></j:slider>
        <j:label value="{player1.getCounter(player1.currentTime, '%M:%S', true)}" class="jpf_countdown" disabled="{!player1.READY}"></j:label>
 
        <j:button background="{!player1.muted ? 'volume.gif':'volume_mute.gif'}" onclick="player1.setAttribute('muted', !player1.muted)" class="jpf_mute"></j:button>
        <j:label value="{Math.round(player1.volume)}" class="jpf_volume" disabled="{player1.muted}"></j:label>
        <j:slider width="80" value="[player1.volume]" min="0" max="100" disabled="{player1.muted}"></j:slider>
    </j:bar>
 --></script><!--wordpress hack--><script type="text/javascript"></script>

Conclusion

We tried to solve the hard problems, gather all available knowledge on scripting player controls and provide them to you in a ‘ready-to-roll’ package. The final release of the video/audio tag will include auto detecting available plugins instead of only trying the preferred plugin for the specified format. This will solve all the multi os problems that are there.

I would love to hear all comments and feedback you have. As the HTML5 spec is subject to change almost every day, I think that the <j:video> and <j:audio> elements will be subject to change as well. I hope this showcase will spur more discussion about the next generation of browser interfaces and we at Javeline would love to chime in with this little contribution.

More to come…

As I mentioned earlier in this article, we’ve been working hard and implemented quite a few more features of the HTML5 specification in the upcoming JPF 1.0 RC1 version, which is due to be released soon, so stay tuned!