HTML5 Rocks

HTML5 Rocks

Web Audio live audio input - now on Android!

By Chris Wilson at

One of the consistent questions I keep fielding over on Stack Overflow is “why doesn’t audio input work?” – to which the answer kept turning out to be “because you’re testing on Android, and we don’t have it hooked up yet.”

Well, I’m happy to announce that the Beta version of Chrome for Android (v31.0.1650+) has support for audio input in Web Audio! Check out my Audio Recorder demo on an Android device with Chrome Beta. We’re still testing it out with all devices; I’ve personally tested on a Nexus 4, a Galaxy S4 and a Nexus 7, but if you run into issues on other devices, please file them.

When I saw the support get checked in, I flipped back through some of the audio input demos I’ve done in the past to find a good demo to show it off. I quickly found that my Audio Recorder demo functioned well on mobile, but it wasn’t really designed for a good user experience on mobile devices.

So, I quickly used the skills I’m teaching in the upcoming Mobile Web Development course to whip it into shape – viewport, media queries, and flexbox to the rescue! Be sure to preregister for the course if you’re interested in taking your web development skills to the mobile world, too!

Live Web Audio Input Enabled!

By Chris Wilson at

I'm really excited by a new feature that went in to yesterday's Chrome Canary build (23.0.1270.0) - the ability to get low-latency access to live audio from a microphone or other audio input on OSX! (This has not yet been enabled on Windows - but don't worry, we're working on it!)

[UPDATE Oct 8, 2012: live audio input is now enabled for Windows, as long as the input and output device are using the same sample rate!]

To enable this, you need to go into chrome://flags/ and enable the "Web Audio Input" item near the bottom, and relaunch the browser; now you're ready to roll! Note: If you're using a microphone, you may need to use headphones for any output in order to avoid feedback. If you are using a different audio source, such as a guitar or external audio feed, or there's no audio output from the demo, this may not be a problem. You can test out live audio input by checking out the spectrum of your input using the live input visualizer.

For those Web Audio coders among you, here's how to request the audio input stream, and get a node to connect to any processing graph you like!

// success callback when requesting audio input stream
function gotStream(stream) {
    window.AudioContext = window.AudioContext || window.webkitAudioContext;
    var audioContext = new AudioContext();

    // Create an AudioNode from the stream.
    var mediaStreamSource = audioContext.createMediaStreamSource( stream );

    // Connect it to the destination to hear yourself (or any other node for processing!)
    mediaStreamSource.connect( audioContext.destination );
}

navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia;
navigator.getUserMedia( {audio:true}, gotStream );

There are many rich possibilities for low-latency audio input, particularly in the musical space. You can see a quick example of how to make use of this in a simple pitch detector I threw together - try plugging in a guitar, or even just whistling into the microphone.

And, as promised, I've added live audio as an input source to the Vocoder I wrote for Google IO - just select "live input" under modulator. You may need to adjust the Modulator Gain and the Synth Level. There's a slight lag due to processing (not due to input latency). Now that I have live audio input, it's time for another round of tweaking!

Finally, you may want to take a look at the collection of my web audio demos - by the time you read this, I may have some more live audio demos up!

HTML5 <audio> and the Web Audio API are BFFs!

By Eric Bidelman at

DEMO

As part of the MediaStream Integration with WebRTC, the Web Audio API recently landed an undercover gem known as createMediaElementSource(). Basically, it allows you to hook up an HTML5 <audio> element as the input source to the API. In layman's terms...you can visualize HTML5 audio, do realtime sound mutations, filtering, etc!

Normally, the Web Audio API works by loading a song via XHR2, file input, whatever,....and you're off. Instead, this hook allows you to combine HTML5 <audio> with the visualization, filter, and processing power of the Web Audio API.

Integrating with<audio> is ideal for streaming fairly long audio assets. Say your file is 2 hours long. You don't want to decode that entire thing! It's also interesting if you want to build a high-level media player API (and UI) for play/pause/seek, but wish to apply some additional processing/analysis.

Here's what it looks like:

// Create an <audio> element dynamically.
var audio = new Audio();
audio.src = 'myfile.mp3';
audio.controls = true;
audio.autoplay = true;
document.body.appendChild(audio);

var context = new webkitAudioContext();
var analyser = context.createAnalyser();

// Wait for window.onload to fire. See crbug.com/112368
window.addEventListener('load', function(e) {
  // Our <audio> element will be the audio source.
  var source = context.createMediaElementSource(audio);
  source.connect(analyser);
  analyser.connect(context.destination);

  // ...call requestAnimationFrame() and render the analyser's output to canvas.
}, false);

As noted in the code, there's a bug that requires the source setup to happen after window.onload.

The next logical step is to fix crbub.com/112367. Once that puppy is ready, you'll be able to wire up WebRTC (the navigator.getUserMedia() API in particular) to pipe audio input (e.g mic, mixer, guitar) to an <audio> tag, then visualize it using the Web Audio API. Mega boom!