HTML5 Rocks

HTML5 Rocks

Chrome Dev Summit: Open Web Platform Summary

By Sam Dutton at

Blink

by Greg Simon & Eric Seidel

Blink is Chrome's open-source rendering engine. The Blink team is evolving the web and addressing the issues encountered by developers.

There have been a number of behind-the-scenes improvements started since our April launch.

First thing we did was to delete half our source, which we didn't necessarily need. We're still not done! And we're not doing this blind: code removal is based on anonymously reported aggregate statistics from Chrome users who opt in to reporting.

We publish a new developer API every six weeks: the same as Chrome's shipping schedule is.

One big change we made when we forked from Blink was to add an intents system: every time before we're going to change the web platform, we send a public announcement to Blink dev announcing our intent to add or remove a feature. Then we go off, and we code it! And then the very next day after the feature is checked in, it's already there shipping in our Canary builds. This feature is off by default, but you can turn it on using about:flags.

Then, on our public mailing list we announce an intent to ship.

At chromestatus.com you can see the features we've worked on, the features we've shipped, and those we're planning to deprecate. You can also check the Chromium Releases blog, which has links to bugs and to our tracker dashboard.

Another big change is that we're removing WebKit prefixes. The intent is not to use Blink prefixes, but to have run-time flags (and not just compile-time flags).

Android WebView has been a big challenge – but HTML5Test shows that things are getting better. We're much closer to desktop in terms of having one set of web platform APIs everywhere (Web Audio is a great example of this!)

But how does the sausage machine work? Every single change we make to Blink is immediately run through over 30,000 tests, not to mention all the Chromium tests that run additionally later. We use 24 hour sheriffing, with thousands of bots, thousands of benchmarks, and systems that throw millions of broken web pages at our engine to make sure it doesn't fall over. We know that mobile is significantly slower, and this is something we're working hard to improve.

So what's new?

  • Web Components: check out Eric Bidelman's talk!
  • Web Animations: complex, synchronized, high performance animations that uses the GPU wherever possible
  • Partial Layout: only compute what you need!
  • CSS Grid
  • Responsive images: srcset or srcN or ?
  • Faster text autosizing, and consistent sub-pixel fonts
  • Skia, the graphic system used by Blink, is moving from GDI to DirectWrite on Windows

We want to know what you have to say!

If you feel C++ in your blood and want to write C++ with us, all of our code is open. You don't have to tell anybody or evangelize to us. You can just simply post a patch or file a bug!

Slides: Blink




Security

by Parisa Tabriz

More people are connected to the web today than ever before – and from more places.

We're connected with our laptops, phones and tablets, and probably soon enough with personal devices and accessories. We access the internet from untrusted and sometimes even hostile networks. With so much of our lives moving online, it's imperative we take steps to protect our data and our users' data.

Above all, as developers we need to understand the necessity and practicality of SSL.

What's SSL? It stands for Secure Sockets Layer, and it's a cryptographic protocol designed to provide communication security over the internet. It guarantees privacy, via encryption and integrity, to prevent snooping or tampering with your internet connection. SSL has it's flaws, but it's the leading way – and really the only way – to ensure any kind of data communication security on the internet.

According to SSL Pulse, a year ago we had about just under 15% of SSL adoption; we're now over 50% adoption.

Two acronyms:

  • TLS: for most intents and purposes the same as SSL. To be precise, SSL 3.1 was renamed to TLS, and TLS is the IETF Standard name. But they're interchangeable!

  • HTTPS: this is HTTP over SSL, just the layering of the security capabilities of SSL and standard HTTP. First the client–server handshake, using public/private key cryptography to create a shared key – which is used by the second part of the SSL protocol to encrypt communication.

Networking on the internet may feel safe, immediate and fast. It feels like we're talking directly to the website. But in reality, it's not a direct connection. Our communications go via a wifi router, an ISP, and potentially other intermediary proxies between your device and the website. Without HTTPS, all our communications is in plain text.

Trouble is, users rarely type in a full URL specifying HTTPS – or they click a link using HTTP. Worse, it's possible to mount a (wo)man-in-the-middle attack and replace HTTPS with HTTP. A tool called SSLstrip introduced in 2009 does just that. Firesheep, from 2010, just listened to opened wifi networks for cookies being sent in the clear: that meant you could listen in on chat, or log in to someone's Facebook account.

But SSL is (relatively) cheap, fast and easy to deploy (check out ssllabs.com and Ilya Grigorik's book High Performance Browser Networking). For non-commercial use, you can even get free certificates from startssl.com! Public Key Pinning is designed to give website operators a means to restrict which certificate authorities can actually issue certificates for their sites.

"In January this year (2010), Gmail switched to using HTTPS for everything by default. .. In order to do this we had to deploy no additional machines and no special hardware. On our production frontend machines, SSL accounts for < 1% of the CPU load, < 10 KB of memory per connection, and < 2% of network overhead…

If you stop reading now you only need to remember one thing: SSL is not computationally expensive any more.”

Overclocking SSL, Adam Langley (Google)

Lastly, a couple of bugs we see most commonly:

  • Mixed content: sites that use HTTP as well as HTTPS. Your user will get annoyed because they have to click a permission button to load content. (Chrome and Firefox actually bar mixed content from iframes.) Make sure that all of your resources on an HTTPS page are loaded by HTTPS, by using relative or scheme-relative URLs for example <style src="//foo.com/style.css">
  • Insecure cookies: sent in the clear via an HTTP connection. Avoid this by setting the secure attribute on cookie headers. You can also use a new "Strict Transport Security" header to require SSL Transport Security (HSTS).

Takeaways

  • If you care about the privacy and integrity of your users' data, you need to be using SSL. It's faster, easier, and cheaper than ever.
  • Avoid common implementation gotchas, like mixed content bugs or not setting the right HTTP header bits.
  • Use relative or scheme relative URLs.
  • Check out some of the new cool stuff, like HSTS and cert pinning

Slides: Got SSL?




Media APIs for the multi-device Web

by Sam Dutton & Jan Linden

Along with a proliferation of new devices and platforms on the web, we're seeing huge growth in audio, video and realtime communication. Online media is transforming the way we consume media of all kinds.

A UK government study found that 53% of adults 'media multi-task' while watching TV: using mobile devices to share and consume media. In many countries TV viewing is down and online viewing is up. In China, for example, in 2012 only 30% of households in Beijing watched TV, down from 70% in 2009. According to the W3C Highlights 2013, 'In the past year video-watching on mobile devices has doubled. This year in the US, the average time spent with digital media per day will surpass TV viewing. Viewing is no longer a passive act. In the US, 87% of entertainment consumers say they use at least one second-screen device while watching television.' According to Cisco 'video ... will be in the range of 80 to 90 percent of global consumer traffic by 2017'. That equates to nearly a million minutes of video every second.

So what do we have for web developers? An ecosystem of media APIs for the open Web: standardized, interoperable technologies that work across multiple platforms.

Takeaways

  • WebRTC provides realtime communication in the browser, and is now widely supported on mobile and desktop. In total there are already over 1.2 billion WebRTC endpoints.
  • Web Audio provides sophisticated tools for audio synthesis and processing.
  • Web MIDI, integrated with Web Audio, allows interaction with MIDI devices.
  • The audio and video elements are now supported on more than 85% of mobile and desktop browsers.
  • Media Source Extensions can be used for adaptive streaming and time shifting.
  • EME enables playback of protected content.
  • Transcripts, captions and the track element enable subtitles, captions, timed metadata, deep linking and deep search.

Slides: Media APIs for the multi-device Web

WebRTC: RTCDataChannel demo, API changes... and Chrome talks to Firefox!

By Sam Dutton at

More good news from our old friend WebRTC...

To be precise: three pieces of good news and a couple of minor API changes.

RTCDataChannel for Chrome

RTCDataChannel has been implemented in Chrome, and there's a great little demo at simpl.info/dc.

This demo shows peer-to-peer communication of arbitrary data – in less than a hundred lines of code. You'll need Chrome 25 or above for this, which at this point means Beta or Canary.

RTCDataChannel makes the most of features built into RTCPeerConnection – not least, the use if the ICE framework to get through firewalls and NATs – and has lots of potential applications where low latency is paramount: for gaming, remote desktop applications, real-time text chat and file transfer.

For more information about RTCDataChannel, take a look at Getting Started with WebRTC.

API changes

Less exciting, but still important: from Chrome 26, some RTCPeerConnection and MediaStream API properties have become getter methods:

  1. MediaStream now has the getAudioTracks() method instead of the audioTracks property, and getVideoTracks() instead of videoTracks.
  2. RTCPeerConnection now has getLocalStreams() instead of localStreams, and getRemoteStreams() instead of remoteStreams.

To get a glimpse of MediaStream in action, take a look at the simpl.info/gum getUserMedia demo. The stream variable is in global scope: examine it from the console. Likewise for RTCPeerConnection at simpl.info/pc: the RTCPeerConnection objects pc1 and pc2 are in global scope.

Chrome <=> Firefox

And in case you missed it, Chrome can now 'talk' to Firefox.

You can try this out now at webrtc.org/demo, which has full instructions, links to source code, and information about API differences.

Tip of the hat to those at Mozilla and Google who made it all happen.

Happy coding! And please let us know of any bugs, either by commenting on this post, or at crbug.com/new.

...and don't forget, you can always get up-to-date implementation information from the excellent chromestatus.com.

Screensharing with WebRTC

By Sam Dutton at

As we reported last week, there's been a lot happening lately with our old friend WebRTC.

Well... here's another first: WebRTC screensharing.

Screengrab of WebRTC screensharing extension, featuring Jake Archibald, Peter Beverloo, Paul Lewis and Sam Dutton

Here's a screencast: youtube.com/watch?v=tD0QtBUZsF4

...and here's the code: github.com/samdutton/rtcshare

In essence, we've built an experimental Chrome extension that uses RTCPeerConnection and chrome.tabCapture to share a live 'video' of a browser tab. If you want to try it out, you'll need Chrome Canary, and you'll need to enable Experimental Extension APIs on the about:flags page.

Our prototype relies heavily on the mighty apprtc.appspot.com demo and, to be frank, it's a bit of a hack! But... it's a proof of concept, and it works.

Here's how we did it:

  1. When the user clicks the extension icon (the 'record button' next to the address bar), the extension's background script background.js, appends an iframe to itself, the src of which is rtcshare.appspot.com. In background.js it's only used to get values such as token and room_key. We told you this was a hack :^}! This is a chopped and channeled version of apprtc.appspot.com. As with the apprtc example, rtcshare.appspot.com is also used for the remote client.

  2. chrome.browserAction.onClicked.addListener(function(tab) {
    	var currentMode = localStorage["capturing"];
    	var newMode = currentMode === "on" ? "off" : "on";
    
    	if (newMode === "on"){ // start capture
    		appendIframe();
    	} else { // stop capture
    		chrome.tabs.getSelected(null, function(tab){
    			localStream.stop();
    			onRemoteHangup();
    		});
    		// set icon, localStorage, etc.
    	}
    }
    
  3. When the iframe has loaded, background.js gets values from it (generated by the rtcshare.appspot.com app) and calls chrome.tabCapture.capture() to start capturing a live stream of the current tab.

  4. function appendIframe(){
    	iframe = document.createElement("iframe");
    	iframe.src="https://rtcshare.appspot.com";
    	document.body.appendChild(iframe);
    	iframe.onload = function(){
    		iframe.contentWindow.postMessage("sendConfig", "*");
    	};
    }
    
    // serialised config object messaged by iframe when it loads
    
    window.addEventListener("message", function(event) {
    	if (event.origin !== "https://rtcshare.appspot.com"){
    		return;
    	}
     	var config = JSON.parse(event.data);
    	room_link = config.room_link; // the remote peer URL
    	token = config.token; // for messaging via Channel API
    	// more parameter set from config
    );
    
    function startCapture(){
    	chrome.tabs.getSelected(null, function(tab) {
    		var selectedTabId = tab.id;
    		chrome.tabCapture.capture({audio:true, video:true}, handleCapture); // bingo!
    	});
    }
    
  5. Once the live stream is available (in other words, a live 'video' of the current tab), background.js kicks off the peer connection process, and signalling is done via rtcshare.appspot.com using XHR and Google's Channel API. All in all, it works like the apprtc demo, except that the video stream communicated to the remote peer is from chrome.tabCapture and not getUserMedia().

  6. function handleCapture(stream){
    	localStream = stream; // used by RTCPeerConnection addStream();
    	initialize(); // start signalling and peer connection process
    }
    
  7. For demo purposes, this prototype extension opens a new tab with the URL provided by rtcshare.appspot.com, which has a 'room number' query string added. Of course, this URL could be opened on another computer, in another place, and THAT might be the start of something useful!

  8. chrome.tabs.create({url: room_link});
    

We envisage a lot of interesting use cases for screensharing and, even at this early stage of development, we're impressed at how responsive and stable plugin-free tab capture and sharing can be.

As ever, we welcome your comments: about this extension and about the WebRTC APIs in general. If you want to learn more about WebRTC, check out the HTML5 Rocks article or our Quick Start Guide.

Happy hacking -- and best wishes for 2013 from everyone at HTML5R and WebRTC!

WebRTC hits Firefox, Android and iOS

By Sam Dutton at

A lot has happened with WebRTC over the last few weeks. Time for an update!

In particular, we're really excited to see WebRTC arriving on multiple browsers and platforms.

getUserMedia is available now in Chrome with no flags, as well as Opera, and Firefox Nightly/Aurora (though for Firefox you'll need to set preferences). Take a look at the cross-browser demo of getUserMedia at simpl.info/gum—and check out Chris Wilson's amazing examples of using getUserMedia as input for Web Audio.

webkitRTCPeerConnection is now in Chrome stable and it's flagless. TURN server support is available in Chrome 24 and above. There's an ultra-simple demo of Chrome's RTCPeerConnection implementation at simpl.info/pc and a great video chat application at apprtc.appspot.com. (A word of explanation about the name: after several iterations, it's currently known as webkitRTCPeerConnection. Other names and implementations have been deprecated. When the standards process has stabilised, the webkit prefix will be removed.)

WebRTC has also now been implemented for desktop in Firefox Nightly and Aurora, and for iOS and Android via the Ericsson Bowser browser.

DataChannel

DataChannel is a WebRTC API for high performance, low latency, peer-to-peer communication of arbritary data. The API is simple—similar to WebSocket—but communication occurs directly between browsers, so DataChannel can be much faster than WebSocket even if a relay (TURN) server is required (when 'hole punching' to cope with firewalls and NATs fails).

DataChannel is planned for version 25 of Chrome, behind a flag – though it may miss this version. This will be for experimentation only, may not be fully functional, and communication won't be possible with the Firefox implementation. DataChannel in later versions should be more stable and will be implemented so as to enable interaction with DataChannel in Firefox.

Firefox Nightly/Aurora supports mozGetUserMedia, mozRTCPeerConnection and DataChannel (but don't forget to set your about:config preferences!)

Here's a screenshot of DataChannel running in Firefox:

This demo is at http://mozilla.github.com/webrtc-landing/data_test.html. Here's a code snippet:

pc1.onconnection = function() {
  log("pc1 onConnection ");
  dc1 = pc1.createDataChannel("This is pc1",{}); // reliable (TCP-like)
  dc1 = pc1.createDataChannel("This is pc1",{outOfOrderAllowed: true, maxRetransmitNum: 0}); // unreliable (UDP-like)
  log("pc1 created channel " + dc1 + " binarytype = " + dc1.binaryType);
  channel = dc1;
  channel.binaryType = "blob";
  log("pc1 new binarytype = " + dc1.binaryType);

  // Since we create the datachannel, don't wait for onDataChannel!
  channel.onmessage = function(evt) {
    if (evt.data instanceof Blob) {
      fancy_log("*** pc2 sent Blob: " + evt.data + ", length=" + evt.data.size,"blue");
    } else {
      fancy_log('pc2 said: ' + evt.data, "blue");
    }
  }
  channel.onopen = function() {
    log("pc1 onopen fired for " + channel);
    channel.send("pc1 says Hello...");
    log("pc1 state: " + channel.state);
  }
  channel.onclose = function() {
    log("pc1 onclose fired");
  };
  log("pc1 state:" + channel.readyState);
      }

More information and demos for the Firefox implementation are available from the hacks.mozilla.org blog. Basic WebRTC support is due for release in Firefox 18 at the beginning of 2013, and support is planned for additional features including getUserMedia and createOffer/Answer constraints, as well as TURN (to allow communication between browsers behind firewalls).

For more information about WebRTC, see Getting Started With WebRTC. There's even a WebRTC book, available in print and several eBook formats.

Resolution Constraints

Constraints have been implemented in Chrome 24 and above. These can be used to set values for video resolution for getUserMedia() and RTCPeerConnection addStream() calls.

There's an example at simpl.info/getusermedia/constraints. Play around with different constraints by setting a breakpoint and tweaking values.

A couple of gotchas... getUserMedia constraints set in one browser tab affect constraints for all tabs opened subsequently. Setting a disallowed value for constraints gives a rather cryptic error message:

navigator.getUserMedia error:  NavigatorUserMediaError {code: 1, PERMISSION_DENIED: 1}

Likewise the error if you try to use getUserMedia from the local file system, not on a server!

Streaming screen capture

Tab Capture is now available in the Chrome Dev channel. This makes it possible to capture the visible area of the tab as a stream, which can then be used locally, or with RTCPeerConnection's addStream(). Very useful for sceencasting and web page sharing. For more information see the WebRTC Tab Content Capture proposal.

Keep us posted by commenting on this update: we'd love to hear what you're doing with these APIs.

...and don't forget to file any bugs you encounter at new.crbug.com!

WebRTC Protothon

By Ilmari Heikkinen at

On March 24th, Google hosted the world’s first WebRTC developer event, the WebRTC Protothon. The hackathon brought together developers and designers from around the world, with browser engineers at hand to give assistance when needed.

For more details, check out the WebRTC Google+ post about the event. There is also a summary document with great photos.

Check out the demos in action either through a video or see the live demos below:

Puzzlible!

We blend!

Team charades

Ping Pong