HTML5 Rocks

Let's face it, AppCache is annoying and has problems [1, 2, 3]. One big limitation is the fact that it's impossible to dynamically cache assets on demand. Essentially, this makes it an all or nothing thing when it comes to taking an app offline. Either everything in the manifest is cached up front, or nothing is cached.

The HTML5 FileSystem API becomes an attractive solution for AppCache's shortcomings. One can programmatically store files and folder hierarchies in the local (sandboxed) filesystem and subsequently add/update/remove individual resources as necessary. My colleague, Boris Smus, even wrote a nice library to manage this type of offline caching in the context of games. The same idea can be extrapolated to work with any type of web app.

crbug.com/89271 is an important fix for the FileSystem API which makes relative filesystem: URL paths work like a charm.

Let's say for example, I've saved index.html in the filesystem's root folder (fs.root), created an img folder, and saved "test.png" in it. The filesystem: URL for those two files would be filesystem:http://example.com/temporary/index.html and filesystem:http://example.com/temporary/img/test.png, respectively. Then, if I wanted to use "test.png" for an img.src, I needed to use its full absolute path: <img src="filesystem:http://example.com/temporary/img/test.png">. That meant rewriting all of the relative urls in index.html to point to corresponding file's filesystem: URL. Not cool! Now, with this bug fix, I can keep the relative path to the file (<img src="img/test.png">) as they will resolve correctly to a filesystem origin.

This feature makes it trivial to pull down a page and save all of its resources offline, while still preserving the exact same folder structure as the online version.

Comments | permalink

There’s various offline related features introduced to modern browsers through HTML5. While offline is convenient, its concept of quota has been left untouched for a long time. The latest version of Chrome browser has the first concept and its implementation of Quota Management API . It handles quota for AppCache, IndexedDB, WebSQL and File System API. Here’s a list of things you should keep in mind when working with Quota Management API in the latest Chrome.

(The specific numbers and details noted below are not a part of HTML5 but the facts in the current Chrome implementation. The API and Quota management in Chrome is still evolving and the details may change over the time.)

• On HTML5 storage, there’s a notion of TEMPORARY storage and PERSISTENT storage.
  • TEMPORARY storage can be used without requesting quota, but may be deleted at the browser’s discretion.
  • PERSISTENT storage is never deleted without the user’s instruction, but usually requires up-front quota request to use.

• For TEMPORARY storage, it is shared between all applications and websites run in the browser.
  • TEMPORARY storage has a default quota of 50% of available disk as a shared pool. (50GB => 25GB) (Not restricted to 1GB anymore)
    [To be more specific, TEMP quota is calucalated by (remaining disk space + TEMP storage space) * 50%. Therefore if apps are using 25GB TEMP storage in total and the current remaining disk space is 25GB that’s already full.]
  • Each application has a limitation to have 20% of the available TEMPORARY storage pool (i.e. 20% of 50% of available disk). (Not restricted to 5Mb anymore)
  • When TEMPORARY storage quota is exceeded, all the data (incl. AppCache, IndexedDB, WebSQL, File System API) stored for oldest used origin gets deleted .
    [Note: since each app can only use up to 20% of the pool, this won’t happen very frequently unless the user has more than 5 active offline apps.]
  • If app tries to make a modification which will result in exceeding TEMPORARY storage quota (20% of pool), an error will be thrown.
  • Each application can query how much data is stored or how much more space is available for the app by calling queryUsageAndQuota() method of Quota API.
  • Requesting more quota against TEMPORARY storage doesn’t do anything.
  • Requesting quota for TEMPORARY storage using webkitRequestFileSystem() doesn’t actually allocate / change quota.

• webkitStorageInfo.queryUsageAndQuota( 
      webkitStorageInfo.TEMPORARY,   // or PERSISTENT 
      usageCallback, 
      errorCallback);

• For PERSISTENT storage, its default quota is 0 and it needs to be explicitly requested per application using requestQuota() of Quota API.
  • The allocated space can be used only by File System API (for PERSISTENT type filesystem) and there’s no such thing like PERSISTENT storage on IndexedDB, WebSQL DB or AppCache (yet).

• webkitStorageInfo.requestQuota( 
      webkitStorageInfo.PERSISTENT
      newQuotaInBytes,
      quotaCallback,
      errorCallback);

• unlimitedStorage on manifest.json of Chrome Web App has been brought as a temporary solution for apps to work without Quota API. So, there’s no guarantee that Chrome will support this feature forever.

• The API is described in WebIDL here: https://groups.google.com/a/chromium.org/group/chromium-html5/msg/5261d24266ba4366?pli=1

Comments | permalink

As you know, the Application Cache helps you run your web app offline, significantly speeds up startup time, and reduces bandwidth bills. All good things to the HTML5 developer!

However, for large apps and sites, it can be cumbersome to add each and every file required for the App Cache. Luckily, there are now tools that can help you auto-generate your App Cache manifest.

Some options include:

Manifested
ManifestR
H5BP build script

Give them a shot, let us know in the comments how they work for you.

Comments | permalink

What is IndexedDB? IndexedDB is an evolving web standard for storage of significant amounts of structured data in the browser and for high performance searches on this data using indexes. In other words, IndexedDB is an object store. It is not the same as a relational database, which has tables with collections rows and columns. It is an important and fundamental difference that affects the way that you design and build your applications (more on the - basic concepts).

So what is new? Changes my friends... we have some changes that are going to throw some errors if we don't handle them with simple syntax change.

From version 17 onwards, Chrome will now throw an error if an IndexedDB transaction is not scoped to an object store. Since all reading and writing of data are done within transactions, we need to create a transaction on a database, specify the scope (such as which object stores you want to access) and determine the kind of access (read only or write).

What does it means in code? Well, instead of passing an empty array to our database.transaction:

var transaction = db.transaction([], IDBTransaction.READ_ONLY);

You should scope to a particular object store, or list of object stores:

// all stores (equivalent to what use to be marked as empty array. )
var transaction = db.transaction(db.objectStoreNames, IDBTransaction.READ_ONLY);

// multiple stores:
var transaction = db.transaction(['ObjectStoreName1', 'ObjectStoreName2'],
    IDBTransaction.READ_ONLY);

// single store - these are equivalent
var transaction = db.transaction(['ObjectStoreName'], IDBTransaction.READ_ONLY);
var transaction = db.transaction('ObjectStoreName', IDBTransaction.READ_ONLY);

You can speed up data access by using the right scope and mode in the transaction. Here's a couple of tips: When defining the scope, specify only the object stores you need. This way, you can run multiple transactions with non-overlapping scopes concurrently. Only specify a READ_WRITE transaction mode when necessary. You can concurrently run multiple READ_ONLY transactions with overlapping scopes, but you can have only one READ_WRITE transaction for an object store.

Other sources:

• The update on chromium.org
• More details on IndexedDB at MDN
• A full example on how to use indexedDB in a simple ToDo list web app

So until next time... keep pushing the web to near territories.

Comments | permalink

If you have a File object (say, one stored using the FileSystem API), it's possible to seek into it and read chunks without reading the entire file into memory:

var url = "filesystem:http://example.com/temporary/myfile.zip";

window.webkitResolveLocalFileSystemURL(url, function(fileEntry) {
  fileEntry.file(function(file) {
    var reader = new FileReader();

    reader.onload = function(e) {
      var ab = e.target.result; // arrayBuffer containing bytes 0-10 of file.
      var uInt8Arr = new Uint8Array(ab);
      ...
    };

    var blob = file.webkitSlice(0, 10, "application/zip");  // mimetype is optional
    reader.readAsArrayBuffer(blob);
  }, errorHandler);
}, errorHandler);

Comments | permalink

With the offline APIs in HTML5, there's no excuse not to provide a flawless offline experience for users. One thing that can help this story is the navigator.onLine property; a feature that recently landed in Chrome dev channel. This property returns true or false depending on whether or not the app has network connectivity:

if (navigator.onLine) {
  console.log('ONLINE!');
} else {
  console.log('Connection flaky');
}

A web app can also listen for online and offline events to determine when the connection is available again or when an app goes offline:

window.addEventListener('online', function(e) {
  // Re-sync data with server.
}, false);

window.addEventListener('offline', function(e) {
  // Queue up events for server.
}, false);

I've posted a working demo at http://html5-demos.appspot.com/static/navigator.onLine.html and more information on offline events can be found in the MDN.

Comments | permalink