Web Storage Size Quota Test

For an introduction to Web Storage, look at my previous post.

Check out the production version on GitHub!

Web Storage is a key feature in my current project, which involves video buffering. But each browser does not support the same amount of web storage, and some even differ in the amount of localStorage and sessionStorage available.

A site aggregates data based on each browser to provide a baseline for each browser. This is useful information, but there I have some serious reservations about using this in a production environment. Mainly because client side user-agent detection is flawed at best and there is an abundance of different browsers and versions in the wild. The only real option was to develop a quick test for each browser, or to live dangerous and handle the exceptions raised when adding too much data (for non-IE browsers). I found a couple libraries, but waiting 2 seconds to determine a cache size would be unacceptable. So I wrote my own, decreasing run time by 10x.

First, I determine if the browser is IE. Again, user-agent detection is not the best means. IE is the only browser that supports .remainingSpace and does not throw when the size has been exceeded. This bool is important for a logic gate when adding to Web Storage.

//remaining space is only available in MSIE
var isIE = localStorage.remainingSpace !== 'undefined';

Next, the array of test strings must be created. These strings are of different lengths and avoid a costly penalty of string copying due to concatenation.

//create the array of different sized objects. hold the length to prevent costly calls to array length property
var arr = buildArray(); 
var length = arr.length;    

function buildArray()
{
    //build array, these will be the values that will be added to the web storage
    var b1 = "0";
    var b10 = increaseLength(b1, 10);
    var b100 = increaseLength(b10, 10);
    var kb1 = increaseLength(b100, 10);
    var kb10 = increaseLength(kb1, 10);
    var kb100 = increaseLength(kb10, 10);
    var mb1 = increaseLength(kb100, 10);
    var mb10 = increaseLength(mb1, 10);

    //return array of various sizes, ordered smallest to largest
    return [b1, b10, b100, kb1, kb10, kb100, mb1, mb10];
}

function increaseLength(string, times)
{
    var temp = [];
    while (times--)
        temp.push(string);
    return temp.join('');    
}  

Now the test can begin, this version tests the three flavors of Web Storage. Some browsers will have different quotas for localStorage and sessionStorage. globalStorage is largely deprecated, and has been replaced by localStorage. Results are posted to a crudely constructed HTML scaffold.

//array of the types of Web Storage
var storage_types = ["localStorage", "sessionStorage", "globalStorage"];

//check for web storage
for (i = 0; i < storage_types.length; i++)
{
    //verify the browser supports this form of web storage
    if (supportsStorage(storage_types[i]))
    {
        //start the stopwatch to time how long the test takes
        var sw = new StopWatch();
        sw.start();

        //clear everything from web storage before starting
        window[storage_types[i]].clear();

        //iterate to find the maximum amount of data that can be stored
        checkSize(storage_types[i]);

        //stop the stopwatch    
        sw.stop();   

        //print the results 
        document.getElementById(storage_types[i] + 'support').innerHTML = storage_types[i] + " is supported by your browser over this protocol";
        document.getElementById(storage_types[i] + 'results').innerHTML = storageSize(storage_types[i]) + " Bytes available for " + storage_types[i] + " on this browser";
        document.getElementById(storage_types[i] + 'time').innerHTML = sw.elapsedTime + " [ms] to test browser " + storage_types[i];

        //clear everything from web storage before starting next iteration (Firefox fix)
        window[storage_types[i]].clear();
    }
    else
    {
        //print the results
        document.getElementById(storage_types[i] + 'support').innerHTML = storage_types[i] + " is not supported by your browser over this protocol";   
        document.getElementById(storage_types[i] + 'results').innerHTML = "N/A";
        document.getElementById(storage_types[i] + 'time').innerHTML = "N/A";
    }
}

The check size function is where the real optimization is. It iterates over the array of test strings, starting at the largest first. Since Web Storage will throw if the size quota has been exceeded, starting at a large value is advantageous for eating up large amounts of space quickly. It will add to the Web Storage until it exceeds the size, then move on to a smaller unit. It does this until there is no space in the Web Storage to add even another byte. Since some browsers allow for an "unlimited" storage quota, there must be something preventing an infinite loop. Counting an iterator is far lest costly than checking the length of the Web Storage object via JSON.stringify, so after 2 additions of the 10MB string we can consider that "unlimited".

function checkSize(type)
{
    //create a key for each storage entry
    var iterator = 0;

    //count the iterations for each entry, this will be used for the "unlimited" cases, which would cause an infinite loop
    //the iterator counter eliminates the need to stringify the entire storage on each iteration and will break at ~20MB
    var arr_iterator = 0;

    //iterate over the array, from largest object to smallest
    for (j = length - 1; j >= 0; j--)
    {
        //reset array iterator
        arr_iterator = 0;

        //iterate until the data can no longer be added to the the web storage
        while (addData(type, iterator++, arr[j]))
        {
            //increment the iterator
            arr_iterator++;

            //if we have added ~20MB, then this is considered "unlimited"
            if (j == length - 1 && arr_iterator >= 2)
                return;
        }
    }
}   

The addData method does not check the current size (outside of IE, which does not throw), and just handles the excess size error if it occurs.

function addData(type, key, data)
{
    //add data to new key, or replace value if the key already exists with new data
    try
    {
        //error isn't thrown in MSIE
        if (isIE)
        {
            if (getRemainingSpace(type) - data.length <= 0)
                return false;
        }

        //add the value to the key, equivalent to localStorage.setItem('key', 'value')
        window[type][key] = data;
        return true;
    }
    catch (e)
    {
        return false;
    }
} 

The remaining helper methods: getRemaininSpace is used for checking the size in IE. Storage size utilizes JSON.stringify to get the size of the Web Storage object. And StopWatch counts the number of milliseconds to run the test.

function getRemainingSpace(type)
{
    //return the number of bytes still available in localStorage
    if (isIE)
        //only IE supports this function
        return window[type].remainingSpace; 
}

function storageSize(type)
{
    //use stringify to determine the size of the web storage object
    return JSON.stringify(window[type]).length;
}

function StopWatch()
{
    //object that will be used to time our tests
    this.startTime = false;
    this.stopTime = false;
    this.elapsedTime = false;

    this.start = function()
    {
        this.startTime = new Date().getTime();
    }
    this.stop = function()
    {
        this.stopTime = new Date().getTime();
        this.elapsedTime = this.stopTime - this.startTime;
    }
}    

This test executes (on my machine) in under 200ms in all browser's I've tested within. The results are the same as those that I found on Web Storage Support Test. The real boon here is the speed at which it takes down the cache size and the avoidance of costly properties or stringifying the dictionary.

Important Lessons!

*No jQuery required.

*IE will not throw when the size quota has been exceeded, but does offer a handy remaining space function (window[{storageType}].remainingSpace).

*Firefox shares localStorage and sessionStorage, by this I mean that if you have 4MB in localStorage then 1MB is available for sessionStorage. Other browsers allow for the double dip.

*Some browsers allow for unlimited sessionStorage, Safari (at least for Windows) is one of these browsers.

*User-Agent detection is not always reliable. Checking for features is usually a better option. For this particular application, only IE has had strange behavior.