The JavaScript Variable Scope article covered the basics of variable scope in JavaScript. Today I’d like to continue that discussion, exploring a more complicated concept called hoisting, where the names of variables declared inside of a function are hoisted to the front of the function, so that they may be accessed, even before they are defined.

How do it…

Here is a simple example of hoisting:

function foo() {
    alert(a);
    var a = 1;
}
foo();

You probably expect that the code above would cause an error, because the variable a is used before it is defined. It will actually alert undefined, because the JavaScript interpreter hoists all variable names used inside a function to the front of the function making them available right away (even though they have no value yet). This means that the code above was interpreted like:

function foo() {
    var a;
    alert(a);
    a = 1;
}
foo();

Now that you know what hoisting is, here is a trickier example¹:

var foo = 1;
function bar() {
	if (!foo) {
		var foo = 10;
	}
	alert(foo);
}
bar();

This will alert 10, because the if statement does not create an execution context (as in other languages), and bar is interpreted as:

var foo = 1;
function bar() {
    var foo;
	if (!foo) {
		foo = 10;
	}
	alert(foo);
}
bar();

The same would happen, even if var foo = 10; was coded in a way that it would never execute:

var foo = 1;
function bar() {
	if (false) {
		var foo = 10;
	}
	alert(foo);
}
bar();

Here is another tricky example, using function names¹:

var foo = 1;
function bar() {
	foo = 10;
	return;
	function foo() {}
}
bar();
alert(foo);

It will alert 1, because bar is interpreted as:

var foo = 1;
function bar() {
    var foo = function() {};
	foo = 10;
	return;
}
bar();
alert(foo);

Keep in mind that hoisting does not occur during scope chain lookups from inner functions:

function foo() {
    console.log(x); // will throw an error
    function bar() {
        x = 1;
    }
    bar();
}
foo();
console.log(x);

The first console.log will throw an error, because x is not hoisted to the execution context of foo. It will be added to window after foo() is called, but not until function bar executes. It will never be added to the execution context of foo().

How it works…

JavaScript has function-level scope, which we call the execution context. Variables declared inside of an execution context are hoisted to the front of the execution context, even if the variables are declared in code that will never execute. The interpreter hoists all variable names, including function names. Variables declared with var will be undefined, while the entire function definition is hoisted. Consequently, I strongly recommend that all variables (including those used by for loops) and functions, should be declared at the front of a function, using a single var statement, to avoid any unintentional hoisting.

function foo(a, b, c) {
    var i = 1,
    	j = 4,
    	baz = "something";

    for (; i < j; i++) {
        alert(baz);
    }
}

Lastly, we should discuss the priority by which variable names enter a scope. There are four basic ways for a variable name to enter an execution context¹:

1. Language-defined: All scopes are, by default, given the names this and arguments.
2. Formal parameters: Functions can have named formal parameters, which are scoped to the body of that function.
3. Function declarations: In the form of function foo() {}.
4. Variable declarations: In the form of var foo;.

For the most part the latest variable name will replace the earlier, so if you define a formal function parameter as foo and then define a function with the same name, then the function would silently replace the formal parameter. The exception to this is variable declarations, which will replace language-defined names, but won’t replace arguments or function declarations. Hoisting allows you to accidentally do some nasty things, especially if you muck around using reserved words (aka. arguments) for variable names. For this reason, I recommend not reusing variable names, but instead always declaring a new variable when you need to store a new value, and never using reserved words.

References

1. http://www.adequatelygood.com/JavaScript-Scoping-and-Hoisting.html

JavaScript scopes variables in a unique way that often confuses developers coming from other languages. First, as discussed in the Context Binding Functions article, the this keyword is dynamic, and second, as we᾿ll discuss today, variable scopes change depending on how you declare them.

How do it…

The most basic way to declare a variable is:

var foo1 = 'bar';

The variable foo1 will be added to the current execution context (window in this case). Here it will be added to a function’s execution context:

function baz() {
    var foo2 = 'bar';
}

In this second case foo2 will only be available inside baz. However, you don’t need to include var when defining a variable:

foo1 = 'bar';

function baz() {
    foo2 = 'bar';
}

This code adds both foo1 and foo2 to the window execution context, even though foo2 is inside a function. It doesn’t matter how deep the function chain is, if there is no var, then the variable will bind to window:

function baz() {
    (function() {
        (function() {
            (function() {
                foo2 = 'bar';
            }());
        }());
    }());
}

Additionally, the window context is available as a first class object, and you can manually bind variables to it:

window.foo1 = 'bar';

function baz() {
    window.foo2 = 'bar';
}

Next, you need to understand that variables defined in the execution context outside of a function are also available in the execution context inside a function:

var foo = 'bar';

function baz() {
    alert(foo); // will alert 'bar'
}

The compiler first looks for the variable foo in the current execution context, then it moves up the scope chain to the next context, and so on until window. An error will be thrown if foo isn’t defined in any execution context in the scope chain. Again, this is true, no matter the how deep the function chain is:

function baz() {
    (function() {
        (function() {
            (function() {
                alert(foo); // will alert 'bar'
            }());
        }());
    }());
}

Function arguments are scoped to the execution context of the function and behave as variables declared with var:

var foo = 'bar1';
function baz(foo) {
    alert(foo); // will be 'bar2'
}
baz('bar2');

How it works…

To summarize, variables can be declared with or without var, but excluding the var will cause the compiler to search the scope chain for the variable, adding it to window if the variable doesn’t exist yet in the scope chain. If there are multiple variables of the same name in the scope chain, the first one found will be used, and all others will be masked. You may, however, access masked variable names on the window object by referencing them as you would an object property (window.NAME_OF_VARIABLE). Lastly, function arguments are defined inside the execution context of the function and will mask any variables in the scope chain of the same name.

Today the EC2 machine that I have been using to run my blog application for the past couple years expires, so I had to move to a new instance. Unfortunately, while I had done a great job of automating the setup for my blog, I had not done a good job of automating the bootstrapping of the operating systems that I use. I think it is really important that system initialization steps be automated, so you can easily deploy additional nodes, rebuild exact systems from scratch, and quickly recover from a disaster. To make things easier, I wrote a bootstrapping script for the two operating systems that I use (OSX and Ubuntu), which sets up the software I need to develop in Python and Node.js, and start the initialization script that is part of my blog repo. This article will introduce my bootstrapping script.

How do it…

The canonical reference will always be at https://gist.github.com/mattsnider/5485500. Here is what the script looked like at the time of this writing:

#!/bin/bash
# expects python is installed with OS distribution

# single line command for execution
# wget -O - <RAW_URL> | bash

# determine environment
if hash apt-get 2>/dev/null; then
    echo "Bootstrapping UBUNTU"
    UBUNTU=true
    # update apt-get and install a C compiler
    sudo apt-get -y update && sudo apt-get -y upgrade
    sudo apt-get -y install gcc
else
    echo "Bootstrapping OSX"
    # on OSX we use brew as the application repo
    ruby -e "$(curl -fsSkL raw.github.com/mxcl/homebrew/go)"
    brew doctor
    OSX=true
fi

# install python, when necessary
if [ $UBUNTU ]; then
    sudo apt-get -y install python-dev python-setuptools
fi

# setup pip and virtualenv
sudo easy_install pip
sudo pip install virtualenv
sudo pip install virtualenvwrapper
mkdir $HOME/.virtualenvs

# install git
if [ $UBUNTU ]; then
    sudo apt-get -y install git
fi
if [ $OSX ]; then
    brew install git
fi

# install node and npm
if [ $UBUNTU ]; then
    sudo apt-get install node npm
fi
if [ $OSX ]; then
    brew install node npm
fi

To run the script as a single command, grab the raw URL from the gist and execute:

wget -O - <RAW_URL> | bash
# eg. wget -O - https://gist.github.com/mattsnider/5485500/raw/ec8b38022145dac41b1ef4ad5dd262cc845b4ee0/bootstrap.sh | bash

How it works…

The script is broken up logically and comments atop each section describe what the section is doing. First, the code determines the operating system by checking if the program apt-get exists. For either OS a temporary variable is created to be used by later if statements. When the OS is Ubuntu, it updates apt-get and ensures that a C compiler is installed. When the OS is OSX, it installs brew which will be used as the application packaging tool for installing new software. The next step is to ensure python and python-setuptools are installed, so we have python and can use easy_install to install pip. This is kept logically separate from the first step, in-case I decide to support additional operating systems in the future. The third step installs pip, virtualenv and virtualenvwrapper, and ensures that the required ~/.virtualenv exists. These tools are a must have for any python developer. Next the script installs git for repository management and lastly it installs node and npm for development in Node.js. At this point the system is ready to run whatever initialization scripts you have for your application. For my blog, I need to manually clone the git repo, create a virtual environment, and install fabric in that environment. Then I have a fabric script that automates the rest of the installation.

I realized this week that I have never dedicated an article to context (scope) binding functions in JavaScript and thought I would remedy that today. JavaScript context binding functions are functions that guarantee the execution context of a callback function, allowing the developer to control what this is set to.

How do it…

We’ll look at two types of binding functions. The first, seamlessly wraps an inner callback function with a provided context, so the arguments are passed directly through inner function to bound function:

function bind(fn, ctx) {
    return function() {
        return fn.apply(ctx, arguments);
    }
}

Here is how to use bind:

el.onclick = bind(function(e) {
    // this === oTargetContext, not el
    // like normal, the only argument will be the event
}, oTargetContext);

The second type allows extra parameters to be provided at the time of binding, so when the wrapped function is called, its signature will be modified:

function bind_extra(fn, ctx, oOtherData) {
    return function() {
        return fn.call(ctx, arguments, oOtherData);
    }
}

Here is how to use bind_extra:

el.onclick = bind_extra(function(aArgs, oOtherData) {
    // this === oTargetContext, not el
    // the normal arguments are passed as argument 1, so aArgs[0] is the event
    // oOtherData === oMoreData
}, oTargetContext, oMoreData);

How it works…

In both examples the binding function takes several arguments and returns a new inner function. The returned function creates a closure around the arguments passed to the binding function, which is used to adjust the scope when the inner function executes. Because the inner function is returned when the binding function executes, it is the inner (anonymous in this case) function that is actually applied to onclick. Thus when the click handler executes, the inner function is called and passed the event as its only argument. The first seamless binding function simply applies the arguments passed to it directly to the original function that was bound. In this case .apply(desiredContext, arrayOfArguments) is used to adjust the context, whilst passing the arguments exactly as it received them. The bound function signature does not need to change. The second binding function explicitly passes the arguments array to the original, bound function as argument 1 and the extra data as argument 2. Since the arguments are enumerated .call(desiredContext, argument1, argument2) is used to adjust the context and pass the necessary arguments. The bound function signature will need to change to expect the arguments provided by the extra binding function. I see the binding pattern used most often in conjunction with prototypal objects, where the event, ajax, or timeout callback functions are attached to the prototype object and for code clarity, this should always refer to the current instance, but there are other uses. In my experience, I try to avoid functions like bind_extra, because they tend to complicate code dependencies and confuse the next person to look at the code.

I recently implemented a JavaScript only communication system for sending messages between windows/tabs (just referred to as window for the remainder of this document) for Ariba.com. We have a very heavy-handed approach that requires identifying each window and queueing messages in the order that are sent, no matter the originating window. And the final kicker is that it needs to work on some older browsers. We settled on a shared communication channel using the browser local storage, or cookies when local storage is unavailable. Since each window has its own UI thread processing the JavaScript, we also had to consider race conditions with reading and writing to the shared data store. Unfortunately, our solution required server-side logic for tab detection, so it is not applicable outside of Ariba. However, there was a lot of good learning to share, so I simplified the communication channel and rewrote the code to not need to keep track of the originating window. This article introduces a system that can be used to send messages between windows using only JavaScript. This technique has some drawbacks to consider before using: it only allows one message to be on the stack at a time, it does not preserve message order (although, they tend to stay in order), and it does not guarantee message delivery (closing a window before all messages have been sent). But it should be fine for loss-tolerant operations, such as sending a message alerting the user that another window about to logout.

How do it…

The full code is available at window_messenger.js and a demo page is setup for testing the code. The first part of the code initializes the data reader/writer and starts a timer to poll for new messages:

init: function() {
	// 1) start the interval timer (called a cycle) to begin polling
	// the message string
	setInterval(WindowMessenger.readMessage, CYCLE_TIMEOUT);

	// 2) decide which engine to use for communication
	// WindowMessengerUseCookie can be set to true to force cookie use
	if (w.localStorage && !w.WindowMessengerUseCookie) {
		// 2a) using HTML5 local storage. session storage was not
		// used because a new session is created in each tab
		this.reader = function(sName) {
			return localStorage.getItem(sName) || '';
		};
		this.writer = function(sName, sValue) {
			return localStorage.setItem(sName, sValue);
		};
	}
	else {
		// 2b) using cookie fallback
		this.reader = function(sName) {
			return readCookie(sName);
		};
		this.writer = function(sName, sValue) {
			return createCookie(sName, sValue, FORWARD_SLASH,
					w.location.hostname, COOKIE_EXPIRES);
		};
	}

	// 3) if a message is already in the system, I'm going to take ownership
	// of it. If I am not the sender, the sender may clear it before me.
	sLastSentMessage = WindowMessenger.reader(KEY_NAME);
	iCycleCount = 1;
},

The postMessage function is called whenever a message should sent through the system:

postMessage: function(sMessage) {
	// 1) current message
	var sCurrentMessage = WindowMessenger.reader(KEY_NAME);

	// 2) is this message being spammed? (should I send it at all)
	if (sLastSentMessage === sMessage || sCurrentMessage === sMessage) {
		fnLog('Not posting "' + sMessage +'", as it was recently sent.');
	}
	else {
		// 3) is there still a message
		if (sCurrentMessage) {
			// 3a) try again in a little while
			fnLog('Queueing message "' + sMessage + '"');
			callDelayed(WindowMessenger.postMessage, [sMessage], CYCLE_TIMEOUT);
		}
		else {
			// 3b) no messages, add this one
			fnLog('Sending post "' + sMessage + '"');
			WindowMessenger.writer(KEY_NAME, sMessage);

			// 4) code against race
			// ensure race condition did not occur and message was written
			setTimeout(function() {
				if (sMessage ===  WindowMessenger.reader(KEY_NAME)) {
					// 4a) record the sent message
					sLastSentMessage = sMessage;
				}
				else {
					fnLog('Race-Condition, re-posting');
					// 4b) revert and re-post
					iCycleCount = 0;
					callDelayed(WindowMessenger.postMessage,
							[sMessage], CYCLE_TIMEOUT);
				}
			}, 500);

			iCycleCount = 1;
		}
	}
},

The requestMessage function is called periodically to poll for new messages, and to remove messages the current window originated:

readMessage: function() {
	// 1) read the window message
	var sCurrentMessage = WindowMessenger.reader(KEY_NAME),
			i, sClosureMessage;

	fnLog('Read window message, contains - ' + sCurrentMessage);

	// 2) is there a message, if not clear last received message
	if (!sCurrentMessage) {
		sLastReceivedMessage = '';
	}
	// 3) is current message equal to the last message
	else if (sCurrentMessage === sLastSentMessage) {
		// 3a) increment the message process counter
		iCycleCount++;

		fnLog('Processing message "' + sCurrentMessage + '" (' +
				iCycleCount + ')');

		// 3b) when max cycles exceeded, remove message key
		if (iCycleCount > NUM_CYCLES_TO_PERSIST_MESSAGE) {
			// refresh the window message to reduce chance of race
			WindowMessenger.writer(KEY_NAME, '');
			sClosureMessage = '' + sLastSentMessage;

			fnLog('Removing message - "' + sCurrentMessage + '"');

			/*
			 3c) code against race
			 */
			setTimeout(function() {
				// 3a) if message key is still in window message,
				// then race occurred
				if (sClosureMessage === WindowMessenger.reader(KEY_NAME)) {
					sClosureMessage = '';
					iCycleCount = 0;
				}
			}, 500);
		}
	}
	// 4) is current message not equal to the last message received,
	// then process it.
	else if (sCurrentMessage !== sLastReceivedMessage) {
		sLastReceivedMessage = sCurrentMessage;
		i = aSubscribers.length - 1;
		fnLog('Passing message to subscribers - ' + sCurrentMessage);

		while (i >= 0) {
			aSubscribers[i--](sCurrentMessage);
		}
	}
},

The subscribe function should be passed a callback function for handling messages. Here is a simple callback to log all messages:

WindowMessenger.subscribe(function(sMessage) {
    console.log(sMessage);
});

How it works…

We’ll break the discussion up into three sections: init, post message, and read message.

init

The init function is called automatically after the window_messenger script is loaded. It first starts a poll timer (default is five seconds) to periodically call the read_message function. Then the code sets up a reader and writer functions to handle reading and writing message internally. The code prefers to use window.localStorage, but will use a cookie when local storage is unavailable. Lastly, it checks to see if there is already a message and updates variables to assume the message originated from this window. Since the messages contain no information indicating what window originated them, there is no way to know if the message originated from this window or another. And if it did originate from this window on a previous page, nobody would ever know to remove the message, since previous page JavaScript variables are lost. So, we assume it originated from this window, and if it did not, everything is still fine, because the real originating window will remove the message before the timer on this window does, and the current window will detect that and move on. We make one assumption about this window not needing to process the message since it just reloaded.

post message

The postMessage function should be passed the string message to send to the other windows. It first checks if the message is currently being sent or sent by another window to prevent spamming messages. Next, when there is already a message being sent, we queue the message using a simple timeout, so it will try to send again after the next read cycle. Otherwise, we write our message and follow it by a short timeout (this should be much shorter than a read cycle and is hardcoded to 500ms). The callback in the timeout double checks that the message is still the value in the data store, ensuring that another window did accidentally overwrite the current message. If a race occurred, we let the other message remain for the next cycle and queue up the message that we tried to write.

read message

The readMessage function is called periodically, and handles reading and removing messages. It first grabs the current message, then makes sure the message is not empty. When the current message is empty we set the sLastReceivedMessage variable to an empty string, so the system can accept the same message again if another window sends it. When the current message is not equal to sLastReceivedMessage, we send the message to all subscribing callback functions and update sLastReceivedMessage to the current message value. When the current message equals sLastSentMessage, we activate logic to handle detecting when to remove the message. A cycle counter is incremented to indicate the number of times this code block executed for the current message. When the counter is greater than the number of cycles to persist (default is two cycles), the message is cleared from the data store. To remove the message, an empty string is written to the shared data store. As with postMessage a short timeout is used to assert there was not a race to update the data store. When a race occurs, we do not need to do anything, because the next read message cycle will attempt to remove the message again. When there is no race, we go ahead and reset the counter and sLastSentMessage. If another window writes the exact same message as the current window is trying to remove during the 500ms timeout, then the race logic will be triggered. This is most likely okay, because the duplicate message was just sent and processed by the other windows, so it is probably a spammy message.

There’s more…

As you can see, sending messages between windows can be done relatively simply, if it doesn’t need to be bullet-proof. I think this could be improved by using an SQL database or similar system in the browser, where messages can be written and removed in order, without worrying about race conditions. Alternatively, a set of 10 to 20 keys could be written to, writing to the next available slot and read out in order. This would allow multiple messages to be written and processed using the same technologies already implemented, but would require more complicated message tracking and handling of race conditions.

Years ago services like Seti@Home introduced us to crowd-sourcing computing cycles in order to analyze big data. Unfortunately, these programs require that you install some software on your computer and keep it running. Lately, I’ve been noodling over applying this same concept to the web, and I thought I’d share my thoughts. When somebody is viewing a webpage, most of the time the browser is relatively idle, and probably their machine as well. For years now, modern browsers have implemented the Web Worker API, exposing a way to tap into those unused computer cycles. The missing step is connecting big data to those computing cycles, and possibly rewarding web surfers or site owners for the used cycles.

How do it…

For starters we need a marketplace where somebody who has a lot of data to analyze can go to buy cycles. This is the hard part of the problem, and what I hope one of you might solve. The gist is that a large amount of segmentable data would be uploaded and a function for analyzing the segments. Sites or users who have opted in to the marketplace include two JavaScript files. The first is the static code to manage the web workers and process data URIs:

(function() {
	var i = 1,
		failCount = 0,
		MAX_FAIL = 10;

	function fetchMoreData(sData) {
		if (! window.WebAnalyzerStop) {
			// fetch more data using JSONP
			var el = document.createElement('script');
			el.async = true;
			el.src = '{PATH_TO_DATA_URL}?data=' + encodeURIComponent(sData);
			document.body.appendChild(el);
		}
	}

	function WebAnalyzerProcessData(sDataUri) {
		try {
			var oWorker = new Worker(sDataUri);

			oWorker.onmessage = function(e) {
				// worker finished. data should be reported back to the server
				//	where it can be recorded and more data can be loaded for processing
				fetchMoreData(e.data);
			};

			oWorker.onerror = function() {
				// worker broke, probably should report somewhere,
				// 	but also get more data for crunching
				failCount++;

				// if something is seriously wrong, let's just stop
				if (MAX_FAIL > failCount) {
					fetchMoreData('');
				}
			};

			console.log("WebWorker " + i++ + " started!");
		}
		catch(e) {
			if (window.console) {
				console.log('WebWorkers are not fully supported by your browser');
			}
		}
	}

	// exposed globally so data loading JSONP requests can start new workers
	window.WebAnalyzerProcessData = WebAnalyzerProcessData;
}());

The second is would be a call to the marketplace server that would return a JavaScript dataURI, and call the WebAnalyzerProcessData function (this is a JSONP system with a known callback function name):

(function() {
    var WebAnalyzerDataUri = "data:text/javascript;charset=US-ASCII,var%20i%20%3D%20500000000%2C%20n%20%3D%200%2C%20dTime%20%3D%20(new%20Date()).getTime()%3Bwhile%20(0%20%3C%20i)%20%7Bn%20%2B%3D%20i%3Bi%20-%3D%201%3B%7DdTime%20%3D%20(new%20Date()).getTime()%20-%20dTime%3BpostMessage(%22Calculated%20%22%20%2B%20n%20%2B%20%22%20in%20%22%20%2B%20dTime%20%2B%20%22ms%22)%3B";

    if (window.WebAnalyzerProcessData) {
        window.WebAnalyzerProcessData(WebAnalyzerDataUri);
    }
}());

If you heard your fan kick on (it will after ~30 seconds), it is because I am demonstrating the system right now. The web worker is adding the numbers between 1 and 500000000, then calling the data URL again (this is a demo, so the data URL does the same thing, but ideally it would be dynamic).

How it works…

The static code manages creating web workers and fetching new data from the dynamic data URL (provided by the marketplace). We expose a global callback function WebAnalyzerProcessData that the dynamic data script can send dataURI encoded JavaScript too. Inside a try/catch block a new web worker is created from this dataURI and the message and error events are subscribed to. The try/catch block is used, because some older browsers do not yet support web workers and IE does not support dataURI blobs. Whether the worker finishes on an error or a message, we fetch a more data, stopping only when MAX_FAIL is reached or somebody sets window.WebAnalyzerStop = true;. Messages are passed to the marketplace using the data query parameter so the analysis can be recorded. The dynamic data URL needs to extract a chunk of data and combine it with the data analysis function, creating a JavaScript file. This file is then encoded (I used encodeURIComponent, but base64 or another compressing encoding can be used as well) as a string and returned dynamically. This dataURI is passed into WebAnalyzerProcessData, where another web worker will be created to process the next chunk of data. If a popular site were to participate millions of chunks of data could be processed every day, helping solve some big data problems. Besides having to build the marketplace, there is onus on the big data provider to provide the marketplace with segmentable data and a function that can process chunks of data. The marketplace would need to combine all the data and provide it back to the big data provider.

There’s more…

This technology really excites me, because there are many large data problems that cannot yet be feasibly solved with a modern computer, but could be solved with millions of computers crushing away at it continuously. Additionally, are computers sit idle most of the time, and this would be a great way to put some of those CPU cycles to good use. If the marketplace is well-implemented then it should be profitable by charging the big data providers to use its system, and be able to pay bounties to sites or users. That said, if the marketplace is not well curated or is intentionally malicious, it could also become the biggest bot-net in the world or possibly be used to attack some of the intractable problems like encryption, and who knows, maybe even Skynet. Overall though, I think the benefits outweigh the risks, and hope some entrepreneur takes on the task of building this system.

Big news in Web Development - Blink Rendering Engine

Google announces that is has forked WebKit and will now be using its own rendering engine, Blink, inside of Chromium. This means in the future web developers will have another platform to test JavaScript and CSS against. The change will affect all Chromium-based browsers, including Opera 14+. If you want to find out more, PPK has written a really detailed article describing the situation.

I recently needed a simple, legacy browser (*cough* IE 6) compatible, solution for sending cross-domain communication between an iframe and its parent window. Fortunately, the URL protocol allows changing the hash part of the URL without causing the page to refresh, and the same is true when updating the URL of an iframe. This means that you can use a hash-hack to communicate between the frames. If you do not need to support < IE 8, the modern technique is to use postMessage to communicate. A full framework would use postMessage when possible, and fallback to the hash-hack technique as a last resort, but this article will only describe the hash-hack technique.

How do it…

We need two pieces of JavaScript, one to be included on the parent iframe and the other to be included on the frame page. To send a message from the parent page, include the following HTML:

<body>
…
<iframe id="idMyIframe" src="http://mattsnider.com/{urlForIframedPage}"></iframe>
…
<script src="http://mattsnider.com/{pathToJs}/hashHackSender.js"></script>
…
</body>

And here is how we send a message:

HashHack.postMessage('idMyIframe', 'Data to send');

To receive messages on the child page, include the following HTML:

<body>
…
<script src="http://mattsnider.com/{pathToJs}/hashHackReceiver.js"></script>
…
</body>

And here is how we listen for messages:

HashHack.onMessage(function(e) {
    // message is stored on e.data
    console.log(e.data);
});

Below is a little demo showing the hash-hack communication. Change the text and click Send Message, to have it show up in the iframe. Send Message The scripts can be downloaded here:

• Hash-Hack Sender
• Hash-Hack Receiver

How it works…

The hash-hack technique uses the hash part of the URL in the iframe to send messages between the parent window and the iframe. This is only necessary if the iframe URL is not from the same domain as the parent window, because normal JavaScript access will be blocked by cross-origin security. The hashHackSender.js script adds the HashHack object to the global namespace, which has a single function postMessage. Calling postMessage will cause the hash of the iframe URL to be replaced with a formatted message that can be decoded by the hashHackReceiver.js. The postMessage requires the id or element reference of the iframe as the first argument and the message as the second. Keep in mind that we are using a URL to send message, so messages should be relatively short, because long messages will exceed the maximum length of URLs in some browsers (~2000 characters). The hashHackReceiver.js script also adds the HashHack object to the global namespace, and exposed a single function onMessage. Call HashHack.onMessage, passing a callback function as the only argument, to subscribe to the communication channel. Whenever the hash of the iframe URL is changed and matches a known format, it is decoded and all callback functions will be called. The callback functions will be passed an object with a single data property, containing the message. A simple interval timeout is used to poll for changes to the hash of the iframe. The goal of this technique is to have one system for communication that can be used on all browsers, including legacy browsers. This is not the most efficient or robust system, but is simple to understand and easy to implement.

There’s more…

There are lots of other strategies for cross-domain iframe communication. This article just focused on a simple, universal technique. For a more complete writeup of various communication strategies and the a list of popular frameworks, see Cross Domain Communications with IFrames.

I recently went through the chore of making sure that all the scripts on this site are loaded asynchronously using JavaScript Deferment, as shown in Django Template Tags for JavaScript Deferment. This ended up causing problems with my JavaScript, because most of my scripts depend on blog.js, which is now loaded asynchronously. I decided to write a simple require system that other scripts could use to ensure that blog.js is already loaded, and today I am sharing that simple system.

How do it…

A small bit of code needs to be added directly to the HTML, exposing the global function __onReady:

<script type="text/javascript">
(function(w) {
    var aFunctions = [];

    w.__onReady = function(fnCallback) {
        aFunctions.push(fnCallback);
    };

    w.__onReady.ready = function() {
        w.__onReady = function(fnCallback) {
            fnCallback(w);
        };

        for (var i = aFunctions.length - 1; i > 0; i -= 1) {
            w.__onReady(aFunctions[i]);
        }
    };
}(window));
</script>

The JavaScript code that is required (the dependency), should call the __onReady.ready function when it finishes (blog.js in my case):

__onReady.ready();

Finally, any code requiring the dependency should be wrapped inside of an __onReady call (in my case, all other JS files):

__onReady(function() {
    // code here as normal
});

How it works…

The global __onReady function must be added before any other JavaScript is included on your page, so you ensure it is always available. When the function is called (and the page is not yet ready) the passed callback function is added to an internal array of callback functions. If the page is ready the callback function is immediately executed and window is passed as the only argument. When the __onReady.ready function is called, it overwrites the behavior of the __onReady function to return immediately, as the required JavaScript is already available. It then iterates through the callback functions in the internal array and calls each of them. I have kept the __onReady code as small as possible, so it has little impact on the weight of the page. It also uses straight JavaScript, so you do not need to depend on a library (which should be loaded asynchronously anyway). One powerful way you could use this, is to load jQuery (or another library) asynchronously but still depend on it in your scripts (also loaded loaded asynchronously). Just add __onReady.ready(); to the end of your library JavaScript file and wrap your other scripts using the __onReady callback pattern. Loading scripts (especially libraries) asynchronously will drastically decrease the delay before a page become usable to the end user.

There’s more…

This is not the only strategy I could have taken for managing dependencies on blog.js, but is the one I liked most. I could have split the file, so that the parts that have to run are included synchronously (probably directly in the HTML) and making blog.js a YUI module that is included by other services. Or I could have included another system, such as requireJS, but this strategy limited the complexity, while requiring few changes.

Python-style decorators (or Java annotations) are a useful feature that is not natively available in JavaScript. This article describes a technique for apply Python-style decorators to JavaScript functions. We’ll call them annotations, so not to confuse them with the decorator pattern.

How do it…

To illustrate what an annotation will be, lets try a simple example:

function alertFoo() {
    alert('foo');
}
alertFoo = annotate(alertFoo).by(alertBar);

We annotate the alertFoo function with the alertBar function (not yet defined). This is equivalent to executing the following code:

alertFoo = alertBar(alertFoo);

Annotations are a form of metaprogramming; they enhance/change the action of the function or method they annotate. For example, if we define the alertBar annotation:

function annotationBar(fnAnnotated) {
	return function() {
		alert("bar");
	}
}

Then the annotated alertFoo will alert bar instead of foo. Go ahead and give it a try: Unannotated alertFoo Annotated alertFoo We replaced the behavior of alertFoo using an annotation. That’s useful, but not terribly powerful. Perhaps, the annotation should check something, hijacking the function if some variable isn’t met, but executing the annotated function normally otherwise. The following function will alert bar when the click counter is odd, but delegate to the annotated function, alerting foo when the counter is even:

function annotationBarWhenTrue(fnAnnotated, fnEval) {
	return function() {
	    if (fnEval()) {
	        alert("bar");
	    }
	    else {
		    fnAnnotated.apply(this, arguments);
		}
	}
}
var iCallCounter = 0;
alertFoo = annotate(alertFoo).by(alertBar, function() {
    return iCallCounter++ % 2; // true when odd
});

Alert bar when click count is odd, foo otherwise As you can see arguments may be passed when annotating a function into the annotation itself and used for evaluation. Additionally, the annotated function may be called directly inside the inner annotating function, allowing for annotation chaining. The last thing to show is annotation chaining. Below we chain two additional alerts onto alertFoo:

function annotationBar(fnAnnotated) {
    return function() {
        alert("bar");
        fnAnnotated.apply(this, arguments);
    }
}

function annotationBaz(fnAnnotated) {
    return function() {
        alert("baz");
        fnAnnotated.apply(this, arguments);
    }
}
alertFoo = annotate(alertFoo).by(annotationBar).by(annotationBaz);
alertFoo();

Alert baz, bar, then foo using chained annotations Once a function is annotated, calling annotate or by functions will continue to use the existing annotation chain, so the above could be rewritten as:

annotate(alertFoo); // adds alertFoo.by
alertFoo = alertFoo.by(annotationBar);
alertFoo = annotate(alertFoo).by(annotationBaz);
alertFoo();

Because the annotate function be improved over time, I have not included the source-code here. You can view the latest version at annotation.js.

How it works…

Annotation.js creates a function called annotate that can be used to annotate existing JavaScript functions. By default annotate will return the function passed in as its only argument, but augment it with a by function that can be used to add annotations. This allows the function calls to use natural language when annotating: annotate(functionToAnnotate).by(myAnnotatingFunction);. The by function supports chaining and returns the the annotated original function (or the last one on the annotation chain, if not the original function). It accepts any number of arguments, but the first argument must be the annotating function; all other arguments will be passed to the annotating function to create the annotation wrapping function. The annotating functions should return a function which is what will be executed when the annotated function is called. Because we are using JavaScript simply calling the annotate function doesn’t update the original function with the annotated version. As shown in all the examples above, you must assign the response of the annotate and by functions to the original variable. Annotations can be used to replace the behavior of a function, augment the behavior of a function, and/or perform other complex operations. Although, not shown above, they are particularly useful, when you want to override the behavior of a single instance of an object, instead of all instances, as might be done by changing the prototype.

There’s more…

The section above describes how to use the annotate function, below will go into details of how it works. To begin, a function is passed into annotate that you want add annotations to. It first checks to see if the function is already annotated by looking for the special by function. This ensures that only one annotation chain exists, and reduces confusion and corner-cases in complex applications. If the function wasn’t previously annotated we initialize the annotation chain with the function to annotate. The annotation chain is used to maintain the call stack of annotations. Lastly, the by function is added to the function to annotate and the function to annotate is returned. At this point no annotations have been made, but the returned function is in an annotation aware state with the by function available. The original function to annotate variable has been modified with the by function, so you do not necessarily need to chain functions, but I think the code is more understandable when you do. The by function is where the annotation magic happens. We first find the function that should be annotated (wrapped by our annotation logic) by looking at the annotation chain. The last item in the chain, is the one that should be annotated next. Initially, this will be the original function to annotate, but will become a stack of each subsequent annotation. We then use a wrapping function to call the annotating function passing the function to annotate and all other arguments that were passed into the by function. The wrapping function is used to allow modifying the arguments array without changing the function that the variable fnAnnotation points to. Thus I can call fnAnnotation using the apply method without needing to enumerate the arguments. The annotating functions are special functions that should accept the function it annotates and any additional arguments that are expected to be passed into the by function. They should return an inner function that does whatever the annotation is expected to do. If the annotation should chain to the function it annotates, simply call fnAnnotated.apply(this, arguments); from the inner function. The last part of the by function pushes the function returned by the annotation function onto the annotation chain, augments it with the by function, and returns it. This is what allows the chaining of the by functions. I find the annotate function intuitive and easy to use, even though explaining it is not, and I hope you do as well. That said, there are two features that I have not yet implemented (but hope to eventually): objects cannot yet be annotated as they can in Python 2.7+, and there is not unannotated function yet. Please leave a comment, describing how you are using annotations in your codebase.

Additional Reading

The annotate function tries to emulate Decorators in Python, so you may find the Wikipedia article helpful.

At my new company (Ariba), I am forced to use a beastly version control system (Perforce) and it was really cramping my style. I couldn’t easily branch or check in small incremental changes (even locally), so I was going days to weeks without checking in, waiting until my code was production ready. This meant if I made a mistake and had to roll back, or wanted to maintain multiple version of a file, I had to manually maintain version and/or redo/undo work. This week, I finally had enough, and decided to augment Ariba’s version control system by using Git locally for better control while I developed. The rest of this article describes how to mirror an existing repository with a Git one. We will be building a fictitious repository named example.

How do it…

Create and initialize the remote Git repo

Create the directory:

mkdir -p /pathToDesiredLocation/example.git

Change to that directory:

cd /pathToDesiredLocation/example.git

Initialize the Git repository:

git init --bare

Gitifying your existing version control

When starting a new project, clone the repository and begin coding:

git clone /pathToDesiredLocation/example.git example

Otherwise, if (like me) you are augmenting an existing repository with Git, move to the root of your existing repository or the desired part of the repository that you want to add Git to:

cd /pathToExistingRepoOrPartOfRepo

Activate Git on that directory:

git --init

Connect the remote repository to the local one:

git remote add origin /pathToDesiredLocation/example.git

Add all the existing files to Git and check it in (this may take a few minutes the first time):

git add .
git commit -m "cloning existing repo"
git push origin master

Maintaining your Git repository

Whenever updating the codebase from your company’s version control system, create a new commit with all changes:

# updated local repo from company repository
git add .
git commit -m "syncing to {CompanyRepoSystem} revision {RevisionNum}"
git push origin master

To make finding previous version easier, try tagging the sync commits:

git tag -m "{CompanyRepoSystem} revision {RevisionNum}" {OptionalCodeVersion} {CommitNum}

View existing tags using:

git tag -l -n

Make sure to push the tags to the remote repository:

git push --tags

How it works…

The first group of steps above show how to turn any directory into a Git repository that can be used as the remote for a local git repository. This article puts it on a local disk, but ideally it should be another server. At Ariba I have my remote on a mapped shared drive, in case my hard drive crashes, and so that my coworkers can easily use the remote as well. The second group of steps show how to initialize your company’s repository (or only part of it) as a Git repository and set the origin to the previously created remote repository. This way, when changes are pushed they live both in the local repository and in the origin repository. If you are not worried about maintaining a remote backup or sharing with peers, then you could get away with only running a local Git repository. I do’t recommend a local only approach, but implementing it would only require the second group of steps and never pushing to the remote. The last group of steps explains keeping the local Git repository in sync with your company’s version control. Syncing, committing, and pushing is all that needs to be done, but using tagging allows for better tracking of when changes were synced with Git and makes rolling back to previous version easier. There is not an example of how to check code into your company’s repository, because I do not know your system. However, here are some guidelines that I am using:

• Do all local development in branches and keep master in sync with your company’s repository to avoid conflicts and make syncing easier.
• Before checking in, update master to your company’s repository, then merge the development branch and push to your company’s repository system.
• If your company’s repository requires checkouts, checkout any modified files before merging the development branch.

There’s more…

The solution written here will work for everybody, but requires you to maintain a Git version control that is separate from your company’s version control. For most version control systems (including perforce) somebody has already written a bridge between it and Git. I recommend trying the bridge first, so you have tighter integration, before maintaining two version control systems. For more Git-related articles, see:

• Using Git Interactive Rebase With Feature Branches
• Git Rebase Versus Git Pull
• Amending and Squashing Commits in Git

Continuing our sorting algorithm discussion, today’s article will cover a technique known as Shell Sort. Shell sort is a quadratic (O(n²)) in-place comparison sort algorithm developed by Donald Shell in 1959 that is basically Insertion Sort with a gap between comparing indexes. The assumption is that comparing farther apart elements can move some out-of-place elements into position faster.

How do it…

The first thing we need is an algorithm to calculate the gaps to evaluate. Here is a simple algorithm that was created by Shell (gaps are an array of values created by n / 2 ^ k):

function createGaps(a) {
    // if a is an array of 100, gaps would be [50, 25, 12, 6, 3, 1]
	var gaps = [];

	for (var i = 0, j = a.length, t; 1 <= (t = Math.floor(j / Math.pow(2, i + 1))); i += 1) {
		gaps[i] = t;

		if (t === 1) {
			break;
		}
	}

	if (gaps[i] !== 1) {
		gaps.push(1);
	}

	return gaps;
}

Here is a the shell sort algorithm in JavaScript:

function shellSort(a) {
	var gaps = createGaps(a),
		temp;

	for (var i = 0, j = gaps.length, gap; i < j; i += 1) {
		gap = gaps[i];

		for (var x = gap, y = a.length; x < y; x += 1) {
			temp = a[x];

            // this performs insertion sort on subarrays
			for (var z = x; z >= gap && a[z - gap] > temp; z -= gap) {
				a[z] = a[z - gap];
			}

			a[z] = temp;
		}
	}
}

Here it is in action: Restart Shell Sort With a New Array If we take a 10,000 element array and sort it, here is how shell sort compares to native array sort (prints to the console): shell Sort Comparison Tests

How it works…

The gap algorithm is simple and there are more efficient ones (see the wikipedia article), but I choose it because it was easy to implement and understand. It creates an array of values n / 2 ^ k, where n is the length of the array and k is an incrementing integer. We stop appending to the array when the final value of 1 is reached. The gap algorithm is critical to reducing the number of swaps made by the inner insertion sort. The shell sort algorithm breaks the array up into sub-arrays and order them. This is a little tricky to understand by looking at the code, so we’ll analyze the first few comparisons made to a 100 value array. The initial gap will be 50, so we will perform insertion sort on the two element sub-arrays made from the values at 0 and 50, 1 and 51, 2 and 52, etc. (i.e. 50 sub-arrays with two values). The second gap will be 25, so we perform insertion sort on the four element sub-arrays made from the values at indexes: 0, 25, 50, and 75; 1, 26, 51, and 76; 2, 27, 52, and 77; etc. (i.e. 25 sub-arrays with four values). And so on until we perform a complete insertion sort on all values once (the complete array with all 100 values). For a random distribution shell sort tends to perform better than other quadratic comparison sorting algorithms, because it moves the values farthest from their final position to that position quicker. However, it will perform worse when the array is sorted already, or the values at each index are near their final position. Additionally, the number of elements in the gap array will affect the performance of this algorithm, so the better the gap the better the performance of shell sort.

There’s more…

For more information on shell sort and creating shell gaps, see the Shell Sort Wikipedia article.

When I was developing for Votizen, I always felt uneasy that we had our settings files with many important passwords and auth keys stored unencrypted on a third-party service. Sure, SSL was used for fetching and committing code, and only a few SSH public keys were allowed access to our repository, but it is one more opportunity for your site to be hacked. I recently read John Resig’s post Keeping Passwords in Source Control and decided with such an easy to follow tutorial, it was time to encrypt my Django settings.py files for my projects. Today’s article describes my process for encrypting files and outlines some of the gotchas that I have faced. We will be using Git and Python/Django, but the lessons can easily be applied to another language and frameworks, as we are mostly going to wrap openssl, which is included on Linux and OSX.

How do it…

To encrypt a file using openssl, use the command:

openssl <cipher> -e -in<file to encrypt> -out<encrypted file name>

To decrypt a file using openssl, use the command:

openssl<cipher> -d -in<encrypted file name> -out<file to encrypt>

So for the Django settings file, using Triple-DES cipher:

openssl des3 -e -in settings.py -out settings.py.bin
openssl des3 -d -in settings.py.bin -out settings.py

Make sure .gitignore excludes settings.py, by adding the line (this replaces the settings_local.py line, if you’re using it):

settings*.py

For those using fabric to simplify scripting, the following functions will be useful:

def decrypt_file(file, cipher='des3'):
    """
    Decrypt a file.
    """
    d = {'cipher': cipher, 'file': file}
    local("openssl %(cipher)s -d -in %(file)s.bin -out %(file)s" % d)
    local("chmod 600 %(file)s" % d)


def encrypt_file(file, cipher='des3'):
    """
    Encrypt a file, default is the settings file.
    """
    local("openssl %(cipher)s -e -in %(file)s -out %(file)s.bin" % {
        'cipher': cipher,
        'file': file,
    })

And to run from the command line:

fab encrypt_file:settings.py
fab decrypt_file:settings.py

How it works…

We are using openssl to encrypt a file with the command openssl -e and to decrypt the file with the command openssl -d. There are a lot of ciphers to choose from (man openssl for more options), but I like Triple-DES, because it is widely used and well tested. When encrypting a file, openssl will prompt for a password. This same password will be used when decrypting the file. This password is the first change to your development flow. You will need to remember it, and share it among your developers. Obviously, make it strong, but also memorable, because writing it down defeats the purpose. We will be checking in the encrypted settings files, instead of the normal ones. Since the encrypted settings files are checked in, when the repository is first checked out, or changes are made to the settings file in the remote repository, developers will need to rerun the decryption command to get the latest settings. If this becomes a problem, setup a githook or some other post-processing command that automatically decrypts settings when there are changes. The next change to your development flow, is when new changes made to the settings file needs to be checked in. In this case, the developer makes their changes, and reruns the encryption command. This will generate a new settings.py.bin, replacing the old one, and is the file that is actually checked into the repository. This is the first big gotcha, since you are ignoring settings files in git, it won’t prompt that changes have been made to settings files. The developer will have to remember to encrypt the settings files. I have not found a good way to automate this, as checking the recentness of the timestamp on the settings files returns false positives if you make a change and then check in several times, or does not work when I change them days ago but are committing today. The fabric functions above, always use the python filename as the file argument, automatically adding .bin to the in or out file, depending on the operation. The .gitignore line above (settings*.py) indicates that all settings files are to be ignored, as long as they following a naming convention that starts with the word settings and ends with .py (such as settings_prod.py and settings_dev.py, or just settings.py). If you are like me and have several settings files, the core file (settings.py) and server specific files (settings_prod.py) that get imported on the correct systems, then you may need to encrypt multiple files. This technique can be applied to as many settings files as is needed for your system. For this blog, I use the following fab commands, to simplify encrypting and decrypting:

def decrypt_settings():
    decrypt_file("settings.py")
    decrypt_file("settings_dev.py")
    decrypt_file("settings_prod.py")

def encrypt_settings():
    encrypt_file("settings.py")
    encrypt_file("settings_dev.py")
    encrypt_file("settings_prod.py")

I always name my encrypted files .bin, instead of using the cipher, as John Resig did in his example. If you intend to use a single cipher, then this adds another level of security (through obscurity), as a hacker needs to determine what cipher was used. However, the ciphers are secure and even if a hacker know the cipher, it will not help the hacker much, so do what works for you.

There’s more…

As you can see, there are pros (security) and cons (workflow changes) to encrypting your settings files. I feel that security far outweighs the changes to my workflow (if a hacker gets your settings file, they pretty much get complete access to your site), but you should decide for yourself. The biggest two problems I have faced are: forgetting to checkin changes to my settings files and dealing with merge conflicts on the encrypted files. The former is hopefully caught by your remote deployment process, before going to the servers (if not, then you’re not testing enough). The latter is a bigger problem. Since encrypted files cannot be merged, if there is a merge conflict, you have to handle it with care. My usual strategy, is to throw away my changes to the encrypted file and except the remote changes. I then backup my decrypted settings file to settings.py.bak and decrypt the remote settings file (overriding my settings file). Afterwards, I copy my changes into the new settings file, re-encrypt it, and push my changes. If my settings change was a one-liner, I will just extracted that line, instead of completely backing up the whole file.

There is an awesome new feature in HTML 5, window.requestAnimationFrame, which tells the browser that you wish to perform an animation and requests that the browser schedule a repaint of the window for the next animation frame¹. This allows you to do animations more efficiently and without using setInterval or series of setTimeout function(s). I have created a Demo that attempts to use the browser’s requestAnimationFrame side-by-side with the legacy polyfill executing.

Getting ready

If your browser supports HTML 5, then the requestAnimationFrame and cancelAnimationFrame will be set by:

// most browsers have an implementation
window.requestAnimationFrame = window.requestAnimationFrame ||
		window.mozRequestAnimationFrame || window.webkitRequestAnimationFrame ||
		window.msRequestAnimationFrame;
window.cancelAnimationFrame = window.cancelAnimationFrame ||
		window.mozCancelAnimationFrame || window.webkitCancelAnimationFrame ||
		window.msCancelAnimationFrame;

If your browser does not support HTML 5, then the requestAnimationFrame and cancelAnimationFrame will be emulated by:

var aAnimQueue = [],
	iRequestId = 0,
	iIntervalId;

// create a mock requestAnimationFrame function
w.requestAnimationFrame = function(callback) {
	aAnimQueue.push([++iRequestId, callback]);

	if (!iIntervalId) {
		iIntervalId = setInterval(function() {
			if (aAnimQueue.length) {
				aAnimQueue.shift()[1](+new Date());
			}
			else {
				// don't continue the interval, if unnecessary
				clearInterval(iIntervalId);
				iIntervalId = undefined;
			}
		}, 1000 / 50);  // estimating support for ~50 frames per second
	}

	return iRequestId;
};

// create a mock cancelAnimationFrame function
w.cancelAnimationFrame = function(requestId) {
	// find the request ID and remove it
	for (var i = 0, j = aAnimQueue.length; i < j; i += 1) {
		if (aAnimQueue[i][0] === requestId) {
			aAnimQueue.splice(i, 1);
			return;
		}
	}
};

Here is a complete code snippet that you can include on your site to support using requestAnimationFrame in all browsers, including legacy:

(function(w) {
    "use strict";
	// most browsers have an implementation
	w.requestAnimationFrame = w.requestAnimationFrame ||
			w.mozRequestAnimationFrame || w.webkitRequestAnimationFrame ||
			w.msRequestAnimationFrame;
	w.cancelAnimationFrame = w.cancelAnimationFrame ||
			w.mozCancelAnimationFrame || w.webkitCancelAnimationFrame ||
			w.msCancelAnimationFrame;

	// polyfill, when necessary
	if (!w.requestAnimationFrame) {
		var aAnimQueue = [],
			iRequestId = 0,
			iIntervalId;

		// create a mock requestAnimationFrame function
		w.requestAnimationFrameLegacy = function(callback) {
			aAnimQueue.push([++iRequestId, callback]);

			if (!iIntervalId) {
				iIntervalId = setInterval(function() {
					if (aAnimQueue.length) {
						aAnimQueue.shift()[1](+new Date());
					}
					else {
						// don't continue the interval, if unnecessary
						clearInterval(iIntervalId);
						iIntervalId = undefined;
					}
				}, 1000 / 50);  // estimating support for 50 frames per second
			}

			return iRequestId;
		};

		// create a mock cancelAnimationFrame function
		w.cancelAnimationFrameLegacy = function(requestId) {
			// find the request ID and remove it
			for (var i = 0, j = aAnimQueue.length; i < j; i += 1) {
				if (aAnimQueue[i][0] === requestId) {
					aAnimQueue.splice(i, 1);
					return;
				}
			}
		};
	}
})(window);

How it works…

The requestAnimationFrame function takes a single argument, the callback to be invoked before the repaint, and returns a positive integer that is the requestID (this will be greater than zero, but do not make any other assumptions), which can be used to stop an animation. You can stop the animation by passing requestID to the cancelAnimationFrame function. The legacy polyfill functions have the exact same signatures, but use an interval for managing animation frames. The legacy requestAnimationFrame appends the callback function and an internal requestID to an animation queue. The requestID is simply a number that increments each time the function is called, ensuring uniqueness. If an animation setInterval is not running, one is started, and the requestID is returned. When there are items in the queue, the setInterval callback function shifts the first element in the queue and calls the animation function, otherwise the interval is terminated for performance. The legacy cancelAnimationFrame searches the animation queue for the animation with the matching requestID, then removes it from the queuing array. It does not worry about stopping the interval, as the interval will stop automatically, if the animation queue is empty, on its next cycle.

There’s more…

There is another related property, animationStartTime, which is supported (prefixed) by some browsers, and indicates when the animation frames began. However, (at the time of this writing) browsers implement it and the timestamp passed into the animation callback function differently. Chrome sets this property to zero and passes the difference (in ms) since the animationStartTime to the animation callback function, while FireFox sets the property to ms since the epoch and passes ms since epoch into the callback function. Until a common standard is agreed upon between the browsers, it is better to implement your own start time system (if you need it), using (new Date()).getTime() or the shorter +new Date(). References:

1. window.requestAnimationFrame on MDN

A couple weeks ago, I mentioned that I was using Processing.js to animate a visualization tool for work. Today’s article showcases the class-based strategy we used to handle animation frames. There is a simple Demo available, if you would like to see Processing.js in action.

Getting ready

To start using processing, you will need three things: the latest Processing.js code, a canvas element on an HTML page with the data-processing-sources attribute set, and a .pde file for your processing code. Here is a sample HTML:

<html>
    <head>
        <title>Processing.js Test App</title>
        <script src="http://mattsnider.com/pathToProcessing/processing.min.js"></script>
    </head>
    <body>
        <canvas data-processing-sources="pathToPde/processingTestApp.pde" width="1000" height="500"> </canvas>
    </body>
</html>

How do it…

In the .pde file, define the animation base class:

function AnimationFrameClass () {}
AnimationFrameClass.prototype = {
    data: null,  // optional data (for internal use of children)
    draw: null,  // required, the function to render this animation
    onend: null,  // optional function to call, when animation is finished
    // required function to indicate when to stop, by default after max_frames
    finished: function() {
        return this.max_frames < this.frame;
    },
    max_frames: 100,  // default maximum number of frames for this animation
    name: null,  // the name of the animation, useful for debugging
    reset: function() {  // restart the frame counter
        this.frame = 0;
    },
    onstart: null,  // optional function to call, before animation begins
    started: false,  // used by draw function
    frame: 0  // required, the frame counter
};

If you are not using a library and need an inheritance function, add the following as well:

function inherit(C, P, p) {
	var F = function () {};
	F.prototype = P.prototype;
	C.prototype = new F();
	C.uber = P.prototype;
	C.uber_constructor = P;
	C.prototype.constructor = C;

	if (p) {
		for (var prop in p) {
			if (p.hasOwnProperty(prop)) {
				C.prototype[prop] = p[prop];
			}
		}
	}

	return C;
}

Here is a simple example that extends the animation base class to draw the background:

color BackgroundColor = 0xFFCCCCCC;
var BackgroundAnim = inherit(function() {}, AnimationFrameClass, {
    draw: function() {
        background(BackgroundColor);
    },

    finished: function() {
        return false;  // never stops
    },

    name: 'Background Animation'
});

Globally, define a queue that AnimationFrameClass instances are appended to and process the queue in the special draw function:

var Queue = [];

// called automatically by processing continuing the animation, draws a frame
void draw() {
    var newQueue = [],
        i = 0,
        j = Queue.length,
        animObj;

    for (; i < j; i += 1) {
        // clear fill and stroke that may have been set by previous animations
        noFill();
        noStroke();
        animObj = Queue[i];

        // fire started function
        if (!animObj.started) {
            animObj.started = true;
            animObj.onstart && animObj.onstart();
        }

        // draw and increment the frame
        animObj.draw();
        animObj.frame += 1;

        // fire the end function
        if (animObj.finished()) {
            animObj.onend && animObj.onend();
        }
        // this animation stays in the queue for the next frame
        else {
            newQueue.push(animObj);
        }
    }

    Queue = newQueue;
}

Don’t forget to define the setup function, which is called automatically by Processing.js on startup:

void setup() {
	size(500, 500);
}

The following would then draw a gray background:

Queue.push(new BackgroundAnim());

How it works…

Processing.js searches the DOM for canvas tags with data-processing-sources attribute defined, downloads the .pde file, and applies a post processor against the code converting it to functioning JavaScript that drives the canvas element. You write your animation logic (the .pde file) in a modified Processing environment, which behaves like the Java-based Processing language, except it also has access to global JavaScript variables. For example, if you include a library like jQuery before Processing.js runs, then the code in the .pde will be able to use jQuery. That said, today’s example uses straight processing logic. The AnimationFrameClass is a simple class that can be extended to include logic for drawing any kind of animation frame. The example above includes a Queue array that all animation instances are pushed onto and processed from. The order that animations are added to the Queue, is the order that they will be rendered during an animation frame, so they should be appended background to foreground. An animation frame is automatically handled by Processing.js when it calls the global draw function. The draw function of each AnimationFrameClass objects will be called each frame. As each animation is iterated over several things happen. If an onstart function is defined and the started property is not set to true, then the onstart function is called. Use onstart to handle one-frame animations or to queue an animation that happens when the current animation first starts. Next the draw function is called and the frame counter incremented. Each subclass of AnimationFrameClass should define its draw function, which is what will be rendered on each frame. If the finished function returns true, then the onend function is called, where you might queue an ending or series animation. If the animation is not finished, it will be appended back onto the Queue for processing again in the next frame. Check out the Demo for a fully functioning app with lots of sample animations.

There’s more…

This article focuses mostly on my frame animation technique, and not all the awesome power of the Processing language and Processing.js. For more details, see the Processing.js documentation.

I have been slowly working to improve my Django Shared project, which I use as the basis for all my Django projects. Recently, I added several new templatetags for deferring content and scripts: async_script, defer_html, defer_script, and render_deferred_html. Today’s article will cover how to use these templatetags in your own projects.

Getting ready

You will need to include django-shared in your own project, or extract the parts from templatetags/common.py. If you use pip, here is the easiest way to include django-shared:

pip install git+https://github.com/mattsnider/django-shared.git

How do it…

The async_script template tag is a drop-in replacement for the static template tag:

{% static "images/hi.jpg" %}

Becomes:

{% async_script "images/hi.jpg" %}

Except you can also specify absolute urls:

{% async_script "http://www.google-analytics.com/ga.js" 0 %}

The defer_html template tag is a block tag that appends whatever HTML is in the block into a list for future rendering:

{% defer_html %}
Something I want rendered later
{% end_defer_html %}

The defer_script template tag combines defer_html with async_script, so rendering of the script tag is deferred:

{% defer_script "images/test.js" %}
{% defer_script "http://www.google-analytics.com/ga.js" 0 %}

Lastly, render_deferred_html template tag renders the tag blocks that were added by defer_html with async_script:

    …
    {% render_deferred_html %}

</body>
</html>

How it works…

The async_script template tag uses the deferment pattern to append the script to the page. The first argument is required and is the relative path from STATIC_ROOT to your file, or the absolute path to a resource. The second argument is optional (True by default) and indicates that you want to prepend STATIC_ROOT to the first argument; set this to a falsy value, if you use an absolute path instead. This template tag causes a blocking thread to be used to download the content of the script (like normal), but defers the actual parsing and execution of the JavaScript until the UI thread is idle. If you include JavaScript libraries on your page, using this will cause it to render substantially faster, but your scripts will need to be smart enough to run asynchronously. Here is sample output:

<script>(function(d) {
var el = d.createElement('script'),
  elScript = d.getElementsByTagName('script')[0];
    el.type = 'text/javascript';
    el.async = true;
    el.src = 'http://www.google-analytics.com/ga.js';
    elScript.parentNode.insertBefore(el, elScript);
}(document));</script>

The defer_html is a block template tag that appends a global list with the HTML content put in the block. It keeps appending until render_deferred_html is called, which will render all the HTML from the list onto the page and truncate the list. These two tags can be used multiple times on the page, if it solves your business needs, but usually the render_deferred_html is put at the end of the document. These tags currently should not be used in a multi-threaded environment. Since it appends to a global in memory array, you will run into concurrency issues. The Django session id could be used as an identifier, but most people run their python webservers in a multi-process, instead of multi-threaded environment, so I won’ tackle the multi-threaded problem, unless people need it. In my own projects, I have not found a need for defer_html, but instead use its sister template tag defer_script. The defer_script template tag generates the same HTML and has the same signature as async_script, but appends the script HTML to the global deferment list for rendering with render_deferred_html. I append almost all of my scripts to the document using this deferment combo, so they are rendered at the end of the document and processed asynchronously. After I made these changes, the average loading time for my pages dropped ~50% (as tracked by Google Analytics), which was a substantial improvement.

I needed a straight (ie. no framework) JavaScript function for parsing query parameters the other day and found one on stack overflow that I liked. Most functions that lookup query parameters, search the url string each time it performs a key lookup. However, the query parameters never change once your JavaScript code starts to run, so it is inefficient and unnecessary to search the query parameters string on each lookup. The lookup function in this article converts the query parameters in to a JavaScript object, so the parameters need only be parsed once. That said, sometimes developers store query parameter like strings in cookies, the DOM, the URL hash, etc., so you might want to parse a string other than window.location.search or need reparse a string. I have modified the original function to make it more versatile.

How do it…

Here is the query parameter parsing function:

function parseQueryString() {
  // encapsulate variables that need only be defined once
  var pl = /\+/g,  // Regex for replacing addition symbol with a space
    search = /([^&=]+)=+([^&]*)/g,
    decode = function (s) { return decodeURIComponent(s.replace(pl, " ")); };

  // overwrite the function, creating closure of single-use variables
  parseQueryString = function(query) {
    var match,
      o = {};

    if ('?' === query.substring(0, 1)) {
      query  = query.substring(1);
    }

    // each match is a query parameter, add them to the object
    while (match = search.exec(query)) {
      o[decode(match[1])] = decode(match[2]);
    }

    return o;
  }

  return parseQueryString.apply(this, arguments);
}

Here is a singleton function to use, so that the parsing function is only called once to parse the query parameters:

function getQueryParams() {
  var o = parseQueryString(window.location.search);

  getQueryParams = function() {
    return o;
  }

  return getQueryParams();
}

The following jsfiddle will show how these strings are processed by the parseQueryString function:

var t1 = ''; // empty string
var t2 = '?param1=value1'; // no & after ?
var t3 = '?&param1=value1&param2=value2'; // ?& to start
var t4 = '&param1=value1&param2=value2'; // no ?
var t5 = '&param1=value1&&param2==value2'; // extra &, extra =
var t6 = 'value1'; // invalid
var t7 = 'value1&param2=value2'; // invalid mixed with with valid
var t8 = 'param1=value+1+!%40%2B%23%24'; // encoded values and plus

To Test Fiddle Here are the results:

t1={}
t2={"param1":"value1"}
t3={"param1":"value1","param2":"value2"}
t4={"param1":"value1","param2":"value2"}
t5={"param1":"value1","param2":"value2"}
t6={}
t7={"param2":"value2"}
t8={"param1":"value 1 !@+#$"}

How it works…

The parseQueryString function takes one argument, the string to parse, and returns an object containing the key/value pairs from the parsed query string. I’m using a singleton pattern to encapsulate the three constants (pl, search, decode) inside the function, making them available only through the closure created by the overriding inner function. The core of the function iterates over the matched results from the search regex, which contains two value groups (the key and the value). These key/value pairs are added to the object, until no more are found. The decode function converts any plus symbols back to spaces and decodes any URI encoded values. I have modified the original regex, so invalid or unset query parameter values are not included in the final object. The stack overflow solution would have included the key, but set its value to empty string ('' in the final object. The getQueryParams function uses the singleton pattern to create a function that evaluates the query parameters once, using parseQueryString, and then just returns the parsed query parameter object.

Unfortunately, I did not have time to finish a blog article this week. I was busy creating a CSS Styleguide and doing visualizations in Processing.js for my day job. I plan to write about Processing in the future, but I don’ have a topic ready yet. However, the Styleguide work inspired me to begin one for MattSnider.com, to help me as I write articles and develop the site. So, let me give you a sneak preview.

• Styleguide for MattSnider.com

You may have already noticed that the styles around the sight have changed a bit, as I unify and normalize. I was surprised to realize how often I was overriding a style in one stylesheet, only to revert it to the initial style again later. This styleguide will help me to see what styles I use on my sight and how they interact with each other. My new approach uses a more smcacss approach, and this rough version of the CSS Styleguide closely resembles the GitHub Styleguide (I love their octolink picture). As I spend more time on it, I expect it to begin to vary more.

Lately, I have been experimenting with various JavaScript MVCs, and I was surprised when AngularJS threw an error, when I changed the argument name in a controller function declaration using a factory resource. The library was obviously doing some magic to parse the names of the arguments of a function and validating it against a previously defined namespace. This was interesting, so I thought I would discover how they were doing it and some possible applications.

How do it…

There are some hacky ways of doing this, which use the arguments.callee and .caller properties, but they are deprecated and may become unavailable in the future, and several approaches which use eval. However it is best to avoid eval and any deprecated feature. The last approach is to use .toString, which is simple and has full cross-browser supported. Here is a function to get the names of the arguments of a function instance:

function getParamNames(fn) {
    var funStr = fn.toString();
    return funStr.slice(funStr.indexOf('(') + 1, funStr.indexOf(')')).match(/([^\s,]+)/g);
}

This snippet shows the output of passing a test function in:

function fnTest(a, b, c, d) {}
getParamNames(fnTest) === ['a', 'b', 'c', 'd'];

One application of getParamNames might be to check that an argument name is used. The following checks the provided function for an arbitrary number of argument names, returning true or false:

function assertFunctionDefines(fn) {
    var aFuncParamNames = getParamNames(fn),
        oFuncParamNames = {},
        i = aFuncParamNames.length - 1;

    while (i >= 0) {
        oFuncParamNames[aFuncParamNames[i]] = 1;
        i -= 1;
    }

    i = arguments.length - 1;

    while (i > 0) {
        if (!oFuncParamNames[arguments[i]]) {
            return false;
        }
        i -= 1;
    }

    return true;
}

This shows a couple of examples using the test function again:

assertFunctionDefines(fnTest, 'a', 'b', 'c', 'd') === true;
assertFunctionDefines(fnTest, 'g') === false;

Lastly, you might want to use the keys of an object to determine which argument should be assigned where when calling a function:

function function_kwargs(fn, o) {
    var argNames = getParamNames(fn),
        nameIndexMap = {},
        i = argNames.length - 1,
        args = [];

    while (0 <= i) {
        nameIndexMap[argNames[i]] = i;
        i -= 1;
    }

    for (i in o) {
        if (o.hasOwnProperty(i)) {
            if (undefined !== nameIndexMap[i]) {
                args[nameIndexMap[i]] = o[i];
            }
        }
    }

    return fn.apply(this, args);
}

Executing the following:

function_kwargs(fnTest, {a: 'foo', c: 'bar'});

Would pass a='foo', b=undefined, c='bar', and d=undefined into the test function. Here is a jsfiddle showing each of the above functions.

How it works…

The getParamNames function calls the .toString method of the function instance, which will return the function code as a string. It then slices the string between the parentheses and uses a regex to create an array of all words separated by spaces and commas. These words are the argument names of the function. This technique works on all modern browsers, although I have not checked on IE < 9. The assertFunctionDefines function simply creates a map of all the known arguments and then tests if the provided argument names are in the map, returning true if they all are, or false if one or more are not. There are nicer ways in jQuery for searching arrays, but I wanted the function to be library independent. Lastly, the function_kwargs maps the keys of the object to its position in the arguments of the provided function. Again, I used native looping, so that the code is library independent. I think this function may have some real practical use for handling configurations objects. Consider the following two simple examples:

Classical Configuration

function MyClass(data, conf) {
    this.data = data;
    this.configuration(conf || {});
}
MyClass.prototype = {
    configuration: function(conf) {
        this.config = {
            option1: conf.option1 || default1,
            option2: conf.option2 || default2,
            ...
        };
    }
}

function_kwargs Configuration

function MyClass(data, conf) {
    this.data = data;
    function_kwargs.call(this, this.configuration, conf || {});
}
MyClass.prototype = {
    configuration: function(option1, option2) {
        this.config = {
            option1: option1 || default1,
            option2: option2 || default2,
            ...
        };
    }
}

We can explicitly define the known options on the configuration function, which requires slightly less code, but is the real benefit is, IMHO, that the code is more obviously documented and easier to read.

Continuing our sorting algorithm discussion, today’s article will cover a technique known as Selection Sort. Selection sort is a simple, quadratic (O(n²)) in-place comparison sort algorithm that divides the array into two lists: an already sorted part and a to-sort part. The algorithm moves for through the array and inserts the next value in-place by searching the unsorted part of the array for the next value. In practice, the selection sort is useful when memory is limited.

How do it…

Here is a simple selection sort algorithm in JavaScript:

function selectionSort(a) {
  for (var i, iMin, j = 0, n = a.length, oTemp; j < n-1; j++) {
    iMin = j;

    for (i = j+1; i < n; i++) {
      if (a[i] < a[iMin]) {
        iMin = i;
      }
    }

    if (iMin !== j) {
      oTemp = a[iMin];
      a[iMin] = a[j];
      a[j] = oTemp;
    }
  }
}

Here it is in action: Restart Selection Sort With a New Array If we take a 10,000 element array and sort it, here is how selection sort compares to native array sort (prints to the console): Selection Sort Comparison Tests Here is the bingo selection sort algorithm variant in JavaScript, which is faster for sorting arrays with duplicate values:

function bingoSort(a) {
  for (var i, iMin, j = 0, n = a.length, oTemp; j < n-1; j++) {
    iMin = j;

    for (i = j+1; i < n; i++) {
      if (a[i] < a[iMin]) {
        iMin = i;
      }
    }

    if (iMin !== j) {
      oTemp = a[iMin];
      a[iMin] = a[j];
      a[j] = oTemp;
    }
  }
}

Here we compare 10,000 element array, sorting it with bingo sort versus selection sort. The array has a values 1 - 5,000 duplicated once each, so bingo sort should perform half as many searches (prints to the console): Bingo Sort Comparison Tests

How it works…

The selection sort algorithm moves forward through the array, starting at index n=0. It searches the remaining unsorted values (n to end of array) for the next smallest value and swaps it with the value at n. Then the index is incremented and the process repeats, until the length-2 index is reached (we know at length-2 that the value length-1 is the maximum value, because it is greater than the value at length-2). There is no performance boost, if the array is already sorted, or if there are multiples of the same value. Of the quadratic sorting algorithm, selection sort usually outperforms bubble sort, because the inner loop usually visits less nodes, but because it visits all remaining nodes searching for the next highest value, it tends to make about twice as many comparisons as insertion sort. If you run the comparison test, you will see that selection sort is 10-20x slower when sorting a 10,000 value array than the native sorting algorithm. There is a common variant known as bingo sort, which can provide a significant performance boost, when an array contains multiple of the same value. The algorithm works, by keeping track of all indices containing the next highest value (instead of just one), and swapping them all each pass. It offers no benefit, if you do not expect duplicate values in an array.

There’s more…

For more information, see the Insertion Sort Wikipedia article.