@adamghill

Pretty sure everything here is wrong

RSS Feed

Callbacks considered a smell

Comments Off
Posted by Adam on December 2, 2012 at 1:24 pm

Here are the ways I have approached dealing with nested callbacks in my own JavaScript code. My samples all set 3 keys in a fake database, then retrieves those 3 keys and prints the values as a decent approximation of dealing with callbacks and I/O. These samples were part of a Intro to Node.js talk I gave at The Motley Fool.

Callbacks

This what was my JavaScript looked like when I first started writing callback-heavy code. It quickly gets unwieldy and hard to refactor because of the nested anonymous functions, and the implicit dependencies.

var db = require('./db/callbackDb');

db.set('key1', 'value1', function(err) {
	if (err) throw err;

	db.set('key2', 'value2', function(err) {
		if (err) throw err;

		db.set('key3', 'value3', function(err) {
			if (err) throw err;

			var str = '';
			db.get('key1', function(err, value) {
				if (err) throw err;
				str += value + ' - ';

				db.get('key2', function(err, value) {
					if (err) throw err;
					str += value + ' - ';

					db.get('key3', function(err, value) {
						if (err) throw err;
						str += value + ' - ';

						console.log(str);
					});
				});
			});
		});
	});
});

Events

If you have the ability to change your underlying code, you can add an EventEmitter so you can follow the observer pattern. My fake evented database has been updated with a wrapper that fires when the functions are finished.

var db = require('./db/eventedDb');
var str = '';
var counter = 0;
db.eventEmitter.on('onGet', function(value) {
  str += value + ' - ';
  counter++;

  if (counter === 3) {
    console.log(str);
  }
});

db.eventEmitter.on('onSet', function(key) {
  db.get(key);
});

db.set('key1', 'value1');
db.set('key2', 'value2');
db.set('key3', 'value3');

Promises

Promises (or defereds or futures) are a way to represent the outcome of an asynchronous call. They used to be included in the core of Node.js, however, they were pulled out around 0.2. However, There are a many user-land modules that attempt to follow the promises interface. For more details on promises, this article explains them in much more detail. For promises to work, you have to return a promise object from the underlying code, so my fake db got some more love.

var db = require('./db/promisesDb');
var str = '';

db.set('key1', 'value1').then(function(key) {
	return db.set('key2', 'value2');
}).then(function() {
	return db.set('key3', 'value3');
}).then(function() {
	return db.get('key1');
}).then(function(value) {
	str += value + ' - ';

	return db.get('key2');
}).then(function(value) {
	str += value + ' - ';

	return db.get('key3');
}).then(function(value) {
	str += value + ' - ';

	console.log(str);
});

Control flow

In addition to the basic promises concept, there are many control flow modules that help sequence asynchronous calls. I used caolan’s async because that is the one I am most comfortable with, but there are a ton floating around. Most of them allow you to parallelize a set of async calls and wait for their end result, or force async calls into a particular series.

var db = require('./db/callbackDb');
var async = require('async');

async.series([
  function(callback) {
    db.set('key1', 'value1', function(err) {
    	callback(err);
    });
  },
  function(callback) {
    db.set('key2', 'value2', function(err) {
    	callback(err);
    });
  },
	function(callback) {
    db.set('key3', 'value3', function(err) {
    	callback(err);
    });
  },
  function(callback) {
    db.get('key1', function(err, value) {
    	callback(err, value);
    });
  },
  function(callback) {
    db.get('key2', function(err, value) {
    	callback(err, value);
    });
  },
  function(callback) {
    db.get('key3', function(err, value) {
    	callback(err, value);
    });
  }
],
function(err, results) {
	// results = [ undefined, undefined, undefined, 'value1', 'value2', 'value3' ]
	var str = '';

	results.forEach(function(result) {
		if (result) {
			str += result + ' - ';
		}
	});

	console.log(str);
});

Named functions and Modules

In my opinion, the best way to escape callback hell is to split your code into modules that do one particular thing with named functions. This is basic code refactoring, but it becomes pretty important when dealing with nested callbacks. My fake database has been wrapped once more which lets our code be more succinct.

var db = require('./db/wrapperDb');

function concatValues(keys, callback) {
	db.getValues(keys, function(err, values) {
		if (err) return callback(err);

		var str = '';

		values.forEach(function(value) {
			str += value + ' - ';
		});

		return callback(null, str);
	})
}

db.setValues(['key1', 'key2', 'key3'], ['value1', 'value2', 'value3'], function(err, keys) {
	if (err) throw err;

	concatValues(keys, function(err, str) {
		if (err) throw err;

		console.log(str);
	});
});

Hopefully this article gives developers some more ammunition when faced with nested anonymous functions in their own forays into JavaScript.

Filed under node.js
Both comments and pings are currently closed.

13 Comments

  • On December 3, 2012 at 4:36 pm Angelo R said

    I think a combination of an event emitter and the named functions is a great way to go. I tend to use named functions for callbacks and event emitters for triggering code within a callback.

  • On December 3, 2012 at 6:52 pm Olly Smith said

    In your control flow example, there’s no need to encapsulate the callback in a function for each step, you can pass the callback directly.

    eg:


    function(callback) {
    db.get('key3', callback);
    }

    • On December 10, 2012 at 7:43 am Nicolas Chambrier said

      You can even replace “function (err, callback) { db.get(‘key3′, callback) }” with “db.get.bind(db, ‘key3′)”

      Well, that may introduce off-topic concept for your readers, but still very useful for conciseness and readability.

  • On December 4, 2012 at 5:41 pm picanteverde said

    Have you tried with monads?
    they look excellent for handling promises
    http://blog.jcoglan.com/2011/03/11/promises-are-the-monad-of-asynchronous-programming/

  • On December 5, 2012 at 10:55 pm Alex said
  • On December 5, 2012 at 11:10 pm Aseem Kishore said

    One that I always heartily recommend and love is something entirely different — instead of helping you at the library level, it helps you at the language level:

    https://github.com/Sage/streamlinejs

    Streamline lets you write “sync”-looking JS that gets “compiled” (transformed) to the callback version. (Or it can compile to use https://github.com/laverdet/node-fibers which is even better.)

    Now you literally eliminate *all* callback noise, including a bunch of extra functions(). You write the algorithm you want to run, not helper code around that algorithm.

    Cheers. =)

    • On December 10, 2012 at 8:02 am Nicolas Chambrier said

      When you talked about naming callbacks, I thought you were going to presentthe first code, but simply “unindented” by declaring your callbacks:

      // shared error-handling logic
      function set (key, value, next) {
      db.set(key, value, function (err) {
      if (err) throw err
      next()
      })
      }
      // and some logic you were repeating
      var str = ”
      function get (key, next) {
      db.get(key, function (err, value) {
      if (err) throw err;
      str += value
      next()
      })
      }

      // now we declare our callbacks
      function set1() { set(‘key1′, ‘value1′, set2) }
      function set2() { set(‘key2′, ‘value2′, set3) }
      function set3() { set(‘key3′, ‘value3′, get1) }
      function get1() { get(‘key1′, get2) }
      function get2() { get(‘key2′, get2) }
      function get3() { get(‘key3′, show) }
      function show() { console.log(str) }

      // start the race
      set1()

      This is of course a naive approach, but can be a good simple first step for beginners.

  • On December 7, 2012 at 5:47 pm Joern said

    In my master thesis, I’m writing a small section about the callback problem right now and found this: http://callbackhell.com/
    I saw your blog entry in the JavaScript Weekly newsletter just now and am really happy to see even more good alternatives how to simplify callback handling.
    Thanks!

  • On December 7, 2012 at 5:47 pm Alon said

    I second the suggestion from other commenters to check out node-fibers and it’s Future implementation. It’s becoming a fairly mainstream approach with frameworks like Meteor wrapping every http request in a fiber. We’ve found it an invaluable tool for writing clear and manageable async code in Node.

    There are a few wrappers for fibers out there including one that we’ve open sources to provide a clean method for using fibers in your code – http://goodeggs.github.com/fibrous/. The major benefit of using fibrous is that the function you write conform to the standard node callback style while internally enabling a more synchronous programming style.

  • On December 8, 2012 at 7:44 am Anita said

    Great article. Each time I pick up some callback-cascading code and have to work out how to change or work with it I find myself trying to rewrite it by using named functions. (At least I have a clue what is supposed to happen with a named function). I thought that it was just my simple mind failing to cope with the latest and greatest in coding fashions.

    I’ll start looking into your other suggestions too.

  • On December 9, 2012 at 3:21 pm Daniel R. said

    The events and named functions and modules examples are not equivalent to the callbacks, promises, or control flow examples. For the events example it requires the db module to guarantee that it fires get events in the same order that the set events where emitted. Likewise for the named functions and modules example the simultaneous db.set and db.get calls within it must return in the same order. Given the variability in timing going against a real asynchronous db additional care would be needed to make all of the examples equivalent.

    • On December 9, 2012 at 5:02 pm Adam said

      Feel free to fork and improve :) .

      • On December 10, 2012 at 7:47 am Nicolas Chambrier said

        Nope, that should now belong to the db module, at least for evented API.
        It’s the “calling” code that must re-order results at reception, as async does.