Pretty sure everything here is wrong
Callbacks considered a smell
Here are the ways I have approached dealing with nested callbacks in my own JavaScript code. My samples all set 3 keys in a fake database, then retrieves those 3 keys and prints the values as a decent approximation of dealing with callbacks and I/O. These samples were part of a Intro to Node.js talk I gave at The Motley Fool.
Callbacks
This what was my JavaScript looked like when I first started writing callback-heavy code. It quickly gets unwieldy and hard to refactor because of the nested anonymous functions, and the implicit dependencies.
var db = require('./db/callbackDb');
db.set('key1', 'value1', function(err) {
if (err) throw err;
db.set('key2', 'value2', function(err) {
if (err) throw err;
db.set('key3', 'value3', function(err) {
if (err) throw err;
var str = '';
db.get('key1', function(err, value) {
if (err) throw err;
str += value + ' - ';
db.get('key2', function(err, value) {
if (err) throw err;
str += value + ' - ';
db.get('key3', function(err, value) {
if (err) throw err;
str += value + ' - ';
console.log(str);
});
});
});
});
});
});
Events
If you have the ability to change your underlying code, you can add an EventEmitter so you can follow the observer pattern. My fake evented database has been updated with a wrapper that fires when the functions are finished.
var db = require('./db/eventedDb');
var str = '';
var counter = 0;
db.eventEmitter.on('onGet', function(value) {
str += value + ' - ';
counter++;
if (counter === 3) {
console.log(str);
}
});
db.eventEmitter.on('onSet', function(key) {
db.get(key);
});
db.set('key1', 'value1');
db.set('key2', 'value2');
db.set('key3', 'value3');
Promises
Promises (or defereds or futures) are a way to represent the outcome of an asynchronous call. They used to be included in the core of Node.js, however, they were pulled out around 0.2. However, There are a many user-land modules that attempt to follow the promises interface. For more details on promises, this article explains them in much more detail. For promises to work, you have to return a promise object from the underlying code, so my fake db got some more love.
var db = require('./db/promisesDb');
var str = '';
db.set('key1', 'value1').then(function(key) {
return db.set('key2', 'value2');
}).then(function() {
return db.set('key3', 'value3');
}).then(function() {
return db.get('key1');
}).then(function(value) {
str += value + ' - ';
return db.get('key2');
}).then(function(value) {
str += value + ' - ';
return db.get('key3');
}).then(function(value) {
str += value + ' - ';
console.log(str);
});
Control flow
In addition to the basic promises concept, there are many control flow modules that help sequence asynchronous calls. I used caolan’s async because that is the one I am most comfortable with, but there are a ton floating around. Most of them allow you to parallelize a set of async calls and wait for their end result, or force async calls into a particular series.
var db = require('./db/callbackDb');
var async = require('async');
async.series([
function(callback) {
db.set('key1', 'value1', function(err) {
callback(err);
});
},
function(callback) {
db.set('key2', 'value2', function(err) {
callback(err);
});
},
function(callback) {
db.set('key3', 'value3', function(err) {
callback(err);
});
},
function(callback) {
db.get('key1', function(err, value) {
callback(err, value);
});
},
function(callback) {
db.get('key2', function(err, value) {
callback(err, value);
});
},
function(callback) {
db.get('key3', function(err, value) {
callback(err, value);
});
}
],
function(err, results) {
// results = [ undefined, undefined, undefined, 'value1', 'value2', 'value3' ]
var str = '';
results.forEach(function(result) {
if (result) {
str += result + ' - ';
}
});
console.log(str);
});
Named functions and Modules
In my opinion, the best way to escape callback hell is to split your code into modules that do one particular thing with named functions. This is basic code refactoring, but it becomes pretty important when dealing with nested callbacks. My fake database has been wrapped once more which lets our code be more succinct.
var db = require('./db/wrapperDb');
function concatValues(keys, callback) {
db.getValues(keys, function(err, values) {
if (err) return callback(err);
var str = '';
values.forEach(function(value) {
str += value + ' - ';
});
return callback(null, str);
})
}
db.setValues(['key1', 'key2', 'key3'], ['value1', 'value2', 'value3'], function(err, keys) {
if (err) throw err;
concatValues(keys, function(err, str) {
if (err) throw err;
console.log(str);
});
});
Hopefully this article gives developers some more ammunition when faced with nested anonymous functions in their own forays into JavaScript.
13 Comments
-
I think a combination of an event emitter and the named functions is a great way to go. I tend to use named functions for callbacks and event emitters for triggering code within a callback.
-
In your control flow example, there’s no need to encapsulate the callback in a function for each step, you can pass the callback directly.
eg:
function(callback) {
db.get('key3', callback);
}
-
You can even replace “function (err, callback) { db.get(‘key3′, callback) }” with “db.get.bind(db, ‘key3′)”
Well, that may introduce off-topic concept for your readers, but still very useful for conciseness and readability.
-
-
Have you tried with monads?
they look excellent for handling promises
http://blog.jcoglan.com/2011/03/11/promises-are-the-monad-of-asynchronous-programming/ -
One that I always heartily recommend and love is something entirely different — instead of helping you at the library level, it helps you at the language level:
https://github.com/Sage/streamlinejs
Streamline lets you write “sync”-looking JS that gets “compiled” (transformed) to the callback version. (Or it can compile to use https://github.com/laverdet/node-fibers which is even better.)
Now you literally eliminate *all* callback noise, including a bunch of extra functions(). You write the algorithm you want to run, not helper code around that algorithm.
Cheers. =)
-
When you talked about naming callbacks, I thought you were going to presentthe first code, but simply “unindented” by declaring your callbacks:
// shared error-handling logic
function set (key, value, next) {
db.set(key, value, function (err) {
if (err) throw err
next()
})
}
// and some logic you were repeating
var str = ”
function get (key, next) {
db.get(key, function (err, value) {
if (err) throw err;
str += value
next()
})
}// now we declare our callbacks
function set1() { set(‘key1′, ‘value1′, set2) }
function set2() { set(‘key2′, ‘value2′, set3) }
function set3() { set(‘key3′, ‘value3′, get1) }
function get1() { get(‘key1′, get2) }
function get2() { get(‘key2′, get2) }
function get3() { get(‘key3′, show) }
function show() { console.log(str) }// start the race
set1()This is of course a naive approach, but can be a good simple first step for beginners.
-
-
In my master thesis, I’m writing a small section about the callback problem right now and found this: http://callbackhell.com/
I saw your blog entry in the JavaScript Weekly newsletter just now and am really happy to see even more good alternatives how to simplify callback handling.
Thanks! -
I second the suggestion from other commenters to check out node-fibers and it’s Future implementation. It’s becoming a fairly mainstream approach with frameworks like Meteor wrapping every http request in a fiber. We’ve found it an invaluable tool for writing clear and manageable async code in Node.
There are a few wrappers for fibers out there including one that we’ve open sources to provide a clean method for using fibers in your code – http://goodeggs.github.com/fibrous/. The major benefit of using fibrous is that the function you write conform to the standard node callback style while internally enabling a more synchronous programming style.
-
Great article. Each time I pick up some callback-cascading code and have to work out how to change or work with it I find myself trying to rewrite it by using named functions. (At least I have a clue what is supposed to happen with a named function). I thought that it was just my simple mind failing to cope with the latest and greatest in coding fashions.
I’ll start looking into your other suggestions too.
-
The events and named functions and modules examples are not equivalent to the callbacks, promises, or control flow examples. For the events example it requires the db module to guarantee that it fires get events in the same order that the set events where emitted. Likewise for the named functions and modules example the simultaneous db.set and db.get calls within it must return in the same order. Given the variability in timing going against a real asynchronous db additional care would be needed to make all of the examples equivalent.
-
Feel free to fork and improve
.-
Nope, that should now belong to the db module, at least for evented API.
It’s the “calling” code that must re-order results at reception, as async does.
-
-
